2024-06-03T02:56:47.957 INFO:root:teuthology version: 0.0.1.dev303+g861a8dc 2024-06-03T02:56:47.958 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/no 6-workunit/suites/fsync-tester}} --verbose --archive /home/teuthworker/archive/teuthology-2024-06-02_21:24:02-fs-squid-distro-default-smithi/7738859 --owner scheduled_teuthology@teuthology --name teuthology-2024-06-02_21:24:02-fs-squid-distro-default-smithi -- /home/teuthworker/archive/teuthology-2024-06-02_21:24:02-fs-squid-distro-default-smithi/7738859/orig.config.yaml 2024-06-03T02:56:48.009 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-06-03T02:56:48.110 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-06-02_21:24:02-fs-squid-distro-default-smithi/7738859 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/no 6-workunit/suites/fsync-tester}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7738859' kernel: client: branch: testing kdb: 1 sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: teuthology-2024-06-02_21:24:02-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - slow metadata IO - SLOW_OPS - slow request sha1: 3d89accfe3201286f000e065a1faea2a88c7a6ba subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported dry_run: true install: ceph: flavor: default sha1: 3d89accfe3201286f000e065a1faea2a88c7a6ba kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - nowsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: dfcc4b3c2c55532c4c04fa47551c9df1dffdf746 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 2064 sha1: 3d89accfe3201286f000e065a1faea2a88c7a6ba sleep_before_teardown: 0 subset: 125/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_dfcc4b3c2c55532c4c04fa47551c9df1dffdf746/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: dfcc4b3c2c55532c4c04fa47551c9df1dffdf746 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi117.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOD4aIiCyafOtJ/JRxMwcSx+Ok3Yy2MVGkX9iUjd+c2th3GNkbblBlBAHkwAuSzVca9mBmolTsJ68GWZ6qIaNuQ= smithi151.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDayn1gBmeKmnboijQwkzgpl40URRaj59GtOgMBoS/qI2lOiCQDL3PNKxbl+Fqi4An1pvqLEVWwW0keuyUV+6pI= smithi183.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBND8tVIzsKVIQLMmvbEX3OEU82I75fGuacr+J5pFZUP1C25Cy99EY6XJ4CxVUUUuIi7q3J5OsbLy+gn19q8SQgM= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/fsync-tester.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 861a8dcf7aa816a26e13f039336f7ed0a9aec0fa timestamp: 2024-06-02_21:24:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2698841 2024-06-03T02:56:48.110 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_dfcc4b3c2c55532c4c04fa47551c9df1dffdf746/qa; will attempt to use it 2024-06-03T02:56:48.111 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_dfcc4b3c2c55532c4c04fa47551c9df1dffdf746/qa/tasks 2024-06-03T02:56:48.111 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-06-03T02:56:48.112 INFO:teuthology.task.internal:Checking packages... 2024-06-03T02:56:48.133 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '3d89accfe3201286f000e065a1faea2a88c7a6ba' 2024-06-03T02:56:48.133 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-06-03T02:56:48.134 INFO:teuthology.packaging:ref: None 2024-06-03T02:56:48.134 INFO:teuthology.packaging:tag: None 2024-06-03T02:56:48.134 INFO:teuthology.packaging:branch: squid 2024-06-03T02:56:48.134 INFO:teuthology.packaging:sha1: 3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-03T02:56:48.134 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-06-03T02:56:48.330 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2952.gdfcc4b3c 2024-06-03T02:56:48.331 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-06-03T02:56:48.337 INFO:teuthology.task.internal:no buildpackages task found 2024-06-03T02:56:48.338 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-06-03T02:56:48.467 INFO:teuthology.task.internal:Saving configuration 2024-06-03T02:56:48.486 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-06-03T02:56:48.573 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-06-03T02:56:48.598 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi117.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-06-02_21:24:02-fs-squid-distro-default-smithi/7738859', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-06-03 02:51:43.998508', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOD4aIiCyafOtJ/JRxMwcSx+Ok3Yy2MVGkX9iUjd+c2th3GNkbblBlBAHkwAuSzVca9mBmolTsJ68GWZ6qIaNuQ='} 2024-06-03T02:56:48.622 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi151.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-06-02_21:24:02-fs-squid-distro-default-smithi/7738859', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-06-03 02:51:43.996817', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDayn1gBmeKmnboijQwkzgpl40URRaj59GtOgMBoS/qI2lOiCQDL3PNKxbl+Fqi4An1pvqLEVWwW0keuyUV+6pI='} 2024-06-03T02:56:48.646 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi183.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-06-02_21:24:02-fs-squid-distro-default-smithi/7738859', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-06-03 02:51:43.999908', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBND8tVIzsKVIQLMmvbEX3OEU82I75fGuacr+J5pFZUP1C25Cy99EY6XJ4CxVUUUuIi7q3J5OsbLy+gn19q8SQgM='} 2024-06-03T02:56:48.646 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-06-03T02:56:48.651 INFO:teuthology.task.internal:roles: ubuntu@smithi117.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-06-03T02:56:48.651 INFO:teuthology.task.internal:roles: ubuntu@smithi151.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-06-03T02:56:48.651 INFO:teuthology.task.internal:roles: ubuntu@smithi183.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-06-03T02:56:48.651 INFO:teuthology.run_tasks:Running task console_log... 2024-06-03T02:56:48.794 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fc8071afbe0>, signals=[15]) 2024-06-03T02:56:48.795 INFO:teuthology.run_tasks:Running task internal.connect... 2024-06-03T02:56:48.830 INFO:teuthology.task.internal:Opening connections... 2024-06-03T02:56:48.830 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi117.front.sepia.ceph.com 2024-06-03T02:56:48.832 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-03T02:56:48.916 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi151.front.sepia.ceph.com 2024-06-03T02:56:48.917 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-03T02:56:48.991 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi183.front.sepia.ceph.com 2024-06-03T02:56:48.992 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-03T02:56:49.065 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-06-03T02:56:49.071 DEBUG:teuthology.orchestra.run.smithi117:> uname -m 2024-06-03T02:56:49.088 INFO:teuthology.orchestra.run.smithi117.stdout:x86_64 2024-06-03T02:56:49.089 DEBUG:teuthology.orchestra.run.smithi117:> cat /etc/os-release 2024-06-03T02:56:49.145 INFO:teuthology.orchestra.run.smithi117.stdout:NAME="CentOS Stream" 2024-06-03T02:56:49.146 INFO:teuthology.orchestra.run.smithi117.stdout:VERSION="9" 2024-06-03T02:56:49.146 INFO:teuthology.orchestra.run.smithi117.stdout:ID="centos" 2024-06-03T02:56:49.146 INFO:teuthology.orchestra.run.smithi117.stdout:ID_LIKE="rhel fedora" 2024-06-03T02:56:49.146 INFO:teuthology.orchestra.run.smithi117.stdout:VERSION_ID="9" 2024-06-03T02:56:49.146 INFO:teuthology.orchestra.run.smithi117.stdout:PLATFORM_ID="platform:el9" 2024-06-03T02:56:49.146 INFO:teuthology.orchestra.run.smithi117.stdout:PRETTY_NAME="CentOS Stream 9" 2024-06-03T02:56:49.146 INFO:teuthology.orchestra.run.smithi117.stdout:ANSI_COLOR="0;31" 2024-06-03T02:56:49.146 INFO:teuthology.orchestra.run.smithi117.stdout:LOGO="fedora-logo-icon" 2024-06-03T02:56:49.146 INFO:teuthology.orchestra.run.smithi117.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-06-03T02:56:49.146 INFO:teuthology.orchestra.run.smithi117.stdout:HOME_URL="https://centos.org/" 2024-06-03T02:56:49.146 INFO:teuthology.orchestra.run.smithi117.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-06-03T02:56:49.146 INFO:teuthology.orchestra.run.smithi117.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-06-03T02:56:49.147 INFO:teuthology.orchestra.run.smithi117.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-06-03T02:56:49.147 INFO:teuthology.lock.ops:Updating smithi117.front.sepia.ceph.com on lock server 2024-06-03T02:56:49.179 DEBUG:teuthology.orchestra.run.smithi151:> uname -m 2024-06-03T02:56:49.197 INFO:teuthology.orchestra.run.smithi151.stdout:x86_64 2024-06-03T02:56:49.197 DEBUG:teuthology.orchestra.run.smithi151:> cat /etc/os-release 2024-06-03T02:56:49.255 INFO:teuthology.orchestra.run.smithi151.stdout:NAME="CentOS Stream" 2024-06-03T02:56:49.255 INFO:teuthology.orchestra.run.smithi151.stdout:VERSION="9" 2024-06-03T02:56:49.255 INFO:teuthology.orchestra.run.smithi151.stdout:ID="centos" 2024-06-03T02:56:49.255 INFO:teuthology.orchestra.run.smithi151.stdout:ID_LIKE="rhel fedora" 2024-06-03T02:56:49.255 INFO:teuthology.orchestra.run.smithi151.stdout:VERSION_ID="9" 2024-06-03T02:56:49.255 INFO:teuthology.orchestra.run.smithi151.stdout:PLATFORM_ID="platform:el9" 2024-06-03T02:56:49.255 INFO:teuthology.orchestra.run.smithi151.stdout:PRETTY_NAME="CentOS Stream 9" 2024-06-03T02:56:49.256 INFO:teuthology.orchestra.run.smithi151.stdout:ANSI_COLOR="0;31" 2024-06-03T02:56:49.256 INFO:teuthology.orchestra.run.smithi151.stdout:LOGO="fedora-logo-icon" 2024-06-03T02:56:49.256 INFO:teuthology.orchestra.run.smithi151.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-06-03T02:56:49.256 INFO:teuthology.orchestra.run.smithi151.stdout:HOME_URL="https://centos.org/" 2024-06-03T02:56:49.256 INFO:teuthology.orchestra.run.smithi151.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-06-03T02:56:49.256 INFO:teuthology.orchestra.run.smithi151.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-06-03T02:56:49.256 INFO:teuthology.orchestra.run.smithi151.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-06-03T02:56:49.256 INFO:teuthology.lock.ops:Updating smithi151.front.sepia.ceph.com on lock server 2024-06-03T02:56:49.279 DEBUG:teuthology.orchestra.run.smithi183:> uname -m 2024-06-03T02:56:49.297 INFO:teuthology.orchestra.run.smithi183.stdout:x86_64 2024-06-03T02:56:49.297 DEBUG:teuthology.orchestra.run.smithi183:> cat /etc/os-release 2024-06-03T02:56:49.355 INFO:teuthology.orchestra.run.smithi183.stdout:NAME="CentOS Stream" 2024-06-03T02:56:49.355 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION="9" 2024-06-03T02:56:49.355 INFO:teuthology.orchestra.run.smithi183.stdout:ID="centos" 2024-06-03T02:56:49.355 INFO:teuthology.orchestra.run.smithi183.stdout:ID_LIKE="rhel fedora" 2024-06-03T02:56:49.355 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION_ID="9" 2024-06-03T02:56:49.355 INFO:teuthology.orchestra.run.smithi183.stdout:PLATFORM_ID="platform:el9" 2024-06-03T02:56:49.355 INFO:teuthology.orchestra.run.smithi183.stdout:PRETTY_NAME="CentOS Stream 9" 2024-06-03T02:56:49.355 INFO:teuthology.orchestra.run.smithi183.stdout:ANSI_COLOR="0;31" 2024-06-03T02:56:49.356 INFO:teuthology.orchestra.run.smithi183.stdout:LOGO="fedora-logo-icon" 2024-06-03T02:56:49.356 INFO:teuthology.orchestra.run.smithi183.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-06-03T02:56:49.356 INFO:teuthology.orchestra.run.smithi183.stdout:HOME_URL="https://centos.org/" 2024-06-03T02:56:49.356 INFO:teuthology.orchestra.run.smithi183.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-06-03T02:56:49.356 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-06-03T02:56:49.356 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-06-03T02:56:49.356 INFO:teuthology.lock.ops:Updating smithi183.front.sepia.ceph.com on lock server 2024-06-03T02:56:49.378 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-06-03T02:56:49.387 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-06-03T02:56:49.599 INFO:teuthology.task.internal:Checking for old test directory... 2024-06-03T02:56:49.600 DEBUG:teuthology.orchestra.run.smithi117:> test '!' -e /home/ubuntu/cephtest 2024-06-03T02:56:49.602 DEBUG:teuthology.orchestra.run.smithi151:> test '!' -e /home/ubuntu/cephtest 2024-06-03T02:56:49.604 DEBUG:teuthology.orchestra.run.smithi183:> test '!' -e /home/ubuntu/cephtest 2024-06-03T02:56:49.621 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-06-03T02:56:49.685 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-06-03T02:56:49.685 DEBUG:teuthology.orchestra.run.smithi117:> test -z $(ls -A /var/lib/ceph) 2024-06-03T02:56:49.688 DEBUG:teuthology.orchestra.run.smithi151:> test -z $(ls -A /var/lib/ceph) 2024-06-03T02:56:49.690 DEBUG:teuthology.orchestra.run.smithi183:> test -z $(ls -A /var/lib/ceph) 2024-06-03T02:56:49.794 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-06-03T02:56:49.869 INFO:teuthology.run_tasks:Running task kernel... 2024-06-03T02:56:49.883 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'} 2024-06-03T02:56:49.883 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-06-03T02:56:49.883 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-06-03T02:56:49.883 DEBUG:teuthology.orchestra.run.smithi117:> test -f /run/.containerenv -o -f /.dockerenv 2024-06-03T02:56:49.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-03T02:56:49.900 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-06-03T02:56:50.144 DEBUG:teuthology.task.kernel:sha1 for client.0 is b9459a26b0888785d05c47deb35abc9d0354ab93 2024-06-03T02:56:50.144 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "b9459a26b0888785d05c47deb35abc9d0354ab93"... 2024-06-03T02:56:50.144 DEBUG:teuthology.orchestra.run.smithi117:> uname -r 2024-06-03T02:56:50.161 INFO:teuthology.orchestra.run.smithi117.stdout:5.14.0-437.el9.x86_64 2024-06-03T02:56:50.161 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs b9459a26b0888785d05c47deb35abc9d0354ab93 2024-06-03T02:56:50.162 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-06-03T02:56:50.162 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum upgrade -y linux-firmware 2024-06-03T02:56:52.463 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.1 MB 00:00 2024-06-03T02:56:54.155 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - AppStream 32 MB/s | 19 MB 00:00 2024-06-03T02:56:58.322 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.2 MB 00:00 2024-06-03T02:57:00.556 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - Extras packages 16 kB/s | 17 kB 00:01 2024-06-03T02:57:01.282 INFO:teuthology.orchestra.run.smithi117.stdout:Extra Packages for Enterprise Linux 36 MB/s | 21 MB 00:00 2024-06-03T02:57:06.399 INFO:teuthology.orchestra.run.smithi117.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-06-03T02:57:08.510 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-06-03T02:57:08.511 INFO:teuthology.orchestra.run.smithi117.stdout:Nothing to do. 2024-06-03T02:57:08.511 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-06-03T02:57:08.616 INFO:teuthology.task.kernel:Downloading kernel b9459a26b0888785d05c47deb35abc9d0354ab93 on client.0... 2024-06-03T02:57:08.617 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b9459a26b0888785d05c47deb35abc9d0354ab93 2024-06-03T02:57:08.765 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/centos/9/flavors/default/x86_64/ 2024-06-03T02:57:08.765 DEBUG:teuthology.orchestra.run.smithi117:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_gb9459a26b088-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/centos/9/flavors/default/x86_64/ --input-file=- 2024-06-03T02:57:08.767 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi117.front.sepia.ceph.com to complete... 2024-06-03T02:57:20.893 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03 02:57:20 URL:https://3.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/centos/9/flavors/default/x86_64/kernel-6.9.0_gb9459a26b088-1.x86_64.rpm [356863406/356863406] -> "/tmp/kernel.x86_64.rpm" [1] 2024-06-03T02:57:20.894 INFO:teuthology.orchestra.run.smithi117.stderr:FINISHED --2024-06-03 02:57:20-- 2024-06-03T02:57:20.894 INFO:teuthology.orchestra.run.smithi117.stderr:Total wall clock time: 12s 2024-06-03T02:57:20.894 INFO:teuthology.orchestra.run.smithi117.stderr:Downloaded: 1 files, 340M in 12s (28.8 MB/s) 2024-06-03T02:57:20.896 INFO:teuthology.task.kernel:Installing kernel b9459a26b0888785d05c47deb35abc9d0354ab93 on client.0... 2024-06-03T02:57:20.897 DEBUG:teuthology.orchestra.run.smithi117:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-06-03T02:57:23.020 INFO:teuthology.orchestra.run.smithi117.stdout:Verifying... ######################################## 2024-06-03T02:57:23.144 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing... ######################################## 2024-06-03T02:57:23.272 INFO:teuthology.orchestra.run.smithi117.stdout:Updating / installing... 2024-06-03T02:57:31.184 INFO:teuthology.orchestra.run.smithi117.stdout:kernel-6.9.0_gb9459a26b088-1 ######################################## 2024-06-03T02:58:10.048 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi117.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-06-03T02:58:10.049 DEBUG:teuthology.orchestra.run.smithi117:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-06-03T02:58:10.132 INFO:teuthology.orchestra.run.smithi117.stdout:/boot/System.map-6.9.0-gb9459a26b088 2024-06-03T02:58:10.132 INFO:teuthology.orchestra.run.smithi117.stdout:/boot/config-6.9.0-gb9459a26b088 2024-06-03T02:58:10.132 INFO:teuthology.orchestra.run.smithi117.stdout:/boot/vmlinuz-6.9.0-gb9459a26b088 2024-06-03T02:58:10.132 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/System.map 2024-06-03T02:58:10.132 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/config 2024-06-03T02:58:10.132 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel 2024-06-03T02:58:10.132 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch 2024-06-03T02:58:10.132 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86 2024-06-03T02:58:10.132 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto 2024-06-03T02:58:10.132 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-06-03T02:58:10.133 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-06-03T02:58:10.133 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-06-03T02:58:10.133 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-06-03T02:58:10.133 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-06-03T02:58:10.133 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-06-03T02:58:10.133 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-06-03T02:58:10.133 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crc32c-intel.ko 2024-06-03T02:58:10.133 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-06-03T02:58:10.133 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-06-03T02:58:10.133 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-06-03T02:58:10.133 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-06-03T02:58:10.133 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-avx2.ko 2024-06-03T02:58:10.133 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-06-03T02:58:10.134 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-06-03T02:58:10.134 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-06-03T02:58:10.134 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-06-03T02:58:10.134 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-06-03T02:58:10.134 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-06-03T02:58:10.134 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-06-03T02:58:10.134 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel 2024-06-03T02:58:10.134 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu 2024-06-03T02:58:10.134 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu/mce 2024-06-03T02:58:10.134 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-06-03T02:58:10.134 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm 2024-06-03T02:58:10.134 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm-amd.ko 2024-06-03T02:58:10.134 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm-intel.ko 2024-06-03T02:58:10.135 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm.ko 2024-06-03T02:58:10.135 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto 2024-06-03T02:58:10.135 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/ansi_cprng.ko 2024-06-03T02:58:10.135 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/anubis.ko 2024-06-03T02:58:10.135 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/arc4.ko 2024-06-03T02:58:10.135 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx 2024-06-03T02:58:10.135 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_memcpy.ko 2024-06-03T02:58:10.135 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_pq.ko 2024-06-03T02:58:10.135 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_raid6_recov.ko 2024-06-03T02:58:10.135 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_tx.ko 2024-06-03T02:58:10.135 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_xor.ko 2024-06-03T02:58:10.135 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/raid6test.ko 2024-06-03T02:58:10.135 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/authenc.ko 2024-06-03T02:58:10.136 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/authencesn.ko 2024-06-03T02:58:10.136 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blake2b_generic.ko 2024-06-03T02:58:10.136 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blowfish_common.ko 2024-06-03T02:58:10.136 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blowfish_generic.ko 2024-06-03T02:58:10.136 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/camellia_generic.ko 2024-06-03T02:58:10.136 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast5_generic.ko 2024-06-03T02:58:10.136 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast6_generic.ko 2024-06-03T02:58:10.136 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast_common.ko 2024-06-03T02:58:10.136 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/ccm.ko 2024-06-03T02:58:10.136 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cmac.ko 2024-06-03T02:58:10.136 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crc32_generic.ko 2024-06-03T02:58:10.136 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crypto_engine.ko 2024-06-03T02:58:10.137 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crypto_user.ko 2024-06-03T02:58:10.137 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cts.ko 2024-06-03T02:58:10.137 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/des_generic.ko 2024-06-03T02:58:10.137 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/dh_generic.ko 2024-06-03T02:58:10.137 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/echainiv.ko 2024-06-03T02:58:10.137 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/essiv.ko 2024-06-03T02:58:10.137 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/fcrypt.ko 2024-06-03T02:58:10.137 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/khazad.ko 2024-06-03T02:58:10.137 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/lz4.ko 2024-06-03T02:58:10.137 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/lz4hc.ko 2024-06-03T02:58:10.137 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/md4.ko 2024-06-03T02:58:10.137 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/michael_mic.ko 2024-06-03T02:58:10.137 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/pcbc.ko 2024-06-03T02:58:10.138 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/pcrypt.ko 2024-06-03T02:58:10.138 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/rmd160.ko 2024-06-03T02:58:10.138 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/seed.ko 2024-06-03T02:58:10.138 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/serpent_generic.ko 2024-06-03T02:58:10.138 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/tcrypt.ko 2024-06-03T02:58:10.138 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/tea.ko 2024-06-03T02:58:10.138 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/twofish_common.ko 2024-06-03T02:58:10.138 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/twofish_generic.ko 2024-06-03T02:58:10.138 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/vmac.ko 2024-06-03T02:58:10.138 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/wp512.ko 2024-06-03T02:58:10.138 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xcbc.ko 2024-06-03T02:58:10.138 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xor.ko 2024-06-03T02:58:10.138 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xxhash_generic.ko 2024-06-03T02:58:10.139 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers 2024-06-03T02:58:10.139 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi 2024-06-03T02:58:10.139 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/acpi_ipmi.ko 2024-06-03T02:58:10.139 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/acpi_pad.ko 2024-06-03T02:58:10.139 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/ec_sys.ko 2024-06-03T02:58:10.139 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/platform_profile.ko 2024-06-03T02:58:10.139 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/sbs.ko 2024-06-03T02:58:10.139 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/sbshc.ko 2024-06-03T02:58:10.139 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/video.ko 2024-06-03T02:58:10.139 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata 2024-06-03T02:58:10.139 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/acard-ahci.ko 2024-06-03T02:58:10.139 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/ahci_platform.ko 2024-06-03T02:58:10.139 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/ata_generic.ko 2024-06-03T02:58:10.139 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/libahci_platform.ko 2024-06-03T02:58:10.140 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_acpi.ko 2024-06-03T02:58:10.140 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ali.ko 2024-06-03T02:58:10.140 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_amd.ko 2024-06-03T02:58:10.140 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_artop.ko 2024-06-03T02:58:10.140 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_atiixp.ko 2024-06-03T02:58:10.140 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_atp867x.ko 2024-06-03T02:58:10.140 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cmd640.ko 2024-06-03T02:58:10.140 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cmd64x.ko 2024-06-03T02:58:10.140 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cypress.ko 2024-06-03T02:58:10.140 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_efar.ko 2024-06-03T02:58:10.140 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt366.ko 2024-06-03T02:58:10.140 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt37x.ko 2024-06-03T02:58:10.140 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt3x2n.ko 2024-06-03T02:58:10.141 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt3x3.ko 2024-06-03T02:58:10.141 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_it8213.ko 2024-06-03T02:58:10.141 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_it821x.ko 2024-06-03T02:58:10.141 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_jmicron.ko 2024-06-03T02:58:10.141 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_marvell.ko 2024-06-03T02:58:10.141 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_mpiix.ko 2024-06-03T02:58:10.141 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_netcell.ko 2024-06-03T02:58:10.141 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ninja32.ko 2024-06-03T02:58:10.141 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ns87410.ko 2024-06-03T02:58:10.141 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ns87415.ko 2024-06-03T02:58:10.141 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_oldpiix.ko 2024-06-03T02:58:10.141 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_opti.ko 2024-06-03T02:58:10.141 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_optidma.ko 2024-06-03T02:58:10.141 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pcmcia.ko 2024-06-03T02:58:10.142 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pdc2027x.ko 2024-06-03T02:58:10.142 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-06-03T02:58:10.142 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_piccolo.ko 2024-06-03T02:58:10.142 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_rdc.ko 2024-06-03T02:58:10.142 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sch.ko 2024-06-03T02:58:10.142 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_serverworks.ko 2024-06-03T02:58:10.142 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sil680.ko 2024-06-03T02:58:10.142 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sis.ko 2024-06-03T02:58:10.142 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sl82c105.ko 2024-06-03T02:58:10.142 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_triflex.ko 2024-06-03T02:58:10.142 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_via.ko 2024-06-03T02:58:10.142 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pdc_adma.ko 2024-06-03T02:58:10.142 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_inic162x.ko 2024-06-03T02:58:10.143 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_mv.ko 2024-06-03T02:58:10.143 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_nv.ko 2024-06-03T02:58:10.143 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_promise.ko 2024-06-03T02:58:10.143 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_qstor.ko 2024-06-03T02:58:10.143 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sil.ko 2024-06-03T02:58:10.143 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sil24.ko 2024-06-03T02:58:10.143 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sis.ko 2024-06-03T02:58:10.143 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_svw.ko 2024-06-03T02:58:10.143 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sx4.ko 2024-06-03T02:58:10.143 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_uli.ko 2024-06-03T02:58:10.143 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_via.ko 2024-06-03T02:58:10.143 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_vsc.ko 2024-06-03T02:58:10.143 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/auxdisplay 2024-06-03T02:58:10.144 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/auxdisplay/ks0108.ko 2024-06-03T02:58:10.144 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base 2024-06-03T02:58:10.144 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base/regmap 2024-06-03T02:58:10.144 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base/regmap/regmap-i2c.ko 2024-06-03T02:58:10.144 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base/regmap/regmap-mmio.ko 2024-06-03T02:58:10.144 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/bcma 2024-06-03T02:58:10.144 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/bcma/bcma.ko 2024-06-03T02:58:10.144 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block 2024-06-03T02:58:10.144 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/aoe 2024-06-03T02:58:10.144 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/aoe/aoe.ko 2024-06-03T02:58:10.144 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/brd.ko 2024-06-03T02:58:10.144 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/drbd 2024-06-03T02:58:10.144 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/drbd/drbd.ko 2024-06-03T02:58:10.145 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/floppy.ko 2024-06-03T02:58:10.145 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/loop.ko 2024-06-03T02:58:10.145 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/mtip32xx 2024-06-03T02:58:10.145 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-06-03T02:58:10.145 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/nbd.ko 2024-06-03T02:58:10.145 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/null_blk 2024-06-03T02:58:10.145 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/null_blk/null_blk.ko 2024-06-03T02:58:10.145 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/pktcdvd.ko 2024-06-03T02:58:10.145 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/rbd.ko 2024-06-03T02:58:10.145 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/virtio_blk.ko 2024-06-03T02:58:10.145 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkback 2024-06-03T02:58:10.145 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-06-03T02:58:10.145 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkfront.ko 2024-06-03T02:58:10.145 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/zram 2024-06-03T02:58:10.146 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/zram/zram.ko 2024-06-03T02:58:10.146 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char 2024-06-03T02:58:10.146 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hangcheck-timer.ko 2024-06-03T02:58:10.146 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random 2024-06-03T02:58:10.146 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/amd-rng.ko 2024-06-03T02:58:10.146 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/intel-rng.ko 2024-06-03T02:58:10.146 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-06-03T02:58:10.146 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/via-rng.ko 2024-06-03T02:58:10.146 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/virtio-rng.ko 2024-06-03T02:58:10.146 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi 2024-06-03T02:58:10.146 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-06-03T02:58:10.146 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-06-03T02:58:10.146 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-06-03T02:58:10.147 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_si.ko 2024-06-03T02:58:10.147 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-06-03T02:58:10.147 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-06-03T02:58:10.147 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/lp.ko 2024-06-03T02:58:10.147 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/mwave 2024-06-03T02:58:10.147 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/mwave/mwave.ko 2024-06-03T02:58:10.147 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ppdev.ko 2024-06-03T02:58:10.147 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tlclk.ko 2024-06-03T02:58:10.147 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm 2024-06-03T02:58:10.147 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm.ko 2024-06-03T02:58:10.147 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_atmel.ko 2024-06-03T02:58:10.147 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_infineon.ko 2024-06-03T02:58:10.147 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_nsc.ko 2024-06-03T02:58:10.147 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_tis.ko 2024-06-03T02:58:10.148 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-06-03T02:58:10.148 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/uv_mmtimer.ko 2024-06-03T02:58:10.148 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/virtio_console.ko 2024-06-03T02:58:10.148 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq 2024-06-03T02:58:10.148 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-06-03T02:58:10.148 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-06-03T02:58:10.148 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/p4-clockmod.ko 2024-06-03T02:58:10.148 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-06-03T02:58:10.148 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/powernow-k8.ko 2024-06-03T02:58:10.148 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/speedstep-lib.ko 2024-06-03T02:58:10.148 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto 2024-06-03T02:58:10.148 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp 2024-06-03T02:58:10.148 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-06-03T02:58:10.148 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp/ccp.ko 2024-06-03T02:58:10.148 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel 2024-06-03T02:58:10.149 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat 2024-06-03T02:58:10.149 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_common 2024-06-03T02:58:10.149 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-06-03T02:58:10.149 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-06-03T02:58:10.149 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-06-03T02:58:10.149 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/padlock-aes.ko 2024-06-03T02:58:10.149 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/padlock-sha.ko 2024-06-03T02:58:10.149 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/virtio 2024-06-03T02:58:10.149 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-06-03T02:58:10.149 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dca 2024-06-03T02:58:10.149 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dca/dca.ko 2024-06-03T02:58:10.149 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/devfreq 2024-06-03T02:58:10.149 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-06-03T02:58:10.150 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma 2024-06-03T02:58:10.150 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/dw 2024-06-03T02:58:10.150 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/dw/dw_dmac.ko 2024-06-03T02:58:10.150 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/ioat 2024-06-03T02:58:10.150 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/ioat/ioatdma.ko 2024-06-03T02:58:10.150 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac 2024-06-03T02:58:10.150 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/amd64_edac.ko 2024-06-03T02:58:10.150 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/e752x_edac.ko 2024-06-03T02:58:10.150 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/edac_mce_amd.ko 2024-06-03T02:58:10.150 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i3000_edac.ko 2024-06-03T02:58:10.150 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i3200_edac.ko 2024-06-03T02:58:10.150 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i5100_edac.ko 2024-06-03T02:58:10.150 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i5400_edac.ko 2024-06-03T02:58:10.151 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i7300_edac.ko 2024-06-03T02:58:10.151 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i7core_edac.ko 2024-06-03T02:58:10.151 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i82975x_edac.ko 2024-06-03T02:58:10.151 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/ie31200_edac.ko 2024-06-03T02:58:10.151 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/sb_edac.ko 2024-06-03T02:58:10.151 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/x38_edac.ko 2024-06-03T02:58:10.151 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware 2024-06-03T02:58:10.151 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware/edd.ko 2024-06-03T02:58:10.151 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware/iscsi_ibft.ko 2024-06-03T02:58:10.151 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio 2024-06-03T02:58:10.151 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-ich.ko 2024-06-03T02:58:10.151 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-viperboard.ko 2024-06-03T02:58:10.151 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid 2024-06-03T02:58:10.152 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-appleir.ko 2024-06-03T02:58:10.152 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-aureal.ko 2024-06-03T02:58:10.152 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-axff.ko 2024-06-03T02:58:10.152 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-dr.ko 2024-06-03T02:58:10.152 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-elecom.ko 2024-06-03T02:58:10.152 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-elo.ko 2024-06-03T02:58:10.152 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-emsff.ko 2024-06-03T02:58:10.152 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gaff.ko 2024-06-03T02:58:10.152 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gt683r.ko 2024-06-03T02:58:10.152 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gyration.ko 2024-06-03T02:58:10.152 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtek-kbd.ko 2024-06-03T02:58:10.152 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtek-mouse.ko 2024-06-03T02:58:10.152 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtekff.ko 2024-06-03T02:58:10.152 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-hyperv.ko 2024-06-03T02:58:10.153 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-icade.ko 2024-06-03T02:58:10.153 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-keytouch.ko 2024-06-03T02:58:10.153 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-kye.ko 2024-06-03T02:58:10.153 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-lcpower.ko 2024-06-03T02:58:10.153 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-led.ko 2024-06-03T02:58:10.153 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-lenovo.ko 2024-06-03T02:58:10.153 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-logitech-dj.ko 2024-06-03T02:58:10.153 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-06-03T02:58:10.153 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-multitouch.ko 2024-06-03T02:58:10.153 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-ortek.ko 2024-06-03T02:58:10.153 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-penmount.ko 2024-06-03T02:58:10.153 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-petalynx.ko 2024-06-03T02:58:10.153 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-picolcd.ko 2024-06-03T02:58:10.153 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-pl.ko 2024-06-03T02:58:10.154 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-plantronics.ko 2024-06-03T02:58:10.154 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-primax.ko 2024-06-03T02:58:10.154 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-rmi.ko 2024-06-03T02:58:10.154 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-arvo.ko 2024-06-03T02:58:10.154 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-common.ko 2024-06-03T02:58:10.154 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-isku.ko 2024-06-03T02:58:10.154 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-kone.ko 2024-06-03T02:58:10.154 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-06-03T02:58:10.154 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-konepure.ko 2024-06-03T02:58:10.154 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-06-03T02:58:10.154 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-lua.ko 2024-06-03T02:58:10.154 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-pyra.ko 2024-06-03T02:58:10.155 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-ryos.ko 2024-06-03T02:58:10.155 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-savu.ko 2024-06-03T02:58:10.155 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat.ko 2024-06-03T02:58:10.155 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-saitek.ko 2024-06-03T02:58:10.155 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-samsung.ko 2024-06-03T02:58:10.155 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sensor-hub.ko 2024-06-03T02:58:10.155 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sjoy.ko 2024-06-03T02:58:10.155 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sony.ko 2024-06-03T02:58:10.155 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-speedlink.ko 2024-06-03T02:58:10.155 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-steelseries.ko 2024-06-03T02:58:10.155 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sunplus.ko 2024-06-03T02:58:10.155 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-thrustmaster.ko 2024-06-03T02:58:10.155 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-tivo.ko 2024-06-03T02:58:10.156 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-tmff.ko 2024-06-03T02:58:10.156 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-topseed.ko 2024-06-03T02:58:10.156 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-twinhan.ko 2024-06-03T02:58:10.156 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-uclogic.ko 2024-06-03T02:58:10.156 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-waltop.ko 2024-06-03T02:58:10.156 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-wiimote.ko 2024-06-03T02:58:10.156 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-xinmo.ko 2024-06-03T02:58:10.156 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-zpff.ko 2024-06-03T02:58:10.156 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-zydacron.ko 2024-06-03T02:58:10.156 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/uhid.ko 2024-06-03T02:58:10.156 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/wacom.ko 2024-06-03T02:58:10.156 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv 2024-06-03T02:58:10.156 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_balloon.ko 2024-06-03T02:58:10.156 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_utils.ko 2024-06-03T02:58:10.157 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_vmbus.ko 2024-06-03T02:58:10.157 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon 2024-06-03T02:58:10.157 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/abituguru.ko 2024-06-03T02:58:10.157 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/abituguru3.ko 2024-06-03T02:58:10.157 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/acpi_power_meter.ko 2024-06-03T02:58:10.157 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ad7414.ko 2024-06-03T02:58:10.157 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ad7418.ko 2024-06-03T02:58:10.157 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adc128d818.ko 2024-06-03T02:58:10.157 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1025.ko 2024-06-03T02:58:10.157 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1026.ko 2024-06-03T02:58:10.157 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1029.ko 2024-06-03T02:58:10.157 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1031.ko 2024-06-03T02:58:10.157 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm9240.ko 2024-06-03T02:58:10.158 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ads7828.ko 2024-06-03T02:58:10.158 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7410.ko 2024-06-03T02:58:10.158 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7411.ko 2024-06-03T02:58:10.158 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7462.ko 2024-06-03T02:58:10.158 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7470.ko 2024-06-03T02:58:10.158 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7475.ko 2024-06-03T02:58:10.158 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7x10.ko 2024-06-03T02:58:10.158 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/amc6821.ko 2024-06-03T02:58:10.158 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/applesmc.ko 2024-06-03T02:58:10.158 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asb100.ko 2024-06-03T02:58:10.158 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asc7621.ko 2024-06-03T02:58:10.158 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asus_atk0110.ko 2024-06-03T02:58:10.158 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/atxp1.ko 2024-06-03T02:58:10.159 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/coretemp.ko 2024-06-03T02:58:10.159 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-06-03T02:58:10.159 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/dme1737.ko 2024-06-03T02:58:10.159 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ds1621.ko 2024-06-03T02:58:10.159 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ds620.ko 2024-06-03T02:58:10.159 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/emc1403.ko 2024-06-03T02:58:10.159 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/emc6w201.ko 2024-06-03T02:58:10.159 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f71805f.ko 2024-06-03T02:58:10.159 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f71882fg.ko 2024-06-03T02:58:10.159 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f75375s.ko 2024-06-03T02:58:10.159 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/fam15h_power.ko 2024-06-03T02:58:10.159 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/fschmd.ko 2024-06-03T02:58:10.159 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/g760a.ko 2024-06-03T02:58:10.160 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/g762.ko 2024-06-03T02:58:10.160 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/gl518sm.ko 2024-06-03T02:58:10.160 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/gl520sm.ko 2024-06-03T02:58:10.160 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/hwmon-vid.ko 2024-06-03T02:58:10.160 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/i5500_temp.ko 2024-06-03T02:58:10.160 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/i5k_amb.ko 2024-06-03T02:58:10.160 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ibmaem.ko 2024-06-03T02:58:10.160 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ibmpex.ko 2024-06-03T02:58:10.160 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ina209.ko 2024-06-03T02:58:10.160 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ina2xx.ko 2024-06-03T02:58:10.160 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/it87.ko 2024-06-03T02:58:10.160 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/k10temp.ko 2024-06-03T02:58:10.160 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/k8temp.ko 2024-06-03T02:58:10.161 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lineage-pem.ko 2024-06-03T02:58:10.161 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm63.ko 2024-06-03T02:58:10.161 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm73.ko 2024-06-03T02:58:10.161 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm75.ko 2024-06-03T02:58:10.161 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm77.ko 2024-06-03T02:58:10.161 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm78.ko 2024-06-03T02:58:10.161 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm80.ko 2024-06-03T02:58:10.161 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm83.ko 2024-06-03T02:58:10.161 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm85.ko 2024-06-03T02:58:10.161 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm87.ko 2024-06-03T02:58:10.161 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm90.ko 2024-06-03T02:58:10.161 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm92.ko 2024-06-03T02:58:10.161 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm93.ko 2024-06-03T02:58:10.161 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95234.ko 2024-06-03T02:58:10.162 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95241.ko 2024-06-03T02:58:10.162 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95245.ko 2024-06-03T02:58:10.162 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc2945.ko 2024-06-03T02:58:10.162 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4151.ko 2024-06-03T02:58:10.162 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4215.ko 2024-06-03T02:58:10.162 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4222.ko 2024-06-03T02:58:10.162 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4245.ko 2024-06-03T02:58:10.162 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4260.ko 2024-06-03T02:58:10.162 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4261.ko 2024-06-03T02:58:10.162 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max16065.ko 2024-06-03T02:58:10.162 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max1619.ko 2024-06-03T02:58:10.162 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max1668.ko 2024-06-03T02:58:10.162 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max197.ko 2024-06-03T02:58:10.163 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6639.ko 2024-06-03T02:58:10.163 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6650.ko 2024-06-03T02:58:10.163 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6697.ko 2024-06-03T02:58:10.163 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/mcp3021.ko 2024-06-03T02:58:10.163 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6683.ko 2024-06-03T02:58:10.163 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6775-core.ko 2024-06-03T02:58:10.163 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6775.ko 2024-06-03T02:58:10.163 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct7802.ko 2024-06-03T02:58:10.163 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pc87360.ko 2024-06-03T02:58:10.163 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pc87427.ko 2024-06-03T02:58:10.163 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pcf8591.ko 2024-06-03T02:58:10.163 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus 2024-06-03T02:58:10.163 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-06-03T02:58:10.163 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-06-03T02:58:10.164 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-06-03T02:58:10.164 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max16064.ko 2024-06-03T02:58:10.164 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max34440.ko 2024-06-03T02:58:10.164 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max8688.ko 2024-06-03T02:58:10.164 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-06-03T02:58:10.164 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-06-03T02:58:10.164 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-06-03T02:58:10.164 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-06-03T02:58:10.164 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-06-03T02:58:10.164 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-06-03T02:58:10.164 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/powr1220.ko 2024-06-03T02:58:10.164 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch5627.ko 2024-06-03T02:58:10.164 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch5636.ko 2024-06-03T02:58:10.165 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch56xx-common.ko 2024-06-03T02:58:10.165 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sht15.ko 2024-06-03T02:58:10.165 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sht21.ko 2024-06-03T02:58:10.165 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/shtc1.ko 2024-06-03T02:58:10.165 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sis5595.ko 2024-06-03T02:58:10.165 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47b397.ko 2024-06-03T02:58:10.165 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47m1.ko 2024-06-03T02:58:10.165 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47m192.ko 2024-06-03T02:58:10.165 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/thmc50.ko 2024-06-03T02:58:10.165 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp102.ko 2024-06-03T02:58:10.165 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp103.ko 2024-06-03T02:58:10.165 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp401.ko 2024-06-03T02:58:10.165 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp421.ko 2024-06-03T02:58:10.165 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/via-cputemp.ko 2024-06-03T02:58:10.166 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/via686a.ko 2024-06-03T02:58:10.166 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/vt1211.ko 2024-06-03T02:58:10.166 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/vt8231.ko 2024-06-03T02:58:10.166 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83627ehf.ko 2024-06-03T02:58:10.166 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83627hf.ko 2024-06-03T02:58:10.166 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83781d.ko 2024-06-03T02:58:10.166 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83791d.ko 2024-06-03T02:58:10.166 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83792d.ko 2024-06-03T02:58:10.166 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83793.ko 2024-06-03T02:58:10.166 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83795.ko 2024-06-03T02:58:10.166 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83l785ts.ko 2024-06-03T02:58:10.166 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83l786ng.ko 2024-06-03T02:58:10.166 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c 2024-06-03T02:58:10.167 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos 2024-06-03T02:58:10.167 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-06-03T02:58:10.167 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-06-03T02:58:10.167 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses 2024-06-03T02:58:10.167 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-06-03T02:58:10.167 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-06-03T02:58:10.167 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-06-03T02:58:10.167 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-06-03T02:58:10.167 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-06-03T02:58:10.167 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-06-03T02:58:10.167 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-06-03T02:58:10.167 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-06-03T02:58:10.167 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-i801.ko 2024-06-03T02:58:10.168 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-isch.ko 2024-06-03T02:58:10.168 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-06-03T02:58:10.168 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-06-03T02:58:10.168 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-06-03T02:58:10.168 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-parport.ko 2024-06-03T02:58:10.168 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-06-03T02:58:10.168 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-06-03T02:58:10.168 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-06-03T02:58:10.168 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-06-03T02:58:10.168 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-06-03T02:58:10.168 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-06-03T02:58:10.168 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-via.ko 2024-06-03T02:58:10.168 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-06-03T02:58:10.168 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-06-03T02:58:10.169 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-dev.ko 2024-06-03T02:58:10.169 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-mux.ko 2024-06-03T02:58:10.169 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-06-03T02:58:10.169 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-smbus.ko 2024-06-03T02:58:10.169 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-stub.ko 2024-06-03T02:58:10.169 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband 2024-06-03T02:58:10.169 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core 2024-06-03T02:58:10.169 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_cm.ko 2024-06-03T02:58:10.169 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_core.ko 2024-06-03T02:58:10.169 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_umad.ko 2024-06-03T02:58:10.169 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-06-03T02:58:10.169 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/iw_cm.ko 2024-06-03T02:58:10.169 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/rdma_cm.ko 2024-06-03T02:58:10.170 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-06-03T02:58:10.170 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw 2024-06-03T02:58:10.170 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/cxgb4 2024-06-03T02:58:10.170 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-06-03T02:58:10.170 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx4 2024-06-03T02:58:10.170 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-06-03T02:58:10.170 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx5 2024-06-03T02:58:10.170 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-06-03T02:58:10.170 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mthca 2024-06-03T02:58:10.170 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-06-03T02:58:10.170 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp 2024-06-03T02:58:10.170 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/ipoib 2024-06-03T02:58:10.171 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-06-03T02:58:10.171 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/iser 2024-06-03T02:58:10.171 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-06-03T02:58:10.171 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/isert 2024-06-03T02:58:10.171 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-06-03T02:58:10.171 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srp 2024-06-03T02:58:10.171 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-06-03T02:58:10.171 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srpt 2024-06-03T02:58:10.171 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-06-03T02:58:10.171 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input 2024-06-03T02:58:10.171 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport 2024-06-03T02:58:10.171 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-06-03T02:58:10.171 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/fm801-gp.ko 2024-06-03T02:58:10.171 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/gameport.ko 2024-06-03T02:58:10.172 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/joydev.ko 2024-06-03T02:58:10.172 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard 2024-06-03T02:58:10.172 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard/gpio_keys.ko 2024-06-03T02:58:10.172 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse 2024-06-03T02:58:10.172 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/appletouch.ko 2024-06-03T02:58:10.172 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/bcm5974.ko 2024-06-03T02:58:10.172 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/cyapatp.ko 2024-06-03T02:58:10.172 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/elan_i2c.ko 2024-06-03T02:58:10.172 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/sermouse.ko 2024-06-03T02:58:10.172 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-06-03T02:58:10.172 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/synaptics_usb.ko 2024-06-03T02:58:10.172 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/vsxxxaa.ko 2024-06-03T02:58:10.172 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/rmi4 2024-06-03T02:58:10.173 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/rmi4/rmi_core.ko 2024-06-03T02:58:10.173 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio 2024-06-03T02:58:10.173 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/altera_ps2.ko 2024-06-03T02:58:10.173 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/arc_ps2.ko 2024-06-03T02:58:10.173 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-06-03T02:58:10.173 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/serio_raw.ko 2024-06-03T02:58:10.173 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/sparse-keymap.ko 2024-06-03T02:58:10.173 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds 2024-06-03T02:58:10.173 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-blinkm.ko 2024-06-03T02:58:10.173 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-lm3530.ko 2024-06-03T02:58:10.173 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-lp3944.ko 2024-06-03T02:58:10.173 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-ss4200.ko 2024-06-03T02:58:10.173 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger 2024-06-03T02:58:10.174 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-06-03T02:58:10.174 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-06-03T02:58:10.174 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-06-03T02:58:10.174 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-06-03T02:58:10.174 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-06-03T02:58:10.174 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-06-03T02:58:10.174 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-06-03T02:58:10.174 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-06-03T02:58:10.174 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-06-03T02:58:10.174 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md 2024-06-03T02:58:10.174 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/bcache 2024-06-03T02:58:10.174 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/bcache/bcache.ko 2024-06-03T02:58:10.175 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-bio-prison.ko 2024-06-03T02:58:10.175 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-cache-smq.ko 2024-06-03T02:58:10.175 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-cache.ko 2024-06-03T02:58:10.175 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-crypt.ko 2024-06-03T02:58:10.175 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-delay.ko 2024-06-03T02:58:10.175 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-flakey.ko 2024-06-03T02:58:10.175 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-log-userspace.ko 2024-06-03T02:58:10.175 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-multipath.ko 2024-06-03T02:58:10.175 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-queue-length.ko 2024-06-03T02:58:10.175 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-raid.ko 2024-06-03T02:58:10.175 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-round-robin.ko 2024-06-03T02:58:10.175 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-service-time.ko 2024-06-03T02:58:10.175 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-switch.ko 2024-06-03T02:58:10.175 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-thin-pool.ko 2024-06-03T02:58:10.176 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-verity.ko 2024-06-03T02:58:10.176 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/persistent-data 2024-06-03T02:58:10.176 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-06-03T02:58:10.176 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid0.ko 2024-06-03T02:58:10.176 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid1.ko 2024-06-03T02:58:10.176 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid10.ko 2024-06-03T02:58:10.176 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid456.ko 2024-06-03T02:58:10.176 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message 2024-06-03T02:58:10.176 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion 2024-06-03T02:58:10.176 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptbase.ko 2024-06-03T02:58:10.176 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptctl.ko 2024-06-03T02:58:10.176 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptfc.ko 2024-06-03T02:58:10.176 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptlan.ko 2024-06-03T02:58:10.177 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptsas.ko 2024-06-03T02:58:10.177 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptscsih.ko 2024-06-03T02:58:10.177 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptspi.ko 2024-06-03T02:58:10.177 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd 2024-06-03T02:58:10.177 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/lpc_ich.ko 2024-06-03T02:58:10.177 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/lpc_sch.ko 2024-06-03T02:58:10.177 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/mfd-core.ko 2024-06-03T02:58:10.177 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/sm501.ko 2024-06-03T02:58:10.177 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/viperboard.ko 2024-06-03T02:58:10.177 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/vx855.ko 2024-06-03T02:58:10.177 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/wl1273-core.ko 2024-06-03T02:58:10.177 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc 2024-06-03T02:58:10.177 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/altera-stapl 2024-06-03T02:58:10.177 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-06-03T02:58:10.178 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/apds9802als.ko 2024-06-03T02:58:10.178 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/apds990x.ko 2024-06-03T02:58:10.178 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/bh1770glc.ko 2024-06-03T02:58:10.178 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/cb710 2024-06-03T02:58:10.178 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/cb710/cb710.ko 2024-06-03T02:58:10.178 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom 2024-06-03T02:58:10.178 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/at24.ko 2024-06-03T02:58:10.178 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-06-03T02:58:10.178 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/max6875.ko 2024-06-03T02:58:10.178 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/enclosure.ko 2024-06-03T02:58:10.178 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/hpilo.ko 2024-06-03T02:58:10.178 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/isl29003.ko 2024-06-03T02:58:10.179 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/isl29020.ko 2024-06-03T02:58:10.179 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d 2024-06-03T02:58:10.179 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-06-03T02:58:10.179 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-06-03T02:58:10.179 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei 2024-06-03T02:58:10.179 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei-me.ko 2024-06-03T02:58:10.179 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei-txe.ko 2024-06-03T02:58:10.179 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei.ko 2024-06-03T02:58:10.179 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-gru 2024-06-03T02:58:10.179 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-gru/gru.ko 2024-06-03T02:58:10.179 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp 2024-06-03T02:58:10.179 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp/xp.ko 2024-06-03T02:58:10.179 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp/xpc.ko 2024-06-03T02:58:10.180 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-06-03T02:58:10.180 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tifm_7xx1.ko 2024-06-03T02:58:10.180 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tifm_core.ko 2024-06-03T02:58:10.180 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tsl2550.ko 2024-06-03T02:58:10.180 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_balloon.ko 2024-06-03T02:58:10.180 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_vmci 2024-06-03T02:58:10.180 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-06-03T02:58:10.180 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net 2024-06-03T02:58:10.180 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/bonding 2024-06-03T02:58:10.180 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/bonding/bonding.ko 2024-06-03T02:58:10.180 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa 2024-06-03T02:58:10.180 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/b53 2024-06-03T02:58:10.180 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/b53/b53_common.ko 2024-06-03T02:58:10.180 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/bcm-sf2.ko 2024-06-03T02:58:10.181 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6060.ko 2024-06-03T02:58:10.181 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6xxx 2024-06-03T02:58:10.181 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-06-03T02:58:10.181 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dummy.ko 2024-06-03T02:58:10.181 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/eql.ko 2024-06-03T02:58:10.181 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet 2024-06-03T02:58:10.181 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com 2024-06-03T02:58:10.181 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-06-03T02:58:10.181 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-06-03T02:58:10.181 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-06-03T02:58:10.181 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-06-03T02:58:10.181 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390 2024-06-03T02:58:10.181 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/8390.ko 2024-06-03T02:58:10.182 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-06-03T02:58:10.182 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-06-03T02:58:10.182 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-06-03T02:58:10.182 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/adaptec 2024-06-03T02:58:10.182 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-06-03T02:58:10.182 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/agere 2024-06-03T02:58:10.182 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/agere/et131x.ko 2024-06-03T02:58:10.182 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/alteon 2024-06-03T02:58:10.182 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-06-03T02:58:10.182 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/altera 2024-06-03T02:58:10.182 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-06-03T02:58:10.182 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd 2024-06-03T02:58:10.182 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-06-03T02:58:10.183 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-06-03T02:58:10.183 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-06-03T02:58:10.183 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros 2024-06-03T02:58:10.183 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/alx 2024-06-03T02:58:10.183 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-06-03T02:58:10.183 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1c 2024-06-03T02:58:10.183 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-06-03T02:58:10.183 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1e 2024-06-03T02:58:10.183 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-06-03T02:58:10.183 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx 2024-06-03T02:58:10.183 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-06-03T02:58:10.183 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-06-03T02:58:10.183 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom 2024-06-03T02:58:10.184 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-06-03T02:58:10.184 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-06-03T02:58:10.184 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-06-03T02:58:10.184 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-06-03T02:58:10.184 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-06-03T02:58:10.184 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/genet 2024-06-03T02:58:10.184 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-06-03T02:58:10.184 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-06-03T02:58:10.184 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade 2024-06-03T02:58:10.184 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade/bna 2024-06-03T02:58:10.184 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-06-03T02:58:10.184 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio 2024-06-03T02:58:10.184 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb 2024-06-03T02:58:10.184 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-06-03T02:58:10.185 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-06-03T02:58:10.185 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-06-03T02:58:10.185 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-06-03T02:58:10.185 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-06-03T02:58:10.185 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-06-03T02:58:10.185 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-06-03T02:58:10.185 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-06-03T02:58:10.185 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-06-03T02:58:10.185 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco 2024-06-03T02:58:10.185 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco/enic 2024-06-03T02:58:10.185 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-06-03T02:58:10.186 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec 2024-06-03T02:58:10.186 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip 2024-06-03T02:58:10.186 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-06-03T02:58:10.186 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-06-03T02:58:10.186 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-06-03T02:58:10.186 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-06-03T02:58:10.186 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-06-03T02:58:10.186 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-06-03T02:58:10.186 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink 2024-06-03T02:58:10.186 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-06-03T02:58:10.186 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-06-03T02:58:10.186 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dnet.ko 2024-06-03T02:58:10.187 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex 2024-06-03T02:58:10.187 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex/benet 2024-06-03T02:58:10.187 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-06-03T02:58:10.187 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ethoc.ko 2024-06-03T02:58:10.187 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/fealnx.ko 2024-06-03T02:58:10.187 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel 2024-06-03T02:58:10.187 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e100.ko 2024-06-03T02:58:10.187 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000 2024-06-03T02:58:10.187 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-06-03T02:58:10.187 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000e 2024-06-03T02:58:10.187 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-06-03T02:58:10.187 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/fm10k 2024-06-03T02:58:10.187 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-06-03T02:58:10.187 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/i40e 2024-06-03T02:58:10.188 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-06-03T02:58:10.188 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/iavf 2024-06-03T02:58:10.188 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-06-03T02:58:10.188 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igb 2024-06-03T02:58:10.188 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-06-03T02:58:10.188 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igbvf 2024-06-03T02:58:10.188 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-06-03T02:58:10.188 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbe 2024-06-03T02:58:10.188 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-06-03T02:58:10.188 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbevf 2024-06-03T02:58:10.188 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-06-03T02:58:10.188 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/jme.ko 2024-06-03T02:58:10.188 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell 2024-06-03T02:58:10.189 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-06-03T02:58:10.189 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/skge.ko 2024-06-03T02:58:10.189 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-06-03T02:58:10.189 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox 2024-06-03T02:58:10.189 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4 2024-06-03T02:58:10.189 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-06-03T02:58:10.189 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-06-03T02:58:10.189 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5 2024-06-03T02:58:10.189 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-06-03T02:58:10.189 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-06-03T02:58:10.189 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/micrel 2024-06-03T02:58:10.189 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-06-03T02:58:10.189 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom 2024-06-03T02:58:10.189 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom/myri10ge 2024-06-03T02:58:10.190 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-06-03T02:58:10.190 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi 2024-06-03T02:58:10.190 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-06-03T02:58:10.190 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-06-03T02:58:10.190 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/neterion 2024-06-03T02:58:10.190 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-06-03T02:58:10.190 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/nvidia 2024-06-03T02:58:10.190 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-06-03T02:58:10.190 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines 2024-06-03T02:58:10.190 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-06-03T02:58:10.190 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-06-03T02:58:10.190 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic 2024-06-03T02:58:10.190 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/netxen 2024-06-03T02:58:10.191 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-06-03T02:58:10.191 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-06-03T02:58:10.191 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-06-03T02:58:10.191 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-06-03T02:58:10.191 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rdc 2024-06-03T02:58:10.191 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-06-03T02:58:10.191 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek 2024-06-03T02:58:10.191 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-06-03T02:58:10.191 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-06-03T02:58:10.191 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/atp.ko 2024-06-03T02:58:10.191 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-06-03T02:58:10.191 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rocker 2024-06-03T02:58:10.191 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-06-03T02:58:10.191 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sfc 2024-06-03T02:58:10.192 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-06-03T02:58:10.192 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/silan 2024-06-03T02:58:10.192 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-06-03T02:58:10.192 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis 2024-06-03T02:58:10.192 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis/sis190.ko 2024-06-03T02:58:10.192 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis/sis900.ko 2024-06-03T02:58:10.192 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc 2024-06-03T02:58:10.192 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-06-03T02:58:10.192 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-06-03T02:58:10.192 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-06-03T02:58:10.192 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-06-03T02:58:10.192 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro 2024-06-03T02:58:10.192 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac 2024-06-03T02:58:10.193 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-06-03T02:58:10.193 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-06-03T02:58:10.193 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun 2024-06-03T02:58:10.193 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/cassini.ko 2024-06-03T02:58:10.193 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/niu.ko 2024-06-03T02:58:10.193 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/sungem.ko 2024-06-03T02:58:10.193 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-06-03T02:58:10.193 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/tehuti 2024-06-03T02:58:10.193 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-06-03T02:58:10.193 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ti 2024-06-03T02:58:10.193 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ti/tlan.ko 2024-06-03T02:58:10.193 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via 2024-06-03T02:58:10.193 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-06-03T02:58:10.193 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-06-03T02:58:10.194 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet 2024-06-03T02:58:10.194 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-06-03T02:58:10.194 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-06-03T02:58:10.194 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/xircom 2024-06-03T02:58:10.194 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-06-03T02:58:10.194 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/geneve.ko 2024-06-03T02:58:10.194 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/hyperv 2024-06-03T02:58:10.194 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-06-03T02:58:10.194 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ieee802154 2024-06-03T02:58:10.194 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ieee802154/fakelb.ko 2024-06-03T02:58:10.194 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ifb.ko 2024-06-03T02:58:10.194 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ipvlan 2024-06-03T02:58:10.194 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ipvlan/ipvlan.ko 2024-06-03T02:58:10.195 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/macvlan.ko 2024-06-03T02:58:10.195 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/macvtap.ko 2024-06-03T02:58:10.195 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio 2024-06-03T02:58:10.195 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio.ko 2024-06-03T02:58:10.195 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-06-03T02:58:10.195 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-06-03T02:58:10.195 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-regmap.ko 2024-06-03T02:58:10.195 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mii.ko 2024-06-03T02:58:10.195 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/net_failover.ko 2024-06-03T02:58:10.195 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/netconsole.ko 2024-06-03T02:58:10.195 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/nlmon.ko 2024-06-03T02:58:10.195 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs 2024-06-03T02:58:10.195 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs/pcs-lynx.ko 2024-06-03T02:58:10.196 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-06-03T02:58:10.196 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy 2024-06-03T02:58:10.196 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/amd.ko 2024-06-03T02:58:10.196 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-06-03T02:58:10.196 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-06-03T02:58:10.196 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm7xxx.ko 2024-06-03T02:58:10.196 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm87xx.ko 2024-06-03T02:58:10.196 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/broadcom.ko 2024-06-03T02:58:10.196 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/cicada.ko 2024-06-03T02:58:10.196 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/davicom.ko 2024-06-03T02:58:10.196 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/dp83640.ko 2024-06-03T02:58:10.196 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/et1011c.ko 2024-06-03T02:58:10.197 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/icplus.ko 2024-06-03T02:58:10.197 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/lxt.ko 2024-06-03T02:58:10.197 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/marvell.ko 2024-06-03T02:58:10.197 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/micrel.ko 2024-06-03T02:58:10.197 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/national.ko 2024-06-03T02:58:10.197 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/phylink.ko 2024-06-03T02:58:10.197 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/qsemi.ko 2024-06-03T02:58:10.197 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/realtek.ko 2024-06-03T02:58:10.197 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/smsc.ko 2024-06-03T02:58:10.197 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/ste10Xp.ko 2024-06-03T02:58:10.197 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/vitesse.ko 2024-06-03T02:58:10.197 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp 2024-06-03T02:58:10.197 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/bsd_comp.ko 2024-06-03T02:58:10.197 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_async.ko 2024-06-03T02:58:10.197 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_deflate.ko 2024-06-03T02:58:10.198 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_generic.ko 2024-06-03T02:58:10.198 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_mppe.ko 2024-06-03T02:58:10.198 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_synctty.ko 2024-06-03T02:58:10.198 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pppoe.ko 2024-06-03T02:58:10.198 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pppox.ko 2024-06-03T02:58:10.198 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pptp.ko 2024-06-03T02:58:10.198 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip 2024-06-03T02:58:10.198 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip/slhc.ko 2024-06-03T02:58:10.198 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip/slip.ko 2024-06-03T02:58:10.198 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/sungem_phy.ko 2024-06-03T02:58:10.198 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/tap.ko 2024-06-03T02:58:10.198 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team 2024-06-03T02:58:10.198 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team.ko 2024-06-03T02:58:10.199 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_activebackup.ko 2024-06-03T02:58:10.199 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_broadcast.ko 2024-06-03T02:58:10.199 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-06-03T02:58:10.199 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_random.ko 2024-06-03T02:58:10.199 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-06-03T02:58:10.199 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/tun.ko 2024-06-03T02:58:10.199 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/veth.ko 2024-06-03T02:58:10.199 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/virtio_net.ko 2024-06-03T02:58:10.199 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vmxnet3 2024-06-03T02:58:10.199 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-06-03T02:58:10.199 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vxlan 2024-06-03T02:58:10.199 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vxlan/vxlan.ko 2024-06-03T02:58:10.199 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netback 2024-06-03T02:58:10.199 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netback/xen-netback.ko 2024-06-03T02:58:10.200 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netfront.ko 2024-06-03T02:58:10.200 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ntb 2024-06-03T02:58:10.200 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ntb/ntb.ko 2024-06-03T02:58:10.200 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme 2024-06-03T02:58:10.200 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host 2024-06-03T02:58:10.200 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host/nvme-core.ko 2024-06-03T02:58:10.200 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host/nvme.ko 2024-06-03T02:58:10.200 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport 2024-06-03T02:58:10.200 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport.ko 2024-06-03T02:58:10.200 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_cs.ko 2024-06-03T02:58:10.200 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_pc.ko 2024-06-03T02:58:10.200 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_serial.ko 2024-06-03T02:58:10.200 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci 2024-06-03T02:58:10.201 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/hotplug 2024-06-03T02:58:10.201 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-06-03T02:58:10.201 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/pcie 2024-06-03T02:58:10.201 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/pcie/aer_inject.ko 2024-06-03T02:58:10.201 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/xen-pcifront.ko 2024-06-03T02:58:10.201 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia 2024-06-03T02:58:10.201 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/i82092.ko 2024-06-03T02:58:10.201 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/pd6729.ko 2024-06-03T02:58:10.201 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/yenta_socket.ko 2024-06-03T02:58:10.201 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl 2024-06-03T02:58:10.201 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl/intel 2024-06-03T02:58:10.201 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-06-03T02:58:10.201 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform 2024-06-03T02:58:10.201 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86 2024-06-03T02:58:10.202 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/acer-wmi.ko 2024-06-03T02:58:10.202 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/acerhdf.ko 2024-06-03T02:58:10.202 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/apple-gmux.ko 2024-06-03T02:58:10.202 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-laptop.ko 2024-06-03T02:58:10.202 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-06-03T02:58:10.202 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-wmi.ko 2024-06-03T02:58:10.202 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/classmate-laptop.ko 2024-06-03T02:58:10.202 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-06-03T02:58:10.202 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-06-03T02:58:10.202 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-06-03T02:58:10.202 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-06-03T02:58:10.202 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/hdaps.ko 2024-06-03T02:58:10.202 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel 2024-06-03T02:58:10.203 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-06-03T02:58:10.203 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel_ips.ko 2024-06-03T02:58:10.203 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/msi-wmi.ko 2024-06-03T02:58:10.203 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/mxm-wmi.ko 2024-06-03T02:58:10.203 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-06-03T02:58:10.203 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/samsung-laptop.ko 2024-06-03T02:58:10.203 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/samsung-q10.ko 2024-06-03T02:58:10.203 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/topstar-laptop.ko 2024-06-03T02:58:10.203 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-06-03T02:58:10.203 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/toshiba_haps.ko 2024-06-03T02:58:10.203 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/wmi-bmof.ko 2024-06-03T02:58:10.203 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/wmi.ko 2024-06-03T02:58:10.203 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap 2024-06-03T02:58:10.203 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap/intel_rapl_common.ko 2024-06-03T02:58:10.204 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap/intel_rapl_msr.ko 2024-06-03T02:58:10.204 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps 2024-06-03T02:58:10.204 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients 2024-06-03T02:58:10.204 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps-gpio.ko 2024-06-03T02:58:10.204 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps-ldisc.ko 2024-06-03T02:58:10.204 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps_parport.ko 2024-06-03T02:58:10.204 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/pps_core.ko 2024-06-03T02:58:10.204 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp 2024-06-03T02:58:10.204 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp/ptp.ko 2024-06-03T02:58:10.204 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp/ptp_kvm.ko 2024-06-03T02:58:10.204 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras 2024-06-03T02:58:10.204 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd 2024-06-03T02:58:10.204 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/atl 2024-06-03T02:58:10.205 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-06-03T02:58:10.205 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/fmpm.ko 2024-06-03T02:58:10.205 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc 2024-06-03T02:58:10.205 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-bq32k.ko 2024-06-03T02:58:10.205 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1286.ko 2024-06-03T02:58:10.205 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1307.ko 2024-06-03T02:58:10.205 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1374.ko 2024-06-03T02:58:10.205 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1511.ko 2024-06-03T02:58:10.205 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1553.ko 2024-06-03T02:58:10.205 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1672.ko 2024-06-03T02:58:10.205 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1742.ko 2024-06-03T02:58:10.205 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds2404.ko 2024-06-03T02:58:10.205 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds3232.ko 2024-06-03T02:58:10.205 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-em3027.ko 2024-06-03T02:58:10.206 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-fm3130.ko 2024-06-03T02:58:10.206 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-isl12022.ko 2024-06-03T02:58:10.206 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-isl1208.ko 2024-06-03T02:58:10.206 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m41t80.ko 2024-06-03T02:58:10.206 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m48t35.ko 2024-06-03T02:58:10.206 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m48t59.ko 2024-06-03T02:58:10.206 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-max6900.ko 2024-06-03T02:58:10.206 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-msm6242.ko 2024-06-03T02:58:10.206 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf2127.ko 2024-06-03T02:58:10.206 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf85063.ko 2024-06-03T02:58:10.206 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8523.ko 2024-06-03T02:58:10.206 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8563.ko 2024-06-03T02:58:10.206 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8583.ko 2024-06-03T02:58:10.207 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rp5c01.ko 2024-06-03T02:58:10.207 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rs5c372.ko 2024-06-03T02:58:10.207 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-06-03T02:58:10.207 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rx8025.ko 2024-06-03T02:58:10.207 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rx8581.ko 2024-06-03T02:58:10.207 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-06-03T02:58:10.207 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-x1205.ko 2024-06-03T02:58:10.207 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi 2024-06-03T02:58:10.207 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-9xxx.ko 2024-06-03T02:58:10.207 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-sas.ko 2024-06-03T02:58:10.207 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-xxxx.ko 2024-06-03T02:58:10.207 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/BusLogic.ko 2024-06-03T02:58:10.207 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/a100u2w.ko 2024-06-03T02:58:10.207 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aacraid 2024-06-03T02:58:10.208 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aacraid/aacraid.ko 2024-06-03T02:58:10.208 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/advansys.ko 2024-06-03T02:58:10.208 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx 2024-06-03T02:58:10.208 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-06-03T02:58:10.208 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-06-03T02:58:10.208 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/am53c974.ko 2024-06-03T02:58:10.208 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/arcmsr 2024-06-03T02:58:10.208 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-06-03T02:58:10.208 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/atp870u.ko 2024-06-03T02:58:10.208 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/be2iscsi 2024-06-03T02:58:10.208 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-06-03T02:58:10.208 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bfa 2024-06-03T02:58:10.208 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bfa/bfa.ko 2024-06-03T02:58:10.209 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2fc 2024-06-03T02:58:10.209 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-06-03T02:58:10.209 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2i 2024-06-03T02:58:10.209 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-06-03T02:58:10.209 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ch.ko 2024-06-03T02:58:10.209 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/csiostor 2024-06-03T02:58:10.209 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/csiostor/csiostor.ko 2024-06-03T02:58:10.209 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi 2024-06-03T02:58:10.209 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb3i 2024-06-03T02:58:10.209 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-06-03T02:58:10.209 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb4i 2024-06-03T02:58:10.209 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-06-03T02:58:10.209 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-06-03T02:58:10.209 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/dc395x.ko 2024-06-03T02:58:10.210 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler 2024-06-03T02:58:10.210 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-06-03T02:58:10.210 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-06-03T02:58:10.210 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-06-03T02:58:10.210 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-06-03T02:58:10.210 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/dmx3191d.ko 2024-06-03T02:58:10.210 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esas2r 2024-06-03T02:58:10.210 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esas2r/esas2r.ko 2024-06-03T02:58:10.210 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esp_scsi.ko 2024-06-03T02:58:10.210 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe 2024-06-03T02:58:10.210 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe/fcoe.ko 2024-06-03T02:58:10.210 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-06-03T02:58:10.210 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fnic 2024-06-03T02:58:10.211 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fnic/fnic.ko 2024-06-03T02:58:10.211 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hpsa.ko 2024-06-03T02:58:10.211 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hptiop.ko 2024-06-03T02:58:10.211 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hv_storvsc.ko 2024-06-03T02:58:10.211 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/initio.ko 2024-06-03T02:58:10.211 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ipr.ko 2024-06-03T02:58:10.211 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ips.ko 2024-06-03T02:58:10.211 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/isci 2024-06-03T02:58:10.211 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/isci/isci.ko 2024-06-03T02:58:10.211 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-06-03T02:58:10.211 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/iscsi_tcp.ko 2024-06-03T02:58:10.211 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libfc 2024-06-03T02:58:10.211 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libfc/libfc.ko 2024-06-03T02:58:10.211 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libiscsi.ko 2024-06-03T02:58:10.212 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libiscsi_tcp.ko 2024-06-03T02:58:10.212 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libsas 2024-06-03T02:58:10.212 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libsas/libsas.ko 2024-06-03T02:58:10.212 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/lpfc 2024-06-03T02:58:10.212 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/lpfc/lpfc.ko 2024-06-03T02:58:10.212 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid 2024-06-03T02:58:10.212 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid.ko 2024-06-03T02:58:10.212 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-06-03T02:58:10.212 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-06-03T02:58:10.212 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-06-03T02:58:10.212 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mpt3sas 2024-06-03T02:58:10.212 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-06-03T02:58:10.212 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvsas 2024-06-03T02:58:10.212 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvsas/mvsas.ko 2024-06-03T02:58:10.213 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvumi.ko 2024-06-03T02:58:10.213 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pm8001 2024-06-03T02:58:10.213 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-06-03T02:58:10.213 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pmcraid.ko 2024-06-03T02:58:10.213 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla1280.ko 2024-06-03T02:58:10.213 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx 2024-06-03T02:58:10.213 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-06-03T02:58:10.213 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-06-03T02:58:10.213 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla4xxx 2024-06-03T02:58:10.213 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-06-03T02:58:10.213 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/raid_class.ko 2024-06-03T02:58:10.213 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_debug.ko 2024-06-03T02:58:10.213 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_fc.ko 2024-06-03T02:58:10.213 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-06-03T02:58:10.214 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_sas.ko 2024-06-03T02:58:10.214 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_spi.ko 2024-06-03T02:58:10.214 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_srp.ko 2024-06-03T02:58:10.214 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ses.ko 2024-06-03T02:58:10.214 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/st.ko 2024-06-03T02:58:10.214 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/stex.ko 2024-06-03T02:58:10.214 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/sym53c8xx_2 2024-06-03T02:58:10.214 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-06-03T02:58:10.214 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/virtio_scsi.ko 2024-06-03T02:58:10.214 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/vmw_pvscsi.ko 2024-06-03T02:58:10.214 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/wd719x.ko 2024-06-03T02:58:10.214 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ssb 2024-06-03T02:58:10.214 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ssb/ssb.ko 2024-06-03T02:58:10.215 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target 2024-06-03T02:58:10.215 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/iscsi 2024-06-03T02:58:10.215 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-06-03T02:58:10.215 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/loopback 2024-06-03T02:58:10.215 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/loopback/tcm_loop.ko 2024-06-03T02:58:10.215 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_file.ko 2024-06-03T02:58:10.215 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_iblock.ko 2024-06-03T02:58:10.215 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_mod.ko 2024-06-03T02:58:10.215 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_pscsi.ko 2024-06-03T02:58:10.215 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_user.ko 2024-06-03T02:58:10.215 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/tcm_fc 2024-06-03T02:58:10.215 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-06-03T02:58:10.215 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal 2024-06-03T02:58:10.215 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel 2024-06-03T02:58:10.216 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal 2024-06-03T02:58:10.216 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-06-03T02:58:10.216 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-06-03T02:58:10.216 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-06-03T02:58:10.216 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-06-03T02:58:10.216 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-06-03T02:58:10.216 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-06-03T02:58:10.216 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-06-03T02:58:10.216 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-06-03T02:58:10.216 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-06-03T02:58:10.216 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-06-03T02:58:10.216 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-06-03T02:58:10.217 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-06-03T02:58:10.217 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-06-03T02:58:10.217 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-06-03T02:58:10.217 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-06-03T02:58:10.217 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-06-03T02:58:10.217 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-06-03T02:58:10.217 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-06-03T02:58:10.217 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty 2024-06-03T02:58:10.217 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/ipwireless 2024-06-03T02:58:10.217 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-06-03T02:58:10.217 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/n_gsm.ko 2024-06-03T02:58:10.217 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/n_hdlc.ko 2024-06-03T02:58:10.217 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/nozomi.ko 2024-06-03T02:58:10.217 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial 2024-06-03T02:58:10.218 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/8250 2024-06-03T02:58:10.218 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-06-03T02:58:10.218 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/arc_uart.ko 2024-06-03T02:58:10.218 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/jsm 2024-06-03T02:58:10.218 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/jsm/jsm.ko 2024-06-03T02:58:10.218 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs 2024-06-03T02:58:10.218 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/core 2024-06-03T02:58:10.218 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/core/ufshcd-core.ko 2024-06-03T02:58:10.218 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/host 2024-06-03T02:58:10.218 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-06-03T02:58:10.218 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio 2024-06-03T02:58:10.218 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio.ko 2024-06-03T02:58:10.218 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_aec.ko 2024-06-03T02:58:10.219 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_cif.ko 2024-06-03T02:58:10.219 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_pci_generic.ko 2024-06-03T02:58:10.219 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_sercos3.ko 2024-06-03T02:58:10.219 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb 2024-06-03T02:58:10.219 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class 2024-06-03T02:58:10.219 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/cdc-acm.ko 2024-06-03T02:58:10.219 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/cdc-wdm.ko 2024-06-03T02:58:10.219 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/usblp.ko 2024-06-03T02:58:10.219 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/usbtmc.ko 2024-06-03T02:58:10.219 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/host 2024-06-03T02:58:10.219 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/host/sl811-hcd.ko 2024-06-03T02:58:10.219 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image 2024-06-03T02:58:10.219 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image/mdc800.ko 2024-06-03T02:58:10.220 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image/microtek.ko 2024-06-03T02:58:10.220 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc 2024-06-03T02:58:10.220 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/adutux.ko 2024-06-03T02:58:10.220 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/appledisplay.ko 2024-06-03T02:58:10.220 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/emi26.ko 2024-06-03T02:58:10.220 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/emi62.ko 2024-06-03T02:58:10.220 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/ezusb.ko 2024-06-03T02:58:10.220 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/idmouse.ko 2024-06-03T02:58:10.220 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/iowarrior.ko 2024-06-03T02:58:10.220 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/isight_firmware.ko 2024-06-03T02:58:10.220 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/ldusb.ko 2024-06-03T02:58:10.220 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/legousbtower.ko 2024-06-03T02:58:10.220 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/sisusbvga 2024-06-03T02:58:10.220 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-06-03T02:58:10.221 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/trancevibrator.ko 2024-06-03T02:58:10.221 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usb3503.ko 2024-06-03T02:58:10.221 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usblcd.ko 2024-06-03T02:58:10.221 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usbsevseg.ko 2024-06-03T02:58:10.221 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/uss720.ko 2024-06-03T02:58:10.221 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/yurex.ko 2024-06-03T02:58:10.221 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/phy 2024-06-03T02:58:10.221 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/phy/phy-generic.ko 2024-06-03T02:58:10.221 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial 2024-06-03T02:58:10.221 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/aircable.ko 2024-06-03T02:58:10.221 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ark3116.ko 2024-06-03T02:58:10.221 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/belkin_sa.ko 2024-06-03T02:58:10.221 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ch341.ko 2024-06-03T02:58:10.222 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cp210x.ko 2024-06-03T02:58:10.222 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cyberjack.ko 2024-06-03T02:58:10.222 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cypress_m8.ko 2024-06-03T02:58:10.222 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/digi_acceleport.ko 2024-06-03T02:58:10.222 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/empeg.ko 2024-06-03T02:58:10.222 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ftdi_sio.ko 2024-06-03T02:58:10.222 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/garmin_gps.ko 2024-06-03T02:58:10.222 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/io_edgeport.ko 2024-06-03T02:58:10.222 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/io_ti.ko 2024-06-03T02:58:10.222 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ipaq.ko 2024-06-03T02:58:10.222 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ipw.ko 2024-06-03T02:58:10.222 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ir-usb.ko 2024-06-03T02:58:10.222 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-06-03T02:58:10.222 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/keyspan.ko 2024-06-03T02:58:10.223 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/keyspan_pda.ko 2024-06-03T02:58:10.223 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/kl5kusb105.ko 2024-06-03T02:58:10.223 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/kobil_sct.ko 2024-06-03T02:58:10.223 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mct_u232.ko 2024-06-03T02:58:10.223 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mos7720.ko 2024-06-03T02:58:10.223 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mos7840.ko 2024-06-03T02:58:10.223 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/navman.ko 2024-06-03T02:58:10.223 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/omninet.ko 2024-06-03T02:58:10.223 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/opticon.ko 2024-06-03T02:58:10.223 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/option.ko 2024-06-03T02:58:10.223 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/oti6858.ko 2024-06-03T02:58:10.223 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/pl2303.ko 2024-06-03T02:58:10.223 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/qcaux.ko 2024-06-03T02:58:10.224 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/qcserial.ko 2024-06-03T02:58:10.224 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/quatech2.ko 2024-06-03T02:58:10.224 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/safe_serial.ko 2024-06-03T02:58:10.224 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/sierra.ko 2024-06-03T02:58:10.224 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/spcp8x5.ko 2024-06-03T02:58:10.224 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ssu100.ko 2024-06-03T02:58:10.224 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/symbolserial.ko 2024-06-03T02:58:10.224 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-06-03T02:58:10.224 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-06-03T02:58:10.224 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb_debug.ko 2024-06-03T02:58:10.224 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb_wwan.ko 2024-06-03T02:58:10.224 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/visor.ko 2024-06-03T02:58:10.224 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/whiteheat.ko 2024-06-03T02:58:10.225 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/xsens_mt.ko 2024-06-03T02:58:10.225 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage 2024-06-03T02:58:10.225 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/uas.ko 2024-06-03T02:58:10.225 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-alauda.ko 2024-06-03T02:58:10.225 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-cypress.ko 2024-06-03T02:58:10.225 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-datafab.ko 2024-06-03T02:58:10.225 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-06-03T02:58:10.225 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-freecom.ko 2024-06-03T02:58:10.225 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-isd200.ko 2024-06-03T02:58:10.225 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-06-03T02:58:10.225 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-karma.ko 2024-06-03T02:58:10.225 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-onetouch.ko 2024-06-03T02:58:10.225 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-realtek.ko 2024-06-03T02:58:10.226 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-sddr09.ko 2024-06-03T02:58:10.226 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-sddr55.ko 2024-06-03T02:58:10.226 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-usbat.ko 2024-06-03T02:58:10.226 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/usb-storage.ko 2024-06-03T02:58:10.226 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip 2024-06-03T02:58:10.226 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/usbip-core.ko 2024-06-03T02:58:10.226 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/usbip-host.ko 2024-06-03T02:58:10.226 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-06-03T02:58:10.226 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio 2024-06-03T02:58:10.226 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci 2024-06-03T02:58:10.226 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-06-03T02:58:10.226 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci/vfio-pci.ko 2024-06-03T02:58:10.227 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/vfio.ko 2024-06-03T02:58:10.227 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-06-03T02:58:10.227 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost 2024-06-03T02:58:10.227 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost.ko 2024-06-03T02:58:10.227 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_iotlb.ko 2024-06-03T02:58:10.227 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_net.ko 2024-06-03T02:58:10.227 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_scsi.ko 2024-06-03T02:58:10.227 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video 2024-06-03T02:58:10.227 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight 2024-06-03T02:58:10.227 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/apple_bl.ko 2024-06-03T02:58:10.227 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/lcd.ko 2024-06-03T02:58:10.227 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/platform_lcd.ko 2024-06-03T02:58:10.227 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio 2024-06-03T02:58:10.228 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio.ko 2024-06-03T02:58:10.228 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_balloon.ko 2024-06-03T02:58:10.228 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_mmio.ko 2024-06-03T02:58:10.228 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_pci.ko 2024-06-03T02:58:10.228 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-06-03T02:58:10.228 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-06-03T02:58:10.228 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_ring.ko 2024-06-03T02:58:10.228 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1 2024-06-03T02:58:10.228 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters 2024-06-03T02:58:10.228 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters/ds2482.ko 2024-06-03T02:58:10.228 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters/ds2490.ko 2024-06-03T02:58:10.228 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves 2024-06-03T02:58:10.228 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-06-03T02:58:10.228 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-06-03T02:58:10.229 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-06-03T02:58:10.229 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-06-03T02:58:10.229 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-06-03T02:58:10.229 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-06-03T02:58:10.229 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-06-03T02:58:10.229 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-06-03T02:58:10.229 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-06-03T02:58:10.229 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_smem.ko 2024-06-03T02:58:10.229 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_therm.ko 2024-06-03T02:58:10.229 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/wire.ko 2024-06-03T02:58:10.229 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog 2024-06-03T02:58:10.229 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/alim1535_wdt.ko 2024-06-03T02:58:10.229 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/alim7101_wdt.ko 2024-06-03T02:58:10.229 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/f71808e_wdt.ko 2024-06-03T02:58:10.230 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/hpwdt.ko 2024-06-03T02:58:10.230 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/i6300esb.ko 2024-06-03T02:58:10.230 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-06-03T02:58:10.230 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/iTCO_wdt.ko 2024-06-03T02:58:10.230 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ib700wdt.ko 2024-06-03T02:58:10.230 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ibmasr.ko 2024-06-03T02:58:10.230 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-06-03T02:58:10.230 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/it8712f_wdt.ko 2024-06-03T02:58:10.230 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/it87_wdt.ko 2024-06-03T02:58:10.230 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/machzwd.ko 2024-06-03T02:58:10.230 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/nv_tco.ko 2024-06-03T02:58:10.230 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/pcwd_pci.ko 2024-06-03T02:58:10.230 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/pcwd_usb.ko 2024-06-03T02:58:10.230 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-06-03T02:58:10.231 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sch311x_wdt.ko 2024-06-03T02:58:10.231 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/softdog.ko 2024-06-03T02:58:10.231 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sp5100_tco.ko 2024-06-03T02:58:10.231 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/via_wdt.ko 2024-06-03T02:58:10.231 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-06-03T02:58:10.231 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83877f_wdt.ko 2024-06-03T02:58:10.231 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83977f_wdt.ko 2024-06-03T02:58:10.231 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/wdt_pci.ko 2024-06-03T02:58:10.231 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/xen_wdt.ko 2024-06-03T02:58:10.231 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen 2024-06-03T02:58:10.231 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-acpi-processor.ko 2024-06-03T02:58:10.231 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-evtchn.ko 2024-06-03T02:58:10.231 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-gntalloc.ko 2024-06-03T02:58:10.232 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-gntdev.ko 2024-06-03T02:58:10.232 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-pciback 2024-06-03T02:58:10.232 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-06-03T02:58:10.232 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-privcmd.ko 2024-06-03T02:58:10.232 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xenfs 2024-06-03T02:58:10.232 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xenfs/xenfs.ko 2024-06-03T02:58:10.232 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs 2024-06-03T02:58:10.232 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/9p 2024-06-03T02:58:10.232 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/9p/9p.ko 2024-06-03T02:58:10.232 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/affs 2024-06-03T02:58:10.232 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/affs/affs.ko 2024-06-03T02:58:10.232 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/befs 2024-06-03T02:58:10.232 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/befs/befs.ko 2024-06-03T02:58:10.232 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/binfmt_misc.ko 2024-06-03T02:58:10.233 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/btrfs 2024-06-03T02:58:10.233 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/btrfs/btrfs.ko 2024-06-03T02:58:10.233 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cachefiles 2024-06-03T02:58:10.233 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cachefiles/cachefiles.ko 2024-06-03T02:58:10.233 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ceph 2024-06-03T02:58:10.233 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ceph/ceph.ko 2024-06-03T02:58:10.233 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/coda 2024-06-03T02:58:10.233 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/coda/coda.ko 2024-06-03T02:58:10.233 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cramfs 2024-06-03T02:58:10.233 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cramfs/cramfs.ko 2024-06-03T02:58:10.233 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/dlm 2024-06-03T02:58:10.233 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/dlm/dlm.ko 2024-06-03T02:58:10.233 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ecryptfs 2024-06-03T02:58:10.233 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ecryptfs/ecryptfs.ko 2024-06-03T02:58:10.234 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/f2fs 2024-06-03T02:58:10.234 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/f2fs/f2fs.ko 2024-06-03T02:58:10.234 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat 2024-06-03T02:58:10.234 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/fat.ko 2024-06-03T02:58:10.234 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/msdos.ko 2024-06-03T02:58:10.234 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/vfat.ko 2024-06-03T02:58:10.234 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse 2024-06-03T02:58:10.234 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse/cuse.ko 2024-06-03T02:58:10.234 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse/fuse.ko 2024-06-03T02:58:10.234 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/gfs2 2024-06-03T02:58:10.234 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/gfs2/gfs2.ko 2024-06-03T02:58:10.234 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfs 2024-06-03T02:58:10.234 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfs/hfs.ko 2024-06-03T02:58:10.235 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfsplus 2024-06-03T02:58:10.235 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfsplus/hfsplus.ko 2024-06-03T02:58:10.235 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/isofs 2024-06-03T02:58:10.235 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/isofs/isofs.ko 2024-06-03T02:58:10.235 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/jfs 2024-06-03T02:58:10.235 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/jfs/jfs.ko 2024-06-03T02:58:10.235 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/lockd 2024-06-03T02:58:10.235 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/lockd/lockd.ko 2024-06-03T02:58:10.235 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/minix 2024-06-03T02:58:10.235 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/minix/minix.ko 2024-06-03T02:58:10.235 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/netfs 2024-06-03T02:58:10.235 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/netfs/netfs.ko 2024-06-03T02:58:10.235 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs 2024-06-03T02:58:10.236 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/blocklayout 2024-06-03T02:58:10.236 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-06-03T02:58:10.236 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/filelayout 2024-06-03T02:58:10.236 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-06-03T02:58:10.236 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/flexfilelayout 2024-06-03T02:58:10.236 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-06-03T02:58:10.236 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfs.ko 2024-06-03T02:58:10.236 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfsv3.ko 2024-06-03T02:58:10.236 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfsv4.ko 2024-06-03T02:58:10.236 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common 2024-06-03T02:58:10.236 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common/grace.ko 2024-06-03T02:58:10.236 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common/nfs_acl.ko 2024-06-03T02:58:10.236 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfsd 2024-06-03T02:58:10.236 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfsd/nfsd.ko 2024-06-03T02:58:10.237 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nilfs2 2024-06-03T02:58:10.237 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nilfs2/nilfs2.ko 2024-06-03T02:58:10.237 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls 2024-06-03T02:58:10.237 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-celtic.ko 2024-06-03T02:58:10.237 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-centeuro.ko 2024-06-03T02:58:10.237 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-croatian.ko 2024-06-03T02:58:10.237 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-cyrillic.ko 2024-06-03T02:58:10.237 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-gaelic.ko 2024-06-03T02:58:10.237 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-greek.ko 2024-06-03T02:58:10.237 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-iceland.ko 2024-06-03T02:58:10.237 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-inuit.ko 2024-06-03T02:58:10.237 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-roman.ko 2024-06-03T02:58:10.237 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-romanian.ko 2024-06-03T02:58:10.237 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-turkish.ko 2024-06-03T02:58:10.238 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1250.ko 2024-06-03T02:58:10.238 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1251.ko 2024-06-03T02:58:10.238 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1255.ko 2024-06-03T02:58:10.238 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp737.ko 2024-06-03T02:58:10.238 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp775.ko 2024-06-03T02:58:10.238 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp850.ko 2024-06-03T02:58:10.238 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp852.ko 2024-06-03T02:58:10.238 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp855.ko 2024-06-03T02:58:10.238 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp857.ko 2024-06-03T02:58:10.238 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp860.ko 2024-06-03T02:58:10.238 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp861.ko 2024-06-03T02:58:10.238 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp862.ko 2024-06-03T02:58:10.238 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp863.ko 2024-06-03T02:58:10.239 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp864.ko 2024-06-03T02:58:10.239 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp865.ko 2024-06-03T02:58:10.239 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp866.ko 2024-06-03T02:58:10.239 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp869.ko 2024-06-03T02:58:10.239 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp874.ko 2024-06-03T02:58:10.239 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp932.ko 2024-06-03T02:58:10.239 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp936.ko 2024-06-03T02:58:10.239 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp949.ko 2024-06-03T02:58:10.239 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp950.ko 2024-06-03T02:58:10.239 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_euc-jp.ko 2024-06-03T02:58:10.239 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-1.ko 2024-06-03T02:58:10.239 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-13.ko 2024-06-03T02:58:10.239 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-14.ko 2024-06-03T02:58:10.240 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-15.ko 2024-06-03T02:58:10.240 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-2.ko 2024-06-03T02:58:10.240 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-3.ko 2024-06-03T02:58:10.240 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-4.ko 2024-06-03T02:58:10.240 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-5.ko 2024-06-03T02:58:10.240 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-6.ko 2024-06-03T02:58:10.240 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-7.ko 2024-06-03T02:58:10.240 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-9.ko 2024-06-03T02:58:10.240 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-r.ko 2024-06-03T02:58:10.240 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-ru.ko 2024-06-03T02:58:10.240 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-u.ko 2024-06-03T02:58:10.240 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_ucs2_utils.ko 2024-06-03T02:58:10.240 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_utf8.ko 2024-06-03T02:58:10.240 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2 2024-06-03T02:58:10.240 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/cluster 2024-06-03T02:58:10.241 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-06-03T02:58:10.241 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlm 2024-06-03T02:58:10.241 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-06-03T02:58:10.241 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlmfs 2024-06-03T02:58:10.241 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-06-03T02:58:10.241 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2.ko 2024-06-03T02:58:10.241 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-06-03T02:58:10.241 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-06-03T02:58:10.241 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-06-03T02:58:10.241 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/overlayfs 2024-06-03T02:58:10.241 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/overlayfs/overlay.ko 2024-06-03T02:58:10.241 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/pstore 2024-06-03T02:58:10.241 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/pstore/ramoops.ko 2024-06-03T02:58:10.242 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/reiserfs 2024-06-03T02:58:10.242 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/reiserfs/reiserfs.ko 2024-06-03T02:58:10.242 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/romfs 2024-06-03T02:58:10.242 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/romfs/romfs.ko 2024-06-03T02:58:10.242 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb 2024-06-03T02:58:10.242 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/client 2024-06-03T02:58:10.242 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/client/cifs.ko 2024-06-03T02:58:10.242 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common 2024-06-03T02:58:10.242 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common/cifs_arc4.ko 2024-06-03T02:58:10.242 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common/cifs_md4.ko 2024-06-03T02:58:10.242 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/squashfs 2024-06-03T02:58:10.242 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/squashfs/squashfs.ko 2024-06-03T02:58:10.243 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/sysv 2024-06-03T02:58:10.243 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/sysv/sysv.ko 2024-06-03T02:58:10.243 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/udf 2024-06-03T02:58:10.243 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/udf/udf.ko 2024-06-03T02:58:10.243 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ufs 2024-06-03T02:58:10.243 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ufs/ufs.ko 2024-06-03T02:58:10.243 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/xfs 2024-06-03T02:58:10.243 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/xfs/xfs.ko 2024-06-03T02:58:10.243 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel 2024-06-03T02:58:10.243 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/rcu 2024-06-03T02:58:10.243 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/rcu/rcutorture.ko 2024-06-03T02:58:10.243 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/torture.ko 2024-06-03T02:58:10.243 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/trace 2024-06-03T02:58:10.243 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-06-03T02:58:10.244 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib 2024-06-03T02:58:10.244 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/asn1_encoder.ko 2024-06-03T02:58:10.244 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crc-itu-t.ko 2024-06-03T02:58:10.244 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crc8.ko 2024-06-03T02:58:10.244 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto 2024-06-03T02:58:10.244 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto/libarc4.ko 2024-06-03T02:58:10.244 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto/libdes.ko 2024-06-03T02:58:10.244 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/libcrc32c.ko 2024-06-03T02:58:10.244 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lru_cache.ko 2024-06-03T02:58:10.244 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4 2024-06-03T02:58:10.244 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4/lz4_compress.ko 2024-06-03T02:58:10.244 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4/lz4hc_compress.ko 2024-06-03T02:58:10.244 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/math 2024-06-03T02:58:10.245 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/math/cordic.ko 2024-06-03T02:58:10.245 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/raid6 2024-06-03T02:58:10.245 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/raid6/raid6_pq.ko 2024-06-03T02:58:10.245 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/reed_solomon 2024-06-03T02:58:10.245 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/reed_solomon/reed_solomon.ko 2024-06-03T02:58:10.245 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_bm.ko 2024-06-03T02:58:10.245 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_fsm.ko 2024-06-03T02:58:10.245 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_kmp.ko 2024-06-03T02:58:10.245 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/zstd 2024-06-03T02:58:10.245 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/zstd/zstd_compress.ko 2024-06-03T02:58:10.245 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/mm 2024-06-03T02:58:10.245 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/mm/hwpoison-inject.ko 2024-06-03T02:58:10.245 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net 2024-06-03T02:58:10.245 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan 2024-06-03T02:58:10.246 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/6lowpan.ko 2024-06-03T02:58:10.246 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_dest.ko 2024-06-03T02:58:10.246 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_fragment.ko 2024-06-03T02:58:10.246 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_hop.ko 2024-06-03T02:58:10.246 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_ipv6.ko 2024-06-03T02:58:10.246 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_mobility.ko 2024-06-03T02:58:10.246 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_routing.ko 2024-06-03T02:58:10.246 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_udp.ko 2024-06-03T02:58:10.246 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802 2024-06-03T02:58:10.246 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/garp.ko 2024-06-03T02:58:10.246 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/mrp.ko 2024-06-03T02:58:10.246 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/p8022.ko 2024-06-03T02:58:10.246 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/psnap.ko 2024-06-03T02:58:10.246 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/stp.ko 2024-06-03T02:58:10.247 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/8021q 2024-06-03T02:58:10.247 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/8021q/8021q.ko 2024-06-03T02:58:10.247 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p 2024-06-03T02:58:10.247 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet.ko 2024-06-03T02:58:10.247 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_fd.ko 2024-06-03T02:58:10.247 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_rdma.ko 2024-06-03T02:58:10.247 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_virtio.ko 2024-06-03T02:58:10.247 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/batman-adv 2024-06-03T02:58:10.247 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/batman-adv/batman-adv.ko 2024-06-03T02:58:10.247 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge 2024-06-03T02:58:10.247 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/br_netfilter.ko 2024-06-03T02:58:10.247 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/bridge.ko 2024-06-03T02:58:10.247 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter 2024-06-03T02:58:10.247 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-06-03T02:58:10.248 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_among.ko 2024-06-03T02:58:10.248 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_arp.ko 2024-06-03T02:58:10.248 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-06-03T02:58:10.248 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-06-03T02:58:10.248 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_ip.ko 2024-06-03T02:58:10.248 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-06-03T02:58:10.248 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_limit.ko 2024-06-03T02:58:10.248 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_log.ko 2024-06-03T02:58:10.248 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_mark.ko 2024-06-03T02:58:10.248 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-06-03T02:58:10.248 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-06-03T02:58:10.248 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-06-03T02:58:10.248 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-06-03T02:58:10.248 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_snat.ko 2024-06-03T02:58:10.249 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_stp.ko 2024-06-03T02:58:10.249 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-06-03T02:58:10.249 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-06-03T02:58:10.249 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-06-03T02:58:10.249 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-06-03T02:58:10.249 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtables.ko 2024-06-03T02:58:10.249 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-06-03T02:58:10.249 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-06-03T02:58:10.249 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-06-03T02:58:10.249 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ceph 2024-06-03T02:58:10.249 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ceph/libceph.ko 2024-06-03T02:58:10.249 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/core 2024-06-03T02:58:10.249 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/core/failover.ko 2024-06-03T02:58:10.249 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/core/pktgen.ko 2024-06-03T02:58:10.250 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp 2024-06-03T02:58:10.250 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp.ko 2024-06-03T02:58:10.250 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_diag.ko 2024-06-03T02:58:10.250 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_ipv4.ko 2024-06-03T02:58:10.250 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_ipv6.ko 2024-06-03T02:58:10.250 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dns_resolver 2024-06-03T02:58:10.250 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dns_resolver/dns_resolver.ko 2024-06-03T02:58:10.250 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa 2024-06-03T02:58:10.250 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/dsa_core.ko 2024-06-03T02:58:10.250 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_brcm.ko 2024-06-03T02:58:10.250 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_dsa.ko 2024-06-03T02:58:10.250 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_none.ko 2024-06-03T02:58:10.250 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_trailer.ko 2024-06-03T02:58:10.251 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154 2024-06-03T02:58:10.251 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/6lowpan 2024-06-03T02:58:10.251 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-06-03T02:58:10.251 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/ieee802154.ko 2024-06-03T02:58:10.251 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/ieee802154_socket.ko 2024-06-03T02:58:10.251 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4 2024-06-03T02:58:10.251 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ah4.ko 2024-06-03T02:58:10.251 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/esp4.ko 2024-06-03T02:58:10.251 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/fou.ko 2024-06-03T02:58:10.251 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/gre.ko 2024-06-03T02:58:10.251 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/inet_diag.ko 2024-06-03T02:58:10.251 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_gre.ko 2024-06-03T02:58:10.251 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_tunnel.ko 2024-06-03T02:58:10.251 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_vti.ko 2024-06-03T02:58:10.252 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ipcomp.ko 2024-06-03T02:58:10.252 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ipip.ko 2024-06-03T02:58:10.252 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter 2024-06-03T02:58:10.252 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arp_tables.ko 2024-06-03T02:58:10.252 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-06-03T02:58:10.252 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-06-03T02:58:10.252 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-06-03T02:58:10.252 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-06-03T02:58:10.252 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-06-03T02:58:10.252 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-06-03T02:58:10.252 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-06-03T02:58:10.252 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-06-03T02:58:10.252 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-06-03T02:58:10.252 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_security.ko 2024-06-03T02:58:10.253 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-06-03T02:58:10.253 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-06-03T02:58:10.253 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-06-03T02:58:10.253 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-06-03T02:58:10.253 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-06-03T02:58:10.253 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-06-03T02:58:10.253 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-06-03T02:58:10.253 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-06-03T02:58:10.253 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-06-03T02:58:10.253 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-06-03T02:58:10.253 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_bic.ko 2024-06-03T02:58:10.253 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_dctcp.ko 2024-06-03T02:58:10.253 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_diag.ko 2024-06-03T02:58:10.253 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_highspeed.ko 2024-06-03T02:58:10.254 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_htcp.ko 2024-06-03T02:58:10.254 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_hybla.ko 2024-06-03T02:58:10.254 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_illinois.ko 2024-06-03T02:58:10.254 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_lp.ko 2024-06-03T02:58:10.254 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_scalable.ko 2024-06-03T02:58:10.254 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_vegas.ko 2024-06-03T02:58:10.254 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_veno.ko 2024-06-03T02:58:10.254 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_westwood.ko 2024-06-03T02:58:10.254 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_yeah.ko 2024-06-03T02:58:10.254 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tunnel4.ko 2024-06-03T02:58:10.254 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/udp_diag.ko 2024-06-03T02:58:10.254 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/udp_tunnel.ko 2024-06-03T02:58:10.255 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/xfrm4_tunnel.ko 2024-06-03T02:58:10.255 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6 2024-06-03T02:58:10.255 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ah6.ko 2024-06-03T02:58:10.255 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/esp6.ko 2024-06-03T02:58:10.255 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/fou6.ko 2024-06-03T02:58:10.255 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_tunnel.ko 2024-06-03T02:58:10.255 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-06-03T02:58:10.255 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_vti.ko 2024-06-03T02:58:10.255 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ipcomp6.ko 2024-06-03T02:58:10.255 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter 2024-06-03T02:58:10.255 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-06-03T02:58:10.255 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-06-03T02:58:10.255 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-06-03T02:58:10.255 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-06-03T02:58:10.256 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-06-03T02:58:10.256 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-06-03T02:58:10.256 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-06-03T02:58:10.256 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-06-03T02:58:10.256 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-06-03T02:58:10.256 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-06-03T02:58:10.256 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-06-03T02:58:10.256 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-06-03T02:58:10.256 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-06-03T02:58:10.256 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-06-03T02:58:10.256 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-06-03T02:58:10.256 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-06-03T02:58:10.256 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-06-03T02:58:10.256 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-06-03T02:58:10.257 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-06-03T02:58:10.257 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-06-03T02:58:10.257 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-06-03T02:58:10.257 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-06-03T02:58:10.257 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-06-03T02:58:10.257 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-06-03T02:58:10.257 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/sit.ko 2024-06-03T02:58:10.257 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/tunnel6.ko 2024-06-03T02:58:10.257 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/xfrm6_tunnel.ko 2024-06-03T02:58:10.257 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/key 2024-06-03T02:58:10.257 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/key/af_key.ko 2024-06-03T02:58:10.257 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp 2024-06-03T02:58:10.257 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_core.ko 2024-06-03T02:58:10.257 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_debugfs.ko 2024-06-03T02:58:10.258 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_eth.ko 2024-06-03T02:58:10.258 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ip.ko 2024-06-03T02:58:10.258 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ip6.ko 2024-06-03T02:58:10.258 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_netlink.ko 2024-06-03T02:58:10.258 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ppp.ko 2024-06-03T02:58:10.258 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/llc 2024-06-03T02:58:10.258 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/llc/llc.ko 2024-06-03T02:58:10.258 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mac802154 2024-06-03T02:58:10.258 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mac802154/mac802154.ko 2024-06-03T02:58:10.258 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mpls 2024-06-03T02:58:10.258 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mpls/mpls_gso.ko 2024-06-03T02:58:10.258 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter 2024-06-03T02:58:10.258 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset 2024-06-03T02:58:10.258 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set.ko 2024-06-03T02:58:10.259 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-06-03T02:58:10.259 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-06-03T02:58:10.259 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-06-03T02:58:10.259 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-06-03T02:58:10.259 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-06-03T02:58:10.259 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-06-03T02:58:10.259 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-06-03T02:58:10.259 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-06-03T02:58:10.259 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-06-03T02:58:10.259 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-06-03T02:58:10.259 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-06-03T02:58:10.259 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-06-03T02:58:10.259 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-06-03T02:58:10.259 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-06-03T02:58:10.260 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-06-03T02:58:10.260 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs 2024-06-03T02:58:10.260 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs.ko 2024-06-03T02:58:10.260 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-06-03T02:58:10.260 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-06-03T02:58:10.260 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-06-03T02:58:10.260 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-06-03T02:58:10.260 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-06-03T02:58:10.260 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-06-03T02:58:10.260 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-06-03T02:58:10.260 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-06-03T02:58:10.260 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-06-03T02:58:10.260 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-06-03T02:58:10.260 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-06-03T02:58:10.261 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-06-03T02:58:10.261 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-06-03T02:58:10.261 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conncount.ko 2024-06-03T02:58:10.261 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack.ko 2024-06-03T02:58:10.261 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-06-03T02:58:10.261 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-06-03T02:58:10.261 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-06-03T02:58:10.261 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_h323.ko 2024-06-03T02:58:10.261 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_irc.ko 2024-06-03T02:58:10.261 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-06-03T02:58:10.261 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-06-03T02:58:10.261 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-06-03T02:58:10.261 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_sane.ko 2024-06-03T02:58:10.261 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_sip.ko 2024-06-03T02:58:10.262 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-06-03T02:58:10.262 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-06-03T02:58:10.262 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_dup_netdev.ko 2024-06-03T02:58:10.262 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_flow_table.ko 2024-06-03T02:58:10.262 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_flow_table_inet.ko 2024-06-03T02:58:10.262 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_log_syslog.ko 2024-06-03T02:58:10.262 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat.ko 2024-06-03T02:58:10.262 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_amanda.ko 2024-06-03T02:58:10.262 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_ftp.ko 2024-06-03T02:58:10.262 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_irc.ko 2024-06-03T02:58:10.262 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_sip.ko 2024-06-03T02:58:10.262 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_tftp.ko 2024-06-03T02:58:10.262 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_synproxy_core.ko 2024-06-03T02:58:10.263 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_tables.ko 2024-06-03T02:58:10.263 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink.ko 2024-06-03T02:58:10.263 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_acct.ko 2024-06-03T02:58:10.263 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_log.ko 2024-06-03T02:58:10.263 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_osf.ko 2024-06-03T02:58:10.263 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_queue.ko 2024-06-03T02:58:10.263 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_chain_nat.ko 2024-06-03T02:58:10.263 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_compat.ko 2024-06-03T02:58:10.263 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_ct.ko 2024-06-03T02:58:10.263 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_dup_netdev.ko 2024-06-03T02:58:10.263 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_fib.ko 2024-06-03T02:58:10.263 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_fwd_netdev.ko 2024-06-03T02:58:10.263 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_hash.ko 2024-06-03T02:58:10.263 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_limit.ko 2024-06-03T02:58:10.264 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_log.ko 2024-06-03T02:58:10.264 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_masq.ko 2024-06-03T02:58:10.264 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_nat.ko 2024-06-03T02:58:10.264 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_numgen.ko 2024-06-03T02:58:10.264 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_queue.ko 2024-06-03T02:58:10.264 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_quota.ko 2024-06-03T02:58:10.264 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_redir.ko 2024-06-03T02:58:10.264 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_reject.ko 2024-06-03T02:58:10.264 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_reject_inet.ko 2024-06-03T02:58:10.264 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_socket.ko 2024-06-03T02:58:10.264 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_tunnel.ko 2024-06-03T02:58:10.264 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_AUDIT.ko 2024-06-03T02:58:10.264 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CHECKSUM.ko 2024-06-03T02:58:10.265 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CLASSIFY.ko 2024-06-03T02:58:10.265 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-06-03T02:58:10.265 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CT.ko 2024-06-03T02:58:10.265 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_DSCP.ko 2024-06-03T02:58:10.265 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_HL.ko 2024-06-03T02:58:10.265 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_HMARK.ko 2024-06-03T02:58:10.265 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_IDLETIMER.ko 2024-06-03T02:58:10.265 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_LED.ko 2024-06-03T02:58:10.265 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_LOG.ko 2024-06-03T02:58:10.265 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_MASQUERADE.ko 2024-06-03T02:58:10.265 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NETMAP.ko 2024-06-03T02:58:10.265 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NFLOG.ko 2024-06-03T02:58:10.265 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NFQUEUE.ko 2024-06-03T02:58:10.265 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_RATEEST.ko 2024-06-03T02:58:10.266 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_REDIRECT.ko 2024-06-03T02:58:10.266 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_SECMARK.ko 2024-06-03T02:58:10.266 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TCPMSS.ko 2024-06-03T02:58:10.266 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-06-03T02:58:10.266 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TEE.ko 2024-06-03T02:58:10.266 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TPROXY.ko 2024-06-03T02:58:10.266 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TRACE.ko 2024-06-03T02:58:10.266 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_addrtype.ko 2024-06-03T02:58:10.266 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_bpf.ko 2024-06-03T02:58:10.266 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cgroup.ko 2024-06-03T02:58:10.266 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cluster.ko 2024-06-03T02:58:10.266 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_comment.ko 2024-06-03T02:58:10.266 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connbytes.ko 2024-06-03T02:58:10.267 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connlabel.ko 2024-06-03T02:58:10.267 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connlimit.ko 2024-06-03T02:58:10.267 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connmark.ko 2024-06-03T02:58:10.267 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_conntrack.ko 2024-06-03T02:58:10.267 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cpu.ko 2024-06-03T02:58:10.267 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_dccp.ko 2024-06-03T02:58:10.267 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_devgroup.ko 2024-06-03T02:58:10.267 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_dscp.ko 2024-06-03T02:58:10.267 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ecn.ko 2024-06-03T02:58:10.267 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_esp.ko 2024-06-03T02:58:10.267 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_hashlimit.ko 2024-06-03T02:58:10.267 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_helper.ko 2024-06-03T02:58:10.267 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_hl.ko 2024-06-03T02:58:10.268 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ipcomp.ko 2024-06-03T02:58:10.268 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_iprange.ko 2024-06-03T02:58:10.268 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ipvs.ko 2024-06-03T02:58:10.268 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_l2tp.ko 2024-06-03T02:58:10.268 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_length.ko 2024-06-03T02:58:10.268 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_limit.ko 2024-06-03T02:58:10.268 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_mac.ko 2024-06-03T02:58:10.268 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_mark.ko 2024-06-03T02:58:10.268 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_multiport.ko 2024-06-03T02:58:10.268 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_nat.ko 2024-06-03T02:58:10.268 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_nfacct.ko 2024-06-03T02:58:10.268 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_osf.ko 2024-06-03T02:58:10.268 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_owner.ko 2024-06-03T02:58:10.269 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_physdev.ko 2024-06-03T02:58:10.269 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_pkttype.ko 2024-06-03T02:58:10.269 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_policy.ko 2024-06-03T02:58:10.269 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_quota.ko 2024-06-03T02:58:10.269 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_rateest.ko 2024-06-03T02:58:10.269 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_realm.ko 2024-06-03T02:58:10.269 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_recent.ko 2024-06-03T02:58:10.269 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_sctp.ko 2024-06-03T02:58:10.269 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_set.ko 2024-06-03T02:58:10.269 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_socket.ko 2024-06-03T02:58:10.269 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_state.ko 2024-06-03T02:58:10.269 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_statistic.ko 2024-06-03T02:58:10.269 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_string.ko 2024-06-03T02:58:10.270 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_tcpmss.ko 2024-06-03T02:58:10.270 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_time.ko 2024-06-03T02:58:10.270 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_u32.ko 2024-06-03T02:58:10.270 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netlink 2024-06-03T02:58:10.270 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netlink/netlink_diag.ko 2024-06-03T02:58:10.270 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/nsh 2024-06-03T02:58:10.270 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/nsh/nsh.ko 2024-06-03T02:58:10.270 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch 2024-06-03T02:58:10.270 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/openvswitch.ko 2024-06-03T02:58:10.270 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-geneve.ko 2024-06-03T02:58:10.270 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-gre.ko 2024-06-03T02:58:10.270 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-vxlan.ko 2024-06-03T02:58:10.270 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/packet 2024-06-03T02:58:10.271 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/packet/af_packet_diag.ko 2024-06-03T02:58:10.271 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds 2024-06-03T02:58:10.271 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds.ko 2024-06-03T02:58:10.271 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds_rdma.ko 2024-06-03T02:58:10.271 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds_tcp.ko 2024-06-03T02:58:10.271 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched 2024-06-03T02:58:10.271 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_csum.ko 2024-06-03T02:58:10.271 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_gact.ko 2024-06-03T02:58:10.271 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_mirred.ko 2024-06-03T02:58:10.271 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_nat.ko 2024-06-03T02:58:10.271 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_pedit.ko 2024-06-03T02:58:10.271 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_police.ko 2024-06-03T02:58:10.271 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_simple.ko 2024-06-03T02:58:10.272 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_skbedit.ko 2024-06-03T02:58:10.272 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_vlan.ko 2024-06-03T02:58:10.272 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_basic.ko 2024-06-03T02:58:10.272 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_bpf.ko 2024-06-03T02:58:10.272 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_flow.ko 2024-06-03T02:58:10.272 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_fw.ko 2024-06-03T02:58:10.272 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_route.ko 2024-06-03T02:58:10.272 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_u32.ko 2024-06-03T02:58:10.272 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_cmp.ko 2024-06-03T02:58:10.272 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_ipset.ko 2024-06-03T02:58:10.272 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_meta.ko 2024-06-03T02:58:10.272 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_nbyte.ko 2024-06-03T02:58:10.272 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_text.ko 2024-06-03T02:58:10.272 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_u32.ko 2024-06-03T02:58:10.273 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_choke.ko 2024-06-03T02:58:10.273 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_codel.ko 2024-06-03T02:58:10.273 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_drr.ko 2024-06-03T02:58:10.273 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_fq.ko 2024-06-03T02:58:10.273 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_gred.ko 2024-06-03T02:58:10.273 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_hfsc.ko 2024-06-03T02:58:10.273 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_hhf.ko 2024-06-03T02:58:10.273 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_htb.ko 2024-06-03T02:58:10.273 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_ingress.ko 2024-06-03T02:58:10.273 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_mqprio.ko 2024-06-03T02:58:10.273 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_mqprio_lib.ko 2024-06-03T02:58:10.273 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_multiq.ko 2024-06-03T02:58:10.273 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_netem.ko 2024-06-03T02:58:10.274 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_pie.ko 2024-06-03T02:58:10.274 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_plug.ko 2024-06-03T02:58:10.274 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_prio.ko 2024-06-03T02:58:10.274 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_qfq.ko 2024-06-03T02:58:10.274 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_red.ko 2024-06-03T02:58:10.274 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_sfb.ko 2024-06-03T02:58:10.274 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_sfq.ko 2024-06-03T02:58:10.274 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_tbf.ko 2024-06-03T02:58:10.274 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_teql.ko 2024-06-03T02:58:10.274 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp 2024-06-03T02:58:10.274 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp/sctp.ko 2024-06-03T02:58:10.274 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp/sctp_diag.ko 2024-06-03T02:58:10.274 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc 2024-06-03T02:58:10.275 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss 2024-06-03T02:58:10.275 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-06-03T02:58:10.275 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-06-03T02:58:10.275 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/sunrpc.ko 2024-06-03T02:58:10.275 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/xprtrdma 2024-06-03T02:58:10.275 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-06-03T02:58:10.275 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc 2024-06-03T02:58:10.275 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc/tipc.ko 2024-06-03T02:58:10.275 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc/tipc_diag.ko 2024-06-03T02:58:10.275 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/unix 2024-06-03T02:58:10.275 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/unix/unix_diag.ko 2024-06-03T02:58:10.275 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock 2024-06-03T02:58:10.275 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-06-03T02:58:10.275 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-06-03T02:58:10.275 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock.ko 2024-06-03T02:58:10.276 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock_diag.ko 2024-06-03T02:58:10.276 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock_loopback.ko 2024-06-03T02:58:10.276 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/xfrm 2024-06-03T02:58:10.276 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/xfrm/xfrm_ipcomp.ko 2024-06-03T02:58:10.276 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security 2024-06-03T02:58:10.276 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys 2024-06-03T02:58:10.276 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/encrypted-keys 2024-06-03T02:58:10.276 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-06-03T02:58:10.276 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/trusted-keys 2024-06-03T02:58:10.276 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/trusted-keys/trusted.ko 2024-06-03T02:58:10.276 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.alias 2024-06-03T02:58:10.276 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.alias.bin 2024-06-03T02:58:10.276 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin 2024-06-03T02:58:10.276 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin.alias.bin 2024-06-03T02:58:10.277 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin.bin 2024-06-03T02:58:10.277 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin.modinfo 2024-06-03T02:58:10.277 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.dep 2024-06-03T02:58:10.277 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.dep.bin 2024-06-03T02:58:10.277 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.devname 2024-06-03T02:58:10.277 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.order 2024-06-03T02:58:10.277 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.softdep 2024-06-03T02:58:10.277 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.symbols 2024-06-03T02:58:10.277 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.symbols.bin 2024-06-03T02:58:10.277 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.9.0-gb9459a26b088/vmlinuz 2024-06-03T02:58:10.281 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-gb9459a26b088 2024-06-03T02:58:10.281 DEBUG:teuthology.orchestra.run.smithi117:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-06-03T02:58:10.304 INFO:teuthology.orchestra.run.smithi117.stdout:postinstall scriptlet (using /bin/sh): 2024-06-03T02:58:10.305 INFO:teuthology.orchestra.run.smithi117.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-06-03T02:58:10.305 INFO:teuthology.orchestra.run.smithi117.stdout: /usr/bin/kernel-install add 6.9.0-gb9459a26b088 /lib/modules/6.9.0-gb9459a26b088/vmlinuz 2024-06-03T02:58:10.305 INFO:teuthology.orchestra.run.smithi117.stdout:fi 2024-06-03T02:58:10.305 INFO:teuthology.orchestra.run.smithi117.stdout:for file in vmlinuz System.map config; do 2024-06-03T02:58:10.305 INFO:teuthology.orchestra.run.smithi117.stdout: if ! cmp --silent "/lib/modules/6.9.0-gb9459a26b088/${file}" "/boot/${file}-6.9.0-gb9459a26b088"; then 2024-06-03T02:58:10.305 INFO:teuthology.orchestra.run.smithi117.stdout: cp "/lib/modules/6.9.0-gb9459a26b088/${file}" "/boot/${file}-6.9.0-gb9459a26b088" 2024-06-03T02:58:10.305 INFO:teuthology.orchestra.run.smithi117.stdout: fi 2024-06-03T02:58:10.305 INFO:teuthology.orchestra.run.smithi117.stdout:done 2024-06-03T02:58:10.305 INFO:teuthology.orchestra.run.smithi117.stdout:if [ -d "/lib/modules/6.9.0-gb9459a26b088/dtb" ] && \ 2024-06-03T02:58:10.305 INFO:teuthology.orchestra.run.smithi117.stdout: ! diff -rq "/lib/modules/6.9.0-gb9459a26b088/dtb" "/boot/dtb-6.9.0-gb9459a26b088" >/dev/null 2>&1; then 2024-06-03T02:58:10.305 INFO:teuthology.orchestra.run.smithi117.stdout: rm -rf "/boot/dtb-6.9.0-gb9459a26b088" 2024-06-03T02:58:10.305 INFO:teuthology.orchestra.run.smithi117.stdout: cp -r "/lib/modules/6.9.0-gb9459a26b088/dtb" "/boot/dtb-6.9.0-gb9459a26b088" 2024-06-03T02:58:10.305 INFO:teuthology.orchestra.run.smithi117.stdout:fi 2024-06-03T02:58:10.306 INFO:teuthology.orchestra.run.smithi117.stdout:if [ ! -e "/lib/modules/6.9.0-gb9459a26b088/modules.dep" ]; then 2024-06-03T02:58:10.306 INFO:teuthology.orchestra.run.smithi117.stdout: /usr/sbin/depmod 6.9.0-gb9459a26b088 2024-06-03T02:58:10.306 INFO:teuthology.orchestra.run.smithi117.stdout:fi 2024-06-03T02:58:10.306 INFO:teuthology.orchestra.run.smithi117.stdout:preuninstall scriptlet (using /bin/sh): 2024-06-03T02:58:10.306 INFO:teuthology.orchestra.run.smithi117.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-06-03T02:58:10.306 INFO:teuthology.orchestra.run.smithi117.stdout:kernel-install remove 6.9.0-gb9459a26b088 2024-06-03T02:58:10.306 INFO:teuthology.orchestra.run.smithi117.stdout:fi 2024-06-03T02:58:10.307 DEBUG:teuthology.orchestra.run.smithi117:> sudo rpm -qi grub2-tools 2024-06-03T02:58:10.384 INFO:teuthology.orchestra.run.smithi117.stdout:Name : grub2-tools 2024-06-03T02:58:10.385 INFO:teuthology.orchestra.run.smithi117.stdout:Epoch : 1 2024-06-03T02:58:10.385 INFO:teuthology.orchestra.run.smithi117.stdout:Version : 2.06 2024-06-03T02:58:10.385 INFO:teuthology.orchestra.run.smithi117.stdout:Release : 68.el9 2024-06-03T02:58:10.385 INFO:teuthology.orchestra.run.smithi117.stdout:Architecture: x86_64 2024-06-03T02:58:10.385 INFO:teuthology.orchestra.run.smithi117.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-06-03T02:58:10.385 INFO:teuthology.orchestra.run.smithi117.stdout:Group : Unspecified 2024-06-03T02:58:10.385 INFO:teuthology.orchestra.run.smithi117.stdout:Size : 8264254 2024-06-03T02:58:10.385 INFO:teuthology.orchestra.run.smithi117.stdout:License : GPLv3+ 2024-06-03T02:58:10.385 INFO:teuthology.orchestra.run.smithi117.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-06-03T02:58:10.385 INFO:teuthology.orchestra.run.smithi117.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-06-03T02:58:10.385 INFO:teuthology.orchestra.run.smithi117.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-06-03T02:58:10.385 INFO:teuthology.orchestra.run.smithi117.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-06-03T02:58:10.386 INFO:teuthology.orchestra.run.smithi117.stdout:Packager : builder@centos.org 2024-06-03T02:58:10.386 INFO:teuthology.orchestra.run.smithi117.stdout:Vendor : CentOS 2024-06-03T02:58:10.386 INFO:teuthology.orchestra.run.smithi117.stdout:URL : http://www.gnu.org/software/grub/ 2024-06-03T02:58:10.386 INFO:teuthology.orchestra.run.smithi117.stdout:Summary : Support tools for GRUB. 2024-06-03T02:58:10.386 INFO:teuthology.orchestra.run.smithi117.stdout:Description : 2024-06-03T02:58:10.386 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T02:58:10.386 INFO:teuthology.orchestra.run.smithi117.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-06-03T02:58:10.386 INFO:teuthology.orchestra.run.smithi117.stdout:customizable bootloader with modular architecture. It supports a rich 2024-06-03T02:58:10.386 INFO:teuthology.orchestra.run.smithi117.stdout:variety of kernel formats, file systems, computer architectures and 2024-06-03T02:58:10.386 INFO:teuthology.orchestra.run.smithi117.stdout:hardware devices. 2024-06-03T02:58:10.386 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T02:58:10.386 INFO:teuthology.orchestra.run.smithi117.stdout:This subpackage provides tools for support of all platforms. 2024-06-03T02:58:10.387 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-06-03T02:58:10.387 INFO:teuthology.task.kernel:Updating grub on smithi117 to boot 6.9.0-gb9459a26b088 2024-06-03T02:58:10.387 DEBUG:teuthology.orchestra.run.smithi117:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-06-03T02:58:10.944 INFO:teuthology.orchestra.run.smithi117.stderr:Generating grub configuration file ... 2024-06-03T02:58:12.323 INFO:teuthology.orchestra.run.smithi117.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-06-03T02:58:12.352 INFO:teuthology.orchestra.run.smithi117.stderr:done 2024-06-03T02:58:12.355 DEBUG:teuthology.orchestra.run.smithi117:> mktemp 2024-06-03T02:58:12.403 INFO:teuthology.orchestra.run.smithi117.stdout:/tmp/tmp.E0ao8exMTR 2024-06-03T02:58:12.403 DEBUG:teuthology.orchestra.run.smithi117:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.E0ao8exMTR 2024-06-03T02:58:12.428 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod 0666 /tmp/tmp.E0ao8exMTR 2024-06-03T02:58:12.582 DEBUG:teuthology.orchestra.remote:smithi117:/tmp/tmp.E0ao8exMTR is 6KB 2024-06-03T02:58:12.632 DEBUG:teuthology.orchestra.run.smithi117:> rm -fr /tmp/tmp.E0ao8exMTR 2024-06-03T02:58:12.646 DEBUG:teuthology.orchestra.run.smithi117:> sudo /bin/ls /boot/loader/entries || true 2024-06-03T02:58:12.708 INFO:teuthology.orchestra.run.smithi117.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-06-03T02:58:12.708 INFO:teuthology.orchestra.run.smithi117.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-06-03T02:58:12.709 INFO:teuthology.orchestra.run.smithi117.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-06-03T02:58:12.709 INFO:teuthology.orchestra.run.smithi117.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-gb9459a26b088.conf 2024-06-03T02:58:12.710 DEBUG:teuthology.orchestra.run.smithi117:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-gb9459a26b088 2024-06-03T02:58:12.848 DEBUG:teuthology.orchestra.run.smithi117:> sudo shutdown -r now 2024-06-03T02:58:42.850 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-06-03T02:58:42.850 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-06-03T02:58:42.851 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-03T02:59:01.278 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.117 2024-06-03T02:59:10.279 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-06-03T02:59:10.280 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-03T02:59:13.342 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.117 2024-06-03T02:59:25.343 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-06-03T02:59:25.344 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-03T03:00:25.346 DEBUG:teuthology.orchestra.remote:timed out 2024-06-03T03:00:40.347 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-06-03T03:00:40.349 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-03T03:00:40.783 DEBUG:teuthology.orchestra.run.smithi117:> true 2024-06-03T03:00:41.351 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-06-03T03:00:41.351 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.9.0-gb9459a26b088"... 2024-06-03T03:00:41.351 DEBUG:teuthology.orchestra.run.smithi117:> uname -r 2024-06-03T03:00:41.400 INFO:teuthology.orchestra.run.smithi117.stdout:6.9.0-gb9459a26b088 2024-06-03T03:00:41.401 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-gb9459a26b088 vs 6.9.0-gb9459a26b088 2024-06-03T03:00:41.401 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-06-03T03:00:41.401 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-06-03T03:00:42.401 DEBUG:teuthology.parallel:result is None 2024-06-03T03:00:42.402 INFO:teuthology.run_tasks:Running task internal.base... 2024-06-03T03:00:42.411 INFO:teuthology.task.internal:Creating test directory... 2024-06-03T03:00:42.411 DEBUG:teuthology.orchestra.run.smithi117:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-06-03T03:00:42.419 DEBUG:teuthology.orchestra.run.smithi151:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-06-03T03:00:42.421 DEBUG:teuthology.orchestra.run.smithi183:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-06-03T03:00:42.467 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-06-03T03:00:42.568 INFO:teuthology.run_tasks:Running task internal.archive... 2024-06-03T03:00:42.689 INFO:teuthology.task.internal:Creating archive directory... 2024-06-03T03:00:42.690 DEBUG:teuthology.orchestra.run.smithi117:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-06-03T03:00:42.697 DEBUG:teuthology.orchestra.run.smithi151:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-06-03T03:00:42.700 DEBUG:teuthology.orchestra.run.smithi183:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-06-03T03:00:42.798 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-06-03T03:00:42.813 INFO:teuthology.task.internal:Enabling coredump saving... 2024-06-03T03:00:42.813 DEBUG:teuthology.orchestra.run.smithi151:> test -f /run/.containerenv -o -f /.dockerenv 2024-06-03T03:00:42.829 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-03T03:00:42.830 DEBUG:teuthology.orchestra.run.smithi183:> test -f /run/.containerenv -o -f /.dockerenv 2024-06-03T03:00:42.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-03T03:00:42.847 DEBUG:teuthology.orchestra.run.smithi117:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-06-03T03:00:42.854 DEBUG:teuthology.orchestra.run.smithi151:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-06-03T03:00:42.873 DEBUG:teuthology.orchestra.run.smithi183:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-06-03T03:00:42.904 INFO:teuthology.orchestra.run.smithi151.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-03T03:00:42.927 INFO:teuthology.orchestra.run.smithi151.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-03T03:00:42.927 INFO:teuthology.orchestra.run.smithi183.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-03T03:00:42.947 INFO:teuthology.orchestra.run.smithi183.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-03T03:00:42.972 INFO:teuthology.orchestra.run.smithi117.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-03T03:00:43.022 INFO:teuthology.orchestra.run.smithi117.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-03T03:00:43.027 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-06-03T03:00:43.034 INFO:teuthology.task.internal:Configuring sudo... 2024-06-03T03:00:43.034 DEBUG:teuthology.orchestra.run.smithi117:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-06-03T03:00:43.039 DEBUG:teuthology.orchestra.run.smithi151:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-06-03T03:00:43.041 DEBUG:teuthology.orchestra.run.smithi183:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-06-03T03:00:43.117 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-06-03T03:00:43.214 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-06-03T03:00:43.215 DEBUG:teuthology.orchestra.run.smithi117:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-06-03T03:00:43.222 DEBUG:teuthology.orchestra.run.smithi151:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-06-03T03:00:43.224 DEBUG:teuthology.orchestra.run.smithi183:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-06-03T03:00:43.271 DEBUG:teuthology.orchestra.run.smithi117:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-03T03:00:43.349 DEBUG:teuthology.orchestra.run.smithi117:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-03T03:00:43.452 DEBUG:teuthology.orchestra.run.smithi117:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-03T03:00:43.527 DEBUG:teuthology.orchestra.run.smithi117:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-03T03:00:43.610 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-06-03T03:00:43.611 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-06-03T03:00:43.693 DEBUG:teuthology.orchestra.run.smithi151:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-03T03:00:43.735 DEBUG:teuthology.orchestra.run.smithi151:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-03T03:00:43.796 DEBUG:teuthology.orchestra.run.smithi151:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-03T03:00:43.878 DEBUG:teuthology.orchestra.run.smithi151:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-03T03:00:43.921 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-06-03T03:00:43.921 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-06-03T03:00:43.987 DEBUG:teuthology.orchestra.run.smithi183:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-03T03:00:44.026 DEBUG:teuthology.orchestra.run.smithi183:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-03T03:00:44.090 DEBUG:teuthology.orchestra.run.smithi183:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-03T03:00:44.167 DEBUG:teuthology.orchestra.run.smithi183:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-03T03:00:44.216 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-06-03T03:00:44.216 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-06-03T03:00:44.281 DEBUG:teuthology.orchestra.run.smithi117:> sudo service rsyslog restart 2024-06-03T03:00:44.289 DEBUG:teuthology.orchestra.run.smithi151:> sudo service rsyslog restart 2024-06-03T03:00:44.291 DEBUG:teuthology.orchestra.run.smithi183:> sudo service rsyslog restart 2024-06-03T03:00:44.327 INFO:teuthology.orchestra.run.smithi151.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-03T03:00:44.358 INFO:teuthology.orchestra.run.smithi183.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-03T03:00:44.396 INFO:teuthology.orchestra.run.smithi117.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-03T03:00:44.777 INFO:teuthology.run_tasks:Running task internal.timer... 2024-06-03T03:00:44.785 INFO:teuthology.task.internal:Starting timer... 2024-06-03T03:00:44.785 INFO:teuthology.run_tasks:Running task pcp... 2024-06-03T03:00:44.906 INFO:teuthology.run_tasks:Running task selinux... 2024-06-03T03:00:44.989 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-06-03T03:00:44.990 DEBUG:teuthology.orchestra.run.smithi117:> sudo service auditd rotate 2024-06-03T03:00:45.100 INFO:teuthology.orchestra.run.smithi117.stdout:Rotating logs: 2024-06-03T03:00:45.105 DEBUG:teuthology.orchestra.run.smithi151:> sudo service auditd rotate 2024-06-03T03:00:45.167 INFO:teuthology.orchestra.run.smithi151.stdout:Rotating logs: 2024-06-03T03:00:45.170 DEBUG:teuthology.orchestra.run.smithi183:> sudo service auditd rotate 2024-06-03T03:00:45.231 INFO:teuthology.orchestra.run.smithi183.stdout:Rotating logs: 2024-06-03T03:00:45.233 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-06-03T03:00:45.233 DEBUG:teuthology.orchestra.run.smithi117:> /usr/sbin/getenforce 2024-06-03T03:00:45.294 INFO:teuthology.orchestra.run.smithi117.stdout:Permissive 2024-06-03T03:00:45.295 DEBUG:teuthology.orchestra.run.smithi151:> /usr/sbin/getenforce 2024-06-03T03:00:45.314 INFO:teuthology.orchestra.run.smithi151.stdout:Permissive 2024-06-03T03:00:45.315 DEBUG:teuthology.orchestra.run.smithi183:> /usr/sbin/getenforce 2024-06-03T03:00:45.337 INFO:teuthology.orchestra.run.smithi183.stdout:Permissive 2024-06-03T03:00:45.337 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi117.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi151.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi183.front.sepia.ceph.com': 'permissive'} 2024-06-03T03:00:45.338 DEBUG:teuthology.orchestra.run.smithi117:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-03T03:00:45.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-03T03:00:45.418 DEBUG:teuthology.orchestra.run.smithi151:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-03T03:00:45.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-03T03:00:45.445 DEBUG:teuthology.orchestra.run.smithi183:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-03T03:00:45.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-03T03:00:45.471 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-06-03T03:00:45.472 DEBUG:teuthology.orchestra.run.smithi117:> sudo /usr/sbin/setenforce permissive 2024-06-03T03:00:45.539 DEBUG:teuthology.orchestra.run.smithi151:> sudo /usr/sbin/setenforce permissive 2024-06-03T03:00:45.563 DEBUG:teuthology.orchestra.run.smithi183:> sudo /usr/sbin/setenforce permissive 2024-06-03T03:00:45.588 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-06-03T03:00:45.597 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-06-03T03:00:45.612 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-06-03T03:00:45.613 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi117.front.sepia.ceph.com,smithi151.front.sepia.ceph.com,smithi183.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-06-03T03:07:02.097 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi117.front.sepia.ceph.com'), Remote(name='ubuntu@smithi151.front.sepia.ceph.com'), Remote(name='ubuntu@smithi183.front.sepia.ceph.com')] 2024-06-03T03:07:02.099 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-06-03T03:07:02.100 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-03T03:07:02.214 DEBUG:teuthology.orchestra.run.smithi117:> true 2024-06-03T03:07:02.381 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-06-03T03:07:02.381 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi151.front.sepia.ceph.com' 2024-06-03T03:07:02.382 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-03T03:07:02.461 DEBUG:teuthology.orchestra.run.smithi151:> true 2024-06-03T03:07:02.547 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi151.front.sepia.ceph.com' 2024-06-03T03:07:02.547 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi183.front.sepia.ceph.com' 2024-06-03T03:07:02.548 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-03T03:07:02.628 DEBUG:teuthology.orchestra.run.smithi183:> true 2024-06-03T03:07:02.707 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi183.front.sepia.ceph.com' 2024-06-03T03:07:02.708 INFO:teuthology.run_tasks:Running task clock... 2024-06-03T03:07:02.720 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-06-03T03:07:02.721 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-06-03T03:07:02.721 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-03T03:07:02.729 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-06-03T03:07:02.729 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-03T03:07:02.731 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-06-03T03:07:02.731 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-03T03:07:02.761 INFO:teuthology.orchestra.run.smithi151.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-06-03T03:07:02.776 INFO:teuthology.orchestra.run.smithi151.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-06-03T03:07:02.779 INFO:teuthology.orchestra.run.smithi183.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-06-03T03:07:02.793 INFO:teuthology.orchestra.run.smithi183.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-06-03T03:07:02.800 INFO:teuthology.orchestra.run.smithi151.stderr:sudo: ntpd: command not found 2024-06-03T03:07:02.811 INFO:teuthology.orchestra.run.smithi151.stdout:506 Cannot talk to daemon 2024-06-03T03:07:02.816 INFO:teuthology.orchestra.run.smithi117.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-06-03T03:07:02.817 INFO:teuthology.orchestra.run.smithi183.stderr:sudo: ntpd: command not found 2024-06-03T03:07:02.825 INFO:teuthology.orchestra.run.smithi151.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-06-03T03:07:02.828 INFO:teuthology.orchestra.run.smithi183.stdout:506 Cannot talk to daemon 2024-06-03T03:07:02.838 INFO:teuthology.orchestra.run.smithi151.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-06-03T03:07:02.841 INFO:teuthology.orchestra.run.smithi183.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-06-03T03:07:02.855 INFO:teuthology.orchestra.run.smithi183.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-06-03T03:07:02.858 INFO:teuthology.orchestra.run.smithi117.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-06-03T03:07:02.892 INFO:teuthology.orchestra.run.smithi151.stderr:bash: line 1: ntpq: command not found 2024-06-03T03:07:02.896 INFO:teuthology.orchestra.run.smithi151.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-03T03:07:02.896 INFO:teuthology.orchestra.run.smithi151.stdout:=============================================================================== 2024-06-03T03:07:02.896 INFO:teuthology.orchestra.run.smithi151.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-03T03:07:02.896 INFO:teuthology.orchestra.run.smithi151.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-03T03:07:02.896 INFO:teuthology.orchestra.run.smithi151.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-03T03:07:02.896 INFO:teuthology.orchestra.run.smithi151.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-03T03:07:02.914 INFO:teuthology.orchestra.run.smithi183.stderr:bash: line 1: ntpq: command not found 2024-06-03T03:07:02.917 INFO:teuthology.orchestra.run.smithi183.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-03T03:07:02.918 INFO:teuthology.orchestra.run.smithi183.stdout:=============================================================================== 2024-06-03T03:07:02.918 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-03T03:07:02.918 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-03T03:07:02.918 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-03T03:07:02.918 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-03T03:07:02.932 INFO:teuthology.orchestra.run.smithi117.stderr:sudo: ntpd: command not found 2024-06-03T03:07:02.965 INFO:teuthology.orchestra.run.smithi117.stdout:506 Cannot talk to daemon 2024-06-03T03:07:03.006 INFO:teuthology.orchestra.run.smithi117.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-06-03T03:07:03.048 INFO:teuthology.orchestra.run.smithi117.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-06-03T03:07:03.186 INFO:teuthology.orchestra.run.smithi117.stderr:bash: line 1: ntpq: command not found 2024-06-03T03:07:03.194 INFO:teuthology.orchestra.run.smithi117.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-03T03:07:03.194 INFO:teuthology.orchestra.run.smithi117.stdout:=============================================================================== 2024-06-03T03:07:03.194 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-03T03:07:03.194 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-03T03:07:03.194 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-03T03:07:03.194 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-03T03:07:03.196 INFO:teuthology.run_tasks:Running task install... 2024-06-03T03:07:03.206 DEBUG:teuthology.task.install:project ceph 2024-06-03T03:07:03.206 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba'}} 2024-06-03T03:07:03.206 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba'} 2024-06-03T03:07:03.206 INFO:teuthology.task.install:Using flavor: default 2024-06-03T03:07:03.211 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-06-03T03:07:03.212 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-06-03T03:07:03.212 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba', 'tag': None, 'wait_for_package': False} 2024-06-03T03:07:03.212 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-03T03:07:03.214 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba', 'tag': None, 'wait_for_package': False} 2024-06-03T03:07:03.215 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-03T03:07:03.215 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba', 'tag': None, 'wait_for_package': False} 2024-06-03T03:07:03.216 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-03T03:07:03.367 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/ 2024-06-03T03:07:03.368 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2946.g3d89accf 2024-06-03T03:07:03.442 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/ 2024-06-03T03:07:03.442 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2946.g3d89accf 2024-06-03T03:07:03.445 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/ 2024-06-03T03:07:03.445 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2946.g3d89accf 2024-06-03T03:07:03.699 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-06-03T03:07:03.700 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-06-03T03:07:03.700 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-06-03T03:07:03.728 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-06-03T03:07:03.728 DEBUG:teuthology.orchestra.run.smithi151:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3d89accfe3201286f000e065a1faea2a88c7a6ba/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-06-03T03:07:03.774 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-06-03T03:07:03.774 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-06-03T03:07:03.774 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-06-03T03:07:03.796 DEBUG:teuthology.orchestra.run.smithi151:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-06-03T03:07:03.801 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-06-03T03:07:03.801 DEBUG:teuthology.orchestra.run.smithi183:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3d89accfe3201286f000e065a1faea2a88c7a6ba/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-06-03T03:07:03.827 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-06-03T03:07:03.827 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-06-03T03:07:03.827 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-06-03T03:07:03.866 DEBUG:teuthology.orchestra.run.smithi183:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-06-03T03:07:03.872 DEBUG:teuthology.orchestra.run.smithi151:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-06-03T03:07:03.917 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-06-03T03:07:03.917 DEBUG:teuthology.orchestra.run.smithi117:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3d89accfe3201286f000e065a1faea2a88c7a6ba/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-06-03T03:07:03.935 INFO:teuthology.orchestra.run.smithi151.stdout:check_obsoletes = 1 2024-06-03T03:07:03.937 DEBUG:teuthology.orchestra.run.smithi151:> sudo yum clean all 2024-06-03T03:07:03.940 DEBUG:teuthology.orchestra.run.smithi183:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-06-03T03:07:04.004 INFO:teuthology.orchestra.run.smithi183.stdout:check_obsoletes = 1 2024-06-03T03:07:04.005 DEBUG:teuthology.orchestra.run.smithi183:> sudo yum clean all 2024-06-03T03:07:04.046 DEBUG:teuthology.orchestra.run.smithi117:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-06-03T03:07:04.160 DEBUG:teuthology.orchestra.run.smithi117:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-06-03T03:07:04.164 INFO:teuthology.orchestra.run.smithi151.stdout:45 files removed 2024-06-03T03:07:04.190 DEBUG:teuthology.orchestra.run.smithi151:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-06-03T03:07:04.229 INFO:teuthology.orchestra.run.smithi183.stdout:45 files removed 2024-06-03T03:07:04.242 INFO:teuthology.orchestra.run.smithi117.stdout:check_obsoletes = 1 2024-06-03T03:07:04.247 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum clean all 2024-06-03T03:07:04.255 DEBUG:teuthology.orchestra.run.smithi183:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-06-03T03:07:04.566 INFO:teuthology.orchestra.run.smithi117.stdout:45 files removed 2024-06-03T03:07:04.607 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-06-03T03:07:05.217 INFO:teuthology.orchestra.run.smithi151.stdout:ceph packages for x86_64 109 kB/s | 83 kB 00:00 2024-06-03T03:07:05.297 INFO:teuthology.orchestra.run.smithi183.stdout:ceph packages for x86_64 107 kB/s | 83 kB 00:00 2024-06-03T03:07:05.818 INFO:teuthology.orchestra.run.smithi151.stdout:ceph noarch packages 21 kB/s | 11 kB 00:00 2024-06-03T03:07:05.880 INFO:teuthology.orchestra.run.smithi117.stdout:ceph packages for x86_64 100 kB/s | 83 kB 00:00 2024-06-03T03:07:05.948 INFO:teuthology.orchestra.run.smithi183.stdout:ceph noarch packages 19 kB/s | 11 kB 00:00 2024-06-03T03:07:06.185 INFO:teuthology.orchestra.run.smithi151.stdout:ceph source packages 5.7 kB/s | 1.9 kB 00:00 2024-06-03T03:07:06.407 INFO:teuthology.orchestra.run.smithi183.stdout:ceph source packages 4.5 kB/s | 1.9 kB 00:00 2024-06-03T03:07:06.447 INFO:teuthology.orchestra.run.smithi117.stdout:ceph noarch packages 25 kB/s | 11 kB 00:00 2024-06-03T03:07:06.602 INFO:teuthology.orchestra.run.smithi151.stdout:CentOS Stream 9 - BaseOS 21 MB/s | 8.1 MB 00:00 2024-06-03T03:07:06.982 INFO:teuthology.orchestra.run.smithi117.stdout:ceph source packages 4.2 kB/s | 1.9 kB 00:00 2024-06-03T03:07:07.857 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - BaseOS 5.7 MB/s | 8.1 MB 00:01 2024-06-03T03:07:08.478 INFO:teuthology.orchestra.run.smithi151.stdout:CentOS Stream 9 - AppStream 24 MB/s | 19 MB 00:00 2024-06-03T03:07:08.729 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - BaseOS 4.9 MB/s | 8.1 MB 00:01 2024-06-03T03:07:09.849 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - AppStream 21 MB/s | 19 MB 00:00 2024-06-03T03:07:10.689 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - AppStream 27 MB/s | 19 MB 00:00 2024-06-03T03:07:12.529 INFO:teuthology.orchestra.run.smithi151.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.2 MB 00:00 2024-06-03T03:07:14.980 INFO:teuthology.orchestra.run.smithi151.stdout:CentOS Stream 9 - Extras packages 13 kB/s | 17 kB 00:01 2024-06-03T03:07:15.139 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.2 MB 00:00 2024-06-03T03:07:15.266 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - CRB 3.3 MB/s | 6.2 MB 00:01 2024-06-03T03:07:16.023 INFO:teuthology.orchestra.run.smithi151.stdout:Extra Packages for Enterprise Linux 24 MB/s | 21 MB 00:00 2024-06-03T03:07:16.776 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - Extras packages 50 kB/s | 17 kB 00:00 2024-06-03T03:07:16.890 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - Extras packages 44 kB/s | 17 kB 00:00 2024-06-03T03:07:17.794 INFO:teuthology.orchestra.run.smithi183.stdout:Extra Packages for Enterprise Linux 24 MB/s | 21 MB 00:00 2024-06-03T03:07:18.746 INFO:teuthology.orchestra.run.smithi117.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-06-03T03:07:21.257 INFO:teuthology.orchestra.run.smithi151.stdout:lab-extras 18 kB/s | 1.7 kB 00:00 2024-06-03T03:07:23.010 INFO:teuthology.orchestra.run.smithi183.stdout:lab-extras 18 kB/s | 1.7 kB 00:00 2024-06-03T03:07:23.558 INFO:teuthology.orchestra.run.smithi151.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-03T03:07:23.559 INFO:teuthology.orchestra.run.smithi151.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-03T03:07:23.570 INFO:teuthology.orchestra.run.smithi151.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-06-03T03:07:23.571 INFO:teuthology.orchestra.run.smithi151.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-06-03T03:07:23.571 INFO:teuthology.orchestra.run.smithi151.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-06-03T03:07:23.572 INFO:teuthology.orchestra.run.smithi151.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-06-03T03:07:23.574 INFO:teuthology.orchestra.run.smithi151.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-06-03T03:07:23.576 INFO:teuthology.orchestra.run.smithi151.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-06-03T03:07:23.577 INFO:teuthology.orchestra.run.smithi151.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-06-03T03:07:23.641 INFO:teuthology.orchestra.run.smithi151.stdout:Dependencies resolved. 2024-06-03T03:07:23.657 INFO:teuthology.orchestra.run.smithi151.stdout:======================================================================================= 2024-06-03T03:07:23.657 INFO:teuthology.orchestra.run.smithi151.stdout: Package Arch Version Repository Size 2024-06-03T03:07:23.657 INFO:teuthology.orchestra.run.smithi151.stdout:======================================================================================= 2024-06-03T03:07:23.657 INFO:teuthology.orchestra.run.smithi151.stdout:Installing: 2024-06-03T03:07:23.657 INFO:teuthology.orchestra.run.smithi151.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-06-03T03:07:23.657 INFO:teuthology.orchestra.run.smithi151.stdout: ceph x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 6.6 k 2024-06-03T03:07:23.658 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-base x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.4 M 2024-06-03T03:07:23.658 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 890 k 2024-06-03T03:07:23.658 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 145 k 2024-06-03T03:07:23.658 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 1.5 M 2024-06-03T03:07:23.658 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 142 k 2024-06-03T03:07:23.658 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 3.5 M 2024-06-03T03:07:23.658 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 7.4 M 2024-06-03T03:07:23.658 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-rook noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 49 k 2024-06-03T03:07:23.658 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-radosgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 10 M 2024-06-03T03:07:23.658 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-test x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 48 M 2024-06-03T03:07:23.658 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-volume noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 264 k 2024-06-03T03:07:23.659 INFO:teuthology.orchestra.run.smithi151.stdout: cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 763 k 2024-06-03T03:07:23.659 INFO:teuthology.orchestra.run.smithi151.stdout: cephfs-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 221 k 2024-06-03T03:07:23.659 INFO:teuthology.orchestra.run.smithi151.stdout: cephfs-shell x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 27 k 2024-06-03T03:07:23.659 INFO:teuthology.orchestra.run.smithi151.stdout: cephfs-top noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 18 k 2024-06-03T03:07:23.659 INFO:teuthology.orchestra.run.smithi151.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-06-03T03:07:23.659 INFO:teuthology.orchestra.run.smithi151.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-06-03T03:07:23.659 INFO:teuthology.orchestra.run.smithi151.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-06-03T03:07:23.659 INFO:teuthology.orchestra.run.smithi151.stdout: libcephfs-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 32 k 2024-06-03T03:07:23.659 INFO:teuthology.orchestra.run.smithi151.stdout: libcephfs2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 747 k 2024-06-03T03:07:23.659 INFO:teuthology.orchestra.run.smithi151.stdout: librados-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 127 k 2024-06-03T03:07:23.659 INFO:teuthology.orchestra.run.smithi151.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-06-03T03:07:23.659 INFO:teuthology.orchestra.run.smithi151.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-06-03T03:07:23.660 INFO:teuthology.orchestra.run.smithi151.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-06-03T03:07:23.660 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-06-03T03:07:23.660 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-06-03T03:07:23.660 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-06-03T03:07:23.660 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cephfs x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 162 k 2024-06-03T03:07:23.660 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rados x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 321 k 2024-06-03T03:07:23.660 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 300 k 2024-06-03T03:07:23.660 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 99 k 2024-06-03T03:07:23.660 INFO:teuthology.orchestra.run.smithi151.stdout: rbd-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 85 k 2024-06-03T03:07:23.660 INFO:teuthology.orchestra.run.smithi151.stdout: rbd-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-03T03:07:23.660 INFO:teuthology.orchestra.run.smithi151.stdout: rbd-nbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 171 k 2024-06-03T03:07:23.660 INFO:teuthology.orchestra.run.smithi151.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-06-03T03:07:23.660 INFO:teuthology.orchestra.run.smithi151.stdout:Upgrading: 2024-06-03T03:07:23.661 INFO:teuthology.orchestra.run.smithi151.stdout: NetworkManager x86_64 1:1.47.91-1.el9 baseos 2.3 M 2024-06-03T03:07:23.661 INFO:teuthology.orchestra.run.smithi151.stdout: NetworkManager-libnm x86_64 1:1.47.91-1.el9 baseos 1.8 M 2024-06-03T03:07:23.661 INFO:teuthology.orchestra.run.smithi151.stdout: NetworkManager-team x86_64 1:1.47.91-1.el9 baseos 41 k 2024-06-03T03:07:23.661 INFO:teuthology.orchestra.run.smithi151.stdout: NetworkManager-tui x86_64 1:1.47.91-1.el9 baseos 248 k 2024-06-03T03:07:23.661 INFO:teuthology.orchestra.run.smithi151.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-06-03T03:07:23.661 INFO:teuthology.orchestra.run.smithi151.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-06-03T03:07:23.661 INFO:teuthology.orchestra.run.smithi151.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-06-03T03:07:23.661 INFO:teuthology.orchestra.run.smithi151.stdout: librados2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.4 M 2024-06-03T03:07:23.661 INFO:teuthology.orchestra.run.smithi151.stdout: librbd1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-03T03:07:23.661 INFO:teuthology.orchestra.run.smithi151.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-06-03T03:07:23.661 INFO:teuthology.orchestra.run.smithi151.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-06-03T03:07:23.661 INFO:teuthology.orchestra.run.smithi151.stdout:Installing dependencies: 2024-06-03T03:07:23.661 INFO:teuthology.orchestra.run.smithi151.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-06-03T03:07:23.662 INFO:teuthology.orchestra.run.smithi151.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-06-03T03:07:23.662 INFO:teuthology.orchestra.run.smithi151.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-06-03T03:07:23.662 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 21 M 2024-06-03T03:07:23.662 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 32 k 2024-06-03T03:07:23.662 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mds x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 2.4 M 2024-06-03T03:07:23.662 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 249 k 2024-06-03T03:07:23.662 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mon x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 4.7 M 2024-06-03T03:07:23.662 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-osd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 17 M 2024-06-03T03:07:23.662 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 16 k 2024-06-03T03:07:23.662 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-selinux x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 25 k 2024-06-03T03:07:23.662 INFO:teuthology.orchestra.run.smithi151.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-06-03T03:07:23.663 INFO:teuthology.orchestra.run.smithi151.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-06-03T03:07:23.663 INFO:teuthology.orchestra.run.smithi151.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-06-03T03:07:23.663 INFO:teuthology.orchestra.run.smithi151.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-06-03T03:07:23.663 INFO:teuthology.orchestra.run.smithi151.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-06-03T03:07:23.663 INFO:teuthology.orchestra.run.smithi151.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-06-03T03:07:23.663 INFO:teuthology.orchestra.run.smithi151.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-06-03T03:07:23.663 INFO:teuthology.orchestra.run.smithi151.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-06-03T03:07:23.663 INFO:teuthology.orchestra.run.smithi151.stdout: libcephsqlite x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 163 k 2024-06-03T03:07:23.663 INFO:teuthology.orchestra.run.smithi151.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-06-03T03:07:23.663 INFO:teuthology.orchestra.run.smithi151.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-06-03T03:07:23.663 INFO:teuthology.orchestra.run.smithi151.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-06-03T03:07:23.663 INFO:teuthology.orchestra.run.smithi151.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-06-03T03:07:23.664 INFO:teuthology.orchestra.run.smithi151.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-06-03T03:07:23.664 INFO:teuthology.orchestra.run.smithi151.stdout: libradosstriper1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 501 k 2024-06-03T03:07:23.664 INFO:teuthology.orchestra.run.smithi151.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-06-03T03:07:23.664 INFO:teuthology.orchestra.run.smithi151.stdout: librgw2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.3 M 2024-06-03T03:07:23.664 INFO:teuthology.orchestra.run.smithi151.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-06-03T03:07:23.664 INFO:teuthology.orchestra.run.smithi151.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-06-03T03:07:23.664 INFO:teuthology.orchestra.run.smithi151.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-06-03T03:07:23.664 INFO:teuthology.orchestra.run.smithi151.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-06-03T03:07:23.664 INFO:teuthology.orchestra.run.smithi151.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-06-03T03:07:23.664 INFO:teuthology.orchestra.run.smithi151.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-06-03T03:07:23.665 INFO:teuthology.orchestra.run.smithi151.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-06-03T03:07:23.665 INFO:teuthology.orchestra.run.smithi151.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-06-03T03:07:23.665 INFO:teuthology.orchestra.run.smithi151.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-06-03T03:07:23.665 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-06-03T03:07:23.665 INFO:teuthology.orchestra.run.smithi151.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-06-03T03:07:23.665 INFO:teuthology.orchestra.run.smithi151.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-06-03T03:07:23.665 INFO:teuthology.orchestra.run.smithi151.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-06-03T03:07:23.665 INFO:teuthology.orchestra.run.smithi151.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-06-03T03:07:23.665 INFO:teuthology.orchestra.run.smithi151.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-06-03T03:07:23.665 INFO:teuthology.orchestra.run.smithi151.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-06-03T03:07:23.665 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-06-03T03:07:23.665 INFO:teuthology.orchestra.run.smithi151.stdout: python3-ceph-argparse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 45 k 2024-06-03T03:07:23.665 INFO:teuthology.orchestra.run.smithi151.stdout: python3-ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 135 k 2024-06-03T03:07:23.665 INFO:teuthology.orchestra.run.smithi151.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-06-03T03:07:23.666 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-06-03T03:07:23.666 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-06-03T03:07:23.666 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-06-03T03:07:23.666 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-06-03T03:07:23.666 INFO:teuthology.orchestra.run.smithi151.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-06-03T03:07:23.666 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-06-03T03:07:23.666 INFO:teuthology.orchestra.run.smithi151.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-06-03T03:07:23.666 INFO:teuthology.orchestra.run.smithi151.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-06-03T03:07:23.666 INFO:teuthology.orchestra.run.smithi151.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-06-03T03:07:23.666 INFO:teuthology.orchestra.run.smithi151.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-06-03T03:07:23.666 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-06-03T03:07:23.666 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-06-03T03:07:23.666 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-06-03T03:07:23.666 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-06-03T03:07:23.667 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-06-03T03:07:23.667 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-06-03T03:07:23.667 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-06-03T03:07:23.667 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-06-03T03:07:23.667 INFO:teuthology.orchestra.run.smithi151.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-06-03T03:07:23.667 INFO:teuthology.orchestra.run.smithi151.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-06-03T03:07:23.667 INFO:teuthology.orchestra.run.smithi151.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-06-03T03:07:23.667 INFO:teuthology.orchestra.run.smithi151.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-06-03T03:07:23.668 INFO:teuthology.orchestra.run.smithi151.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-06-03T03:07:23.668 INFO:teuthology.orchestra.run.smithi151.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-06-03T03:07:23.668 INFO:teuthology.orchestra.run.smithi151.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-06-03T03:07:23.668 INFO:teuthology.orchestra.run.smithi151.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-06-03T03:07:23.668 INFO:teuthology.orchestra.run.smithi151.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-06-03T03:07:23.668 INFO:teuthology.orchestra.run.smithi151.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-06-03T03:07:23.668 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-06-03T03:07:23.668 INFO:teuthology.orchestra.run.smithi151.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-06-03T03:07:23.668 INFO:teuthology.orchestra.run.smithi151.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-06-03T03:07:23.668 INFO:teuthology.orchestra.run.smithi151.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-06-03T03:07:23.668 INFO:teuthology.orchestra.run.smithi151.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-06-03T03:07:23.668 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-06-03T03:07:23.668 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-06-03T03:07:23.669 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-06-03T03:07:23.669 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-06-03T03:07:23.669 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-06-03T03:07:23.669 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-06-03T03:07:23.669 INFO:teuthology.orchestra.run.smithi151.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-06-03T03:07:23.669 INFO:teuthology.orchestra.run.smithi151.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-06-03T03:07:23.669 INFO:teuthology.orchestra.run.smithi151.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-06-03T03:07:23.669 INFO:teuthology.orchestra.run.smithi151.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-06-03T03:07:23.669 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-06-03T03:07:23.669 INFO:teuthology.orchestra.run.smithi151.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-06-03T03:07:23.669 INFO:teuthology.orchestra.run.smithi151.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-06-03T03:07:23.669 INFO:teuthology.orchestra.run.smithi151.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-06-03T03:07:23.669 INFO:teuthology.orchestra.run.smithi151.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-06-03T03:07:23.670 INFO:teuthology.orchestra.run.smithi151.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-06-03T03:07:23.670 INFO:teuthology.orchestra.run.smithi151.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-06-03T03:07:23.670 INFO:teuthology.orchestra.run.smithi151.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-06-03T03:07:23.670 INFO:teuthology.orchestra.run.smithi151.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-06-03T03:07:23.670 INFO:teuthology.orchestra.run.smithi151.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-06-03T03:07:23.670 INFO:teuthology.orchestra.run.smithi151.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-06-03T03:07:23.670 INFO:teuthology.orchestra.run.smithi151.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-06-03T03:07:23.670 INFO:teuthology.orchestra.run.smithi151.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-06-03T03:07:23.670 INFO:teuthology.orchestra.run.smithi151.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-06-03T03:07:23.670 INFO:teuthology.orchestra.run.smithi151.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-06-03T03:07:23.670 INFO:teuthology.orchestra.run.smithi151.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-06-03T03:07:23.670 INFO:teuthology.orchestra.run.smithi151.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-06-03T03:07:23.671 INFO:teuthology.orchestra.run.smithi151.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-06-03T03:07:23.671 INFO:teuthology.orchestra.run.smithi151.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-06-03T03:07:23.671 INFO:teuthology.orchestra.run.smithi151.stdout:Installing weak dependencies: 2024-06-03T03:07:23.671 INFO:teuthology.orchestra.run.smithi151.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-06-03T03:07:23.671 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-06-03T03:07:23.671 INFO:teuthology.orchestra.run.smithi151.stdout:Transaction Summary 2024-06-03T03:07:23.671 INFO:teuthology.orchestra.run.smithi151.stdout:======================================================================================= 2024-06-03T03:07:23.671 INFO:teuthology.orchestra.run.smithi151.stdout:Install 148 Packages 2024-06-03T03:07:23.671 INFO:teuthology.orchestra.run.smithi151.stdout:Upgrade 11 Packages 2024-06-03T03:07:23.671 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-06-03T03:07:23.672 INFO:teuthology.orchestra.run.smithi151.stdout:Total download size: 225 M 2024-06-03T03:07:23.672 INFO:teuthology.orchestra.run.smithi151.stdout:Downloading Packages: 2024-06-03T03:07:24.550 INFO:teuthology.orchestra.run.smithi151.stdout:(1/159): ceph-19.0.0-2946.g3d89accf.el9.x86_64. 31 kB/s | 6.6 kB 00:00 2024-06-03T03:07:24.947 INFO:teuthology.orchestra.run.smithi117.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-06-03T03:07:25.225 INFO:teuthology.orchestra.run.smithi151.stdout:(2/159): ceph-base-19.0.0-2946.g3d89accf.el9.x8 6.0 MB/s | 5.4 MB 00:00 2024-06-03T03:07:25.303 INFO:teuthology.orchestra.run.smithi183.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-03T03:07:25.303 INFO:teuthology.orchestra.run.smithi183.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-03T03:07:25.314 INFO:teuthology.orchestra.run.smithi183.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-06-03T03:07:25.315 INFO:teuthology.orchestra.run.smithi183.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-06-03T03:07:25.316 INFO:teuthology.orchestra.run.smithi183.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-06-03T03:07:25.317 INFO:teuthology.orchestra.run.smithi183.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-06-03T03:07:25.318 INFO:teuthology.orchestra.run.smithi183.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-06-03T03:07:25.320 INFO:teuthology.orchestra.run.smithi183.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-06-03T03:07:25.321 INFO:teuthology.orchestra.run.smithi183.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-06-03T03:07:25.333 INFO:teuthology.orchestra.run.smithi151.stdout:(3/159): ceph-immutable-object-cache-19.0.0-294 1.3 MB/s | 145 kB 00:00 2024-06-03T03:07:25.368 INFO:teuthology.orchestra.run.smithi151.stdout:(4/159): ceph-fuse-19.0.0-2946.g3d89accf.el9.x8 1.1 MB/s | 890 kB 00:00 2024-06-03T03:07:25.383 INFO:teuthology.orchestra.run.smithi183.stdout:Dependencies resolved. 2024-06-03T03:07:25.408 INFO:teuthology.orchestra.run.smithi183.stdout:======================================================================================= 2024-06-03T03:07:25.408 INFO:teuthology.orchestra.run.smithi183.stdout: Package Arch Version Repository Size 2024-06-03T03:07:25.408 INFO:teuthology.orchestra.run.smithi183.stdout:======================================================================================= 2024-06-03T03:07:25.409 INFO:teuthology.orchestra.run.smithi183.stdout:Installing: 2024-06-03T03:07:25.409 INFO:teuthology.orchestra.run.smithi183.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-06-03T03:07:25.409 INFO:teuthology.orchestra.run.smithi183.stdout: ceph x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 6.6 k 2024-06-03T03:07:25.409 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-base x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.4 M 2024-06-03T03:07:25.409 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 890 k 2024-06-03T03:07:25.409 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 145 k 2024-06-03T03:07:25.409 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 1.5 M 2024-06-03T03:07:25.409 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 142 k 2024-06-03T03:07:25.409 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 3.5 M 2024-06-03T03:07:25.409 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 7.4 M 2024-06-03T03:07:25.409 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-rook noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 49 k 2024-06-03T03:07:25.410 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-radosgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 10 M 2024-06-03T03:07:25.410 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-test x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 48 M 2024-06-03T03:07:25.410 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-volume noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 264 k 2024-06-03T03:07:25.410 INFO:teuthology.orchestra.run.smithi183.stdout: cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 763 k 2024-06-03T03:07:25.410 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 221 k 2024-06-03T03:07:25.410 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-shell x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 27 k 2024-06-03T03:07:25.410 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-top noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 18 k 2024-06-03T03:07:25.410 INFO:teuthology.orchestra.run.smithi183.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-06-03T03:07:25.410 INFO:teuthology.orchestra.run.smithi183.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-06-03T03:07:25.410 INFO:teuthology.orchestra.run.smithi183.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-06-03T03:07:25.410 INFO:teuthology.orchestra.run.smithi183.stdout: libcephfs-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 32 k 2024-06-03T03:07:25.411 INFO:teuthology.orchestra.run.smithi183.stdout: libcephfs2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 747 k 2024-06-03T03:07:25.411 INFO:teuthology.orchestra.run.smithi183.stdout: librados-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 127 k 2024-06-03T03:07:25.411 INFO:teuthology.orchestra.run.smithi183.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-06-03T03:07:25.411 INFO:teuthology.orchestra.run.smithi183.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-06-03T03:07:25.411 INFO:teuthology.orchestra.run.smithi183.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-06-03T03:07:25.411 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-06-03T03:07:25.411 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-06-03T03:07:25.411 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-06-03T03:07:25.411 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cephfs x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 162 k 2024-06-03T03:07:25.411 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rados x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 321 k 2024-06-03T03:07:25.411 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 300 k 2024-06-03T03:07:25.411 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 99 k 2024-06-03T03:07:25.411 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 85 k 2024-06-03T03:07:25.412 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-03T03:07:25.412 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-nbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 171 k 2024-06-03T03:07:25.412 INFO:teuthology.orchestra.run.smithi183.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-06-03T03:07:25.412 INFO:teuthology.orchestra.run.smithi183.stdout:Upgrading: 2024-06-03T03:07:25.412 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager x86_64 1:1.47.91-1.el9 baseos 2.3 M 2024-06-03T03:07:25.412 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager-libnm x86_64 1:1.47.91-1.el9 baseos 1.8 M 2024-06-03T03:07:25.412 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager-team x86_64 1:1.47.91-1.el9 baseos 41 k 2024-06-03T03:07:25.412 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager-tui x86_64 1:1.47.91-1.el9 baseos 248 k 2024-06-03T03:07:25.412 INFO:teuthology.orchestra.run.smithi183.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-06-03T03:07:25.412 INFO:teuthology.orchestra.run.smithi183.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-06-03T03:07:25.412 INFO:teuthology.orchestra.run.smithi183.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-06-03T03:07:25.412 INFO:teuthology.orchestra.run.smithi183.stdout: librados2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.4 M 2024-06-03T03:07:25.413 INFO:teuthology.orchestra.run.smithi183.stdout: librbd1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-03T03:07:25.413 INFO:teuthology.orchestra.run.smithi183.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-06-03T03:07:25.413 INFO:teuthology.orchestra.run.smithi183.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-06-03T03:07:25.413 INFO:teuthology.orchestra.run.smithi183.stdout:Installing dependencies: 2024-06-03T03:07:25.413 INFO:teuthology.orchestra.run.smithi183.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-06-03T03:07:25.413 INFO:teuthology.orchestra.run.smithi183.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-06-03T03:07:25.413 INFO:teuthology.orchestra.run.smithi183.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-06-03T03:07:25.414 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 21 M 2024-06-03T03:07:25.414 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 32 k 2024-06-03T03:07:25.414 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mds x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 2.4 M 2024-06-03T03:07:25.414 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 249 k 2024-06-03T03:07:25.414 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mon x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 4.7 M 2024-06-03T03:07:25.414 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-osd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 17 M 2024-06-03T03:07:25.414 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 16 k 2024-06-03T03:07:25.414 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-selinux x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 25 k 2024-06-03T03:07:25.414 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-06-03T03:07:25.414 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-06-03T03:07:25.414 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-06-03T03:07:25.414 INFO:teuthology.orchestra.run.smithi183.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-06-03T03:07:25.415 INFO:teuthology.orchestra.run.smithi183.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-06-03T03:07:25.415 INFO:teuthology.orchestra.run.smithi183.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-06-03T03:07:25.415 INFO:teuthology.orchestra.run.smithi183.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-06-03T03:07:25.415 INFO:teuthology.orchestra.run.smithi183.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-06-03T03:07:25.415 INFO:teuthology.orchestra.run.smithi183.stdout: libcephsqlite x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 163 k 2024-06-03T03:07:25.415 INFO:teuthology.orchestra.run.smithi183.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-06-03T03:07:25.415 INFO:teuthology.orchestra.run.smithi183.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-06-03T03:07:25.415 INFO:teuthology.orchestra.run.smithi183.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-06-03T03:07:25.415 INFO:teuthology.orchestra.run.smithi183.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-06-03T03:07:25.415 INFO:teuthology.orchestra.run.smithi183.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-06-03T03:07:25.415 INFO:teuthology.orchestra.run.smithi183.stdout: libradosstriper1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 501 k 2024-06-03T03:07:25.416 INFO:teuthology.orchestra.run.smithi183.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-06-03T03:07:25.416 INFO:teuthology.orchestra.run.smithi183.stdout: librgw2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.3 M 2024-06-03T03:07:25.416 INFO:teuthology.orchestra.run.smithi183.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-06-03T03:07:25.416 INFO:teuthology.orchestra.run.smithi183.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-06-03T03:07:25.416 INFO:teuthology.orchestra.run.smithi183.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-06-03T03:07:25.416 INFO:teuthology.orchestra.run.smithi183.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-06-03T03:07:25.416 INFO:teuthology.orchestra.run.smithi183.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-06-03T03:07:25.416 INFO:teuthology.orchestra.run.smithi183.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-06-03T03:07:25.416 INFO:teuthology.orchestra.run.smithi183.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-06-03T03:07:25.416 INFO:teuthology.orchestra.run.smithi183.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-06-03T03:07:25.416 INFO:teuthology.orchestra.run.smithi183.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-06-03T03:07:25.416 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-06-03T03:07:25.416 INFO:teuthology.orchestra.run.smithi183.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-06-03T03:07:25.416 INFO:teuthology.orchestra.run.smithi183.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-06-03T03:07:25.417 INFO:teuthology.orchestra.run.smithi183.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-06-03T03:07:25.417 INFO:teuthology.orchestra.run.smithi183.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-06-03T03:07:25.417 INFO:teuthology.orchestra.run.smithi183.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-06-03T03:07:25.417 INFO:teuthology.orchestra.run.smithi183.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-06-03T03:07:25.417 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-06-03T03:07:25.417 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ceph-argparse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 45 k 2024-06-03T03:07:25.417 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 135 k 2024-06-03T03:07:25.417 INFO:teuthology.orchestra.run.smithi183.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-06-03T03:07:25.417 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-06-03T03:07:25.417 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-06-03T03:07:25.417 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-06-03T03:07:25.417 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-06-03T03:07:25.417 INFO:teuthology.orchestra.run.smithi183.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-06-03T03:07:25.418 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-06-03T03:07:25.418 INFO:teuthology.orchestra.run.smithi183.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-06-03T03:07:25.418 INFO:teuthology.orchestra.run.smithi183.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-06-03T03:07:25.418 INFO:teuthology.orchestra.run.smithi183.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-06-03T03:07:25.418 INFO:teuthology.orchestra.run.smithi183.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-06-03T03:07:25.418 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-06-03T03:07:25.418 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-06-03T03:07:25.418 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-06-03T03:07:25.418 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-06-03T03:07:25.418 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-06-03T03:07:25.418 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-06-03T03:07:25.418 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-06-03T03:07:25.419 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-06-03T03:07:25.419 INFO:teuthology.orchestra.run.smithi183.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-06-03T03:07:25.419 INFO:teuthology.orchestra.run.smithi183.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-06-03T03:07:25.419 INFO:teuthology.orchestra.run.smithi183.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-06-03T03:07:25.419 INFO:teuthology.orchestra.run.smithi183.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-06-03T03:07:25.419 INFO:teuthology.orchestra.run.smithi183.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-06-03T03:07:25.419 INFO:teuthology.orchestra.run.smithi183.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-06-03T03:07:25.419 INFO:teuthology.orchestra.run.smithi183.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-06-03T03:07:25.419 INFO:teuthology.orchestra.run.smithi183.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-06-03T03:07:25.419 INFO:teuthology.orchestra.run.smithi183.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-06-03T03:07:25.419 INFO:teuthology.orchestra.run.smithi183.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-06-03T03:07:25.419 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-06-03T03:07:25.420 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-06-03T03:07:25.420 INFO:teuthology.orchestra.run.smithi183.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-06-03T03:07:25.420 INFO:teuthology.orchestra.run.smithi183.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-06-03T03:07:25.420 INFO:teuthology.orchestra.run.smithi183.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-06-03T03:07:25.420 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-06-03T03:07:25.420 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-06-03T03:07:25.420 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-06-03T03:07:25.420 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-06-03T03:07:25.420 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-06-03T03:07:25.420 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-06-03T03:07:25.420 INFO:teuthology.orchestra.run.smithi183.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-06-03T03:07:25.420 INFO:teuthology.orchestra.run.smithi183.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-06-03T03:07:25.420 INFO:teuthology.orchestra.run.smithi183.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-06-03T03:07:25.421 INFO:teuthology.orchestra.run.smithi183.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-06-03T03:07:25.421 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-06-03T03:07:25.421 INFO:teuthology.orchestra.run.smithi183.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-06-03T03:07:25.421 INFO:teuthology.orchestra.run.smithi183.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-06-03T03:07:25.421 INFO:teuthology.orchestra.run.smithi183.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-06-03T03:07:25.421 INFO:teuthology.orchestra.run.smithi183.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-06-03T03:07:25.421 INFO:teuthology.orchestra.run.smithi183.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-06-03T03:07:25.421 INFO:teuthology.orchestra.run.smithi183.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-06-03T03:07:25.421 INFO:teuthology.orchestra.run.smithi183.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-06-03T03:07:25.421 INFO:teuthology.orchestra.run.smithi183.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-06-03T03:07:25.421 INFO:teuthology.orchestra.run.smithi183.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-06-03T03:07:25.421 INFO:teuthology.orchestra.run.smithi183.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-06-03T03:07:25.421 INFO:teuthology.orchestra.run.smithi183.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-06-03T03:07:25.422 INFO:teuthology.orchestra.run.smithi183.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-06-03T03:07:25.422 INFO:teuthology.orchestra.run.smithi183.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-06-03T03:07:25.422 INFO:teuthology.orchestra.run.smithi183.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-06-03T03:07:25.422 INFO:teuthology.orchestra.run.smithi183.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-06-03T03:07:25.422 INFO:teuthology.orchestra.run.smithi183.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-06-03T03:07:25.422 INFO:teuthology.orchestra.run.smithi183.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-06-03T03:07:25.422 INFO:teuthology.orchestra.run.smithi183.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-06-03T03:07:25.422 INFO:teuthology.orchestra.run.smithi183.stdout:Installing weak dependencies: 2024-06-03T03:07:25.422 INFO:teuthology.orchestra.run.smithi183.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-06-03T03:07:25.422 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-06-03T03:07:25.422 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction Summary 2024-06-03T03:07:25.422 INFO:teuthology.orchestra.run.smithi183.stdout:======================================================================================= 2024-06-03T03:07:25.422 INFO:teuthology.orchestra.run.smithi183.stdout:Install 148 Packages 2024-06-03T03:07:25.423 INFO:teuthology.orchestra.run.smithi183.stdout:Upgrade 11 Packages 2024-06-03T03:07:25.423 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-06-03T03:07:25.423 INFO:teuthology.orchestra.run.smithi183.stdout:Total download size: 225 M 2024-06-03T03:07:25.423 INFO:teuthology.orchestra.run.smithi183.stdout:Downloading Packages: 2024-06-03T03:07:25.851 INFO:teuthology.orchestra.run.smithi151.stdout:(5/159): ceph-common-19.0.0-2946.g3d89accf.el9. 14 MB/s | 21 MB 00:01 2024-06-03T03:07:26.010 INFO:teuthology.orchestra.run.smithi151.stdout:(6/159): ceph-mds-19.0.0-2946.g3d89accf.el9.x86 3.5 MB/s | 2.4 MB 00:00 2024-06-03T03:07:26.168 INFO:teuthology.orchestra.run.smithi151.stdout:(7/159): ceph-mon-19.0.0-2946.g3d89accf.el9.x86 15 MB/s | 4.7 MB 00:00 2024-06-03T03:07:26.378 INFO:teuthology.orchestra.run.smithi183.stdout:(1/159): ceph-19.0.0-2946.g3d89accf.el9.x86_64. 25 kB/s | 6.6 kB 00:00 2024-06-03T03:07:26.527 INFO:teuthology.orchestra.run.smithi151.stdout:(8/159): ceph-mgr-19.0.0-2946.g3d89accf.el9.x86 1.3 MB/s | 1.5 MB 00:01 2024-06-03T03:07:26.652 INFO:teuthology.orchestra.run.smithi151.stdout:(9/159): ceph-selinux-19.0.0-2946.g3d89accf.el9 201 kB/s | 25 kB 00:00 2024-06-03T03:07:26.919 INFO:teuthology.orchestra.run.smithi151.stdout:(10/159): ceph-radosgw-19.0.0-2946.g3d89accf.el 14 MB/s | 10 MB 00:00 2024-06-03T03:07:27.002 INFO:teuthology.orchestra.run.smithi151.stdout:(11/159): cephfs-mirror-19.0.0-2946.g3d89accf.e 2.6 MB/s | 221 kB 00:00 2024-06-03T03:07:27.078 INFO:teuthology.orchestra.run.smithi151.stdout:(12/159): cephfs-shell-19.0.0-2946.g3d89accf.el 361 kB/s | 27 kB 00:00 2024-06-03T03:07:27.145 INFO:teuthology.orchestra.run.smithi151.stdout:(13/159): libcephfs-devel-19.0.0-2946.g3d89accf 476 kB/s | 32 kB 00:00 2024-06-03T03:07:27.253 INFO:teuthology.orchestra.run.smithi151.stdout:(14/159): libcephfs2-19.0.0-2946.g3d89accf.el9. 6.7 MB/s | 747 kB 00:00 2024-06-03T03:07:27.312 INFO:teuthology.orchestra.run.smithi183.stdout:(2/159): ceph-fuse-19.0.0-2946.g3d89accf.el9.x8 953 kB/s | 890 kB 00:00 2024-06-03T03:07:27.328 INFO:teuthology.orchestra.run.smithi151.stdout:(15/159): libcephsqlite-19.0.0-2946.g3d89accf.e 2.1 MB/s | 163 kB 00:00 2024-06-03T03:07:27.412 INFO:teuthology.orchestra.run.smithi151.stdout:(16/159): librados-devel-19.0.0-2946.g3d89accf. 1.5 MB/s | 127 kB 00:00 2024-06-03T03:07:27.420 INFO:teuthology.orchestra.run.smithi183.stdout:(3/159): ceph-immutable-object-cache-19.0.0-294 1.3 MB/s | 145 kB 00:00 2024-06-03T03:07:27.504 INFO:teuthology.orchestra.run.smithi151.stdout:(17/159): libradosstriper1-19.0.0-2946.g3d89acc 5.3 MB/s | 501 kB 00:00 2024-06-03T03:07:27.576 INFO:teuthology.orchestra.run.smithi117.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-03T03:07:27.578 INFO:teuthology.orchestra.run.smithi117.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-03T03:07:27.599 INFO:teuthology.orchestra.run.smithi117.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-06-03T03:07:27.600 INFO:teuthology.orchestra.run.smithi117.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-06-03T03:07:27.601 INFO:teuthology.orchestra.run.smithi117.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-06-03T03:07:27.603 INFO:teuthology.orchestra.run.smithi117.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-06-03T03:07:27.605 INFO:teuthology.orchestra.run.smithi117.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-06-03T03:07:27.609 INFO:teuthology.orchestra.run.smithi117.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-06-03T03:07:27.610 INFO:teuthology.orchestra.run.smithi117.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-06-03T03:07:27.683 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-06-03T03:07:27.701 INFO:teuthology.orchestra.run.smithi117.stdout:======================================================================================= 2024-06-03T03:07:27.701 INFO:teuthology.orchestra.run.smithi117.stdout: Package Arch Version Repository Size 2024-06-03T03:07:27.701 INFO:teuthology.orchestra.run.smithi117.stdout:======================================================================================= 2024-06-03T03:07:27.701 INFO:teuthology.orchestra.run.smithi117.stdout:Installing: 2024-06-03T03:07:27.701 INFO:teuthology.orchestra.run.smithi117.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-06-03T03:07:27.702 INFO:teuthology.orchestra.run.smithi117.stdout: ceph x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 6.6 k 2024-06-03T03:07:27.702 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-base x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.4 M 2024-06-03T03:07:27.702 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 890 k 2024-06-03T03:07:27.702 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 145 k 2024-06-03T03:07:27.702 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 1.5 M 2024-06-03T03:07:27.702 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 142 k 2024-06-03T03:07:27.702 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 3.5 M 2024-06-03T03:07:27.702 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 7.4 M 2024-06-03T03:07:27.702 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-rook noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 49 k 2024-06-03T03:07:27.702 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-radosgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 10 M 2024-06-03T03:07:27.702 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-test x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 48 M 2024-06-03T03:07:27.702 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-volume noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 264 k 2024-06-03T03:07:27.703 INFO:teuthology.orchestra.run.smithi117.stdout: cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 763 k 2024-06-03T03:07:27.703 INFO:teuthology.orchestra.run.smithi117.stdout: cephfs-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 221 k 2024-06-03T03:07:27.703 INFO:teuthology.orchestra.run.smithi117.stdout: cephfs-shell x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 27 k 2024-06-03T03:07:27.703 INFO:teuthology.orchestra.run.smithi117.stdout: cephfs-top noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 18 k 2024-06-03T03:07:27.703 INFO:teuthology.orchestra.run.smithi117.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-06-03T03:07:27.703 INFO:teuthology.orchestra.run.smithi117.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-06-03T03:07:27.703 INFO:teuthology.orchestra.run.smithi117.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-06-03T03:07:27.703 INFO:teuthology.orchestra.run.smithi117.stdout: libcephfs-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 32 k 2024-06-03T03:07:27.703 INFO:teuthology.orchestra.run.smithi117.stdout: libcephfs2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 747 k 2024-06-03T03:07:27.703 INFO:teuthology.orchestra.run.smithi117.stdout: librados-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 127 k 2024-06-03T03:07:27.703 INFO:teuthology.orchestra.run.smithi117.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-06-03T03:07:27.703 INFO:teuthology.orchestra.run.smithi117.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-06-03T03:07:27.703 INFO:teuthology.orchestra.run.smithi117.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-06-03T03:07:27.704 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-06-03T03:07:27.704 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-06-03T03:07:27.704 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-06-03T03:07:27.704 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cephfs x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 162 k 2024-06-03T03:07:27.704 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rados x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 321 k 2024-06-03T03:07:27.704 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 300 k 2024-06-03T03:07:27.704 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 99 k 2024-06-03T03:07:27.704 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 85 k 2024-06-03T03:07:27.704 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-03T03:07:27.704 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-nbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 171 k 2024-06-03T03:07:27.704 INFO:teuthology.orchestra.run.smithi117.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-06-03T03:07:27.704 INFO:teuthology.orchestra.run.smithi117.stdout:Upgrading: 2024-06-03T03:07:27.705 INFO:teuthology.orchestra.run.smithi117.stdout: NetworkManager x86_64 1:1.47.91-1.el9 baseos 2.3 M 2024-06-03T03:07:27.705 INFO:teuthology.orchestra.run.smithi117.stdout: NetworkManager-libnm x86_64 1:1.47.91-1.el9 baseos 1.8 M 2024-06-03T03:07:27.705 INFO:teuthology.orchestra.run.smithi117.stdout: NetworkManager-team x86_64 1:1.47.91-1.el9 baseos 41 k 2024-06-03T03:07:27.705 INFO:teuthology.orchestra.run.smithi117.stdout: NetworkManager-tui x86_64 1:1.47.91-1.el9 baseos 248 k 2024-06-03T03:07:27.705 INFO:teuthology.orchestra.run.smithi117.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-06-03T03:07:27.705 INFO:teuthology.orchestra.run.smithi117.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-06-03T03:07:27.705 INFO:teuthology.orchestra.run.smithi117.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-06-03T03:07:27.705 INFO:teuthology.orchestra.run.smithi117.stdout: librados2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.4 M 2024-06-03T03:07:27.705 INFO:teuthology.orchestra.run.smithi117.stdout: librbd1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-03T03:07:27.705 INFO:teuthology.orchestra.run.smithi117.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-06-03T03:07:27.705 INFO:teuthology.orchestra.run.smithi117.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-06-03T03:07:27.705 INFO:teuthology.orchestra.run.smithi117.stdout:Installing dependencies: 2024-06-03T03:07:27.705 INFO:teuthology.orchestra.run.smithi117.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-06-03T03:07:27.705 INFO:teuthology.orchestra.run.smithi117.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-06-03T03:07:27.706 INFO:teuthology.orchestra.run.smithi117.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-06-03T03:07:27.706 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 21 M 2024-06-03T03:07:27.706 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 32 k 2024-06-03T03:07:27.706 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mds x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 2.4 M 2024-06-03T03:07:27.706 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 249 k 2024-06-03T03:07:27.706 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mon x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 4.7 M 2024-06-03T03:07:27.706 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-osd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 17 M 2024-06-03T03:07:27.706 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 16 k 2024-06-03T03:07:27.706 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-selinux x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 25 k 2024-06-03T03:07:27.706 INFO:teuthology.orchestra.run.smithi117.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-06-03T03:07:27.706 INFO:teuthology.orchestra.run.smithi117.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-06-03T03:07:27.706 INFO:teuthology.orchestra.run.smithi117.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-06-03T03:07:27.707 INFO:teuthology.orchestra.run.smithi117.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-06-03T03:07:27.707 INFO:teuthology.orchestra.run.smithi117.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-06-03T03:07:27.707 INFO:teuthology.orchestra.run.smithi117.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-06-03T03:07:27.707 INFO:teuthology.orchestra.run.smithi117.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-06-03T03:07:27.707 INFO:teuthology.orchestra.run.smithi117.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-06-03T03:07:27.707 INFO:teuthology.orchestra.run.smithi117.stdout: libcephsqlite x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 163 k 2024-06-03T03:07:27.707 INFO:teuthology.orchestra.run.smithi117.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-06-03T03:07:27.707 INFO:teuthology.orchestra.run.smithi117.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-06-03T03:07:27.707 INFO:teuthology.orchestra.run.smithi117.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-06-03T03:07:27.707 INFO:teuthology.orchestra.run.smithi117.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-06-03T03:07:27.707 INFO:teuthology.orchestra.run.smithi117.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-06-03T03:07:27.707 INFO:teuthology.orchestra.run.smithi117.stdout: libradosstriper1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 501 k 2024-06-03T03:07:27.707 INFO:teuthology.orchestra.run.smithi117.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-06-03T03:07:27.708 INFO:teuthology.orchestra.run.smithi117.stdout: librgw2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.3 M 2024-06-03T03:07:27.708 INFO:teuthology.orchestra.run.smithi117.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-06-03T03:07:27.708 INFO:teuthology.orchestra.run.smithi117.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-06-03T03:07:27.708 INFO:teuthology.orchestra.run.smithi117.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-06-03T03:07:27.708 INFO:teuthology.orchestra.run.smithi117.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-06-03T03:07:27.708 INFO:teuthology.orchestra.run.smithi117.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-06-03T03:07:27.708 INFO:teuthology.orchestra.run.smithi117.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-06-03T03:07:27.708 INFO:teuthology.orchestra.run.smithi117.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-06-03T03:07:27.708 INFO:teuthology.orchestra.run.smithi117.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-06-03T03:07:27.708 INFO:teuthology.orchestra.run.smithi117.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-06-03T03:07:27.708 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-06-03T03:07:27.708 INFO:teuthology.orchestra.run.smithi117.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-06-03T03:07:27.708 INFO:teuthology.orchestra.run.smithi117.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-06-03T03:07:27.708 INFO:teuthology.orchestra.run.smithi117.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-06-03T03:07:27.709 INFO:teuthology.orchestra.run.smithi117.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-06-03T03:07:27.709 INFO:teuthology.orchestra.run.smithi117.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-06-03T03:07:27.709 INFO:teuthology.orchestra.run.smithi117.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-06-03T03:07:27.709 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-06-03T03:07:27.709 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ceph-argparse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 45 k 2024-06-03T03:07:27.709 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 135 k 2024-06-03T03:07:27.709 INFO:teuthology.orchestra.run.smithi117.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-06-03T03:07:27.709 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-06-03T03:07:27.709 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-06-03T03:07:27.709 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-06-03T03:07:27.709 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-06-03T03:07:27.709 INFO:teuthology.orchestra.run.smithi117.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-06-03T03:07:27.709 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-06-03T03:07:27.710 INFO:teuthology.orchestra.run.smithi117.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-06-03T03:07:27.710 INFO:teuthology.orchestra.run.smithi117.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-06-03T03:07:27.710 INFO:teuthology.orchestra.run.smithi117.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-06-03T03:07:27.710 INFO:teuthology.orchestra.run.smithi117.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-06-03T03:07:27.710 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-06-03T03:07:27.710 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-06-03T03:07:27.710 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-06-03T03:07:27.710 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-06-03T03:07:27.710 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-06-03T03:07:27.710 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-06-03T03:07:27.710 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-06-03T03:07:27.710 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-06-03T03:07:27.711 INFO:teuthology.orchestra.run.smithi117.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-06-03T03:07:27.711 INFO:teuthology.orchestra.run.smithi117.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-06-03T03:07:27.711 INFO:teuthology.orchestra.run.smithi117.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-06-03T03:07:27.711 INFO:teuthology.orchestra.run.smithi117.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-06-03T03:07:27.711 INFO:teuthology.orchestra.run.smithi117.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-06-03T03:07:27.711 INFO:teuthology.orchestra.run.smithi117.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-06-03T03:07:27.711 INFO:teuthology.orchestra.run.smithi117.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-06-03T03:07:27.711 INFO:teuthology.orchestra.run.smithi117.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-06-03T03:07:27.711 INFO:teuthology.orchestra.run.smithi117.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-06-03T03:07:27.711 INFO:teuthology.orchestra.run.smithi117.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-06-03T03:07:27.711 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-06-03T03:07:27.711 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-06-03T03:07:27.711 INFO:teuthology.orchestra.run.smithi117.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-06-03T03:07:27.711 INFO:teuthology.orchestra.run.smithi117.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-06-03T03:07:27.712 INFO:teuthology.orchestra.run.smithi117.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-06-03T03:07:27.712 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-06-03T03:07:27.712 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-06-03T03:07:27.712 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-06-03T03:07:27.712 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-06-03T03:07:27.712 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-06-03T03:07:27.712 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-06-03T03:07:27.712 INFO:teuthology.orchestra.run.smithi117.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-06-03T03:07:27.712 INFO:teuthology.orchestra.run.smithi117.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-06-03T03:07:27.712 INFO:teuthology.orchestra.run.smithi117.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-06-03T03:07:27.712 INFO:teuthology.orchestra.run.smithi117.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-06-03T03:07:27.712 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-06-03T03:07:27.712 INFO:teuthology.orchestra.run.smithi117.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-06-03T03:07:27.713 INFO:teuthology.orchestra.run.smithi117.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-06-03T03:07:27.713 INFO:teuthology.orchestra.run.smithi117.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-06-03T03:07:27.713 INFO:teuthology.orchestra.run.smithi117.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-06-03T03:07:27.713 INFO:teuthology.orchestra.run.smithi117.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-06-03T03:07:27.713 INFO:teuthology.orchestra.run.smithi117.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-06-03T03:07:27.713 INFO:teuthology.orchestra.run.smithi117.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-06-03T03:07:27.713 INFO:teuthology.orchestra.run.smithi117.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-06-03T03:07:27.713 INFO:teuthology.orchestra.run.smithi117.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-06-03T03:07:27.713 INFO:teuthology.orchestra.run.smithi117.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-06-03T03:07:27.713 INFO:teuthology.orchestra.run.smithi117.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-06-03T03:07:27.713 INFO:teuthology.orchestra.run.smithi117.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-06-03T03:07:27.713 INFO:teuthology.orchestra.run.smithi117.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-06-03T03:07:27.713 INFO:teuthology.orchestra.run.smithi117.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-06-03T03:07:27.713 INFO:teuthology.orchestra.run.smithi117.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-06-03T03:07:27.714 INFO:teuthology.orchestra.run.smithi117.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-06-03T03:07:27.714 INFO:teuthology.orchestra.run.smithi117.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-06-03T03:07:27.714 INFO:teuthology.orchestra.run.smithi117.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-06-03T03:07:27.714 INFO:teuthology.orchestra.run.smithi117.stdout:Installing weak dependencies: 2024-06-03T03:07:27.714 INFO:teuthology.orchestra.run.smithi117.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-06-03T03:07:27.714 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T03:07:27.714 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2024-06-03T03:07:27.714 INFO:teuthology.orchestra.run.smithi117.stdout:======================================================================================= 2024-06-03T03:07:27.714 INFO:teuthology.orchestra.run.smithi117.stdout:Install 148 Packages 2024-06-03T03:07:27.714 INFO:teuthology.orchestra.run.smithi117.stdout:Upgrade 11 Packages 2024-06-03T03:07:27.714 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T03:07:27.715 INFO:teuthology.orchestra.run.smithi117.stdout:Total download size: 225 M 2024-06-03T03:07:27.715 INFO:teuthology.orchestra.run.smithi117.stdout:Downloading Packages: 2024-06-03T03:07:27.879 INFO:teuthology.orchestra.run.smithi183.stdout:(4/159): ceph-base-19.0.0-2946.g3d89accf.el9.x8 3.0 MB/s | 5.4 MB 00:01 2024-06-03T03:07:28.071 INFO:teuthology.orchestra.run.smithi151.stdout:(18/159): ceph-osd-19.0.0-2946.g3d89accf.el9.x8 8.1 MB/s | 17 MB 00:02 2024-06-03T03:07:28.147 INFO:teuthology.orchestra.run.smithi151.stdout:(19/159): librgw2-19.0.0-2946.g3d89accf.el9.x86 8.3 MB/s | 5.3 MB 00:00 2024-06-03T03:07:28.172 INFO:teuthology.orchestra.run.smithi151.stdout:(20/159): python3-ceph-argparse-19.0.0-2946.g3d 451 kB/s | 45 kB 00:00 2024-06-03T03:07:28.187 INFO:teuthology.orchestra.run.smithi183.stdout:(5/159): ceph-mgr-19.0.0-2946.g3d89accf.el9.x86 4.8 MB/s | 1.5 MB 00:00 2024-06-03T03:07:28.197 INFO:teuthology.orchestra.run.smithi151.stdout:(21/159): python3-ceph-common-19.0.0-2946.g3d89 2.6 MB/s | 135 kB 00:00 2024-06-03T03:07:28.222 INFO:teuthology.orchestra.run.smithi151.stdout:(22/159): python3-cephfs-19.0.0-2946.g3d89accf. 3.2 MB/s | 162 kB 00:00 2024-06-03T03:07:28.256 INFO:teuthology.orchestra.run.smithi151.stdout:(23/159): python3-rados-19.0.0-2946.g3d89accf.e 5.4 MB/s | 321 kB 00:00 2024-06-03T03:07:28.298 INFO:teuthology.orchestra.run.smithi151.stdout:(24/159): python3-rbd-19.0.0-2946.g3d89accf.el9 3.9 MB/s | 300 kB 00:00 2024-06-03T03:07:28.321 INFO:teuthology.orchestra.run.smithi183.stdout:(6/159): ceph-mds-19.0.0-2946.g3d89accf.el9.x86 2.6 MB/s | 2.4 MB 00:00 2024-06-03T03:07:28.323 INFO:teuthology.orchestra.run.smithi151.stdout:(25/159): python3-rgw-19.0.0-2946.g3d89accf.el9 1.5 MB/s | 99 kB 00:00 2024-06-03T03:07:28.348 INFO:teuthology.orchestra.run.smithi151.stdout:(26/159): rbd-fuse-19.0.0-2946.g3d89accf.el9.x8 1.7 MB/s | 85 kB 00:00 2024-06-03T03:07:28.432 INFO:teuthology.orchestra.run.smithi151.stdout:(27/159): rbd-nbd-19.0.0-2946.g3d89accf.el9.x86 2.0 MB/s | 171 kB 00:00 2024-06-03T03:07:28.532 INFO:teuthology.orchestra.run.smithi151.stdout:(28/159): ceph-grafana-dashboards-19.0.0-2946.g 317 kB/s | 32 kB 00:00 2024-06-03T03:07:28.616 INFO:teuthology.orchestra.run.smithi151.stdout:(29/159): ceph-mgr-cephadm-19.0.0-2946.g3d89acc 1.7 MB/s | 142 kB 00:00 2024-06-03T03:07:28.675 INFO:teuthology.orchestra.run.smithi151.stdout:(30/159): rbd-mirror-19.0.0-2946.g3d89accf.el9. 8.8 MB/s | 3.1 MB 00:00 2024-06-03T03:07:29.092 INFO:teuthology.orchestra.run.smithi117.stdout:(1/159): ceph-19.0.0-2946.g3d89accf.el9.x86_64. 16 kB/s | 6.6 kB 00:00 2024-06-03T03:07:29.308 INFO:teuthology.orchestra.run.smithi151.stdout:(31/159): ceph-mgr-dashboard-19.0.0-2946.g3d89a 5.1 MB/s | 3.5 MB 00:00 2024-06-03T03:07:29.329 INFO:teuthology.orchestra.run.smithi183.stdout:(7/159): ceph-common-19.0.0-2946.g3d89accf.el9. 6.5 MB/s | 21 MB 00:03 2024-06-03T03:07:29.394 INFO:teuthology.orchestra.run.smithi151.stdout:(32/159): ceph-mgr-modules-core-19.0.0-2946.g3d 2.8 MB/s | 249 kB 00:00 2024-06-03T03:07:29.494 INFO:teuthology.orchestra.run.smithi151.stdout:(33/159): ceph-mgr-rook-19.0.0-2946.g3d89accf.e 494 kB/s | 49 kB 00:00 2024-06-03T03:07:29.578 INFO:teuthology.orchestra.run.smithi151.stdout:(34/159): ceph-prometheus-alerts-19.0.0-2946.g3 198 kB/s | 16 kB 00:00 2024-06-03T03:07:29.629 INFO:teuthology.orchestra.run.smithi183.stdout:(8/159): ceph-mon-19.0.0-2946.g3d89accf.el9.x86 3.3 MB/s | 4.7 MB 00:01 2024-06-03T03:07:29.644 INFO:teuthology.orchestra.run.smithi117.stdout:(2/159): ceph-fuse-19.0.0-2946.g3d89accf.el9.x8 1.6 MB/s | 890 kB 00:00 2024-06-03T03:07:29.661 INFO:teuthology.orchestra.run.smithi151.stdout:(35/159): ceph-volume-19.0.0-2946.g3d89accf.el9 3.1 MB/s | 264 kB 00:00 2024-06-03T03:07:29.680 INFO:teuthology.orchestra.run.smithi183.stdout:(9/159): ceph-selinux-19.0.0-2946.g3d89accf.el9 502 kB/s | 25 kB 00:00 2024-06-03T03:07:29.726 INFO:teuthology.orchestra.run.smithi117.stdout:(3/159): ceph-immutable-object-cache-19.0.0-294 1.8 MB/s | 145 kB 00:00 2024-06-03T03:07:30.045 INFO:teuthology.orchestra.run.smithi151.stdout:(36/159): ceph-mgr-diskprediction-local-19.0.0- 5.4 MB/s | 7.4 MB 00:01 2024-06-03T03:07:30.051 INFO:teuthology.orchestra.run.smithi117.stdout:(4/159): ceph-base-19.0.0-2946.g3d89accf.el9.x8 3.9 MB/s | 5.4 MB 00:01 2024-06-03T03:07:30.095 INFO:teuthology.orchestra.run.smithi151.stdout:(37/159): cephadm-19.0.0-2946.g3d89accf.el9.noa 1.7 MB/s | 763 kB 00:00 2024-06-03T03:07:30.120 INFO:teuthology.orchestra.run.smithi151.stdout:(38/159): cephfs-top-19.0.0-2946.g3d89accf.el9. 243 kB/s | 18 kB 00:00 2024-06-03T03:07:30.221 INFO:teuthology.orchestra.run.smithi151.stdout:(39/159): fuse-2.9.9-15.el9.x86_64.rpm 642 kB/s | 80 kB 00:00 2024-06-03T03:07:30.271 INFO:teuthology.orchestra.run.smithi151.stdout:(40/159): libquadmath-11.4.1-3.el9.x86_64.rpm 3.8 MB/s | 195 kB 00:00 2024-06-03T03:07:30.296 INFO:teuthology.orchestra.run.smithi151.stdout:(41/159): libgfortran-11.4.1-3.el9.x86_64.rpm 4.5 MB/s | 803 kB 00:00 2024-06-03T03:07:30.318 INFO:teuthology.orchestra.run.smithi117.stdout:(5/159): ceph-mgr-19.0.0-2946.g3d89accf.el9.x86 5.6 MB/s | 1.5 MB 00:00 2024-06-03T03:07:30.322 INFO:teuthology.orchestra.run.smithi151.stdout:(42/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.9 MB/s | 253 kB 00:00 2024-06-03T03:07:30.355 INFO:teuthology.orchestra.run.smithi151.stdout:(43/159): python3-cryptography-36.0.1-4.el9.x86 21 MB/s | 1.2 MB 00:00 2024-06-03T03:07:30.380 INFO:teuthology.orchestra.run.smithi151.stdout:(44/159): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-06-03T03:07:30.414 INFO:teuthology.orchestra.run.smithi151.stdout:(45/159): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-06-03T03:07:30.439 INFO:teuthology.orchestra.run.smithi151.stdout:(46/159): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-06-03T03:07:30.465 INFO:teuthology.orchestra.run.smithi151.stdout:(47/159): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-06-03T03:07:30.498 INFO:teuthology.orchestra.run.smithi151.stdout:(48/159): automake-1.16.2-8.el9.noarch.rpm 12 MB/s | 689 kB 00:00 2024-06-03T03:07:30.510 INFO:teuthology.orchestra.run.smithi117.stdout:(6/159): ceph-mds-19.0.0-2946.g3d89accf.el9.x86 3.0 MB/s | 2.4 MB 00:00 2024-06-03T03:07:30.532 INFO:teuthology.orchestra.run.smithi151.stdout:(49/159): bison-3.7.4-5.el9.x86_64.rpm 14 MB/s | 943 kB 00:00 2024-06-03T03:07:30.558 INFO:teuthology.orchestra.run.smithi151.stdout:(50/159): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-06-03T03:07:30.574 INFO:teuthology.orchestra.run.smithi151.stdout:(51/159): elfutils-libelf-devel-0.191-4.el9.x86 520 kB/s | 22 kB 00:00 2024-06-03T03:07:30.599 INFO:teuthology.orchestra.run.smithi151.stdout:(52/159): flex-2.6.4-9.el9.x86_64.rpm 7.3 MB/s | 313 kB 00:00 2024-06-03T03:07:30.616 INFO:teuthology.orchestra.run.smithi151.stdout:(53/159): flexiblas-3.0.4-8.el9.x86_64.rpm 781 kB/s | 33 kB 00:00 2024-06-03T03:07:30.650 INFO:teuthology.orchestra.run.smithi151.stdout:(54/159): flexiblas-openblas-openmp-3.0.4-8.el9 548 kB/s | 18 kB 00:00 2024-06-03T03:07:30.700 INFO:teuthology.orchestra.run.smithi151.stdout:(55/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 30 MB/s | 3.0 MB 00:00 2024-06-03T03:07:30.717 INFO:teuthology.orchestra.run.smithi151.stdout:(56/159): libaio-devel-0.3.111-13.el9.x86_64.rp 166 kB/s | 11 kB 00:00 2024-06-03T03:07:30.743 INFO:teuthology.orchestra.run.smithi151.stdout:(57/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.8 MB/s | 160 kB 00:00 2024-06-03T03:07:30.759 INFO:teuthology.orchestra.run.smithi151.stdout:(58/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-06-03T03:07:30.793 INFO:teuthology.orchestra.run.smithi151.stdout:(59/159): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 662 kB 00:00 2024-06-03T03:07:30.827 INFO:teuthology.orchestra.run.smithi151.stdout:(60/159): libtool-2.4.6-45.el9.x86_64.rpm 8.5 MB/s | 582 kB 00:00 2024-06-03T03:07:30.838 INFO:teuthology.orchestra.run.smithi183.stdout:(10/159): ceph-osd-19.0.0-2946.g3d89accf.el9.x8 6.7 MB/s | 17 MB 00:02 2024-06-03T03:07:30.843 INFO:teuthology.orchestra.run.smithi151.stdout:(61/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 507 kB/s | 25 kB 00:00 2024-06-03T03:07:30.869 INFO:teuthology.orchestra.run.smithi151.stdout:(62/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-06-03T03:07:30.894 INFO:teuthology.orchestra.run.smithi151.stdout:(63/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-06-03T03:07:30.897 INFO:teuthology.orchestra.run.smithi183.stdout:(11/159): cephfs-mirror-19.0.0-2946.g3d89accf.e 3.7 MB/s | 221 kB 00:00 2024-06-03T03:07:30.919 INFO:teuthology.orchestra.run.smithi151.stdout:(64/159): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-06-03T03:07:30.935 INFO:teuthology.orchestra.run.smithi117.stdout:(7/159): ceph-mon-19.0.0-2946.g3d89accf.el9.x86 7.6 MB/s | 4.7 MB 00:00 2024-06-03T03:07:30.936 INFO:teuthology.orchestra.run.smithi151.stdout:(65/159): openblas-0.3.21-2.el9.x86_64.rpm 839 kB/s | 35 kB 00:00 2024-06-03T03:07:30.939 INFO:teuthology.orchestra.run.smithi183.stdout:(12/159): cephfs-shell-19.0.0-2946.g3d89accf.el 647 kB/s | 27 kB 00:00 2024-06-03T03:07:31.014 INFO:teuthology.orchestra.run.smithi183.stdout:(13/159): libcephfs-devel-19.0.0-2946.g3d89accf 424 kB/s | 32 kB 00:00 2024-06-03T03:07:31.037 INFO:teuthology.orchestra.run.smithi151.stdout:(66/159): openblas-openmp-0.3.21-2.el9.x86_64.r 41 MB/s | 4.7 MB 00:00 2024-06-03T03:07:31.104 INFO:teuthology.orchestra.run.smithi151.stdout:(67/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 27 MB/s | 4.4 MB 00:00 2024-06-03T03:07:31.146 INFO:teuthology.orchestra.run.smithi151.stdout:(68/159): postgresql-13.14-1.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-06-03T03:07:31.148 INFO:teuthology.orchestra.run.smithi183.stdout:(14/159): libcephfs2-19.0.0-2946.g3d89accf.el9. 5.5 MB/s | 747 kB 00:00 2024-06-03T03:07:31.179 INFO:teuthology.orchestra.run.smithi151.stdout:(69/159): postgresql-contrib-13.14-1.el9.x86_64 12 MB/s | 911 kB 00:00 2024-06-03T03:07:31.204 INFO:teuthology.orchestra.run.smithi151.stdout:(70/159): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 133 kB 00:00 2024-06-03T03:07:31.223 INFO:teuthology.orchestra.run.smithi183.stdout:(15/159): libcephsqlite-19.0.0-2946.g3d89accf.e 2.1 MB/s | 163 kB 00:00 2024-06-03T03:07:31.255 INFO:teuthology.orchestra.run.smithi151.stdout:(71/159): protobuf-3.14.0-13.el9.x86_64.rpm 20 MB/s | 1.0 MB 00:00 2024-06-03T03:07:31.298 INFO:teuthology.orchestra.run.smithi183.stdout:(16/159): librados-devel-19.0.0-2946.g3d89accf. 1.6 MB/s | 127 kB 00:00 2024-06-03T03:07:31.330 INFO:teuthology.orchestra.run.smithi151.stdout:(72/159): postgresql-server-13.14-1.el9.x86_64. 38 MB/s | 5.8 MB 00:00 2024-06-03T03:07:31.356 INFO:teuthology.orchestra.run.smithi151.stdout:(73/159): python3-attrs-20.3.0-7.el9.noarch.rpm 873 kB/s | 87 kB 00:00 2024-06-03T03:07:31.398 INFO:teuthology.orchestra.run.smithi151.stdout:(74/159): python3-colorama-0.4.6-3.el9.noarch.r 1.4 MB/s | 61 kB 00:00 2024-06-03T03:07:31.432 INFO:teuthology.orchestra.run.smithi183.stdout:(17/159): libradosstriper1-19.0.0-2946.g3d89acc 3.7 MB/s | 501 kB 00:00 2024-06-03T03:07:31.481 INFO:teuthology.orchestra.run.smithi151.stdout:(75/159): python3-babel-2.9.1-2.el9.noarch.rpm 40 MB/s | 6.0 MB 00:00 2024-06-03T03:07:31.506 INFO:teuthology.orchestra.run.smithi151.stdout:(76/159): python3-devel-3.9.18-3.el9.x86_64.rpm 2.2 MB/s | 244 kB 00:00 2024-06-03T03:07:31.531 INFO:teuthology.orchestra.run.smithi151.stdout:(77/159): python3-jinja2-2.11.3-6.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-06-03T03:07:31.556 INFO:teuthology.orchestra.run.smithi151.stdout:(78/159): python3-jsonpatch-1.21-16.el9.noarch. 513 kB/s | 26 kB 00:00 2024-06-03T03:07:31.582 INFO:teuthology.orchestra.run.smithi151.stdout:(79/159): python3-jsonpointer-2.0-4.el9.noarch. 382 kB/s | 19 kB 00:00 2024-06-03T03:07:31.607 INFO:teuthology.orchestra.run.smithi151.stdout:(80/159): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-06-03T03:07:31.632 INFO:teuthology.orchestra.run.smithi151.stdout:(81/159): python3-markupsafe-1.1.1-12.el9.x86_6 693 kB/s | 35 kB 00:00 2024-06-03T03:07:31.674 INFO:teuthology.orchestra.run.smithi151.stdout:(82/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 5.9 MB/s | 250 kB 00:00 2024-06-03T03:07:31.741 INFO:teuthology.orchestra.run.smithi151.stdout:(83/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 38 MB/s | 5.1 MB 00:00 2024-06-03T03:07:31.767 INFO:teuthology.orchestra.run.smithi151.stdout:(84/159): python3-oauthlib-3.1.1-5.el9.noarch.r 2.4 MB/s | 222 kB 00:00 2024-06-03T03:07:31.792 INFO:teuthology.orchestra.run.smithi151.stdout:(85/159): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-06-03T03:07:31.817 INFO:teuthology.orchestra.run.smithi151.stdout:(86/159): python3-prettytable-0.7.2-27.el9.noar 844 kB/s | 42 kB 00:00 2024-06-03T03:07:31.842 INFO:teuthology.orchestra.run.smithi151.stdout:(87/159): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-06-03T03:07:31.868 INFO:teuthology.orchestra.run.smithi151.stdout:(88/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-06-03T03:07:31.893 INFO:teuthology.orchestra.run.smithi151.stdout:(89/159): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-06-03T03:07:31.918 INFO:teuthology.orchestra.run.smithi151.stdout:(90/159): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-06-03T03:07:31.944 INFO:teuthology.orchestra.run.smithi151.stdout:(91/159): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-06-03T03:07:31.985 INFO:teuthology.orchestra.run.smithi151.stdout:(92/159): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-06-03T03:07:32.028 INFO:teuthology.orchestra.run.smithi151.stdout:(93/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.0 MB/s | 44 kB 00:00 2024-06-03T03:07:32.069 INFO:teuthology.orchestra.run.smithi151.stdout:(94/159): qatlib-24.02.0-1.el9.x86_64.rpm 5.2 MB/s | 221 kB 00:00 2024-06-03T03:07:32.085 INFO:teuthology.orchestra.run.smithi117.stdout:(8/159): ceph-radosgw-19.0.0-2946.g3d89accf.el9 8.9 MB/s | 10 MB 00:01 2024-06-03T03:07:32.111 INFO:teuthology.orchestra.run.smithi151.stdout:(95/159): qatlib-service-24.02.0-1.el9.x86_64.r 865 kB/s | 36 kB 00:00 2024-06-03T03:07:32.124 INFO:teuthology.orchestra.run.smithi183.stdout:(18/159): librgw2-19.0.0-2946.g3d89accf.el9.x86 7.7 MB/s | 5.3 MB 00:00 2024-06-03T03:07:32.144 INFO:teuthology.orchestra.run.smithi117.stdout:(9/159): ceph-selinux-19.0.0-2946.g3d89accf.el9 438 kB/s | 25 kB 00:00 2024-06-03T03:07:32.174 INFO:teuthology.orchestra.run.smithi183.stdout:(19/159): python3-ceph-argparse-19.0.0-2946.g3d 904 kB/s | 45 kB 00:00 2024-06-03T03:07:32.249 INFO:teuthology.orchestra.run.smithi183.stdout:(20/159): python3-ceph-common-19.0.0-2946.g3d89 1.8 MB/s | 135 kB 00:00 2024-06-03T03:07:32.279 INFO:teuthology.orchestra.run.smithi151.stdout:(96/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 45 MB/s | 16 MB 00:00 2024-06-03T03:07:32.304 INFO:teuthology.orchestra.run.smithi151.stdout:(97/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 247 kB/s | 47 kB 00:00 2024-06-03T03:07:32.324 INFO:teuthology.orchestra.run.smithi183.stdout:(21/159): python3-cephfs-19.0.0-2946.g3d89accf. 2.1 MB/s | 162 kB 00:00 2024-06-03T03:07:32.329 INFO:teuthology.orchestra.run.smithi151.stdout:(98/159): socat-1.7.4.1-5.el9.x86_64.rpm 6.0 MB/s | 305 kB 00:00 2024-06-03T03:07:32.355 INFO:teuthology.orchestra.run.smithi151.stdout:(99/159): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-06-03T03:07:32.379 INFO:teuthology.orchestra.run.smithi151.stdout:(100/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.8 MB/s | 94 kB 00:00 2024-06-03T03:07:32.413 INFO:teuthology.orchestra.run.smithi151.stdout:(101/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-06-03T03:07:32.416 INFO:teuthology.orchestra.run.smithi183.stdout:(22/159): python3-rados-19.0.0-2946.g3d89accf.e 3.4 MB/s | 321 kB 00:00 2024-06-03T03:07:32.430 INFO:teuthology.orchestra.run.smithi151.stdout:(102/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 893 kB/s | 45 kB 00:00 2024-06-03T03:07:32.500 INFO:teuthology.orchestra.run.smithi183.stdout:(23/159): python3-rbd-19.0.0-2946.g3d89accf.el9 3.5 MB/s | 300 kB 00:00 2024-06-03T03:07:32.522 INFO:teuthology.orchestra.run.smithi151.stdout:(103/159): lua-devel-5.4.4-4.el9.x86_64.rpm 206 kB/s | 22 kB 00:00 2024-06-03T03:07:32.575 INFO:teuthology.orchestra.run.smithi183.stdout:(24/159): python3-rgw-19.0.0-2946.g3d89accf.el9 1.3 MB/s | 99 kB 00:00 2024-06-03T03:07:32.651 INFO:teuthology.orchestra.run.smithi183.stdout:(25/159): rbd-fuse-19.0.0-2946.g3d89accf.el9.x8 1.1 MB/s | 85 kB 00:00 2024-06-03T03:07:32.689 INFO:teuthology.orchestra.run.smithi151.stdout:(104/159): protobuf-compiler-3.14.0-13.el9.x86_ 3.3 MB/s | 863 kB 00:00 2024-06-03T03:07:32.989 INFO:teuthology.orchestra.run.smithi151.stdout:(105/159): abseil-cpp-20211102.0-3.el9.x86_64.r 1.1 MB/s | 548 kB 00:00 2024-06-03T03:07:33.056 INFO:teuthology.orchestra.run.smithi151.stdout:(106/159): grpc-data-1.46.7-10.el9.noarch.rpm 292 kB/s | 19 kB 00:00 2024-06-03T03:07:33.098 INFO:teuthology.orchestra.run.smithi151.stdout:(107/159): gperftools-libs-2.9.1-2.el9.x86_64.r 756 kB/s | 309 kB 00:00 2024-06-03T03:07:33.101 INFO:teuthology.orchestra.run.smithi183.stdout:(26/159): rbd-mirror-19.0.0-2946.g3d89accf.el9. 6.9 MB/s | 3.1 MB 00:00 2024-06-03T03:07:33.165 INFO:teuthology.orchestra.run.smithi151.stdout:(108/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 389 kB/s | 26 kB 00:00 2024-06-03T03:07:33.184 INFO:teuthology.orchestra.run.smithi183.stdout:(27/159): rbd-nbd-19.0.0-2946.g3d89accf.el9.x86 2.0 MB/s | 171 kB 00:00 2024-06-03T03:07:33.257 INFO:teuthology.orchestra.run.smithi151.stdout:(109/159): liboath-2.6.7-2.el9.x86_64.rpm 530 kB/s | 49 kB 00:00 2024-06-03T03:07:33.260 INFO:teuthology.orchestra.run.smithi183.stdout:(28/159): ceph-grafana-dashboards-19.0.0-2946.g 423 kB/s | 32 kB 00:00 2024-06-03T03:07:33.343 INFO:teuthology.orchestra.run.smithi183.stdout:(29/159): ceph-mgr-cephadm-19.0.0-2946.g3d89acc 1.7 MB/s | 142 kB 00:00 2024-06-03T03:07:33.365 INFO:teuthology.orchestra.run.smithi151.stdout:(110/159): libarrow-9.0.0-11.el9.x86_64.rpm 14 MB/s | 4.4 MB 00:00 2024-06-03T03:07:33.391 INFO:teuthology.orchestra.run.smithi151.stdout:(111/159): libunwind-1.6.2-1.el9.x86_64.rpm 505 kB/s | 67 kB 00:00 2024-06-03T03:07:33.441 INFO:teuthology.orchestra.run.smithi151.stdout:(112/159): luarocks-3.9.2-5.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-06-03T03:07:33.525 INFO:teuthology.orchestra.run.smithi151.stdout:(113/159): python3-asyncssh-2.13.2-5.el9.noarch 6.4 MB/s | 548 kB 00:00 2024-06-03T03:07:33.558 INFO:teuthology.orchestra.run.smithi151.stdout:(114/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.9 MB/s | 839 kB 00:00 2024-06-03T03:07:33.600 INFO:teuthology.orchestra.run.smithi151.stdout:(115/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 578 kB/s | 43 kB 00:00 2024-06-03T03:07:33.625 INFO:teuthology.orchestra.run.smithi151.stdout:(116/159): python3-cachetools-4.2.4-1.el9.noarc 481 kB/s | 32 kB 00:00 2024-06-03T03:07:33.627 INFO:teuthology.orchestra.run.smithi183.stdout:(30/159): ceph-radosgw-19.0.0-2946.g3d89accf.el 2.4 MB/s | 10 MB 00:04 2024-06-03T03:07:33.667 INFO:teuthology.orchestra.run.smithi151.stdout:(117/159): python3-certifi-2021.10.8-2.el9.noar 223 kB/s | 15 kB 00:00 2024-06-03T03:07:33.701 INFO:teuthology.orchestra.run.smithi151.stdout:(118/159): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2024-06-03T03:07:33.743 INFO:teuthology.orchestra.run.smithi151.stdout:(119/159): python3-cherrypy-18.6.1-2.el9.noarch 4.7 MB/s | 358 kB 00:00 2024-06-03T03:07:33.785 INFO:teuthology.orchestra.run.smithi151.stdout:(120/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.5 MB/s | 303 kB 00:00 2024-06-03T03:07:33.818 INFO:teuthology.orchestra.run.smithi151.stdout:(121/159): python3-google-auth-2.29.0-1.el9.noa 2.7 MB/s | 206 kB 00:00 2024-06-03T03:07:33.869 INFO:teuthology.orchestra.run.smithi183.stdout:(31/159): ceph-mgr-dashboard-19.0.0-2946.g3d89a 6.7 MB/s | 3.5 MB 00:00 2024-06-03T03:07:33.928 INFO:teuthology.orchestra.run.smithi151.stdout:(122/159): python3-grpcio-tools-1.46.7-10.el9.x 1.3 MB/s | 144 kB 00:00 2024-06-03T03:07:33.952 INFO:teuthology.orchestra.run.smithi183.stdout:(32/159): ceph-mgr-modules-core-19.0.0-2946.g3d 2.9 MB/s | 249 kB 00:00 2024-06-03T03:07:33.978 INFO:teuthology.orchestra.run.smithi151.stdout:(123/159): python3-grpcio-1.46.7-10.el9.x86_64. 11 MB/s | 2.0 MB 00:00 2024-06-03T03:07:34.003 INFO:teuthology.orchestra.run.smithi151.stdout:(124/159): python3-jaraco-8.2.1-3.el9.noarch.rp 142 kB/s | 11 kB 00:00 2024-06-03T03:07:34.028 INFO:teuthology.orchestra.run.smithi183.stdout:(33/159): ceph-mgr-rook-19.0.0-2946.g3d89accf.e 657 kB/s | 49 kB 00:00 2024-06-03T03:07:34.045 INFO:teuthology.orchestra.run.smithi151.stdout:(125/159): python3-jaraco-classes-3.2.1-5.el9.n 264 kB/s | 18 kB 00:00 2024-06-03T03:07:34.070 INFO:teuthology.orchestra.run.smithi151.stdout:(126/159): python3-jaraco-collections-3.0.0-8.e 347 kB/s | 23 kB 00:00 2024-06-03T03:07:34.129 INFO:teuthology.orchestra.run.smithi151.stdout:(127/159): python3-jaraco-functools-3.5.0-2.el9 232 kB/s | 19 kB 00:00 2024-06-03T03:07:34.154 INFO:teuthology.orchestra.run.smithi151.stdout:(128/159): python3-jaraco-text-3.2.0-6.el9.noar 251 kB/s | 20 kB 00:00 2024-06-03T03:07:34.161 INFO:teuthology.orchestra.run.smithi183.stdout:(34/159): ceph-prometheus-alerts-19.0.0-2946.g3 123 kB/s | 16 kB 00:00 2024-06-03T03:07:34.221 INFO:teuthology.orchestra.run.smithi151.stdout:(129/159): python3-kubernetes-26.1.0-2.el9.noar 11 MB/s | 1.0 MB 00:00 2024-06-03T03:07:34.245 INFO:teuthology.orchestra.run.smithi183.stdout:(35/159): ceph-volume-19.0.0-2946.g3d89accf.el9 3.1 MB/s | 264 kB 00:00 2024-06-03T03:07:34.246 INFO:teuthology.orchestra.run.smithi151.stdout:(130/159): python3-logutils-0.3.5-21.el9.noarch 503 kB/s | 46 kB 00:00 2024-06-03T03:07:34.297 INFO:teuthology.orchestra.run.smithi151.stdout:(131/159): python3-more-itertools-8.12.0-2.el9. 1.0 MB/s | 79 kB 00:00 2024-06-03T03:07:34.321 INFO:teuthology.orchestra.run.smithi151.stdout:(132/159): python3-natsort-7.1.1-5.el9.noarch.r 768 kB/s | 58 kB 00:00 2024-06-03T03:07:34.372 INFO:teuthology.orchestra.run.smithi151.stdout:(133/159): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-06-03T03:07:34.397 INFO:teuthology.orchestra.run.smithi151.stdout:(134/159): python3-portend-3.1.0-2.el9.noarch.r 218 kB/s | 16 kB 00:00 2024-06-03T03:07:34.403 INFO:teuthology.orchestra.run.smithi183.stdout:(36/159): cephadm-19.0.0-2946.g3d89accf.el9.noa 4.7 MB/s | 763 kB 00:00 2024-06-03T03:07:34.448 INFO:teuthology.orchestra.run.smithi151.stdout:(135/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-06-03T03:07:34.462 INFO:teuthology.orchestra.run.smithi183.stdout:(37/159): cephfs-top-19.0.0-2946.g3d89accf.el9. 313 kB/s | 18 kB 00:00 2024-06-03T03:07:34.473 INFO:teuthology.orchestra.run.smithi151.stdout:(136/159): python3-pyperclip-1.8.2-2.el9.noarch 375 kB/s | 28 kB 00:00 2024-06-03T03:07:34.515 INFO:teuthology.orchestra.run.smithi151.stdout:(137/159): python3-repoze-lru-0.7-16.el9.noarch 460 kB/s | 31 kB 00:00 2024-06-03T03:07:34.548 INFO:teuthology.orchestra.run.smithi151.stdout:(138/159): python3-routes-2.5.1-5.el9.noarch.rp 2.5 MB/s | 188 kB 00:00 2024-06-03T03:07:34.582 INFO:teuthology.orchestra.run.smithi151.stdout:(139/159): python3-rsa-4.9-2.el9.noarch.rpm 883 kB/s | 59 kB 00:00 2024-06-03T03:07:34.588 INFO:teuthology.orchestra.run.smithi183.stdout:(38/159): fuse-2.9.9-15.el9.x86_64.rpm 641 kB/s | 80 kB 00:00 2024-06-03T03:07:34.616 INFO:teuthology.orchestra.run.smithi151.stdout:(140/159): python3-tempora-5.0.0-2.el9.noarch.r 534 kB/s | 36 kB 00:00 2024-06-03T03:07:34.619 INFO:teuthology.orchestra.run.smithi117.stdout:(10/159): ceph-osd-19.0.0-2946.g3d89accf.el9.x8 4.1 MB/s | 17 MB 00:04 2024-06-03T03:07:34.658 INFO:teuthology.orchestra.run.smithi151.stdout:(141/159): python3-typing-extensions-4.4.0-2.el 678 kB/s | 51 kB 00:00 2024-06-03T03:07:34.691 INFO:teuthology.orchestra.run.smithi151.stdout:(142/159): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-06-03T03:07:34.705 INFO:teuthology.orchestra.run.smithi183.stdout:(39/159): libgfortran-11.4.1-3.el9.x86_64.rpm 6.7 MB/s | 803 kB 00:00 2024-06-03T03:07:34.733 INFO:teuthology.orchestra.run.smithi151.stdout:(143/159): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-06-03T03:07:34.736 INFO:teuthology.orchestra.run.smithi117.stdout:(11/159): cephfs-mirror-19.0.0-2946.g3d89accf.e 1.9 MB/s | 221 kB 00:00 2024-06-03T03:07:34.746 INFO:teuthology.orchestra.run.smithi183.stdout:(40/159): libquadmath-11.4.1-3.el9.x86_64.rpm 4.6 MB/s | 195 kB 00:00 2024-06-03T03:07:34.775 INFO:teuthology.orchestra.run.smithi151.stdout:(144/159): python3-werkzeug-2.0.3-3.el9.1.noarc 5.0 MB/s | 427 kB 00:00 2024-06-03T03:07:34.788 INFO:teuthology.orchestra.run.smithi183.stdout:(41/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 6.0 MB/s | 253 kB 00:00 2024-06-03T03:07:34.800 INFO:teuthology.orchestra.run.smithi151.stdout:(145/159): python3-zc-lockfile-2.0-10.el9.noarc 298 kB/s | 20 kB 00:00 2024-06-03T03:07:34.803 INFO:teuthology.orchestra.run.smithi117.stdout:(12/159): cephfs-shell-19.0.0-2946.g3d89accf.el 410 kB/s | 27 kB 00:00 2024-06-03T03:07:34.850 INFO:teuthology.orchestra.run.smithi151.stdout:(146/159): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-06-03T03:07:34.864 INFO:teuthology.orchestra.run.smithi183.stdout:(42/159): python3-cryptography-36.0.1-4.el9.x86 16 MB/s | 1.2 MB 00:00 2024-06-03T03:07:34.870 INFO:teuthology.orchestra.run.smithi117.stdout:(13/159): libcephfs-devel-19.0.0-2946.g3d89accf 485 kB/s | 32 kB 00:00 2024-06-03T03:07:34.901 INFO:teuthology.orchestra.run.smithi151.stdout:(147/159): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-06-03T03:07:34.906 INFO:teuthology.orchestra.run.smithi183.stdout:(43/159): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-06-03T03:07:34.948 INFO:teuthology.orchestra.run.smithi183.stdout:(44/159): python3-pycparser-2.20-6.el9.noarch.r 3.2 MB/s | 135 kB 00:00 2024-06-03T03:07:34.990 INFO:teuthology.orchestra.run.smithi183.stdout:(45/159): python3-requests-2.25.1-8.el9.noarch. 2.9 MB/s | 125 kB 00:00 2024-06-03T03:07:35.032 INFO:teuthology.orchestra.run.smithi183.stdout:(46/159): python3-urllib3-1.26.5-5.el9.noarch.r 5.0 MB/s | 215 kB 00:00 2024-06-03T03:07:35.120 INFO:teuthology.orchestra.run.smithi117.stdout:(14/159): libcephfs2-19.0.0-2946.g3d89accf.el9. 2.9 MB/s | 747 kB 00:00 2024-06-03T03:07:35.165 INFO:teuthology.orchestra.run.smithi183.stdout:(47/159): automake-1.16.2-8.el9.noarch.rpm 5.1 MB/s | 689 kB 00:00 2024-06-03T03:07:35.212 INFO:teuthology.orchestra.run.smithi117.stdout:(15/159): libcephsqlite-19.0.0-2946.g3d89accf.e 1.8 MB/s | 163 kB 00:00 2024-06-03T03:07:35.224 INFO:teuthology.orchestra.run.smithi183.stdout:(48/159): bison-3.7.4-5.el9.x86_64.rpm 16 MB/s | 943 kB 00:00 2024-06-03T03:07:35.266 INFO:teuthology.orchestra.run.smithi183.stdout:(49/159): boost-program-options-1.75.0-8.el9.x8 2.5 MB/s | 107 kB 00:00 2024-06-03T03:07:35.300 INFO:teuthology.orchestra.run.smithi183.stdout:(50/159): elfutils-libelf-devel-0.191-4.el9.x86 653 kB/s | 22 kB 00:00 2024-06-03T03:07:35.337 INFO:teuthology.orchestra.run.smithi117.stdout:(16/159): librados-devel-19.0.0-2946.g3d89accf. 1.0 MB/s | 127 kB 00:00 2024-06-03T03:07:35.350 INFO:teuthology.orchestra.run.smithi183.stdout:(51/159): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-06-03T03:07:35.392 INFO:teuthology.orchestra.run.smithi183.stdout:(52/159): flexiblas-3.0.4-8.el9.x86_64.rpm 788 kB/s | 33 kB 00:00 2024-06-03T03:07:35.475 INFO:teuthology.orchestra.run.smithi183.stdout:(53/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 36 MB/s | 3.0 MB 00:00 2024-06-03T03:07:35.509 INFO:teuthology.orchestra.run.smithi183.stdout:(54/159): flexiblas-openblas-openmp-3.0.4-8.el9 546 kB/s | 18 kB 00:00 2024-06-03T03:07:35.543 INFO:teuthology.orchestra.run.smithi183.stdout:(55/159): libaio-devel-0.3.111-13.el9.x86_64.rp 335 kB/s | 11 kB 00:00 2024-06-03T03:07:35.585 INFO:teuthology.orchestra.run.smithi183.stdout:(56/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.8 MB/s | 160 kB 00:00 2024-06-03T03:07:35.596 INFO:teuthology.orchestra.run.smithi117.stdout:(17/159): libradosstriper1-19.0.0-2946.g3d89acc 1.9 MB/s | 501 kB 00:00 2024-06-03T03:07:35.618 INFO:teuthology.orchestra.run.smithi183.stdout:(57/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.3 MB/s | 45 kB 00:00 2024-06-03T03:07:35.668 INFO:teuthology.orchestra.run.smithi151.stdout:(148/159): librbd1-19.0.0-2946.g3d89accf.el9.x8 4.1 MB/s | 3.1 MB 00:00 2024-06-03T03:07:35.668 INFO:teuthology.orchestra.run.smithi183.stdout:(58/159): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 662 kB 00:00 2024-06-03T03:07:35.710 INFO:teuthology.orchestra.run.smithi183.stdout:(59/159): libtool-2.4.6-45.el9.x86_64.rpm 14 MB/s | 582 kB 00:00 2024-06-03T03:07:35.744 INFO:teuthology.orchestra.run.smithi183.stdout:(60/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 764 kB/s | 25 kB 00:00 2024-06-03T03:07:35.778 INFO:teuthology.orchestra.run.smithi183.stdout:(61/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-06-03T03:07:35.810 INFO:teuthology.orchestra.run.smithi151.stdout:(149/159): librados2-19.0.0-2946.g3d89accf.el9. 3.5 MB/s | 3.4 MB 00:00 2024-06-03T03:07:35.820 INFO:teuthology.orchestra.run.smithi183.stdout:(62/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 6.9 MB/s | 292 kB 00:00 2024-06-03T03:07:35.860 INFO:teuthology.orchestra.run.smithi151.stdout:(150/159): NetworkManager-1.47.91-1.el9.x86_64. 12 MB/s | 2.3 MB 00:00 2024-06-03T03:07:35.862 INFO:teuthology.orchestra.run.smithi183.stdout:(63/159): lua-5.4.4-4.el9.x86_64.rpm 4.4 MB/s | 188 kB 00:00 2024-06-03T03:07:35.893 INFO:teuthology.orchestra.run.smithi151.stdout:(151/159): NetworkManager-team-1.47.91-1.el9.x8 1.2 MB/s | 41 kB 00:00 2024-06-03T03:07:35.895 INFO:teuthology.orchestra.run.smithi183.stdout:(64/159): openblas-0.3.21-2.el9.x86_64.rpm 1.0 MB/s | 35 kB 00:00 2024-06-03T03:07:35.936 INFO:teuthology.orchestra.run.smithi151.stdout:(152/159): NetworkManager-tui-1.47.91-1.el9.x86 5.8 MB/s | 248 kB 00:00 2024-06-03T03:07:35.969 INFO:teuthology.orchestra.run.smithi151.stdout:(153/159): elfutils-debuginfod-client-0.191-4.e 1.1 MB/s | 37 kB 00:00 2024-06-03T03:07:36.004 INFO:teuthology.orchestra.run.smithi183.stdout:(65/159): openblas-openmp-0.3.21-2.el9.x86_64.r 44 MB/s | 4.7 MB 00:00 2024-06-03T03:07:36.036 INFO:teuthology.orchestra.run.smithi151.stdout:(154/159): NetworkManager-libnm-1.47.91-1.el9.x 8.0 MB/s | 1.8 MB 00:00 2024-06-03T03:07:36.062 INFO:teuthology.orchestra.run.smithi151.stdout:(155/159): elfutils-libelf-0.191-4.el9.x86_64.r 2.2 MB/s | 207 kB 00:00 2024-06-03T03:07:36.087 INFO:teuthology.orchestra.run.smithi151.stdout:(156/159): elfutils-libs-0.191-4.el9.x86_64.rpm 5.0 MB/s | 260 kB 00:00 2024-06-03T03:07:36.121 INFO:teuthology.orchestra.run.smithi183.stdout:(66/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 38 MB/s | 4.4 MB 00:00 2024-06-03T03:07:36.129 INFO:teuthology.orchestra.run.smithi151.stdout:(157/159): openssl-3.2.1-1.el9.x86_64.rpm 20 MB/s | 1.3 MB 00:00 2024-06-03T03:07:36.180 INFO:teuthology.orchestra.run.smithi183.stdout:(67/159): postgresql-13.14-1.el9.x86_64.rpm 27 MB/s | 1.6 MB 00:00 2024-06-03T03:07:36.187 INFO:teuthology.orchestra.run.smithi151.stdout:(158/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 24 MB/s | 2.4 MB 00:00 2024-06-03T03:07:36.230 INFO:teuthology.orchestra.run.smithi183.stdout:(68/159): postgresql-contrib-13.14-1.el9.x86_64 18 MB/s | 911 kB 00:00 2024-06-03T03:07:36.272 INFO:teuthology.orchestra.run.smithi183.stdout:(69/159): postgresql-private-libs-13.14-1.el9.x 3.1 MB/s | 133 kB 00:00 2024-06-03T03:07:36.398 INFO:teuthology.orchestra.run.smithi183.stdout:(70/159): postgresql-server-13.14-1.el9.x86_64. 46 MB/s | 5.8 MB 00:00 2024-06-03T03:07:36.448 INFO:teuthology.orchestra.run.smithi183.stdout:(71/159): protobuf-3.14.0-13.el9.x86_64.rpm 20 MB/s | 1.0 MB 00:00 2024-06-03T03:07:36.481 INFO:teuthology.orchestra.run.smithi183.stdout:(72/159): python3-attrs-20.3.0-7.el9.noarch.rpm 2.6 MB/s | 87 kB 00:00 2024-06-03T03:07:36.598 INFO:teuthology.orchestra.run.smithi183.stdout:(73/159): ceph-mgr-diskprediction-local-19.0.0- 2.5 MB/s | 7.4 MB 00:02 2024-06-03T03:07:36.682 INFO:teuthology.orchestra.run.smithi183.stdout:(74/159): python3-babel-2.9.1-2.el9.noarch.rpm 30 MB/s | 6.0 MB 00:00 2024-06-03T03:07:36.724 INFO:teuthology.orchestra.run.smithi183.stdout:(75/159): python3-devel-3.9.18-3.el9.x86_64.rpm 5.8 MB/s | 244 kB 00:00 2024-06-03T03:07:36.741 INFO:teuthology.orchestra.run.smithi183.stdout:(76/159): python3-colorama-0.4.6-3.el9.noarch.r 431 kB/s | 61 kB 00:00 2024-06-03T03:07:36.774 INFO:teuthology.orchestra.run.smithi183.stdout:(77/159): python3-jinja2-2.11.3-6.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-06-03T03:07:36.791 INFO:teuthology.orchestra.run.smithi183.stdout:(78/159): python3-jsonpatch-1.21-16.el9.noarch. 511 kB/s | 26 kB 00:00 2024-06-03T03:07:36.817 INFO:teuthology.orchestra.run.smithi183.stdout:(79/159): python3-jsonpointer-2.0-4.el9.noarch. 457 kB/s | 19 kB 00:00 2024-06-03T03:07:36.850 INFO:teuthology.orchestra.run.smithi183.stdout:(80/159): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-06-03T03:07:36.875 INFO:teuthology.orchestra.run.smithi183.stdout:(81/159): python3-markupsafe-1.1.1-12.el9.x86_6 594 kB/s | 35 kB 00:00 2024-06-03T03:07:36.913 INFO:teuthology.orchestra.run.smithi117.stdout:(18/159): ceph-test-19.0.0-2946.g3d89accf.el9.x 10 MB/s | 48 MB 00:04 2024-06-03T03:07:36.917 INFO:teuthology.orchestra.run.smithi183.stdout:(82/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 5.9 MB/s | 250 kB 00:00 2024-06-03T03:07:36.960 INFO:teuthology.orchestra.run.smithi183.stdout:(83/159): python3-oauthlib-3.1.1-5.el9.noarch.r 5.2 MB/s | 222 kB 00:00 2024-06-03T03:07:36.963 INFO:teuthology.orchestra.run.smithi117.stdout:(19/159): python3-ceph-argparse-19.0.0-2946.g3d 926 kB/s | 45 kB 00:00 2024-06-03T03:07:37.035 INFO:teuthology.orchestra.run.smithi183.stdout:(84/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 28 MB/s | 5.1 MB 00:00 2024-06-03T03:07:37.060 INFO:teuthology.orchestra.run.smithi183.stdout:(85/159): python3-packaging-20.9-5.el9.noarch.r 772 kB/s | 77 kB 00:00 2024-06-03T03:07:37.085 INFO:teuthology.orchestra.run.smithi183.stdout:(86/159): python3-prettytable-0.7.2-27.el9.noar 846 kB/s | 42 kB 00:00 2024-06-03T03:07:37.105 INFO:teuthology.orchestra.run.smithi117.stdout:(20/159): librgw2-19.0.0-2946.g3d89accf.el9.x86 3.5 MB/s | 5.3 MB 00:01 2024-06-03T03:07:37.111 INFO:teuthology.orchestra.run.smithi183.stdout:(87/159): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-06-03T03:07:37.136 INFO:teuthology.orchestra.run.smithi183.stdout:(88/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-06-03T03:07:37.138 INFO:teuthology.orchestra.run.smithi117.stdout:(21/159): python3-ceph-common-19.0.0-2946.g3d89 775 kB/s | 135 kB 00:00 2024-06-03T03:07:37.161 INFO:teuthology.orchestra.run.smithi183.stdout:(89/159): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-06-03T03:07:37.181 INFO:teuthology.orchestra.run.smithi117.stdout:(22/159): python3-cephfs-19.0.0-2946.g3d89accf. 2.1 MB/s | 162 kB 00:00 2024-06-03T03:07:37.186 INFO:teuthology.orchestra.run.smithi183.stdout:(90/159): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-06-03T03:07:37.212 INFO:teuthology.orchestra.run.smithi183.stdout:(91/159): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-06-03T03:07:37.239 INFO:teuthology.orchestra.run.smithi117.stdout:(23/159): python3-rados-19.0.0-2946.g3d89accf.e 3.2 MB/s | 321 kB 00:00 2024-06-03T03:07:37.253 INFO:teuthology.orchestra.run.smithi183.stdout:(92/159): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-06-03T03:07:37.290 INFO:teuthology.orchestra.run.smithi117.stdout:(24/159): python3-rbd-19.0.0-2946.g3d89accf.el9 2.7 MB/s | 300 kB 00:00 2024-06-03T03:07:37.312 INFO:teuthology.orchestra.run.smithi183.stdout:(93/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 758 kB/s | 44 kB 00:00 2024-06-03T03:07:37.315 INFO:teuthology.orchestra.run.smithi117.stdout:(25/159): python3-rgw-19.0.0-2946.g3d89accf.el9 1.3 MB/s | 99 kB 00:00 2024-06-03T03:07:37.340 INFO:teuthology.orchestra.run.smithi117.stdout:(26/159): rbd-fuse-19.0.0-2946.g3d89accf.el9.x8 1.7 MB/s | 85 kB 00:00 2024-06-03T03:07:37.363 INFO:teuthology.orchestra.run.smithi183.stdout:(94/159): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-06-03T03:07:37.399 INFO:teuthology.orchestra.run.smithi117.stdout:(27/159): rbd-nbd-19.0.0-2946.g3d89accf.el9.x86 2.9 MB/s | 171 kB 00:00 2024-06-03T03:07:37.405 INFO:teuthology.orchestra.run.smithi183.stdout:(95/159): qatlib-service-24.02.0-1.el9.x86_64.r 866 kB/s | 36 kB 00:00 2024-06-03T03:07:37.449 INFO:teuthology.orchestra.run.smithi117.stdout:(28/159): ceph-grafana-dashboards-19.0.0-2946.g 648 kB/s | 32 kB 00:00 2024-06-03T03:07:37.531 INFO:teuthology.orchestra.run.smithi117.stdout:(29/159): ceph-mgr-cephadm-19.0.0-2946.g3d89acc 1.7 MB/s | 142 kB 00:00 2024-06-03T03:07:37.580 INFO:teuthology.orchestra.run.smithi183.stdout:(96/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 41 MB/s | 16 MB 00:00 2024-06-03T03:07:37.605 INFO:teuthology.orchestra.run.smithi183.stdout:(97/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 237 kB/s | 47 kB 00:00 2024-06-03T03:07:37.630 INFO:teuthology.orchestra.run.smithi183.stdout:(98/159): socat-1.7.4.1-5.el9.x86_64.rpm 6.0 MB/s | 305 kB 00:00 2024-06-03T03:07:37.656 INFO:teuthology.orchestra.run.smithi183.stdout:(99/159): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-06-03T03:07:37.681 INFO:teuthology.orchestra.run.smithi183.stdout:(100/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.8 MB/s | 94 kB 00:00 2024-06-03T03:07:37.707 INFO:teuthology.orchestra.run.smithi183.stdout:(101/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-06-03T03:07:37.732 INFO:teuthology.orchestra.run.smithi183.stdout:(102/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 895 kB/s | 45 kB 00:00 2024-06-03T03:07:37.783 INFO:teuthology.orchestra.run.smithi117.stdout:(30/159): ceph-common-19.0.0-2946.g3d89accf.el9 2.3 MB/s | 21 MB 00:09 2024-06-03T03:07:37.865 INFO:teuthology.orchestra.run.smithi183.stdout:(103/159): lua-devel-5.4.4-4.el9.x86_64.rpm 141 kB/s | 22 kB 00:00 2024-06-03T03:07:37.883 INFO:teuthology.orchestra.run.smithi117.stdout:(31/159): rbd-mirror-19.0.0-2946.g3d89accf.el9. 5.5 MB/s | 3.1 MB 00:00 2024-06-03T03:07:37.924 INFO:teuthology.orchestra.run.smithi183.stdout:(104/159): protobuf-compiler-3.14.0-13.el9.x86_ 4.4 MB/s | 863 kB 00:00 2024-06-03T03:07:37.942 INFO:teuthology.orchestra.run.smithi117.stdout:(32/159): ceph-mgr-modules-core-19.0.0-2946.g3d 4.2 MB/s | 249 kB 00:00 2024-06-03T03:07:37.984 INFO:teuthology.orchestra.run.smithi117.stdout:(33/159): ceph-mgr-rook-19.0.0-2946.g3d89accf.e 1.2 MB/s | 49 kB 00:00 2024-06-03T03:07:38.026 INFO:teuthology.orchestra.run.smithi117.stdout:(34/159): ceph-prometheus-alerts-19.0.0-2946.g3 404 kB/s | 16 kB 00:00 2024-06-03T03:07:38.084 INFO:teuthology.orchestra.run.smithi117.stdout:(35/159): ceph-volume-19.0.0-2946.g3d89accf.el9 4.5 MB/s | 264 kB 00:00 2024-06-03T03:07:38.158 INFO:teuthology.orchestra.run.smithi183.stdout:(105/159): abseil-cpp-20211102.0-3.el9.x86_64.r 1.8 MB/s | 548 kB 00:00 2024-06-03T03:07:38.183 INFO:teuthology.orchestra.run.smithi183.stdout:(106/159): gperftools-libs-2.9.1-2.el9.x86_64.r 1.2 MB/s | 309 kB 00:00 2024-06-03T03:07:38.201 INFO:teuthology.orchestra.run.smithi117.stdout:(36/159): cephadm-19.0.0-2946.g3d89accf.el9.noa 6.4 MB/s | 763 kB 00:00 2024-06-03T03:07:38.208 INFO:teuthology.orchestra.run.smithi183.stdout:(107/159): grpc-data-1.46.7-10.el9.noarch.rpm 387 kB/s | 19 kB 00:00 2024-06-03T03:07:38.267 INFO:teuthology.orchestra.run.smithi183.stdout:(108/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 447 kB/s | 26 kB 00:00 2024-06-03T03:07:38.285 INFO:teuthology.orchestra.run.smithi117.stdout:(37/159): ceph-mgr-dashboard-19.0.0-2946.g3d89a 4.7 MB/s | 3.5 MB 00:00 2024-06-03T03:07:38.302 INFO:teuthology.orchestra.run.smithi117.stdout:(38/159): cephfs-top-19.0.0-2946.g3d89accf.el9. 183 kB/s | 18 kB 00:00 2024-06-03T03:07:38.325 INFO:teuthology.orchestra.run.smithi183.stdout:(109/159): liboath-2.6.7-2.el9.x86_64.rpm 834 kB/s | 49 kB 00:00 2024-06-03T03:07:38.384 INFO:teuthology.orchestra.run.smithi183.stdout:(110/159): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-06-03T03:07:38.451 INFO:teuthology.orchestra.run.smithi183.stdout:(111/159): libarrow-9.0.0-11.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2024-06-03T03:07:38.468 INFO:teuthology.orchestra.run.smithi183.stdout:(112/159): luarocks-3.9.2-5.el9.noarch.rpm 1.8 MB/s | 151 kB 00:00 2024-06-03T03:07:38.486 INFO:teuthology.orchestra.run.smithi117.stdout:(39/159): fuse-2.9.9-15.el9.x86_64.rpm 403 kB/s | 80 kB 00:00 2024-06-03T03:07:38.527 INFO:teuthology.orchestra.run.smithi183.stdout:(113/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 11 MB/s | 839 kB 00:00 2024-06-03T03:07:38.552 INFO:teuthology.orchestra.run.smithi183.stdout:(114/159): python3-asyncssh-2.13.2-5.el9.noarch 6.4 MB/s | 548 kB 00:00 2024-06-03T03:07:38.578 INFO:teuthology.orchestra.run.smithi117.stdout:(40/159): libquadmath-11.4.1-3.el9.x86_64.rpm 2.1 MB/s | 195 kB 00:00 2024-06-03T03:07:38.593 INFO:teuthology.orchestra.run.smithi183.stdout:(115/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 651 kB/s | 43 kB 00:00 2024-06-03T03:07:38.619 INFO:teuthology.orchestra.run.smithi183.stdout:(116/159): python3-cachetools-4.2.4-1.el9.noarc 479 kB/s | 32 kB 00:00 2024-06-03T03:07:38.628 INFO:teuthology.orchestra.run.smithi117.stdout:(41/159): libgfortran-11.4.1-3.el9.x86_64.rpm 2.4 MB/s | 803 kB 00:00 2024-06-03T03:07:38.644 INFO:teuthology.orchestra.run.smithi183.stdout:(117/159): python3-certifi-2021.10.8-2.el9.noar 298 kB/s | 15 kB 00:00 2024-06-03T03:07:38.662 INFO:teuthology.orchestra.run.smithi117.stdout:(42/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.0 MB/s | 253 kB 00:00 2024-06-03T03:07:38.678 INFO:teuthology.orchestra.run.smithi183.stdout:(118/159): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-06-03T03:07:38.703 INFO:teuthology.orchestra.run.smithi183.stdout:(119/159): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-06-03T03:07:38.720 INFO:teuthology.orchestra.run.smithi117.stdout:(43/159): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-06-03T03:07:38.745 INFO:teuthology.orchestra.run.smithi183.stdout:(120/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.4 MB/s | 303 kB 00:00 2024-06-03T03:07:38.770 INFO:teuthology.orchestra.run.smithi183.stdout:(121/159): python3-google-auth-2.29.0-1.el9.noa 3.0 MB/s | 206 kB 00:00 2024-06-03T03:07:38.804 INFO:teuthology.orchestra.run.smithi117.stdout:(44/159): python3-pycparser-2.20-6.el9.noarch.r 1.6 MB/s | 135 kB 00:00 2024-06-03T03:07:38.829 INFO:teuthology.orchestra.run.smithi183.stdout:(122/159): python3-grpcio-tools-1.46.7-10.el9.x 2.4 MB/s | 144 kB 00:00 2024-06-03T03:07:38.845 INFO:teuthology.orchestra.run.smithi117.stdout:(45/159): python3-cryptography-36.0.1-4.el9.x86 5.7 MB/s | 1.2 MB 00:00 2024-06-03T03:07:38.879 INFO:teuthology.orchestra.run.smithi117.stdout:(46/159): python3-requests-2.25.1-8.el9.noarch. 1.6 MB/s | 125 kB 00:00 2024-06-03T03:07:38.896 INFO:teuthology.orchestra.run.smithi183.stdout:(123/159): python3-grpcio-1.46.7-10.el9.x86_64. 14 MB/s | 2.0 MB 00:00 2024-06-03T03:07:38.913 INFO:teuthology.orchestra.run.smithi117.stdout:(47/159): python3-urllib3-1.26.5-5.el9.noarch.r 3.2 MB/s | 215 kB 00:00 2024-06-03T03:07:38.921 INFO:teuthology.orchestra.run.smithi183.stdout:(124/159): python3-jaraco-8.2.1-3.el9.noarch.rp 116 kB/s | 11 kB 00:00 2024-06-03T03:07:38.947 INFO:teuthology.orchestra.run.smithi183.stdout:(125/159): python3-jaraco-classes-3.2.1-5.el9.n 352 kB/s | 18 kB 00:00 2024-06-03T03:07:38.980 INFO:teuthology.orchestra.run.smithi183.stdout:(126/159): python3-jaraco-collections-3.0.0-8.e 397 kB/s | 23 kB 00:00 2024-06-03T03:07:39.006 INFO:teuthology.orchestra.run.smithi183.stdout:(127/159): python3-jaraco-functools-3.5.0-2.el9 332 kB/s | 19 kB 00:00 2024-06-03T03:07:39.031 INFO:teuthology.orchestra.run.smithi183.stdout:(128/159): python3-jaraco-text-3.2.0-6.el9.noar 392 kB/s | 20 kB 00:00 2024-06-03T03:07:39.038 INFO:teuthology.orchestra.run.smithi117.stdout:(48/159): automake-1.16.2-8.el9.noarch.rpm 4.3 MB/s | 689 kB 00:00 2024-06-03T03:07:39.090 INFO:teuthology.orchestra.run.smithi183.stdout:(129/159): python3-kubernetes-26.1.0-2.el9.noar 12 MB/s | 1.0 MB 00:00 2024-06-03T03:07:39.115 INFO:teuthology.orchestra.run.smithi183.stdout:(130/159): python3-logutils-0.3.5-21.el9.noarch 554 kB/s | 46 kB 00:00 2024-06-03T03:07:39.122 INFO:teuthology.orchestra.run.smithi117.stdout:(49/159): bison-3.7.4-5.el9.x86_64.rpm 4.4 MB/s | 943 kB 00:00 2024-06-03T03:07:39.147 INFO:teuthology.orchestra.run.smithi117.stdout:(50/159): boost-program-options-1.75.0-8.el9.x8 993 kB/s | 107 kB 00:00 2024-06-03T03:07:39.148 INFO:teuthology.orchestra.run.smithi183.stdout:(131/159): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-06-03T03:07:39.172 INFO:teuthology.orchestra.run.smithi117.stdout:(51/159): elfutils-libelf-devel-0.191-4.el9.x86 439 kB/s | 22 kB 00:00 2024-06-03T03:07:39.174 INFO:teuthology.orchestra.run.smithi183.stdout:(132/159): python3-natsort-7.1.1-5.el9.noarch.r 983 kB/s | 58 kB 00:00 2024-06-03T03:07:39.206 INFO:teuthology.orchestra.run.smithi117.stdout:(52/159): flex-2.6.4-9.el9.x86_64.rpm 5.3 MB/s | 313 kB 00:00 2024-06-03T03:07:39.231 INFO:teuthology.orchestra.run.smithi117.stdout:(53/159): flexiblas-3.0.4-8.el9.x86_64.rpm 567 kB/s | 33 kB 00:00 2024-06-03T03:07:39.273 INFO:teuthology.orchestra.run.smithi117.stdout:(54/159): flexiblas-openblas-openmp-3.0.4-8.el9 447 kB/s | 18 kB 00:00 2024-06-03T03:07:39.307 INFO:teuthology.orchestra.run.smithi117.stdout:(55/159): libaio-devel-0.3.111-13.el9.x86_64.rp 344 kB/s | 11 kB 00:00 2024-06-03T03:07:39.333 INFO:teuthology.orchestra.run.smithi183.stdout:(133/159): python3-pecan-1.4.2-3.el9.noarch.rpm 1.4 MB/s | 272 kB 00:00 2024-06-03T03:07:39.349 INFO:teuthology.orchestra.run.smithi117.stdout:(56/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.8 MB/s | 160 kB 00:00 2024-06-03T03:07:39.358 INFO:teuthology.orchestra.run.smithi183.stdout:(134/159): python3-portend-3.1.0-2.el9.noarch.r 89 kB/s | 16 kB 00:00 2024-06-03T03:07:39.392 INFO:teuthology.orchestra.run.smithi183.stdout:(135/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-06-03T03:07:39.417 INFO:teuthology.orchestra.run.smithi183.stdout:(136/159): python3-pyperclip-1.8.2-2.el9.noarch 482 kB/s | 28 kB 00:00 2024-06-03T03:07:39.433 INFO:teuthology.orchestra.run.smithi117.stdout:(57/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 13 MB/s | 3.0 MB 00:00 2024-06-03T03:07:39.449 INFO:teuthology.orchestra.run.smithi117.stdout:(58/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 454 kB/s | 45 kB 00:00 2024-06-03T03:07:39.450 INFO:teuthology.orchestra.run.smithi183.stdout:(137/159): python3-repoze-lru-0.7-16.el9.noarch 527 kB/s | 31 kB 00:00 2024-06-03T03:07:39.476 INFO:teuthology.orchestra.run.smithi183.stdout:(138/159): python3-routes-2.5.1-5.el9.noarch.rp 3.1 MB/s | 188 kB 00:00 2024-06-03T03:07:39.500 INFO:teuthology.orchestra.run.smithi117.stdout:(59/159): librdkafka-1.6.1-102.el9.x86_64.rpm 9.8 MB/s | 662 kB 00:00 2024-06-03T03:07:39.501 INFO:teuthology.orchestra.run.smithi183.stdout:(139/159): python3-rsa-4.9-2.el9.noarch.rpm 1.1 MB/s | 59 kB 00:00 2024-06-03T03:07:39.535 INFO:teuthology.orchestra.run.smithi183.stdout:(140/159): python3-tempora-5.0.0-2.el9.noarch.r 611 kB/s | 36 kB 00:00 2024-06-03T03:07:39.542 INFO:teuthology.orchestra.run.smithi117.stdout:(60/159): libtool-2.4.6-45.el9.x86_64.rpm 6.2 MB/s | 582 kB 00:00 2024-06-03T03:07:39.559 INFO:teuthology.orchestra.run.smithi117.stdout:(61/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 441 kB/s | 25 kB 00:00 2024-06-03T03:07:39.576 INFO:teuthology.orchestra.run.smithi117.stdout:(62/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-06-03T03:07:39.602 INFO:teuthology.orchestra.run.smithi183.stdout:(141/159): python3-typing-extensions-4.4.0-2.el 509 kB/s | 51 kB 00:00 2024-06-03T03:07:39.627 INFO:teuthology.orchestra.run.smithi183.stdout:(142/159): python3-webob-1.8.7-6.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-06-03T03:07:39.652 INFO:teuthology.orchestra.run.smithi183.stdout:(143/159): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-06-03T03:07:39.686 INFO:teuthology.orchestra.run.smithi183.stdout:(144/159): python3-werkzeug-2.0.3-3.el9.1.noarc 7.1 MB/s | 427 kB 00:00 2024-06-03T03:07:39.711 INFO:teuthology.orchestra.run.smithi183.stdout:(145/159): python3-zc-lockfile-2.0-10.el9.noarc 341 kB/s | 20 kB 00:00 2024-06-03T03:07:39.727 INFO:teuthology.orchestra.run.smithi117.stdout:(63/159): ceph-mgr-diskprediction-local-19.0.0- 3.8 MB/s | 7.4 MB 00:01 2024-06-03T03:07:39.762 INFO:teuthology.orchestra.run.smithi117.stdout:(64/159): lua-5.4.4-4.el9.x86_64.rpm 1.0 MB/s | 188 kB 00:00 2024-06-03T03:07:39.770 INFO:teuthology.orchestra.run.smithi183.stdout:(146/159): re2-20211101-3.el9.x86_64.rpm 2.2 MB/s | 192 kB 00:00 2024-06-03T03:07:39.793 INFO:teuthology.orchestra.run.smithi117.stdout:(65/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.2 MB/s | 292 kB 00:00 2024-06-03T03:07:39.810 INFO:teuthology.orchestra.run.smithi117.stdout:(66/159): openblas-0.3.21-2.el9.x86_64.rpm 426 kB/s | 35 kB 00:00 2024-06-03T03:07:39.820 INFO:teuthology.orchestra.run.smithi183.stdout:(147/159): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-06-03T03:07:39.961 INFO:teuthology.orchestra.run.smithi117.stdout:(67/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 27 MB/s | 4.4 MB 00:00 2024-06-03T03:07:40.159 INFO:teuthology.orchestra.run.smithi117.stdout:(68/159): openblas-openmp-0.3.21-2.el9.x86_64.r 12 MB/s | 4.7 MB 00:00 2024-06-03T03:07:40.212 INFO:teuthology.orchestra.run.smithi117.stdout:(69/159): postgresql-13.14-1.el9.x86_64.rpm 4.0 MB/s | 1.6 MB 00:00 2024-06-03T03:07:40.253 INFO:teuthology.orchestra.run.smithi117.stdout:(70/159): postgresql-contrib-13.14-1.el9.x86_64 3.0 MB/s | 911 kB 00:00 2024-06-03T03:07:40.270 INFO:teuthology.orchestra.run.smithi183.stdout:(148/159): ceph-test-19.0.0-2946.g3d89accf.el9. 4.6 MB/s | 48 MB 00:10 2024-06-03T03:07:40.287 INFO:teuthology.orchestra.run.smithi117.stdout:(71/159): postgresql-private-libs-13.14-1.el9.x 1.0 MB/s | 133 kB 00:00 2024-06-03T03:07:40.337 INFO:teuthology.orchestra.run.smithi183.stdout:(149/159): librados2-19.0.0-2946.g3d89accf.el9. 6.0 MB/s | 3.4 MB 00:00 2024-06-03T03:07:40.355 INFO:teuthology.orchestra.run.smithi117.stdout:(72/159): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-06-03T03:07:40.404 INFO:teuthology.orchestra.run.smithi117.stdout:(73/159): protobuf-3.14.0-13.el9.x86_64.rpm 6.7 MB/s | 1.0 MB 00:00 2024-06-03T03:07:40.446 INFO:teuthology.orchestra.run.smithi117.stdout:(74/159): python3-colorama-0.4.6-3.el9.noarch.r 1.5 MB/s | 61 kB 00:00 2024-06-03T03:07:40.488 INFO:teuthology.orchestra.run.smithi183.stdout:(150/159): NetworkManager-1.47.91-1.el9.x86_64. 11 MB/s | 2.3 MB 00:00 2024-06-03T03:07:40.496 INFO:teuthology.orchestra.run.smithi117.stdout:(75/159): python3-devel-3.9.18-3.el9.x86_64.rpm 4.9 MB/s | 244 kB 00:00 2024-06-03T03:07:40.530 INFO:teuthology.orchestra.run.smithi183.stdout:(151/159): NetworkManager-libnm-1.47.91-1.el9.x 9.4 MB/s | 1.8 MB 00:00 2024-06-03T03:07:40.547 INFO:teuthology.orchestra.run.smithi117.stdout:(76/159): python3-jinja2-2.11.3-6.el9.noarch.rp 4.9 MB/s | 248 kB 00:00 2024-06-03T03:07:40.555 INFO:teuthology.orchestra.run.smithi183.stdout:(152/159): NetworkManager-team-1.47.91-1.el9.x8 621 kB/s | 41 kB 00:00 2024-06-03T03:07:40.580 INFO:teuthology.orchestra.run.smithi117.stdout:(77/159): python3-jsonpatch-1.21-16.el9.noarch. 788 kB/s | 26 kB 00:00 2024-06-03T03:07:40.581 INFO:teuthology.orchestra.run.smithi183.stdout:(153/159): NetworkManager-tui-1.47.91-1.el9.x86 4.8 MB/s | 248 kB 00:00 2024-06-03T03:07:40.605 INFO:teuthology.orchestra.run.smithi183.stdout:(154/159): elfutils-debuginfod-client-0.191-4.e 731 kB/s | 37 kB 00:00 2024-06-03T03:07:40.614 INFO:teuthology.orchestra.run.smithi117.stdout:(78/159): python3-jsonpointer-2.0-4.el9.noarch. 590 kB/s | 19 kB 00:00 2024-06-03T03:07:40.631 INFO:teuthology.orchestra.run.smithi183.stdout:(155/159): elfutils-libelf-0.191-4.el9.x86_64.r 4.0 MB/s | 207 kB 00:00 2024-06-03T03:07:40.656 INFO:teuthology.orchestra.run.smithi183.stdout:(156/159): elfutils-libs-0.191-4.el9.x86_64.rpm 5.1 MB/s | 260 kB 00:00 2024-06-03T03:07:40.681 INFO:teuthology.orchestra.run.smithi117.stdout:(79/159): python3-mako-1.1.4-6.el9.noarch.rpm 2.6 MB/s | 172 kB 00:00 2024-06-03T03:07:40.698 INFO:teuthology.orchestra.run.smithi183.stdout:(157/159): openssl-3.2.1-1.el9.x86_64.rpm 20 MB/s | 1.3 MB 00:00 2024-06-03T03:07:40.757 INFO:teuthology.orchestra.run.smithi183.stdout:(158/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 24 MB/s | 2.4 MB 00:00 2024-06-03T03:07:40.790 INFO:teuthology.orchestra.run.smithi117.stdout:(80/159): postgresql-server-13.14-1.el9.x86_64. 10 MB/s | 5.8 MB 00:00 2024-06-03T03:07:40.806 INFO:teuthology.orchestra.run.smithi117.stdout:(81/159): python3-markupsafe-1.1.1-12.el9.x86_6 279 kB/s | 35 kB 00:00 2024-06-03T03:07:40.857 INFO:teuthology.orchestra.run.smithi117.stdout:(82/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 5.0 MB/s | 250 kB 00:00 2024-06-03T03:07:40.949 INFO:teuthology.orchestra.run.smithi183.stdout:(159/159): librbd1-19.0.0-2946.g3d89accf.el9.x8 2.8 MB/s | 3.1 MB 00:01 2024-06-03T03:07:40.953 INFO:teuthology.orchestra.run.smithi183.stdout:-------------------------------------------------------------------------------- 2024-06-03T03:07:40.953 INFO:teuthology.orchestra.run.smithi183.stdout:Total 14 MB/s | 225 MB 00:15 2024-06-03T03:07:40.999 INFO:teuthology.orchestra.run.smithi117.stdout:(83/159): python3-babel-2.9.1-2.el9.noarch.rpm 9.3 MB/s | 6.0 MB 00:00 2024-06-03T03:07:41.032 INFO:teuthology.orchestra.run.smithi117.stdout:(84/159): python3-oauthlib-3.1.1-5.el9.noarch.r 1.2 MB/s | 222 kB 00:00 2024-06-03T03:07:41.058 INFO:teuthology.orchestra.run.smithi117.stdout:(85/159): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-06-03T03:07:41.083 INFO:teuthology.orchestra.run.smithi117.stdout:(86/159): python3-prettytable-0.7.2-27.el9.noar 856 kB/s | 42 kB 00:00 2024-06-03T03:07:41.117 INFO:teuthology.orchestra.run.smithi117.stdout:(87/159): python3-protobuf-3.14.0-13.el9.noarch 4.6 MB/s | 269 kB 00:00 2024-06-03T03:07:41.150 INFO:teuthology.orchestra.run.smithi117.stdout:(88/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 159 kB 00:00 2024-06-03T03:07:41.184 INFO:teuthology.orchestra.run.smithi117.stdout:(89/159): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 279 kB 00:00 2024-06-03T03:07:41.209 INFO:teuthology.orchestra.run.smithi117.stdout:(90/159): python3-pytz-2021.1-5.el9.noarch.rpm 884 kB/s | 51 kB 00:00 2024-06-03T03:07:41.243 INFO:teuthology.orchestra.run.smithi117.stdout:(91/159): python3-requests-oauthlib-1.3.0-12.el 923 kB/s | 54 kB 00:00 2024-06-03T03:07:41.302 INFO:teuthology.orchestra.run.smithi117.stdout:(92/159): python3-toml-0.10.2-6.el9.noarch.rpm 723 kB/s | 42 kB 00:00 2024-06-03T03:07:41.344 INFO:teuthology.orchestra.run.smithi117.stdout:(93/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.1 MB/s | 44 kB 00:00 2024-06-03T03:07:41.394 INFO:teuthology.orchestra.run.smithi117.stdout:(94/159): qatlib-24.02.0-1.el9.x86_64.rpm 4.4 MB/s | 221 kB 00:00 2024-06-03T03:07:41.502 INFO:teuthology.orchestra.run.smithi117.stdout:(95/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 7.2 MB/s | 5.1 MB 00:00 2024-06-03T03:07:41.527 INFO:teuthology.orchestra.run.smithi117.stdout:(96/159): qatlib-service-24.02.0-1.el9.x86_64.r 270 kB/s | 36 kB 00:00 2024-06-03T03:07:41.553 INFO:teuthology.orchestra.run.smithi117.stdout:(97/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 962 kB/s | 47 kB 00:00 2024-06-03T03:07:41.615 INFO:teuthology.orchestra.run.smithi117.stdout:(98/159): socat-1.7.4.1-5.el9.x86_64.rpm 3.4 MB/s | 305 kB 00:00 2024-06-03T03:07:41.645 INFO:teuthology.orchestra.run.smithi117.stdout:(99/159): uuid-1.6.2-55.el9.x86_64.rpm 634 kB/s | 58 kB 00:00 2024-06-03T03:07:41.670 INFO:teuthology.orchestra.run.smithi117.stdout:(100/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.7 MB/s | 94 kB 00:00 2024-06-03T03:07:41.696 INFO:teuthology.orchestra.run.smithi117.stdout:(101/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.3 MB/s | 64 kB 00:00 2024-06-03T03:07:41.721 INFO:teuthology.orchestra.run.smithi117.stdout:(102/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 907 kB/s | 45 kB 00:00 2024-06-03T03:07:41.780 INFO:teuthology.orchestra.run.smithi117.stdout:(103/159): lua-devel-5.4.4-4.el9.x86_64.rpm 269 kB/s | 22 kB 00:00 2024-06-03T03:07:42.088 INFO:teuthology.orchestra.run.smithi117.stdout:(104/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 19 MB/s | 16 MB 00:00 2024-06-03T03:07:42.130 INFO:teuthology.orchestra.run.smithi117.stdout:(105/159): protobuf-compiler-3.14.0-13.el9.x86_ 2.1 MB/s | 863 kB 00:00 2024-06-03T03:07:42.160 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction check 2024-06-03T03:07:42.316 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction check succeeded. 2024-06-03T03:07:42.316 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction test 2024-06-03T03:07:42.422 INFO:teuthology.orchestra.run.smithi117.stdout:(106/159): grpc-data-1.46.7-10.el9.noarch.rpm 67 kB/s | 19 kB 00:00 2024-06-03T03:07:42.472 INFO:teuthology.orchestra.run.smithi117.stdout:(107/159): abseil-cpp-20211102.0-3.el9.x86_64.r 793 kB/s | 548 kB 00:00 2024-06-03T03:07:42.573 INFO:teuthology.orchestra.run.smithi117.stdout:(108/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 262 kB/s | 26 kB 00:00 2024-06-03T03:07:42.606 INFO:teuthology.orchestra.run.smithi117.stdout:(109/159): gperftools-libs-2.9.1-2.el9.x86_64.r 601 kB/s | 309 kB 00:00 2024-06-03T03:07:42.648 INFO:teuthology.orchestra.run.smithi117.stdout:(110/159): liboath-2.6.7-2.el9.x86_64.rpm 653 kB/s | 49 kB 00:00 2024-06-03T03:07:42.690 INFO:teuthology.orchestra.run.smithi117.stdout:(111/159): libunwind-1.6.2-1.el9.x86_64.rpm 813 kB/s | 67 kB 00:00 2024-06-03T03:07:42.732 INFO:teuthology.orchestra.run.smithi117.stdout:(112/159): luarocks-3.9.2-5.el9.noarch.rpm 1.8 MB/s | 151 kB 00:00 2024-06-03T03:07:42.880 INFO:teuthology.orchestra.run.smithi117.stdout:(113/159): python3-asyncssh-2.13.2-5.el9.noarch 3.6 MB/s | 548 kB 00:00 2024-06-03T03:07:42.920 INFO:teuthology.orchestra.run.smithi151.stdout:(159/159): ceph-test-19.0.0-2946.g3d89accf.el9. 3.0 MB/s | 48 MB 00:16 2024-06-03T03:07:42.927 INFO:teuthology.orchestra.run.smithi151.stdout:-------------------------------------------------------------------------------- 2024-06-03T03:07:42.927 INFO:teuthology.orchestra.run.smithi151.stdout:Total 12 MB/s | 225 MB 00:19 2024-06-03T03:07:42.932 INFO:teuthology.orchestra.run.smithi117.stdout:(114/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.4 MB/s | 839 kB 00:00 2024-06-03T03:07:42.958 INFO:teuthology.orchestra.run.smithi117.stdout:(115/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 566 kB/s | 43 kB 00:00 2024-06-03T03:07:43.000 INFO:teuthology.orchestra.run.smithi117.stdout:(116/159): python3-cachetools-4.2.4-1.el9.noarc 488 kB/s | 32 kB 00:00 2024-06-03T03:07:43.025 INFO:teuthology.orchestra.run.smithi117.stdout:(117/159): python3-certifi-2021.10.8-2.el9.noar 226 kB/s | 15 kB 00:00 2024-06-03T03:07:43.150 INFO:teuthology.orchestra.run.smithi117.stdout:(118/159): libarrow-9.0.0-11.el9.x86_64.rpm 6.1 MB/s | 4.4 MB 00:00 2024-06-03T03:07:43.209 INFO:teuthology.orchestra.run.smithi117.stdout:(119/159): python3-cheroot-8.6.0-4.el9.noarch.r 824 kB/s | 172 kB 00:00 2024-06-03T03:07:43.251 INFO:teuthology.orchestra.run.smithi117.stdout:(120/159): python3-cherrypy-18.6.1-2.el9.noarch 1.6 MB/s | 358 kB 00:00 2024-06-03T03:07:43.285 INFO:teuthology.orchestra.run.smithi117.stdout:(121/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.2 MB/s | 303 kB 00:00 2024-06-03T03:07:43.318 INFO:teuthology.orchestra.run.smithi117.stdout:(122/159): python3-google-auth-2.29.0-1.el9.noa 1.9 MB/s | 206 kB 00:00 2024-06-03T03:07:43.361 INFO:teuthology.orchestra.run.smithi117.stdout:(123/159): python3-grpcio-tools-1.46.7-10.el9.x 1.9 MB/s | 144 kB 00:00 2024-06-03T03:07:43.386 INFO:teuthology.orchestra.run.smithi117.stdout:(124/159): python3-jaraco-8.2.1-3.el9.noarch.rp 161 kB/s | 11 kB 00:00 2024-06-03T03:07:43.470 INFO:teuthology.orchestra.run.smithi117.stdout:(125/159): python3-grpcio-1.46.7-10.el9.x86_64. 9.4 MB/s | 2.0 MB 00:00 2024-06-03T03:07:43.494 INFO:teuthology.orchestra.run.smithi117.stdout:(126/159): python3-jaraco-classes-3.2.1-5.el9.n 133 kB/s | 18 kB 00:00 2024-06-03T03:07:43.545 INFO:teuthology.orchestra.run.smithi117.stdout:(127/159): python3-jaraco-collections-3.0.0-8.e 147 kB/s | 23 kB 00:00 2024-06-03T03:07:43.570 INFO:teuthology.orchestra.run.smithi117.stdout:(128/159): python3-jaraco-functools-3.5.0-2.el9 195 kB/s | 19 kB 00:00 2024-06-03T03:07:43.620 INFO:teuthology.orchestra.run.smithi117.stdout:(129/159): python3-jaraco-text-3.2.0-6.el9.noar 265 kB/s | 20 kB 00:00 2024-06-03T03:07:43.645 INFO:teuthology.orchestra.run.smithi117.stdout:(130/159): python3-logutils-0.3.5-21.el9.noarch 620 kB/s | 46 kB 00:00 2024-06-03T03:07:43.704 INFO:teuthology.orchestra.run.smithi117.stdout:(131/159): python3-kubernetes-26.1.0-2.el9.noar 6.5 MB/s | 1.0 MB 00:00 2024-06-03T03:07:43.729 INFO:teuthology.orchestra.run.smithi117.stdout:(132/159): python3-natsort-7.1.1-5.el9.noarch.r 696 kB/s | 58 kB 00:00 2024-06-03T03:07:43.755 INFO:teuthology.orchestra.run.smithi117.stdout:(133/159): python3-more-itertools-8.12.0-2.el9. 592 kB/s | 79 kB 00:00 2024-06-03T03:07:43.788 INFO:teuthology.orchestra.run.smithi117.stdout:(134/159): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-06-03T03:07:43.813 INFO:teuthology.orchestra.run.smithi117.stdout:(135/159): python3-portend-3.1.0-2.el9.noarch.r 197 kB/s | 16 kB 00:00 2024-06-03T03:07:43.864 INFO:teuthology.orchestra.run.smithi117.stdout:(136/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 832 kB/s | 90 kB 00:00 2024-06-03T03:07:43.889 INFO:teuthology.orchestra.run.smithi117.stdout:(137/159): python3-pyperclip-1.8.2-2.el9.noarch 283 kB/s | 28 kB 00:00 2024-06-03T03:07:43.914 INFO:teuthology.orchestra.run.smithi117.stdout:(138/159): python3-repoze-lru-0.7-16.el9.noarch 308 kB/s | 31 kB 00:00 2024-06-03T03:07:43.948 INFO:teuthology.orchestra.run.smithi117.stdout:(139/159): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-06-03T03:07:43.990 INFO:teuthology.orchestra.run.smithi117.stdout:(140/159): python3-tempora-5.0.0-2.el9.noarch.r 484 kB/s | 36 kB 00:00 2024-06-03T03:07:44.015 INFO:teuthology.orchestra.run.smithi117.stdout:(141/159): python3-rsa-4.9-2.el9.noarch.rpm 595 kB/s | 59 kB 00:00 2024-06-03T03:07:44.040 INFO:teuthology.orchestra.run.smithi117.stdout:(142/159): python3-typing-extensions-4.4.0-2.el 556 kB/s | 51 kB 00:00 2024-06-03T03:07:44.074 INFO:teuthology.orchestra.run.smithi117.stdout:(143/159): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-06-03T03:07:44.099 INFO:teuthology.orchestra.run.smithi117.stdout:(144/159): python3-websocket-client-1.2.3-2.el9 1.1 MB/s | 90 kB 00:00 2024-06-03T03:07:44.115 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction test succeeded. 2024-06-03T03:07:44.116 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction 2024-06-03T03:07:44.162 INFO:teuthology.orchestra.run.smithi151.stdout:Running transaction check 2024-06-03T03:07:44.174 INFO:teuthology.orchestra.run.smithi117.stdout:(145/159): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-06-03T03:07:44.200 INFO:teuthology.orchestra.run.smithi117.stdout:(146/159): python3-zc-lockfile-2.0-10.el9.noarc 160 kB/s | 20 kB 00:00 2024-06-03T03:07:44.233 INFO:teuthology.orchestra.run.smithi117.stdout:(147/159): re2-20211101-3.el9.x86_64.rpm 1.4 MB/s | 192 kB 00:00 2024-06-03T03:07:44.320 INFO:teuthology.orchestra.run.smithi151.stdout:Transaction check succeeded. 2024-06-03T03:07:44.320 INFO:teuthology.orchestra.run.smithi151.stdout:Running transaction test 2024-06-03T03:07:44.342 INFO:teuthology.orchestra.run.smithi117.stdout:(148/159): thrift-0.15.0-2.el9.x86_64.rpm 9.5 MB/s | 1.6 MB 00:00 2024-06-03T03:07:44.559 INFO:teuthology.orchestra.run.smithi117.stdout:(149/159): librbd1-19.0.0-2946.g3d89accf.el9.x8 9.7 MB/s | 3.1 MB 00:00 2024-06-03T03:07:44.734 INFO:teuthology.orchestra.run.smithi117.stdout:(150/159): librados2-19.0.0-2946.g3d89accf.el9. 6.3 MB/s | 3.4 MB 00:00 2024-06-03T03:07:44.826 INFO:teuthology.orchestra.run.smithi117.stdout:(151/159): NetworkManager-1.47.91-1.el9.x86_64. 4.8 MB/s | 2.3 MB 00:00 2024-06-03T03:07:44.902 INFO:teuthology.orchestra.run.smithi117.stdout:(152/159): NetworkManager-tui-1.47.91-1.el9.x86 3.3 MB/s | 248 kB 00:00 2024-06-03T03:07:44.943 INFO:teuthology.orchestra.run.smithi117.stdout:(153/159): NetworkManager-team-1.47.91-1.el9.x8 199 kB/s | 41 kB 00:00 2024-06-03T03:07:44.961 INFO:teuthology.orchestra.run.smithi117.stdout:(154/159): elfutils-debuginfod-client-0.191-4.e 635 kB/s | 37 kB 00:00 2024-06-03T03:07:45.010 INFO:teuthology.orchestra.run.smithi117.stdout:(155/159): NetworkManager-libnm-1.47.91-1.el9.x 4.0 MB/s | 1.8 MB 00:00 2024-06-03T03:07:45.044 INFO:teuthology.orchestra.run.smithi117.stdout:(156/159): elfutils-libs-0.191-4.el9.x86_64.rpm 3.1 MB/s | 260 kB 00:00 2024-06-03T03:07:45.110 INFO:teuthology.orchestra.run.smithi117.stdout:(157/159): elfutils-libelf-0.191-4.el9.x86_64.r 1.2 MB/s | 207 kB 00:00 2024-06-03T03:07:45.312 INFO:teuthology.orchestra.run.smithi117.stdout:(158/159): openssl-3.2.1-1.el9.x86_64.rpm 4.5 MB/s | 1.3 MB 00:00 2024-06-03T03:07:45.386 INFO:teuthology.orchestra.run.smithi117.stdout:(159/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 7.0 MB/s | 2.4 MB 00:00 2024-06-03T03:07:45.399 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2024-06-03T03:07:45.400 INFO:teuthology.orchestra.run.smithi117.stdout:Total 13 MB/s | 225 MB 00:17 2024-06-03T03:07:46.177 INFO:teuthology.orchestra.run.smithi151.stdout:Transaction test succeeded. 2024-06-03T03:07:46.178 INFO:teuthology.orchestra.run.smithi151.stdout:Running transaction 2024-06-03T03:07:48.601 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction check 2024-06-03T03:07:48.853 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction check succeeded. 2024-06-03T03:07:48.853 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction test 2024-06-03T03:07:50.485 INFO:teuthology.orchestra.run.smithi183.stdout: Preparing : 1/1 2024-06-03T03:07:50.569 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-06-03T03:07:50.607 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-06-03T03:07:50.660 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-06-03T03:07:50.694 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-06-03T03:07:50.720 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-06-03T03:07:50.763 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-06-03T03:07:51.086 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-06-03T03:07:51.188 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-03T03:07:51.309 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-03T03:07:51.347 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-03T03:07:51.349 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction test succeeded. 2024-06-03T03:07:51.354 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction 2024-06-03T03:07:51.422 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-03T03:07:51.451 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 10/170 2024-06-03T03:07:51.484 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-06-03T03:07:51.664 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-06-03T03:07:51.707 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-06-03T03:07:51.727 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-03T03:07:51.857 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-03T03:07:51.880 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/170 2024-06-03T03:07:51.926 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/170 2024-06-03T03:07:51.948 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-06-03T03:07:51.971 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-06-03T03:07:52.131 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-06-03T03:07:52.261 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-06-03T03:07:52.411 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-06-03T03:07:52.455 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-06-03T03:07:52.551 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-03T03:07:52.617 INFO:teuthology.orchestra.run.smithi151.stdout: Preparing : 1/1 2024-06-03T03:07:52.624 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-03T03:07:52.666 INFO:teuthology.orchestra.run.smithi151.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-06-03T03:07:52.712 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-06-03T03:07:52.766 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-06-03T03:07:52.799 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-06-03T03:07:52.819 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-03T03:07:52.825 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-06-03T03:07:52.868 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-06-03T03:07:52.926 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-06-03T03:07:53.019 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-06-03T03:07:53.071 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 22/170 2024-06-03T03:07:53.114 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/170 2024-06-03T03:07:53.188 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/170 2024-06-03T03:07:53.207 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-06-03T03:07:53.221 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 25/170 2024-06-03T03:07:53.265 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/170 2024-06-03T03:07:53.297 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/170 2024-06-03T03:07:53.309 INFO:teuthology.orchestra.run.smithi151.stdout: Upgrading : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-03T03:07:53.325 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/170 2024-06-03T03:07:53.394 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/170 2024-06-03T03:07:53.423 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-03T03:07:53.447 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 30/170 2024-06-03T03:07:53.452 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-03T03:07:53.478 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/170 2024-06-03T03:07:53.513 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 32/170 2024-06-03T03:07:53.543 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-03T03:07:53.573 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 10/170 2024-06-03T03:07:53.590 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 33/170 2024-06-03T03:07:53.606 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-06-03T03:07:53.624 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 34/170 2024-06-03T03:07:53.667 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 35/170 2024-06-03T03:07:53.696 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 36/170 2024-06-03T03:07:53.765 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/170 2024-06-03T03:07:53.799 INFO:teuthology.orchestra.run.smithi151.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-06-03T03:07:53.810 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/170 2024-06-03T03:07:53.846 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-06-03T03:07:53.874 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-03T03:07:53.892 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/170 2024-06-03T03:07:53.976 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-03T03:07:54.000 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/170 2024-06-03T03:07:54.027 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/170 2024-06-03T03:07:54.046 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/170 2024-06-03T03:07:54.068 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-06-03T03:07:54.079 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/170 2024-06-03T03:07:54.087 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-06-03T03:07:54.243 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-06-03T03:07:54.372 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-06-03T03:07:54.523 INFO:teuthology.orchestra.run.smithi151.stdout: Upgrading : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-06-03T03:07:54.584 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-06-03T03:07:54.604 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/170 2024-06-03T03:07:54.691 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/170 2024-06-03T03:07:54.699 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-03T03:07:54.720 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/170 2024-06-03T03:07:54.756 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/170 2024-06-03T03:07:54.776 INFO:teuthology.orchestra.run.smithi151.stdout: Upgrading : NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-03T03:07:54.813 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 46/170 2024-06-03T03:07:54.852 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 47/170 2024-06-03T03:07:54.918 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 48/170 2024-06-03T03:07:54.981 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-03T03:07:55.068 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 49/170 2024-06-03T03:07:55.093 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-06-03T03:07:55.129 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 50/170 2024-06-03T03:07:55.184 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 51/170 2024-06-03T03:07:55.195 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-06-03T03:07:55.232 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 22/170 2024-06-03T03:07:55.256 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-06-03T03:07:55.275 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/170 2024-06-03T03:07:55.288 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-06-03T03:07:55.325 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-06-03T03:07:55.349 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/170 2024-06-03T03:07:55.371 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-06-03T03:07:55.382 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 25/170 2024-06-03T03:07:55.407 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-06-03T03:07:55.426 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/170 2024-06-03T03:07:55.450 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-06-03T03:07:55.459 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/170 2024-06-03T03:07:55.486 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/170 2024-06-03T03:07:55.492 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-06-03T03:07:55.555 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/170 2024-06-03T03:07:55.608 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 30/170 2024-06-03T03:07:55.628 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-06-03T03:07:55.639 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/170 2024-06-03T03:07:55.669 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-06-03T03:07:55.674 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 32/170 2024-06-03T03:07:55.710 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-06-03T03:07:55.753 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 33/170 2024-06-03T03:07:55.785 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 34/170 2024-06-03T03:07:55.805 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-06-03T03:07:55.828 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 35/170 2024-06-03T03:07:55.844 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 63/170 2024-06-03T03:07:55.857 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 36/170 2024-06-03T03:07:55.884 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-06-03T03:07:55.910 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-06-03T03:07:55.927 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/170 2024-06-03T03:07:55.934 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-06-03T03:07:55.972 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/170 2024-06-03T03:07:56.009 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-06-03T03:07:56.046 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/170 2024-06-03T03:07:56.136 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-06-03T03:07:56.179 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/170 2024-06-03T03:07:56.232 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/170 2024-06-03T03:07:56.308 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-06-03T03:07:56.328 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-06-03T03:07:56.362 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-06-03T03:07:56.402 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-06-03T03:07:56.441 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-06-03T03:07:56.471 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-06-03T03:07:56.506 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-06-03T03:07:56.539 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-06-03T03:07:56.580 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-06-03T03:07:56.623 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-06-03T03:07:56.710 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-06-03T03:07:56.753 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/170 2024-06-03T03:07:56.761 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-06-03T03:07:56.793 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-06-03T03:07:56.827 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/170 2024-06-03T03:07:56.839 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-06-03T03:07:56.857 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/170 2024-06-03T03:07:56.893 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/170 2024-06-03T03:07:56.937 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-06-03T03:07:56.950 INFO:teuthology.orchestra.run.smithi151.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 46/170 2024-06-03T03:07:56.986 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-06-03T03:07:56.988 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 47/170 2024-06-03T03:07:57.030 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 48/170 2024-06-03T03:07:57.194 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 49/170 2024-06-03T03:07:57.266 INFO:teuthology.orchestra.run.smithi151.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 50/170 2024-06-03T03:07:57.321 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 51/170 2024-06-03T03:07:57.394 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-06-03T03:07:57.425 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-06-03T03:07:57.461 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-06-03T03:07:57.508 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-06-03T03:07:57.544 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-06-03T03:07:57.587 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-06-03T03:07:57.629 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-06-03T03:07:57.712 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-06-03T03:07:57.758 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-06-03T03:07:57.765 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-06-03T03:07:57.800 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-06-03T03:07:57.814 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-06-03T03:07:57.855 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-06-03T03:07:57.898 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-06-03T03:07:57.948 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-06-03T03:07:57.954 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-06-03T03:07:57.980 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 63/170 2024-06-03T03:07:57.999 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-06-03T03:07:58.021 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-06-03T03:07:58.046 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-06-03T03:07:58.071 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-06-03T03:07:58.104 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-06-03T03:07:58.144 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-06-03T03:07:58.145 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-06-03T03:07:58.320 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-06-03T03:07:58.339 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-06-03T03:07:58.350 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-06-03T03:07:58.370 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-06-03T03:07:58.384 INFO:teuthology.orchestra.run.smithi117.stdout: Preparing : 1/1 2024-06-03T03:07:58.397 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-06-03T03:07:58.427 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-06-03T03:07:58.432 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-06-03T03:07:58.471 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-06-03T03:07:58.492 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-06-03T03:07:58.502 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-06-03T03:07:58.532 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-06-03T03:07:58.567 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-06-03T03:07:58.567 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-06-03T03:07:58.600 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-06-03T03:07:58.641 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-06-03T03:07:58.685 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-06-03T03:07:58.687 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-06-03T03:07:58.747 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-06-03T03:07:58.770 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-06-03T03:07:58.778 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-06-03T03:07:58.823 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-06-03T03:07:58.854 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-06-03T03:07:58.878 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-06-03T03:07:58.901 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-06-03T03:07:58.998 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-06-03T03:07:59.047 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-06-03T03:07:59.067 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-06-03T03:07:59.180 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-06-03T03:07:59.346 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-06-03T03:07:59.459 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-03T03:07:59.609 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-03T03:07:59.643 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-03T03:07:59.769 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-03T03:07:59.771 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-06-03T03:07:59.800 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 10/170 2024-06-03T03:07:59.819 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-06-03T03:07:59.853 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-06-03T03:07:59.870 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-06-03T03:07:59.952 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-06-03T03:08:00.008 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-06-03T03:08:00.053 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-06-03T03:08:00.139 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-06-03T03:08:00.180 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-06-03T03:08:00.196 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-06-03T03:08:00.265 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-06-03T03:08:00.290 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-03T03:08:00.356 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-06-03T03:08:00.386 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-06-03T03:08:00.401 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-06-03T03:08:00.427 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-03T03:08:00.458 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/170 2024-06-03T03:08:00.464 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-06-03T03:08:00.585 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/170 2024-06-03T03:08:00.594 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-06-03T03:08:00.617 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-06-03T03:08:00.621 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-06-03T03:08:00.655 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-06-03T03:08:00.696 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-06-03T03:08:00.965 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-06-03T03:08:00.985 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-06-03T03:08:01.009 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 103/170 2024-06-03T03:08:01.101 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 103/170 2024-06-03T03:08:01.116 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-06-03T03:08:01.131 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 104/170 2024-06-03T03:08:01.186 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-06-03T03:08:01.226 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-06-03T03:08:01.379 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-06-03T03:08:01.567 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-06-03T03:08:01.654 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-06-03T03:08:01.654 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-06-03T03:08:01.687 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 107/170 2024-06-03T03:08:01.758 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 107/170 2024-06-03T03:08:01.780 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 108/170 2024-06-03T03:08:01.821 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-03T03:08:01.953 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-03T03:08:02.283 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-03T03:08:02.466 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-06-03T03:08:02.477 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-06-03T03:08:02.538 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-06-03T03:08:02.564 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-06-03T03:08:02.597 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-06-03T03:08:02.610 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-06-03T03:08:02.658 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 22/170 2024-06-03T03:08:02.706 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/170 2024-06-03T03:08:02.830 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/170 2024-06-03T03:08:02.880 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 25/170 2024-06-03T03:08:02.910 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-06-03T03:08:02.952 INFO:teuthology.orchestra.run.smithi151.stdout: Upgrading : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 103/170 2024-06-03T03:08:02.959 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/170 2024-06-03T03:08:03.011 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/170 2024-06-03T03:08:03.042 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 103/170 2024-06-03T03:08:03.050 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/170 2024-06-03T03:08:03.074 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 104/170 2024-06-03T03:08:03.110 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-06-03T03:08:03.168 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/170 2024-06-03T03:08:03.263 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 30/170 2024-06-03T03:08:03.310 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/170 2024-06-03T03:08:03.368 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 32/170 2024-06-03T03:08:03.485 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 33/170 2024-06-03T03:08:03.545 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 34/170 2024-06-03T03:08:03.583 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-06-03T03:08:03.622 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 35/170 2024-06-03T03:08:03.622 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 107/170 2024-06-03T03:08:03.655 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 36/170 2024-06-03T03:08:03.692 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 107/170 2024-06-03T03:08:03.715 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 108/170 2024-06-03T03:08:03.777 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/170 2024-06-03T03:08:03.843 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/170 2024-06-03T03:08:03.960 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/170 2024-06-03T03:08:04.054 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-03T03:08:04.085 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-03T03:08:04.142 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/170 2024-06-03T03:08:04.219 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/170 2024-06-03T03:08:04.626 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-03T03:08:04.663 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 110/170 2024-06-03T03:08:04.757 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 110/170 2024-06-03T03:08:04.757 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-06-03T03:08:04.757 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-06-03T03:08:04.757 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-06-03T03:08:04.798 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 111/170 2024-06-03T03:08:04.919 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/170 2024-06-03T03:08:05.024 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/170 2024-06-03T03:08:05.059 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/170 2024-06-03T03:08:05.133 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/170 2024-06-03T03:08:05.202 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 46/170 2024-06-03T03:08:05.255 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 47/170 2024-06-03T03:08:05.326 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 48/170 2024-06-03T03:08:05.550 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 49/170 2024-06-03T03:08:05.613 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 50/170 2024-06-03T03:08:05.692 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 51/170 2024-06-03T03:08:05.801 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-06-03T03:08:05.838 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-06-03T03:08:05.877 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-06-03T03:08:05.942 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-06-03T03:08:05.982 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-03T03:08:05.985 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-06-03T03:08:06.028 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-03T03:08:06.074 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-06-03T03:08:06.141 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-06-03T03:08:06.343 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-06-03T03:08:06.395 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-06-03T03:08:06.448 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-06-03T03:08:06.569 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-03T03:08:06.576 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-06-03T03:08:06.606 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 110/170 2024-06-03T03:08:06.629 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 63/170 2024-06-03T03:08:06.697 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 110/170 2024-06-03T03:08:06.697 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-06-03T03:08:06.697 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-06-03T03:08:06.697 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-06-03T03:08:06.723 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 111/170 2024-06-03T03:08:06.745 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-06-03T03:08:06.788 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-06-03T03:08:06.820 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-06-03T03:08:06.999 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-06-03T03:08:07.179 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-06-03T03:08:07.233 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-06-03T03:08:07.271 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-06-03T03:08:07.342 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-06-03T03:08:07.405 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-06-03T03:08:07.464 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-06-03T03:08:07.513 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-06-03T03:08:07.559 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-06-03T03:08:07.644 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-06-03T03:08:07.729 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-06-03T03:08:07.768 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-06-03T03:08:07.914 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-06-03T03:08:07.989 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-06-03T03:08:08.030 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-06-03T03:08:08.100 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-06-03T03:08:08.245 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-06-03T03:08:08.326 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-06-03T03:08:09.282 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-06-03T03:08:09.350 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-06-03T03:08:09.383 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-06-03T03:08:09.506 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-06-03T03:08:09.566 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-06-03T03:08:09.619 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-06-03T03:08:09.728 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-06-03T03:08:09.768 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-06-03T03:08:10.026 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-06-03T03:08:10.057 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-06-03T03:08:10.162 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-06-03T03:08:10.979 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-06-03T03:08:11.278 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-06-03T03:08:12.909 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-06-03T03:08:13.013 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-06-03T03:08:13.048 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-06-03T03:08:13.084 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-06-03T03:08:13.450 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-06-03T03:08:13.509 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 103/170 2024-06-03T03:08:13.648 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 103/170 2024-06-03T03:08:13.682 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 104/170 2024-06-03T03:08:13.738 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-06-03T03:08:14.274 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-06-03T03:08:14.311 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 107/170 2024-06-03T03:08:14.402 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 107/170 2024-06-03T03:08:14.430 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 108/170 2024-06-03T03:08:16.319 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 111/170 2024-06-03T03:08:16.319 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /sys 2024-06-03T03:08:16.319 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /proc 2024-06-03T03:08:16.319 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /mnt 2024-06-03T03:08:16.319 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /var/tmp 2024-06-03T03:08:16.320 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /home 2024-06-03T03:08:16.320 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /root 2024-06-03T03:08:16.320 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /tmp 2024-06-03T03:08:16.320 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-06-03T03:08:16.488 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 112/170 2024-06-03T03:08:16.504 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 112/170 2024-06-03T03:08:16.504 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T03:08:16.504 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-06-03T03:08:16.504 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-06-03T03:08:17.361 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-03T03:08:17.418 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-03T03:08:17.975 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 113/170 2024-06-03T03:08:18.001 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 113/170 2024-06-03T03:08:18.001 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T03:08:18.001 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-06-03T03:08:18.001 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-03T03:08:18.001 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-03T03:08:18.001 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-06-03T03:08:18.059 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-03T03:08:18.105 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 110/170 2024-06-03T03:08:18.225 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 114/170 2024-06-03T03:08:18.248 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 114/170 2024-06-03T03:08:18.248 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T03:08:18.248 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-06-03T03:08:18.248 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-03T03:08:18.248 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-03T03:08:18.248 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-06-03T03:08:18.275 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 110/170 2024-06-03T03:08:18.275 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-06-03T03:08:18.275 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-06-03T03:08:18.275 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T03:08:18.313 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 111/170 2024-06-03T03:08:18.502 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 111/170 2024-06-03T03:08:18.502 INFO:teuthology.orchestra.run.smithi151.stdout:skipping the directory /sys 2024-06-03T03:08:18.502 INFO:teuthology.orchestra.run.smithi151.stdout:skipping the directory /proc 2024-06-03T03:08:18.502 INFO:teuthology.orchestra.run.smithi151.stdout:skipping the directory /mnt 2024-06-03T03:08:18.502 INFO:teuthology.orchestra.run.smithi151.stdout:skipping the directory /var/tmp 2024-06-03T03:08:18.503 INFO:teuthology.orchestra.run.smithi151.stdout:skipping the directory /home 2024-06-03T03:08:18.503 INFO:teuthology.orchestra.run.smithi151.stdout:skipping the directory /root 2024-06-03T03:08:18.503 INFO:teuthology.orchestra.run.smithi151.stdout:skipping the directory /tmp 2024-06-03T03:08:18.503 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-06-03T03:08:18.669 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 112/170 2024-06-03T03:08:18.685 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 112/170 2024-06-03T03:08:18.685 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T03:08:18.685 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-06-03T03:08:18.685 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-06-03T03:08:18.693 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 115/170 2024-06-03T03:08:18.715 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 115/170 2024-06-03T03:08:18.715 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T03:08:18.715 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-06-03T03:08:18.715 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-03T03:08:18.715 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-03T03:08:18.715 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-06-03T03:08:18.848 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-06-03T03:08:18.905 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-06-03T03:08:18.960 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-06-03T03:08:19.144 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-06-03T03:08:19.202 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-06-03T03:08:19.279 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-06-03T03:08:19.367 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-06-03T03:08:19.498 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-06-03T03:08:19.536 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-06-03T03:08:19.576 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-06-03T03:08:19.603 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-06-03T03:08:20.031 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 127/170 2024-06-03T03:08:20.077 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 127/170 2024-06-03T03:08:20.109 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 128/170 2024-06-03T03:08:20.171 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 129/170 2024-06-03T03:08:20.173 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 113/170 2024-06-03T03:08:20.199 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 113/170 2024-06-03T03:08:20.199 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T03:08:20.199 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-06-03T03:08:20.199 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-03T03:08:20.199 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-03T03:08:20.199 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-06-03T03:08:20.208 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 130/170 2024-06-03T03:08:20.432 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 114/170 2024-06-03T03:08:20.455 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 114/170 2024-06-03T03:08:20.455 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T03:08:20.455 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-06-03T03:08:20.455 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-03T03:08:20.455 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-03T03:08:20.455 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-06-03T03:08:20.900 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 115/170 2024-06-03T03:08:20.922 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 115/170 2024-06-03T03:08:20.922 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T03:08:20.922 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-06-03T03:08:20.922 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-03T03:08:20.923 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-03T03:08:20.923 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-06-03T03:08:21.030 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-06-03T03:08:21.079 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-06-03T03:08:21.134 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-06-03T03:08:21.241 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 130/170 2024-06-03T03:08:21.294 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 131/170 2024-06-03T03:08:21.320 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-06-03T03:08:21.368 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-06-03T03:08:21.446 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-06-03T03:08:21.535 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-06-03T03:08:21.676 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-06-03T03:08:21.728 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-06-03T03:08:21.768 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-06-03T03:08:21.796 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-06-03T03:08:22.241 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 127/170 2024-06-03T03:08:22.260 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 131/170 2024-06-03T03:08:22.271 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 127/170 2024-06-03T03:08:22.285 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 132/170 2024-06-03T03:08:22.302 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 128/170 2024-06-03T03:08:22.364 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 129/170 2024-06-03T03:08:22.387 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 132/170 2024-06-03T03:08:22.394 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 130/170 2024-06-03T03:08:22.565 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 133/170 2024-06-03T03:08:22.596 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 134/170 2024-06-03T03:08:22.619 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 134/170 2024-06-03T03:08:22.619 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T03:08:22.620 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-06-03T03:08:22.620 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-03T03:08:22.620 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-03T03:08:22.620 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-06-03T03:08:22.663 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 135/170 2024-06-03T03:08:22.674 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 135/170 2024-06-03T03:08:22.774 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 136/170 2024-06-03T03:08:22.829 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 137/170 2024-06-03T03:08:23.430 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 130/170 2024-06-03T03:08:23.463 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 131/170 2024-06-03T03:08:23.734 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-06-03T03:08:23.766 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 139/170 2024-06-03T03:08:23.789 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 139/170 2024-06-03T03:08:23.789 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T03:08:23.789 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-06-03T03:08:23.789 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-03T03:08:23.789 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-03T03:08:23.790 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-06-03T03:08:23.833 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 140/170 2024-06-03T03:08:23.855 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 140/170 2024-06-03T03:08:23.855 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T03:08:23.855 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-06-03T03:08:23.855 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-06-03T03:08:23.909 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 141/170 2024-06-03T03:08:23.932 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 141/170 2024-06-03T03:08:23.932 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T03:08:23.932 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-06-03T03:08:23.932 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-03T03:08:23.932 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-03T03:08:23.932 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-06-03T03:08:24.234 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/170 2024-06-03T03:08:24.258 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/170 2024-06-03T03:08:24.258 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T03:08:24.258 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-06-03T03:08:24.258 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-03T03:08:24.258 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-03T03:08:24.258 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-06-03T03:08:24.428 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 131/170 2024-06-03T03:08:24.464 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 132/170 2024-06-03T03:08:24.568 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 132/170 2024-06-03T03:08:24.744 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 133/170 2024-06-03T03:08:24.775 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 134/170 2024-06-03T03:08:24.798 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 134/170 2024-06-03T03:08:24.799 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T03:08:24.799 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-06-03T03:08:24.799 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-03T03:08:24.799 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-03T03:08:24.799 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-06-03T03:08:24.842 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 135/170 2024-06-03T03:08:24.853 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 135/170 2024-06-03T03:08:24.968 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 136/170 2024-06-03T03:08:25.016 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 137/170 2024-06-03T03:08:25.914 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-06-03T03:08:25.954 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 139/170 2024-06-03T03:08:25.978 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 139/170 2024-06-03T03:08:25.978 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T03:08:25.978 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-06-03T03:08:25.978 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-03T03:08:25.979 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-03T03:08:25.979 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-06-03T03:08:26.022 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 140/170 2024-06-03T03:08:26.044 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 140/170 2024-06-03T03:08:26.044 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T03:08:26.044 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-06-03T03:08:26.044 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-06-03T03:08:26.089 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 141/170 2024-06-03T03:08:26.112 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 141/170 2024-06-03T03:08:26.112 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T03:08:26.112 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-06-03T03:08:26.112 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-03T03:08:26.112 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-03T03:08:26.112 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-06-03T03:08:26.415 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/170 2024-06-03T03:08:26.438 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/170 2024-06-03T03:08:26.438 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T03:08:26.438 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-06-03T03:08:26.438 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-03T03:08:26.438 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-03T03:08:26.438 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-06-03T03:08:28.780 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 143/170 2024-06-03T03:08:28.821 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 144/170 2024-06-03T03:08:28.861 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 145/170 2024-06-03T03:08:28.894 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-06-03T03:08:28.921 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-06-03T03:08:29.102 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 148/170 2024-06-03T03:08:29.151 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-06-03T03:08:29.183 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 150/170 2024-06-03T03:08:29.224 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-06-03T03:08:29.253 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : NetworkManager-team-1:1.47.91-1.el9.x86_64 152/170 2024-06-03T03:08:29.300 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : NetworkManager-tui-1:1.47.91-1.el9.x86_64 153/170 2024-06-03T03:08:29.937 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-03T03:08:29.973 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-03T03:08:29.993 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-03T03:08:30.824 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 155/170 2024-06-03T03:08:30.910 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-06-03T03:08:30.936 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 143/170 2024-06-03T03:08:30.951 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-06-03T03:08:30.977 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 144/170 2024-06-03T03:08:31.032 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-06-03T03:08:31.035 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 145/170 2024-06-03T03:08:31.073 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-06-03T03:08:31.076 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-06-03T03:08:31.098 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-06-03T03:08:31.112 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-06-03T03:08:31.123 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-06-03T03:08:31.149 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-06-03T03:08:31.149 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-06-03T03:08:31.233 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-06-03T03:08:31.258 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-06-03T03:08:31.258 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-06-03T03:08:31.302 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 148/170 2024-06-03T03:08:31.351 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-06-03T03:08:31.351 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-06-03T03:08:31.376 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-06-03T03:08:31.381 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-03T03:08:31.381 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-03T03:08:31.384 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 150/170 2024-06-03T03:08:31.424 INFO:teuthology.orchestra.run.smithi151.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-06-03T03:08:31.454 INFO:teuthology.orchestra.run.smithi151.stdout: Upgrading : NetworkManager-team-1:1.47.91-1.el9.x86_64 152/170 2024-06-03T03:08:31.501 INFO:teuthology.orchestra.run.smithi151.stdout: Upgrading : NetworkManager-tui-1:1.47.91-1.el9.x86_64 153/170 2024-06-03T03:08:31.561 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-03T03:08:31.562 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-06-03T03:08:31.637 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-06-03T03:08:31.662 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-06-03T03:08:31.726 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-06-03T03:08:31.830 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 111/170 2024-06-03T03:08:31.830 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /sys 2024-06-03T03:08:31.830 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /proc 2024-06-03T03:08:31.831 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /mnt 2024-06-03T03:08:31.831 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /var/tmp 2024-06-03T03:08:31.831 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /home 2024-06-03T03:08:31.831 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /root 2024-06-03T03:08:31.831 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /tmp 2024-06-03T03:08:31.831 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T03:08:32.070 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-03T03:08:32.071 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 112/170 2024-06-03T03:08:32.109 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-03T03:08:32.121 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 112/170 2024-06-03T03:08:32.121 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T03:08:32.121 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-06-03T03:08:32.121 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T03:08:32.129 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-03T03:08:32.978 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 155/170 2024-06-03T03:08:33.067 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-06-03T03:08:33.108 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-06-03T03:08:33.189 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-06-03T03:08:33.230 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-06-03T03:08:33.254 INFO:teuthology.orchestra.run.smithi151.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-06-03T03:08:33.280 INFO:teuthology.orchestra.run.smithi151.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-06-03T03:08:33.307 INFO:teuthology.orchestra.run.smithi151.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-06-03T03:08:33.307 INFO:teuthology.orchestra.run.smithi151.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-06-03T03:08:33.392 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-06-03T03:08:33.417 INFO:teuthology.orchestra.run.smithi151.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-06-03T03:08:33.417 INFO:teuthology.orchestra.run.smithi151.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-06-03T03:08:33.514 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-06-03T03:08:33.539 INFO:teuthology.orchestra.run.smithi151.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-06-03T03:08:33.544 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-03T03:08:33.544 INFO:teuthology.orchestra.run.smithi151.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-03T03:08:33.724 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-03T03:08:33.724 INFO:teuthology.orchestra.run.smithi151.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-06-03T03:08:33.755 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 113/170 2024-06-03T03:08:33.809 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-06-03T03:08:33.844 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 113/170 2024-06-03T03:08:33.844 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T03:08:33.844 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-06-03T03:08:33.844 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-03T03:08:33.844 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-03T03:08:33.845 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T03:08:33.848 INFO:teuthology.orchestra.run.smithi151.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-06-03T03:08:33.921 INFO:teuthology.orchestra.run.smithi151.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-06-03T03:08:34.097 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 114/170 2024-06-03T03:08:34.173 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 114/170 2024-06-03T03:08:34.173 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T03:08:34.173 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-06-03T03:08:34.173 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-03T03:08:34.173 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-03T03:08:34.173 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T03:08:34.664 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 115/170 2024-06-03T03:08:34.738 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 115/170 2024-06-03T03:08:34.739 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T03:08:34.739 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-06-03T03:08:34.739 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-03T03:08:34.739 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-03T03:08:34.739 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T03:08:34.874 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-06-03T03:08:34.936 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-06-03T03:08:35.017 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-06-03T03:08:35.265 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-06-03T03:08:35.325 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-06-03T03:08:35.412 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-06-03T03:08:35.520 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-06-03T03:08:35.704 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-06-03T03:08:35.766 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-06-03T03:08:35.806 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-06-03T03:08:35.857 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-06-03T03:08:36.542 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 127/170 2024-06-03T03:08:36.585 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 127/170 2024-06-03T03:08:36.638 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 128/170 2024-06-03T03:08:36.751 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 129/170 2024-06-03T03:08:36.794 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 130/170 2024-06-03T03:08:37.058 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-06-03T03:08:37.059 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 1/170 2024-06-03T03:08:37.059 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2/170 2024-06-03T03:08:37.059 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 3/170 2024-06-03T03:08:37.059 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 4/170 2024-06-03T03:08:37.059 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 5/170 2024-06-03T03:08:37.059 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 6/170 2024-06-03T03:08:37.059 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 7/170 2024-06-03T03:08:37.059 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-03T03:08:37.059 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-03T03:08:37.059 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 10/170 2024-06-03T03:08:37.059 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 11/170 2024-06-03T03:08:37.060 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 12/170 2024-06-03T03:08:37.060 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 13/170 2024-06-03T03:08:37.060 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-03T03:08:37.060 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 15/170 2024-06-03T03:08:37.060 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 16/170 2024-06-03T03:08:37.060 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 17/170 2024-06-03T03:08:37.060 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 18/170 2024-06-03T03:08:37.060 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 19/170 2024-06-03T03:08:37.060 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 20/170 2024-06-03T03:08:37.060 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 21/170 2024-06-03T03:08:37.060 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 22/170 2024-06-03T03:08:37.060 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 23/170 2024-06-03T03:08:37.060 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 24/170 2024-06-03T03:08:37.061 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 25/170 2024-06-03T03:08:37.061 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 26/170 2024-06-03T03:08:37.061 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 27/170 2024-06-03T03:08:37.061 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 28/170 2024-06-03T03:08:37.061 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 29/170 2024-06-03T03:08:37.061 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 30/170 2024-06-03T03:08:37.061 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 31/170 2024-06-03T03:08:37.061 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 32/170 2024-06-03T03:08:37.061 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 33/170 2024-06-03T03:08:37.061 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 34/170 2024-06-03T03:08:37.061 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 35/170 2024-06-03T03:08:37.061 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 36/170 2024-06-03T03:08:37.061 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 37/170 2024-06-03T03:08:37.062 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 38/170 2024-06-03T03:08:37.062 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 39/170 2024-06-03T03:08:37.062 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-06-03T03:08:37.062 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-06-03T03:08:37.062 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-06-03T03:08:37.062 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-06-03T03:08:37.062 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-06-03T03:08:37.062 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-06-03T03:08:37.062 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-06-03T03:08:37.062 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-06-03T03:08:37.062 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-06-03T03:08:37.062 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-06-03T03:08:37.062 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-06-03T03:08:37.063 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-06-03T03:08:37.063 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-06-03T03:08:37.063 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-06-03T03:08:37.063 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-06-03T03:08:37.063 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-06-03T03:08:37.063 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-06-03T03:08:37.063 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-06-03T03:08:37.063 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-06-03T03:08:37.063 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-06-03T03:08:37.063 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-06-03T03:08:37.063 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-06-03T03:08:37.063 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-06-03T03:08:37.063 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-06-03T03:08:37.064 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-06-03T03:08:37.064 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-06-03T03:08:37.064 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-06-03T03:08:37.064 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-06-03T03:08:37.064 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-06-03T03:08:37.064 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-06-03T03:08:37.064 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-06-03T03:08:37.064 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-06-03T03:08:37.064 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-06-03T03:08:37.064 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-06-03T03:08:37.064 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-06-03T03:08:37.064 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-06-03T03:08:37.064 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-06-03T03:08:37.064 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-06-03T03:08:37.065 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-06-03T03:08:37.065 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-06-03T03:08:37.065 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-06-03T03:08:37.065 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-06-03T03:08:37.065 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-06-03T03:08:37.065 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-06-03T03:08:37.065 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-06-03T03:08:37.065 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-06-03T03:08:37.065 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-06-03T03:08:37.065 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-06-03T03:08:37.065 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-06-03T03:08:37.065 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-06-03T03:08:37.065 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-06-03T03:08:37.066 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-06-03T03:08:37.066 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-06-03T03:08:37.066 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-06-03T03:08:37.066 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-06-03T03:08:37.066 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-06-03T03:08:37.066 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-06-03T03:08:37.066 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-06-03T03:08:37.066 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-06-03T03:08:37.066 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-06-03T03:08:37.066 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-06-03T03:08:37.066 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-06-03T03:08:37.066 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-06-03T03:08:37.066 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-06-03T03:08:37.067 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-06-03T03:08:37.067 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-06-03T03:08:37.067 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-06-03T03:08:37.067 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-06-03T03:08:37.067 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-06-03T03:08:37.067 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-06-03T03:08:37.067 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-06-03T03:08:37.067 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-06-03T03:08:37.067 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-06-03T03:08:37.067 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-06-03T03:08:37.067 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-06-03T03:08:37.067 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-06-03T03:08:37.067 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-06-03T03:08:37.068 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-06-03T03:08:37.068 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-06-03T03:08:37.068 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-06-03T03:08:37.068 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-06-03T03:08:37.068 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-06-03T03:08:37.068 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/170 2024-06-03T03:08:37.068 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-06-03T03:08:37.068 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-06-03T03:08:37.068 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-06-03T03:08:37.068 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-06-03T03:08:37.068 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-06-03T03:08:37.068 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-06-03T03:08:37.068 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-06-03T03:08:37.068 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-06-03T03:08:37.069 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-06-03T03:08:37.069 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-06-03T03:08:37.069 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-06-03T03:08:37.069 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-06-03T03:08:37.069 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-06-03T03:08:37.069 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-06-03T03:08:37.069 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-06-03T03:08:37.069 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-06-03T03:08:37.069 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-06-03T03:08:37.069 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-06-03T03:08:37.069 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-06-03T03:08:37.069 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-06-03T03:08:37.069 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-06-03T03:08:37.070 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-06-03T03:08:37.070 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-06-03T03:08:37.070 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-06-03T03:08:37.070 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-06-03T03:08:37.070 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-06-03T03:08:37.070 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 149/170 2024-06-03T03:08:37.070 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-06-03T03:08:37.070 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 151/170 2024-06-03T03:08:37.070 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-06-03T03:08:37.070 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-1:1.47.91-1.el9.x86_64 153/170 2024-06-03T03:08:37.070 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-06-03T03:08:37.070 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 155/170 2024-06-03T03:08:37.070 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-06-03T03:08:37.071 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-team-1:1.47.91-1.el9.x86_64 157/170 2024-06-03T03:08:37.071 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-06-03T03:08:37.071 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-tui-1:1.47.91-1.el9.x86_64 159/170 2024-06-03T03:08:37.071 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-06-03T03:08:37.071 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-06-03T03:08:37.071 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-06-03T03:08:37.071 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-06-03T03:08:37.071 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-06-03T03:08:37.071 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-06-03T03:08:37.071 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-06-03T03:08:37.071 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-06-03T03:08:37.071 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-06-03T03:08:37.071 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-06-03T03:08:38.071 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 130/170 2024-06-03T03:08:38.121 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 131/170 2024-06-03T03:08:39.213 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 131/170 2024-06-03T03:08:39.247 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 132/170 2024-06-03T03:08:39.391 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 132/170 2024-06-03T03:08:39.561 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 133/170 2024-06-03T03:08:39.590 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 134/170 2024-06-03T03:08:39.660 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 134/170 2024-06-03T03:08:39.660 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T03:08:39.660 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-06-03T03:08:39.660 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-03T03:08:39.660 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-03T03:08:39.660 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T03:08:39.710 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-06-03T03:08:39.710 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 1/170 2024-06-03T03:08:39.710 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2/170 2024-06-03T03:08:39.710 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 3/170 2024-06-03T03:08:39.710 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 4/170 2024-06-03T03:08:39.710 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 5/170 2024-06-03T03:08:39.710 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 6/170 2024-06-03T03:08:39.711 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 7/170 2024-06-03T03:08:39.711 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-03T03:08:39.711 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-03T03:08:39.711 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 10/170 2024-06-03T03:08:39.711 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 11/170 2024-06-03T03:08:39.711 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 12/170 2024-06-03T03:08:39.711 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 13/170 2024-06-03T03:08:39.711 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-03T03:08:39.711 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 15/170 2024-06-03T03:08:39.711 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 16/170 2024-06-03T03:08:39.711 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 17/170 2024-06-03T03:08:39.711 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 18/170 2024-06-03T03:08:39.711 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 19/170 2024-06-03T03:08:39.712 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 20/170 2024-06-03T03:08:39.712 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 21/170 2024-06-03T03:08:39.712 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 22/170 2024-06-03T03:08:39.712 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 23/170 2024-06-03T03:08:39.712 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 24/170 2024-06-03T03:08:39.712 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 25/170 2024-06-03T03:08:39.712 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 26/170 2024-06-03T03:08:39.712 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 27/170 2024-06-03T03:08:39.712 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 28/170 2024-06-03T03:08:39.712 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 29/170 2024-06-03T03:08:39.712 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 30/170 2024-06-03T03:08:39.712 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 31/170 2024-06-03T03:08:39.712 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 32/170 2024-06-03T03:08:39.712 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 33/170 2024-06-03T03:08:39.713 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 34/170 2024-06-03T03:08:39.713 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 35/170 2024-06-03T03:08:39.713 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 36/170 2024-06-03T03:08:39.713 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 37/170 2024-06-03T03:08:39.713 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 38/170 2024-06-03T03:08:39.713 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 39/170 2024-06-03T03:08:39.713 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-06-03T03:08:39.713 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-06-03T03:08:39.713 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-06-03T03:08:39.713 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-06-03T03:08:39.713 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-06-03T03:08:39.713 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-06-03T03:08:39.713 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-06-03T03:08:39.713 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-06-03T03:08:39.714 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-06-03T03:08:39.714 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-06-03T03:08:39.714 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-06-03T03:08:39.714 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-06-03T03:08:39.714 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-06-03T03:08:39.714 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-06-03T03:08:39.714 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-06-03T03:08:39.714 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-06-03T03:08:39.714 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-06-03T03:08:39.714 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-06-03T03:08:39.714 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-06-03T03:08:39.714 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-06-03T03:08:39.715 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-06-03T03:08:39.715 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-06-03T03:08:39.715 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-06-03T03:08:39.715 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-06-03T03:08:39.715 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-06-03T03:08:39.715 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-06-03T03:08:39.715 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-06-03T03:08:39.715 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-06-03T03:08:39.715 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-06-03T03:08:39.715 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-06-03T03:08:39.715 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-06-03T03:08:39.715 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-06-03T03:08:39.715 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-06-03T03:08:39.715 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-06-03T03:08:39.716 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-06-03T03:08:39.716 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-06-03T03:08:39.716 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-06-03T03:08:39.716 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-06-03T03:08:39.716 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-06-03T03:08:39.716 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-06-03T03:08:39.716 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-06-03T03:08:39.716 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-06-03T03:08:39.716 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-06-03T03:08:39.716 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-06-03T03:08:39.716 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-06-03T03:08:39.716 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-06-03T03:08:39.716 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-06-03T03:08:39.716 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-06-03T03:08:39.717 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-06-03T03:08:39.717 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-06-03T03:08:39.717 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-06-03T03:08:39.717 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-06-03T03:08:39.717 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-06-03T03:08:39.717 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-06-03T03:08:39.717 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-06-03T03:08:39.717 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-06-03T03:08:39.717 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-06-03T03:08:39.717 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-06-03T03:08:39.717 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-06-03T03:08:39.717 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-06-03T03:08:39.717 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-06-03T03:08:39.717 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-06-03T03:08:39.717 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-06-03T03:08:39.718 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-06-03T03:08:39.718 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-06-03T03:08:39.718 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-06-03T03:08:39.718 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-06-03T03:08:39.718 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-06-03T03:08:39.718 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-06-03T03:08:39.718 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-06-03T03:08:39.718 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-06-03T03:08:39.718 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-06-03T03:08:39.718 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-06-03T03:08:39.718 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-06-03T03:08:39.718 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-06-03T03:08:39.718 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-06-03T03:08:39.718 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-06-03T03:08:39.719 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-06-03T03:08:39.719 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-06-03T03:08:39.719 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-06-03T03:08:39.719 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-06-03T03:08:39.719 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-06-03T03:08:39.719 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/170 2024-06-03T03:08:39.719 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-06-03T03:08:39.719 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-06-03T03:08:39.719 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-06-03T03:08:39.719 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-06-03T03:08:39.719 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-06-03T03:08:39.719 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-06-03T03:08:39.719 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-06-03T03:08:39.719 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-06-03T03:08:39.720 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-06-03T03:08:39.720 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-06-03T03:08:39.720 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-06-03T03:08:39.720 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-06-03T03:08:39.720 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-06-03T03:08:39.720 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-06-03T03:08:39.720 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-06-03T03:08:39.720 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-06-03T03:08:39.720 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-06-03T03:08:39.720 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-06-03T03:08:39.720 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-06-03T03:08:39.720 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-06-03T03:08:39.720 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-06-03T03:08:39.720 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-06-03T03:08:39.720 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-06-03T03:08:39.721 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-06-03T03:08:39.721 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-06-03T03:08:39.721 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-06-03T03:08:39.721 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 149/170 2024-06-03T03:08:39.721 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-06-03T03:08:39.721 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 151/170 2024-06-03T03:08:39.721 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-06-03T03:08:39.721 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : NetworkManager-1:1.47.91-1.el9.x86_64 153/170 2024-06-03T03:08:39.721 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-06-03T03:08:39.721 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 155/170 2024-06-03T03:08:39.721 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-06-03T03:08:39.721 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : NetworkManager-team-1:1.47.91-1.el9.x86_64 157/170 2024-06-03T03:08:39.721 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-06-03T03:08:39.721 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : NetworkManager-tui-1:1.47.91-1.el9.x86_64 159/170 2024-06-03T03:08:39.722 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-06-03T03:08:39.722 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-06-03T03:08:39.722 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-06-03T03:08:39.722 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-06-03T03:08:39.722 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-06-03T03:08:39.722 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-06-03T03:08:39.722 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-06-03T03:08:39.722 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-06-03T03:08:39.722 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-06-03T03:08:39.722 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-06-03T03:08:39.723 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 135/170 2024-06-03T03:08:39.754 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 135/170 2024-06-03T03:08:39.904 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 136/170 2024-06-03T03:08:39.976 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 137/170 2024-06-03T03:08:40.057 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-06-03T03:08:40.057 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-06-03T03:08:40.057 INFO:teuthology.orchestra.run.smithi183.stdout:Upgraded: 2024-06-03T03:08:40.057 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager-1:1.47.91-1.el9.x86_64 2024-06-03T03:08:40.057 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 2024-06-03T03:08:40.057 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager-team-1:1.47.91-1.el9.x86_64 2024-06-03T03:08:40.058 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager-tui-1:1.47.91-1.el9.x86_64 2024-06-03T03:08:40.058 INFO:teuthology.orchestra.run.smithi183.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-06-03T03:08:40.058 INFO:teuthology.orchestra.run.smithi183.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-06-03T03:08:40.058 INFO:teuthology.orchestra.run.smithi183.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-06-03T03:08:40.058 INFO:teuthology.orchestra.run.smithi183.stdout: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:40.058 INFO:teuthology.orchestra.run.smithi183.stdout: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:40.058 INFO:teuthology.orchestra.run.smithi183.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-06-03T03:08:40.058 INFO:teuthology.orchestra.run.smithi183.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-06-03T03:08:40.058 INFO:teuthology.orchestra.run.smithi183.stdout:Installed: 2024-06-03T03:08:40.058 INFO:teuthology.orchestra.run.smithi183.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-06-03T03:08:40.058 INFO:teuthology.orchestra.run.smithi183.stdout: automake-1.16.2-8.el9.noarch 2024-06-03T03:08:40.059 INFO:teuthology.orchestra.run.smithi183.stdout: bison-3.7.4-5.el9.x86_64 2024-06-03T03:08:40.059 INFO:teuthology.orchestra.run.smithi183.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-06-03T03:08:40.059 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:40.059 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:40.059 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:40.059 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:40.059 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T03:08:40.059 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-immutable-object-cache-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:40.059 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:40.059 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:40.059 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T03:08:40.059 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T03:08:40.059 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T03:08:40.060 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T03:08:40.060 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T03:08:40.060 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:40.060 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:40.060 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T03:08:40.060 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:40.060 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:40.060 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:40.060 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T03:08:40.060 INFO:teuthology.orchestra.run.smithi183.stdout: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T03:08:40.060 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:40.060 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:40.060 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T03:08:40.061 INFO:teuthology.orchestra.run.smithi183.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-06-03T03:08:40.061 INFO:teuthology.orchestra.run.smithi183.stdout: flex-2.6.4-9.el9.x86_64 2024-06-03T03:08:40.061 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-06-03T03:08:40.061 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-06-03T03:08:40.061 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-06-03T03:08:40.061 INFO:teuthology.orchestra.run.smithi183.stdout: fuse-2.9.9-15.el9.x86_64 2024-06-03T03:08:40.061 INFO:teuthology.orchestra.run.smithi183.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-06-03T03:08:40.061 INFO:teuthology.orchestra.run.smithi183.stdout: grpc-data-1.46.7-10.el9.noarch 2024-06-03T03:08:40.061 INFO:teuthology.orchestra.run.smithi183.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-06-03T03:08:40.061 INFO:teuthology.orchestra.run.smithi183.stdout: libarrow-9.0.0-11.el9.x86_64 2024-06-03T03:08:40.061 INFO:teuthology.orchestra.run.smithi183.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-06-03T03:08:40.061 INFO:teuthology.orchestra.run.smithi183.stdout: libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:40.061 INFO:teuthology.orchestra.run.smithi183.stdout: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:40.062 INFO:teuthology.orchestra.run.smithi183.stdout: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:40.062 INFO:teuthology.orchestra.run.smithi183.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-06-03T03:08:40.062 INFO:teuthology.orchestra.run.smithi183.stdout: liboath-2.6.7-2.el9.x86_64 2024-06-03T03:08:40.062 INFO:teuthology.orchestra.run.smithi183.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-06-03T03:08:40.062 INFO:teuthology.orchestra.run.smithi183.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-06-03T03:08:40.062 INFO:teuthology.orchestra.run.smithi183.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-06-03T03:08:40.062 INFO:teuthology.orchestra.run.smithi183.stdout: librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:40.062 INFO:teuthology.orchestra.run.smithi183.stdout: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:40.062 INFO:teuthology.orchestra.run.smithi183.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-06-03T03:08:40.062 INFO:teuthology.orchestra.run.smithi183.stdout: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:40.062 INFO:teuthology.orchestra.run.smithi183.stdout: libtool-2.4.6-45.el9.x86_64 2024-06-03T03:08:40.062 INFO:teuthology.orchestra.run.smithi183.stdout: libunwind-1.6.2-1.el9.x86_64 2024-06-03T03:08:40.063 INFO:teuthology.orchestra.run.smithi183.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-06-03T03:08:40.063 INFO:teuthology.orchestra.run.smithi183.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-06-03T03:08:40.063 INFO:teuthology.orchestra.run.smithi183.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-06-03T03:08:40.063 INFO:teuthology.orchestra.run.smithi183.stdout: lua-5.4.4-4.el9.x86_64 2024-06-03T03:08:40.063 INFO:teuthology.orchestra.run.smithi183.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-06-03T03:08:40.063 INFO:teuthology.orchestra.run.smithi183.stdout: luarocks-3.9.2-5.el9.noarch 2024-06-03T03:08:40.063 INFO:teuthology.orchestra.run.smithi183.stdout: openblas-0.3.21-2.el9.x86_64 2024-06-03T03:08:40.063 INFO:teuthology.orchestra.run.smithi183.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-06-03T03:08:40.063 INFO:teuthology.orchestra.run.smithi183.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-06-03T03:08:40.063 INFO:teuthology.orchestra.run.smithi183.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-06-03T03:08:40.063 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-13.14-1.el9.x86_64 2024-06-03T03:08:40.063 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-06-03T03:08:40.063 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-06-03T03:08:40.064 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-server-13.14-1.el9.x86_64 2024-06-03T03:08:40.064 INFO:teuthology.orchestra.run.smithi183.stdout: protobuf-3.14.0-13.el9.x86_64 2024-06-03T03:08:40.064 INFO:teuthology.orchestra.run.smithi183.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-06-03T03:08:40.064 INFO:teuthology.orchestra.run.smithi183.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-06-03T03:08:40.064 INFO:teuthology.orchestra.run.smithi183.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-06-03T03:08:40.064 INFO:teuthology.orchestra.run.smithi183.stdout: python3-babel-2.9.1-2.el9.noarch 2024-06-03T03:08:40.064 INFO:teuthology.orchestra.run.smithi183.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-06-03T03:08:40.064 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-06-03T03:08:40.064 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:40.064 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:40.064 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:40.064 INFO:teuthology.orchestra.run.smithi183.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-06-03T03:08:40.064 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-06-03T03:08:40.064 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-06-03T03:08:40.065 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-06-03T03:08:40.065 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-06-03T03:08:40.065 INFO:teuthology.orchestra.run.smithi183.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-06-03T03:08:40.065 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-06-03T03:08:40.065 INFO:teuthology.orchestra.run.smithi183.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-06-03T03:08:40.065 INFO:teuthology.orchestra.run.smithi183.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-06-03T03:08:40.065 INFO:teuthology.orchestra.run.smithi183.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-06-03T03:08:40.065 INFO:teuthology.orchestra.run.smithi183.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-06-03T03:08:40.065 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-06-03T03:08:40.065 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-06-03T03:08:40.065 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-06-03T03:08:40.065 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-06-03T03:08:40.065 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-06-03T03:08:40.066 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-06-03T03:08:40.066 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-06-03T03:08:40.066 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-06-03T03:08:40.066 INFO:teuthology.orchestra.run.smithi183.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-06-03T03:08:40.066 INFO:teuthology.orchestra.run.smithi183.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-06-03T03:08:40.066 INFO:teuthology.orchestra.run.smithi183.stdout: python3-mako-1.1.4-6.el9.noarch 2024-06-03T03:08:40.066 INFO:teuthology.orchestra.run.smithi183.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-06-03T03:08:40.066 INFO:teuthology.orchestra.run.smithi183.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-06-03T03:08:40.066 INFO:teuthology.orchestra.run.smithi183.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-06-03T03:08:40.066 INFO:teuthology.orchestra.run.smithi183.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-06-03T03:08:40.066 INFO:teuthology.orchestra.run.smithi183.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-06-03T03:08:40.066 INFO:teuthology.orchestra.run.smithi183.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-06-03T03:08:40.066 INFO:teuthology.orchestra.run.smithi183.stdout: python3-packaging-20.9-5.el9.noarch 2024-06-03T03:08:40.067 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-06-03T03:08:40.067 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ply-3.11-14.el9.noarch 2024-06-03T03:08:40.067 INFO:teuthology.orchestra.run.smithi183.stdout: python3-portend-3.1.0-2.el9.noarch 2024-06-03T03:08:40.067 INFO:teuthology.orchestra.run.smithi183.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-06-03T03:08:40.067 INFO:teuthology.orchestra.run.smithi183.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-06-03T03:08:40.067 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-06-03T03:08:40.067 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-06-03T03:08:40.067 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-06-03T03:08:40.067 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pycparser-2.20-6.el9.noarch 2024-06-03T03:08:40.067 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-06-03T03:08:40.067 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pytz-2021.1-5.el9.noarch 2024-06-03T03:08:40.067 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:40.067 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:40.068 INFO:teuthology.orchestra.run.smithi183.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-06-03T03:08:40.068 INFO:teuthology.orchestra.run.smithi183.stdout: python3-requests-2.25.1-8.el9.noarch 2024-06-03T03:08:40.068 INFO:teuthology.orchestra.run.smithi183.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-06-03T03:08:40.068 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:40.068 INFO:teuthology.orchestra.run.smithi183.stdout: python3-routes-2.5.1-5.el9.noarch 2024-06-03T03:08:40.068 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rsa-4.9-2.el9.noarch 2024-06-03T03:08:40.068 INFO:teuthology.orchestra.run.smithi183.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-06-03T03:08:40.068 INFO:teuthology.orchestra.run.smithi183.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-06-03T03:08:40.068 INFO:teuthology.orchestra.run.smithi183.stdout: python3-toml-0.10.2-6.el9.noarch 2024-06-03T03:08:40.068 INFO:teuthology.orchestra.run.smithi183.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-06-03T03:08:40.068 INFO:teuthology.orchestra.run.smithi183.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-06-03T03:08:40.068 INFO:teuthology.orchestra.run.smithi183.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-06-03T03:08:40.068 INFO:teuthology.orchestra.run.smithi183.stdout: python3-webob-1.8.7-6.el9.noarch 2024-06-03T03:08:40.068 INFO:teuthology.orchestra.run.smithi183.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-06-03T03:08:40.069 INFO:teuthology.orchestra.run.smithi183.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-06-03T03:08:40.069 INFO:teuthology.orchestra.run.smithi183.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-06-03T03:08:40.069 INFO:teuthology.orchestra.run.smithi183.stdout: qatlib-24.02.0-1.el9.x86_64 2024-06-03T03:08:40.069 INFO:teuthology.orchestra.run.smithi183.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-06-03T03:08:40.069 INFO:teuthology.orchestra.run.smithi183.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-06-03T03:08:40.069 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:40.069 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:40.069 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:40.069 INFO:teuthology.orchestra.run.smithi183.stdout: re2-1:20211101-3.el9.x86_64 2024-06-03T03:08:40.069 INFO:teuthology.orchestra.run.smithi183.stdout: socat-1.7.4.1-5.el9.x86_64 2024-06-03T03:08:40.069 INFO:teuthology.orchestra.run.smithi183.stdout: thrift-0.15.0-2.el9.x86_64 2024-06-03T03:08:40.069 INFO:teuthology.orchestra.run.smithi183.stdout: uuid-1.6.2-55.el9.x86_64 2024-06-03T03:08:40.069 INFO:teuthology.orchestra.run.smithi183.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-06-03T03:08:40.069 INFO:teuthology.orchestra.run.smithi183.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-06-03T03:08:40.070 INFO:teuthology.orchestra.run.smithi183.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-06-03T03:08:40.070 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-06-03T03:08:40.070 INFO:teuthology.orchestra.run.smithi183.stdout:Complete! 2024-06-03T03:08:40.512 DEBUG:teuthology.parallel:result is None 2024-06-03T03:08:40.956 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-06-03T03:08:40.985 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 139/170 2024-06-03T03:08:41.052 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 139/170 2024-06-03T03:08:41.053 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T03:08:41.053 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-06-03T03:08:41.053 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-03T03:08:41.053 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-03T03:08:41.053 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T03:08:41.110 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 140/170 2024-06-03T03:08:41.179 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 140/170 2024-06-03T03:08:41.179 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T03:08:41.179 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-06-03T03:08:41.179 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T03:08:41.236 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 141/170 2024-06-03T03:08:41.310 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 141/170 2024-06-03T03:08:41.310 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T03:08:41.311 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-06-03T03:08:41.311 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-03T03:08:41.311 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-03T03:08:41.311 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T03:08:41.686 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/170 2024-06-03T03:08:41.756 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/170 2024-06-03T03:08:41.757 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T03:08:41.757 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-06-03T03:08:41.757 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-03T03:08:41.757 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-03T03:08:41.757 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T03:08:42.342 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-06-03T03:08:42.343 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-06-03T03:08:42.343 INFO:teuthology.orchestra.run.smithi151.stdout:Upgraded: 2024-06-03T03:08:42.343 INFO:teuthology.orchestra.run.smithi151.stdout: NetworkManager-1:1.47.91-1.el9.x86_64 2024-06-03T03:08:42.343 INFO:teuthology.orchestra.run.smithi151.stdout: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 2024-06-03T03:08:42.343 INFO:teuthology.orchestra.run.smithi151.stdout: NetworkManager-team-1:1.47.91-1.el9.x86_64 2024-06-03T03:08:42.343 INFO:teuthology.orchestra.run.smithi151.stdout: NetworkManager-tui-1:1.47.91-1.el9.x86_64 2024-06-03T03:08:42.343 INFO:teuthology.orchestra.run.smithi151.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-06-03T03:08:42.343 INFO:teuthology.orchestra.run.smithi151.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-06-03T03:08:42.343 INFO:teuthology.orchestra.run.smithi151.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-06-03T03:08:42.343 INFO:teuthology.orchestra.run.smithi151.stdout: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:42.343 INFO:teuthology.orchestra.run.smithi151.stdout: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:42.344 INFO:teuthology.orchestra.run.smithi151.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-06-03T03:08:42.344 INFO:teuthology.orchestra.run.smithi151.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-06-03T03:08:42.344 INFO:teuthology.orchestra.run.smithi151.stdout:Installed: 2024-06-03T03:08:42.344 INFO:teuthology.orchestra.run.smithi151.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-06-03T03:08:42.344 INFO:teuthology.orchestra.run.smithi151.stdout: automake-1.16.2-8.el9.noarch 2024-06-03T03:08:42.344 INFO:teuthology.orchestra.run.smithi151.stdout: bison-3.7.4-5.el9.x86_64 2024-06-03T03:08:42.344 INFO:teuthology.orchestra.run.smithi151.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-06-03T03:08:42.344 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:42.344 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:42.344 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:42.344 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:42.344 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T03:08:42.344 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-immutable-object-cache-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:42.345 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:42.345 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:42.345 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T03:08:42.345 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T03:08:42.345 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T03:08:42.345 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T03:08:42.345 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T03:08:42.345 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:42.345 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:42.345 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T03:08:42.345 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:42.345 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:42.346 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:42.346 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T03:08:42.346 INFO:teuthology.orchestra.run.smithi151.stdout: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T03:08:42.346 INFO:teuthology.orchestra.run.smithi151.stdout: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:42.346 INFO:teuthology.orchestra.run.smithi151.stdout: cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:42.346 INFO:teuthology.orchestra.run.smithi151.stdout: cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T03:08:42.346 INFO:teuthology.orchestra.run.smithi151.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-06-03T03:08:42.346 INFO:teuthology.orchestra.run.smithi151.stdout: flex-2.6.4-9.el9.x86_64 2024-06-03T03:08:42.346 INFO:teuthology.orchestra.run.smithi151.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-06-03T03:08:42.346 INFO:teuthology.orchestra.run.smithi151.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-06-03T03:08:42.346 INFO:teuthology.orchestra.run.smithi151.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-06-03T03:08:42.346 INFO:teuthology.orchestra.run.smithi151.stdout: fuse-2.9.9-15.el9.x86_64 2024-06-03T03:08:42.346 INFO:teuthology.orchestra.run.smithi151.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-06-03T03:08:42.347 INFO:teuthology.orchestra.run.smithi151.stdout: grpc-data-1.46.7-10.el9.noarch 2024-06-03T03:08:42.347 INFO:teuthology.orchestra.run.smithi151.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-06-03T03:08:42.347 INFO:teuthology.orchestra.run.smithi151.stdout: libarrow-9.0.0-11.el9.x86_64 2024-06-03T03:08:42.347 INFO:teuthology.orchestra.run.smithi151.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-06-03T03:08:42.347 INFO:teuthology.orchestra.run.smithi151.stdout: libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:42.347 INFO:teuthology.orchestra.run.smithi151.stdout: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:42.347 INFO:teuthology.orchestra.run.smithi151.stdout: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:42.347 INFO:teuthology.orchestra.run.smithi151.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-06-03T03:08:42.347 INFO:teuthology.orchestra.run.smithi151.stdout: liboath-2.6.7-2.el9.x86_64 2024-06-03T03:08:42.347 INFO:teuthology.orchestra.run.smithi151.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-06-03T03:08:42.347 INFO:teuthology.orchestra.run.smithi151.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-06-03T03:08:42.347 INFO:teuthology.orchestra.run.smithi151.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-06-03T03:08:42.348 INFO:teuthology.orchestra.run.smithi151.stdout: librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:42.348 INFO:teuthology.orchestra.run.smithi151.stdout: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:42.348 INFO:teuthology.orchestra.run.smithi151.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-06-03T03:08:42.348 INFO:teuthology.orchestra.run.smithi151.stdout: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:42.348 INFO:teuthology.orchestra.run.smithi151.stdout: libtool-2.4.6-45.el9.x86_64 2024-06-03T03:08:42.348 INFO:teuthology.orchestra.run.smithi151.stdout: libunwind-1.6.2-1.el9.x86_64 2024-06-03T03:08:42.348 INFO:teuthology.orchestra.run.smithi151.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-06-03T03:08:42.348 INFO:teuthology.orchestra.run.smithi151.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-06-03T03:08:42.348 INFO:teuthology.orchestra.run.smithi151.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-06-03T03:08:42.348 INFO:teuthology.orchestra.run.smithi151.stdout: lua-5.4.4-4.el9.x86_64 2024-06-03T03:08:42.348 INFO:teuthology.orchestra.run.smithi151.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-06-03T03:08:42.348 INFO:teuthology.orchestra.run.smithi151.stdout: luarocks-3.9.2-5.el9.noarch 2024-06-03T03:08:42.348 INFO:teuthology.orchestra.run.smithi151.stdout: openblas-0.3.21-2.el9.x86_64 2024-06-03T03:08:42.349 INFO:teuthology.orchestra.run.smithi151.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-06-03T03:08:42.349 INFO:teuthology.orchestra.run.smithi151.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-06-03T03:08:42.349 INFO:teuthology.orchestra.run.smithi151.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-06-03T03:08:42.349 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql-13.14-1.el9.x86_64 2024-06-03T03:08:42.349 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-06-03T03:08:42.349 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-06-03T03:08:42.349 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql-server-13.14-1.el9.x86_64 2024-06-03T03:08:42.349 INFO:teuthology.orchestra.run.smithi151.stdout: protobuf-3.14.0-13.el9.x86_64 2024-06-03T03:08:42.349 INFO:teuthology.orchestra.run.smithi151.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-06-03T03:08:42.349 INFO:teuthology.orchestra.run.smithi151.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-06-03T03:08:42.349 INFO:teuthology.orchestra.run.smithi151.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-06-03T03:08:42.349 INFO:teuthology.orchestra.run.smithi151.stdout: python3-babel-2.9.1-2.el9.noarch 2024-06-03T03:08:42.349 INFO:teuthology.orchestra.run.smithi151.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-06-03T03:08:42.350 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-06-03T03:08:42.350 INFO:teuthology.orchestra.run.smithi151.stdout: python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:42.350 INFO:teuthology.orchestra.run.smithi151.stdout: python3-ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:42.350 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:42.350 INFO:teuthology.orchestra.run.smithi151.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-06-03T03:08:42.350 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-06-03T03:08:42.350 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-06-03T03:08:42.350 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-06-03T03:08:42.350 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-06-03T03:08:42.350 INFO:teuthology.orchestra.run.smithi151.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-06-03T03:08:42.350 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-06-03T03:08:42.350 INFO:teuthology.orchestra.run.smithi151.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-06-03T03:08:42.350 INFO:teuthology.orchestra.run.smithi151.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-06-03T03:08:42.351 INFO:teuthology.orchestra.run.smithi151.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-06-03T03:08:42.351 INFO:teuthology.orchestra.run.smithi151.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-06-03T03:08:42.351 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-06-03T03:08:42.351 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-06-03T03:08:42.351 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-06-03T03:08:42.351 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-06-03T03:08:42.351 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-06-03T03:08:42.351 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-06-03T03:08:42.351 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-06-03T03:08:42.351 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-06-03T03:08:42.351 INFO:teuthology.orchestra.run.smithi151.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-06-03T03:08:42.351 INFO:teuthology.orchestra.run.smithi151.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-06-03T03:08:42.351 INFO:teuthology.orchestra.run.smithi151.stdout: python3-mako-1.1.4-6.el9.noarch 2024-06-03T03:08:42.352 INFO:teuthology.orchestra.run.smithi151.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-06-03T03:08:42.352 INFO:teuthology.orchestra.run.smithi151.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-06-03T03:08:42.352 INFO:teuthology.orchestra.run.smithi151.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-06-03T03:08:42.352 INFO:teuthology.orchestra.run.smithi151.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-06-03T03:08:42.352 INFO:teuthology.orchestra.run.smithi151.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-06-03T03:08:42.352 INFO:teuthology.orchestra.run.smithi151.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-06-03T03:08:42.352 INFO:teuthology.orchestra.run.smithi151.stdout: python3-packaging-20.9-5.el9.noarch 2024-06-03T03:08:42.352 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-06-03T03:08:42.352 INFO:teuthology.orchestra.run.smithi151.stdout: python3-ply-3.11-14.el9.noarch 2024-06-03T03:08:42.352 INFO:teuthology.orchestra.run.smithi151.stdout: python3-portend-3.1.0-2.el9.noarch 2024-06-03T03:08:42.352 INFO:teuthology.orchestra.run.smithi151.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-06-03T03:08:42.352 INFO:teuthology.orchestra.run.smithi151.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-06-03T03:08:42.352 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-06-03T03:08:42.353 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-06-03T03:08:42.353 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-06-03T03:08:42.353 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pycparser-2.20-6.el9.noarch 2024-06-03T03:08:42.353 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-06-03T03:08:42.353 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pytz-2021.1-5.el9.noarch 2024-06-03T03:08:42.353 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:42.353 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:42.353 INFO:teuthology.orchestra.run.smithi151.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-06-03T03:08:42.353 INFO:teuthology.orchestra.run.smithi151.stdout: python3-requests-2.25.1-8.el9.noarch 2024-06-03T03:08:42.353 INFO:teuthology.orchestra.run.smithi151.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-06-03T03:08:42.353 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:42.353 INFO:teuthology.orchestra.run.smithi151.stdout: python3-routes-2.5.1-5.el9.noarch 2024-06-03T03:08:42.353 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rsa-4.9-2.el9.noarch 2024-06-03T03:08:42.353 INFO:teuthology.orchestra.run.smithi151.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-06-03T03:08:42.354 INFO:teuthology.orchestra.run.smithi151.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-06-03T03:08:42.354 INFO:teuthology.orchestra.run.smithi151.stdout: python3-toml-0.10.2-6.el9.noarch 2024-06-03T03:08:42.354 INFO:teuthology.orchestra.run.smithi151.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-06-03T03:08:42.354 INFO:teuthology.orchestra.run.smithi151.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-06-03T03:08:42.354 INFO:teuthology.orchestra.run.smithi151.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-06-03T03:08:42.354 INFO:teuthology.orchestra.run.smithi151.stdout: python3-webob-1.8.7-6.el9.noarch 2024-06-03T03:08:42.354 INFO:teuthology.orchestra.run.smithi151.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-06-03T03:08:42.354 INFO:teuthology.orchestra.run.smithi151.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-06-03T03:08:42.354 INFO:teuthology.orchestra.run.smithi151.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-06-03T03:08:42.354 INFO:teuthology.orchestra.run.smithi151.stdout: qatlib-24.02.0-1.el9.x86_64 2024-06-03T03:08:42.354 INFO:teuthology.orchestra.run.smithi151.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-06-03T03:08:42.354 INFO:teuthology.orchestra.run.smithi151.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-06-03T03:08:42.354 INFO:teuthology.orchestra.run.smithi151.stdout: rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:42.355 INFO:teuthology.orchestra.run.smithi151.stdout: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:42.355 INFO:teuthology.orchestra.run.smithi151.stdout: rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:42.355 INFO:teuthology.orchestra.run.smithi151.stdout: re2-1:20211101-3.el9.x86_64 2024-06-03T03:08:42.355 INFO:teuthology.orchestra.run.smithi151.stdout: socat-1.7.4.1-5.el9.x86_64 2024-06-03T03:08:42.355 INFO:teuthology.orchestra.run.smithi151.stdout: thrift-0.15.0-2.el9.x86_64 2024-06-03T03:08:42.355 INFO:teuthology.orchestra.run.smithi151.stdout: uuid-1.6.2-55.el9.x86_64 2024-06-03T03:08:42.355 INFO:teuthology.orchestra.run.smithi151.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-06-03T03:08:42.355 INFO:teuthology.orchestra.run.smithi151.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-06-03T03:08:42.355 INFO:teuthology.orchestra.run.smithi151.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-06-03T03:08:42.355 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-06-03T03:08:42.356 INFO:teuthology.orchestra.run.smithi151.stdout:Complete! 2024-06-03T03:08:42.851 DEBUG:teuthology.parallel:result is None 2024-06-03T03:08:46.624 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 143/170 2024-06-03T03:08:46.692 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 144/170 2024-06-03T03:08:46.755 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 145/170 2024-06-03T03:08:46.790 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-06-03T03:08:46.829 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-06-03T03:08:47.091 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 148/170 2024-06-03T03:08:47.145 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-06-03T03:08:47.179 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 150/170 2024-06-03T03:08:47.235 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-06-03T03:08:47.308 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : NetworkManager-team-1:1.47.91-1.el9.x86_64 152/170 2024-06-03T03:08:47.353 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : NetworkManager-tui-1:1.47.91-1.el9.x86_64 153/170 2024-06-03T03:08:48.387 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-03T03:08:48.433 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-03T03:08:48.497 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-03T03:08:49.888 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 155/170 2024-06-03T03:08:49.998 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-06-03T03:08:50.060 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-06-03T03:08:50.220 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-06-03T03:08:50.258 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-06-03T03:08:50.290 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-06-03T03:08:50.325 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-06-03T03:08:50.374 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-06-03T03:08:50.374 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-06-03T03:08:50.516 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-06-03T03:08:50.562 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-06-03T03:08:50.563 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-06-03T03:08:50.691 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-06-03T03:08:50.755 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-06-03T03:08:50.788 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-03T03:08:50.788 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-03T03:08:51.134 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-03T03:08:51.135 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-06-03T03:08:51.271 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-06-03T03:08:51.316 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-06-03T03:08:51.454 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-06-03T03:08:51.704 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-06-03T03:08:51.704 INFO:teuthology.orchestra.run.smithi117.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-06-03T03:08:51.704 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T03:08:57.265 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 1/170 2024-06-03T03:08:57.265 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2/170 2024-06-03T03:08:57.265 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 3/170 2024-06-03T03:08:57.265 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 4/170 2024-06-03T03:08:57.266 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 5/170 2024-06-03T03:08:57.266 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 6/170 2024-06-03T03:08:57.266 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 7/170 2024-06-03T03:08:57.266 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-03T03:08:57.266 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-03T03:08:57.266 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 10/170 2024-06-03T03:08:57.266 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 11/170 2024-06-03T03:08:57.266 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 12/170 2024-06-03T03:08:57.266 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 13/170 2024-06-03T03:08:57.266 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-03T03:08:57.266 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 15/170 2024-06-03T03:08:57.266 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 16/170 2024-06-03T03:08:57.267 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 17/170 2024-06-03T03:08:57.267 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 18/170 2024-06-03T03:08:57.267 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 19/170 2024-06-03T03:08:57.267 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 20/170 2024-06-03T03:08:57.267 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 21/170 2024-06-03T03:08:57.267 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 22/170 2024-06-03T03:08:57.267 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 23/170 2024-06-03T03:08:57.267 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 24/170 2024-06-03T03:08:57.267 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 25/170 2024-06-03T03:08:57.267 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 26/170 2024-06-03T03:08:57.267 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 27/170 2024-06-03T03:08:57.267 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 28/170 2024-06-03T03:08:57.267 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 29/170 2024-06-03T03:08:57.268 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 30/170 2024-06-03T03:08:57.268 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 31/170 2024-06-03T03:08:57.268 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 32/170 2024-06-03T03:08:57.268 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 33/170 2024-06-03T03:08:57.268 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 34/170 2024-06-03T03:08:57.268 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 35/170 2024-06-03T03:08:57.268 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 36/170 2024-06-03T03:08:57.268 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 37/170 2024-06-03T03:08:57.268 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 38/170 2024-06-03T03:08:57.268 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 39/170 2024-06-03T03:08:57.268 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-06-03T03:08:57.268 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-06-03T03:08:57.268 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-06-03T03:08:57.268 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-06-03T03:08:57.269 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-06-03T03:08:57.269 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-06-03T03:08:57.269 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-06-03T03:08:57.269 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-06-03T03:08:57.269 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-06-03T03:08:57.269 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-06-03T03:08:57.269 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-06-03T03:08:57.269 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-06-03T03:08:57.269 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-06-03T03:08:57.269 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-06-03T03:08:57.269 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-06-03T03:08:57.269 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-06-03T03:08:57.269 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-06-03T03:08:57.270 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-06-03T03:08:57.270 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-06-03T03:08:57.270 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-06-03T03:08:57.270 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-06-03T03:08:57.270 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-06-03T03:08:57.270 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-06-03T03:08:57.270 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-06-03T03:08:57.270 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-06-03T03:08:57.270 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-06-03T03:08:57.270 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-06-03T03:08:57.270 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-06-03T03:08:57.270 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-06-03T03:08:57.271 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-06-03T03:08:57.271 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-06-03T03:08:57.271 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-06-03T03:08:57.271 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-06-03T03:08:57.271 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-06-03T03:08:57.271 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-06-03T03:08:57.271 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-06-03T03:08:57.271 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-06-03T03:08:57.271 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-06-03T03:08:57.271 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-06-03T03:08:57.271 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-06-03T03:08:57.271 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-06-03T03:08:57.271 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-06-03T03:08:57.272 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-06-03T03:08:57.272 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-06-03T03:08:57.272 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-06-03T03:08:57.272 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-06-03T03:08:57.272 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-06-03T03:08:57.272 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-06-03T03:08:57.272 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-06-03T03:08:57.272 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-06-03T03:08:57.272 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-06-03T03:08:57.272 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-06-03T03:08:57.272 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-06-03T03:08:57.272 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-06-03T03:08:57.273 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-06-03T03:08:57.273 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-06-03T03:08:57.273 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-06-03T03:08:57.273 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-06-03T03:08:57.273 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-06-03T03:08:57.273 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-06-03T03:08:57.273 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-06-03T03:08:57.273 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-06-03T03:08:57.273 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-06-03T03:08:57.273 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-06-03T03:08:57.273 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-06-03T03:08:57.273 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-06-03T03:08:57.274 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-06-03T03:08:57.274 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-06-03T03:08:57.274 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-06-03T03:08:57.274 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-06-03T03:08:57.274 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-06-03T03:08:57.274 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-06-03T03:08:57.274 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-06-03T03:08:57.274 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-06-03T03:08:57.274 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-06-03T03:08:57.274 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-06-03T03:08:57.274 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-06-03T03:08:57.274 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-06-03T03:08:57.275 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-06-03T03:08:57.275 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-06-03T03:08:57.275 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-06-03T03:08:57.275 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-06-03T03:08:57.275 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/170 2024-06-03T03:08:57.275 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-06-03T03:08:57.275 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-06-03T03:08:57.275 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-06-03T03:08:57.275 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-06-03T03:08:57.275 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-06-03T03:08:57.275 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-06-03T03:08:57.275 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-06-03T03:08:57.275 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-06-03T03:08:57.276 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-06-03T03:08:57.276 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-06-03T03:08:57.276 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-06-03T03:08:57.276 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-06-03T03:08:57.276 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-06-03T03:08:57.276 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-06-03T03:08:57.276 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-06-03T03:08:57.276 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-06-03T03:08:57.276 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-06-03T03:08:57.276 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-06-03T03:08:57.276 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-06-03T03:08:57.276 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-06-03T03:08:57.277 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-06-03T03:08:57.277 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-06-03T03:08:57.277 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-06-03T03:08:57.277 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-06-03T03:08:57.277 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-06-03T03:08:57.277 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-06-03T03:08:57.277 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 149/170 2024-06-03T03:08:57.277 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-06-03T03:08:57.277 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 151/170 2024-06-03T03:08:57.277 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-06-03T03:08:57.277 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : NetworkManager-1:1.47.91-1.el9.x86_64 153/170 2024-06-03T03:08:57.277 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-06-03T03:08:57.277 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 155/170 2024-06-03T03:08:57.278 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-06-03T03:08:57.278 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : NetworkManager-team-1:1.47.91-1.el9.x86_64 157/170 2024-06-03T03:08:57.278 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-06-03T03:08:57.278 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : NetworkManager-tui-1:1.47.91-1.el9.x86_64 159/170 2024-06-03T03:08:57.278 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-06-03T03:08:57.278 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-06-03T03:08:57.278 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-06-03T03:08:57.278 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-06-03T03:08:57.278 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-06-03T03:08:57.278 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-06-03T03:08:57.278 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-06-03T03:08:57.278 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-06-03T03:08:57.278 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-06-03T03:08:57.279 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-06-03T03:08:59.614 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-06-03T03:08:59.614 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T03:08:59.614 INFO:teuthology.orchestra.run.smithi117.stdout:Upgraded: 2024-06-03T03:08:59.614 INFO:teuthology.orchestra.run.smithi117.stdout: NetworkManager-1:1.47.91-1.el9.x86_64 2024-06-03T03:08:59.615 INFO:teuthology.orchestra.run.smithi117.stdout: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 2024-06-03T03:08:59.615 INFO:teuthology.orchestra.run.smithi117.stdout: NetworkManager-team-1:1.47.91-1.el9.x86_64 2024-06-03T03:08:59.615 INFO:teuthology.orchestra.run.smithi117.stdout: NetworkManager-tui-1:1.47.91-1.el9.x86_64 2024-06-03T03:08:59.615 INFO:teuthology.orchestra.run.smithi117.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-06-03T03:08:59.615 INFO:teuthology.orchestra.run.smithi117.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-06-03T03:08:59.615 INFO:teuthology.orchestra.run.smithi117.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-06-03T03:08:59.615 INFO:teuthology.orchestra.run.smithi117.stdout: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:59.615 INFO:teuthology.orchestra.run.smithi117.stdout: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:59.615 INFO:teuthology.orchestra.run.smithi117.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-06-03T03:08:59.615 INFO:teuthology.orchestra.run.smithi117.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-06-03T03:08:59.615 INFO:teuthology.orchestra.run.smithi117.stdout:Installed: 2024-06-03T03:08:59.615 INFO:teuthology.orchestra.run.smithi117.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-06-03T03:08:59.615 INFO:teuthology.orchestra.run.smithi117.stdout: automake-1.16.2-8.el9.noarch 2024-06-03T03:08:59.616 INFO:teuthology.orchestra.run.smithi117.stdout: bison-3.7.4-5.el9.x86_64 2024-06-03T03:08:59.616 INFO:teuthology.orchestra.run.smithi117.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-06-03T03:08:59.616 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:59.616 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:59.616 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:59.616 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:59.616 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T03:08:59.616 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-immutable-object-cache-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:59.616 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:59.616 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:59.616 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T03:08:59.616 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T03:08:59.616 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T03:08:59.617 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T03:08:59.617 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T03:08:59.617 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:59.617 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:59.617 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T03:08:59.617 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:59.617 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:59.617 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:59.617 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T03:08:59.617 INFO:teuthology.orchestra.run.smithi117.stdout: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T03:08:59.617 INFO:teuthology.orchestra.run.smithi117.stdout: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:59.617 INFO:teuthology.orchestra.run.smithi117.stdout: cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:59.617 INFO:teuthology.orchestra.run.smithi117.stdout: cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T03:08:59.618 INFO:teuthology.orchestra.run.smithi117.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-06-03T03:08:59.618 INFO:teuthology.orchestra.run.smithi117.stdout: flex-2.6.4-9.el9.x86_64 2024-06-03T03:08:59.618 INFO:teuthology.orchestra.run.smithi117.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-06-03T03:08:59.618 INFO:teuthology.orchestra.run.smithi117.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-06-03T03:08:59.618 INFO:teuthology.orchestra.run.smithi117.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-06-03T03:08:59.618 INFO:teuthology.orchestra.run.smithi117.stdout: fuse-2.9.9-15.el9.x86_64 2024-06-03T03:08:59.618 INFO:teuthology.orchestra.run.smithi117.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-06-03T03:08:59.618 INFO:teuthology.orchestra.run.smithi117.stdout: grpc-data-1.46.7-10.el9.noarch 2024-06-03T03:08:59.618 INFO:teuthology.orchestra.run.smithi117.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-06-03T03:08:59.618 INFO:teuthology.orchestra.run.smithi117.stdout: libarrow-9.0.0-11.el9.x86_64 2024-06-03T03:08:59.618 INFO:teuthology.orchestra.run.smithi117.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-06-03T03:08:59.618 INFO:teuthology.orchestra.run.smithi117.stdout: libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:59.619 INFO:teuthology.orchestra.run.smithi117.stdout: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:59.619 INFO:teuthology.orchestra.run.smithi117.stdout: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:59.619 INFO:teuthology.orchestra.run.smithi117.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-06-03T03:08:59.619 INFO:teuthology.orchestra.run.smithi117.stdout: liboath-2.6.7-2.el9.x86_64 2024-06-03T03:08:59.619 INFO:teuthology.orchestra.run.smithi117.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-06-03T03:08:59.619 INFO:teuthology.orchestra.run.smithi117.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-06-03T03:08:59.619 INFO:teuthology.orchestra.run.smithi117.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-06-03T03:08:59.619 INFO:teuthology.orchestra.run.smithi117.stdout: librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:59.619 INFO:teuthology.orchestra.run.smithi117.stdout: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:59.619 INFO:teuthology.orchestra.run.smithi117.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-06-03T03:08:59.619 INFO:teuthology.orchestra.run.smithi117.stdout: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:59.619 INFO:teuthology.orchestra.run.smithi117.stdout: libtool-2.4.6-45.el9.x86_64 2024-06-03T03:08:59.619 INFO:teuthology.orchestra.run.smithi117.stdout: libunwind-1.6.2-1.el9.x86_64 2024-06-03T03:08:59.619 INFO:teuthology.orchestra.run.smithi117.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-06-03T03:08:59.620 INFO:teuthology.orchestra.run.smithi117.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-06-03T03:08:59.620 INFO:teuthology.orchestra.run.smithi117.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-06-03T03:08:59.620 INFO:teuthology.orchestra.run.smithi117.stdout: lua-5.4.4-4.el9.x86_64 2024-06-03T03:08:59.620 INFO:teuthology.orchestra.run.smithi117.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-06-03T03:08:59.620 INFO:teuthology.orchestra.run.smithi117.stdout: luarocks-3.9.2-5.el9.noarch 2024-06-03T03:08:59.620 INFO:teuthology.orchestra.run.smithi117.stdout: openblas-0.3.21-2.el9.x86_64 2024-06-03T03:08:59.620 INFO:teuthology.orchestra.run.smithi117.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-06-03T03:08:59.620 INFO:teuthology.orchestra.run.smithi117.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-06-03T03:08:59.620 INFO:teuthology.orchestra.run.smithi117.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-06-03T03:08:59.620 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-13.14-1.el9.x86_64 2024-06-03T03:08:59.620 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-06-03T03:08:59.620 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-06-03T03:08:59.620 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-server-13.14-1.el9.x86_64 2024-06-03T03:08:59.620 INFO:teuthology.orchestra.run.smithi117.stdout: protobuf-3.14.0-13.el9.x86_64 2024-06-03T03:08:59.621 INFO:teuthology.orchestra.run.smithi117.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-06-03T03:08:59.621 INFO:teuthology.orchestra.run.smithi117.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-06-03T03:08:59.621 INFO:teuthology.orchestra.run.smithi117.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-06-03T03:08:59.621 INFO:teuthology.orchestra.run.smithi117.stdout: python3-babel-2.9.1-2.el9.noarch 2024-06-03T03:08:59.621 INFO:teuthology.orchestra.run.smithi117.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-06-03T03:08:59.621 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-06-03T03:08:59.621 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:59.621 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:59.621 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:59.621 INFO:teuthology.orchestra.run.smithi117.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-06-03T03:08:59.621 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-06-03T03:08:59.621 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-06-03T03:08:59.621 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-06-03T03:08:59.622 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-06-03T03:08:59.622 INFO:teuthology.orchestra.run.smithi117.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-06-03T03:08:59.622 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-06-03T03:08:59.622 INFO:teuthology.orchestra.run.smithi117.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-06-03T03:08:59.622 INFO:teuthology.orchestra.run.smithi117.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-06-03T03:08:59.622 INFO:teuthology.orchestra.run.smithi117.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-06-03T03:08:59.622 INFO:teuthology.orchestra.run.smithi117.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-06-03T03:08:59.622 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-06-03T03:08:59.622 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-06-03T03:08:59.622 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-06-03T03:08:59.622 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-06-03T03:08:59.622 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-06-03T03:08:59.623 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-06-03T03:08:59.623 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-06-03T03:08:59.623 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-06-03T03:08:59.623 INFO:teuthology.orchestra.run.smithi117.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-06-03T03:08:59.623 INFO:teuthology.orchestra.run.smithi117.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-06-03T03:08:59.623 INFO:teuthology.orchestra.run.smithi117.stdout: python3-mako-1.1.4-6.el9.noarch 2024-06-03T03:08:59.623 INFO:teuthology.orchestra.run.smithi117.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-06-03T03:08:59.623 INFO:teuthology.orchestra.run.smithi117.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-06-03T03:08:59.623 INFO:teuthology.orchestra.run.smithi117.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-06-03T03:08:59.623 INFO:teuthology.orchestra.run.smithi117.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-06-03T03:08:59.623 INFO:teuthology.orchestra.run.smithi117.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-06-03T03:08:59.623 INFO:teuthology.orchestra.run.smithi117.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-06-03T03:08:59.623 INFO:teuthology.orchestra.run.smithi117.stdout: python3-packaging-20.9-5.el9.noarch 2024-06-03T03:08:59.624 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-06-03T03:08:59.624 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ply-3.11-14.el9.noarch 2024-06-03T03:08:59.624 INFO:teuthology.orchestra.run.smithi117.stdout: python3-portend-3.1.0-2.el9.noarch 2024-06-03T03:08:59.624 INFO:teuthology.orchestra.run.smithi117.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-06-03T03:08:59.624 INFO:teuthology.orchestra.run.smithi117.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-06-03T03:08:59.624 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-06-03T03:08:59.624 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-06-03T03:08:59.624 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-06-03T03:08:59.624 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pycparser-2.20-6.el9.noarch 2024-06-03T03:08:59.624 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-06-03T03:08:59.624 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pytz-2021.1-5.el9.noarch 2024-06-03T03:08:59.624 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:59.624 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:59.624 INFO:teuthology.orchestra.run.smithi117.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-06-03T03:08:59.625 INFO:teuthology.orchestra.run.smithi117.stdout: python3-requests-2.25.1-8.el9.noarch 2024-06-03T03:08:59.625 INFO:teuthology.orchestra.run.smithi117.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-06-03T03:08:59.625 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:59.625 INFO:teuthology.orchestra.run.smithi117.stdout: python3-routes-2.5.1-5.el9.noarch 2024-06-03T03:08:59.625 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rsa-4.9-2.el9.noarch 2024-06-03T03:08:59.625 INFO:teuthology.orchestra.run.smithi117.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-06-03T03:08:59.625 INFO:teuthology.orchestra.run.smithi117.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-06-03T03:08:59.625 INFO:teuthology.orchestra.run.smithi117.stdout: python3-toml-0.10.2-6.el9.noarch 2024-06-03T03:08:59.625 INFO:teuthology.orchestra.run.smithi117.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-06-03T03:08:59.625 INFO:teuthology.orchestra.run.smithi117.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-06-03T03:08:59.625 INFO:teuthology.orchestra.run.smithi117.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-06-03T03:08:59.625 INFO:teuthology.orchestra.run.smithi117.stdout: python3-webob-1.8.7-6.el9.noarch 2024-06-03T03:08:59.625 INFO:teuthology.orchestra.run.smithi117.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-06-03T03:08:59.625 INFO:teuthology.orchestra.run.smithi117.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-06-03T03:08:59.626 INFO:teuthology.orchestra.run.smithi117.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-06-03T03:08:59.626 INFO:teuthology.orchestra.run.smithi117.stdout: qatlib-24.02.0-1.el9.x86_64 2024-06-03T03:08:59.626 INFO:teuthology.orchestra.run.smithi117.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-06-03T03:08:59.626 INFO:teuthology.orchestra.run.smithi117.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-06-03T03:08:59.626 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:59.626 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:59.626 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T03:08:59.626 INFO:teuthology.orchestra.run.smithi117.stdout: re2-1:20211101-3.el9.x86_64 2024-06-03T03:08:59.626 INFO:teuthology.orchestra.run.smithi117.stdout: socat-1.7.4.1-5.el9.x86_64 2024-06-03T03:08:59.626 INFO:teuthology.orchestra.run.smithi117.stdout: thrift-0.15.0-2.el9.x86_64 2024-06-03T03:08:59.626 INFO:teuthology.orchestra.run.smithi117.stdout: uuid-1.6.2-55.el9.x86_64 2024-06-03T03:08:59.626 INFO:teuthology.orchestra.run.smithi117.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-06-03T03:08:59.626 INFO:teuthology.orchestra.run.smithi117.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-06-03T03:08:59.627 INFO:teuthology.orchestra.run.smithi117.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-06-03T03:08:59.627 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T03:08:59.627 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-06-03T03:09:00.263 DEBUG:teuthology.parallel:result is None 2024-06-03T03:09:00.263 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-03T03:09:00.487 DEBUG:teuthology.orchestra.run.smithi117:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-06-03T03:09:00.539 INFO:teuthology.orchestra.run.smithi117.stdout:19.0.0-2946.g3d89accf.el9 2024-06-03T03:09:00.540 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2946.g3d89accf.el9 2024-06-03T03:09:00.540 INFO:teuthology.task.install:The correct ceph version 19.0.0-2946.g3d89accf is installed. 2024-06-03T03:09:00.542 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-03T03:09:00.758 DEBUG:teuthology.orchestra.run.smithi151:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-06-03T03:09:00.784 INFO:teuthology.orchestra.run.smithi151.stdout:19.0.0-2946.g3d89accf.el9 2024-06-03T03:09:00.784 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2946.g3d89accf.el9 2024-06-03T03:09:00.784 INFO:teuthology.task.install:The correct ceph version 19.0.0-2946.g3d89accf is installed. 2024-06-03T03:09:00.786 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-03T03:09:00.923 DEBUG:teuthology.orchestra.run.smithi183:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-06-03T03:09:00.949 INFO:teuthology.orchestra.run.smithi183.stdout:19.0.0-2946.g3d89accf.el9 2024-06-03T03:09:00.949 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2946.g3d89accf.el9 2024-06-03T03:09:00.949 INFO:teuthology.task.install:The correct ceph version 19.0.0-2946.g3d89accf is installed. 2024-06-03T03:09:00.951 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-06-03T03:09:00.951 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-06-03T03:09:00.951 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-06-03T03:09:01.022 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-06-03T03:09:01.023 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-06-03T03:09:01.054 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-06-03T03:09:01.054 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-06-03T03:09:01.087 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-06-03T03:09:01.087 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-06-03T03:09:01.087 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/usr/bin/daemon-helper 2024-06-03T03:09:01.153 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-06-03T03:09:01.256 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-06-03T03:09:01.256 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/usr/bin/daemon-helper 2024-06-03T03:09:01.283 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-06-03T03:09:01.348 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-06-03T03:09:01.348 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/usr/bin/daemon-helper 2024-06-03T03:09:01.377 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-06-03T03:09:01.444 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-06-03T03:09:01.444 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-06-03T03:09:01.444 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/usr/bin/adjust-ulimits 2024-06-03T03:09:01.513 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-06-03T03:09:01.615 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-06-03T03:09:01.615 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/usr/bin/adjust-ulimits 2024-06-03T03:09:01.639 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-06-03T03:09:01.703 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-06-03T03:09:01.703 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/usr/bin/adjust-ulimits 2024-06-03T03:09:01.727 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-06-03T03:09:01.790 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-06-03T03:09:01.791 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-06-03T03:09:01.791 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/usr/bin/stdin-killer 2024-06-03T03:09:01.855 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-06-03T03:09:01.960 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-06-03T03:09:01.961 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/usr/bin/stdin-killer 2024-06-03T03:09:01.984 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-06-03T03:09:02.048 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-06-03T03:09:02.048 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/usr/bin/stdin-killer 2024-06-03T03:09:02.071 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-06-03T03:09:02.134 INFO:teuthology.run_tasks:Running task cephadm... 2024-06-03T03:09:02.224 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-06-03T03:09:02.225 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-03T03:09:02.225 INFO:tasks.cephadm:Cluster fsid is a0e8473a-2156-11ef-bc9b-c7b262605968 2024-06-03T03:09:02.225 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-06-03T03:09:02.226 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.117', 'mon.b': '172.21.15.151', 'mon.c': '172.21.15.183'} 2024-06-03T03:09:02.226 INFO:tasks.cephadm:First mon is mon.a on smithi117 2024-06-03T03:09:02.226 INFO:tasks.cephadm:First mgr is x 2024-06-03T03:09:02.226 INFO:tasks.cephadm:Normalizing hostnames... 2024-06-03T03:09:02.226 DEBUG:teuthology.orchestra.run.smithi117:> sudo hostname $(hostname -s) 2024-06-03T03:09:02.289 DEBUG:teuthology.orchestra.run.smithi151:> sudo hostname $(hostname -s) 2024-06-03T03:09:02.313 DEBUG:teuthology.orchestra.run.smithi183:> sudo hostname $(hostname -s) 2024-06-03T03:09:02.336 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-06-03T03:09:02.337 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-03T03:09:02.478 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/55273/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2946-g3d89accf', 'node_name': '172.21.2.14+braggi14', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2946.g3d89accf'}, 'url': 'https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-30 17:04:17.570799', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-06-03T03:09:02.691 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref squid, sha1 3d89accfe3201286f000e065a1faea2a88c7a6ba from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-03T03:09:02.692 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/x86_64/flavors/default/cephadm 2024-06-03T03:09:02.693 DEBUG:teuthology.orchestra.run.smithi117:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-06-03T03:09:03.330 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r--. 1 ubuntu ubuntu 781907 Jun 3 03:09 /home/ubuntu/cephtest/cephadm 2024-06-03T03:09:03.331 DEBUG:teuthology.orchestra.run.smithi151:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-06-03T03:09:03.748 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r--. 1 ubuntu ubuntu 781907 Jun 3 03:09 /home/ubuntu/cephtest/cephadm 2024-06-03T03:09:03.748 DEBUG:teuthology.orchestra.run.smithi183:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-06-03T03:09:04.361 INFO:teuthology.orchestra.run.smithi183.stdout:-rw-r--r--. 1 ubuntu ubuntu 781907 Jun 3 03:09 /home/ubuntu/cephtest/cephadm 2024-06-03T03:09:04.361 DEBUG:teuthology.orchestra.run.smithi117:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-06-03T03:09:04.402 DEBUG:teuthology.orchestra.run.smithi151:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-06-03T03:09:04.420 DEBUG:teuthology.orchestra.run.smithi183:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-06-03T03:09:04.455 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba on all hosts... 2024-06-03T03:09:04.455 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba pull 2024-06-03T03:09:04.462 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba pull 2024-06-03T03:09:04.464 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba pull 2024-06-03T03:09:05.290 INFO:teuthology.orchestra.run.smithi183.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba... 2024-06-03T03:09:05.383 INFO:teuthology.orchestra.run.smithi151.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba... 2024-06-03T03:09:06.195 INFO:teuthology.orchestra.run.smithi117.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba... 2024-06-03T03:09:33.316 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-06-03T03:09:33.317 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_version": "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)", 2024-06-03T03:09:33.317 INFO:teuthology.orchestra.run.smithi183.stdout: "image_id": "b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1", 2024-06-03T03:09:33.317 INFO:teuthology.orchestra.run.smithi183.stdout: "repo_digests": [ 2024-06-03T03:09:33.317 INFO:teuthology.orchestra.run.smithi183.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9" 2024-06-03T03:09:33.317 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-06-03T03:09:33.317 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-06-03T03:09:52.482 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-06-03T03:09:52.482 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_version": "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)", 2024-06-03T03:09:52.482 INFO:teuthology.orchestra.run.smithi151.stdout: "image_id": "b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1", 2024-06-03T03:09:52.482 INFO:teuthology.orchestra.run.smithi151.stdout: "repo_digests": [ 2024-06-03T03:09:52.483 INFO:teuthology.orchestra.run.smithi151.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9" 2024-06-03T03:09:52.483 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-06-03T03:09:52.483 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-06-03T03:09:55.177 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-06-03T03:09:55.178 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_version": "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)", 2024-06-03T03:09:55.178 INFO:teuthology.orchestra.run.smithi117.stdout: "image_id": "b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1", 2024-06-03T03:09:55.178 INFO:teuthology.orchestra.run.smithi117.stdout: "repo_digests": [ 2024-06-03T03:09:55.178 INFO:teuthology.orchestra.run.smithi117.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9" 2024-06-03T03:09:55.178 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-06-03T03:09:55.178 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-06-03T03:09:55.222 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /etc/ceph 2024-06-03T03:09:55.311 DEBUG:teuthology.orchestra.run.smithi151:> sudo mkdir -p /etc/ceph 2024-06-03T03:09:55.344 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /etc/ceph 2024-06-03T03:09:55.376 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod 777 /etc/ceph 2024-06-03T03:09:55.459 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod 777 /etc/ceph 2024-06-03T03:09:55.487 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod 777 /etc/ceph 2024-06-03T03:09:55.517 INFO:tasks.cephadm:Writing seed config... 2024-06-03T03:09:55.518 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-06-03T03:09:55.519 INFO:tasks.cephadm: override: [client] debug client = 20 2024-06-03T03:09:55.519 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-06-03T03:09:55.519 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-06-03T03:09:55.519 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-06-03T03:09:55.519 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-06-03T03:09:55.519 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-06-03T03:09:55.519 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-06-03T03:09:55.519 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-06-03T03:09:55.519 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-06-03T03:09:55.519 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-06-03T03:09:55.519 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-06-03T03:09:55.519 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-06-03T03:09:55.519 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-06-03T03:09:55.519 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-06-03T03:09:55.520 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-06-03T03:09:55.520 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-06-03T03:09:55.520 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-06-03T03:09:55.520 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-06-03T03:09:55.520 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-06-03T03:09:55.520 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-06-03T03:09:55.520 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-06-03T03:09:55.520 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-06-03T03:09:55.520 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-06-03T03:09:55.520 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-06-03T03:09:55.520 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-06-03T03:09:55.520 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-06-03T03:09:55.520 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-06-03T03:09:55.521 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-06-03T03:09:55.521 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-06-03T03:09:55.521 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-06-03T03:09:55.521 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-06-03T03:09:55.521 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-06-03T03:09:55.521 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-06-03T03:09:55.521 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-06-03T03:09:55.521 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-06-03T03:09:55.521 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-06-03T03:09:55.521 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-06-03T03:09:55.521 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-06-03T03:09:55.521 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-06-03T03:09:55.521 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-06-03T03:09:55.521 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-06-03T03:09:55.521 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-06-03T03:09:55.522 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-06-03T03:09:55.522 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-06-03T03:09:55.522 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-06-03T03:09:55.522 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-06-03T03:09:55.523 DEBUG:teuthology.orchestra.run.smithi117:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-06-03T03:09:55.579 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = a0e8473a-2156-11ef-bc9b-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-06-03T03:09:55.579 DEBUG:teuthology.orchestra.run.smithi117:mon.a> sudo journalctl -f -n 0 -u ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@mon.a.service 2024-06-03T03:09:55.626 DEBUG:teuthology.orchestra.run.smithi117:mgr.x> sudo journalctl -f -n 0 -u ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@mgr.x.service 2024-06-03T03:09:55.673 INFO:tasks.cephadm:Bootstrapping... 2024-06-03T03:09:55.673 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba -v bootstrap --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.117 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-06-03T03:09:55.940 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2024-06-03T03:09:55.940 INFO:teuthology.orchestra.run.smithi117.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba', '-v', 'bootstrap', '--fsid', 'a0e8473a-2156-11ef-bc9b-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.117', '--skip-admin-label'] 2024-06-03T03:09:55.940 INFO:teuthology.orchestra.run.smithi117.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-06-03T03:09:55.940 INFO:teuthology.orchestra.run.smithi117.stdout:Verifying podman|docker is present... 2024-06-03T03:09:55.998 INFO:teuthology.orchestra.run.smithi117.stdout:/bin/podman: stdout 5.0.2 2024-06-03T03:09:55.998 INFO:teuthology.orchestra.run.smithi117.stdout:Verifying lvm2 is present... 2024-06-03T03:09:55.998 INFO:teuthology.orchestra.run.smithi117.stdout:Verifying time synchronization is in place... 2024-06-03T03:09:56.016 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-06-03T03:09:56.016 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-06-03T03:09:56.034 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-06-03T03:09:56.034 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stdout inactive 2024-06-03T03:09:56.054 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stdout enabled 2024-06-03T03:09:56.071 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stdout active 2024-06-03T03:09:56.071 INFO:teuthology.orchestra.run.smithi117.stdout:Unit chronyd.service is enabled and running 2024-06-03T03:09:56.071 INFO:teuthology.orchestra.run.smithi117.stdout:Repeating the final host check... 2024-06-03T03:09:56.125 INFO:teuthology.orchestra.run.smithi117.stdout:/bin/podman: stdout 5.0.2 2024-06-03T03:09:56.126 INFO:teuthology.orchestra.run.smithi117.stdout:podman (/bin/podman) version 5.0.2 is present 2024-06-03T03:09:56.126 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl is present 2024-06-03T03:09:56.126 INFO:teuthology.orchestra.run.smithi117.stdout:lvcreate is present 2024-06-03T03:09:56.142 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-06-03T03:09:56.142 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-06-03T03:09:56.156 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-06-03T03:09:56.156 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stdout inactive 2024-06-03T03:09:56.175 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stdout enabled 2024-06-03T03:09:56.193 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stdout active 2024-06-03T03:09:56.194 INFO:teuthology.orchestra.run.smithi117.stdout:Unit chronyd.service is enabled and running 2024-06-03T03:09:56.194 INFO:teuthology.orchestra.run.smithi117.stdout:Host looks OK 2024-06-03T03:09:56.194 INFO:teuthology.orchestra.run.smithi117.stdout:Cluster fsid: a0e8473a-2156-11ef-bc9b-c7b262605968 2024-06-03T03:09:56.195 INFO:teuthology.orchestra.run.smithi117.stdout:Acquiring lock 139818057688448 on /run/cephadm/a0e8473a-2156-11ef-bc9b-c7b262605968.lock 2024-06-03T03:09:56.195 INFO:teuthology.orchestra.run.smithi117.stdout:Lock 139818057688448 acquired on /run/cephadm/a0e8473a-2156-11ef-bc9b-c7b262605968.lock 2024-06-03T03:09:56.196 INFO:teuthology.orchestra.run.smithi117.stdout:Verifying IP 172.21.15.117 port 3300 ... 2024-06-03T03:09:56.197 INFO:teuthology.orchestra.run.smithi117.stdout:Verifying IP 172.21.15.117 port 6789 ... 2024-06-03T03:09:56.197 INFO:teuthology.orchestra.run.smithi117.stdout:Base mon IP(s) is [172.21.15.117:3300, 172.21.15.117:6789], mon addrv is [v2:172.21.15.117:3300,v1:172.21.15.117:6789] 2024-06-03T03:09:56.212 INFO:teuthology.orchestra.run.smithi117.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.117 metric 100 2024-06-03T03:09:56.212 INFO:teuthology.orchestra.run.smithi117.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.117 metric 100 2024-06-03T03:09:56.225 INFO:teuthology.orchestra.run.smithi117.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-06-03T03:09:56.225 INFO:teuthology.orchestra.run.smithi117.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-06-03T03:09:56.226 INFO:teuthology.orchestra.run.smithi117.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1780sec hoplimit 64 pref medium 2024-06-03T03:09:56.237 INFO:teuthology.orchestra.run.smithi117.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-06-03T03:09:56.237 INFO:teuthology.orchestra.run.smithi117.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-06-03T03:09:56.237 INFO:teuthology.orchestra.run.smithi117.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-06-03T03:09:56.237 INFO:teuthology.orchestra.run.smithi117.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-06-03T03:09:56.237 INFO:teuthology.orchestra.run.smithi117.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:4463/64 scope link 2024-06-03T03:09:56.237 INFO:teuthology.orchestra.run.smithi117.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-06-03T03:09:56.239 INFO:teuthology.orchestra.run.smithi117.stdout:Mon IP `172.21.15.117` is in CIDR network `172.21.0.0/20` 2024-06-03T03:09:56.240 INFO:teuthology.orchestra.run.smithi117.stdout:Mon IP `172.21.15.117` is in CIDR network `172.21.0.0/20` 2024-06-03T03:09:56.240 INFO:teuthology.orchestra.run.smithi117.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-06-03T03:09:56.242 INFO:teuthology.orchestra.run.smithi117.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-06-03T03:09:56.245 INFO:teuthology.orchestra.run.smithi117.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba... 2024-06-03T03:09:56.982 INFO:teuthology.orchestra.run.smithi117.stdout:/bin/podman: stdout b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 2024-06-03T03:09:56.982 INFO:teuthology.orchestra.run.smithi117.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba... 2024-06-03T03:09:56.982 INFO:teuthology.orchestra.run.smithi117.stdout:/bin/podman: stderr Getting image source signatures 2024-06-03T03:09:56.982 INFO:teuthology.orchestra.run.smithi117.stdout:/bin/podman: stderr Copying blob sha256:53b2d4191269e9d20972bfa652565296fad6514cf6a3d99f19db644875f67876 2024-06-03T03:09:56.982 INFO:teuthology.orchestra.run.smithi117.stdout:/bin/podman: stderr Copying blob sha256:3ca969c7b6d79946c6f10490fffc00028babe7df67f50ed53c7bc209325e0612 2024-06-03T03:09:56.982 INFO:teuthology.orchestra.run.smithi117.stdout:/bin/podman: stderr Copying config sha256:b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 2024-06-03T03:09:56.982 INFO:teuthology.orchestra.run.smithi117.stdout:/bin/podman: stderr Writing manifest to image destination 2024-06-03T03:09:58.400 INFO:teuthology.orchestra.run.smithi117.stdout:ceph: stdout ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc) 2024-06-03T03:09:58.400 INFO:teuthology.orchestra.run.smithi117.stdout:Ceph version: ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc) 2024-06-03T03:09:58.400 INFO:teuthology.orchestra.run.smithi117.stdout:Extracting ceph user uid/gid from container image... 2024-06-03T03:09:59.601 INFO:teuthology.orchestra.run.smithi117.stdout:stat: stdout 167 167 2024-06-03T03:09:59.601 INFO:teuthology.orchestra.run.smithi117.stdout:Creating initial keys... 2024-06-03T03:10:00.866 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-authtool: stdout AQAINF1m+4g0CRAA5A5Xcv17br7a2NUzeaXN7w== 2024-06-03T03:10:02.231 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-authtool: stdout AQAJNF1mCupnGhAAQEt2FzZP18UxGEe1VOrAqA== 2024-06-03T03:10:03.461 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-authtool: stdout AQAKNF1mBcCdKxAARU6qOLwguffT7AIL7Yi3EQ== 2024-06-03T03:10:03.462 INFO:teuthology.orchestra.run.smithi117.stdout:Creating initial monmap... 2024-06-03T03:10:04.611 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-06-03T03:10:04.612 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-06-03T03:10:04.612 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to a0e8473a-2156-11ef-bc9b-c7b262605968 2024-06-03T03:10:04.612 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-06-03T03:10:04.612 INFO:teuthology.orchestra.run.smithi117.stdout:monmaptool for a [v2:172.21.15.117:3300,v1:172.21.15.117:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-06-03T03:10:04.612 INFO:teuthology.orchestra.run.smithi117.stdout:setting min_mon_release = quincy 2024-06-03T03:10:04.612 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/monmaptool: set fsid to a0e8473a-2156-11ef-bc9b-c7b262605968 2024-06-03T03:10:04.612 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-06-03T03:10:04.612 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T03:10:04.612 INFO:teuthology.orchestra.run.smithi117.stdout:Creating mon... 2024-06-03T03:10:06.033 INFO:teuthology.orchestra.run.smithi117.stdout:create mon.a on 2024-06-03T03:10:06.388 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-06-03T03:10:06.626 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-06-03T03:10:06.968 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-a0e8473a-2156-11ef-bc9b-c7b262605968.target → /etc/systemd/system/ceph-a0e8473a-2156-11ef-bc9b-c7b262605968.target. 2024-06-03T03:10:06.968 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-a0e8473a-2156-11ef-bc9b-c7b262605968.target → /etc/systemd/system/ceph-a0e8473a-2156-11ef-bc9b-c7b262605968.target. 2024-06-03T03:10:07.382 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@mon.a 2024-06-03T03:10:07.382 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stderr Failed to reset failed state of unit ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@mon.a.service: Unit ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@mon.a.service not loaded. 2024-06-03T03:10:07.662 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-a0e8473a-2156-11ef-bc9b-c7b262605968.target.wants/ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@mon.a.service → /etc/systemd/system/ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@.service. 2024-06-03T03:10:08.361 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:08 smithi117 podman[31268]: 2024-06-03 03:10:08.071965668 +0000 UTC m=+0.140960308 container create 237af53f6dad612fd2dd1ac1e001c8c0f0c311f0a006af9dd72ba7f41b0e38c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mon-a, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-03T03:10:08.361 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:08 smithi117 podman[31268]: 2024-06-03 03:10:08.359475914 +0000 UTC m=+0.428470634 container init 237af53f6dad612fd2dd1ac1e001c8c0f0c311f0a006af9dd72ba7f41b0e38c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mon-a, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-03T03:10:08.480 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-06-03T03:10:08.480 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stdout disabled 2024-06-03T03:10:08.497 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-06-03T03:10:08.497 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stdout inactive 2024-06-03T03:10:08.497 INFO:teuthology.orchestra.run.smithi117.stdout:firewalld.service is not enabled 2024-06-03T03:10:08.497 INFO:teuthology.orchestra.run.smithi117.stdout:Not possible to enable service . firewalld.service is not available 2024-06-03T03:10:08.498 INFO:teuthology.orchestra.run.smithi117.stdout:Waiting for mon to start... 2024-06-03T03:10:08.498 INFO:teuthology.orchestra.run.smithi117.stdout:Waiting for mon... 2024-06-03T03:10:08.649 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:08 smithi117 podman[31268]: 2024-06-03 03:10:08.369961957 +0000 UTC m=+0.438956599 container start 237af53f6dad612fd2dd1ac1e001c8c0f0c311f0a006af9dd72ba7f41b0e38c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mon-a, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-06-03T03:10:08.649 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:08 smithi117 bash[31268]: 237af53f6dad612fd2dd1ac1e001c8c0f0c311f0a006af9dd72ba7f41b0e38c4 2024-06-03T03:10:08.649 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:08 smithi117 ceph-mon[31287]: mkfs a0e8473a-2156-11ef-bc9b-c7b262605968 2024-06-03T03:10:08.649 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:08 smithi117 ceph-mon[31287]: mon.a is new leader, mons a in quorum (ranks 0) 2024-06-03T03:10:08.649 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:08 smithi117 systemd[1]: Started Ceph mon.a for a0e8473a-2156-11ef-bc9b-c7b262605968. 2024-06-03T03:10:09.575 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:09 smithi117 ceph-mon[31287]: mon.a is new leader, mons a in quorum (ranks 0) 2024-06-03T03:10:09.575 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:09 smithi117 ceph-mon[31287]: monmap epoch 1 2024-06-03T03:10:09.575 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:09 smithi117 ceph-mon[31287]: fsid a0e8473a-2156-11ef-bc9b-c7b262605968 2024-06-03T03:10:09.575 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:09 smithi117 ceph-mon[31287]: last_changed 2024-06-03T03:10:04.002179+0000 2024-06-03T03:10:09.576 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:09 smithi117 ceph-mon[31287]: created 2024-06-03T03:10:04.002179+0000 2024-06-03T03:10:09.576 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:09 smithi117 ceph-mon[31287]: min_mon_release 19 (squid) 2024-06-03T03:10:09.576 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:09 smithi117 ceph-mon[31287]: election_strategy: 1 2024-06-03T03:10:09.576 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:09 smithi117 ceph-mon[31287]: 0: [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] mon.a 2024-06-03T03:10:09.576 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:09 smithi117 ceph-mon[31287]: fsmap 2024-06-03T03:10:09.576 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:09 smithi117 ceph-mon[31287]: osdmap e1: 0 total, 0 up, 0 in 2024-06-03T03:10:09.576 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:09 smithi117 ceph-mon[31287]: mgrmap e1: no daemons active 2024-06-03T03:10:09.576 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:09 smithi117 ceph-mon[31287]: from='client.? 172.21.15.117:0/3456591400' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-06-03T03:10:09.827 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout cluster: 2024-06-03T03:10:09.827 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout id: a0e8473a-2156-11ef-bc9b-c7b262605968 2024-06-03T03:10:09.827 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-06-03T03:10:09.827 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout 2024-06-03T03:10:09.828 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout services: 2024-06-03T03:10:09.828 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.794531s) 2024-06-03T03:10:09.828 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-06-03T03:10:09.828 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-06-03T03:10:09.828 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout 2024-06-03T03:10:09.828 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout data: 2024-06-03T03:10:09.828 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-06-03T03:10:09.828 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-06-03T03:10:09.828 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-06-03T03:10:09.828 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout pgs: 2024-06-03T03:10:09.828 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout 2024-06-03T03:10:09.828 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:09.147+0000 7f2cc3000700 1 Processor -- start 2024-06-03T03:10:09.829 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:09.147+0000 7f2cc3000700 1 -- start start 2024-06-03T03:10:09.829 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:09.148+0000 7f2cc3000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2cbc081fb0 0x7f2cbc0823d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:10:09.829 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:09.148+0000 7f2cc3000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2cbc0829a0 con 0x7f2cbc081fb0 2024-06-03T03:10:09.829 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:09.149+0000 7f2cbbe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2cbc081fb0 0x7f2cbc0823d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:10:09.829 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:09.149+0000 7f2cbbe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2cbc081fb0 0x7f2cbc0823d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:60240/0 (socket says 172.21.15.117:60240) 2024-06-03T03:10:09.829 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:09.149+0000 7f2cbbe00700 1 -- 172.21.15.117:0/3750658195 learned_addr learned my addr 172.21.15.117:0/3750658195 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:10:09.829 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:09.150+0000 7f2cbbe00700 1 -- 172.21.15.117:0/3750658195 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cbc10f770 con 0x7f2cbc081fb0 2024-06-03T03:10:09.829 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:09.150+0000 7f2cbbe00700 1 --2- 172.21.15.117:0/3750658195 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2cbc081fb0 0x7f2cbc0823d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f2cac0098a0 tx=0x7f2cac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a2ce08687de16e1 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:10:09.829 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:09.150+0000 7f2cbaa00700 1 -- 172.21.15.117:0/3750658195 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2cac004160 con 0x7f2cbc081fb0 2024-06-03T03:10:09.829 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:09.150+0000 7f2cbaa00700 1 -- 172.21.15.117:0/3750658195 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cac004300 con 0x7f2cbc081fb0 2024-06-03T03:10:09.829 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:09.150+0000 7f2cbaa00700 1 -- 172.21.15.117:0/3750658195 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2cac0036a0 con 0x7f2cbc081fb0 2024-06-03T03:10:09.829 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:09.151+0000 7f2cc3000700 1 -- 172.21.15.117:0/3750658195 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2cbc081fb0 msgr2=0x7f2cbc0823d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:10:09.830 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:09.151+0000 7f2cc3000700 1 --2- 172.21.15.117:0/3750658195 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2cbc081fb0 0x7f2cbc0823d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f2cac0098a0 tx=0x7f2cac009bb0 comp rx=0 tx=0).stop 2024-06-03T03:10:09.830 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:09.151+0000 7f2cc3000700 1 -- 172.21.15.117:0/3750658195 shutdown_connections 2024-06-03T03:10:09.830 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:09.151+0000 7f2cc3000700 1 --2- 172.21.15.117:0/3750658195 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2cbc081fb0 0x7f2cbc0823d0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:10:09.830 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:09.151+0000 7f2cc3000700 1 -- 172.21.15.117:0/3750658195 >> 172.21.15.117:0/3750658195 conn(0x7f2cbc07d710 msgr2=0x7f2cbc07fb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:10:09.830 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:09.151+0000 7f2cc3000700 1 -- 172.21.15.117:0/3750658195 shutdown_connections 2024-06-03T03:10:09.830 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:09.151+0000 7f2cc3000700 1 -- 172.21.15.117:0/3750658195 wait complete. 2024-06-03T03:10:09.830 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:09.152+0000 7f2cc3000700 1 Processor -- start 2024-06-03T03:10:09.830 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:09.152+0000 7f2cc3000700 1 -- start start 2024-06-03T03:10:09.830 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:09.153+0000 7f2cc3000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2cbc081fb0 0x7f2cbc1a1c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:10:09.830 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:09.153+0000 7f2cc3000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2cbc10fca0 con 0x7f2cbc081fb0 2024-06-03T03:10:09.830 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:09.153+0000 7f2cbbe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2cbc081fb0 0x7f2cbc1a1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:10:09.830 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:09.153+0000 7f2cbbe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2cbc081fb0 0x7f2cbc1a1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:60252/0 (socket says 172.21.15.117:60252) 2024-06-03T03:10:09.831 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:09.153+0000 7f2cbbe00700 1 -- 172.21.15.117:0/3456591400 learned_addr learned my addr 172.21.15.117:0/3456591400 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:10:09.831 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:09.154+0000 7f2cbbe00700 1 -- 172.21.15.117:0/3456591400 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cbc1a2150 con 0x7f2cbc081fb0 2024-06-03T03:10:09.831 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:09.154+0000 7f2cbbe00700 1 --2- 172.21.15.117:0/3456591400 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2cbc081fb0 0x7f2cbc1a1c10 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f2cac000c00 tx=0x7f2cac004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:10:09.831 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:09.154+0000 7f2cb8c00700 1 -- 172.21.15.117:0/3456591400 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2cac003db0 con 0x7f2cbc081fb0 2024-06-03T03:10:09.831 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:09.154+0000 7f2cb8c00700 1 -- 172.21.15.117:0/3456591400 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cac011630 con 0x7f2cbc081fb0 2024-06-03T03:10:09.831 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:09.154+0000 7f2cb8c00700 1 -- 172.21.15.117:0/3456591400 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2cac011950 con 0x7f2cbc081fb0 2024-06-03T03:10:09.831 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:09.154+0000 7f2cc3000700 1 -- 172.21.15.117:0/3456591400 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cbc1a23e0 con 0x7f2cbc081fb0 2024-06-03T03:10:09.831 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:09.155+0000 7f2cc3000700 1 -- 172.21.15.117:0/3456591400 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cbc1a50d0 con 0x7f2cbc081fb0 2024-06-03T03:10:09.831 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:09.155+0000 7f2cb8c00700 1 -- 172.21.15.117:0/3456591400 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2cac011af0 con 0x7f2cbc081fb0 2024-06-03T03:10:09.831 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:09.155+0000 7f2cb8c00700 1 -- 172.21.15.117:0/3456591400 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2cac01c450 con 0x7f2cbc081fb0 2024-06-03T03:10:09.831 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:09.156+0000 7f2cc3000700 1 -- 172.21.15.117:0/3456591400 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cbc063af0 con 0x7f2cbc081fb0 2024-06-03T03:10:09.831 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:09.159+0000 7f2cb8c00700 1 -- 172.21.15.117:0/3456591400 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f2cac032020 con 0x7f2cbc081fb0 2024-06-03T03:10:09.832 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:09.237+0000 7f2cc3000700 1 -- 172.21.15.117:0/3456591400 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f2cbc02dda0 con 0x7f2cbc081fb0 2024-06-03T03:10:09.832 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:09.238+0000 7f2cb8c00700 1 -- 172.21.15.117:0/3456591400 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f2cac004260 con 0x7f2cbc081fb0 2024-06-03T03:10:09.832 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:09.239+0000 7f2cc3000700 1 -- 172.21.15.117:0/3456591400 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2cbc081fb0 msgr2=0x7f2cbc1a1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:10:09.832 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:09.239+0000 7f2cc3000700 1 --2- 172.21.15.117:0/3456591400 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2cbc081fb0 0x7f2cbc1a1c10 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f2cac000c00 tx=0x7f2cac004750 comp rx=0 tx=0).stop 2024-06-03T03:10:09.832 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:09.239+0000 7f2cc3000700 1 -- 172.21.15.117:0/3456591400 shutdown_connections 2024-06-03T03:10:09.832 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:09.239+0000 7f2cc3000700 1 --2- 172.21.15.117:0/3456591400 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2cbc081fb0 0x7f2cbc1a1c10 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:10:09.832 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:09.239+0000 7f2cc3000700 1 -- 172.21.15.117:0/3456591400 >> 172.21.15.117:0/3456591400 conn(0x7f2cbc07d710 msgr2=0x7f2cbc1988d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:10:09.832 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:09.239+0000 7f2cc3000700 1 -- 172.21.15.117:0/3456591400 shutdown_connections 2024-06-03T03:10:09.832 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:09.239+0000 7f2cc3000700 1 -- 172.21.15.117:0/3456591400 wait complete. 2024-06-03T03:10:09.832 INFO:teuthology.orchestra.run.smithi117.stdout:mon is available 2024-06-03T03:10:09.832 INFO:teuthology.orchestra.run.smithi117.stdout:Assimilating anything we can from ceph.conf... 2024-06-03T03:10:10.916 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:10 smithi117 ceph-mon[31287]: from='client.? 172.21.15.117:0/3229643908' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-06-03T03:10:10.916 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:10 smithi117 ceph-mon[31287]: from='client.? 172.21.15.117:0/3229643908' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-06-03T03:10:11.237 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout 2024-06-03T03:10:11.237 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout [global] 2024-06-03T03:10:11.238 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout fsid = a0e8473a-2156-11ef-bc9b-c7b262605968 2024-06-03T03:10:11.238 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.117:3300,v1:172.21.15.117:6789] 2024-06-03T03:10:11.239 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-06-03T03:10:11.239 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-06-03T03:10:11.239 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-06-03T03:10:11.239 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-06-03T03:10:11.239 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout 2024-06-03T03:10:11.239 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout [mgr] 2024-06-03T03:10:11.239 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-06-03T03:10:11.239 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout 2024-06-03T03:10:11.241 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout [osd] 2024-06-03T03:10:11.241 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-06-03T03:10:11.241 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-06-03T03:10:11.242 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-06-03T03:10:11.242 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-06-03T03:10:11.242 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-06-03T03:10:11.242 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:10.459+0000 7f262ac00700 1 Processor -- start 2024-06-03T03:10:11.242 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:10.459+0000 7f262ac00700 1 -- start start 2024-06-03T03:10:11.242 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:10.460+0000 7f262ac00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f262410a1a0 0x7f262410a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:10:11.242 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:10.460+0000 7f262ac00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f262410ab90 con 0x7f262410a1a0 2024-06-03T03:10:11.242 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:10.460+0000 7f2623e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f262410a1a0 0x7f262410a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:10:11.242 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:10.460+0000 7f2623e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f262410a1a0 0x7f262410a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:60258/0 (socket says 172.21.15.117:60258) 2024-06-03T03:10:11.242 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:10.460+0000 7f2623e00700 1 -- 172.21.15.117:0/613750142 learned_addr learned my addr 172.21.15.117:0/613750142 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:10:11.242 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:10.461+0000 7f2623e00700 1 -- 172.21.15.117:0/613750142 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f262410b390 con 0x7f262410a1a0 2024-06-03T03:10:11.242 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:10.461+0000 7f2623e00700 1 --2- 172.21.15.117:0/613750142 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f262410a1a0 0x7f262410a5c0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f26140098a0 tx=0x7f2614009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbc63e9479578659 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:10:11.242 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:10.461+0000 7f2622a00700 1 -- 172.21.15.117:0/613750142 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2614004160 con 0x7f262410a1a0 2024-06-03T03:10:11.243 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:10.461+0000 7f2622a00700 1 -- 172.21.15.117:0/613750142 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2614004300 con 0x7f262410a1a0 2024-06-03T03:10:11.243 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:10.461+0000 7f2622a00700 1 -- 172.21.15.117:0/613750142 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26140036a0 con 0x7f262410a1a0 2024-06-03T03:10:11.243 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:10.462+0000 7f262ac00700 1 -- 172.21.15.117:0/613750142 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f262410a1a0 msgr2=0x7f262410a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:10:11.246 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:10.462+0000 7f262ac00700 1 --2- 172.21.15.117:0/613750142 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f262410a1a0 0x7f262410a5c0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f26140098a0 tx=0x7f2614009bb0 comp rx=0 tx=0).stop 2024-06-03T03:10:11.246 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:10.462+0000 7f262ac00700 1 -- 172.21.15.117:0/613750142 shutdown_connections 2024-06-03T03:10:11.246 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:10.462+0000 7f262ac00700 1 --2- 172.21.15.117:0/613750142 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f262410a1a0 0x7f262410a5c0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:10:11.246 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:10.462+0000 7f262ac00700 1 -- 172.21.15.117:0/613750142 >> 172.21.15.117:0/613750142 conn(0x7f262407d710 msgr2=0x7f262407db60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:10:11.247 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:10.462+0000 7f262ac00700 1 -- 172.21.15.117:0/613750142 shutdown_connections 2024-06-03T03:10:11.247 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:10.463+0000 7f262ac00700 1 -- 172.21.15.117:0/613750142 wait complete. 2024-06-03T03:10:11.247 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:10.463+0000 7f262ac00700 1 Processor -- start 2024-06-03T03:10:11.247 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:10.464+0000 7f262ac00700 1 -- start start 2024-06-03T03:10:11.247 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:10.464+0000 7f262ac00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f262410a1a0 0x7f262419d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:10:11.247 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:10.464+0000 7f262ac00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f262410b860 con 0x7f262410a1a0 2024-06-03T03:10:11.247 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:10.464+0000 7f2623e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f262410a1a0 0x7f262419d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:10:11.247 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:10.464+0000 7f2623e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f262410a1a0 0x7f262419d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:60272/0 (socket says 172.21.15.117:60272) 2024-06-03T03:10:11.247 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:10.464+0000 7f2623e00700 1 -- 172.21.15.117:0/3229643908 learned_addr learned my addr 172.21.15.117:0/3229643908 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:10:11.247 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:10.465+0000 7f2623e00700 1 -- 172.21.15.117:0/3229643908 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f262419de00 con 0x7f262410a1a0 2024-06-03T03:10:11.247 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:10.465+0000 7f2623e00700 1 --2- 172.21.15.117:0/3229643908 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f262410a1a0 0x7f262419d8c0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f2614000c00 tx=0x7f2614004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:10:11.247 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:10.465+0000 7f2620c00700 1 -- 172.21.15.117:0/3229643908 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2614003db0 con 0x7f262410a1a0 2024-06-03T03:10:11.248 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:10.465+0000 7f2620c00700 1 -- 172.21.15.117:0/3229643908 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2614011630 con 0x7f262410a1a0 2024-06-03T03:10:11.248 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:10.465+0000 7f2620c00700 1 -- 172.21.15.117:0/3229643908 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2614011950 con 0x7f262410a1a0 2024-06-03T03:10:11.248 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:10.465+0000 7f262ac00700 1 -- 172.21.15.117:0/3229643908 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f262419e090 con 0x7f262410a1a0 2024-06-03T03:10:11.248 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:10.465+0000 7f262ac00700 1 -- 172.21.15.117:0/3229643908 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26241a0d80 con 0x7f262410a1a0 2024-06-03T03:10:11.248 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:10.466+0000 7f2620c00700 1 -- 172.21.15.117:0/3229643908 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2614011af0 con 0x7f262410a1a0 2024-06-03T03:10:11.248 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:10.466+0000 7f2620c00700 1 -- 172.21.15.117:0/3229643908 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f261401c450 con 0x7f262410a1a0 2024-06-03T03:10:11.248 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:10.467+0000 7f262ac00700 1 -- 172.21.15.117:0/3229643908 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2624063af0 con 0x7f262410a1a0 2024-06-03T03:10:11.248 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:10.469+0000 7f2620c00700 1 -- 172.21.15.117:0/3229643908 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f2614032020 con 0x7f262410a1a0 2024-06-03T03:10:11.248 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:10.545+0000 7f262ac00700 1 -- 172.21.15.117:0/3229643908 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f262402dda0 con 0x7f262410a1a0 2024-06-03T03:10:11.248 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:10.556+0000 7f2620c00700 1 -- 172.21.15.117:0/3229643908 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+431 (secure 0 0 0) 0x7f2614004260 con 0x7f262410a1a0 2024-06-03T03:10:11.248 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:10.558+0000 7f2620c00700 1 -- 172.21.15.117:0/3229643908 <== mon.0 v2:172.21.15.117:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f261401b450 con 0x7f262410a1a0 2024-06-03T03:10:11.248 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:10.559+0000 7f262ac00700 1 -- 172.21.15.117:0/3229643908 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f262410a1a0 msgr2=0x7f262419d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:10:11.248 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:10.559+0000 7f262ac00700 1 --2- 172.21.15.117:0/3229643908 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f262410a1a0 0x7f262419d8c0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f2614000c00 tx=0x7f2614004750 comp rx=0 tx=0).stop 2024-06-03T03:10:11.248 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:10.560+0000 7f262ac00700 1 -- 172.21.15.117:0/3229643908 shutdown_connections 2024-06-03T03:10:11.249 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:10.560+0000 7f262ac00700 1 --2- 172.21.15.117:0/3229643908 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f262410a1a0 0x7f262419d8c0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:10:11.249 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:10.560+0000 7f262ac00700 1 -- 172.21.15.117:0/3229643908 >> 172.21.15.117:0/3229643908 conn(0x7f262407d710 msgr2=0x7f26241944d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:10:11.249 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:10.560+0000 7f262ac00700 1 -- 172.21.15.117:0/3229643908 shutdown_connections 2024-06-03T03:10:11.249 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:10.560+0000 7f262ac00700 1 -- 172.21.15.117:0/3229643908 wait complete. 2024-06-03T03:10:11.249 INFO:teuthology.orchestra.run.smithi117.stdout:Generating new minimal ceph.conf... 2024-06-03T03:10:12.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:11 smithi117 ceph-mon[31287]: from='client.? 172.21.15.117:0/3433017484' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:10:13.699 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:11.832+0000 7fad6d000700 1 Processor -- start 2024-06-03T03:10:13.699 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:11.832+0000 7fad6d000700 1 -- start start 2024-06-03T03:10:13.700 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:11.833+0000 7fad6d000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad6810a250 0x7fad6810a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:10:13.700 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:11.833+0000 7fad6d000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fad6810ac40 con 0x7fad6810a250 2024-06-03T03:10:13.700 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:11.833+0000 7fad65e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad6810a250 0x7fad6810a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:10:13.700 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:11.833+0000 7fad65e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad6810a250 0x7fad6810a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:60280/0 (socket says 172.21.15.117:60280) 2024-06-03T03:10:13.700 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:11.833+0000 7fad65e00700 1 -- 172.21.15.117:0/1833808084 learned_addr learned my addr 172.21.15.117:0/1833808084 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:10:13.700 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:11.834+0000 7fad65e00700 1 -- 172.21.15.117:0/1833808084 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad6810b340 con 0x7fad6810a250 2024-06-03T03:10:13.700 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:11.834+0000 7fad65e00700 1 --2- 172.21.15.117:0/1833808084 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad6810a250 0x7fad6810a670 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fad580098a0 tx=0x7fad58009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72436f35096801c5 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:10:13.700 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:11.834+0000 7fad64a00700 1 -- 172.21.15.117:0/1833808084 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad58004160 con 0x7fad6810a250 2024-06-03T03:10:13.700 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:11.835+0000 7fad64a00700 1 -- 172.21.15.117:0/1833808084 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fad58004300 con 0x7fad6810a250 2024-06-03T03:10:13.700 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:11.835+0000 7fad64a00700 1 -- 172.21.15.117:0/1833808084 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad58011b50 con 0x7fad6810a250 2024-06-03T03:10:13.700 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:11.835+0000 7fad6d000700 1 -- 172.21.15.117:0/1833808084 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad6810a250 msgr2=0x7fad6810a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:10:13.700 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:11.835+0000 7fad6d000700 1 --2- 172.21.15.117:0/1833808084 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad6810a250 0x7fad6810a670 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fad580098a0 tx=0x7fad58009bb0 comp rx=0 tx=0).stop 2024-06-03T03:10:13.700 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:11.835+0000 7fad6d000700 1 -- 172.21.15.117:0/1833808084 shutdown_connections 2024-06-03T03:10:13.701 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:11.835+0000 7fad6d000700 1 --2- 172.21.15.117:0/1833808084 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad6810a250 0x7fad6810a670 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:10:13.701 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:11.835+0000 7fad6d000700 1 -- 172.21.15.117:0/1833808084 >> 172.21.15.117:0/1833808084 conn(0x7fad6807d550 msgr2=0x7fad6807d980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:10:13.701 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:11.835+0000 7fad6d000700 1 -- 172.21.15.117:0/1833808084 shutdown_connections 2024-06-03T03:10:13.701 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:11.835+0000 7fad6d000700 1 -- 172.21.15.117:0/1833808084 wait complete. 2024-06-03T03:10:13.701 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:11.836+0000 7fad6d000700 1 Processor -- start 2024-06-03T03:10:13.701 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:11.836+0000 7fad6d000700 1 -- start start 2024-06-03T03:10:13.701 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:11.837+0000 7fad6d000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad6810a250 0x7fad6819e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:10:13.701 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:11.837+0000 7fad6d000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fad6810e570 con 0x7fad6810a250 2024-06-03T03:10:13.701 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:11.837+0000 7fad65e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad6810a250 0x7fad6819e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:10:13.701 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:11.837+0000 7fad65e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad6810a250 0x7fad6819e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:60284/0 (socket says 172.21.15.117:60284) 2024-06-03T03:10:13.701 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:11.837+0000 7fad65e00700 1 -- 172.21.15.117:0/3433017484 learned_addr learned my addr 172.21.15.117:0/3433017484 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:10:13.701 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:11.837+0000 7fad65e00700 1 -- 172.21.15.117:0/3433017484 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad6819e690 con 0x7fad6810a250 2024-06-03T03:10:13.702 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:11.838+0000 7fad65e00700 1 --2- 172.21.15.117:0/3433017484 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad6810a250 0x7fad6819e150 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fad58000c00 tx=0x7fad58011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:10:13.702 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:11.838+0000 7fad5ea00700 1 -- 172.21.15.117:0/3433017484 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad58003b20 con 0x7fad6810a250 2024-06-03T03:10:13.702 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:11.838+0000 7fad5ea00700 1 -- 172.21.15.117:0/3433017484 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fad58003cc0 con 0x7fad6810a250 2024-06-03T03:10:13.702 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:11.838+0000 7fad5ea00700 1 -- 172.21.15.117:0/3433017484 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad58024910 con 0x7fad6810a250 2024-06-03T03:10:13.702 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:11.838+0000 7fad6d000700 1 -- 172.21.15.117:0/3433017484 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad6819e920 con 0x7fad6810a250 2024-06-03T03:10:13.702 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:11.838+0000 7fad6d000700 1 -- 172.21.15.117:0/3433017484 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad681a1610 con 0x7fad6810a250 2024-06-03T03:10:13.702 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:11.838+0000 7fad5ea00700 1 -- 172.21.15.117:0/3433017484 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fad58022020 con 0x7fad6810a250 2024-06-03T03:10:13.702 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:11.839+0000 7fad5ea00700 1 -- 172.21.15.117:0/3433017484 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fad58011e00 con 0x7fad6810a250 2024-06-03T03:10:13.702 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:11.839+0000 7fad6d000700 1 -- 172.21.15.117:0/3433017484 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad68063af0 con 0x7fad6810a250 2024-06-03T03:10:13.702 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:11.842+0000 7fad5ea00700 1 -- 172.21.15.117:0/3433017484 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fad5802ddf0 con 0x7fad6810a250 2024-06-03T03:10:13.702 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:11.918+0000 7fad6d000700 1 -- 172.21.15.117:0/3433017484 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fad681a17e0 con 0x7fad6810a250 2024-06-03T03:10:13.702 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:11.919+0000 7fad5ea00700 1 -- 172.21.15.117:0/3433017484 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+177 (secure 0 0 0) 0x7fad58032030 con 0x7fad6810a250 2024-06-03T03:10:13.703 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:11.920+0000 7fad6d000700 1 -- 172.21.15.117:0/3433017484 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad6810a250 msgr2=0x7fad6819e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:10:13.703 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:11.920+0000 7fad6d000700 1 --2- 172.21.15.117:0/3433017484 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad6810a250 0x7fad6819e150 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fad58000c00 tx=0x7fad58011c10 comp rx=0 tx=0).stop 2024-06-03T03:10:13.703 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:11.920+0000 7fad6d000700 1 -- 172.21.15.117:0/3433017484 shutdown_connections 2024-06-03T03:10:13.703 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:11.920+0000 7fad6d000700 1 --2- 172.21.15.117:0/3433017484 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad6810a250 0x7fad6819e150 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:10:13.703 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:11.920+0000 7fad6d000700 1 -- 172.21.15.117:0/3433017484 >> 172.21.15.117:0/3433017484 conn(0x7fad6807d550 msgr2=0x7fad68107db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:10:13.703 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:11.920+0000 7fad6d000700 1 -- 172.21.15.117:0/3433017484 shutdown_connections 2024-06-03T03:10:13.703 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:11.920+0000 7fad6d000700 1 -- 172.21.15.117:0/3433017484 wait complete. 2024-06-03T03:10:13.703 INFO:teuthology.orchestra.run.smithi117.stdout:Restarting the monitor... 2024-06-03T03:10:13.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:13 smithi117 systemd[1]: Stopping Ceph mon.a for a0e8473a-2156-11ef-bc9b-c7b262605968... 2024-06-03T03:10:17.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:17 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mon-a[31283]: 2024-06-03T03:10:17.552+0000 7f1608400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T03:10:17.948 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:17 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mon-a[31283]: 2024-06-03T03:10:17.552+0000 7f1608400700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-06-03T03:10:17.948 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:17 smithi117 podman[31529]: 2024-06-03 03:10:17.749702215 +0000 UTC m=+3.681236639 container died 237af53f6dad612fd2dd1ac1e001c8c0f0c311f0a006af9dd72ba7f41b0e38c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-06-03T03:10:22.680 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:22 smithi117 podman[31529]: 2024-06-03 03:10:22.478173558 +0000 UTC m=+8.409708005 container cleanup 237af53f6dad612fd2dd1ac1e001c8c0f0c311f0a006af9dd72ba7f41b0e38c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mon-a, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528) 2024-06-03T03:10:22.680 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:22 smithi117 bash[31529]: ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mon-a 2024-06-03T03:10:22.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:22 smithi117 podman[31540]: 2024-06-03 03:10:22.678921818 +0000 UTC m=+4.913896373 container remove 237af53f6dad612fd2dd1ac1e001c8c0f0c311f0a006af9dd72ba7f41b0e38c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mon-a, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240528) 2024-06-03T03:10:23.449 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 systemd[1]: ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@mon.a.service: Deactivated successfully. 2024-06-03T03:10:23.450 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 systemd[1]: Stopped Ceph mon.a for a0e8473a-2156-11ef-bc9b-c7b262605968. 2024-06-03T03:10:23.450 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 systemd[1]: ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@mon.a.service: Consumed 1.911s CPU time. 2024-06-03T03:10:23.450 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 systemd[1]: Starting Ceph mon.a for a0e8473a-2156-11ef-bc9b-c7b262605968... 2024-06-03T03:10:23.713 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 podman[31630]: 2024-06-03 03:10:23.415186901 +0000 UTC m=+0.036170573 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-03T03:10:23.714 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 podman[31630]: 2024-06-03 03:10:23.521361957 +0000 UTC m=+0.142345664 container create 92ba733bffb8ae7bb2913d8cd0c8a11cfbc4718d08f5f82680e75aed6d029674 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0) 2024-06-03T03:10:23.892 INFO:teuthology.orchestra.run.smithi117.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-06-03T03:10:24.176 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 podman[31630]: 2024-06-03 03:10:23.799747068 +0000 UTC m=+0.420730809 container init 92ba733bffb8ae7bb2913d8cd0c8a11cfbc4718d08f5f82680e75aed6d029674 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mon-a, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240528) 2024-06-03T03:10:24.176 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 podman[31630]: 2024-06-03 03:10:23.810693774 +0000 UTC m=+0.431677430 container start 92ba733bffb8ae7bb2913d8cd0c8a11cfbc4718d08f5f82680e75aed6d029674 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mon-a, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-03T03:10:24.177 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: set uid:gid to 167:167 (ceph:ceph) 2024-06-03T03:10:24.177 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc), process ceph-mon, pid 2 2024-06-03T03:10:24.177 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: pidfile_write: ignore empty --pid-file 2024-06-03T03:10:24.177 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: load: jerasure load: lrc 2024-06-03T03:10:24.177 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: RocksDB version: 7.9.2 2024-06-03T03:10:24.177 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Git sha 0 2024-06-03T03:10:24.177 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Compile date 2024-05-30 15:42:47 2024-06-03T03:10:24.177 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: DB SUMMARY 2024-06-03T03:10:24.177 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: DB Session ID: ZID35G2JDYGLPYL0ONM3 2024-06-03T03:10:24.177 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: CURRENT file: CURRENT 2024-06-03T03:10:24.177 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: IDENTITY file: IDENTITY 2024-06-03T03:10:24.177 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-06-03T03:10:24.178 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-06-03T03:10:24.178 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104894 ; 2024-06-03T03:10:24.178 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.error_if_exists: 0 2024-06-03T03:10:24.178 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.create_if_missing: 0 2024-06-03T03:10:24.178 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.paranoid_checks: 1 2024-06-03T03:10:24.178 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.flush_verify_memtable_count: 1 2024-06-03T03:10:24.178 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-06-03T03:10:24.178 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-06-03T03:10:24.178 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.env: 0x55d464fe7c60 2024-06-03T03:10:24.178 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.fs: PosixFileSystem 2024-06-03T03:10:24.178 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.info_log: 0x55d4985d1b60 2024-06-03T03:10:24.178 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.max_file_opening_threads: 16 2024-06-03T03:10:24.178 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.statistics: (nil) 2024-06-03T03:10:24.179 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.use_fsync: 0 2024-06-03T03:10:24.179 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.max_log_file_size: 0 2024-06-03T03:10:24.179 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-06-03T03:10:24.179 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.log_file_time_to_roll: 0 2024-06-03T03:10:24.179 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.keep_log_file_num: 1000 2024-06-03T03:10:24.179 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.recycle_log_file_num: 0 2024-06-03T03:10:24.179 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.allow_fallocate: 1 2024-06-03T03:10:24.179 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.allow_mmap_reads: 0 2024-06-03T03:10:24.180 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.allow_mmap_writes: 0 2024-06-03T03:10:24.180 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.use_direct_reads: 0 2024-06-03T03:10:24.180 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-06-03T03:10:24.180 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.create_missing_column_families: 0 2024-06-03T03:10:24.180 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.db_log_dir: 2024-06-03T03:10:24.180 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.wal_dir: 2024-06-03T03:10:24.180 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.table_cache_numshardbits: 6 2024-06-03T03:10:24.180 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.WAL_ttl_seconds: 0 2024-06-03T03:10:24.180 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.WAL_size_limit_MB: 0 2024-06-03T03:10:24.180 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-06-03T03:10:24.180 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-06-03T03:10:24.180 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.is_fd_close_on_exec: 1 2024-06-03T03:10:24.180 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.advise_random_on_open: 1 2024-06-03T03:10:24.181 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.db_write_buffer_size: 0 2024-06-03T03:10:24.181 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.write_buffer_manager: 0x55d4985a21e0 2024-06-03T03:10:24.181 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-06-03T03:10:24.181 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-06-03T03:10:24.181 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.use_adaptive_mutex: 0 2024-06-03T03:10:24.181 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.rate_limiter: (nil) 2024-06-03T03:10:24.181 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-06-03T03:10:24.181 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.wal_recovery_mode: 2 2024-06-03T03:10:24.181 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.enable_thread_tracking: 0 2024-06-03T03:10:24.181 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.enable_pipelined_write: 0 2024-06-03T03:10:24.181 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.unordered_write: 0 2024-06-03T03:10:24.181 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-06-03T03:10:24.181 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-06-03T03:10:24.181 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-06-03T03:10:24.182 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-06-03T03:10:24.182 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.row_cache: None 2024-06-03T03:10:24.182 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.wal_filter: None 2024-06-03T03:10:24.182 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-06-03T03:10:24.183 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.allow_ingest_behind: 0 2024-06-03T03:10:24.183 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.two_write_queues: 0 2024-06-03T03:10:24.183 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.manual_wal_flush: 0 2024-06-03T03:10:24.183 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.wal_compression: 0 2024-06-03T03:10:24.183 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.atomic_flush: 0 2024-06-03T03:10:24.183 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-06-03T03:10:24.183 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.persist_stats_to_disk: 0 2024-06-03T03:10:24.183 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.write_dbid_to_manifest: 0 2024-06-03T03:10:24.183 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.log_readahead_size: 0 2024-06-03T03:10:24.183 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-06-03T03:10:24.183 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.best_efforts_recovery: 0 2024-06-03T03:10:24.184 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-06-03T03:10:24.184 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-06-03T03:10:24.184 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.allow_data_in_errors: 0 2024-06-03T03:10:24.184 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.db_host_id: __hostname__ 2024-06-03T03:10:24.184 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.enforce_single_del_contracts: true 2024-06-03T03:10:24.184 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.max_background_jobs: 2 2024-06-03T03:10:24.184 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.max_background_compactions: -1 2024-06-03T03:10:24.184 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.max_subcompactions: 1 2024-06-03T03:10:24.184 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-06-03T03:10:24.184 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-06-03T03:10:24.184 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.delayed_write_rate : 16777216 2024-06-03T03:10:24.184 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.max_total_wal_size: 0 2024-06-03T03:10:24.184 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-06-03T03:10:24.185 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.stats_dump_period_sec: 600 2024-06-03T03:10:24.185 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.stats_persist_period_sec: 600 2024-06-03T03:10:24.185 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-06-03T03:10:24.185 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.max_open_files: -1 2024-06-03T03:10:24.185 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.bytes_per_sync: 0 2024-06-03T03:10:24.185 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.wal_bytes_per_sync: 0 2024-06-03T03:10:24.185 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.strict_bytes_per_sync: 0 2024-06-03T03:10:24.185 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.compaction_readahead_size: 0 2024-06-03T03:10:24.185 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.max_background_flushes: -1 2024-06-03T03:10:24.185 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Compression algorithms supported: 2024-06-03T03:10:24.185 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-06-03T03:10:24.185 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: kZSTD supported: 0 2024-06-03T03:10:24.186 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: kXpressCompression supported: 0 2024-06-03T03:10:24.186 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: kLZ4HCCompression supported: 1 2024-06-03T03:10:24.186 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: kZlibCompression supported: 1 2024-06-03T03:10:24.186 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: kSnappyCompression supported: 1 2024-06-03T03:10:24.186 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: kLZ4Compression supported: 1 2024-06-03T03:10:24.186 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: kBZip2Compression supported: 0 2024-06-03T03:10:24.186 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Fast CRC32 supported: Supported on x86 2024-06-03T03:10:24.186 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: DMutex implementation: pthread_mutex_t 2024-06-03T03:10:24.186 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-06-03T03:10:24.186 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-06-03T03:10:24.186 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-06-03T03:10:24.186 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.merge_operator: 2024-06-03T03:10:24.186 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.compaction_filter: None 2024-06-03T03:10:24.186 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.compaction_filter_factory: None 2024-06-03T03:10:24.187 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.sst_partitioner_factory: None 2024-06-03T03:10:24.187 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.memtable_factory: SkipListFactory 2024-06-03T03:10:24.187 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.table_factory: BlockBasedTable 2024-06-03T03:10:24.187 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55d4985d1c80) 2024-06-03T03:10:24.187 INFO:journalctl@ceph.mon.a.smithi117.stdout: cache_index_and_filter_blocks: 1 2024-06-03T03:10:24.187 INFO:journalctl@ceph.mon.a.smithi117.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-06-03T03:10:24.187 INFO:journalctl@ceph.mon.a.smithi117.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-06-03T03:10:24.187 INFO:journalctl@ceph.mon.a.smithi117.stdout: pin_top_level_index_and_filter: 1 2024-06-03T03:10:24.187 INFO:journalctl@ceph.mon.a.smithi117.stdout: index_type: 0 2024-06-03T03:10:24.187 INFO:journalctl@ceph.mon.a.smithi117.stdout: data_block_index_type: 0 2024-06-03T03:10:24.187 INFO:journalctl@ceph.mon.a.smithi117.stdout: index_shortening: 1 2024-06-03T03:10:24.187 INFO:journalctl@ceph.mon.a.smithi117.stdout: data_block_hash_table_util_ratio: 0.750000 2024-06-03T03:10:24.187 INFO:journalctl@ceph.mon.a.smithi117.stdout: checksum: 4 2024-06-03T03:10:24.188 INFO:journalctl@ceph.mon.a.smithi117.stdout: no_block_cache: 0 2024-06-03T03:10:24.188 INFO:journalctl@ceph.mon.a.smithi117.stdout: block_cache: 0x55d4985d3090 2024-06-03T03:10:24.188 INFO:journalctl@ceph.mon.a.smithi117.stdout: block_cache_name: BinnedLRUCache 2024-06-03T03:10:24.188 INFO:journalctl@ceph.mon.a.smithi117.stdout: block_cache_options: 2024-06-03T03:10:24.188 INFO:journalctl@ceph.mon.a.smithi117.stdout: capacity : 536870912 2024-06-03T03:10:24.188 INFO:journalctl@ceph.mon.a.smithi117.stdout: num_shard_bits : 4 2024-06-03T03:10:24.188 INFO:journalctl@ceph.mon.a.smithi117.stdout: strict_capacity_limit : 0 2024-06-03T03:10:24.188 INFO:journalctl@ceph.mon.a.smithi117.stdout: high_pri_pool_ratio: 0.000 2024-06-03T03:10:24.188 INFO:journalctl@ceph.mon.a.smithi117.stdout: block_cache_compressed: (nil) 2024-06-03T03:10:24.188 INFO:journalctl@ceph.mon.a.smithi117.stdout: persistent_cache: (nil) 2024-06-03T03:10:24.188 INFO:journalctl@ceph.mon.a.smithi117.stdout: block_size: 4096 2024-06-03T03:10:24.188 INFO:journalctl@ceph.mon.a.smithi117.stdout: block_size_deviation: 10 2024-06-03T03:10:24.188 INFO:journalctl@ceph.mon.a.smithi117.stdout: block_restart_interval: 16 2024-06-03T03:10:24.189 INFO:journalctl@ceph.mon.a.smithi117.stdout: index_block_restart_interval: 1 2024-06-03T03:10:24.189 INFO:journalctl@ceph.mon.a.smithi117.stdout: metadata_block_size: 4096 2024-06-03T03:10:24.189 INFO:journalctl@ceph.mon.a.smithi117.stdout: partition_filters: 0 2024-06-03T03:10:24.189 INFO:journalctl@ceph.mon.a.smithi117.stdout: use_delta_encoding: 1 2024-06-03T03:10:24.189 INFO:journalctl@ceph.mon.a.smithi117.stdout: filter_policy: bloomfilter 2024-06-03T03:10:24.189 INFO:journalctl@ceph.mon.a.smithi117.stdout: whole_key_filtering: 1 2024-06-03T03:10:24.189 INFO:journalctl@ceph.mon.a.smithi117.stdout: verify_compression: 0 2024-06-03T03:10:24.189 INFO:journalctl@ceph.mon.a.smithi117.stdout: read_amp_bytes_per_bit: 0 2024-06-03T03:10:24.189 INFO:journalctl@ceph.mon.a.smithi117.stdout: format_version: 5 2024-06-03T03:10:24.189 INFO:journalctl@ceph.mon.a.smithi117.stdout: enable_index_compression: 1 2024-06-03T03:10:24.189 INFO:journalctl@ceph.mon.a.smithi117.stdout: block_align: 0 2024-06-03T03:10:24.189 INFO:journalctl@ceph.mon.a.smithi117.stdout: max_auto_readahead_size: 262144 2024-06-03T03:10:24.189 INFO:journalctl@ceph.mon.a.smithi117.stdout: prepopulate_block_cache: 0 2024-06-03T03:10:24.190 INFO:journalctl@ceph.mon.a.smithi117.stdout: initial_auto_readahead_size: 8192 2024-06-03T03:10:24.190 INFO:journalctl@ceph.mon.a.smithi117.stdout: num_file_reads_for_auto_readahead: 2 2024-06-03T03:10:24.190 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.write_buffer_size: 33554432 2024-06-03T03:10:24.190 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.max_write_buffer_number: 2 2024-06-03T03:10:24.190 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.compression: NoCompression 2024-06-03T03:10:24.190 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.bottommost_compression: Disabled 2024-06-03T03:10:24.190 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.prefix_extractor: nullptr 2024-06-03T03:10:24.190 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-06-03T03:10:24.190 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.num_levels: 7 2024-06-03T03:10:24.190 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-06-03T03:10:24.190 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-06-03T03:10:24.190 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-06-03T03:10:24.190 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-06-03T03:10:24.191 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-06-03T03:10:24.191 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-06-03T03:10:24.191 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-06-03T03:10:24.191 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-06-03T03:10:24.191 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-06-03T03:10:24.191 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-06-03T03:10:24.191 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-06-03T03:10:24.191 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-06-03T03:10:24.191 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.compression_opts.window_bits: -14 2024-06-03T03:10:24.191 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.compression_opts.level: 32767 2024-06-03T03:10:24.191 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.compression_opts.strategy: 0 2024-06-03T03:10:24.191 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-06-03T03:10:24.191 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-06-03T03:10:24.191 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-06-03T03:10:24.192 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-06-03T03:10:24.192 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.compression_opts.enabled: false 2024-06-03T03:10:24.192 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-06-03T03:10:24.192 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-06-03T03:10:24.192 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-06-03T03:10:24.192 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-06-03T03:10:24.192 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.target_file_size_base: 67108864 2024-06-03T03:10:24.192 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.target_file_size_multiplier: 1 2024-06-03T03:10:24.192 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-06-03T03:10:24.192 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-06-03T03:10:24.193 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-06-03T03:10:24.193 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-06-03T03:10:24.193 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-06-03T03:10:24.193 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-06-03T03:10:24.193 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-06-03T03:10:24.193 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-06-03T03:10:24.193 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-06-03T03:10:24.193 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-06-03T03:10:24.193 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-06-03T03:10:24.193 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-06-03T03:10:24.193 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-06-03T03:10:24.193 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.arena_block_size: 1048576 2024-06-03T03:10:24.194 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-06-03T03:10:24.194 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-06-03T03:10:24.194 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.disable_auto_compactions: 0 2024-06-03T03:10:24.194 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-06-03T03:10:24.194 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-06-03T03:10:24.194 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-06-03T03:10:24.194 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-06-03T03:10:24.194 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-06-03T03:10:24.194 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-06-03T03:10:24.194 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-06-03T03:10:24.194 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-06-03T03:10:24.194 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-06-03T03:10:24.194 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-06-03T03:10:24.195 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.table_properties_collectors: 2024-06-03T03:10:24.195 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.inplace_update_support: 0 2024-06-03T03:10:24.195 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.inplace_update_num_locks: 10000 2024-06-03T03:10:24.195 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-06-03T03:10:24.195 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-06-03T03:10:24.195 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.memtable_huge_page_size: 0 2024-06-03T03:10:24.195 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.bloom_locality: 0 2024-06-03T03:10:24.195 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.max_successive_merges: 0 2024-06-03T03:10:24.195 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.optimize_filters_for_hits: 0 2024-06-03T03:10:24.195 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.paranoid_file_checks: 0 2024-06-03T03:10:24.195 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.force_consistency_checks: 1 2024-06-03T03:10:24.195 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.report_bg_io_stats: 0 2024-06-03T03:10:24.195 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.ttl: 2592000 2024-06-03T03:10:24.195 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.periodic_compaction_seconds: 0 2024-06-03T03:10:24.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-06-03T03:10:24.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-06-03T03:10:24.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.enable_blob_files: false 2024-06-03T03:10:24.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.min_blob_size: 0 2024-06-03T03:10:24.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.blob_file_size: 268435456 2024-06-03T03:10:24.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.blob_compression_type: NoCompression 2024-06-03T03:10:24.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.enable_blob_garbage_collection: false 2024-06-03T03:10:24.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-06-03T03:10:24.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-06-03T03:10:24.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-06-03T03:10:24.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.blob_file_starting_level: 0 2024-06-03T03:10:24.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-06-03T03:10:24.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-06-03T03:10:24.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-06-03T03:10:24.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 94531048-f023-4f02-8a06-5b906b3bf410 2024-06-03T03:10:24.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717384223867546, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-06-03T03:10:24.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-06-03T03:10:24.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717384223869870, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100006, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 326, "table_properties": {"data_size": 97917, "index_size": 307, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14195, "raw_average_key_size": 49, "raw_value_size": 90204, "raw_average_value_size": 316, "num_data_blocks": 12, "num_entries": 285, "num_filter_entries": 285, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1717384223, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "94531048-f023-4f02-8a06-5b906b3bf410", "db_session_id": "ZID35G2JDYGLPYL0ONM3", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-06-03T03:10:24.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717384223870023, "job": 1, "event": "recovery_finished"} 2024-06-03T03:10:24.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-06-03T03:10:24.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-06-03T03:10:24.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55d4986c2000 2024-06-03T03:10:24.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: DB pointer 0x55d4986ac000 2024-06-03T03:10:24.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-06-03T03:10:24.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-06-03T03:10:24.198 INFO:journalctl@ceph.mon.a.smithi117.stdout: ** DB Stats ** 2024-06-03T03:10:24.198 INFO:journalctl@ceph.mon.a.smithi117.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-03T03:10:24.198 INFO:journalctl@ceph.mon.a.smithi117.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-06-03T03:10:24.198 INFO:journalctl@ceph.mon.a.smithi117.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-03T03:10:24.198 INFO:journalctl@ceph.mon.a.smithi117.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-03T03:10:24.199 INFO:journalctl@ceph.mon.a.smithi117.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-06-03T03:10:24.199 INFO:journalctl@ceph.mon.a.smithi117.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-03T03:10:24.199 INFO:journalctl@ceph.mon.a.smithi117.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-03T03:10:24.199 INFO:journalctl@ceph.mon.a.smithi117.stdout: 2024-06-03T03:10:24.199 INFO:journalctl@ceph.mon.a.smithi117.stdout: ** Compaction Stats [default] ** 2024-06-03T03:10:24.199 INFO:journalctl@ceph.mon.a.smithi117.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-06-03T03:10:24.199 INFO:journalctl@ceph.mon.a.smithi117.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-06-03T03:10:24.199 INFO:journalctl@ceph.mon.a.smithi117.stdout: L0 2/0 99.50 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 59.4 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-06-03T03:10:24.199 INFO:journalctl@ceph.mon.a.smithi117.stdout: Sum 2/0 99.50 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 59.4 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-06-03T03:10:24.199 INFO:journalctl@ceph.mon.a.smithi117.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 59.4 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-06-03T03:10:24.199 INFO:journalctl@ceph.mon.a.smithi117.stdout: 2024-06-03T03:10:24.199 INFO:journalctl@ceph.mon.a.smithi117.stdout: ** Compaction Stats [default] ** 2024-06-03T03:10:24.199 INFO:journalctl@ceph.mon.a.smithi117.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-06-03T03:10:24.199 INFO:journalctl@ceph.mon.a.smithi117.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-06-03T03:10:24.200 INFO:journalctl@ceph.mon.a.smithi117.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 59.4 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-06-03T03:10:24.200 INFO:journalctl@ceph.mon.a.smithi117.stdout: 2024-06-03T03:10:24.200 INFO:journalctl@ceph.mon.a.smithi117.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-06-03T03:10:24.200 INFO:journalctl@ceph.mon.a.smithi117.stdout: 2024-06-03T03:10:24.200 INFO:journalctl@ceph.mon.a.smithi117.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-03T03:10:24.200 INFO:journalctl@ceph.mon.a.smithi117.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-06-03T03:10:24.200 INFO:journalctl@ceph.mon.a.smithi117.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-06-03T03:10:24.200 INFO:journalctl@ceph.mon.a.smithi117.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-06-03T03:10:24.200 INFO:journalctl@ceph.mon.a.smithi117.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-06-03T03:10:24.200 INFO:journalctl@ceph.mon.a.smithi117.stdout: AddFile(Keys): cumulative 0, interval 0 2024-06-03T03:10:24.200 INFO:journalctl@ceph.mon.a.smithi117.stdout: Cumulative compaction: 0.00 GB write, 13.44 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-03T03:10:24.200 INFO:journalctl@ceph.mon.a.smithi117.stdout: Interval compaction: 0.00 GB write, 13.44 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-03T03:10:24.200 INFO:journalctl@ceph.mon.a.smithi117.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-06-03T03:10:24.201 INFO:journalctl@ceph.mon.a.smithi117.stdout: Block cache BinnedLRUCache@0x55d4985d3090#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.2e-05 secs_since: 0 2024-06-03T03:10:24.201 INFO:journalctl@ceph.mon.a.smithi117.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-06-03T03:10:24.201 INFO:journalctl@ceph.mon.a.smithi117.stdout: 2024-06-03T03:10:24.201 INFO:journalctl@ceph.mon.a.smithi117.stdout: ** File Read Latency Histogram By Level [default] ** 2024-06-03T03:10:24.201 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: starting mon.a rank 0 at public addrs [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] at bind addrs [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid a0e8473a-2156-11ef-bc9b-c7b262605968 2024-06-03T03:10:24.201 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: mon.a@-1(???) e1 preinit fsid a0e8473a-2156-11ef-bc9b-c7b262605968 2024-06-03T03:10:24.201 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: mon.a@-1(???).mds e1 new map 2024-06-03T03:10:24.201 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: mon.a@-1(???).mds e1 print_map 2024-06-03T03:10:24.201 INFO:journalctl@ceph.mon.a.smithi117.stdout: e1 2024-06-03T03:10:24.201 INFO:journalctl@ceph.mon.a.smithi117.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-06-03T03:10:24.201 INFO:journalctl@ceph.mon.a.smithi117.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-06-03T03:10:24.201 INFO:journalctl@ceph.mon.a.smithi117.stdout: legacy client fscid: -1 2024-06-03T03:10:24.201 INFO:journalctl@ceph.mon.a.smithi117.stdout: 2024-06-03T03:10:24.202 INFO:journalctl@ceph.mon.a.smithi117.stdout: No filesystems configured 2024-06-03T03:10:24.202 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-06-03T03:10:24.202 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-06-03T03:10:24.202 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-06-03T03:10:24.202 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-06-03T03:10:24.202 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-06-03T03:10:24.202 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 bash[31630]: 92ba733bffb8ae7bb2913d8cd0c8a11cfbc4718d08f5f82680e75aed6d029674 2024-06-03T03:10:24.202 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 systemd[1]: Started Ceph mon.a for a0e8473a-2156-11ef-bc9b-c7b262605968. 2024-06-03T03:10:24.202 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: mon.a is new leader, mons a in quorum (ranks 0) 2024-06-03T03:10:24.202 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: monmap epoch 1 2024-06-03T03:10:24.202 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: fsid a0e8473a-2156-11ef-bc9b-c7b262605968 2024-06-03T03:10:24.202 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: last_changed 2024-06-03T03:10:04.002179+0000 2024-06-03T03:10:24.202 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: created 2024-06-03T03:10:04.002179+0000 2024-06-03T03:10:24.202 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: min_mon_release 19 (squid) 2024-06-03T03:10:24.203 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: election_strategy: 1 2024-06-03T03:10:24.203 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: 0: [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] mon.a 2024-06-03T03:10:24.203 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: fsmap 2024-06-03T03:10:24.203 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: osdmap e1: 0 total, 0 up, 0 in 2024-06-03T03:10:24.203 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:23 smithi117 ceph-mon[31649]: mgrmap e1: no daemons active 2024-06-03T03:10:25.232 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:24.489+0000 7f0617e00700 1 Processor -- start 2024-06-03T03:10:25.233 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:24.489+0000 7f0617e00700 1 -- start start 2024-06-03T03:10:25.233 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:24.489+0000 7f0617e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f061007e720 0x7f061007cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:10:25.233 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:24.489+0000 7f0617e00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f061007d1a0 con 0x7f061007e720 2024-06-03T03:10:25.233 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:24.490+0000 7f0615400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f061007e720 0x7f061007cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:10:25.234 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:24.490+0000 7f0615400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f061007e720 0x7f061007cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37734/0 (socket says 172.21.15.117:37734) 2024-06-03T03:10:25.234 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:24.490+0000 7f0615400700 1 -- 172.21.15.117:0/1470484515 learned_addr learned my addr 172.21.15.117:0/1470484515 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:10:25.234 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:24.491+0000 7f0615400700 1 -- 172.21.15.117:0/1470484515 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f061007d320 con 0x7f061007e720 2024-06-03T03:10:25.234 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:24.491+0000 7f0615400700 1 --2- 172.21.15.117:0/1470484515 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f061007e720 0x7f061007cbd0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f06040098a0 tx=0x7f0604009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bd22eb058a6912e server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:10:25.235 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:24.491+0000 7f060fe00700 1 -- 172.21.15.117:0/1470484515 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0604004160 con 0x7f061007e720 2024-06-03T03:10:25.235 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:24.491+0000 7f060fe00700 1 -- 172.21.15.117:0/1470484515 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0604004300 con 0x7f061007e720 2024-06-03T03:10:25.235 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:24.491+0000 7f060fe00700 1 -- 172.21.15.117:0/1470484515 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0604011b50 con 0x7f061007e720 2024-06-03T03:10:25.235 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:24.492+0000 7f0617e00700 1 -- 172.21.15.117:0/1470484515 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f061007e720 msgr2=0x7f061007cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:10:25.235 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:24.492+0000 7f0617e00700 1 --2- 172.21.15.117:0/1470484515 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f061007e720 0x7f061007cbd0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f06040098a0 tx=0x7f0604009bb0 comp rx=0 tx=0).stop 2024-06-03T03:10:25.236 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:24.492+0000 7f0617e00700 1 -- 172.21.15.117:0/1470484515 shutdown_connections 2024-06-03T03:10:25.236 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:24.492+0000 7f0617e00700 1 --2- 172.21.15.117:0/1470484515 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f061007e720 0x7f061007cbd0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:10:25.236 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:24.492+0000 7f0617e00700 1 -- 172.21.15.117:0/1470484515 >> 172.21.15.117:0/1470484515 conn(0x7f0610103530 msgr2=0x7f0610105950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:10:25.236 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:24.492+0000 7f0617e00700 1 -- 172.21.15.117:0/1470484515 shutdown_connections 2024-06-03T03:10:25.236 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:24.493+0000 7f0617e00700 1 -- 172.21.15.117:0/1470484515 wait complete. 2024-06-03T03:10:25.237 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:24.494+0000 7f0617e00700 1 Processor -- start 2024-06-03T03:10:25.237 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:24.494+0000 7f0617e00700 1 -- start start 2024-06-03T03:10:25.237 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:24.494+0000 7f0617e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f061007e720 0x7f061019e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:10:25.237 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:24.494+0000 7f0617e00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f061010e540 con 0x7f061007e720 2024-06-03T03:10:25.237 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:24.494+0000 7f0615400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f061007e720 0x7f061019e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:10:25.237 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:24.494+0000 7f0615400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f061007e720 0x7f061019e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37736/0 (socket says 172.21.15.117:37736) 2024-06-03T03:10:25.238 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:24.494+0000 7f0615400700 1 -- 172.21.15.117:0/2589328924 learned_addr learned my addr 172.21.15.117:0/2589328924 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:10:25.238 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:24.495+0000 7f0615400700 1 -- 172.21.15.117:0/2589328924 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f061019e600 con 0x7f061007e720 2024-06-03T03:10:25.238 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:24.495+0000 7f0615400700 1 --2- 172.21.15.117:0/2589328924 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f061007e720 0x7f061019e0c0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f0604011b60 tx=0x7f0604011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:10:25.238 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:24.495+0000 7f060e000700 1 -- 172.21.15.117:0/2589328924 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0604003b20 con 0x7f061007e720 2024-06-03T03:10:25.239 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:24.495+0000 7f060e000700 1 -- 172.21.15.117:0/2589328924 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0604003cc0 con 0x7f061007e720 2024-06-03T03:10:25.239 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:24.495+0000 7f0617e00700 1 -- 172.21.15.117:0/2589328924 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f061019e890 con 0x7f061007e720 2024-06-03T03:10:25.239 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:24.496+0000 7f060e000700 1 -- 172.21.15.117:0/2589328924 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0604024840 con 0x7f061007e720 2024-06-03T03:10:25.239 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:24.496+0000 7f0617e00700 1 -- 172.21.15.117:0/2589328924 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06101a1580 con 0x7f061007e720 2024-06-03T03:10:25.239 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:24.496+0000 7f060e000700 1 -- 172.21.15.117:0/2589328924 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0604022020 con 0x7f061007e720 2024-06-03T03:10:25.239 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:24.496+0000 7f060e000700 1 -- 172.21.15.117:0/2589328924 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0604011e00 con 0x7f061007e720 2024-06-03T03:10:25.240 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:24.497+0000 7f0617e00700 1 -- 172.21.15.117:0/2589328924 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0610063af0 con 0x7f061007e720 2024-06-03T03:10:25.240 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:24.500+0000 7f060e000700 1 -- 172.21.15.117:0/2589328924 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f0604043dd0 con 0x7f061007e720 2024-06-03T03:10:25.240 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:24.575+0000 7f0617e00700 1 -- 172.21.15.117:0/2589328924 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f06101a1750 con 0x7f061007e720 2024-06-03T03:10:25.240 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:24.578+0000 7f060e000700 1 -- 172.21.15.117:0/2589328924 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f0604032030 con 0x7f061007e720 2024-06-03T03:10:25.240 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:24.580+0000 7f0617e00700 1 -- 172.21.15.117:0/2589328924 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f061007e720 msgr2=0x7f061019e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:10:25.241 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:24.580+0000 7f0617e00700 1 --2- 172.21.15.117:0/2589328924 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f061007e720 0x7f061019e0c0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f0604011b60 tx=0x7f0604011c40 comp rx=0 tx=0).stop 2024-06-03T03:10:25.241 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:24.580+0000 7f0617e00700 1 -- 172.21.15.117:0/2589328924 shutdown_connections 2024-06-03T03:10:25.241 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:24.580+0000 7f0617e00700 1 --2- 172.21.15.117:0/2589328924 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f061007e720 0x7f061019e0c0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:10:25.241 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:24.580+0000 7f0617e00700 1 -- 172.21.15.117:0/2589328924 >> 172.21.15.117:0/2589328924 conn(0x7f0610103530 msgr2=0x7f0610104080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:10:25.241 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:24.580+0000 7f0617e00700 1 -- 172.21.15.117:0/2589328924 shutdown_connections 2024-06-03T03:10:25.241 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:24.580+0000 7f0617e00700 1 -- 172.21.15.117:0/2589328924 wait complete. 2024-06-03T03:10:25.241 INFO:teuthology.orchestra.run.smithi117.stdout:Wrote config to /etc/ceph/ceph.conf 2024-06-03T03:10:25.254 INFO:teuthology.orchestra.run.smithi117.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-06-03T03:10:25.254 INFO:teuthology.orchestra.run.smithi117.stdout:Creating mgr... 2024-06-03T03:10:25.255 INFO:teuthology.orchestra.run.smithi117.stdout:Verifying port 0.0.0.0:9283 ... 2024-06-03T03:10:25.256 INFO:teuthology.orchestra.run.smithi117.stdout:Verifying port 0.0.0.0:8765 ... 2024-06-03T03:10:25.659 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@mgr.x 2024-06-03T03:10:25.659 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stderr Failed to reset failed state of unit ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@mgr.x.service: Unit ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@mgr.x.service not loaded. 2024-06-03T03:10:25.849 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:25 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/2589328924' entity='client.admin' 2024-06-03T03:10:25.914 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-a0e8473a-2156-11ef-bc9b-c7b262605968.target.wants/ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@.service. 2024-06-03T03:10:26.197 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:10:25 smithi117 systemd[1]: Starting Ceph mgr.x for a0e8473a-2156-11ef-bc9b-c7b262605968... 2024-06-03T03:10:26.618 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:10:26 smithi117 podman[31868]: 2024-06-03 03:10:26.196180236 +0000 UTC m=+0.031613050 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-03T03:10:26.618 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:10:26 smithi117 podman[31868]: 2024-06-03 03:10:26.306256278 +0000 UTC m=+0.141689152 container create 31f67206095a987561f775a5b1af7dffd82d46ca8b1fd2e7e43d1c74edfbbbd6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-03T03:10:26.618 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:10:26 smithi117 podman[31868]: 2024-06-03 03:10:26.526466586 +0000 UTC m=+0.361899490 container init 31f67206095a987561f775a5b1af7dffd82d46ca8b1fd2e7e43d1c74edfbbbd6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-06-03T03:10:26.618 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:10:26 smithi117 podman[31868]: 2024-06-03 03:10:26.536132515 +0000 UTC m=+0.371565355 container start 31f67206095a987561f775a5b1af7dffd82d46ca8b1fd2e7e43d1c74edfbbbd6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-06-03T03:10:26.618 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:10:26 smithi117 bash[31868]: 31f67206095a987561f775a5b1af7dffd82d46ca8b1fd2e7e43d1c74edfbbbd6 2024-06-03T03:10:26.653 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-06-03T03:10:26.653 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stdout disabled 2024-06-03T03:10:26.670 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-06-03T03:10:26.670 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stdout inactive 2024-06-03T03:10:26.670 INFO:teuthology.orchestra.run.smithi117.stdout:firewalld.service is not enabled 2024-06-03T03:10:26.670 INFO:teuthology.orchestra.run.smithi117.stdout:Not possible to enable service . firewalld.service is not available 2024-06-03T03:10:26.693 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-06-03T03:10:26.694 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stdout disabled 2024-06-03T03:10:26.715 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-06-03T03:10:26.715 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stdout inactive 2024-06-03T03:10:26.715 INFO:teuthology.orchestra.run.smithi117.stdout:firewalld.service is not enabled 2024-06-03T03:10:26.716 INFO:teuthology.orchestra.run.smithi117.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-06-03T03:10:26.716 INFO:teuthology.orchestra.run.smithi117.stdout:Waiting for mgr to start... 2024-06-03T03:10:26.716 INFO:teuthology.orchestra.run.smithi117.stdout:Waiting for mgr... 2024-06-03T03:10:26.868 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:10:26 smithi117 systemd[1]: Started Ceph mgr.x for a0e8473a-2156-11ef-bc9b-c7b262605968. 2024-06-03T03:10:26.868 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:10:26 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:10:26.777+0000 7fa7a4507200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-03T03:10:27.183 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:10:27 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:10:27.040+0000 7fa7a4507200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-03T03:10:27.183 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:10:27 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:10:27.181+0000 7fa7a4507200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-03T03:10:27.820 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:10:27 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:10:27.585+0000 7fa7a4507200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-03T03:10:27.821 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:10:27 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:10:27.703+0000 7fa7a4507200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-03T03:10:28.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:28 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/2642502208' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-06-03T03:10:28.660 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout 2024-06-03T03:10:28.660 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout { 2024-06-03T03:10:28.660 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "fsid": "a0e8473a-2156-11ef-bc9b-c7b262605968", 2024-06-03T03:10:28.660 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "health": { 2024-06-03T03:10:28.660 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-06-03T03:10:28.660 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-06-03T03:10:28.660 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-06-03T03:10:28.661 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-06-03T03:10:28.661 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-06-03T03:10:28.661 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-06-03T03:10:28.661 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout 0 2024-06-03T03:10:28.661 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout ], 2024-06-03T03:10:28.661 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-06-03T03:10:28.661 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "a" 2024-06-03T03:10:28.661 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout ], 2024-06-03T03:10:28.661 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "quorum_age": 4, 2024-06-03T03:10:28.661 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "monmap": { 2024-06-03T03:10:28.661 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-03T03:10:28.661 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-06-03T03:10:28.661 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-06-03T03:10:28.661 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-06-03T03:10:28.661 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-06-03T03:10:28.662 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-03T03:10:28.662 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-06-03T03:10:28.662 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-06-03T03:10:28.662 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-06-03T03:10:28.662 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-06-03T03:10:28.662 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-06-03T03:10:28.662 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-06-03T03:10:28.662 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-06-03T03:10:28.662 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-06-03T03:10:28.669 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-06-03T03:10:28.669 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-06-03T03:10:28.669 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-06-03T03:10:28.669 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-06-03T03:10:28.669 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-06-03T03:10:28.669 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-06-03T03:10:28.669 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-06-03T03:10:28.669 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-06-03T03:10:28.670 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-06-03T03:10:28.670 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-06-03T03:10:28.670 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-03T03:10:28.670 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-06-03T03:10:28.670 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-06-03T03:10:28.670 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-06-03T03:10:28.670 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-06-03T03:10:28.670 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "available": false, 2024-06-03T03:10:28.670 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-06-03T03:10:28.670 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "modules": [ 2024-06-03T03:10:28.670 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "iostat", 2024-06-03T03:10:28.670 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "nfs", 2024-06-03T03:10:28.670 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "restful" 2024-06-03T03:10:28.671 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout ], 2024-06-03T03:10:28.671 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-03T03:10:28.671 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-06-03T03:10:28.671 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-06-03T03:10:28.671 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-03T03:10:28.671 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "modified": "2024-06-03T03:10:08.447523+0000", 2024-06-03T03:10:28.671 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-03T03:10:28.671 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-06-03T03:10:28.671 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-06-03T03:10:28.671 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout } 2024-06-03T03:10:28.671 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:27.861+0000 7fd711a00700 1 Processor -- start 2024-06-03T03:10:28.671 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:27.861+0000 7fd711a00700 1 -- start start 2024-06-03T03:10:28.671 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:27.862+0000 7fd711a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd70c107f70 0x7fd70c108390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:10:28.671 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:27.862+0000 7fd711a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd70c1088d0 con 0x7fd70c107f70 2024-06-03T03:10:28.672 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:27.862+0000 7fd70aa00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd70c107f70 0x7fd70c108390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:10:28.672 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:27.863+0000 7fd70aa00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd70c107f70 0x7fd70c108390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:59690/0 (socket says 172.21.15.117:59690) 2024-06-03T03:10:28.672 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:27.863+0000 7fd70aa00700 1 -- 172.21.15.117:0/1894458907 learned_addr learned my addr 172.21.15.117:0/1894458907 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:10:28.672 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:27.863+0000 7fd70aa00700 1 -- 172.21.15.117:0/1894458907 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd70c109080 con 0x7fd70c107f70 2024-06-03T03:10:28.672 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:27.863+0000 7fd70aa00700 1 --2- 172.21.15.117:0/1894458907 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd70c107f70 0x7fd70c108390 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc0098a0 tx=0x7fd6fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acf871daa2167746 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:10:28.672 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:27.864+0000 7fd709600700 1 -- 172.21.15.117:0/1894458907 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd6fc004160 con 0x7fd70c107f70 2024-06-03T03:10:28.672 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:27.864+0000 7fd709600700 1 -- 172.21.15.117:0/1894458907 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd6fc004300 con 0x7fd70c107f70 2024-06-03T03:10:28.672 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:27.864+0000 7fd709600700 1 -- 172.21.15.117:0/1894458907 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd6fc011b50 con 0x7fd70c107f70 2024-06-03T03:10:28.672 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:27.864+0000 7fd711a00700 1 -- 172.21.15.117:0/1894458907 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd70c107f70 msgr2=0x7fd70c108390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:10:28.672 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:27.864+0000 7fd711a00700 1 --2- 172.21.15.117:0/1894458907 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd70c107f70 0x7fd70c108390 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc0098a0 tx=0x7fd6fc009bb0 comp rx=0 tx=0).stop 2024-06-03T03:10:28.672 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:27.865+0000 7fd711a00700 1 -- 172.21.15.117:0/1894458907 shutdown_connections 2024-06-03T03:10:28.672 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:27.865+0000 7fd711a00700 1 --2- 172.21.15.117:0/1894458907 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd70c107f70 0x7fd70c108390 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:10:28.672 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:27.865+0000 7fd711a00700 1 -- 172.21.15.117:0/1894458907 >> 172.21.15.117:0/1894458907 conn(0x7fd70c103530 msgr2=0x7fd70c105950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:10:28.672 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:27.865+0000 7fd711a00700 1 -- 172.21.15.117:0/1894458907 shutdown_connections 2024-06-03T03:10:28.673 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:27.865+0000 7fd711a00700 1 -- 172.21.15.117:0/1894458907 wait complete. 2024-06-03T03:10:28.673 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:27.866+0000 7fd711a00700 1 Processor -- start 2024-06-03T03:10:28.673 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:27.866+0000 7fd711a00700 1 -- start start 2024-06-03T03:10:28.673 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:27.866+0000 7fd711a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd70c107f70 0x7fd70c19e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:10:28.673 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:27.866+0000 7fd711a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd70c10c4c0 con 0x7fd70c107f70 2024-06-03T03:10:28.673 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:27.866+0000 7fd70aa00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd70c107f70 0x7fd70c19e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:10:28.673 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:27.866+0000 7fd70aa00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd70c107f70 0x7fd70c19e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:59694/0 (socket says 172.21.15.117:59694) 2024-06-03T03:10:28.673 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:27.866+0000 7fd70aa00700 1 -- 172.21.15.117:0/2642502208 learned_addr learned my addr 172.21.15.117:0/2642502208 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:10:28.673 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:27.867+0000 7fd70aa00700 1 -- 172.21.15.117:0/2642502208 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd70c19e610 con 0x7fd70c107f70 2024-06-03T03:10:28.673 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:27.867+0000 7fd70aa00700 1 --2- 172.21.15.117:0/2642502208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd70c107f70 0x7fd70c19e0d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc011b60 tx=0x7fd6fc011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:10:28.673 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:27.867+0000 7fd703400700 1 -- 172.21.15.117:0/2642502208 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd6fc003b20 con 0x7fd70c107f70 2024-06-03T03:10:28.673 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:27.867+0000 7fd703400700 1 -- 172.21.15.117:0/2642502208 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd6fc003cc0 con 0x7fd70c107f70 2024-06-03T03:10:28.673 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:27.867+0000 7fd711a00700 1 -- 172.21.15.117:0/2642502208 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd70c19e8a0 con 0x7fd70c107f70 2024-06-03T03:10:28.674 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:27.867+0000 7fd703400700 1 -- 172.21.15.117:0/2642502208 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd6fc024840 con 0x7fd70c107f70 2024-06-03T03:10:28.674 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:27.867+0000 7fd711a00700 1 -- 172.21.15.117:0/2642502208 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd70c1a1590 con 0x7fd70c107f70 2024-06-03T03:10:28.674 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:27.868+0000 7fd703400700 1 -- 172.21.15.117:0/2642502208 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd6fc022020 con 0x7fd70c107f70 2024-06-03T03:10:28.674 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:27.868+0000 7fd703400700 1 -- 172.21.15.117:0/2642502208 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd6fc011e00 con 0x7fd70c107f70 2024-06-03T03:10:28.674 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:27.868+0000 7fd711a00700 1 -- 172.21.15.117:0/2642502208 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd70c063af0 con 0x7fd70c107f70 2024-06-03T03:10:28.674 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:27.871+0000 7fd703400700 1 -- 172.21.15.117:0/2642502208 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fd6fc043dd0 con 0x7fd70c107f70 2024-06-03T03:10:28.674 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:27.951+0000 7fd711a00700 1 -- 172.21.15.117:0/2642502208 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fd70c1a1b00 con 0x7fd70c107f70 2024-06-03T03:10:28.674 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:27.951+0000 7fd703400700 1 -- 172.21.15.117:0/2642502208 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fd6fc032030 con 0x7fd70c107f70 2024-06-03T03:10:28.674 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:27.953+0000 7fd711a00700 1 -- 172.21.15.117:0/2642502208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd70c107f70 msgr2=0x7fd70c19e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:10:28.674 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:27.953+0000 7fd711a00700 1 --2- 172.21.15.117:0/2642502208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd70c107f70 0x7fd70c19e0d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc011b60 tx=0x7fd6fc011c40 comp rx=0 tx=0).stop 2024-06-03T03:10:28.674 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:27.953+0000 7fd711a00700 1 -- 172.21.15.117:0/2642502208 shutdown_connections 2024-06-03T03:10:28.674 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:27.953+0000 7fd711a00700 1 --2- 172.21.15.117:0/2642502208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd70c107f70 0x7fd70c19e0d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:10:28.674 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:27.953+0000 7fd711a00700 1 -- 172.21.15.117:0/2642502208 >> 172.21.15.117:0/2642502208 conn(0x7fd70c103530 msgr2=0x7fd70c104050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:10:28.674 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:27.953+0000 7fd711a00700 1 -- 172.21.15.117:0/2642502208 shutdown_connections 2024-06-03T03:10:28.675 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:27.953+0000 7fd711a00700 1 -- 172.21.15.117:0/2642502208 wait complete. 2024-06-03T03:10:28.675 INFO:teuthology.orchestra.run.smithi117.stdout:mgr not available, waiting (1/15)... 2024-06-03T03:10:29.696 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:10:29 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:10:29.297+0000 7fa7a4507200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-03T03:10:29.696 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:10:29 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:10:29.514+0000 7fa7a4507200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-03T03:10:30.100 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:10:29 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:10:29.724+0000 7fa7a4507200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-03T03:10:30.100 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:10:30 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:10:30.098+0000 7fa7a4507200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-03T03:10:30.446 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:10:30 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:10:30.220+0000 7fa7a4507200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-03T03:10:30.720 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:10:30 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:10:30.468+0000 7fa7a4507200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-03T03:10:30.720 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:10:30 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:10:30.719+0000 7fa7a4507200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-03T03:10:31.696 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:31 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/2993493635' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-06-03T03:10:31.697 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:10:31 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:10:31.521+0000 7fa7a4507200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-03T03:10:32.079 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:10:32 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:10:32.077+0000 7fa7a4507200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-03T03:10:32.110 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout 2024-06-03T03:10:32.110 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout { 2024-06-03T03:10:32.110 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "fsid": "a0e8473a-2156-11ef-bc9b-c7b262605968", 2024-06-03T03:10:32.110 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "health": { 2024-06-03T03:10:32.110 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-06-03T03:10:32.110 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-06-03T03:10:32.110 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-06-03T03:10:32.110 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-06-03T03:10:32.110 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-06-03T03:10:32.110 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-06-03T03:10:32.111 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout 0 2024-06-03T03:10:32.111 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout ], 2024-06-03T03:10:32.111 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-06-03T03:10:32.111 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "a" 2024-06-03T03:10:32.111 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout ], 2024-06-03T03:10:32.111 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-06-03T03:10:32.111 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "monmap": { 2024-06-03T03:10:32.111 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-03T03:10:32.111 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-06-03T03:10:32.111 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-06-03T03:10:32.111 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-06-03T03:10:32.111 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-06-03T03:10:32.111 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-03T03:10:32.112 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-06-03T03:10:32.112 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-06-03T03:10:32.112 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-06-03T03:10:32.112 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-06-03T03:10:32.112 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-06-03T03:10:32.112 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-06-03T03:10:32.112 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-06-03T03:10:32.112 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-06-03T03:10:32.112 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-06-03T03:10:32.112 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-06-03T03:10:32.112 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-06-03T03:10:32.112 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-06-03T03:10:32.112 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-06-03T03:10:32.112 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-06-03T03:10:32.112 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-06-03T03:10:32.113 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-06-03T03:10:32.113 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-06-03T03:10:32.113 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-06-03T03:10:32.113 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-03T03:10:32.113 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-06-03T03:10:32.113 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-06-03T03:10:32.119 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-06-03T03:10:32.119 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-06-03T03:10:32.119 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "available": false, 2024-06-03T03:10:32.119 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-06-03T03:10:32.119 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "modules": [ 2024-06-03T03:10:32.119 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "iostat", 2024-06-03T03:10:32.119 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "nfs", 2024-06-03T03:10:32.120 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "restful" 2024-06-03T03:10:32.120 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout ], 2024-06-03T03:10:32.120 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-03T03:10:32.120 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-06-03T03:10:32.120 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-06-03T03:10:32.120 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-03T03:10:32.120 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "modified": "2024-06-03T03:10:08.447523+0000", 2024-06-03T03:10:32.120 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-03T03:10:32.120 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-06-03T03:10:32.120 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-06-03T03:10:32.120 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout } 2024-06-03T03:10:32.120 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:31.295+0000 7f77cc000700 1 Processor -- start 2024-06-03T03:10:32.120 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:31.296+0000 7f77cc000700 1 -- start start 2024-06-03T03:10:32.121 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:31.296+0000 7f77cc000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77c4107fa0 0x7f77c41083e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:10:32.121 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:31.296+0000 7f77cc000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f77c4108920 con 0x7f77c4107fa0 2024-06-03T03:10:32.121 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:31.297+0000 7f77c9600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77c4107fa0 0x7f77c41083e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:10:32.121 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:31.297+0000 7f77c9600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77c4107fa0 0x7f77c41083e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:59696/0 (socket says 172.21.15.117:59696) 2024-06-03T03:10:32.121 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:31.297+0000 7f77c9600700 1 -- 172.21.15.117:0/3446477723 learned_addr learned my addr 172.21.15.117:0/3446477723 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:10:32.121 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:31.297+0000 7f77c9600700 1 -- 172.21.15.117:0/3446477723 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77c41090e0 con 0x7f77c4107fa0 2024-06-03T03:10:32.121 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:31.298+0000 7f77c9600700 1 --2- 172.21.15.117:0/3446477723 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77c4107fa0 0x7f77c41083e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f77b80098a0 tx=0x7f77b8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ca69f170b9320b0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:10:32.121 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:31.298+0000 7f77c3e00700 1 -- 172.21.15.117:0/3446477723 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77b8004160 con 0x7f77c4107fa0 2024-06-03T03:10:32.121 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:31.298+0000 7f77c3e00700 1 -- 172.21.15.117:0/3446477723 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f77b8004300 con 0x7f77c4107fa0 2024-06-03T03:10:32.121 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:31.298+0000 7f77c3e00700 1 -- 172.21.15.117:0/3446477723 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77b8011b50 con 0x7f77c4107fa0 2024-06-03T03:10:32.121 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:31.299+0000 7f77cc000700 1 -- 172.21.15.117:0/3446477723 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77c4107fa0 msgr2=0x7f77c41083e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:10:32.121 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:31.299+0000 7f77cc000700 1 --2- 172.21.15.117:0/3446477723 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77c4107fa0 0x7f77c41083e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f77b80098a0 tx=0x7f77b8009bb0 comp rx=0 tx=0).stop 2024-06-03T03:10:32.121 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:31.299+0000 7f77cc000700 1 -- 172.21.15.117:0/3446477723 shutdown_connections 2024-06-03T03:10:32.122 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:31.299+0000 7f77cc000700 1 --2- 172.21.15.117:0/3446477723 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77c4107fa0 0x7f77c41083e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:10:32.122 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:31.299+0000 7f77cc000700 1 -- 172.21.15.117:0/3446477723 >> 172.21.15.117:0/3446477723 conn(0x7f77c41035b0 msgr2=0x7f77c41059d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:10:32.122 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:31.299+0000 7f77cc000700 1 -- 172.21.15.117:0/3446477723 shutdown_connections 2024-06-03T03:10:32.122 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:31.299+0000 7f77cc000700 1 -- 172.21.15.117:0/3446477723 wait complete. 2024-06-03T03:10:32.122 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:31.300+0000 7f77cc000700 1 Processor -- start 2024-06-03T03:10:32.122 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:31.300+0000 7f77cc000700 1 -- start start 2024-06-03T03:10:32.122 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:31.300+0000 7f77cc000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77c4107fa0 0x7f77c419e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:10:32.122 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:31.300+0000 7f77cc000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f77c410c3a0 con 0x7f77c4107fa0 2024-06-03T03:10:32.122 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:31.301+0000 7f77c9600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77c4107fa0 0x7f77c419e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:10:32.122 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:31.301+0000 7f77c9600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77c4107fa0 0x7f77c419e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:59698/0 (socket says 172.21.15.117:59698) 2024-06-03T03:10:32.122 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:31.301+0000 7f77c9600700 1 -- 172.21.15.117:0/2993493635 learned_addr learned my addr 172.21.15.117:0/2993493635 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:10:32.122 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:31.301+0000 7f77c9600700 1 -- 172.21.15.117:0/2993493635 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77c419e650 con 0x7f77c4107fa0 2024-06-03T03:10:32.122 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:31.301+0000 7f77c9600700 1 --2- 172.21.15.117:0/2993493635 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77c4107fa0 0x7f77c419e110 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f77b8000c00 tx=0x7f77b8011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:10:32.123 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:31.301+0000 7f77c2000700 1 -- 172.21.15.117:0/2993493635 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77b8003b20 con 0x7f77c4107fa0 2024-06-03T03:10:32.123 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:31.302+0000 7f77c2000700 1 -- 172.21.15.117:0/2993493635 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f77b8003cc0 con 0x7f77c4107fa0 2024-06-03T03:10:32.123 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:31.302+0000 7f77cc000700 1 -- 172.21.15.117:0/2993493635 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77c419e8e0 con 0x7f77c4107fa0 2024-06-03T03:10:32.123 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:31.302+0000 7f77c2000700 1 -- 172.21.15.117:0/2993493635 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77b8024910 con 0x7f77c4107fa0 2024-06-03T03:10:32.123 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:31.302+0000 7f77cc000700 1 -- 172.21.15.117:0/2993493635 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77c41a15d0 con 0x7f77c4107fa0 2024-06-03T03:10:32.123 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:31.302+0000 7f77c2000700 1 -- 172.21.15.117:0/2993493635 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f77b8022020 con 0x7f77c4107fa0 2024-06-03T03:10:32.123 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:31.302+0000 7f77c2000700 1 -- 172.21.15.117:0/2993493635 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f77b8011e00 con 0x7f77c4107fa0 2024-06-03T03:10:32.123 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:31.303+0000 7f77cc000700 1 -- 172.21.15.117:0/2993493635 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77c410c520 con 0x7f77c4107fa0 2024-06-03T03:10:32.123 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:31.307+0000 7f77c2000700 1 -- 172.21.15.117:0/2993493635 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f77b802ddf0 con 0x7f77c4107fa0 2024-06-03T03:10:32.123 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:31.381+0000 7f77cc000700 1 -- 172.21.15.117:0/2993493635 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f77c4063af0 con 0x7f77c4107fa0 2024-06-03T03:10:32.123 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:31.382+0000 7f77c2000700 1 -- 172.21.15.117:0/2993493635 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f77b8032030 con 0x7f77c4107fa0 2024-06-03T03:10:32.123 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:31.383+0000 7f77cc000700 1 -- 172.21.15.117:0/2993493635 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77c4107fa0 msgr2=0x7f77c419e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:10:32.123 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:31.383+0000 7f77cc000700 1 --2- 172.21.15.117:0/2993493635 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77c4107fa0 0x7f77c419e110 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f77b8000c00 tx=0x7f77b8011c10 comp rx=0 tx=0).stop 2024-06-03T03:10:32.123 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:31.384+0000 7f77cc000700 1 -- 172.21.15.117:0/2993493635 shutdown_connections 2024-06-03T03:10:32.124 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:31.384+0000 7f77cc000700 1 --2- 172.21.15.117:0/2993493635 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77c4107fa0 0x7f77c419e110 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:10:32.124 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:31.384+0000 7f77cc000700 1 -- 172.21.15.117:0/2993493635 >> 172.21.15.117:0/2993493635 conn(0x7f77c41035b0 msgr2=0x7f77c4104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:10:32.124 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:31.384+0000 7f77cc000700 1 -- 172.21.15.117:0/2993493635 shutdown_connections 2024-06-03T03:10:32.124 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:31.384+0000 7f77cc000700 1 -- 172.21.15.117:0/2993493635 wait complete. 2024-06-03T03:10:32.124 INFO:teuthology.orchestra.run.smithi117.stdout:mgr not available, waiting (2/15)... 2024-06-03T03:10:32.446 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:10:32 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:10:32.210+0000 7fa7a4507200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-03T03:10:32.946 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:10:32 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:10:32.469+0000 7fa7a4507200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-03T03:10:33.446 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:10:33 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:10:33.103+0000 7fa7a4507200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-03T03:10:33.446 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:10:33 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:10:33.223+0000 7fa7a4507200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-03T03:10:33.840 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:10:33 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:10:33.539+0000 7fa7a4507200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-03T03:10:33.840 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:10:33 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:10:33.715+0000 7fa7a4507200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-03T03:10:33.840 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:10:33 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:10:33.839+0000 7fa7a4507200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-03T03:10:34.196 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:10:33 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:10:33.988+0000 7fa7a4507200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-03T03:10:34.582 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:10:34 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:10:34.356+0000 7fa7a4507200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-03T03:10:34.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:34 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/4082662156' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-06-03T03:10:35.416 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:10:35 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:10:35.126+0000 7fa7a4507200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-03T03:10:35.478 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout 2024-06-03T03:10:35.478 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout { 2024-06-03T03:10:35.478 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "fsid": "a0e8473a-2156-11ef-bc9b-c7b262605968", 2024-06-03T03:10:35.478 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "health": { 2024-06-03T03:10:35.478 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-06-03T03:10:35.478 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-06-03T03:10:35.478 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-06-03T03:10:35.478 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-06-03T03:10:35.479 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-06-03T03:10:35.479 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-06-03T03:10:35.479 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout 0 2024-06-03T03:10:35.479 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout ], 2024-06-03T03:10:35.479 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-06-03T03:10:35.479 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "a" 2024-06-03T03:10:35.479 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout ], 2024-06-03T03:10:35.479 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-06-03T03:10:35.479 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "monmap": { 2024-06-03T03:10:35.479 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-03T03:10:35.479 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-06-03T03:10:35.479 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-06-03T03:10:35.479 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-06-03T03:10:35.480 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-06-03T03:10:35.480 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-03T03:10:35.480 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-06-03T03:10:35.480 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-06-03T03:10:35.480 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-06-03T03:10:35.480 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-06-03T03:10:35.480 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-06-03T03:10:35.480 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-06-03T03:10:35.480 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-06-03T03:10:35.480 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-06-03T03:10:35.480 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-06-03T03:10:35.480 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-06-03T03:10:35.480 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-06-03T03:10:35.481 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-06-03T03:10:35.481 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-06-03T03:10:35.481 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-06-03T03:10:35.481 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-06-03T03:10:35.481 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-06-03T03:10:35.481 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-06-03T03:10:35.481 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-06-03T03:10:35.481 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-03T03:10:35.481 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-06-03T03:10:35.481 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-06-03T03:10:35.481 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-06-03T03:10:35.482 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-06-03T03:10:35.482 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "available": false, 2024-06-03T03:10:35.482 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-06-03T03:10:35.482 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "modules": [ 2024-06-03T03:10:35.482 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "iostat", 2024-06-03T03:10:35.482 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "nfs", 2024-06-03T03:10:35.482 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "restful" 2024-06-03T03:10:35.482 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout ], 2024-06-03T03:10:35.482 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-03T03:10:35.482 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-06-03T03:10:35.482 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-06-03T03:10:35.482 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-03T03:10:35.483 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "modified": "2024-06-03T03:10:08.447523+0000", 2024-06-03T03:10:35.483 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-03T03:10:35.483 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-06-03T03:10:35.483 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-06-03T03:10:35.483 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout } 2024-06-03T03:10:35.483 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:34.646+0000 7fb384600700 1 Processor -- start 2024-06-03T03:10:35.483 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:34.646+0000 7fb384600700 1 -- start start 2024-06-03T03:10:35.483 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:34.647+0000 7fb384600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb37c107fa0 0x7fb37c1083e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:10:35.483 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:34.647+0000 7fb384600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb37c108920 con 0x7fb37c107fa0 2024-06-03T03:10:35.483 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:34.647+0000 7fb381c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb37c107fa0 0x7fb37c1083e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:10:35.483 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:34.647+0000 7fb381c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb37c107fa0 0x7fb37c1083e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:59700/0 (socket says 172.21.15.117:59700) 2024-06-03T03:10:35.483 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:34.647+0000 7fb381c00700 1 -- 172.21.15.117:0/769023407 learned_addr learned my addr 172.21.15.117:0/769023407 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:10:35.483 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:34.648+0000 7fb381c00700 1 -- 172.21.15.117:0/769023407 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb37c1090e0 con 0x7fb37c107fa0 2024-06-03T03:10:35.484 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:34.648+0000 7fb381c00700 1 --2- 172.21.15.117:0/769023407 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb37c107fa0 0x7fb37c1083e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb3700098a0 tx=0x7fb370009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfd78df761f9e15 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:10:35.484 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:34.648+0000 7fb37be00700 1 -- 172.21.15.117:0/769023407 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb370004160 con 0x7fb37c107fa0 2024-06-03T03:10:35.484 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:34.648+0000 7fb37be00700 1 -- 172.21.15.117:0/769023407 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb370004300 con 0x7fb37c107fa0 2024-06-03T03:10:35.484 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:34.648+0000 7fb37be00700 1 -- 172.21.15.117:0/769023407 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb370011b50 con 0x7fb37c107fa0 2024-06-03T03:10:35.484 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:34.649+0000 7fb384600700 1 -- 172.21.15.117:0/769023407 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb37c107fa0 msgr2=0x7fb37c1083e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:10:35.484 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:34.649+0000 7fb384600700 1 --2- 172.21.15.117:0/769023407 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb37c107fa0 0x7fb37c1083e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb3700098a0 tx=0x7fb370009bb0 comp rx=0 tx=0).stop 2024-06-03T03:10:35.484 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:34.649+0000 7fb384600700 1 -- 172.21.15.117:0/769023407 shutdown_connections 2024-06-03T03:10:35.484 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:34.649+0000 7fb384600700 1 --2- 172.21.15.117:0/769023407 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb37c107fa0 0x7fb37c1083e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:10:35.484 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:34.649+0000 7fb384600700 1 -- 172.21.15.117:0/769023407 >> 172.21.15.117:0/769023407 conn(0x7fb37c1035b0 msgr2=0x7fb37c1059d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:10:35.484 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:34.649+0000 7fb384600700 1 -- 172.21.15.117:0/769023407 shutdown_connections 2024-06-03T03:10:35.484 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:34.649+0000 7fb384600700 1 -- 172.21.15.117:0/769023407 wait complete. 2024-06-03T03:10:35.484 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:34.650+0000 7fb384600700 1 Processor -- start 2024-06-03T03:10:35.484 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:34.650+0000 7fb384600700 1 -- start start 2024-06-03T03:10:35.485 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:34.650+0000 7fb384600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb37c107fa0 0x7fb37c19e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:10:35.485 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:34.651+0000 7fb384600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb37c10c3a0 con 0x7fb37c107fa0 2024-06-03T03:10:35.485 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:34.651+0000 7fb381c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb37c107fa0 0x7fb37c19e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:10:35.485 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:34.651+0000 7fb381c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb37c107fa0 0x7fb37c19e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:59702/0 (socket says 172.21.15.117:59702) 2024-06-03T03:10:35.485 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:34.651+0000 7fb381c00700 1 -- 172.21.15.117:0/4082662156 learned_addr learned my addr 172.21.15.117:0/4082662156 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:10:35.485 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:34.651+0000 7fb381c00700 1 -- 172.21.15.117:0/4082662156 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb37c19e650 con 0x7fb37c107fa0 2024-06-03T03:10:35.485 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:34.651+0000 7fb381c00700 1 --2- 172.21.15.117:0/4082662156 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb37c107fa0 0x7fb37c19e110 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb370000c00 tx=0x7fb370011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:10:35.485 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:34.652+0000 7fb37a000700 1 -- 172.21.15.117:0/4082662156 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb370003b20 con 0x7fb37c107fa0 2024-06-03T03:10:35.485 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:34.652+0000 7fb37a000700 1 -- 172.21.15.117:0/4082662156 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb370003cc0 con 0x7fb37c107fa0 2024-06-03T03:10:35.485 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:34.652+0000 7fb37a000700 1 -- 172.21.15.117:0/4082662156 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb370024910 con 0x7fb37c107fa0 2024-06-03T03:10:35.485 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:34.652+0000 7fb384600700 1 -- 172.21.15.117:0/4082662156 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb37c19e8e0 con 0x7fb37c107fa0 2024-06-03T03:10:35.485 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:34.652+0000 7fb384600700 1 -- 172.21.15.117:0/4082662156 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb37c1a15d0 con 0x7fb37c107fa0 2024-06-03T03:10:35.486 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:34.652+0000 7fb37a000700 1 -- 172.21.15.117:0/4082662156 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb370022020 con 0x7fb37c107fa0 2024-06-03T03:10:35.486 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:34.652+0000 7fb37a000700 1 -- 172.21.15.117:0/4082662156 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb370011e00 con 0x7fb37c107fa0 2024-06-03T03:10:35.486 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:34.653+0000 7fb384600700 1 -- 172.21.15.117:0/4082662156 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb37c10c520 con 0x7fb37c107fa0 2024-06-03T03:10:35.486 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:34.656+0000 7fb37a000700 1 -- 172.21.15.117:0/4082662156 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fb37001acd0 con 0x7fb37c107fa0 2024-06-03T03:10:35.486 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:34.734+0000 7fb384600700 1 -- 172.21.15.117:0/4082662156 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fb37c063af0 con 0x7fb37c107fa0 2024-06-03T03:10:35.486 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:34.735+0000 7fb37a000700 1 -- 172.21.15.117:0/4082662156 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1240 (secure 0 0 0) 0x7fb370032030 con 0x7fb37c107fa0 2024-06-03T03:10:35.486 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:34.737+0000 7fb384600700 1 -- 172.21.15.117:0/4082662156 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb37c107fa0 msgr2=0x7fb37c19e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:10:35.486 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:34.737+0000 7fb384600700 1 --2- 172.21.15.117:0/4082662156 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb37c107fa0 0x7fb37c19e110 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb370000c00 tx=0x7fb370011c10 comp rx=0 tx=0).stop 2024-06-03T03:10:35.486 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:34.737+0000 7fb384600700 1 -- 172.21.15.117:0/4082662156 shutdown_connections 2024-06-03T03:10:35.486 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:34.737+0000 7fb384600700 1 --2- 172.21.15.117:0/4082662156 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb37c107fa0 0x7fb37c19e110 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:10:35.486 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:34.737+0000 7fb384600700 1 -- 172.21.15.117:0/4082662156 >> 172.21.15.117:0/4082662156 conn(0x7fb37c1035b0 msgr2=0x7fb37c1040a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:10:35.486 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:34.737+0000 7fb384600700 1 -- 172.21.15.117:0/4082662156 shutdown_connections 2024-06-03T03:10:35.487 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:34.737+0000 7fb384600700 1 -- 172.21.15.117:0/4082662156 wait complete. 2024-06-03T03:10:35.487 INFO:teuthology.orchestra.run.smithi117.stdout:mgr not available, waiting (3/15)... 2024-06-03T03:10:35.696 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:10:35 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:10:35.415+0000 7fa7a4507200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-03T03:10:36.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:35 smithi117 ceph-mon[31649]: Activating manager daemon x 2024-06-03T03:10:36.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:35 smithi117 ceph-mon[31649]: mgrmap e2: x(active, starting, since 0.0102092s) 2024-06-03T03:10:36.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:35 smithi117 ceph-mon[31649]: from='mgr.14100 172.21.15.117:0/2092201765' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-03T03:10:36.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:35 smithi117 ceph-mon[31649]: from='mgr.14100 172.21.15.117:0/2092201765' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-03T03:10:36.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:35 smithi117 ceph-mon[31649]: from='mgr.14100 172.21.15.117:0/2092201765' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-03T03:10:36.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:35 smithi117 ceph-mon[31649]: from='mgr.14100 172.21.15.117:0/2092201765' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-03T03:10:36.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:35 smithi117 ceph-mon[31649]: from='mgr.14100 172.21.15.117:0/2092201765' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-03T03:10:36.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:35 smithi117 ceph-mon[31649]: from='mgr.14100 172.21.15.117:0/2092201765' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-03T03:10:36.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:35 smithi117 ceph-mon[31649]: from='mgr.14100 172.21.15.117:0/2092201765' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-03T03:10:36.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:35 smithi117 ceph-mon[31649]: from='mgr.14100 172.21.15.117:0/2092201765' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-03T03:10:36.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:35 smithi117 ceph-mon[31649]: Manager daemon x is now available 2024-06-03T03:10:36.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:35 smithi117 ceph-mon[31649]: from='mgr.14100 172.21.15.117:0/2092201765' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-03T03:10:36.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:35 smithi117 ceph-mon[31649]: from='mgr.14100 172.21.15.117:0/2092201765' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-03T03:10:36.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:35 smithi117 ceph-mon[31649]: from='mgr.14100 172.21.15.117:0/2092201765' entity='mgr.x' 2024-06-03T03:10:36.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:35 smithi117 ceph-mon[31649]: from='mgr.14100 172.21.15.117:0/2092201765' entity='mgr.x' 2024-06-03T03:10:36.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:35 smithi117 ceph-mon[31649]: from='mgr.14100 172.21.15.117:0/2092201765' entity='mgr.x' 2024-06-03T03:10:37.696 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:37 smithi117 ceph-mon[31649]: mgrmap e3: x(active, since 1.0181s) 2024-06-03T03:10:38.696 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:38 smithi117 ceph-mon[31649]: mgrmap e4: x(active, since 2s) 2024-06-03T03:10:39.139 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout 2024-06-03T03:10:39.139 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout { 2024-06-03T03:10:39.140 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "fsid": "a0e8473a-2156-11ef-bc9b-c7b262605968", 2024-06-03T03:10:39.140 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "health": { 2024-06-03T03:10:39.140 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-06-03T03:10:39.140 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-06-03T03:10:39.141 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-06-03T03:10:39.141 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-06-03T03:10:39.141 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-06-03T03:10:39.141 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-06-03T03:10:39.141 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout 0 2024-06-03T03:10:39.142 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout ], 2024-06-03T03:10:39.142 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-06-03T03:10:39.142 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "a" 2024-06-03T03:10:39.142 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout ], 2024-06-03T03:10:39.142 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-06-03T03:10:39.142 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "monmap": { 2024-06-03T03:10:39.142 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-03T03:10:39.142 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-06-03T03:10:39.144 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-06-03T03:10:39.144 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-06-03T03:10:39.144 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-06-03T03:10:39.144 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-03T03:10:39.144 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-06-03T03:10:39.144 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-06-03T03:10:39.144 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-06-03T03:10:39.144 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-06-03T03:10:39.144 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-06-03T03:10:39.144 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-06-03T03:10:39.147 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-06-03T03:10:39.147 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-06-03T03:10:39.147 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-06-03T03:10:39.147 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-06-03T03:10:39.148 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-06-03T03:10:39.148 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-06-03T03:10:39.148 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-06-03T03:10:39.148 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-06-03T03:10:39.148 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-06-03T03:10:39.148 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-06-03T03:10:39.148 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-06-03T03:10:39.148 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-06-03T03:10:39.148 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-03T03:10:39.148 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-06-03T03:10:39.148 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-06-03T03:10:39.148 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-06-03T03:10:39.148 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-06-03T03:10:39.149 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "available": true, 2024-06-03T03:10:39.149 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-06-03T03:10:39.149 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "modules": [ 2024-06-03T03:10:39.149 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "iostat", 2024-06-03T03:10:39.149 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "nfs", 2024-06-03T03:10:39.149 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "restful" 2024-06-03T03:10:39.149 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout ], 2024-06-03T03:10:39.149 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-03T03:10:39.149 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-06-03T03:10:39.149 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-06-03T03:10:39.149 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-03T03:10:39.150 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "modified": "2024-06-03T03:10:08.447523+0000", 2024-06-03T03:10:39.155 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-03T03:10:39.155 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-06-03T03:10:39.155 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-06-03T03:10:39.155 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout } 2024-06-03T03:10:39.155 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:38.132+0000 7f5515400700 1 Processor -- start 2024-06-03T03:10:39.155 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:38.132+0000 7f5515400700 1 -- start start 2024-06-03T03:10:39.155 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:38.132+0000 7f5515400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5510081e10 0x7f5510082230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:10:39.156 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:38.132+0000 7f5515400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5510082800 con 0x7f5510081e10 2024-06-03T03:10:39.156 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:38.133+0000 7f550fe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5510081e10 0x7f5510082230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:10:39.156 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:38.133+0000 7f550fe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5510081e10 0x7f5510082230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56874/0 (socket says 172.21.15.117:56874) 2024-06-03T03:10:39.156 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:38.133+0000 7f550fe00700 1 -- 172.21.15.117:0/3661274334 learned_addr learned my addr 172.21.15.117:0/3661274334 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:10:39.156 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:38.133+0000 7f550fe00700 1 -- 172.21.15.117:0/3661274334 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f551010f600 con 0x7f5510081e10 2024-06-03T03:10:39.156 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:38.133+0000 7f550fe00700 1 --2- 172.21.15.117:0/3661274334 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5510081e10 0x7f5510082230 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f55000098a0 tx=0x7f5500009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=accd8f14a3a597de server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:10:39.156 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:38.134+0000 7f550ea00700 1 -- 172.21.15.117:0/3661274334 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5500004160 con 0x7f5510081e10 2024-06-03T03:10:39.156 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:38.134+0000 7f550ea00700 1 -- 172.21.15.117:0/3661274334 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5500004300 con 0x7f5510081e10 2024-06-03T03:10:39.156 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:38.134+0000 7f550ea00700 1 -- 172.21.15.117:0/3661274334 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5500011b50 con 0x7f5510081e10 2024-06-03T03:10:39.156 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:38.134+0000 7f5515400700 1 -- 172.21.15.117:0/3661274334 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5510081e10 msgr2=0x7f5510082230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:10:39.157 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:38.134+0000 7f5515400700 1 --2- 172.21.15.117:0/3661274334 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5510081e10 0x7f5510082230 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f55000098a0 tx=0x7f5500009bb0 comp rx=0 tx=0).stop 2024-06-03T03:10:39.157 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:38.134+0000 7f5515400700 1 -- 172.21.15.117:0/3661274334 shutdown_connections 2024-06-03T03:10:39.157 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:38.134+0000 7f5515400700 1 --2- 172.21.15.117:0/3661274334 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5510081e10 0x7f5510082230 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:10:39.157 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:38.134+0000 7f5515400700 1 -- 172.21.15.117:0/3661274334 >> 172.21.15.117:0/3661274334 conn(0x7f551007d370 msgr2=0x7f551007f7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:10:39.157 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:38.135+0000 7f5515400700 1 -- 172.21.15.117:0/3661274334 shutdown_connections 2024-06-03T03:10:39.157 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:38.135+0000 7f5515400700 1 -- 172.21.15.117:0/3661274334 wait complete. 2024-06-03T03:10:39.157 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:38.136+0000 7f5515400700 1 Processor -- start 2024-06-03T03:10:39.157 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:38.136+0000 7f5515400700 1 -- start start 2024-06-03T03:10:39.157 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:38.136+0000 7f5515400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5510081e10 0x7f55101a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:10:39.157 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:38.136+0000 7f5515400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f55101129b0 con 0x7f5510081e10 2024-06-03T03:10:39.157 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:38.136+0000 7f550fe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5510081e10 0x7f55101a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:10:39.157 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:38.136+0000 7f550fe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5510081e10 0x7f55101a2410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56886/0 (socket says 172.21.15.117:56886) 2024-06-03T03:10:39.157 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:38.136+0000 7f550fe00700 1 -- 172.21.15.117:0/3241504469 learned_addr learned my addr 172.21.15.117:0/3241504469 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:10:39.158 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:38.137+0000 7f550fe00700 1 -- 172.21.15.117:0/3241504469 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55101a2950 con 0x7f5510081e10 2024-06-03T03:10:39.158 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:38.137+0000 7f550fe00700 1 --2- 172.21.15.117:0/3241504469 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5510081e10 0x7f55101a2410 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5500011b60 tx=0x7f5500011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:10:39.158 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:38.137+0000 7f550cc00700 1 -- 172.21.15.117:0/3241504469 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5500003970 con 0x7f5510081e10 2024-06-03T03:10:39.158 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:38.137+0000 7f550cc00700 1 -- 172.21.15.117:0/3241504469 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5500003b10 con 0x7f5510081e10 2024-06-03T03:10:39.158 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:38.137+0000 7f550cc00700 1 -- 172.21.15.117:0/3241504469 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5500024580 con 0x7f5510081e10 2024-06-03T03:10:39.158 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:38.137+0000 7f5515400700 1 -- 172.21.15.117:0/3241504469 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55101a2be0 con 0x7f5510081e10 2024-06-03T03:10:39.158 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:38.138+0000 7f5515400700 1 -- 172.21.15.117:0/3241504469 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55101a58d0 con 0x7f5510081e10 2024-06-03T03:10:39.158 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:38.139+0000 7f5515400700 1 -- 172.21.15.117:0/3241504469 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55100695b0 con 0x7f5510081e10 2024-06-03T03:10:39.158 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:38.139+0000 7f550cc00700 1 -- 172.21.15.117:0/3241504469 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f550001a430 con 0x7f5510081e10 2024-06-03T03:10:39.158 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:38.139+0000 7f550cc00700 1 --2- 172.21.15.117:0/3241504469 >> [v2:172.21.15.117:6800/2424554985,v1:172.21.15.117:6801/2424554985] conn(0x7f54f4039050 0x7f54f403b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:10:39.158 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:38.139+0000 7f550cc00700 1 -- 172.21.15.117:0/3241504469 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5500011e00 con 0x7f5510081e10 2024-06-03T03:10:39.158 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:38.140+0000 7f550f400700 1 --2- 172.21.15.117:0/3241504469 >> [v2:172.21.15.117:6800/2424554985,v1:172.21.15.117:6801/2424554985] conn(0x7f54f4039050 0x7f54f403b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:10:39.159 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:38.141+0000 7f550f400700 1 --2- 172.21.15.117:0/3241504469 >> [v2:172.21.15.117:6800/2424554985,v1:172.21.15.117:6801/2424554985] conn(0x7f54f4039050 0x7f54f403b510 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f54fc006fd0 tx=0x7f54fc006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:10:39.159 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:38.146+0000 7f550cc00700 1 -- 172.21.15.117:0/3241504469 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5500016260 con 0x7f5510081e10 2024-06-03T03:10:39.159 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:38.449+0000 7f5515400700 1 -- 172.21.15.117:0/3241504469 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f55101a5aa0 con 0x7f5510081e10 2024-06-03T03:10:39.159 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:38.450+0000 7f550cc00700 1 -- 172.21.15.117:0/3241504469 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f5500019bf0 con 0x7f5510081e10 2024-06-03T03:10:39.159 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:38.455+0000 7f5515400700 1 -- 172.21.15.117:0/3241504469 >> [v2:172.21.15.117:6800/2424554985,v1:172.21.15.117:6801/2424554985] conn(0x7f54f4039050 msgr2=0x7f54f403b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:10:39.159 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:38.455+0000 7f5515400700 1 --2- 172.21.15.117:0/3241504469 >> [v2:172.21.15.117:6800/2424554985,v1:172.21.15.117:6801/2424554985] conn(0x7f54f4039050 0x7f54f403b510 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f54fc006fd0 tx=0x7f54fc006e40 comp rx=0 tx=0).stop 2024-06-03T03:10:39.159 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:38.455+0000 7f5515400700 1 -- 172.21.15.117:0/3241504469 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5510081e10 msgr2=0x7f55101a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:10:39.159 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:38.455+0000 7f5515400700 1 --2- 172.21.15.117:0/3241504469 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5510081e10 0x7f55101a2410 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5500011b60 tx=0x7f5500011c40 comp rx=0 tx=0).stop 2024-06-03T03:10:39.160 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:38.455+0000 7f5515400700 1 -- 172.21.15.117:0/3241504469 shutdown_connections 2024-06-03T03:10:39.160 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:38.455+0000 7f5515400700 1 --2- 172.21.15.117:0/3241504469 >> [v2:172.21.15.117:6800/2424554985,v1:172.21.15.117:6801/2424554985] conn(0x7f54f4039050 0x7f54f403b510 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:10:39.160 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:38.455+0000 7f5515400700 1 --2- 172.21.15.117:0/3241504469 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5510081e10 0x7f55101a2410 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:10:39.160 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:38.455+0000 7f5515400700 1 -- 172.21.15.117:0/3241504469 >> 172.21.15.117:0/3241504469 conn(0x7f551007d370 msgr2=0x7f551007de40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:10:39.160 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:38.456+0000 7f5515400700 1 -- 172.21.15.117:0/3241504469 shutdown_connections 2024-06-03T03:10:39.160 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:38.456+0000 7f5515400700 1 -- 172.21.15.117:0/3241504469 wait complete. 2024-06-03T03:10:39.160 INFO:teuthology.orchestra.run.smithi117.stdout:mgr is available 2024-06-03T03:10:39.696 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:39 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/3241504469' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-06-03T03:10:40.655 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:40 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/2452955051' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-06-03T03:10:40.665 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout 2024-06-03T03:10:40.665 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout [global] 2024-06-03T03:10:40.665 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout fsid = a0e8473a-2156-11ef-bc9b-c7b262605968 2024-06-03T03:10:40.665 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-06-03T03:10:40.672 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-06-03T03:10:40.672 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-06-03T03:10:40.672 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-06-03T03:10:40.672 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout 2024-06-03T03:10:40.672 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout [mgr] 2024-06-03T03:10:40.672 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-06-03T03:10:40.672 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout 2024-06-03T03:10:40.672 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout [osd] 2024-06-03T03:10:40.673 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-06-03T03:10:40.673 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-06-03T03:10:40.673 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-06-03T03:10:40.673 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-06-03T03:10:40.673 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-06-03T03:10:40.673 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:39.735+0000 7fb810a00700 1 Processor -- start 2024-06-03T03:10:40.673 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:39.735+0000 7fb810a00700 1 -- start start 2024-06-03T03:10:40.673 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:39.736+0000 7fb810a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb80c10a100 0x7fb80c10a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:10:40.673 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:39.736+0000 7fb810a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb80c10aaf0 con 0x7fb80c10a100 2024-06-03T03:10:40.673 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:39.736+0000 7fb80a800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb80c10a100 0x7fb80c10a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:10:40.673 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:39.736+0000 7fb80a800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb80c10a100 0x7fb80c10a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56890/0 (socket says 172.21.15.117:56890) 2024-06-03T03:10:40.673 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:39.736+0000 7fb80a800700 1 -- 172.21.15.117:0/2892267003 learned_addr learned my addr 172.21.15.117:0/2892267003 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:10:40.673 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:39.737+0000 7fb80a800700 1 -- 172.21.15.117:0/2892267003 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb80c10b1c0 con 0x7fb80c10a100 2024-06-03T03:10:40.674 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:39.737+0000 7fb80a800700 1 --2- 172.21.15.117:0/2892267003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb80c10a100 0x7fb80c10a520 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc0098a0 tx=0x7fb7fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7053f8d770d8682 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:10:40.674 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:39.738+0000 7fb809400700 1 -- 172.21.15.117:0/2892267003 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb7fc004160 con 0x7fb80c10a100 2024-06-03T03:10:40.674 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:39.738+0000 7fb809400700 1 -- 172.21.15.117:0/2892267003 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb7fc004300 con 0x7fb80c10a100 2024-06-03T03:10:40.674 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:39.738+0000 7fb809400700 1 -- 172.21.15.117:0/2892267003 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb7fc011b50 con 0x7fb80c10a100 2024-06-03T03:10:40.674 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:39.738+0000 7fb810a00700 1 -- 172.21.15.117:0/2892267003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb80c10a100 msgr2=0x7fb80c10a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:10:40.674 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:39.738+0000 7fb810a00700 1 --2- 172.21.15.117:0/2892267003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb80c10a100 0x7fb80c10a520 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc0098a0 tx=0x7fb7fc009bb0 comp rx=0 tx=0).stop 2024-06-03T03:10:40.674 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:39.738+0000 7fb810a00700 1 -- 172.21.15.117:0/2892267003 shutdown_connections 2024-06-03T03:10:40.674 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:39.738+0000 7fb810a00700 1 --2- 172.21.15.117:0/2892267003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb80c10a100 0x7fb80c10a520 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:10:40.674 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:39.738+0000 7fb810a00700 1 -- 172.21.15.117:0/2892267003 >> 172.21.15.117:0/2892267003 conn(0x7fb80c07d370 msgr2=0x7fb80c07d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:10:40.674 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:39.739+0000 7fb810a00700 1 -- 172.21.15.117:0/2892267003 shutdown_connections 2024-06-03T03:10:40.674 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:39.739+0000 7fb810a00700 1 -- 172.21.15.117:0/2892267003 wait complete. 2024-06-03T03:10:40.674 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:39.740+0000 7fb810a00700 1 Processor -- start 2024-06-03T03:10:40.675 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:39.740+0000 7fb810a00700 1 -- start start 2024-06-03T03:10:40.675 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:39.741+0000 7fb810a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb80c10a100 0x7fb80c19dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:10:40.675 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:39.741+0000 7fb810a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb80c10e570 con 0x7fb80c10a100 2024-06-03T03:10:40.675 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:39.741+0000 7fb80a800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb80c10a100 0x7fb80c19dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:10:40.675 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:39.741+0000 7fb80a800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb80c10a100 0x7fb80c19dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56894/0 (socket says 172.21.15.117:56894) 2024-06-03T03:10:40.675 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:39.741+0000 7fb80a800700 1 -- 172.21.15.117:0/2452955051 learned_addr learned my addr 172.21.15.117:0/2452955051 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:10:40.675 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:39.741+0000 7fb80a800700 1 -- 172.21.15.117:0/2452955051 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb80c19e500 con 0x7fb80c10a100 2024-06-03T03:10:40.675 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:39.742+0000 7fb80a800700 1 --2- 172.21.15.117:0/2452955051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb80c10a100 0x7fb80c19dfc0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc011b60 tx=0x7fb7fc011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:10:40.675 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:39.742+0000 7fb803400700 1 -- 172.21.15.117:0/2452955051 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb7fc003970 con 0x7fb80c10a100 2024-06-03T03:10:40.675 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:39.742+0000 7fb803400700 1 -- 172.21.15.117:0/2452955051 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb7fc003b10 con 0x7fb80c10a100 2024-06-03T03:10:40.675 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:39.742+0000 7fb803400700 1 -- 172.21.15.117:0/2452955051 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb7fc024610 con 0x7fb80c10a100 2024-06-03T03:10:40.675 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:39.742+0000 7fb810a00700 1 -- 172.21.15.117:0/2452955051 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb80c19e790 con 0x7fb80c10a100 2024-06-03T03:10:40.675 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:39.742+0000 7fb810a00700 1 -- 172.21.15.117:0/2452955051 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb80c1a1480 con 0x7fb80c10a100 2024-06-03T03:10:40.675 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:39.743+0000 7fb803400700 1 -- 172.21.15.117:0/2452955051 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7fb7fc01a9f0 con 0x7fb80c10a100 2024-06-03T03:10:40.675 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:39.743+0000 7fb810a00700 1 -- 172.21.15.117:0/2452955051 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb80c0695b0 con 0x7fb80c10a100 2024-06-03T03:10:40.676 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:39.744+0000 7fb803400700 1 --2- 172.21.15.117:0/2452955051 >> [v2:172.21.15.117:6800/2424554985,v1:172.21.15.117:6801/2424554985] conn(0x7fb7f0039050 0x7fb7f003b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:10:40.676 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:39.744+0000 7fb803400700 1 -- 172.21.15.117:0/2452955051 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb7fc011e00 con 0x7fb80c10a100 2024-06-03T03:10:40.676 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:39.744+0000 7fb809e00700 1 --2- 172.21.15.117:0/2452955051 >> [v2:172.21.15.117:6800/2424554985,v1:172.21.15.117:6801/2424554985] conn(0x7fb7f0039050 0x7fb7f003b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:10:40.676 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:39.745+0000 7fb809e00700 1 --2- 172.21.15.117:0/2452955051 >> [v2:172.21.15.117:6800/2424554985,v1:172.21.15.117:6801/2424554985] conn(0x7fb7f0039050 0x7fb7f003b510 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb7f8006fd0 tx=0x7fb7f8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:10:40.676 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:39.749+0000 7fb803400700 1 -- 172.21.15.117:0/2452955051 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb7fc00b3e0 con 0x7fb80c10a100 2024-06-03T03:10:40.676 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:39.972+0000 7fb810a00700 1 -- 172.21.15.117:0/2452955051 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fb80c1a2bf0 con 0x7fb80c10a100 2024-06-03T03:10:40.676 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:39.973+0000 7fb803400700 1 -- 172.21.15.117:0/2452955051 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+373 (secure 0 0 0) 0x7fb7fc04cbb0 con 0x7fb80c10a100 2024-06-03T03:10:40.676 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:39.976+0000 7fb810a00700 1 -- 172.21.15.117:0/2452955051 >> [v2:172.21.15.117:6800/2424554985,v1:172.21.15.117:6801/2424554985] conn(0x7fb7f0039050 msgr2=0x7fb7f003b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:10:40.676 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:39.976+0000 7fb810a00700 1 --2- 172.21.15.117:0/2452955051 >> [v2:172.21.15.117:6800/2424554985,v1:172.21.15.117:6801/2424554985] conn(0x7fb7f0039050 0x7fb7f003b510 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb7f8006fd0 tx=0x7fb7f8006e40 comp rx=0 tx=0).stop 2024-06-03T03:10:40.676 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:39.976+0000 7fb810a00700 1 -- 172.21.15.117:0/2452955051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb80c10a100 msgr2=0x7fb80c19dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:10:40.676 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:39.976+0000 7fb810a00700 1 --2- 172.21.15.117:0/2452955051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb80c10a100 0x7fb80c19dfc0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc011b60 tx=0x7fb7fc011c40 comp rx=0 tx=0).stop 2024-06-03T03:10:40.676 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:39.977+0000 7fb810a00700 1 -- 172.21.15.117:0/2452955051 shutdown_connections 2024-06-03T03:10:40.676 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:39.977+0000 7fb810a00700 1 --2- 172.21.15.117:0/2452955051 >> [v2:172.21.15.117:6800/2424554985,v1:172.21.15.117:6801/2424554985] conn(0x7fb7f0039050 0x7fb7f003b510 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:10:40.676 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:39.977+0000 7fb810a00700 1 --2- 172.21.15.117:0/2452955051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb80c10a100 0x7fb80c19dfc0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:10:40.677 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:39.977+0000 7fb810a00700 1 -- 172.21.15.117:0/2452955051 >> 172.21.15.117:0/2452955051 conn(0x7fb80c07d370 msgr2=0x7fb80c107db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:10:40.677 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:39.977+0000 7fb810a00700 1 -- 172.21.15.117:0/2452955051 shutdown_connections 2024-06-03T03:10:40.677 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:39.977+0000 7fb810a00700 1 -- 172.21.15.117:0/2452955051 wait complete. 2024-06-03T03:10:40.677 INFO:teuthology.orchestra.run.smithi117.stdout:Enabling cephadm module... 2024-06-03T03:10:41.922 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:41 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/1106029024' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-06-03T03:10:41.923 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:10:41 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: ignoring --setuser ceph since I am not root 2024-06-03T03:10:41.923 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:10:41 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: ignoring --setgroup ceph since I am not root 2024-06-03T03:10:41.923 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:10:41 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:10:41.921+0000 7f64cc58a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-03T03:10:42.196 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:10:42 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:10:42.162+0000 7f64cc58a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-03T03:10:42.417 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:41.281+0000 7fa3a9e00700 1 Processor -- start 2024-06-03T03:10:42.418 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:41.281+0000 7fa3a9e00700 1 -- start start 2024-06-03T03:10:42.418 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:41.282+0000 7fa3a9e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3a41089f0 0x7fa3a410ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:10:42.418 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:41.282+0000 7fa3a9e00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a407c700 con 0x7fa3a41089f0 2024-06-03T03:10:42.418 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:41.282+0000 7fa3a2a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3a41089f0 0x7fa3a410ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:10:42.418 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:41.282+0000 7fa3a2a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3a41089f0 0x7fa3a410ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56904/0 (socket says 172.21.15.117:56904) 2024-06-03T03:10:42.418 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:41.282+0000 7fa3a2a00700 1 -- 172.21.15.117:0/1371289077 learned_addr learned my addr 172.21.15.117:0/1371289077 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:10:42.418 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:41.283+0000 7fa3a2a00700 1 -- 172.21.15.117:0/1371289077 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3a410b320 con 0x7fa3a41089f0 2024-06-03T03:10:42.418 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:41.283+0000 7fa3a2a00700 1 --2- 172.21.15.117:0/1371289077 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3a41089f0 0x7fa3a410ade0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa3940098a0 tx=0x7fa394009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58c76afdb5664916 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:10:42.418 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:41.283+0000 7fa3a1600700 1 -- 172.21.15.117:0/1371289077 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa394004160 con 0x7fa3a41089f0 2024-06-03T03:10:42.418 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:41.283+0000 7fa3a1600700 1 -- 172.21.15.117:0/1371289077 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa394004300 con 0x7fa3a41089f0 2024-06-03T03:10:42.418 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:41.283+0000 7fa3a1600700 1 -- 172.21.15.117:0/1371289077 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa394011b50 con 0x7fa3a41089f0 2024-06-03T03:10:42.418 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:41.284+0000 7fa3a9e00700 1 -- 172.21.15.117:0/1371289077 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3a41089f0 msgr2=0x7fa3a410ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:10:42.419 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:41.284+0000 7fa3a9e00700 1 --2- 172.21.15.117:0/1371289077 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3a41089f0 0x7fa3a410ade0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa3940098a0 tx=0x7fa394009bb0 comp rx=0 tx=0).stop 2024-06-03T03:10:42.419 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:41.284+0000 7fa3a9e00700 1 -- 172.21.15.117:0/1371289077 shutdown_connections 2024-06-03T03:10:42.419 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:41.284+0000 7fa3a9e00700 1 --2- 172.21.15.117:0/1371289077 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3a41089f0 0x7fa3a410ade0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:10:42.419 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:41.284+0000 7fa3a9e00700 1 -- 172.21.15.117:0/1371289077 >> 172.21.15.117:0/1371289077 conn(0x7fa3a4102420 msgr2=0x7fa3a4104840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:10:42.419 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:41.284+0000 7fa3a9e00700 1 -- 172.21.15.117:0/1371289077 shutdown_connections 2024-06-03T03:10:42.419 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:41.284+0000 7fa3a9e00700 1 -- 172.21.15.117:0/1371289077 wait complete. 2024-06-03T03:10:42.419 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:41.286+0000 7fa3a9e00700 1 Processor -- start 2024-06-03T03:10:42.419 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:41.286+0000 7fa3a9e00700 1 -- start start 2024-06-03T03:10:42.419 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:41.287+0000 7fa3a9e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3a41089f0 0x7fa3a419bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:10:42.419 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:41.287+0000 7fa3a9e00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a410c340 con 0x7fa3a41089f0 2024-06-03T03:10:42.419 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:41.287+0000 7fa3a2a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3a41089f0 0x7fa3a419bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:10:42.419 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:41.287+0000 7fa3a2a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3a41089f0 0x7fa3a419bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56916/0 (socket says 172.21.15.117:56916) 2024-06-03T03:10:42.419 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:41.287+0000 7fa3a2a00700 1 -- 172.21.15.117:0/1106029024 learned_addr learned my addr 172.21.15.117:0/1106029024 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:10:42.419 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:41.287+0000 7fa3a2a00700 1 -- 172.21.15.117:0/1106029024 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3a419c410 con 0x7fa3a41089f0 2024-06-03T03:10:42.420 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:41.288+0000 7fa3a2a00700 1 --2- 172.21.15.117:0/1106029024 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3a41089f0 0x7fa3a419bed0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa394011670 tx=0x7fa394011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:10:42.420 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:41.288+0000 7fa3a8a00700 1 -- 172.21.15.117:0/1106029024 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa3940039c0 con 0x7fa3a41089f0 2024-06-03T03:10:42.420 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:41.288+0000 7fa3a8a00700 1 -- 172.21.15.117:0/1106029024 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa394011e50 con 0x7fa3a41089f0 2024-06-03T03:10:42.420 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:41.288+0000 7fa3a9e00700 1 -- 172.21.15.117:0/1106029024 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3a419c6a0 con 0x7fa3a41089f0 2024-06-03T03:10:42.420 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:41.288+0000 7fa3a8a00700 1 -- 172.21.15.117:0/1106029024 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa39401a5c0 con 0x7fa3a41089f0 2024-06-03T03:10:42.420 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:41.288+0000 7fa3a9e00700 1 -- 172.21.15.117:0/1106029024 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3a419f390 con 0x7fa3a41089f0 2024-06-03T03:10:42.420 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:41.289+0000 7fa3a8a00700 1 -- 172.21.15.117:0/1106029024 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7fa394024430 con 0x7fa3a41089f0 2024-06-03T03:10:42.420 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:41.289+0000 7fa3a9e00700 1 -- 172.21.15.117:0/1106029024 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3a4063af0 con 0x7fa3a41089f0 2024-06-03T03:10:42.420 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:41.290+0000 7fa3a8a00700 1 --2- 172.21.15.117:0/1106029024 >> [v2:172.21.15.117:6800/2424554985,v1:172.21.15.117:6801/2424554985] conn(0x7fa388039000 0x7fa38803b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:10:42.420 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:41.290+0000 7fa3a8a00700 1 -- 172.21.15.117:0/1106029024 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa39404d310 con 0x7fa3a41089f0 2024-06-03T03:10:42.420 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:41.290+0000 7fa3a2000700 1 --2- 172.21.15.117:0/1106029024 >> [v2:172.21.15.117:6800/2424554985,v1:172.21.15.117:6801/2424554985] conn(0x7fa388039000 0x7fa38803b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:10:42.420 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:41.291+0000 7fa3a2000700 1 --2- 172.21.15.117:0/1106029024 >> [v2:172.21.15.117:6800/2424554985,v1:172.21.15.117:6801/2424554985] conn(0x7fa388039000 0x7fa38803b4c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa390006fd0 tx=0x7fa390006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:10:42.420 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:41.295+0000 7fa3a8a00700 1 -- 172.21.15.117:0/1106029024 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa39404cd30 con 0x7fa3a41089f0 2024-06-03T03:10:42.421 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:41.576+0000 7fa3a9e00700 1 -- 172.21.15.117:0/1106029024 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7fa3a419f560 con 0x7fa3a41089f0 2024-06-03T03:10:42.421 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:41.629+0000 7fa3a8a00700 1 -- 172.21.15.117:0/1106029024 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7fa39402d3f0 con 0x7fa3a41089f0 2024-06-03T03:10:42.421 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:41.632+0000 7fa3a8a00700 1 -- 172.21.15.117:0/1106029024 <== mon.0 v2:172.21.15.117:3300/0 8 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7fa394024720 con 0x7fa3a41089f0 2024-06-03T03:10:42.421 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:41.633+0000 7fa3a9e00700 1 -- 172.21.15.117:0/1106029024 >> [v2:172.21.15.117:6800/2424554985,v1:172.21.15.117:6801/2424554985] conn(0x7fa388039000 msgr2=0x7fa38803b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:10:42.421 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:41.633+0000 7fa3a9e00700 1 --2- 172.21.15.117:0/1106029024 >> [v2:172.21.15.117:6800/2424554985,v1:172.21.15.117:6801/2424554985] conn(0x7fa388039000 0x7fa38803b4c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa390006fd0 tx=0x7fa390006e40 comp rx=0 tx=0).stop 2024-06-03T03:10:42.421 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:41.633+0000 7fa3a9e00700 1 -- 172.21.15.117:0/1106029024 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3a41089f0 msgr2=0x7fa3a419bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:10:42.421 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:41.633+0000 7fa3a9e00700 1 --2- 172.21.15.117:0/1106029024 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3a41089f0 0x7fa3a419bed0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa394011670 tx=0x7fa394011b60 comp rx=0 tx=0).stop 2024-06-03T03:10:42.421 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:41.633+0000 7fa3a9e00700 1 -- 172.21.15.117:0/1106029024 shutdown_connections 2024-06-03T03:10:42.422 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:41.633+0000 7fa3a9e00700 1 --2- 172.21.15.117:0/1106029024 >> [v2:172.21.15.117:6800/2424554985,v1:172.21.15.117:6801/2424554985] conn(0x7fa388039000 0x7fa38803b4c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:10:42.422 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:41.633+0000 7fa3a9e00700 1 --2- 172.21.15.117:0/1106029024 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3a41089f0 0x7fa3a419bed0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:10:42.422 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:41.633+0000 7fa3a9e00700 1 -- 172.21.15.117:0/1106029024 >> 172.21.15.117:0/1106029024 conn(0x7fa3a4102420 msgr2=0x7fa3a4102f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:10:42.422 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:41.634+0000 7fa3a9e00700 1 -- 172.21.15.117:0/1106029024 shutdown_connections 2024-06-03T03:10:42.422 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:41.634+0000 7fa3a9e00700 1 -- 172.21.15.117:0/1106029024 wait complete. 2024-06-03T03:10:42.569 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:10:42 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:10:42.296+0000 7f64cc58a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-03T03:10:42.830 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:42 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/1106029024' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-06-03T03:10:42.830 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:42 smithi117 ceph-mon[31649]: mgrmap e5: x(active, since 6s) 2024-06-03T03:10:42.831 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:10:42 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:10:42.709+0000 7f64cc58a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-03T03:10:43.196 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:10:42 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:10:42.829+0000 7f64cc58a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-03T03:10:43.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:43 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/1475542701' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-06-03T03:10:43.995 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout { 2024-06-03T03:10:43.996 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-06-03T03:10:43.996 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "available": true, 2024-06-03T03:10:43.996 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-06-03T03:10:43.996 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-06-03T03:10:43.996 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout } 2024-06-03T03:10:43.996 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:43.034+0000 7fd016e00700 1 Processor -- start 2024-06-03T03:10:43.996 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:43.034+0000 7fd016e00700 1 -- start start 2024-06-03T03:10:43.996 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:43.034+0000 7fd016e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd01007de00 0x7fd01007c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:10:43.996 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:43.034+0000 7fd016e00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd01007c840 con 0x7fd01007de00 2024-06-03T03:10:43.996 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:43.035+0000 7fd00fe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd01007de00 0x7fd01007c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:10:43.996 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:43.035+0000 7fd00fe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd01007de00 0x7fd01007c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56932/0 (socket says 172.21.15.117:56932) 2024-06-03T03:10:43.996 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:43.035+0000 7fd00fe00700 1 -- 172.21.15.117:0/2029144191 learned_addr learned my addr 172.21.15.117:0/2029144191 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:10:43.996 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:43.035+0000 7fd00fe00700 1 -- 172.21.15.117:0/2029144191 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd01007c9c0 con 0x7fd01007de00 2024-06-03T03:10:43.996 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:43.036+0000 7fd00fe00700 1 --2- 172.21.15.117:0/2029144191 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd01007de00 0x7fd01007c300 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd0000098a0 tx=0x7fd000009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ecd457251e2089f server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:10:43.997 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:43.036+0000 7fd00ea00700 1 -- 172.21.15.117:0/2029144191 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd000004160 con 0x7fd01007de00 2024-06-03T03:10:43.997 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:43.036+0000 7fd00ea00700 1 -- 172.21.15.117:0/2029144191 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd000004300 con 0x7fd01007de00 2024-06-03T03:10:43.997 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:43.036+0000 7fd00ea00700 1 -- 172.21.15.117:0/2029144191 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd000011b50 con 0x7fd01007de00 2024-06-03T03:10:43.997 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:43.037+0000 7fd016e00700 1 -- 172.21.15.117:0/2029144191 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd01007de00 msgr2=0x7fd01007c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:10:43.997 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:43.037+0000 7fd016e00700 1 --2- 172.21.15.117:0/2029144191 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd01007de00 0x7fd01007c300 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd0000098a0 tx=0x7fd000009bb0 comp rx=0 tx=0).stop 2024-06-03T03:10:43.997 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:43.037+0000 7fd016e00700 1 -- 172.21.15.117:0/2029144191 shutdown_connections 2024-06-03T03:10:43.997 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:43.037+0000 7fd016e00700 1 --2- 172.21.15.117:0/2029144191 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd01007de00 0x7fd01007c300 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:10:43.997 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:43.037+0000 7fd016e00700 1 -- 172.21.15.117:0/2029144191 >> 172.21.15.117:0/2029144191 conn(0x7fd0101035b0 msgr2=0x7fd0101059d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:10:43.997 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:43.037+0000 7fd016e00700 1 -- 172.21.15.117:0/2029144191 shutdown_connections 2024-06-03T03:10:43.997 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:43.037+0000 7fd016e00700 1 -- 172.21.15.117:0/2029144191 wait complete. 2024-06-03T03:10:43.997 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:43.038+0000 7fd016e00700 1 Processor -- start 2024-06-03T03:10:43.997 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:43.039+0000 7fd016e00700 1 -- start start 2024-06-03T03:10:43.997 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:43.039+0000 7fd016e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd01007de00 0x7fd01019e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:10:43.998 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:43.039+0000 7fd016e00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd01010e5a0 con 0x7fd01007de00 2024-06-03T03:10:43.998 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:43.039+0000 7fd00fe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd01007de00 0x7fd01019e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:10:43.998 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:43.039+0000 7fd00fe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd01007de00 0x7fd01019e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56948/0 (socket says 172.21.15.117:56948) 2024-06-03T03:10:43.998 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:43.039+0000 7fd00fe00700 1 -- 172.21.15.117:0/1475542701 learned_addr learned my addr 172.21.15.117:0/1475542701 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:10:43.998 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:43.040+0000 7fd00fe00700 1 -- 172.21.15.117:0/1475542701 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd01019e610 con 0x7fd01007de00 2024-06-03T03:10:43.998 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:43.040+0000 7fd00fe00700 1 --2- 172.21.15.117:0/1475542701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd01007de00 0x7fd01019e0d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd000011b60 tx=0x7fd000011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:10:43.998 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:43.040+0000 7fd00cc00700 1 -- 172.21.15.117:0/1475542701 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd000003b20 con 0x7fd01007de00 2024-06-03T03:10:43.998 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:43.040+0000 7fd00cc00700 1 -- 172.21.15.117:0/1475542701 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd000003cc0 con 0x7fd01007de00 2024-06-03T03:10:43.998 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:43.040+0000 7fd016e00700 1 -- 172.21.15.117:0/1475542701 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd01019e8a0 con 0x7fd01007de00 2024-06-03T03:10:43.998 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:43.040+0000 7fd00cc00700 1 -- 172.21.15.117:0/1475542701 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd000024840 con 0x7fd01007de00 2024-06-03T03:10:43.998 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:43.041+0000 7fd016e00700 1 -- 172.21.15.117:0/1475542701 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0101a1590 con 0x7fd01007de00 2024-06-03T03:10:43.998 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:43.042+0000 7fd00cc00700 1 -- 172.21.15.117:0/1475542701 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7fd000022020 con 0x7fd01007de00 2024-06-03T03:10:43.998 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:43.042+0000 7fd016e00700 1 -- 172.21.15.117:0/1475542701 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd010063af0 con 0x7fd01007de00 2024-06-03T03:10:43.999 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:43.042+0000 7fd00cc00700 1 --2- 172.21.15.117:0/1475542701 >> [v2:172.21.15.117:6800/2424554985,v1:172.21.15.117:6801/2424554985] conn(0x7fcff40390a0 0x7fcff403b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:10:43.999 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:43.042+0000 7fd00cc00700 1 -- 172.21.15.117:0/1475542701 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd000011e00 con 0x7fd01007de00 2024-06-03T03:10:43.999 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:43.042+0000 7fd00f400700 1 -- 172.21.15.117:0/1475542701 >> [v2:172.21.15.117:6800/2424554985,v1:172.21.15.117:6801/2424554985] conn(0x7fcff40390a0 msgr2=0x7fcff403b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.117:6800/2424554985 2024-06-03T03:10:43.999 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:43.042+0000 7fd00f400700 1 --2- 172.21.15.117:0/1475542701 >> [v2:172.21.15.117:6800/2424554985,v1:172.21.15.117:6801/2424554985] conn(0x7fcff40390a0 0x7fcff403b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-03T03:10:43.999 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:43.048+0000 7fd00cc00700 1 -- 172.21.15.117:0/1475542701 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd00000ba30 con 0x7fd01007de00 2024-06-03T03:10:43.999 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:43.243+0000 7fd00f400700 1 -- 172.21.15.117:0/1475542701 >> [v2:172.21.15.117:6800/2424554985,v1:172.21.15.117:6801/2424554985] conn(0x7fcff40390a0 msgr2=0x7fcff403b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.117:6800/2424554985 2024-06-03T03:10:43.999 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:43.243+0000 7fd00f400700 1 --2- 172.21.15.117:0/1475542701 >> [v2:172.21.15.117:6800/2424554985,v1:172.21.15.117:6801/2424554985] conn(0x7fcff40390a0 0x7fcff403b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-03T03:10:43.999 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:43.320+0000 7fd016e00700 1 -- 172.21.15.117:0/1475542701 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fd0101a1760 con 0x7fd01007de00 2024-06-03T03:10:43.999 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:43.321+0000 7fd00cc00700 1 -- 172.21.15.117:0/1475542701 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7fd00000fdd0 con 0x7fd01007de00 2024-06-03T03:10:43.999 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:43.323+0000 7fd016e00700 1 -- 172.21.15.117:0/1475542701 >> [v2:172.21.15.117:6800/2424554985,v1:172.21.15.117:6801/2424554985] conn(0x7fcff40390a0 msgr2=0x7fcff403b560 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-03T03:10:43.999 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:43.323+0000 7fd016e00700 1 --2- 172.21.15.117:0/1475542701 >> [v2:172.21.15.117:6800/2424554985,v1:172.21.15.117:6801/2424554985] conn(0x7fcff40390a0 0x7fcff403b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:10:43.999 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:43.323+0000 7fd016e00700 1 -- 172.21.15.117:0/1475542701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd01007de00 msgr2=0x7fd01019e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:10:43.999 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:43.323+0000 7fd016e00700 1 --2- 172.21.15.117:0/1475542701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd01007de00 0x7fd01019e0d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd000011b60 tx=0x7fd000011c40 comp rx=0 tx=0).stop 2024-06-03T03:10:43.999 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:43.323+0000 7fd016e00700 1 -- 172.21.15.117:0/1475542701 shutdown_connections 2024-06-03T03:10:44.000 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:43.323+0000 7fd016e00700 1 --2- 172.21.15.117:0/1475542701 >> [v2:172.21.15.117:6800/2424554985,v1:172.21.15.117:6801/2424554985] conn(0x7fcff40390a0 0x7fcff403b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:10:44.000 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:43.323+0000 7fd016e00700 1 --2- 172.21.15.117:0/1475542701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd01007de00 0x7fd01019e0d0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:10:44.000 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:43.323+0000 7fd016e00700 1 -- 172.21.15.117:0/1475542701 >> 172.21.15.117:0/1475542701 conn(0x7fd0101035b0 msgr2=0x7fd0101040f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:10:44.000 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:43.323+0000 7fd016e00700 1 -- 172.21.15.117:0/1475542701 shutdown_connections 2024-06-03T03:10:44.000 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:43.323+0000 7fd016e00700 1 -- 172.21.15.117:0/1475542701 wait complete. 2024-06-03T03:10:44.000 INFO:teuthology.orchestra.run.smithi117.stdout:Waiting for the mgr to restart... 2024-06-03T03:10:44.000 INFO:teuthology.orchestra.run.smithi117.stdout:Waiting for mgr epoch 5... 2024-06-03T03:10:44.544 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:10:44 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:10:44.417+0000 7f64cc58a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-03T03:10:44.872 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:10:44 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:10:44.649+0000 7f64cc58a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-03T03:10:44.872 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:10:44 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:10:44.871+0000 7f64cc58a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-03T03:10:45.446 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:10:45 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:10:45.251+0000 7f64cc58a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-03T03:10:45.447 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:10:45 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:10:45.373+0000 7f64cc58a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-03T03:10:45.879 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:10:45 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:10:45.623+0000 7f64cc58a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-03T03:10:45.880 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:10:45 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:10:45.878+0000 7f64cc58a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-03T03:10:46.946 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:10:46 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:10:46.655+0000 7f64cc58a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-03T03:10:47.577 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:10:47 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:10:47.201+0000 7f64cc58a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-03T03:10:47.577 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:10:47 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:10:47.331+0000 7f64cc58a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-03T03:10:47.946 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:10:47 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:10:47.598+0000 7f64cc58a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-03T03:10:48.690 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:10:48 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:10:48.241+0000 7f64cc58a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-03T03:10:48.690 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:10:48 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:10:48.366+0000 7f64cc58a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-03T03:10:48.690 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:10:48 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:10:48.688+0000 7f64cc58a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-03T03:10:48.946 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:10:48 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:10:48.867+0000 7f64cc58a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-03T03:10:49.450 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:10:48 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:10:48.992+0000 7f64cc58a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-03T03:10:49.450 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:10:49 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:10:49.147+0000 7f64cc58a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-03T03:10:49.946 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:10:49 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:10:49.514+0000 7f64cc58a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-03T03:10:50.551 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:10:50 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:10:50.262+0000 7f64cc58a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-03T03:10:50.946 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:10:50 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:10:50.549+0000 7f64cc58a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-03T03:10:50.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:50 smithi117 ceph-mon[31649]: Active manager daemon x restarted 2024-06-03T03:10:50.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:50 smithi117 ceph-mon[31649]: Activating manager daemon x 2024-06-03T03:10:50.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:50 smithi117 ceph-mon[31649]: osdmap e2: 0 total, 0 up, 0 in 2024-06-03T03:10:50.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:50 smithi117 ceph-mon[31649]: mgrmap e6: x(active, starting, since 0.00706654s) 2024-06-03T03:10:50.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:50 smithi117 ceph-mon[31649]: from='mgr.14120 172.21.15.117:0/175906377' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-03T03:10:50.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:50 smithi117 ceph-mon[31649]: from='mgr.14120 172.21.15.117:0/175906377' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-03T03:10:50.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:50 smithi117 ceph-mon[31649]: from='mgr.14120 172.21.15.117:0/175906377' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-03T03:10:50.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:50 smithi117 ceph-mon[31649]: from='mgr.14120 172.21.15.117:0/175906377' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-03T03:10:50.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:50 smithi117 ceph-mon[31649]: from='mgr.14120 172.21.15.117:0/175906377' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-03T03:10:50.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:50 smithi117 ceph-mon[31649]: Manager daemon x is now available 2024-06-03T03:10:50.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:50 smithi117 ceph-mon[31649]: from='mgr.14120 172.21.15.117:0/175906377' entity='mgr.x' 2024-06-03T03:10:51.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:51 smithi117 ceph-mon[31649]: Found migration_current of "None". Setting to last migration. 2024-06-03T03:10:51.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:51 smithi117 ceph-mon[31649]: from='mgr.14120 172.21.15.117:0/175906377' entity='mgr.x' 2024-06-03T03:10:51.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:51 smithi117 ceph-mon[31649]: from='mgr.14120 172.21.15.117:0/175906377' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:10:51.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:51 smithi117 ceph-mon[31649]: from='mgr.14120 172.21.15.117:0/175906377' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:10:51.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:51 smithi117 ceph-mon[31649]: from='mgr.14120 172.21.15.117:0/175906377' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-03T03:10:51.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:51 smithi117 ceph-mon[31649]: from='mgr.14120 172.21.15.117:0/175906377' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-03T03:10:51.948 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:51 smithi117 ceph-mon[31649]: from='mgr.14120 172.21.15.117:0/175906377' entity='mgr.x' 2024-06-03T03:10:51.948 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:51 smithi117 ceph-mon[31649]: from='mgr.14120 172.21.15.117:0/175906377' entity='mgr.x' 2024-06-03T03:10:51.948 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:51 smithi117 ceph-mon[31649]: mgrmap e7: x(active, since 1.0112s) 2024-06-03T03:10:52.286 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout { 2024-06-03T03:10:52.286 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-06-03T03:10:52.286 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "initialized": true 2024-06-03T03:10:52.286 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout } 2024-06-03T03:10:52.287 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:44.562+0000 7f454a600700 1 Processor -- start 2024-06-03T03:10:52.287 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:44.563+0000 7f454a600700 1 -- start start 2024-06-03T03:10:52.287 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:44.563+0000 7f454a600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4544106880 0x7f4544106ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:10:52.287 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:44.563+0000 7f454a600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f454407be30 con 0x7f4544106880 2024-06-03T03:10:52.287 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:44.563+0000 7f4543400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4544106880 0x7f4544106ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:10:52.287 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:44.563+0000 7f4543400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4544106880 0x7f4544106ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56958/0 (socket says 172.21.15.117:56958) 2024-06-03T03:10:52.294 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:44.563+0000 7f4543400700 1 -- 172.21.15.117:0/3275519030 learned_addr learned my addr 172.21.15.117:0/3275519030 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:10:52.294 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:44.564+0000 7f4543400700 1 -- 172.21.15.117:0/3275519030 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45441071e0 con 0x7f4544106880 2024-06-03T03:10:52.294 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:44.564+0000 7f4543400700 1 --2- 172.21.15.117:0/3275519030 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4544106880 0x7f4544106ca0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f45340098a0 tx=0x7f4534009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f07b805f36678145 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:10:52.294 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:44.564+0000 7f4542000700 1 -- 172.21.15.117:0/3275519030 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4534004160 con 0x7f4544106880 2024-06-03T03:10:52.294 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:44.565+0000 7f4542000700 1 -- 172.21.15.117:0/3275519030 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4534004300 con 0x7f4544106880 2024-06-03T03:10:52.294 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:44.565+0000 7f4542000700 1 -- 172.21.15.117:0/3275519030 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4534011b50 con 0x7f4544106880 2024-06-03T03:10:52.294 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:44.565+0000 7f454a600700 1 -- 172.21.15.117:0/3275519030 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4544106880 msgr2=0x7f4544106ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:10:52.294 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:44.565+0000 7f454a600700 1 --2- 172.21.15.117:0/3275519030 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4544106880 0x7f4544106ca0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f45340098a0 tx=0x7f4534009bb0 comp rx=0 tx=0).stop 2024-06-03T03:10:52.294 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:44.565+0000 7f454a600700 1 -- 172.21.15.117:0/3275519030 shutdown_connections 2024-06-03T03:10:52.294 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:44.565+0000 7f454a600700 1 --2- 172.21.15.117:0/3275519030 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4544106880 0x7f4544106ca0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:10:52.294 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:44.565+0000 7f454a600700 1 -- 172.21.15.117:0/3275519030 >> 172.21.15.117:0/3275519030 conn(0x7f45441024e0 msgr2=0x7f4544104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:10:52.295 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:44.566+0000 7f454a600700 1 -- 172.21.15.117:0/3275519030 shutdown_connections 2024-06-03T03:10:52.295 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:44.566+0000 7f454a600700 1 -- 172.21.15.117:0/3275519030 wait complete. 2024-06-03T03:10:52.295 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:44.567+0000 7f454a600700 1 Processor -- start 2024-06-03T03:10:52.295 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:44.567+0000 7f454a600700 1 -- start start 2024-06-03T03:10:52.295 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:44.567+0000 7f454a600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4544106880 0x7f454419e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:10:52.295 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:44.567+0000 7f454a600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4544108200 con 0x7f4544106880 2024-06-03T03:10:52.295 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:44.568+0000 7f4543400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4544106880 0x7f454419e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:10:52.295 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:44.568+0000 7f4543400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4544106880 0x7f454419e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56970/0 (socket says 172.21.15.117:56970) 2024-06-03T03:10:52.295 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:44.568+0000 7f4543400700 1 -- 172.21.15.117:0/1567395612 learned_addr learned my addr 172.21.15.117:0/1567395612 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:10:52.295 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:44.568+0000 7f4543400700 1 -- 172.21.15.117:0/1567395612 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f454419e660 con 0x7f4544106880 2024-06-03T03:10:52.295 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:44.568+0000 7f4543400700 1 --2- 172.21.15.117:0/1567395612 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4544106880 0x7f454419e120 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4534011b60 tx=0x7f4534011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:10:52.295 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:44.569+0000 7f453be00700 1 -- 172.21.15.117:0/1567395612 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4534003b20 con 0x7f4544106880 2024-06-03T03:10:52.295 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:44.569+0000 7f453be00700 1 -- 172.21.15.117:0/1567395612 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4534003cc0 con 0x7f4544106880 2024-06-03T03:10:52.295 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:44.569+0000 7f453be00700 1 -- 172.21.15.117:0/1567395612 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4534024840 con 0x7f4544106880 2024-06-03T03:10:52.295 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:44.569+0000 7f454a600700 1 -- 172.21.15.117:0/1567395612 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f454419e8f0 con 0x7f4544106880 2024-06-03T03:10:52.296 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:44.569+0000 7f454a600700 1 -- 172.21.15.117:0/1567395612 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45441a15e0 con 0x7f4544106880 2024-06-03T03:10:52.296 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:44.570+0000 7f453be00700 1 -- 172.21.15.117:0/1567395612 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f4534022020 con 0x7f4544106880 2024-06-03T03:10:52.296 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:44.570+0000 7f453be00700 1 --2- 172.21.15.117:0/1567395612 >> [v2:172.21.15.117:6800/2424554985,v1:172.21.15.117:6801/2424554985] conn(0x7f4528039050 0x7f452803b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:10:52.296 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:44.570+0000 7f453be00700 1 -- 172.21.15.117:0/1567395612 --> [v2:172.21.15.117:6800/2424554985,v1:172.21.15.117:6801/2424554985] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f452803bc20 con 0x7f4528039050 2024-06-03T03:10:52.296 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:44.570+0000 7f453be00700 1 -- 172.21.15.117:0/1567395612 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4534011e00 con 0x7f4544106880 2024-06-03T03:10:52.296 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:44.571+0000 7f4542a00700 1 -- 172.21.15.117:0/1567395612 >> [v2:172.21.15.117:6800/2424554985,v1:172.21.15.117:6801/2424554985] conn(0x7f4528039050 msgr2=0x7f452803b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.117:6800/2424554985 2024-06-03T03:10:52.296 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:44.571+0000 7f4542a00700 1 --2- 172.21.15.117:0/1567395612 >> [v2:172.21.15.117:6800/2424554985,v1:172.21.15.117:6801/2424554985] conn(0x7f4528039050 0x7f452803b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-03T03:10:52.296 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:44.772+0000 7f4542a00700 1 -- 172.21.15.117:0/1567395612 >> [v2:172.21.15.117:6800/2424554985,v1:172.21.15.117:6801/2424554985] conn(0x7f4528039050 msgr2=0x7f452803b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.117:6800/2424554985 2024-06-03T03:10:52.296 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:44.772+0000 7f4542a00700 1 --2- 172.21.15.117:0/1567395612 >> [v2:172.21.15.117:6800/2424554985,v1:172.21.15.117:6801/2424554985] conn(0x7f4528039050 0x7f452803b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-03T03:10:52.296 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:45.173+0000 7f4542a00700 1 -- 172.21.15.117:0/1567395612 >> [v2:172.21.15.117:6800/2424554985,v1:172.21.15.117:6801/2424554985] conn(0x7f4528039050 msgr2=0x7f452803b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.117:6800/2424554985 2024-06-03T03:10:52.296 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:45.173+0000 7f4542a00700 1 --2- 172.21.15.117:0/1567395612 >> [v2:172.21.15.117:6800/2424554985,v1:172.21.15.117:6801/2424554985] conn(0x7f4528039050 0x7f452803b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-06-03T03:10:52.296 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:45.974+0000 7f4542a00700 1 -- 172.21.15.117:0/1567395612 >> [v2:172.21.15.117:6800/2424554985,v1:172.21.15.117:6801/2424554985] conn(0x7f4528039050 msgr2=0x7f452803b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.117:6800/2424554985 2024-06-03T03:10:52.296 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:45.974+0000 7f4542a00700 1 --2- 172.21.15.117:0/1567395612 >> [v2:172.21.15.117:6800/2424554985,v1:172.21.15.117:6801/2424554985] conn(0x7f4528039050 0x7f452803b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-06-03T03:10:52.296 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:47.576+0000 7f4542a00700 1 -- 172.21.15.117:0/1567395612 >> [v2:172.21.15.117:6800/2424554985,v1:172.21.15.117:6801/2424554985] conn(0x7f4528039050 msgr2=0x7f452803b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.117:6800/2424554985 2024-06-03T03:10:52.296 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:47.576+0000 7f4542a00700 1 --2- 172.21.15.117:0/1567395612 >> [v2:172.21.15.117:6800/2424554985,v1:172.21.15.117:6801/2424554985] conn(0x7f4528039050 0x7f452803b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-06-03T03:10:52.297 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:50.558+0000 7f453be00700 1 -- 172.21.15.117:0/1567395612 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mgrmap(e 6) ==== 46054+0+0 (secure 0 0 0) 0x7f453401a430 con 0x7f4544106880 2024-06-03T03:10:52.297 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:50.559+0000 7f453be00700 1 -- 172.21.15.117:0/1567395612 >> [v2:172.21.15.117:6800/2424554985,v1:172.21.15.117:6801/2424554985] conn(0x7f4528039050 msgr2=0x7f452803b510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-03T03:10:52.297 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:50.559+0000 7f453be00700 1 --2- 172.21.15.117:0/1567395612 >> [v2:172.21.15.117:6800/2424554985,v1:172.21.15.117:6801/2424554985] conn(0x7f4528039050 0x7f452803b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:10:52.297 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:51.561+0000 7f453be00700 1 -- 172.21.15.117:0/1567395612 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7f453404e780 con 0x7f4544106880 2024-06-03T03:10:52.297 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:51.561+0000 7f453be00700 1 --2- 172.21.15.117:0/1567395612 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7f452803c980 0x7f452803ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:10:52.297 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:51.561+0000 7f453be00700 1 -- 172.21.15.117:0/1567395612 --> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f453400af30 con 0x7f452803c980 2024-06-03T03:10:52.297 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:51.562+0000 7f4542a00700 1 --2- 172.21.15.117:0/1567395612 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7f452803c980 0x7f452803ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:10:52.297 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:51.564+0000 7f4542a00700 1 --2- 172.21.15.117:0/1567395612 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7f452803c980 0x7f452803ed70 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4530003a10 tx=0x7f4530009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:10:52.297 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:51.565+0000 7f453be00700 1 -- 172.21.15.117:0/1567395612 <== mgr.14120 v2:172.21.15.117:6800/2142313590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f453400af30 con 0x7f452803c980 2024-06-03T03:10:52.297 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:51.573+0000 7f454a600700 1 -- 172.21.15.117:0/1567395612 --> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f45440695b0 con 0x7f452803c980 2024-06-03T03:10:52.297 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:51.573+0000 7f453be00700 1 -- 172.21.15.117:0/1567395612 <== mgr.14120 v2:172.21.15.117:6800/2142313590 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f45440695b0 con 0x7f452803c980 2024-06-03T03:10:52.297 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:51.574+0000 7f454a600700 1 -- 172.21.15.117:0/1567395612 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7f452803c980 msgr2=0x7f452803ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:10:52.297 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:51.574+0000 7f454a600700 1 --2- 172.21.15.117:0/1567395612 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7f452803c980 0x7f452803ed70 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4530003a10 tx=0x7f4530009290 comp rx=0 tx=0).stop 2024-06-03T03:10:52.297 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:51.574+0000 7f454a600700 1 -- 172.21.15.117:0/1567395612 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4544106880 msgr2=0x7f454419e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:10:52.297 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:51.574+0000 7f454a600700 1 --2- 172.21.15.117:0/1567395612 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4544106880 0x7f454419e120 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4534011b60 tx=0x7f4534011c40 comp rx=0 tx=0).stop 2024-06-03T03:10:52.298 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:51.574+0000 7f454a600700 1 -- 172.21.15.117:0/1567395612 shutdown_connections 2024-06-03T03:10:52.298 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:51.574+0000 7f454a600700 1 --2- 172.21.15.117:0/1567395612 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7f452803c980 0x7f452803ed70 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:10:52.298 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:51.574+0000 7f454a600700 1 --2- 172.21.15.117:0/1567395612 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4544106880 0x7f454419e120 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:10:52.298 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:51.574+0000 7f454a600700 1 --2- 172.21.15.117:0/1567395612 >> [v2:172.21.15.117:6800/2424554985,v1:172.21.15.117:6801/2424554985] conn(0x7f4528039050 0x7f452803b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:10:52.298 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:51.574+0000 7f454a600700 1 -- 172.21.15.117:0/1567395612 >> 172.21.15.117:0/1567395612 conn(0x7f45441024e0 msgr2=0x7f4544194de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:10:52.298 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:51.574+0000 7f454a600700 1 -- 172.21.15.117:0/1567395612 shutdown_connections 2024-06-03T03:10:52.298 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:51.574+0000 7f454a600700 1 -- 172.21.15.117:0/1567395612 wait complete. 2024-06-03T03:10:52.298 INFO:teuthology.orchestra.run.smithi117.stdout:mgr epoch 5 is available 2024-06-03T03:10:52.299 INFO:teuthology.orchestra.run.smithi117.stdout:Setting orchestrator backend to cephadm... 2024-06-03T03:10:52.900 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:52 smithi117 ceph-mon[31649]: [03/Jun/2024:03:10:51] ENGINE Bus STARTING 2024-06-03T03:10:52.900 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:52 smithi117 ceph-mon[31649]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-06-03T03:10:52.901 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:52 smithi117 ceph-mon[31649]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-06-03T03:10:52.901 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:52 smithi117 ceph-mon[31649]: [03/Jun/2024:03:10:51] ENGINE Serving on http://172.21.15.117:8765 2024-06-03T03:10:52.901 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:52 smithi117 ceph-mon[31649]: from='mgr.14120 172.21.15.117:0/175906377' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:10:53.950 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:53 smithi117 ceph-mon[31649]: [03/Jun/2024:03:10:51] ENGINE Serving on https://172.21.15.117:7150 2024-06-03T03:10:53.950 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:53 smithi117 ceph-mon[31649]: [03/Jun/2024:03:10:51] ENGINE Bus STARTED 2024-06-03T03:10:53.950 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:53 smithi117 ceph-mon[31649]: mgrmap e8: x(active, since 2s) 2024-06-03T03:10:53.950 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:53 smithi117 ceph-mon[31649]: from='mgr.14120 172.21.15.117:0/175906377' entity='mgr.x' 2024-06-03T03:10:53.950 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:53 smithi117 ceph-mon[31649]: from='mgr.14120 172.21.15.117:0/175906377' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:10:54.112 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:53.065+0000 7fc795800700 1 Processor -- start 2024-06-03T03:10:54.112 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:53.065+0000 7fc795800700 1 -- start start 2024-06-03T03:10:54.112 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:53.065+0000 7fc795800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc79007ed30 0x7fc79007f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:10:54.112 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:53.065+0000 7fc795800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc79007f720 con 0x7fc79007ed30 2024-06-03T03:10:54.112 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:53.066+0000 7fc78fe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc79007ed30 0x7fc79007f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:10:54.113 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:53.066+0000 7fc78fe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc79007ed30 0x7fc79007f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:58774/0 (socket says 172.21.15.117:58774) 2024-06-03T03:10:54.113 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:53.066+0000 7fc78fe00700 1 -- 172.21.15.117:0/2026139178 learned_addr learned my addr 172.21.15.117:0/2026139178 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:10:54.113 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:53.066+0000 7fc78fe00700 1 -- 172.21.15.117:0/2026139178 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc79007ff80 con 0x7fc79007ed30 2024-06-03T03:10:54.113 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:53.066+0000 7fc78fe00700 1 --2- 172.21.15.117:0/2026139178 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc79007ed30 0x7fc79007f150 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc7800098a0 tx=0x7fc780009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41b497d267e5a2b2 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:10:54.113 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:53.067+0000 7fc78ea00700 1 -- 172.21.15.117:0/2026139178 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc780004160 con 0x7fc79007ed30 2024-06-03T03:10:54.113 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:53.067+0000 7fc78ea00700 1 -- 172.21.15.117:0/2026139178 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc780004300 con 0x7fc79007ed30 2024-06-03T03:10:54.113 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:53.067+0000 7fc78ea00700 1 -- 172.21.15.117:0/2026139178 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc780011b50 con 0x7fc79007ed30 2024-06-03T03:10:54.114 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:53.067+0000 7fc795800700 1 -- 172.21.15.117:0/2026139178 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc79007ed30 msgr2=0x7fc79007f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:10:54.114 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:53.067+0000 7fc795800700 1 --2- 172.21.15.117:0/2026139178 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc79007ed30 0x7fc79007f150 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc7800098a0 tx=0x7fc780009bb0 comp rx=0 tx=0).stop 2024-06-03T03:10:54.114 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:53.067+0000 7fc795800700 1 -- 172.21.15.117:0/2026139178 shutdown_connections 2024-06-03T03:10:54.114 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:53.068+0000 7fc795800700 1 --2- 172.21.15.117:0/2026139178 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc79007ed30 0x7fc79007f150 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:10:54.114 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:53.068+0000 7fc795800700 1 -- 172.21.15.117:0/2026139178 >> 172.21.15.117:0/2026139178 conn(0x7fc79007d360 msgr2=0x7fc79007d770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:10:54.114 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:53.068+0000 7fc795800700 1 -- 172.21.15.117:0/2026139178 shutdown_connections 2024-06-03T03:10:54.115 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:53.068+0000 7fc795800700 1 -- 172.21.15.117:0/2026139178 wait complete. 2024-06-03T03:10:54.115 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:53.069+0000 7fc795800700 1 Processor -- start 2024-06-03T03:10:54.115 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:53.069+0000 7fc795800700 1 -- start start 2024-06-03T03:10:54.115 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:53.069+0000 7fc795800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc79007ed30 0x7fc7901a2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:10:54.115 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:53.069+0000 7fc795800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc790081340 con 0x7fc79007ed30 2024-06-03T03:10:54.115 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:53.069+0000 7fc78fe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc79007ed30 0x7fc7901a2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:10:54.115 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:53.069+0000 7fc78fe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc79007ed30 0x7fc7901a2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:58778/0 (socket says 172.21.15.117:58778) 2024-06-03T03:10:54.115 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:53.069+0000 7fc78fe00700 1 -- 172.21.15.117:0/4066172539 learned_addr learned my addr 172.21.15.117:0/4066172539 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:10:54.116 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:53.070+0000 7fc78fe00700 1 -- 172.21.15.117:0/4066172539 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7901a28c0 con 0x7fc79007ed30 2024-06-03T03:10:54.116 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:53.070+0000 7fc78fe00700 1 --2- 172.21.15.117:0/4066172539 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc79007ed30 0x7fc7901a2380 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc780011b60 tx=0x7fc780011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:10:54.116 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:53.070+0000 7fc78cc00700 1 -- 172.21.15.117:0/4066172539 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc780003970 con 0x7fc79007ed30 2024-06-03T03:10:54.116 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:53.070+0000 7fc78cc00700 1 -- 172.21.15.117:0/4066172539 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc780003b10 con 0x7fc79007ed30 2024-06-03T03:10:54.116 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:53.070+0000 7fc78cc00700 1 -- 172.21.15.117:0/4066172539 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc780024610 con 0x7fc79007ed30 2024-06-03T03:10:54.116 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:53.070+0000 7fc795800700 1 -- 172.21.15.117:0/4066172539 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7901a2b50 con 0x7fc79007ed30 2024-06-03T03:10:54.116 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:53.070+0000 7fc795800700 1 -- 172.21.15.117:0/4066172539 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7901a5840 con 0x7fc79007ed30 2024-06-03T03:10:54.116 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:53.071+0000 7fc78cc00700 1 -- 172.21.15.117:0/4066172539 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fc78001a9f0 con 0x7fc79007ed30 2024-06-03T03:10:54.116 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:53.071+0000 7fc795800700 1 -- 172.21.15.117:0/4066172539 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7900695b0 con 0x7fc79007ed30 2024-06-03T03:10:54.116 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:53.072+0000 7fc78cc00700 1 --2- 172.21.15.117:0/4066172539 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7fc7740390a0 0x7fc77403b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:10:54.116 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:53.072+0000 7fc78cc00700 1 -- 172.21.15.117:0/4066172539 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc780011e00 con 0x7fc79007ed30 2024-06-03T03:10:54.116 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:53.072+0000 7fc78f400700 1 --2- 172.21.15.117:0/4066172539 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7fc7740390a0 0x7fc77403b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:10:54.116 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:53.073+0000 7fc78f400700 1 --2- 172.21.15.117:0/4066172539 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7fc7740390a0 0x7fc77403b560 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc77c006fd0 tx=0x7fc77c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:10:54.117 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:53.078+0000 7fc78cc00700 1 -- 172.21.15.117:0/4066172539 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc78004cc10 con 0x7fc79007ed30 2024-06-03T03:10:54.117 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:53.321+0000 7fc795800700 1 -- 172.21.15.117:0/4066172539 --> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7fc79002df60 con 0x7fc7740390a0 2024-06-03T03:10:54.117 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:53.334+0000 7fc78cc00700 1 -- 172.21.15.117:0/4066172539 <== mgr.14120 v2:172.21.15.117:6800/2142313590 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc79002df60 con 0x7fc7740390a0 2024-06-03T03:10:54.117 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:53.337+0000 7fc795800700 1 -- 172.21.15.117:0/4066172539 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7fc7740390a0 msgr2=0x7fc77403b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:10:54.117 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:53.337+0000 7fc795800700 1 --2- 172.21.15.117:0/4066172539 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7fc7740390a0 0x7fc77403b560 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc77c006fd0 tx=0x7fc77c006e40 comp rx=0 tx=0).stop 2024-06-03T03:10:54.117 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:53.337+0000 7fc795800700 1 -- 172.21.15.117:0/4066172539 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc79007ed30 msgr2=0x7fc7901a2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:10:54.117 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:53.337+0000 7fc795800700 1 --2- 172.21.15.117:0/4066172539 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc79007ed30 0x7fc7901a2380 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc780011b60 tx=0x7fc780011c40 comp rx=0 tx=0).stop 2024-06-03T03:10:54.117 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:53.338+0000 7fc795800700 1 -- 172.21.15.117:0/4066172539 shutdown_connections 2024-06-03T03:10:54.117 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:53.338+0000 7fc795800700 1 --2- 172.21.15.117:0/4066172539 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7fc7740390a0 0x7fc77403b560 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:10:54.117 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:53.338+0000 7fc795800700 1 --2- 172.21.15.117:0/4066172539 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc79007ed30 0x7fc7901a2380 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:10:54.117 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:53.338+0000 7fc795800700 1 -- 172.21.15.117:0/4066172539 >> 172.21.15.117:0/4066172539 conn(0x7fc79007d360 msgr2=0x7fc790108a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:10:54.117 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:53.338+0000 7fc795800700 1 -- 172.21.15.117:0/4066172539 shutdown_connections 2024-06-03T03:10:54.117 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:53.338+0000 7fc795800700 1 -- 172.21.15.117:0/4066172539 wait complete. 2024-06-03T03:10:54.680 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:54 smithi117 ceph-mon[31649]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:10:55.717 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout value unchanged 2024-06-03T03:10:55.717 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:54.704+0000 7f23d4a00700 1 Processor -- start 2024-06-03T03:10:55.717 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:54.704+0000 7f23d4a00700 1 -- start start 2024-06-03T03:10:55.717 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:54.704+0000 7f23d4a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23d010a010 0x7f23d010a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:10:55.717 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:54.704+0000 7f23d4a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f23d010aa00 con 0x7f23d010a010 2024-06-03T03:10:55.717 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:54.705+0000 7f23ce800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23d010a010 0x7f23d010a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:10:55.717 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:54.705+0000 7f23ce800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23d010a010 0x7f23d010a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:58786/0 (socket says 172.21.15.117:58786) 2024-06-03T03:10:55.718 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:54.705+0000 7f23ce800700 1 -- 172.21.15.117:0/3892250935 learned_addr learned my addr 172.21.15.117:0/3892250935 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:10:55.718 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:54.705+0000 7f23ce800700 1 -- 172.21.15.117:0/3892250935 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23d010b1c0 con 0x7f23d010a010 2024-06-03T03:10:55.718 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:54.705+0000 7f23ce800700 1 --2- 172.21.15.117:0/3892250935 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23d010a010 0x7f23d010a430 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f23c00098a0 tx=0x7f23c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10bcfa650ac26304 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:10:55.718 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:54.706+0000 7f23cd400700 1 -- 172.21.15.117:0/3892250935 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f23c0004160 con 0x7f23d010a010 2024-06-03T03:10:55.718 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:54.706+0000 7f23cd400700 1 -- 172.21.15.117:0/3892250935 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f23c0004300 con 0x7f23d010a010 2024-06-03T03:10:55.718 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:54.706+0000 7f23cd400700 1 -- 172.21.15.117:0/3892250935 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f23c0011b50 con 0x7f23d010a010 2024-06-03T03:10:55.718 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:54.706+0000 7f23d4a00700 1 -- 172.21.15.117:0/3892250935 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23d010a010 msgr2=0x7f23d010a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:10:55.718 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:54.706+0000 7f23d4a00700 1 --2- 172.21.15.117:0/3892250935 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23d010a010 0x7f23d010a430 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f23c00098a0 tx=0x7f23c0009bb0 comp rx=0 tx=0).stop 2024-06-03T03:10:55.718 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:54.707+0000 7f23d4a00700 1 -- 172.21.15.117:0/3892250935 shutdown_connections 2024-06-03T03:10:55.718 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:54.707+0000 7f23d4a00700 1 --2- 172.21.15.117:0/3892250935 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23d010a010 0x7f23d010a430 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:10:55.718 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:54.707+0000 7f23d4a00700 1 -- 172.21.15.117:0/3892250935 >> 172.21.15.117:0/3892250935 conn(0x7f23d007d3d0 msgr2=0x7f23d007d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:10:55.718 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:54.707+0000 7f23d4a00700 1 -- 172.21.15.117:0/3892250935 shutdown_connections 2024-06-03T03:10:55.719 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:54.707+0000 7f23d4a00700 1 -- 172.21.15.117:0/3892250935 wait complete. 2024-06-03T03:10:55.719 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:54.708+0000 7f23d4a00700 1 Processor -- start 2024-06-03T03:10:55.719 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:54.708+0000 7f23d4a00700 1 -- start start 2024-06-03T03:10:55.719 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:54.708+0000 7f23d4a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23d010a010 0x7f23d019dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:10:55.719 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:54.708+0000 7f23d4a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f23d010e570 con 0x7f23d010a010 2024-06-03T03:10:55.719 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:54.708+0000 7f23ce800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23d010a010 0x7f23d019dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:10:55.719 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:54.708+0000 7f23ce800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23d010a010 0x7f23d019dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:58800/0 (socket says 172.21.15.117:58800) 2024-06-03T03:10:55.719 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:54.708+0000 7f23ce800700 1 -- 172.21.15.117:0/3827187470 learned_addr learned my addr 172.21.15.117:0/3827187470 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:10:55.719 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:54.709+0000 7f23ce800700 1 -- 172.21.15.117:0/3827187470 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23d019e520 con 0x7f23d010a010 2024-06-03T03:10:55.719 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:54.709+0000 7f23ce800700 1 --2- 172.21.15.117:0/3827187470 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23d010a010 0x7f23d019dfe0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f23c0011850 tx=0x7f23c0011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:10:55.719 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:54.709+0000 7f23c7400700 1 -- 172.21.15.117:0/3827187470 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f23c0003af0 con 0x7f23d010a010 2024-06-03T03:10:55.719 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:54.709+0000 7f23c7400700 1 -- 172.21.15.117:0/3827187470 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f23c0020070 con 0x7f23d010a010 2024-06-03T03:10:55.719 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:54.709+0000 7f23c7400700 1 -- 172.21.15.117:0/3827187470 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f23c00195a0 con 0x7f23d010a010 2024-06-03T03:10:55.720 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:54.709+0000 7f23d4a00700 1 -- 172.21.15.117:0/3827187470 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23d019e7b0 con 0x7f23d010a010 2024-06-03T03:10:55.720 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:54.709+0000 7f23d4a00700 1 -- 172.21.15.117:0/3827187470 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23d01a14a0 con 0x7f23d010a010 2024-06-03T03:10:55.720 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:54.710+0000 7f23d4a00700 1 -- 172.21.15.117:0/3827187470 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23d00695b0 con 0x7f23d010a010 2024-06-03T03:10:55.720 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:54.711+0000 7f23c7400700 1 -- 172.21.15.117:0/3827187470 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f23c00198a0 con 0x7f23d010a010 2024-06-03T03:10:55.720 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:54.711+0000 7f23c7400700 1 --2- 172.21.15.117:0/3827187470 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7f23b4039050 0x7f23b403b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:10:55.720 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:54.711+0000 7f23c7400700 1 -- 172.21.15.117:0/3827187470 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f23c0011e00 con 0x7f23d010a010 2024-06-03T03:10:55.720 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:54.711+0000 7f23cde00700 1 --2- 172.21.15.117:0/3827187470 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7f23b4039050 0x7f23b403b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:10:55.720 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:54.712+0000 7f23cde00700 1 --2- 172.21.15.117:0/3827187470 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7f23b4039050 0x7f23b403b510 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f23bc006fd0 tx=0x7f23bc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:10:55.720 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:54.717+0000 7f23c7400700 1 -- 172.21.15.117:0/3827187470 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f23c000e260 con 0x7f23d010a010 2024-06-03T03:10:55.720 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:54.940+0000 7f23d4a00700 1 -- 172.21.15.117:0/3827187470 --> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f23d0107bd0 con 0x7f23b4039050 2024-06-03T03:10:55.720 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:54.941+0000 7f23c7400700 1 -- 172.21.15.117:0/3827187470 <== mgr.14120 v2:172.21.15.117:6800/2142313590 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f23d0107bd0 con 0x7f23b4039050 2024-06-03T03:10:55.720 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:54.944+0000 7f23d4a00700 1 -- 172.21.15.117:0/3827187470 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7f23b4039050 msgr2=0x7f23b403b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:10:55.721 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:54.944+0000 7f23d4a00700 1 --2- 172.21.15.117:0/3827187470 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7f23b4039050 0x7f23b403b510 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f23bc006fd0 tx=0x7f23bc006e40 comp rx=0 tx=0).stop 2024-06-03T03:10:55.721 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:54.944+0000 7f23d4a00700 1 -- 172.21.15.117:0/3827187470 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23d010a010 msgr2=0x7f23d019dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:10:55.721 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:54.944+0000 7f23d4a00700 1 --2- 172.21.15.117:0/3827187470 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23d010a010 0x7f23d019dfe0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f23c0011850 tx=0x7f23c0011c10 comp rx=0 tx=0).stop 2024-06-03T03:10:55.721 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:54.944+0000 7f23d4a00700 1 -- 172.21.15.117:0/3827187470 shutdown_connections 2024-06-03T03:10:55.721 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:54.944+0000 7f23d4a00700 1 --2- 172.21.15.117:0/3827187470 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7f23b4039050 0x7f23b403b510 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:10:55.721 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:54.944+0000 7f23d4a00700 1 --2- 172.21.15.117:0/3827187470 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23d010a010 0x7f23d019dfe0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:10:55.721 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:54.944+0000 7f23d4a00700 1 -- 172.21.15.117:0/3827187470 >> 172.21.15.117:0/3827187470 conn(0x7f23d007d3d0 msgr2=0x7f23d01074c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:10:55.721 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:54.945+0000 7f23d4a00700 1 -- 172.21.15.117:0/3827187470 shutdown_connections 2024-06-03T03:10:55.721 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:54.945+0000 7f23d4a00700 1 -- 172.21.15.117:0/3827187470 wait complete. 2024-06-03T03:10:55.721 INFO:teuthology.orchestra.run.smithi117.stdout:Generating ssh key... 2024-06-03T03:10:56.020 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:55 smithi117 ceph-mon[31649]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:10:57.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:56 smithi117 ceph-mon[31649]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:10:57.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:56 smithi117 ceph-mon[31649]: Generating ssh key... 2024-06-03T03:10:57.068 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:10:57 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: Generating public/private rsa key pair. 2024-06-03T03:10:57.068 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:10:57 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: Your identification has been saved in /tmp/tmpvav4u5iw/key. 2024-06-03T03:10:57.068 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:10:57 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: Your public key has been saved in /tmp/tmpvav4u5iw/key.pub. 2024-06-03T03:10:57.068 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:10:57 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: The key fingerprint is: 2024-06-03T03:10:57.068 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:10:57 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: SHA256:iOZyX1JlFJOWIBTi4pyn70aRdv8IdZ4wMDaEfBwpPH4 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968 2024-06-03T03:10:57.068 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:10:57 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: The key's randomart image is: 2024-06-03T03:10:57.068 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:10:57 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: +---[RSA 3072]----+ 2024-06-03T03:10:57.068 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:10:57 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: | o.=*+..++ | 2024-06-03T03:10:57.068 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:10:57 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: | .*.O. .+. | 2024-06-03T03:10:57.068 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:10:57 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: | ...* + .o | 2024-06-03T03:10:57.068 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:10:57 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: | o o=.E.+o. | 2024-06-03T03:10:57.068 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:10:57 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: | +.++.oS= . | 2024-06-03T03:10:57.068 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:10:57 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: | =. ... o | 2024-06-03T03:10:57.068 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:10:57 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: | o.o ...o | 2024-06-03T03:10:57.069 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:10:57 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: | +.. o. . | 2024-06-03T03:10:57.069 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:10:57 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: | oo . | 2024-06-03T03:10:57.069 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:10:57 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: +----[SHA256]-----+ 2024-06-03T03:10:57.746 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:56.323+0000 7efde0800700 1 Processor -- start 2024-06-03T03:10:57.748 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:56.323+0000 7efde0800700 1 -- start start 2024-06-03T03:10:57.748 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:56.323+0000 7efde0800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efdd8106870 0x7efdd8106c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:10:57.748 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:56.323+0000 7efde0800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7efdd807be30 con 0x7efdd8106870 2024-06-03T03:10:57.748 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:56.324+0000 7efddde00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efdd8106870 0x7efdd8106c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:10:57.748 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:56.324+0000 7efddde00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efdd8106870 0x7efdd8106c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:58816/0 (socket says 172.21.15.117:58816) 2024-06-03T03:10:57.748 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:56.324+0000 7efddde00700 1 -- 172.21.15.117:0/4061069606 learned_addr learned my addr 172.21.15.117:0/4061069606 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:10:57.748 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:56.324+0000 7efddde00700 1 -- 172.21.15.117:0/4061069606 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdd81071d0 con 0x7efdd8106870 2024-06-03T03:10:57.751 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:56.325+0000 7efddde00700 1 --2- 172.21.15.117:0/4061069606 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efdd8106870 0x7efdd8106c90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7efdcc0098a0 tx=0x7efdcc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=948a5f243eb191f6 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:10:57.751 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:56.325+0000 7efddca00700 1 -- 172.21.15.117:0/4061069606 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efdcc004160 con 0x7efdd8106870 2024-06-03T03:10:57.751 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:56.325+0000 7efddca00700 1 -- 172.21.15.117:0/4061069606 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7efdcc004300 con 0x7efdd8106870 2024-06-03T03:10:57.751 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:56.325+0000 7efddca00700 1 -- 172.21.15.117:0/4061069606 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efdcc011b50 con 0x7efdd8106870 2024-06-03T03:10:57.751 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:56.325+0000 7efde0800700 1 -- 172.21.15.117:0/4061069606 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efdd8106870 msgr2=0x7efdd8106c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:10:57.751 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:56.325+0000 7efde0800700 1 --2- 172.21.15.117:0/4061069606 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efdd8106870 0x7efdd8106c90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7efdcc0098a0 tx=0x7efdcc009bb0 comp rx=0 tx=0).stop 2024-06-03T03:10:57.751 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:56.326+0000 7efde0800700 1 -- 172.21.15.117:0/4061069606 shutdown_connections 2024-06-03T03:10:57.751 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:56.326+0000 7efde0800700 1 --2- 172.21.15.117:0/4061069606 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efdd8106870 0x7efdd8106c90 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:10:57.751 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:56.326+0000 7efde0800700 1 -- 172.21.15.117:0/4061069606 >> 172.21.15.117:0/4061069606 conn(0x7efdd8102450 msgr2=0x7efdd8104870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:10:57.751 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:56.326+0000 7efde0800700 1 -- 172.21.15.117:0/4061069606 shutdown_connections 2024-06-03T03:10:57.751 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:56.326+0000 7efde0800700 1 -- 172.21.15.117:0/4061069606 wait complete. 2024-06-03T03:10:57.751 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:56.327+0000 7efde0800700 1 Processor -- start 2024-06-03T03:10:57.752 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:56.327+0000 7efde0800700 1 -- start start 2024-06-03T03:10:57.752 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:56.327+0000 7efde0800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efdd8106870 0x7efdd819e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:10:57.752 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:56.327+0000 7efde0800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7efdd8108260 con 0x7efdd8106870 2024-06-03T03:10:57.752 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:56.327+0000 7efddde00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efdd8106870 0x7efdd819e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:10:57.752 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:56.327+0000 7efddde00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efdd8106870 0x7efdd819e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:58824/0 (socket says 172.21.15.117:58824) 2024-06-03T03:10:57.752 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:56.328+0000 7efddde00700 1 -- 172.21.15.117:0/3788175782 learned_addr learned my addr 172.21.15.117:0/3788175782 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:10:57.752 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:56.328+0000 7efddde00700 1 -- 172.21.15.117:0/3788175782 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdd819e650 con 0x7efdd8106870 2024-06-03T03:10:57.752 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:56.328+0000 7efddde00700 1 --2- 172.21.15.117:0/3788175782 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efdd8106870 0x7efdd819e110 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7efdcc011b60 tx=0x7efdcc011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:10:57.752 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:56.328+0000 7efdd2a00700 1 -- 172.21.15.117:0/3788175782 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efdcc003b20 con 0x7efdd8106870 2024-06-03T03:10:57.755 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:56.328+0000 7efdd2a00700 1 -- 172.21.15.117:0/3788175782 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7efdcc003cc0 con 0x7efdd8106870 2024-06-03T03:10:57.755 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:56.328+0000 7efdd2a00700 1 -- 172.21.15.117:0/3788175782 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efdcc024840 con 0x7efdd8106870 2024-06-03T03:10:57.755 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:56.328+0000 7efde0800700 1 -- 172.21.15.117:0/3788175782 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdd819e8e0 con 0x7efdd8106870 2024-06-03T03:10:57.755 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:56.329+0000 7efde0800700 1 -- 172.21.15.117:0/3788175782 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdd81a15d0 con 0x7efdd8106870 2024-06-03T03:10:57.755 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:56.330+0000 7efdd2a00700 1 -- 172.21.15.117:0/3788175782 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7efdcc022020 con 0x7efdd8106870 2024-06-03T03:10:57.755 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:56.330+0000 7efde0800700 1 -- 172.21.15.117:0/3788175782 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdd8063af0 con 0x7efdd8106870 2024-06-03T03:10:57.755 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:56.330+0000 7efdd2a00700 1 --2- 172.21.15.117:0/3788175782 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7efdc00390a0 0x7efdc003b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:10:57.755 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:56.330+0000 7efdd2a00700 1 -- 172.21.15.117:0/3788175782 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7efdcc011e00 con 0x7efdd8106870 2024-06-03T03:10:57.755 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:56.330+0000 7efddd400700 1 --2- 172.21.15.117:0/3788175782 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7efdc00390a0 0x7efdc003b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:10:57.755 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:56.331+0000 7efddd400700 1 --2- 172.21.15.117:0/3788175782 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7efdc00390a0 0x7efdc003b560 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7efdc8006fd0 tx=0x7efdc8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:10:57.755 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:56.336+0000 7efdd2a00700 1 -- 172.21.15.117:0/3788175782 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7efdcc04cb30 con 0x7efdd8106870 2024-06-03T03:10:57.756 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:56.557+0000 7efde0800700 1 -- 172.21.15.117:0/3788175782 --> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7efdd802df80 con 0x7efdc00390a0 2024-06-03T03:10:57.756 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:57.048+0000 7efdd2a00700 1 -- 172.21.15.117:0/3788175782 <== mgr.14120 v2:172.21.15.117:6800/2142313590 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7efdd802df80 con 0x7efdc00390a0 2024-06-03T03:10:57.756 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:57.051+0000 7efde0800700 1 -- 172.21.15.117:0/3788175782 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7efdc00390a0 msgr2=0x7efdc003b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:10:57.756 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:57.051+0000 7efde0800700 1 --2- 172.21.15.117:0/3788175782 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7efdc00390a0 0x7efdc003b560 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7efdc8006fd0 tx=0x7efdc8006e40 comp rx=0 tx=0).stop 2024-06-03T03:10:57.756 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:57.051+0000 7efde0800700 1 -- 172.21.15.117:0/3788175782 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efdd8106870 msgr2=0x7efdd819e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:10:57.756 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:57.051+0000 7efde0800700 1 --2- 172.21.15.117:0/3788175782 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efdd8106870 0x7efdd819e110 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7efdcc011b60 tx=0x7efdcc011c40 comp rx=0 tx=0).stop 2024-06-03T03:10:57.756 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:57.051+0000 7efde0800700 1 -- 172.21.15.117:0/3788175782 shutdown_connections 2024-06-03T03:10:57.756 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:57.051+0000 7efde0800700 1 --2- 172.21.15.117:0/3788175782 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7efdc00390a0 0x7efdc003b560 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:10:57.756 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:57.051+0000 7efde0800700 1 --2- 172.21.15.117:0/3788175782 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efdd8106870 0x7efdd819e110 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:10:57.756 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:57.051+0000 7efde0800700 1 -- 172.21.15.117:0/3788175782 >> 172.21.15.117:0/3788175782 conn(0x7efdd8102450 msgr2=0x7efdd8194d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:10:57.756 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:57.051+0000 7efde0800700 1 -- 172.21.15.117:0/3788175782 shutdown_connections 2024-06-03T03:10:57.756 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:57.051+0000 7efde0800700 1 -- 172.21.15.117:0/3788175782 wait complete. 2024-06-03T03:10:58.309 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:58 smithi117 ceph-mon[31649]: from='mgr.14120 172.21.15.117:0/175906377' entity='mgr.x' 2024-06-03T03:10:58.309 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:58 smithi117 ceph-mon[31649]: from='mgr.14120 172.21.15.117:0/175906377' entity='mgr.x' 2024-06-03T03:10:59.312 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:10:59 smithi117 ceph-mon[31649]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:10:59.331 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDk9chwckqyNL+exGwktePMvfnjZu1PoS16S7g4tpE8O9j8b18USig1upkmhoI1nasKfujRnMxCFlRCKNN2DWYx3xkXfrwvLyjxQxEDiVlj88IhkIU03sTTMEfkghw6jUNEWznlWExf1VRiEdPd8GS7Uqeo2KogRGTDwUkjOaB3nvBu2Ot5ljE0kC0tTK41kT8oqgyT6L+4kAnPAKs+H5KalftCVqpdDVBB+xa1Uuj4UNdiBKYSqdCII5Ch92sOrWTkFykOinkvUju2Pge3FwJJ+m9CC1OQT0IgCZCU8DobO6Kg66Pu/CNsm2Z/QZwE4c9P6Bt3BTAwLt7Ne74/P9iLAuNdr4BoD5aPuHiMwJnm6vwuD1/KP6zKxzlHFTFZnaK9MO20FPGpZHZxW3HjGyVBx3apmDhoeIb3mZSZqaP5SdltmHxT+Nne4D8RjkJxIjNwi1cbQXUaQ6GDibL53PKb+3sEKE9B/nUEbfCQWvIro63q4Y4CcaYbH01iVaCenGc= ceph-a0e8473a-2156-11ef-bc9b-c7b262605968 2024-06-03T03:10:59.331 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:58.378+0000 7fdfd4a00700 1 Processor -- start 2024-06-03T03:10:59.331 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:58.378+0000 7fdfd4a00700 1 -- start start 2024-06-03T03:10:59.331 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:58.378+0000 7fdfd4a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfd007ef90 0x7fdfd007f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:10:59.331 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:58.378+0000 7fdfd4a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd007f980 con 0x7fdfd007ef90 2024-06-03T03:10:59.331 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:58.379+0000 7fdfcde00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfd007ef90 0x7fdfd007f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:10:59.331 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:58.379+0000 7fdfcde00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfd007ef90 0x7fdfd007f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:60530/0 (socket says 172.21.15.117:60530) 2024-06-03T03:10:59.331 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:58.379+0000 7fdfcde00700 1 -- 172.21.15.117:0/2091182458 learned_addr learned my addr 172.21.15.117:0/2091182458 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:10:59.332 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:58.379+0000 7fdfcde00700 1 -- 172.21.15.117:0/2091182458 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfd0080180 con 0x7fdfd007ef90 2024-06-03T03:10:59.332 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:58.380+0000 7fdfcde00700 1 --2- 172.21.15.117:0/2091182458 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfd007ef90 0x7fdfd007f3b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fdfc00098a0 tx=0x7fdfc0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d59b57c83cddd4e server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:10:59.332 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:58.380+0000 7fdfcca00700 1 -- 172.21.15.117:0/2091182458 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdfc0004160 con 0x7fdfd007ef90 2024-06-03T03:10:59.332 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:58.380+0000 7fdfcca00700 1 -- 172.21.15.117:0/2091182458 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdfc0004300 con 0x7fdfd007ef90 2024-06-03T03:10:59.332 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:58.380+0000 7fdfcca00700 1 -- 172.21.15.117:0/2091182458 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdfc0011b50 con 0x7fdfd007ef90 2024-06-03T03:10:59.332 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:58.381+0000 7fdfd4a00700 1 -- 172.21.15.117:0/2091182458 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfd007ef90 msgr2=0x7fdfd007f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:10:59.332 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:58.381+0000 7fdfd4a00700 1 --2- 172.21.15.117:0/2091182458 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfd007ef90 0x7fdfd007f3b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fdfc00098a0 tx=0x7fdfc0009bb0 comp rx=0 tx=0).stop 2024-06-03T03:10:59.332 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:58.381+0000 7fdfd4a00700 1 -- 172.21.15.117:0/2091182458 shutdown_connections 2024-06-03T03:10:59.332 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:58.381+0000 7fdfd4a00700 1 --2- 172.21.15.117:0/2091182458 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfd007ef90 0x7fdfd007f3b0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:10:59.332 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:58.381+0000 7fdfd4a00700 1 -- 172.21.15.117:0/2091182458 >> 172.21.15.117:0/2091182458 conn(0x7fdfd007d710 msgr2=0x7fdfd007db60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:10:59.332 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:58.381+0000 7fdfd4a00700 1 -- 172.21.15.117:0/2091182458 shutdown_connections 2024-06-03T03:10:59.332 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:58.381+0000 7fdfd4a00700 1 -- 172.21.15.117:0/2091182458 wait complete. 2024-06-03T03:10:59.333 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:58.382+0000 7fdfd4a00700 1 Processor -- start 2024-06-03T03:10:59.333 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:58.382+0000 7fdfd4a00700 1 -- start start 2024-06-03T03:10:59.333 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:58.382+0000 7fdfd4a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfd007ef90 0x7fdfd01a2550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:10:59.333 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:58.382+0000 7fdfd4a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd0081540 con 0x7fdfd007ef90 2024-06-03T03:10:59.333 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:58.383+0000 7fdfcde00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfd007ef90 0x7fdfd01a2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:10:59.333 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:58.383+0000 7fdfcde00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfd007ef90 0x7fdfd01a2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:60540/0 (socket says 172.21.15.117:60540) 2024-06-03T03:10:59.333 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:58.383+0000 7fdfcde00700 1 -- 172.21.15.117:0/3509059140 learned_addr learned my addr 172.21.15.117:0/3509059140 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:10:59.333 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:58.383+0000 7fdfcde00700 1 -- 172.21.15.117:0/3509059140 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfd01a2a90 con 0x7fdfd007ef90 2024-06-03T03:10:59.333 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:58.383+0000 7fdfcde00700 1 --2- 172.21.15.117:0/3509059140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfd007ef90 0x7fdfd01a2550 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fdfc0000c00 tx=0x7fdfc0011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:10:59.333 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:58.383+0000 7fdfc6a00700 1 -- 172.21.15.117:0/3509059140 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdfc0003b20 con 0x7fdfd007ef90 2024-06-03T03:10:59.333 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:58.384+0000 7fdfc6a00700 1 -- 172.21.15.117:0/3509059140 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdfc0003cc0 con 0x7fdfd007ef90 2024-06-03T03:10:59.333 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:58.384+0000 7fdfc6a00700 1 -- 172.21.15.117:0/3509059140 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdfc0024910 con 0x7fdfd007ef90 2024-06-03T03:10:59.334 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:58.384+0000 7fdfd4a00700 1 -- 172.21.15.117:0/3509059140 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfd01a2d20 con 0x7fdfd007ef90 2024-06-03T03:10:59.334 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:58.384+0000 7fdfd4a00700 1 -- 172.21.15.117:0/3509059140 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfd01a5a10 con 0x7fdfd007ef90 2024-06-03T03:10:59.334 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:58.385+0000 7fdfd4a00700 1 -- 172.21.15.117:0/3509059140 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfd0063af0 con 0x7fdfd007ef90 2024-06-03T03:10:59.334 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:58.385+0000 7fdfc6a00700 1 -- 172.21.15.117:0/3509059140 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fdfc0022020 con 0x7fdfd007ef90 2024-06-03T03:10:59.334 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:58.385+0000 7fdfc6a00700 1 --2- 172.21.15.117:0/3509059140 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7fdfb40390a0 0x7fdfb403b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:10:59.334 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:58.385+0000 7fdfc6a00700 1 -- 172.21.15.117:0/3509059140 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fdfc0011e00 con 0x7fdfd007ef90 2024-06-03T03:10:59.334 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:58.386+0000 7fdfcd400700 1 --2- 172.21.15.117:0/3509059140 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7fdfb40390a0 0x7fdfb403b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:10:59.334 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:58.386+0000 7fdfcd400700 1 --2- 172.21.15.117:0/3509059140 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7fdfb40390a0 0x7fdfb403b560 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fdfb8006fd0 tx=0x7fdfb8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:10:59.334 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:58.391+0000 7fdfc6a00700 1 -- 172.21.15.117:0/3509059140 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fdfc000f2e0 con 0x7fdfd007ef90 2024-06-03T03:10:59.334 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:58.617+0000 7fdfd4a00700 1 -- 172.21.15.117:0/3509059140 --> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7fdfd00628d0 con 0x7fdfb40390a0 2024-06-03T03:10:59.334 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:58.619+0000 7fdfc6a00700 1 -- 172.21.15.117:0/3509059140 <== mgr.14120 v2:172.21.15.117:6800/2142313590 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7fdfd00628d0 con 0x7fdfb40390a0 2024-06-03T03:10:59.334 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:58.621+0000 7fdfd4a00700 1 -- 172.21.15.117:0/3509059140 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7fdfb40390a0 msgr2=0x7fdfb403b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:10:59.335 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:58.621+0000 7fdfd4a00700 1 --2- 172.21.15.117:0/3509059140 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7fdfb40390a0 0x7fdfb403b560 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fdfb8006fd0 tx=0x7fdfb8006e40 comp rx=0 tx=0).stop 2024-06-03T03:10:59.335 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:58.621+0000 7fdfd4a00700 1 -- 172.21.15.117:0/3509059140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfd007ef90 msgr2=0x7fdfd01a2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:10:59.335 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:58.621+0000 7fdfd4a00700 1 --2- 172.21.15.117:0/3509059140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfd007ef90 0x7fdfd01a2550 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fdfc0000c00 tx=0x7fdfc0011c10 comp rx=0 tx=0).stop 2024-06-03T03:10:59.335 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:58.621+0000 7fdfd4a00700 1 -- 172.21.15.117:0/3509059140 shutdown_connections 2024-06-03T03:10:59.335 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:58.621+0000 7fdfd4a00700 1 --2- 172.21.15.117:0/3509059140 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7fdfb40390a0 0x7fdfb403b560 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:10:59.335 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:58.621+0000 7fdfd4a00700 1 --2- 172.21.15.117:0/3509059140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfd007ef90 0x7fdfd01a2550 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:10:59.335 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:58.621+0000 7fdfd4a00700 1 -- 172.21.15.117:0/3509059140 >> 172.21.15.117:0/3509059140 conn(0x7fdfd007d710 msgr2=0x7fdfd0109050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:10:59.335 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:58.622+0000 7fdfd4a00700 1 -- 172.21.15.117:0/3509059140 shutdown_connections 2024-06-03T03:10:59.335 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:58.622+0000 7fdfd4a00700 1 -- 172.21.15.117:0/3509059140 wait complete. 2024-06-03T03:10:59.335 INFO:teuthology.orchestra.run.smithi117.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-06-03T03:10:59.335 INFO:teuthology.orchestra.run.smithi117.stdout:Adding key to root@localhost authorized_keys... 2024-06-03T03:10:59.336 INFO:teuthology.orchestra.run.smithi117.stdout:Adding host smithi117... 2024-06-03T03:11:01.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:00 smithi117 ceph-mon[31649]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi117", "addr": "172.21.15.117", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:11:02.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:01 smithi117 ceph-mon[31649]: Deploying cephadm binary to smithi117 2024-06-03T03:11:06.941 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout Added host 'smithi117' with addr '172.21.15.117' 2024-06-03T03:11:06.942 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:59.962+0000 7f9221c00700 1 Processor -- start 2024-06-03T03:11:06.942 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:59.962+0000 7f9221c00700 1 -- start start 2024-06-03T03:11:06.942 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:59.962+0000 7f9221c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f921c107f30 0x7f921c108350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:11:06.942 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:59.963+0000 7f9221c00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f921c108890 con 0x7f921c107f30 2024-06-03T03:11:06.942 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:59.963+0000 7f921aa00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f921c107f30 0x7f921c108350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:11:06.942 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:59.963+0000 7f921aa00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f921c107f30 0x7f921c108350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:60552/0 (socket says 172.21.15.117:60552) 2024-06-03T03:11:06.942 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:59.963+0000 7f921aa00700 1 -- 172.21.15.117:0/131408265 learned_addr learned my addr 172.21.15.117:0/131408265 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:11:06.942 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:59.963+0000 7f921aa00700 1 -- 172.21.15.117:0/131408265 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f921c109090 con 0x7f921c107f30 2024-06-03T03:11:06.942 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:59.964+0000 7f921aa00700 1 --2- 172.21.15.117:0/131408265 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f921c107f30 0x7f921c108350 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f920c0098a0 tx=0x7f920c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97a71b2ed9394fb0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:11:06.942 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:59.964+0000 7f9219600700 1 -- 172.21.15.117:0/131408265 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f920c004160 con 0x7f921c107f30 2024-06-03T03:11:06.942 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:59.964+0000 7f9219600700 1 -- 172.21.15.117:0/131408265 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f920c004300 con 0x7f921c107f30 2024-06-03T03:11:06.942 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:59.964+0000 7f9219600700 1 -- 172.21.15.117:0/131408265 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f920c011b50 con 0x7f921c107f30 2024-06-03T03:11:06.943 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:59.965+0000 7f9221c00700 1 -- 172.21.15.117:0/131408265 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f921c107f30 msgr2=0x7f921c108350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:11:06.943 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:59.965+0000 7f9221c00700 1 --2- 172.21.15.117:0/131408265 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f921c107f30 0x7f921c108350 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f920c0098a0 tx=0x7f920c009bb0 comp rx=0 tx=0).stop 2024-06-03T03:11:06.943 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:59.965+0000 7f9221c00700 1 -- 172.21.15.117:0/131408265 shutdown_connections 2024-06-03T03:11:06.943 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:59.965+0000 7f9221c00700 1 --2- 172.21.15.117:0/131408265 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f921c107f30 0x7f921c108350 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:11:06.943 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:59.965+0000 7f9221c00700 1 -- 172.21.15.117:0/131408265 >> 172.21.15.117:0/131408265 conn(0x7f921c103530 msgr2=0x7f921c105950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:11:06.943 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:59.965+0000 7f9221c00700 1 -- 172.21.15.117:0/131408265 shutdown_connections 2024-06-03T03:11:06.943 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:59.965+0000 7f9221c00700 1 -- 172.21.15.117:0/131408265 wait complete. 2024-06-03T03:11:06.943 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:59.966+0000 7f9221c00700 1 Processor -- start 2024-06-03T03:11:06.946 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:59.966+0000 7f9221c00700 1 -- start start 2024-06-03T03:11:06.946 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:59.966+0000 7f9221c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f921c107f30 0x7f921c19bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:11:06.947 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:59.966+0000 7f9221c00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f921c10c4d0 con 0x7f921c107f30 2024-06-03T03:11:06.947 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:59.967+0000 7f921aa00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f921c107f30 0x7f921c19bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:11:06.947 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:59.967+0000 7f921aa00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f921c107f30 0x7f921c19bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:60556/0 (socket says 172.21.15.117:60556) 2024-06-03T03:11:06.947 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:59.967+0000 7f921aa00700 1 -- 172.21.15.117:0/1366634329 learned_addr learned my addr 172.21.15.117:0/1366634329 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:11:06.947 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:59.967+0000 7f921aa00700 1 -- 172.21.15.117:0/1366634329 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f921c19c450 con 0x7f921c107f30 2024-06-03T03:11:06.947 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:59.967+0000 7f921aa00700 1 --2- 172.21.15.117:0/1366634329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f921c107f30 0x7f921c19bf10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f920c011670 tx=0x7f920c011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:11:06.947 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:59.968+0000 7f9213400700 1 -- 172.21.15.117:0/1366634329 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f920c0039c0 con 0x7f921c107f30 2024-06-03T03:11:06.947 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:59.968+0000 7f9213400700 1 -- 172.21.15.117:0/1366634329 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f920c011e50 con 0x7f921c107f30 2024-06-03T03:11:06.947 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:59.968+0000 7f9221c00700 1 -- 172.21.15.117:0/1366634329 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f921c19c6e0 con 0x7f921c107f30 2024-06-03T03:11:06.947 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:59.968+0000 7f9213400700 1 -- 172.21.15.117:0/1366634329 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f920c01a5c0 con 0x7f921c107f30 2024-06-03T03:11:06.947 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:59.968+0000 7f9221c00700 1 -- 172.21.15.117:0/1366634329 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f921c19f3d0 con 0x7f921c107f30 2024-06-03T03:11:06.947 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:59.969+0000 7f9213400700 1 -- 172.21.15.117:0/1366634329 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f920c024430 con 0x7f921c107f30 2024-06-03T03:11:06.947 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:59.969+0000 7f9221c00700 1 -- 172.21.15.117:0/1366634329 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f921c063af0 con 0x7f921c107f30 2024-06-03T03:11:06.948 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:59.970+0000 7f9213400700 1 --2- 172.21.15.117:0/1366634329 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7f9200039050 0x7f920003b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:11:06.948 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:59.970+0000 7f9213400700 1 -- 172.21.15.117:0/1366634329 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f920c04d4f0 con 0x7f921c107f30 2024-06-03T03:11:06.948 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:59.970+0000 7f921a000700 1 --2- 172.21.15.117:0/1366634329 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7f9200039050 0x7f920003b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:11:06.948 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:59.971+0000 7f921a000700 1 --2- 172.21.15.117:0/1366634329 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7f9200039050 0x7f920003b510 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9208006fd0 tx=0x7f9208006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:11:06.948 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:10:59.976+0000 7f9213400700 1 -- 172.21.15.117:0/1366634329 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f920c04ce40 con 0x7f921c107f30 2024-06-03T03:11:06.948 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:00.204+0000 7f9221c00700 1 -- 172.21.15.117:0/1366634329 --> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi117", "addr": "172.21.15.117", "target": ["mon-mgr", ""]}) -- 0x7f921c104780 con 0x7f9200039050 2024-06-03T03:11:06.948 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:06.195+0000 7f9213400700 1 -- 172.21.15.117:0/1366634329 <== mgr.14120 v2:172.21.15.117:6800/2142313590 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f921c104780 con 0x7f9200039050 2024-06-03T03:11:06.948 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:06.199+0000 7f9221c00700 1 -- 172.21.15.117:0/1366634329 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7f9200039050 msgr2=0x7f920003b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:11:06.948 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:06.199+0000 7f9221c00700 1 --2- 172.21.15.117:0/1366634329 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7f9200039050 0x7f920003b510 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9208006fd0 tx=0x7f9208006e40 comp rx=0 tx=0).stop 2024-06-03T03:11:06.948 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:06.199+0000 7f9221c00700 1 -- 172.21.15.117:0/1366634329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f921c107f30 msgr2=0x7f921c19bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:11:06.948 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:06.199+0000 7f9221c00700 1 --2- 172.21.15.117:0/1366634329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f921c107f30 0x7f921c19bf10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f920c011670 tx=0x7f920c011b60 comp rx=0 tx=0).stop 2024-06-03T03:11:06.948 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:06.200+0000 7f9221c00700 1 -- 172.21.15.117:0/1366634329 shutdown_connections 2024-06-03T03:11:06.948 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:06.200+0000 7f9221c00700 1 --2- 172.21.15.117:0/1366634329 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7f9200039050 0x7f920003b510 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:11:06.949 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:06.200+0000 7f9221c00700 1 --2- 172.21.15.117:0/1366634329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f921c107f30 0x7f921c19bf10 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:11:06.949 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:06.200+0000 7f9221c00700 1 -- 172.21.15.117:0/1366634329 >> 172.21.15.117:0/1366634329 conn(0x7f921c103530 msgr2=0x7f921c104070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:11:06.949 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:06.200+0000 7f9221c00700 1 -- 172.21.15.117:0/1366634329 shutdown_connections 2024-06-03T03:11:06.949 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:06.200+0000 7f9221c00700 1 -- 172.21.15.117:0/1366634329 wait complete. 2024-06-03T03:11:06.949 INFO:teuthology.orchestra.run.smithi117.stdout:Deploying unmanaged mon service... 2024-06-03T03:11:07.449 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:07 smithi117 ceph-mon[31649]: from='mgr.14120 172.21.15.117:0/175906377' entity='mgr.x' 2024-06-03T03:11:07.449 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:07 smithi117 ceph-mon[31649]: Added host smithi117 2024-06-03T03:11:07.449 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:07 smithi117 ceph-mon[31649]: from='mgr.14120 172.21.15.117:0/175906377' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:11:08.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:08 smithi117 ceph-mon[31649]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:11:08.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:08 smithi117 ceph-mon[31649]: Saving service mon spec with placement count:5 2024-06-03T03:11:08.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:08 smithi117 ceph-mon[31649]: from='mgr.14120 172.21.15.117:0/175906377' entity='mgr.x' 2024-06-03T03:11:09.104 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-06-03T03:11:09.104 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:07.616+0000 7fc093600700 1 Processor -- start 2024-06-03T03:11:09.104 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:07.616+0000 7fc093600700 1 -- start start 2024-06-03T03:11:09.104 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:07.617+0000 7fc093600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc08c107f30 0x7fc08c108350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:11:09.105 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:07.617+0000 7fc093600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc08c108890 con 0x7fc08c107f30 2024-06-03T03:11:09.105 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:07.617+0000 7fc090c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc08c107f30 0x7fc08c108350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:11:09.105 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:07.617+0000 7fc090c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc08c107f30 0x7fc08c108350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34142/0 (socket says 172.21.15.117:34142) 2024-06-03T03:11:09.105 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:07.617+0000 7fc090c00700 1 -- 172.21.15.117:0/3462217981 learned_addr learned my addr 172.21.15.117:0/3462217981 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:11:09.105 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:07.618+0000 7fc090c00700 1 -- 172.21.15.117:0/3462217981 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc08c109090 con 0x7fc08c107f30 2024-06-03T03:11:09.105 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:07.618+0000 7fc090c00700 1 --2- 172.21.15.117:0/3462217981 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc08c107f30 0x7fc08c108350 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc0800098a0 tx=0x7fc080009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebc31101781a72e7 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:11:09.105 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:07.618+0000 7fc08b400700 1 -- 172.21.15.117:0/3462217981 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc080004160 con 0x7fc08c107f30 2024-06-03T03:11:09.105 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:07.618+0000 7fc08b400700 1 -- 172.21.15.117:0/3462217981 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc080004300 con 0x7fc08c107f30 2024-06-03T03:11:09.105 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:07.618+0000 7fc08b400700 1 -- 172.21.15.117:0/3462217981 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc080011b50 con 0x7fc08c107f30 2024-06-03T03:11:09.105 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:07.619+0000 7fc093600700 1 -- 172.21.15.117:0/3462217981 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc08c107f30 msgr2=0x7fc08c108350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:11:09.105 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:07.619+0000 7fc093600700 1 --2- 172.21.15.117:0/3462217981 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc08c107f30 0x7fc08c108350 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc0800098a0 tx=0x7fc080009bb0 comp rx=0 tx=0).stop 2024-06-03T03:11:09.106 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:07.619+0000 7fc093600700 1 -- 172.21.15.117:0/3462217981 shutdown_connections 2024-06-03T03:11:09.106 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:07.619+0000 7fc093600700 1 --2- 172.21.15.117:0/3462217981 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc08c107f30 0x7fc08c108350 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:11:09.106 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:07.619+0000 7fc093600700 1 -- 172.21.15.117:0/3462217981 >> 172.21.15.117:0/3462217981 conn(0x7fc08c103530 msgr2=0x7fc08c105950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:11:09.106 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:07.619+0000 7fc093600700 1 -- 172.21.15.117:0/3462217981 shutdown_connections 2024-06-03T03:11:09.106 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:07.619+0000 7fc093600700 1 -- 172.21.15.117:0/3462217981 wait complete. 2024-06-03T03:11:09.106 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:07.620+0000 7fc093600700 1 Processor -- start 2024-06-03T03:11:09.106 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:07.620+0000 7fc093600700 1 -- start start 2024-06-03T03:11:09.106 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:07.621+0000 7fc093600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc08c107f30 0x7fc08c19bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:11:09.106 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:07.621+0000 7fc093600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc08c10c4d0 con 0x7fc08c107f30 2024-06-03T03:11:09.106 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:07.621+0000 7fc090c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc08c107f30 0x7fc08c19bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:11:09.106 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:07.621+0000 7fc090c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc08c107f30 0x7fc08c19bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34148/0 (socket says 172.21.15.117:34148) 2024-06-03T03:11:09.107 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:07.621+0000 7fc090c00700 1 -- 172.21.15.117:0/3766053195 learned_addr learned my addr 172.21.15.117:0/3766053195 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:11:09.107 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:07.621+0000 7fc090c00700 1 -- 172.21.15.117:0/3766053195 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc08c19c480 con 0x7fc08c107f30 2024-06-03T03:11:09.107 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:07.622+0000 7fc090c00700 1 --2- 172.21.15.117:0/3766053195 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc08c107f30 0x7fc08c19bf40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc080011b60 tx=0x7fc080011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:11:09.107 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:07.622+0000 7fc089600700 1 -- 172.21.15.117:0/3766053195 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc080003b20 con 0x7fc08c107f30 2024-06-03T03:11:09.107 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:07.622+0000 7fc089600700 1 -- 172.21.15.117:0/3766053195 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc080003cc0 con 0x7fc08c107f30 2024-06-03T03:11:09.107 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:07.622+0000 7fc093600700 1 -- 172.21.15.117:0/3766053195 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc08c19c710 con 0x7fc08c107f30 2024-06-03T03:11:09.107 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:07.622+0000 7fc089600700 1 -- 172.21.15.117:0/3766053195 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc080024840 con 0x7fc08c107f30 2024-06-03T03:11:09.107 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:07.622+0000 7fc093600700 1 -- 172.21.15.117:0/3766053195 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc08c19f400 con 0x7fc08c107f30 2024-06-03T03:11:09.107 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:07.623+0000 7fc089600700 1 -- 172.21.15.117:0/3766053195 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fc080022020 con 0x7fc08c107f30 2024-06-03T03:11:09.107 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:07.623+0000 7fc093600700 1 -- 172.21.15.117:0/3766053195 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc08c063af0 con 0x7fc08c107f30 2024-06-03T03:11:09.107 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:07.623+0000 7fc089600700 1 --2- 172.21.15.117:0/3766053195 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7fc0740390a0 0x7fc07403b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:11:09.107 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:07.623+0000 7fc089600700 1 -- 172.21.15.117:0/3766053195 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc080011e00 con 0x7fc08c107f30 2024-06-03T03:11:09.108 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:07.624+0000 7fc08be00700 1 --2- 172.21.15.117:0/3766053195 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7fc0740390a0 0x7fc07403b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:11:09.108 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:07.624+0000 7fc08be00700 1 --2- 172.21.15.117:0/3766053195 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7fc0740390a0 0x7fc07403b560 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc078006fd0 tx=0x7fc078006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:11:09.108 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:07.629+0000 7fc089600700 1 -- 172.21.15.117:0/3766053195 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc08004cc90 con 0x7fc08c107f30 2024-06-03T03:11:09.108 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:07.875+0000 7fc093600700 1 -- 172.21.15.117:0/3766053195 --> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fc08c1046e0 con 0x7fc0740390a0 2024-06-03T03:11:09.108 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:07.882+0000 7fc089600700 1 -- 172.21.15.117:0/3766053195 <== mgr.14120 v2:172.21.15.117:6800/2142313590 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fc08c1046e0 con 0x7fc0740390a0 2024-06-03T03:11:09.108 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:07.884+0000 7fc093600700 1 -- 172.21.15.117:0/3766053195 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7fc0740390a0 msgr2=0x7fc07403b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:11:09.108 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:07.884+0000 7fc093600700 1 --2- 172.21.15.117:0/3766053195 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7fc0740390a0 0x7fc07403b560 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc078006fd0 tx=0x7fc078006e40 comp rx=0 tx=0).stop 2024-06-03T03:11:09.108 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:07.884+0000 7fc093600700 1 -- 172.21.15.117:0/3766053195 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc08c107f30 msgr2=0x7fc08c19bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:11:09.108 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:07.884+0000 7fc093600700 1 --2- 172.21.15.117:0/3766053195 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc08c107f30 0x7fc08c19bf40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc080011b60 tx=0x7fc080011c40 comp rx=0 tx=0).stop 2024-06-03T03:11:09.108 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:07.884+0000 7fc093600700 1 -- 172.21.15.117:0/3766053195 shutdown_connections 2024-06-03T03:11:09.108 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:07.884+0000 7fc093600700 1 --2- 172.21.15.117:0/3766053195 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7fc0740390a0 0x7fc07403b560 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:11:09.108 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:07.884+0000 7fc093600700 1 --2- 172.21.15.117:0/3766053195 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc08c107f30 0x7fc08c19bf40 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:11:09.108 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:07.884+0000 7fc093600700 1 -- 172.21.15.117:0/3766053195 >> 172.21.15.117:0/3766053195 conn(0x7fc08c103530 msgr2=0x7fc08c103fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:11:09.109 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:07.885+0000 7fc093600700 1 -- 172.21.15.117:0/3766053195 shutdown_connections 2024-06-03T03:11:09.109 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:07.885+0000 7fc093600700 1 -- 172.21.15.117:0/3766053195 wait complete. 2024-06-03T03:11:09.109 INFO:teuthology.orchestra.run.smithi117.stdout:Deploying unmanaged mgr service... 2024-06-03T03:11:10.820 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-06-03T03:11:10.820 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:09.749+0000 7ff8e3e00700 1 Processor -- start 2024-06-03T03:11:10.820 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:09.749+0000 7ff8e3e00700 1 -- start start 2024-06-03T03:11:10.820 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:09.749+0000 7ff8e3e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8e410a010 0x7ff8e410a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:11:10.821 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:09.749+0000 7ff8e3e00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e410aa00 con 0x7ff8e410a010 2024-06-03T03:11:10.821 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:09.749+0000 7ff8e2a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8e410a010 0x7ff8e410a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:11:10.821 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:09.749+0000 7ff8e2a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8e410a010 0x7ff8e410a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34160/0 (socket says 172.21.15.117:34160) 2024-06-03T03:11:10.821 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:09.749+0000 7ff8e2a00700 1 -- 172.21.15.117:0/3970321451 learned_addr learned my addr 172.21.15.117:0/3970321451 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:11:10.821 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:09.749+0000 7ff8e2a00700 1 -- 172.21.15.117:0/3970321451 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8e410b1c0 con 0x7ff8e410a010 2024-06-03T03:11:10.821 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:09.750+0000 7ff8e2a00700 1 --2- 172.21.15.117:0/3970321451 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8e410a010 0x7ff8e410a430 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff8d40098a0 tx=0x7ff8d4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9cb699d2bdba055 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:11:10.821 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:09.750+0000 7ff8e1600700 1 -- 172.21.15.117:0/3970321451 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8d4004160 con 0x7ff8e410a010 2024-06-03T03:11:10.821 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:09.750+0000 7ff8e1600700 1 -- 172.21.15.117:0/3970321451 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff8d4004300 con 0x7ff8e410a010 2024-06-03T03:11:10.821 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:09.750+0000 7ff8e1600700 1 -- 172.21.15.117:0/3970321451 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8d4011b50 con 0x7ff8e410a010 2024-06-03T03:11:10.821 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:09.751+0000 7ff8e3e00700 1 -- 172.21.15.117:0/3970321451 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8e410a010 msgr2=0x7ff8e410a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:11:10.821 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:09.751+0000 7ff8e3e00700 1 --2- 172.21.15.117:0/3970321451 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8e410a010 0x7ff8e410a430 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff8d40098a0 tx=0x7ff8d4009bb0 comp rx=0 tx=0).stop 2024-06-03T03:11:10.822 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:09.751+0000 7ff8e3e00700 1 -- 172.21.15.117:0/3970321451 shutdown_connections 2024-06-03T03:11:10.822 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:09.751+0000 7ff8e3e00700 1 --2- 172.21.15.117:0/3970321451 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8e410a010 0x7ff8e410a430 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:11:10.822 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:09.751+0000 7ff8e3e00700 1 -- 172.21.15.117:0/3970321451 >> 172.21.15.117:0/3970321451 conn(0x7ff8e407d3d0 msgr2=0x7ff8e407d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:11:10.822 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:09.751+0000 7ff8e3e00700 1 -- 172.21.15.117:0/3970321451 shutdown_connections 2024-06-03T03:11:10.822 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:09.751+0000 7ff8e3e00700 1 -- 172.21.15.117:0/3970321451 wait complete. 2024-06-03T03:11:10.822 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:09.752+0000 7ff8e3e00700 1 Processor -- start 2024-06-03T03:11:10.822 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:09.752+0000 7ff8e3e00700 1 -- start start 2024-06-03T03:11:10.822 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:09.752+0000 7ff8e3e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8e410a010 0x7ff8e419dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:11:10.822 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:09.752+0000 7ff8e3e00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e410e570 con 0x7ff8e410a010 2024-06-03T03:11:10.822 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:09.753+0000 7ff8e2a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8e410a010 0x7ff8e419dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:11:10.822 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:09.753+0000 7ff8e2a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8e410a010 0x7ff8e419dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34170/0 (socket says 172.21.15.117:34170) 2024-06-03T03:11:10.822 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:09.753+0000 7ff8e2a00700 1 -- 172.21.15.117:0/1491080225 learned_addr learned my addr 172.21.15.117:0/1491080225 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:11:10.823 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:09.753+0000 7ff8e2a00700 1 -- 172.21.15.117:0/1491080225 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8e419e520 con 0x7ff8e410a010 2024-06-03T03:11:10.823 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:09.753+0000 7ff8e2a00700 1 --2- 172.21.15.117:0/1491080225 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8e410a010 0x7ff8e419dfe0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff8d4011850 tx=0x7ff8d4011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:11:10.823 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:09.753+0000 7ff8e8a00700 1 -- 172.21.15.117:0/1491080225 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8d4003af0 con 0x7ff8e410a010 2024-06-03T03:11:10.823 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:09.754+0000 7ff8e8a00700 1 -- 172.21.15.117:0/1491080225 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff8d4020070 con 0x7ff8e410a010 2024-06-03T03:11:10.823 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:09.754+0000 7ff8e3e00700 1 -- 172.21.15.117:0/1491080225 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8e419e7b0 con 0x7ff8e410a010 2024-06-03T03:11:10.823 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:09.754+0000 7ff8e8a00700 1 -- 172.21.15.117:0/1491080225 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8d40195a0 con 0x7ff8e410a010 2024-06-03T03:11:10.823 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:09.754+0000 7ff8e3e00700 1 -- 172.21.15.117:0/1491080225 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8e41a14a0 con 0x7ff8e410a010 2024-06-03T03:11:10.823 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:09.755+0000 7ff8e3e00700 1 -- 172.21.15.117:0/1491080225 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8e40695b0 con 0x7ff8e410a010 2024-06-03T03:11:10.823 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:09.755+0000 7ff8e8a00700 1 -- 172.21.15.117:0/1491080225 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7ff8d4019850 con 0x7ff8e410a010 2024-06-03T03:11:10.823 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:09.755+0000 7ff8e8a00700 1 --2- 172.21.15.117:0/1491080225 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7ff8c80390a0 0x7ff8c803b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:11:10.823 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:09.755+0000 7ff8e8a00700 1 -- 172.21.15.117:0/1491080225 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff8d4011e00 con 0x7ff8e410a010 2024-06-03T03:11:10.823 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:09.756+0000 7ff8e2000700 1 --2- 172.21.15.117:0/1491080225 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7ff8c80390a0 0x7ff8c803b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:11:10.824 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:09.756+0000 7ff8e2000700 1 --2- 172.21.15.117:0/1491080225 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7ff8c80390a0 0x7ff8c803b560 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0006fd0 tx=0x7ff8d0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:11:10.824 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:09.761+0000 7ff8e8a00700 1 -- 172.21.15.117:0/1491080225 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff8d400e260 con 0x7ff8e410a010 2024-06-03T03:11:10.824 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:09.998+0000 7ff8e8a00700 1 -- 172.21.15.117:0/1491080225 <== mon.0 v2:172.21.15.117:3300/0 7 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8d402d710 con 0x7ff8e410a010 2024-06-03T03:11:10.824 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:10.003+0000 7ff8e3e00700 1 -- 172.21.15.117:0/1491080225 --> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7ff8e4107bd0 con 0x7ff8c80390a0 2024-06-03T03:11:10.824 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:10.028+0000 7ff8e8a00700 1 -- 172.21.15.117:0/1491080225 <== mgr.14120 v2:172.21.15.117:6800/2142313590 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7ff8e4107bd0 con 0x7ff8c80390a0 2024-06-03T03:11:10.824 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:10.032+0000 7ff8e3e00700 1 -- 172.21.15.117:0/1491080225 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7ff8c80390a0 msgr2=0x7ff8c803b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:11:10.824 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:10.032+0000 7ff8e3e00700 1 --2- 172.21.15.117:0/1491080225 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7ff8c80390a0 0x7ff8c803b560 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0006fd0 tx=0x7ff8d0006e40 comp rx=0 tx=0).stop 2024-06-03T03:11:10.824 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:10.032+0000 7ff8e3e00700 1 -- 172.21.15.117:0/1491080225 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8e410a010 msgr2=0x7ff8e419dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:11:10.824 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:10.032+0000 7ff8e3e00700 1 --2- 172.21.15.117:0/1491080225 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8e410a010 0x7ff8e419dfe0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff8d4011850 tx=0x7ff8d4011c10 comp rx=0 tx=0).stop 2024-06-03T03:11:10.824 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:10.032+0000 7ff8e3e00700 1 -- 172.21.15.117:0/1491080225 shutdown_connections 2024-06-03T03:11:10.824 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:10.032+0000 7ff8e3e00700 1 --2- 172.21.15.117:0/1491080225 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7ff8c80390a0 0x7ff8c803b560 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:11:10.824 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:10.032+0000 7ff8e3e00700 1 --2- 172.21.15.117:0/1491080225 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8e410a010 0x7ff8e419dfe0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:11:10.824 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:10.032+0000 7ff8e3e00700 1 -- 172.21.15.117:0/1491080225 >> 172.21.15.117:0/1491080225 conn(0x7ff8e407d3d0 msgr2=0x7ff8e41074c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:11:10.824 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:10.032+0000 7ff8e3e00700 1 -- 172.21.15.117:0/1491080225 shutdown_connections 2024-06-03T03:11:10.825 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:10.032+0000 7ff8e3e00700 1 -- 172.21.15.117:0/1491080225 wait complete. 2024-06-03T03:11:11.153 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:10 smithi117 ceph-mon[31649]: from='mgr.14120 172.21.15.117:0/175906377' entity='mgr.x' 2024-06-03T03:11:11.154 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:10 smithi117 ceph-mon[31649]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:11:11.154 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:10 smithi117 ceph-mon[31649]: Saving service mgr spec with placement count:2 2024-06-03T03:11:11.154 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:10 smithi117 ceph-mon[31649]: from='mgr.14120 172.21.15.117:0/175906377' entity='mgr.x' 2024-06-03T03:11:11.154 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:10 smithi117 ceph-mon[31649]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:11:11.154 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:10 smithi117 ceph-mon[31649]: from='mgr.14120 172.21.15.117:0/175906377' entity='mgr.x' 2024-06-03T03:11:12.587 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:11.432+0000 7faf71200700 1 Processor -- start 2024-06-03T03:11:12.587 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:11.433+0000 7faf71200700 1 -- start start 2024-06-03T03:11:12.587 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:11.433+0000 7faf71200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf6c07e770 0x7faf6c07cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:11:12.587 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:11.433+0000 7faf71200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7faf6c07d1a0 con 0x7faf6c07e770 2024-06-03T03:11:12.587 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:11.433+0000 7faf69e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf6c07e770 0x7faf6c07cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:11:12.587 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:11.433+0000 7faf69e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf6c07e770 0x7faf6c07cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34184/0 (socket says 172.21.15.117:34184) 2024-06-03T03:11:12.588 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:11.433+0000 7faf69e00700 1 -- 172.21.15.117:0/1565064707 learned_addr learned my addr 172.21.15.117:0/1565064707 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:11:12.588 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:11.434+0000 7faf69e00700 1 -- 172.21.15.117:0/1565064707 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf6c07d320 con 0x7faf6c07e770 2024-06-03T03:11:12.588 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:11.434+0000 7faf69e00700 1 --2- 172.21.15.117:0/1565064707 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf6c07e770 0x7faf6c07cbd0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7faf5c0098a0 tx=0x7faf5c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45998113ab6d9820 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:11:12.588 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:11.434+0000 7faf68a00700 1 -- 172.21.15.117:0/1565064707 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faf5c004160 con 0x7faf6c07e770 2024-06-03T03:11:12.588 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:11.434+0000 7faf68a00700 1 -- 172.21.15.117:0/1565064707 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf5c004300 con 0x7faf6c07e770 2024-06-03T03:11:12.588 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:11.434+0000 7faf68a00700 1 -- 172.21.15.117:0/1565064707 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faf5c011b20 con 0x7faf6c07e770 2024-06-03T03:11:12.588 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:11.435+0000 7faf71200700 1 -- 172.21.15.117:0/1565064707 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf6c07e770 msgr2=0x7faf6c07cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:11:12.588 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:11.435+0000 7faf71200700 1 --2- 172.21.15.117:0/1565064707 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf6c07e770 0x7faf6c07cbd0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7faf5c0098a0 tx=0x7faf5c009bb0 comp rx=0 tx=0).stop 2024-06-03T03:11:12.588 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:11.435+0000 7faf71200700 1 -- 172.21.15.117:0/1565064707 shutdown_connections 2024-06-03T03:11:12.588 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:11.435+0000 7faf71200700 1 --2- 172.21.15.117:0/1565064707 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf6c07e770 0x7faf6c07cbd0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:11:12.588 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:11.435+0000 7faf71200700 1 -- 172.21.15.117:0/1565064707 >> 172.21.15.117:0/1565064707 conn(0x7faf6c103530 msgr2=0x7faf6c105950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:11:12.588 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:11.435+0000 7faf71200700 1 -- 172.21.15.117:0/1565064707 shutdown_connections 2024-06-03T03:11:12.588 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:11.435+0000 7faf71200700 1 -- 172.21.15.117:0/1565064707 wait complete. 2024-06-03T03:11:12.589 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:11.436+0000 7faf71200700 1 Processor -- start 2024-06-03T03:11:12.589 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:11.437+0000 7faf71200700 1 -- start start 2024-06-03T03:11:12.589 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:11.437+0000 7faf71200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf6c07e770 0x7faf6c19e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:11:12.589 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:11.437+0000 7faf71200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7faf6c10e5a0 con 0x7faf6c07e770 2024-06-03T03:11:12.589 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:11.437+0000 7faf69e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf6c07e770 0x7faf6c19e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:11:12.589 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:11.437+0000 7faf69e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf6c07e770 0x7faf6c19e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34192/0 (socket says 172.21.15.117:34192) 2024-06-03T03:11:12.589 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:11.437+0000 7faf69e00700 1 -- 172.21.15.117:0/3458169729 learned_addr learned my addr 172.21.15.117:0/3458169729 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:11:12.589 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:11.438+0000 7faf69e00700 1 -- 172.21.15.117:0/3458169729 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf6c19e5e0 con 0x7faf6c07e770 2024-06-03T03:11:12.589 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:11.438+0000 7faf69e00700 1 --2- 172.21.15.117:0/3458169729 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf6c07e770 0x7faf6c19e0a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7faf5c0117e0 tx=0x7faf5c011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:11:12.589 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:11.438+0000 7faf62a00700 1 -- 172.21.15.117:0/3458169729 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faf5c003bf0 con 0x7faf6c07e770 2024-06-03T03:11:12.589 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:11.438+0000 7faf62a00700 1 -- 172.21.15.117:0/3458169729 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf5c003d90 con 0x7faf6c07e770 2024-06-03T03:11:12.589 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:11.438+0000 7faf71200700 1 -- 172.21.15.117:0/3458169729 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf6c19e870 con 0x7faf6c07e770 2024-06-03T03:11:12.589 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:11.439+0000 7faf62a00700 1 -- 172.21.15.117:0/3458169729 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faf5c024800 con 0x7faf6c07e770 2024-06-03T03:11:12.590 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:11.439+0000 7faf71200700 1 -- 172.21.15.117:0/3458169729 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf6c19ecd0 con 0x7faf6c07e770 2024-06-03T03:11:12.590 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:11.440+0000 7faf71200700 1 -- 172.21.15.117:0/3458169729 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf6c063af0 con 0x7faf6c07e770 2024-06-03T03:11:12.590 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:11.440+0000 7faf62a00700 1 -- 172.21.15.117:0/3458169729 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7faf5c022020 con 0x7faf6c07e770 2024-06-03T03:11:12.590 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:11.441+0000 7faf62a00700 1 --2- 172.21.15.117:0/3458169729 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7faf500390a0 0x7faf5003b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:11:12.590 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:11.441+0000 7faf62a00700 1 -- 172.21.15.117:0/3458169729 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7faf5c04d2d0 con 0x7faf6c07e770 2024-06-03T03:11:12.590 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:11.441+0000 7faf69400700 1 --2- 172.21.15.117:0/3458169729 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7faf500390a0 0x7faf5003b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:11:12.590 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:11.442+0000 7faf69400700 1 --2- 172.21.15.117:0/3458169729 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7faf500390a0 0x7faf5003b560 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7faf58006fd0 tx=0x7faf58006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:11:12.590 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:11.446+0000 7faf62a00700 1 -- 172.21.15.117:0/3458169729 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7faf5c02ad60 con 0x7faf6c07e770 2024-06-03T03:11:12.590 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:11.675+0000 7faf71200700 1 -- 172.21.15.117:0/3458169729 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7faf6c1a1560 con 0x7faf6c07e770 2024-06-03T03:11:12.590 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:11.679+0000 7faf62a00700 1 -- 172.21.15.117:0/3458169729 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7faf5c01a430 con 0x7faf6c07e770 2024-06-03T03:11:12.590 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:11.682+0000 7faf71200700 1 -- 172.21.15.117:0/3458169729 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7faf500390a0 msgr2=0x7faf5003b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:11:12.590 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:11.682+0000 7faf71200700 1 --2- 172.21.15.117:0/3458169729 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7faf500390a0 0x7faf5003b560 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7faf58006fd0 tx=0x7faf58006e40 comp rx=0 tx=0).stop 2024-06-03T03:11:12.590 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:11.682+0000 7faf71200700 1 -- 172.21.15.117:0/3458169729 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf6c07e770 msgr2=0x7faf6c19e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:11:12.591 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:11.682+0000 7faf71200700 1 --2- 172.21.15.117:0/3458169729 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf6c07e770 0x7faf6c19e0a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7faf5c0117e0 tx=0x7faf5c011ba0 comp rx=0 tx=0).stop 2024-06-03T03:11:12.591 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:11.682+0000 7faf71200700 1 -- 172.21.15.117:0/3458169729 shutdown_connections 2024-06-03T03:11:12.591 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:11.682+0000 7faf71200700 1 --2- 172.21.15.117:0/3458169729 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7faf500390a0 0x7faf5003b560 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:11:12.591 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:11.682+0000 7faf71200700 1 --2- 172.21.15.117:0/3458169729 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf6c07e770 0x7faf6c19e0a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:11:12.591 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:11.682+0000 7faf71200700 1 -- 172.21.15.117:0/3458169729 >> 172.21.15.117:0/3458169729 conn(0x7faf6c103530 msgr2=0x7faf6c103fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:11:12.591 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:11.682+0000 7faf71200700 1 -- 172.21.15.117:0/3458169729 shutdown_connections 2024-06-03T03:11:12.591 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:11.682+0000 7faf71200700 1 -- 172.21.15.117:0/3458169729 wait complete. 2024-06-03T03:11:12.857 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:12 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/3458169729' entity='client.admin' 2024-06-03T03:11:13.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:13 smithi117 ceph-mon[31649]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:11:13.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:13 smithi117 ceph-mon[31649]: from='mgr.14120 172.21.15.117:0/175906377' entity='mgr.x' 2024-06-03T03:11:14.404 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:13.473+0000 7fae06400700 1 Processor -- start 2024-06-03T03:11:14.405 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:13.474+0000 7fae06400700 1 -- start start 2024-06-03T03:11:14.405 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:13.474+0000 7fae06400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae0007e720 0x7fae0007cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:11:14.405 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:13.474+0000 7fae06400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fae0007d1a0 con 0x7fae0007e720 2024-06-03T03:11:14.405 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:13.475+0000 7fadff400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae0007e720 0x7fae0007cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:11:14.405 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:13.475+0000 7fadff400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae0007e720 0x7fae0007cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34196/0 (socket says 172.21.15.117:34196) 2024-06-03T03:11:14.405 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:13.475+0000 7fadff400700 1 -- 172.21.15.117:0/1068290213 learned_addr learned my addr 172.21.15.117:0/1068290213 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:11:14.405 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:13.475+0000 7fadff400700 1 -- 172.21.15.117:0/1068290213 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae0007d320 con 0x7fae0007e720 2024-06-03T03:11:14.405 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:13.476+0000 7fadff400700 1 --2- 172.21.15.117:0/1068290213 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae0007e720 0x7fae0007cbd0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fadf00098a0 tx=0x7fadf0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6b04955a47f4f82 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:11:14.405 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:13.476+0000 7fadfe000700 1 -- 172.21.15.117:0/1068290213 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fadf0004160 con 0x7fae0007e720 2024-06-03T03:11:14.405 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:13.476+0000 7fadfe000700 1 -- 172.21.15.117:0/1068290213 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadf0004300 con 0x7fae0007e720 2024-06-03T03:11:14.405 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:13.476+0000 7fadfe000700 1 -- 172.21.15.117:0/1068290213 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fadf0011b20 con 0x7fae0007e720 2024-06-03T03:11:14.405 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:13.477+0000 7fae06400700 1 -- 172.21.15.117:0/1068290213 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae0007e720 msgr2=0x7fae0007cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:11:14.406 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:13.477+0000 7fae06400700 1 --2- 172.21.15.117:0/1068290213 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae0007e720 0x7fae0007cbd0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fadf00098a0 tx=0x7fadf0009bb0 comp rx=0 tx=0).stop 2024-06-03T03:11:14.406 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:13.477+0000 7fae06400700 1 -- 172.21.15.117:0/1068290213 shutdown_connections 2024-06-03T03:11:14.406 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:13.477+0000 7fae06400700 1 --2- 172.21.15.117:0/1068290213 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae0007e720 0x7fae0007cbd0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:11:14.406 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:13.477+0000 7fae06400700 1 -- 172.21.15.117:0/1068290213 >> 172.21.15.117:0/1068290213 conn(0x7fae00103530 msgr2=0x7fae00105950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:11:14.406 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:13.477+0000 7fae06400700 1 -- 172.21.15.117:0/1068290213 shutdown_connections 2024-06-03T03:11:14.406 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:13.477+0000 7fae06400700 1 -- 172.21.15.117:0/1068290213 wait complete. 2024-06-03T03:11:14.406 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:13.478+0000 7fae06400700 1 Processor -- start 2024-06-03T03:11:14.406 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:13.478+0000 7fae06400700 1 -- start start 2024-06-03T03:11:14.406 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:13.479+0000 7fae06400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae0007e720 0x7fae0019e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:11:14.406 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:13.479+0000 7fae06400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fae0010e590 con 0x7fae0007e720 2024-06-03T03:11:14.406 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:13.479+0000 7fadff400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae0007e720 0x7fae0019e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:11:14.406 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:13.479+0000 7fadff400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae0007e720 0x7fae0019e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34204/0 (socket says 172.21.15.117:34204) 2024-06-03T03:11:14.406 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:13.479+0000 7fadff400700 1 -- 172.21.15.117:0/3942909596 learned_addr learned my addr 172.21.15.117:0/3942909596 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:11:14.407 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:13.479+0000 7fadff400700 1 -- 172.21.15.117:0/3942909596 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae0019e5b0 con 0x7fae0007e720 2024-06-03T03:11:14.407 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:13.480+0000 7fadff400700 1 --2- 172.21.15.117:0/3942909596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae0007e720 0x7fae0019e070 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fadf00117e0 tx=0x7fadf0011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:11:14.407 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:13.480+0000 7fadf7e00700 1 -- 172.21.15.117:0/3942909596 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fadf0003ba0 con 0x7fae0007e720 2024-06-03T03:11:14.407 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:13.480+0000 7fadf7e00700 1 -- 172.21.15.117:0/3942909596 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadf001a910 con 0x7fae0007e720 2024-06-03T03:11:14.407 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:13.480+0000 7fadf7e00700 1 -- 172.21.15.117:0/3942909596 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fadf00248f0 con 0x7fae0007e720 2024-06-03T03:11:14.407 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:13.480+0000 7fae06400700 1 -- 172.21.15.117:0/3942909596 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae0019e840 con 0x7fae0007e720 2024-06-03T03:11:14.407 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:13.480+0000 7fae06400700 1 -- 172.21.15.117:0/3942909596 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae0019eca0 con 0x7fae0007e720 2024-06-03T03:11:14.407 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:13.482+0000 7fae06400700 1 -- 172.21.15.117:0/3942909596 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae00063af0 con 0x7fae0007e720 2024-06-03T03:11:14.407 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:13.482+0000 7fadf7e00700 1 -- 172.21.15.117:0/3942909596 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fadf0022020 con 0x7fae0007e720 2024-06-03T03:11:14.407 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:13.482+0000 7fadf7e00700 1 --2- 172.21.15.117:0/3942909596 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7fade40390a0 0x7fade403b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:11:14.407 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:13.482+0000 7fadf7e00700 1 -- 172.21.15.117:0/3942909596 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fadf004d580 con 0x7fae0007e720 2024-06-03T03:11:14.407 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:13.483+0000 7fadfea00700 1 --2- 172.21.15.117:0/3942909596 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7fade40390a0 0x7fade403b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:11:14.407 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:13.483+0000 7fadfea00700 1 --2- 172.21.15.117:0/3942909596 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7fade40390a0 0x7fade403b560 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fadec006fd0 tx=0x7fadec006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:11:14.408 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:13.488+0000 7fadf7e00700 1 -- 172.21.15.117:0/3942909596 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fadf000f2e0 con 0x7fae0007e720 2024-06-03T03:11:14.408 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:13.715+0000 7fae06400700 1 -- 172.21.15.117:0/3942909596 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7fae001a1730 con 0x7fae0007e720 2024-06-03T03:11:14.408 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:13.721+0000 7fadf7e00700 1 -- 172.21.15.117:0/3942909596 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7fadf004ced0 con 0x7fae0007e720 2024-06-03T03:11:14.408 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:13.725+0000 7fae06400700 1 -- 172.21.15.117:0/3942909596 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7fade40390a0 msgr2=0x7fade403b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:11:14.408 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:13.725+0000 7fae06400700 1 --2- 172.21.15.117:0/3942909596 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7fade40390a0 0x7fade403b560 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fadec006fd0 tx=0x7fadec006e40 comp rx=0 tx=0).stop 2024-06-03T03:11:14.408 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:13.725+0000 7fae06400700 1 -- 172.21.15.117:0/3942909596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae0007e720 msgr2=0x7fae0019e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:11:14.408 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:13.725+0000 7fae06400700 1 --2- 172.21.15.117:0/3942909596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae0007e720 0x7fae0019e070 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fadf00117e0 tx=0x7fadf0011ba0 comp rx=0 tx=0).stop 2024-06-03T03:11:14.408 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:13.725+0000 7fae06400700 1 -- 172.21.15.117:0/3942909596 shutdown_connections 2024-06-03T03:11:14.408 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:13.725+0000 7fae06400700 1 --2- 172.21.15.117:0/3942909596 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7fade40390a0 0x7fade403b560 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:11:14.408 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:13.725+0000 7fae06400700 1 --2- 172.21.15.117:0/3942909596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae0007e720 0x7fae0019e070 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:11:14.408 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:13.725+0000 7fae06400700 1 -- 172.21.15.117:0/3942909596 >> 172.21.15.117:0/3942909596 conn(0x7fae00103530 msgr2=0x7fae00103fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:11:14.408 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:13.725+0000 7fae06400700 1 -- 172.21.15.117:0/3942909596 shutdown_connections 2024-06-03T03:11:14.408 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:13.725+0000 7fae06400700 1 -- 172.21.15.117:0/3942909596 wait complete. 2024-06-03T03:11:14.409 INFO:teuthology.orchestra.run.smithi117.stdout:Enabling the dashboard module... 2024-06-03T03:11:14.982 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:14 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/3942909596' entity='client.admin' 2024-06-03T03:11:16.033 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:15 smithi117 ceph-mon[31649]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:11:16.033 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:15 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/2859585088' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-06-03T03:11:16.033 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:11:15 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: ignoring --setuser ceph since I am not root 2024-06-03T03:11:16.033 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:11:15 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: ignoring --setgroup ceph since I am not root 2024-06-03T03:11:16.396 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:11:16 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:11:16.031+0000 7fbc41c36200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-03T03:11:16.396 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:11:16 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:11:16.272+0000 7fbc41c36200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-03T03:11:16.411 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:15.029+0000 7f8e84a00700 1 Processor -- start 2024-06-03T03:11:16.411 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:15.029+0000 7f8e84a00700 1 -- start start 2024-06-03T03:11:16.411 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:15.029+0000 7f8e84a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e80107d80 0x7f8e801081a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:11:16.411 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:15.029+0000 7f8e84a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8e80108770 con 0x7f8e80107d80 2024-06-03T03:11:16.411 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:15.030+0000 7f8e7f400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e80107d80 0x7f8e801081a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:11:16.411 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:15.030+0000 7f8e7f400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e80107d80 0x7f8e801081a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34216/0 (socket says 172.21.15.117:34216) 2024-06-03T03:11:16.411 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:15.030+0000 7f8e7f400700 1 -- 172.21.15.117:0/880464399 learned_addr learned my addr 172.21.15.117:0/880464399 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:11:16.411 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:15.030+0000 7f8e7f400700 1 -- 172.21.15.117:0/880464399 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e80108f80 con 0x7f8e80107d80 2024-06-03T03:11:16.412 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:15.031+0000 7f8e7f400700 1 --2- 172.21.15.117:0/880464399 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e80107d80 0x7f8e801081a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8e700098a0 tx=0x7f8e70009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c91f327019d7432a server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:11:16.412 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:15.031+0000 7f8e7e000700 1 -- 172.21.15.117:0/880464399 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8e70004160 con 0x7f8e80107d80 2024-06-03T03:11:16.412 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:15.031+0000 7f8e7e000700 1 -- 172.21.15.117:0/880464399 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e70004300 con 0x7f8e80107d80 2024-06-03T03:11:16.412 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:15.031+0000 7f8e7e000700 1 -- 172.21.15.117:0/880464399 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8e70011b20 con 0x7f8e80107d80 2024-06-03T03:11:16.412 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:15.032+0000 7f8e84a00700 1 -- 172.21.15.117:0/880464399 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e80107d80 msgr2=0x7f8e801081a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:11:16.412 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:15.032+0000 7f8e84a00700 1 --2- 172.21.15.117:0/880464399 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e80107d80 0x7f8e801081a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8e700098a0 tx=0x7f8e70009bb0 comp rx=0 tx=0).stop 2024-06-03T03:11:16.412 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:15.032+0000 7f8e84a00700 1 -- 172.21.15.117:0/880464399 shutdown_connections 2024-06-03T03:11:16.412 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:15.032+0000 7f8e84a00700 1 --2- 172.21.15.117:0/880464399 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e80107d80 0x7f8e801081a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:11:16.412 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:15.032+0000 7f8e84a00700 1 -- 172.21.15.117:0/880464399 >> 172.21.15.117:0/880464399 conn(0x7f8e80103320 msgr2=0x7f8e80105760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:11:16.412 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:15.032+0000 7f8e84a00700 1 -- 172.21.15.117:0/880464399 shutdown_connections 2024-06-03T03:11:16.412 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:15.032+0000 7f8e84a00700 1 -- 172.21.15.117:0/880464399 wait complete. 2024-06-03T03:11:16.412 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:15.033+0000 7f8e84a00700 1 Processor -- start 2024-06-03T03:11:16.412 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:15.033+0000 7f8e84a00700 1 -- start start 2024-06-03T03:11:16.413 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:15.033+0000 7f8e84a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e80107d80 0x7f8e8019bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:11:16.413 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:15.033+0000 7f8e84a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8e8010c330 con 0x7f8e80107d80 2024-06-03T03:11:16.413 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:15.034+0000 7f8e7f400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e80107d80 0x7f8e8019bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:11:16.413 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:15.034+0000 7f8e7f400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e80107d80 0x7f8e8019bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34218/0 (socket says 172.21.15.117:34218) 2024-06-03T03:11:16.413 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:15.034+0000 7f8e7f400700 1 -- 172.21.15.117:0/2859585088 learned_addr learned my addr 172.21.15.117:0/2859585088 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:11:16.413 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:15.034+0000 7f8e7f400700 1 -- 172.21.15.117:0/2859585088 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e8019c2d0 con 0x7f8e80107d80 2024-06-03T03:11:16.413 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:15.034+0000 7f8e7f400700 1 --2- 172.21.15.117:0/2859585088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e80107d80 0x7f8e8019bd90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8e70000c00 tx=0x7f8e70011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:11:16.413 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:15.034+0000 7f8e77e00700 1 -- 172.21.15.117:0/2859585088 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8e70003b60 con 0x7f8e80107d80 2024-06-03T03:11:16.413 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:15.035+0000 7f8e77e00700 1 -- 172.21.15.117:0/2859585088 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e7001a910 con 0x7f8e80107d80 2024-06-03T03:11:16.413 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:15.035+0000 7f8e84a00700 1 -- 172.21.15.117:0/2859585088 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e8019c560 con 0x7f8e80107d80 2024-06-03T03:11:16.413 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:15.035+0000 7f8e77e00700 1 -- 172.21.15.117:0/2859585088 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8e70024930 con 0x7f8e80107d80 2024-06-03T03:11:16.413 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:15.035+0000 7f8e84a00700 1 -- 172.21.15.117:0/2859585088 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e8019f210 con 0x7f8e80107d80 2024-06-03T03:11:16.413 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:15.036+0000 7f8e77e00700 1 -- 172.21.15.117:0/2859585088 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f8e70022020 con 0x7f8e80107d80 2024-06-03T03:11:16.413 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:15.036+0000 7f8e84a00700 1 -- 172.21.15.117:0/2859585088 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e800695b0 con 0x7f8e80107d80 2024-06-03T03:11:16.414 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:15.036+0000 7f8e77e00700 1 --2- 172.21.15.117:0/2859585088 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7f8e64039050 0x7f8e6403b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:11:16.414 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:15.036+0000 7f8e77e00700 1 -- 172.21.15.117:0/2859585088 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f8e7004d280 con 0x7f8e80107d80 2024-06-03T03:11:16.414 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:15.037+0000 7f8e7ea00700 1 --2- 172.21.15.117:0/2859585088 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7f8e64039050 0x7f8e6403b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:11:16.414 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:15.037+0000 7f8e7ea00700 1 --2- 172.21.15.117:0/2859585088 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7f8e64039050 0x7f8e6403b510 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c006fd0 tx=0x7f8e6c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:11:16.414 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:15.042+0000 7f8e77e00700 1 -- 172.21.15.117:0/2859585088 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8e7001ab70 con 0x7f8e80107d80 2024-06-03T03:11:16.414 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:15.344+0000 7f8e84a00700 1 -- 172.21.15.117:0/2859585088 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f8e8019f710 con 0x7f8e80107d80 2024-06-03T03:11:16.414 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:15.729+0000 7f8e77e00700 1 -- 172.21.15.117:0/2859585088 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f8e7002d3f0 con 0x7f8e80107d80 2024-06-03T03:11:16.414 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:15.732+0000 7f8e77e00700 1 -- 172.21.15.117:0/2859585088 <== mon.0 v2:172.21.15.117:3300/0 8 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f8e7001a430 con 0x7f8e80107d80 2024-06-03T03:11:16.415 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:15.733+0000 7f8e84a00700 1 -- 172.21.15.117:0/2859585088 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7f8e64039050 msgr2=0x7f8e6403b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:11:16.415 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:15.733+0000 7f8e84a00700 1 --2- 172.21.15.117:0/2859585088 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7f8e64039050 0x7f8e6403b510 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c006fd0 tx=0x7f8e6c006e40 comp rx=0 tx=0).stop 2024-06-03T03:11:16.415 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:15.733+0000 7f8e84a00700 1 -- 172.21.15.117:0/2859585088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e80107d80 msgr2=0x7f8e8019bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:11:16.415 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:15.733+0000 7f8e84a00700 1 --2- 172.21.15.117:0/2859585088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e80107d80 0x7f8e8019bd90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8e70000c00 tx=0x7f8e70011c10 comp rx=0 tx=0).stop 2024-06-03T03:11:16.415 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:15.733+0000 7f8e84a00700 1 -- 172.21.15.117:0/2859585088 shutdown_connections 2024-06-03T03:11:16.415 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:15.733+0000 7f8e84a00700 1 --2- 172.21.15.117:0/2859585088 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7f8e64039050 0x7f8e6403b510 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:11:16.415 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:15.733+0000 7f8e84a00700 1 --2- 172.21.15.117:0/2859585088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e80107d80 0x7f8e8019bd90 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:11:16.415 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:15.733+0000 7f8e84a00700 1 -- 172.21.15.117:0/2859585088 >> 172.21.15.117:0/2859585088 conn(0x7f8e80103320 msgr2=0x7f8e80103e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:11:16.415 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:15.734+0000 7f8e84a00700 1 -- 172.21.15.117:0/2859585088 shutdown_connections 2024-06-03T03:11:16.415 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:15.734+0000 7f8e84a00700 1 -- 172.21.15.117:0/2859585088 wait complete. 2024-06-03T03:11:16.665 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:11:16 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:11:16.404+0000 7fbc41c36200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-03T03:11:16.934 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:16 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/2859585088' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-06-03T03:11:16.934 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:16 smithi117 ceph-mon[31649]: mgrmap e9: x(active, since 25s) 2024-06-03T03:11:16.934 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:11:16 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:11:16.815+0000 7fbc41c36200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-03T03:11:17.196 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:11:16 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:11:16.932+0000 7fbc41c36200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-03T03:11:17.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:17 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/642713605' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-06-03T03:11:17.993 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout { 2024-06-03T03:11:17.993 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-06-03T03:11:17.993 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "available": true, 2024-06-03T03:11:17.994 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-06-03T03:11:17.994 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-06-03T03:11:17.994 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout } 2024-06-03T03:11:17.994 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:17.007+0000 7fc4d9200700 1 Processor -- start 2024-06-03T03:11:17.994 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:17.007+0000 7fc4d9200700 1 -- start start 2024-06-03T03:11:17.994 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:17.007+0000 7fc4d9200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4d407de80 0x7fc4d407c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:11:17.994 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:17.008+0000 7fc4d9200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d407c8b0 con 0x7fc4d407de80 2024-06-03T03:11:17.994 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:17.008+0000 7fc4d3400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4d407de80 0x7fc4d407c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:11:17.994 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:17.008+0000 7fc4d3400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4d407de80 0x7fc4d407c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:42550/0 (socket says 172.21.15.117:42550) 2024-06-03T03:11:17.994 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:17.008+0000 7fc4d3400700 1 -- 172.21.15.117:0/3450458185 learned_addr learned my addr 172.21.15.117:0/3450458185 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:11:17.994 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:17.008+0000 7fc4d3400700 1 -- 172.21.15.117:0/3450458185 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4d407ca30 con 0x7fc4d407de80 2024-06-03T03:11:17.995 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:17.009+0000 7fc4d3400700 1 --2- 172.21.15.117:0/3450458185 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4d407de80 0x7fc4d407c2e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc4c40098a0 tx=0x7fc4c4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fb65d9e1c9f9de9 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:11:17.995 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:17.009+0000 7fc4d2000700 1 -- 172.21.15.117:0/3450458185 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4c4004160 con 0x7fc4d407de80 2024-06-03T03:11:17.995 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:17.009+0000 7fc4d2000700 1 -- 172.21.15.117:0/3450458185 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4c4004300 con 0x7fc4d407de80 2024-06-03T03:11:17.995 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:17.009+0000 7fc4d2000700 1 -- 172.21.15.117:0/3450458185 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4c4011b20 con 0x7fc4d407de80 2024-06-03T03:11:17.995 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:17.009+0000 7fc4d9200700 1 -- 172.21.15.117:0/3450458185 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4d407de80 msgr2=0x7fc4d407c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:11:17.995 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:17.009+0000 7fc4d9200700 1 --2- 172.21.15.117:0/3450458185 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4d407de80 0x7fc4d407c2e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc4c40098a0 tx=0x7fc4c4009bb0 comp rx=0 tx=0).stop 2024-06-03T03:11:17.995 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:17.010+0000 7fc4d9200700 1 -- 172.21.15.117:0/3450458185 shutdown_connections 2024-06-03T03:11:17.995 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:17.010+0000 7fc4d9200700 1 --2- 172.21.15.117:0/3450458185 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4d407de80 0x7fc4d407c2e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:11:17.995 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:17.010+0000 7fc4d9200700 1 -- 172.21.15.117:0/3450458185 >> 172.21.15.117:0/3450458185 conn(0x7fc4d41032e0 msgr2=0x7fc4d4105720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:11:17.995 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:17.010+0000 7fc4d9200700 1 -- 172.21.15.117:0/3450458185 shutdown_connections 2024-06-03T03:11:17.995 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:17.010+0000 7fc4d9200700 1 -- 172.21.15.117:0/3450458185 wait complete. 2024-06-03T03:11:17.995 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:17.011+0000 7fc4d9200700 1 Processor -- start 2024-06-03T03:11:17.995 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:17.011+0000 7fc4d9200700 1 -- start start 2024-06-03T03:11:17.995 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:17.012+0000 7fc4d9200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4d407de80 0x7fc4d419dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:11:17.996 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:17.012+0000 7fc4d9200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d410e3c0 con 0x7fc4d407de80 2024-06-03T03:11:17.996 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:17.012+0000 7fc4d3400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4d407de80 0x7fc4d419dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:11:17.996 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:17.012+0000 7fc4d3400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4d407de80 0x7fc4d419dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:42556/0 (socket says 172.21.15.117:42556) 2024-06-03T03:11:17.996 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:17.012+0000 7fc4d3400700 1 -- 172.21.15.117:0/642713605 learned_addr learned my addr 172.21.15.117:0/642713605 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:11:17.996 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:17.012+0000 7fc4d3400700 1 -- 172.21.15.117:0/642713605 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4d419e420 con 0x7fc4d407de80 2024-06-03T03:11:17.996 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:17.013+0000 7fc4d3400700 1 --2- 172.21.15.117:0/642713605 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4d407de80 0x7fc4d419dee0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc4c4011b60 tx=0x7fc4c4011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:11:17.996 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:17.013+0000 7fc4cbe00700 1 -- 172.21.15.117:0/642713605 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4c4003b60 con 0x7fc4d407de80 2024-06-03T03:11:17.996 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:17.013+0000 7fc4cbe00700 1 -- 172.21.15.117:0/642713605 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4c401a910 con 0x7fc4d407de80 2024-06-03T03:11:17.996 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:17.013+0000 7fc4d9200700 1 -- 172.21.15.117:0/642713605 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4d419e6b0 con 0x7fc4d407de80 2024-06-03T03:11:17.996 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:17.013+0000 7fc4cbe00700 1 -- 172.21.15.117:0/642713605 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4c4024930 con 0x7fc4d407de80 2024-06-03T03:11:17.996 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:17.013+0000 7fc4d9200700 1 -- 172.21.15.117:0/642713605 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4d419eb10 con 0x7fc4d407de80 2024-06-03T03:11:17.996 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:17.014+0000 7fc4d9200700 1 -- 172.21.15.117:0/642713605 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4d40695b0 con 0x7fc4d407de80 2024-06-03T03:11:17.996 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:17.015+0000 7fc4cbe00700 1 -- 172.21.15.117:0/642713605 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7fc4c4022020 con 0x7fc4d407de80 2024-06-03T03:11:17.997 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:17.015+0000 7fc4cbe00700 1 --2- 172.21.15.117:0/642713605 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7fc4b80390a0 0x7fc4b803b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:11:17.997 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:17.015+0000 7fc4cbe00700 1 -- 172.21.15.117:0/642713605 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc4c404d2a0 con 0x7fc4d407de80 2024-06-03T03:11:17.997 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:17.015+0000 7fc4d2a00700 1 -- 172.21.15.117:0/642713605 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7fc4b80390a0 msgr2=0x7fc4b803b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.117:6800/2142313590 2024-06-03T03:11:17.997 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:17.015+0000 7fc4d2a00700 1 --2- 172.21.15.117:0/642713605 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7fc4b80390a0 0x7fc4b803b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-03T03:11:17.997 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:17.021+0000 7fc4cbe00700 1 -- 172.21.15.117:0/642713605 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc4c400b3e0 con 0x7fc4d407de80 2024-06-03T03:11:17.997 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:17.216+0000 7fc4d2a00700 1 -- 172.21.15.117:0/642713605 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7fc4b80390a0 msgr2=0x7fc4b803b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.117:6800/2142313590 2024-06-03T03:11:17.997 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:17.216+0000 7fc4d2a00700 1 --2- 172.21.15.117:0/642713605 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7fc4b80390a0 0x7fc4b803b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-03T03:11:17.997 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:17.298+0000 7fc4d9200700 1 -- 172.21.15.117:0/642713605 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fc4d41a1580 con 0x7fc4d407de80 2024-06-03T03:11:17.997 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:17.299+0000 7fc4cbe00700 1 -- 172.21.15.117:0/642713605 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7fc4c402ddb0 con 0x7fc4d407de80 2024-06-03T03:11:17.997 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:17.302+0000 7fc4d9200700 1 -- 172.21.15.117:0/642713605 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7fc4b80390a0 msgr2=0x7fc4b803b560 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-03T03:11:17.997 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:17.302+0000 7fc4d9200700 1 --2- 172.21.15.117:0/642713605 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7fc4b80390a0 0x7fc4b803b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:11:17.997 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:17.302+0000 7fc4d9200700 1 -- 172.21.15.117:0/642713605 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4d407de80 msgr2=0x7fc4d419dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:11:17.997 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:17.302+0000 7fc4d9200700 1 --2- 172.21.15.117:0/642713605 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4d407de80 0x7fc4d419dee0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc4c4011b60 tx=0x7fc4c4011c40 comp rx=0 tx=0).stop 2024-06-03T03:11:17.997 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:17.302+0000 7fc4d9200700 1 -- 172.21.15.117:0/642713605 shutdown_connections 2024-06-03T03:11:17.998 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:17.302+0000 7fc4d9200700 1 --2- 172.21.15.117:0/642713605 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7fc4b80390a0 0x7fc4b803b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:11:17.998 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:17.302+0000 7fc4d9200700 1 --2- 172.21.15.117:0/642713605 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4d407de80 0x7fc4d419dee0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:11:17.998 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:17.302+0000 7fc4d9200700 1 -- 172.21.15.117:0/642713605 >> 172.21.15.117:0/642713605 conn(0x7fc4d41032e0 msgr2=0x7fc4d4103e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:11:17.998 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:17.303+0000 7fc4d9200700 1 -- 172.21.15.117:0/642713605 shutdown_connections 2024-06-03T03:11:17.998 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:17.303+0000 7fc4d9200700 1 -- 172.21.15.117:0/642713605 wait complete. 2024-06-03T03:11:17.998 INFO:teuthology.orchestra.run.smithi117.stdout:Waiting for the mgr to restart... 2024-06-03T03:11:17.998 INFO:teuthology.orchestra.run.smithi117.stdout:Waiting for mgr epoch 9... 2024-06-03T03:11:18.578 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:11:18 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:11:18.507+0000 7fbc41c36200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-03T03:11:18.946 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:11:18 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:11:18.732+0000 7fbc41c36200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-03T03:11:19.328 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:11:18 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:11:18.947+0000 7fbc41c36200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-03T03:11:19.328 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:11:19 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:11:19.326+0000 7fbc41c36200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-03T03:11:19.696 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:11:19 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:11:19.448+0000 7fbc41c36200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-03T03:11:19.953 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:11:19 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:11:19.699+0000 7fbc41c36200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-03T03:11:19.953 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:11:19 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:11:19.952+0000 7fbc41c36200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-03T03:11:21.196 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:11:20 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:11:20.730+0000 7fbc41c36200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-03T03:11:21.591 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:11:21 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:11:21.277+0000 7fbc41c36200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-03T03:11:21.591 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:11:21 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:11:21.405+0000 7fbc41c36200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-03T03:11:21.946 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:11:21 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:11:21.669+0000 7fbc41c36200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-03T03:11:22.696 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:11:22 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:11:22.319+0000 7fbc41c36200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-03T03:11:22.697 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:11:22 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:11:22.442+0000 7fbc41c36200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-03T03:11:23.068 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:11:22 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:11:22.763+0000 7fbc41c36200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-03T03:11:23.069 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:11:22 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:11:22.941+0000 7fbc41c36200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-03T03:11:23.069 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:11:23 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:11:23.067+0000 7fbc41c36200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-03T03:11:23.446 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:11:23 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:11:23.220+0000 7fbc41c36200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-03T03:11:23.946 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:11:23 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:11:23.588+0000 7fbc41c36200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-03T03:11:24.653 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:11:24 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:11:24.357+0000 7fbc41c36200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-03T03:11:24.946 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:11:24 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:11:24.652+0000 7fbc41c36200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-03T03:11:24.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:24 smithi117 ceph-mon[31649]: Active manager daemon x restarted 2024-06-03T03:11:24.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:24 smithi117 ceph-mon[31649]: Activating manager daemon x 2024-06-03T03:11:24.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:24 smithi117 ceph-mon[31649]: osdmap e3: 0 total, 0 up, 0 in 2024-06-03T03:11:24.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:24 smithi117 ceph-mon[31649]: mgrmap e10: x(active, starting, since 0.0062852s) 2024-06-03T03:11:24.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:24 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-03T03:11:24.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:24 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-03T03:11:24.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:24 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-03T03:11:24.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:24 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-03T03:11:24.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:24 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-03T03:11:24.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:24 smithi117 ceph-mon[31649]: Manager daemon x is now available 2024-06-03T03:11:25.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:25 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:11:25.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:25 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-03T03:11:25.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:25 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-03T03:11:25.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:25 smithi117 ceph-mon[31649]: mgrmap e11: x(active, since 1.00948s) 2024-06-03T03:11:26.427 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout { 2024-06-03T03:11:26.427 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-06-03T03:11:26.428 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "initialized": true 2024-06-03T03:11:26.428 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout } 2024-06-03T03:11:26.428 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:18.577+0000 7f7144a00700 1 Processor -- start 2024-06-03T03:11:26.429 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:18.577+0000 7f7144a00700 1 -- start start 2024-06-03T03:11:26.429 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:18.577+0000 7f7144a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7140108a10 0x7f714010ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:11:26.429 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:18.577+0000 7f7144a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f714007beb0 con 0x7f7140108a10 2024-06-03T03:11:26.429 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:18.578+0000 7f713de00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7140108a10 0x7f714010ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:11:26.429 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:18.578+0000 7f713de00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7140108a10 0x7f714010ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:42562/0 (socket says 172.21.15.117:42562) 2024-06-03T03:11:26.429 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:18.578+0000 7f713de00700 1 -- 172.21.15.117:0/3547610182 learned_addr learned my addr 172.21.15.117:0/3547610182 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:11:26.431 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:18.578+0000 7f713de00700 1 -- 172.21.15.117:0/3547610182 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f714010b340 con 0x7f7140108a10 2024-06-03T03:11:26.431 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:18.579+0000 7f713de00700 1 --2- 172.21.15.117:0/3547610182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7140108a10 0x7f714010ae00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f71300098a0 tx=0x7f7130009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33914df2c2e49970 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:11:26.431 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:18.579+0000 7f713ca00700 1 -- 172.21.15.117:0/3547610182 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7130004160 con 0x7f7140108a10 2024-06-03T03:11:26.431 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:18.579+0000 7f713ca00700 1 -- 172.21.15.117:0/3547610182 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7130004300 con 0x7f7140108a10 2024-06-03T03:11:26.431 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:18.579+0000 7f713ca00700 1 -- 172.21.15.117:0/3547610182 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7130011b20 con 0x7f7140108a10 2024-06-03T03:11:26.431 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:18.580+0000 7f7144a00700 1 -- 172.21.15.117:0/3547610182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7140108a10 msgr2=0x7f714010ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:11:26.431 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:18.580+0000 7f7144a00700 1 --2- 172.21.15.117:0/3547610182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7140108a10 0x7f714010ae00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f71300098a0 tx=0x7f7130009bb0 comp rx=0 tx=0).stop 2024-06-03T03:11:26.431 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:18.580+0000 7f7144a00700 1 -- 172.21.15.117:0/3547610182 shutdown_connections 2024-06-03T03:11:26.432 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:18.580+0000 7f7144a00700 1 --2- 172.21.15.117:0/3547610182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7140108a10 0x7f714010ae00 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:11:26.432 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:18.580+0000 7f7144a00700 1 -- 172.21.15.117:0/3547610182 >> 172.21.15.117:0/3547610182 conn(0x7f71401024e0 msgr2=0x7f7140104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:11:26.432 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:18.580+0000 7f7144a00700 1 -- 172.21.15.117:0/3547610182 shutdown_connections 2024-06-03T03:11:26.432 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:18.580+0000 7f7144a00700 1 -- 172.21.15.117:0/3547610182 wait complete. 2024-06-03T03:11:26.432 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:18.581+0000 7f7144a00700 1 Processor -- start 2024-06-03T03:11:26.432 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:18.581+0000 7f7144a00700 1 -- start start 2024-06-03T03:11:26.434 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:18.582+0000 7f7144a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7140108a10 0x7f7140108290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:11:26.434 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:18.582+0000 7f7144a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f714010c3b0 con 0x7f7140108a10 2024-06-03T03:11:26.434 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:18.582+0000 7f713de00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7140108a10 0x7f7140108290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:11:26.434 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:18.582+0000 7f713de00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7140108a10 0x7f7140108290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:42564/0 (socket says 172.21.15.117:42564) 2024-06-03T03:11:26.434 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:18.582+0000 7f713de00700 1 -- 172.21.15.117:0/2177582694 learned_addr learned my addr 172.21.15.117:0/2177582694 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:11:26.434 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:18.582+0000 7f713de00700 1 -- 172.21.15.117:0/2177582694 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71401087d0 con 0x7f7140108a10 2024-06-03T03:11:26.434 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:18.582+0000 7f713de00700 1 --2- 172.21.15.117:0/2177582694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7140108a10 0x7f7140108290 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f71300117e0 tx=0x7f7130011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:11:26.434 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:18.583+0000 7f7136a00700 1 -- 172.21.15.117:0/2177582694 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7130003ba0 con 0x7f7140108a10 2024-06-03T03:11:26.435 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:18.583+0000 7f7136a00700 1 -- 172.21.15.117:0/2177582694 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f713001a910 con 0x7f7140108a10 2024-06-03T03:11:26.435 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:18.583+0000 7f7144a00700 1 -- 172.21.15.117:0/2177582694 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71401069c0 con 0x7f7140108a10 2024-06-03T03:11:26.435 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:18.583+0000 7f7136a00700 1 -- 172.21.15.117:0/2177582694 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f71300248f0 con 0x7f7140108a10 2024-06-03T03:11:26.435 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:18.583+0000 7f7144a00700 1 -- 172.21.15.117:0/2177582694 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7140106e80 con 0x7f7140108a10 2024-06-03T03:11:26.435 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:18.584+0000 7f7136a00700 1 -- 172.21.15.117:0/2177582694 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f7130022020 con 0x7f7140108a10 2024-06-03T03:11:26.435 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:18.584+0000 7f7136a00700 1 --2- 172.21.15.117:0/2177582694 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7f7124039100 0x7f712403b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:11:26.435 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:18.584+0000 7f7136a00700 1 -- 172.21.15.117:0/2177582694 --> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f712403bcd0 con 0x7f7124039100 2024-06-03T03:11:26.435 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:18.584+0000 7f7136a00700 1 -- 172.21.15.117:0/2177582694 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f713004d6e0 con 0x7f7140108a10 2024-06-03T03:11:26.435 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:18.585+0000 7f713d400700 1 -- 172.21.15.117:0/2177582694 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7f7124039100 msgr2=0x7f712403b5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.117:6800/2142313590 2024-06-03T03:11:26.435 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:18.585+0000 7f713d400700 1 --2- 172.21.15.117:0/2177582694 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7f7124039100 0x7f712403b5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-03T03:11:26.435 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:18.786+0000 7f713d400700 1 -- 172.21.15.117:0/2177582694 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7f7124039100 msgr2=0x7f712403b5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.117:6800/2142313590 2024-06-03T03:11:26.435 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:18.786+0000 7f713d400700 1 --2- 172.21.15.117:0/2177582694 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7f7124039100 0x7f712403b5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-03T03:11:26.438 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:19.187+0000 7f713d400700 1 -- 172.21.15.117:0/2177582694 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7f7124039100 msgr2=0x7f712403b5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.117:6800/2142313590 2024-06-03T03:11:26.438 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:19.187+0000 7f713d400700 1 --2- 172.21.15.117:0/2177582694 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7f7124039100 0x7f712403b5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-06-03T03:11:26.438 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:19.988+0000 7f713d400700 1 -- 172.21.15.117:0/2177582694 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7f7124039100 msgr2=0x7f712403b5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.117:6800/2142313590 2024-06-03T03:11:26.438 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:19.988+0000 7f713d400700 1 --2- 172.21.15.117:0/2177582694 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7f7124039100 0x7f712403b5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-06-03T03:11:26.439 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:21.590+0000 7f713d400700 1 -- 172.21.15.117:0/2177582694 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7f7124039100 msgr2=0x7f712403b5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.117:6800/2142313590 2024-06-03T03:11:26.439 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:21.590+0000 7f713d400700 1 --2- 172.21.15.117:0/2177582694 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7f7124039100 0x7f712403b5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-06-03T03:11:26.439 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:24.659+0000 7f7136a00700 1 -- 172.21.15.117:0/2177582694 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mgrmap(e 10) ==== 46067+0+0 (secure 0 0 0) 0x7f713002d3f0 con 0x7f7140108a10 2024-06-03T03:11:26.439 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:24.660+0000 7f7136a00700 1 -- 172.21.15.117:0/2177582694 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7f7124039100 msgr2=0x7f712403b5c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-03T03:11:26.439 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:24.660+0000 7f7136a00700 1 --2- 172.21.15.117:0/2177582694 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7f7124039100 0x7f712403b5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:11:26.439 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:25.661+0000 7f7136a00700 1 -- 172.21.15.117:0/2177582694 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7f713004f510 con 0x7f7140108a10 2024-06-03T03:11:26.439 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:25.661+0000 7f7136a00700 1 --2- 172.21.15.117:0/2177582694 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f712403cbc0 0x7f712403efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:11:26.439 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:25.662+0000 7f7136a00700 1 -- 172.21.15.117:0/2177582694 --> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f712403bcd0 con 0x7f712403cbc0 2024-06-03T03:11:26.439 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:25.662+0000 7f713d400700 1 --2- 172.21.15.117:0/2177582694 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f712403cbc0 0x7f712403efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:11:26.439 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:25.664+0000 7f713d400700 1 --2- 172.21.15.117:0/2177582694 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f712403cbc0 0x7f712403efb0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f712c003a10 tx=0x7f712c009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:11:26.439 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:25.665+0000 7f7136a00700 1 -- 172.21.15.117:0/2177582694 <== mgr.14152 v2:172.21.15.117:6800/1398160887 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f712403bcd0 con 0x7f712403cbc0 2024-06-03T03:11:26.439 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:25.672+0000 7f7144a00700 1 -- 172.21.15.117:0/2177582694 --> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f71400695b0 con 0x7f712403cbc0 2024-06-03T03:11:26.439 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:25.673+0000 7f7136a00700 1 -- 172.21.15.117:0/2177582694 <== mgr.14152 v2:172.21.15.117:6800/1398160887 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f71400695b0 con 0x7f712403cbc0 2024-06-03T03:11:26.439 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:25.673+0000 7f7144a00700 1 -- 172.21.15.117:0/2177582694 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f712403cbc0 msgr2=0x7f712403efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:11:26.440 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:25.673+0000 7f7144a00700 1 --2- 172.21.15.117:0/2177582694 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f712403cbc0 0x7f712403efb0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f712c003a10 tx=0x7f712c009270 comp rx=0 tx=0).stop 2024-06-03T03:11:26.440 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:25.673+0000 7f7144a00700 1 -- 172.21.15.117:0/2177582694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7140108a10 msgr2=0x7f7140108290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:11:26.440 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:25.673+0000 7f7144a00700 1 --2- 172.21.15.117:0/2177582694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7140108a10 0x7f7140108290 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f71300117e0 tx=0x7f7130011ba0 comp rx=0 tx=0).stop 2024-06-03T03:11:26.440 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:25.674+0000 7f7144a00700 1 -- 172.21.15.117:0/2177582694 shutdown_connections 2024-06-03T03:11:26.440 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:25.674+0000 7f7144a00700 1 --2- 172.21.15.117:0/2177582694 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f712403cbc0 0x7f712403efb0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:11:26.440 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:25.674+0000 7f7144a00700 1 --2- 172.21.15.117:0/2177582694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7140108a10 0x7f7140108290 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:11:26.440 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:25.674+0000 7f7144a00700 1 --2- 172.21.15.117:0/2177582694 >> [v2:172.21.15.117:6800/2142313590,v1:172.21.15.117:6801/2142313590] conn(0x7f7124039100 0x7f712403b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:11:26.440 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:25.674+0000 7f7144a00700 1 -- 172.21.15.117:0/2177582694 >> 172.21.15.117:0/2177582694 conn(0x7f71401024e0 msgr2=0x7f7140102fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:11:26.440 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:25.674+0000 7f7144a00700 1 -- 172.21.15.117:0/2177582694 shutdown_connections 2024-06-03T03:11:26.440 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:25.674+0000 7f7144a00700 1 -- 172.21.15.117:0/2177582694 wait complete. 2024-06-03T03:11:26.440 INFO:teuthology.orchestra.run.smithi117.stdout:mgr epoch 9 is available 2024-06-03T03:11:26.440 INFO:teuthology.orchestra.run.smithi117.stdout:Generating a dashboard self-signed certificate... 2024-06-03T03:11:26.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:26 smithi117 ceph-mon[31649]: [03/Jun/2024:03:11:24] ENGINE Bus STARTING 2024-06-03T03:11:26.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:26 smithi117 ceph-mon[31649]: [03/Jun/2024:03:11:25] ENGINE Serving on http://172.21.15.117:8765 2024-06-03T03:11:26.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:26 smithi117 ceph-mon[31649]: [03/Jun/2024:03:11:25] ENGINE Serving on https://172.21.15.117:7150 2024-06-03T03:11:26.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:26 smithi117 ceph-mon[31649]: [03/Jun/2024:03:11:25] ENGINE Bus STARTED 2024-06-03T03:11:26.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:26 smithi117 ceph-mon[31649]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-06-03T03:11:26.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:26 smithi117 ceph-mon[31649]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-06-03T03:11:28.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:27 smithi117 ceph-mon[31649]: mgrmap e12: x(active, since 2s) 2024-06-03T03:11:28.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:27 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:11:28.598 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-06-03T03:11:28.598 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:27.565+0000 7faed1400700 1 Processor -- start 2024-06-03T03:11:28.599 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:27.565+0000 7faed1400700 1 -- start start 2024-06-03T03:11:28.599 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:27.565+0000 7faed1400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faecc10a170 0x7faecc10a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:11:28.599 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:27.565+0000 7faed1400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7faecc10ab60 con 0x7faecc10a170 2024-06-03T03:11:28.599 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:27.566+0000 7faeca800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faecc10a170 0x7faecc10a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:11:28.599 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:27.566+0000 7faeca800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faecc10a170 0x7faecc10a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39124/0 (socket says 172.21.15.117:39124) 2024-06-03T03:11:28.599 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:27.566+0000 7faeca800700 1 -- 172.21.15.117:0/3683493476 learned_addr learned my addr 172.21.15.117:0/3683493476 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:11:28.599 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:27.566+0000 7faeca800700 1 -- 172.21.15.117:0/3683493476 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faecc10b370 con 0x7faecc10a170 2024-06-03T03:11:28.599 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:27.567+0000 7faeca800700 1 --2- 172.21.15.117:0/3683493476 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faecc10a170 0x7faecc10a590 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7faebc0098a0 tx=0x7faebc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b48e0ad6609dabe6 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:11:28.599 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:27.567+0000 7faec9400700 1 -- 172.21.15.117:0/3683493476 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faebc004160 con 0x7faecc10a170 2024-06-03T03:11:28.599 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:27.567+0000 7faec9400700 1 -- 172.21.15.117:0/3683493476 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faebc004300 con 0x7faecc10a170 2024-06-03T03:11:28.599 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:27.567+0000 7faec9400700 1 -- 172.21.15.117:0/3683493476 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faebc011b20 con 0x7faecc10a170 2024-06-03T03:11:28.600 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:27.568+0000 7faed1400700 1 -- 172.21.15.117:0/3683493476 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faecc10a170 msgr2=0x7faecc10a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:11:28.600 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:27.568+0000 7faed1400700 1 --2- 172.21.15.117:0/3683493476 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faecc10a170 0x7faecc10a590 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7faebc0098a0 tx=0x7faebc009bb0 comp rx=0 tx=0).stop 2024-06-03T03:11:28.600 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:27.568+0000 7faed1400700 1 -- 172.21.15.117:0/3683493476 shutdown_connections 2024-06-03T03:11:28.600 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:27.568+0000 7faed1400700 1 --2- 172.21.15.117:0/3683493476 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faecc10a170 0x7faecc10a590 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:11:28.600 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:27.568+0000 7faed1400700 1 -- 172.21.15.117:0/3683493476 >> 172.21.15.117:0/3683493476 conn(0x7faecc07d6c0 msgr2=0x7faecc07daf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:11:28.600 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:27.568+0000 7faed1400700 1 -- 172.21.15.117:0/3683493476 shutdown_connections 2024-06-03T03:11:28.600 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:27.568+0000 7faed1400700 1 -- 172.21.15.117:0/3683493476 wait complete. 2024-06-03T03:11:28.600 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:27.569+0000 7faed1400700 1 Processor -- start 2024-06-03T03:11:28.600 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:27.570+0000 7faed1400700 1 -- start start 2024-06-03T03:11:28.600 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:27.570+0000 7faed1400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faecc10a170 0x7faecc1a24d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:11:28.600 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:27.570+0000 7faed1400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7faecc10e720 con 0x7faecc10a170 2024-06-03T03:11:28.600 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:27.570+0000 7faeca800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faecc10a170 0x7faecc1a24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:11:28.601 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:27.570+0000 7faeca800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faecc10a170 0x7faecc1a24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39136/0 (socket says 172.21.15.117:39136) 2024-06-03T03:11:28.601 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:27.570+0000 7faeca800700 1 -- 172.21.15.117:0/3011208155 learned_addr learned my addr 172.21.15.117:0/3011208155 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:11:28.601 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:27.571+0000 7faeca800700 1 -- 172.21.15.117:0/3011208155 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faecc1a2a10 con 0x7faecc10a170 2024-06-03T03:11:28.601 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:27.571+0000 7faeca800700 1 --2- 172.21.15.117:0/3011208155 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faecc10a170 0x7faecc1a24d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7faebc0117e0 tx=0x7faebc011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:11:28.601 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:27.571+0000 7faec3400700 1 -- 172.21.15.117:0/3011208155 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faebc003bf0 con 0x7faecc10a170 2024-06-03T03:11:28.601 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:27.571+0000 7faec3400700 1 -- 172.21.15.117:0/3011208155 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faebc01a910 con 0x7faecc10a170 2024-06-03T03:11:28.601 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:27.571+0000 7faed1400700 1 -- 172.21.15.117:0/3011208155 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faecc1a2ca0 con 0x7faecc10a170 2024-06-03T03:11:28.601 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:27.571+0000 7faec3400700 1 -- 172.21.15.117:0/3011208155 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faebc0249a0 con 0x7faecc10a170 2024-06-03T03:11:28.601 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:27.571+0000 7faed1400700 1 -- 172.21.15.117:0/3011208155 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faecc1a3100 con 0x7faecc10a170 2024-06-03T03:11:28.601 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:27.573+0000 7faec3400700 1 -- 172.21.15.117:0/3011208155 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7faebc022020 con 0x7faecc10a170 2024-06-03T03:11:28.601 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:27.573+0000 7faed1400700 1 -- 172.21.15.117:0/3011208155 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faecc063af0 con 0x7faecc10a170 2024-06-03T03:11:28.601 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:27.573+0000 7faec3400700 1 --2- 172.21.15.117:0/3011208155 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7faeb0039050 0x7faeb003b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:11:28.602 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:27.573+0000 7faec3400700 1 -- 172.21.15.117:0/3011208155 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7faebc04d490 con 0x7faecc10a170 2024-06-03T03:11:28.602 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:27.573+0000 7faec9e00700 1 --2- 172.21.15.117:0/3011208155 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7faeb0039050 0x7faeb003b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:11:28.602 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:27.574+0000 7faec9e00700 1 --2- 172.21.15.117:0/3011208155 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7faeb0039050 0x7faeb003b510 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7faeb8006fd0 tx=0x7faeb8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:11:28.602 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:27.579+0000 7faec3400700 1 -- 172.21.15.117:0/3011208155 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7faebc011e20 con 0x7faecc10a170 2024-06-03T03:11:28.602 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:27.812+0000 7faed1400700 1 -- 172.21.15.117:0/3011208155 --> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7faecc107b90 con 0x7faeb0039050 2024-06-03T03:11:28.602 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:27.905+0000 7faec3400700 1 -- 172.21.15.117:0/3011208155 <== mgr.14152 v2:172.21.15.117:6800/1398160887 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7faecc107b90 con 0x7faeb0039050 2024-06-03T03:11:28.602 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:27.909+0000 7faed1400700 1 -- 172.21.15.117:0/3011208155 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7faeb0039050 msgr2=0x7faeb003b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:11:28.602 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:27.909+0000 7faed1400700 1 --2- 172.21.15.117:0/3011208155 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7faeb0039050 0x7faeb003b510 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7faeb8006fd0 tx=0x7faeb8006e40 comp rx=0 tx=0).stop 2024-06-03T03:11:28.602 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:27.909+0000 7faed1400700 1 -- 172.21.15.117:0/3011208155 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faecc10a170 msgr2=0x7faecc1a24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:11:28.602 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:27.909+0000 7faed1400700 1 --2- 172.21.15.117:0/3011208155 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faecc10a170 0x7faecc1a24d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7faebc0117e0 tx=0x7faebc011c10 comp rx=0 tx=0).stop 2024-06-03T03:11:28.602 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:27.909+0000 7faed1400700 1 -- 172.21.15.117:0/3011208155 shutdown_connections 2024-06-03T03:11:28.602 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:27.909+0000 7faed1400700 1 --2- 172.21.15.117:0/3011208155 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7faeb0039050 0x7faeb003b510 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:11:28.603 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:27.909+0000 7faed1400700 1 --2- 172.21.15.117:0/3011208155 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faecc10a170 0x7faecc1a24d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:11:28.603 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:27.909+0000 7faed1400700 1 -- 172.21.15.117:0/3011208155 >> 172.21.15.117:0/3011208155 conn(0x7faecc07d6c0 msgr2=0x7faecc107480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:11:28.603 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:27.909+0000 7faed1400700 1 -- 172.21.15.117:0/3011208155 shutdown_connections 2024-06-03T03:11:28.603 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:27.910+0000 7faed1400700 1 -- 172.21.15.117:0/3011208155 wait complete. 2024-06-03T03:11:28.603 INFO:teuthology.orchestra.run.smithi117.stdout:Creating initial admin user... 2024-06-03T03:11:28.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:28 smithi117 ceph-mon[31649]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:11:28.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:28 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:11:28.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:28 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:11:30.507 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$wqGAN2BwyH4RU7XwZyScseVUaE..Yinfy8hm7h7EPiYznLeotbSQW", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1717384289, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-06-03T03:11:30.507 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:29.307+0000 7fa80e800700 1 Processor -- start 2024-06-03T03:11:30.507 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:29.307+0000 7fa80e800700 1 -- start start 2024-06-03T03:11:30.507 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:29.307+0000 7fa80e800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa808107d80 0x7fa8081081a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:11:30.507 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:29.307+0000 7fa80e800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa808108770 con 0x7fa808107d80 2024-06-03T03:11:30.507 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:29.308+0000 7fa80d400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa808107d80 0x7fa8081081a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:11:30.507 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:29.308+0000 7fa80d400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa808107d80 0x7fa8081081a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39140/0 (socket says 172.21.15.117:39140) 2024-06-03T03:11:30.507 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:29.308+0000 7fa80d400700 1 -- 172.21.15.117:0/60644563 learned_addr learned my addr 172.21.15.117:0/60644563 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:11:30.507 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:29.308+0000 7fa80d400700 1 -- 172.21.15.117:0/60644563 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa808108f80 con 0x7fa808107d80 2024-06-03T03:11:30.507 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:29.309+0000 7fa80d400700 1 --2- 172.21.15.117:0/60644563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa808107d80 0x7fa8081081a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc009e80 tx=0x7fa7fc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0ac946e41d9e3cc server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:11:30.512 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:29.309+0000 7fa807e00700 1 -- 172.21.15.117:0/60644563 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa7fc017070 con 0x7fa808107d80 2024-06-03T03:11:30.512 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:29.309+0000 7fa807e00700 1 -- 172.21.15.117:0/60644563 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7fc00b8c0 con 0x7fa808107d80 2024-06-03T03:11:30.512 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:29.309+0000 7fa807e00700 1 -- 172.21.15.117:0/60644563 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa7fc0045c0 con 0x7fa808107d80 2024-06-03T03:11:30.512 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:29.310+0000 7fa80e800700 1 -- 172.21.15.117:0/60644563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa808107d80 msgr2=0x7fa8081081a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:11:30.512 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:29.310+0000 7fa80e800700 1 --2- 172.21.15.117:0/60644563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa808107d80 0x7fa8081081a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc009e80 tx=0x7fa7fc00b330 comp rx=0 tx=0).stop 2024-06-03T03:11:30.512 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:29.310+0000 7fa80e800700 1 -- 172.21.15.117:0/60644563 shutdown_connections 2024-06-03T03:11:30.512 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:29.310+0000 7fa80e800700 1 --2- 172.21.15.117:0/60644563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa808107d80 0x7fa8081081a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:11:30.513 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:29.310+0000 7fa80e800700 1 -- 172.21.15.117:0/60644563 >> 172.21.15.117:0/60644563 conn(0x7fa808103320 msgr2=0x7fa808105760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:11:30.513 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:29.310+0000 7fa80e800700 1 -- 172.21.15.117:0/60644563 shutdown_connections 2024-06-03T03:11:30.513 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:29.310+0000 7fa80e800700 1 -- 172.21.15.117:0/60644563 wait complete. 2024-06-03T03:11:30.513 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:29.311+0000 7fa80e800700 1 Processor -- start 2024-06-03T03:11:30.513 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:29.311+0000 7fa80e800700 1 -- start start 2024-06-03T03:11:30.513 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:29.312+0000 7fa80e800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa808107d80 0x7fa80807db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:11:30.513 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:29.312+0000 7fa80e800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa80810c330 con 0x7fa808107d80 2024-06-03T03:11:30.513 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:29.312+0000 7fa80d400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa808107d80 0x7fa80807db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:11:30.513 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:29.312+0000 7fa80d400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa808107d80 0x7fa80807db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39144/0 (socket says 172.21.15.117:39144) 2024-06-03T03:11:30.513 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:29.312+0000 7fa80d400700 1 -- 172.21.15.117:0/3741180244 learned_addr learned my addr 172.21.15.117:0/3741180244 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:11:30.513 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:29.312+0000 7fa80d400700 1 -- 172.21.15.117:0/3741180244 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa80807e0b0 con 0x7fa808107d80 2024-06-03T03:11:30.513 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:29.313+0000 7fa80d400700 1 --2- 172.21.15.117:0/3741180244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa808107d80 0x7fa80807db70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc009530 tx=0x7fa7fc003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:11:30.514 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:29.313+0000 7fa806000700 1 -- 172.21.15.117:0/3741180244 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa7fc020020 con 0x7fa808107d80 2024-06-03T03:11:30.514 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:29.313+0000 7fa806000700 1 -- 172.21.15.117:0/3741180244 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7fc00bba0 con 0x7fa808107d80 2024-06-03T03:11:30.514 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:29.313+0000 7fa806000700 1 -- 172.21.15.117:0/3741180244 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa7fc017040 con 0x7fa808107d80 2024-06-03T03:11:30.514 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:29.313+0000 7fa80e800700 1 -- 172.21.15.117:0/3741180244 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa80807c2a0 con 0x7fa808107d80 2024-06-03T03:11:30.514 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:29.313+0000 7fa80e800700 1 -- 172.21.15.117:0/3741180244 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa80807c7e0 con 0x7fa808107d80 2024-06-03T03:11:30.514 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:29.314+0000 7fa80e800700 1 -- 172.21.15.117:0/3741180244 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8080695b0 con 0x7fa808107d80 2024-06-03T03:11:30.514 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:29.314+0000 7fa806000700 1 -- 172.21.15.117:0/3741180244 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7fa7fc023020 con 0x7fa808107d80 2024-06-03T03:11:30.514 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:29.315+0000 7fa806000700 1 --2- 172.21.15.117:0/3741180244 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fa7f00390a0 0x7fa7f003b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:11:30.514 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:29.315+0000 7fa806000700 1 -- 172.21.15.117:0/3741180244 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa7fc00f050 con 0x7fa808107d80 2024-06-03T03:11:30.514 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:29.315+0000 7fa80ca00700 1 --2- 172.21.15.117:0/3741180244 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fa7f00390a0 0x7fa7f003b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:11:30.514 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:29.316+0000 7fa80ca00700 1 --2- 172.21.15.117:0/3741180244 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fa7f00390a0 0x7fa7f003b560 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa7f4006fd0 tx=0x7fa7f4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:11:30.514 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:29.321+0000 7fa806000700 1 -- 172.21.15.117:0/3741180244 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa7fc02a020 con 0x7fa808107d80 2024-06-03T03:11:30.514 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:29.559+0000 7fa80e800700 1 -- 172.21.15.117:0/3741180244 --> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7fa80807cf30 con 0x7fa7f00390a0 2024-06-03T03:11:30.515 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:29.815+0000 7fa806000700 1 -- 172.21.15.117:0/3741180244 <== mgr.14152 v2:172.21.15.117:6800/1398160887 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7fa80807cf30 con 0x7fa7f00390a0 2024-06-03T03:11:30.515 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:29.818+0000 7fa80e800700 1 -- 172.21.15.117:0/3741180244 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fa7f00390a0 msgr2=0x7fa7f003b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:11:30.515 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:29.818+0000 7fa80e800700 1 --2- 172.21.15.117:0/3741180244 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fa7f00390a0 0x7fa7f003b560 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa7f4006fd0 tx=0x7fa7f4006e40 comp rx=0 tx=0).stop 2024-06-03T03:11:30.515 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:29.818+0000 7fa80e800700 1 -- 172.21.15.117:0/3741180244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa808107d80 msgr2=0x7fa80807db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:11:30.515 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:29.818+0000 7fa80e800700 1 --2- 172.21.15.117:0/3741180244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa808107d80 0x7fa80807db70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc009530 tx=0x7fa7fc003d00 comp rx=0 tx=0).stop 2024-06-03T03:11:30.515 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:29.818+0000 7fa80e800700 1 -- 172.21.15.117:0/3741180244 shutdown_connections 2024-06-03T03:11:30.515 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:29.818+0000 7fa80e800700 1 --2- 172.21.15.117:0/3741180244 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fa7f00390a0 0x7fa7f003b560 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:11:30.515 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:29.818+0000 7fa80e800700 1 --2- 172.21.15.117:0/3741180244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa808107d80 0x7fa80807db70 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:11:30.515 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:29.818+0000 7fa80e800700 1 -- 172.21.15.117:0/3741180244 >> 172.21.15.117:0/3741180244 conn(0x7fa808103320 msgr2=0x7fa808103e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:11:30.515 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:29.818+0000 7fa80e800700 1 -- 172.21.15.117:0/3741180244 shutdown_connections 2024-06-03T03:11:30.515 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:29.818+0000 7fa80e800700 1 -- 172.21.15.117:0/3741180244 wait complete. 2024-06-03T03:11:30.515 INFO:teuthology.orchestra.run.smithi117.stdout:Fetching dashboard port number... 2024-06-03T03:11:30.747 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:30 smithi117 ceph-mon[31649]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:11:30.748 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:30 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:11:30.748 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:30 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:11:31.697 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:31 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/4222503213' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-06-03T03:11:32.649 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout 8443 2024-06-03T03:11:32.649 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:31.232+0000 7f6e0d400700 1 Processor -- start 2024-06-03T03:11:32.649 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:31.232+0000 7f6e0d400700 1 -- start start 2024-06-03T03:11:32.649 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:31.232+0000 7f6e0d400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e08107fa0 0x7f6e081083e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:11:32.649 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:31.232+0000 7f6e0d400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6e08108920 con 0x7f6e08107fa0 2024-06-03T03:11:32.650 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:31.233+0000 7f6e06800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e08107fa0 0x7f6e081083e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:11:32.652 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:31.233+0000 7f6e06800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e08107fa0 0x7f6e081083e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39158/0 (socket says 172.21.15.117:39158) 2024-06-03T03:11:32.652 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:31.233+0000 7f6e06800700 1 -- 172.21.15.117:0/3192920471 learned_addr learned my addr 172.21.15.117:0/3192920471 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:11:32.652 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:31.233+0000 7f6e06800700 1 -- 172.21.15.117:0/3192920471 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e081090e0 con 0x7f6e08107fa0 2024-06-03T03:11:32.652 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:31.233+0000 7f6e06800700 1 --2- 172.21.15.117:0/3192920471 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e08107fa0 0x7f6e081083e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6df80098a0 tx=0x7f6df8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=702faef1544fe1a1 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:11:32.652 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:31.234+0000 7f6e05400700 1 -- 172.21.15.117:0/3192920471 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6df8004160 con 0x7f6e08107fa0 2024-06-03T03:11:32.652 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:31.234+0000 7f6e05400700 1 -- 172.21.15.117:0/3192920471 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6df8004300 con 0x7f6e08107fa0 2024-06-03T03:11:32.652 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:31.234+0000 7f6e05400700 1 -- 172.21.15.117:0/3192920471 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6df8011b20 con 0x7f6e08107fa0 2024-06-03T03:11:32.652 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:31.234+0000 7f6e0d400700 1 -- 172.21.15.117:0/3192920471 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e08107fa0 msgr2=0x7f6e081083e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:11:32.652 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:31.234+0000 7f6e0d400700 1 --2- 172.21.15.117:0/3192920471 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e08107fa0 0x7f6e081083e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6df80098a0 tx=0x7f6df8009bb0 comp rx=0 tx=0).stop 2024-06-03T03:11:32.653 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:31.235+0000 7f6e0d400700 1 -- 172.21.15.117:0/3192920471 shutdown_connections 2024-06-03T03:11:32.653 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:31.235+0000 7f6e0d400700 1 --2- 172.21.15.117:0/3192920471 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e08107fa0 0x7f6e081083e0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:11:32.653 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:31.235+0000 7f6e0d400700 1 -- 172.21.15.117:0/3192920471 >> 172.21.15.117:0/3192920471 conn(0x7f6e081035b0 msgr2=0x7f6e081059d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:11:32.653 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:31.235+0000 7f6e0d400700 1 -- 172.21.15.117:0/3192920471 shutdown_connections 2024-06-03T03:11:32.653 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:31.235+0000 7f6e0d400700 1 -- 172.21.15.117:0/3192920471 wait complete. 2024-06-03T03:11:32.653 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:31.236+0000 7f6e0d400700 1 Processor -- start 2024-06-03T03:11:32.653 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:31.236+0000 7f6e0d400700 1 -- start start 2024-06-03T03:11:32.653 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:31.236+0000 7f6e0d400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e08107fa0 0x7f6e0819e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:11:32.653 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:31.236+0000 7f6e0d400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6e0810c3a0 con 0x7f6e08107fa0 2024-06-03T03:11:32.653 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:31.237+0000 7f6e06800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e08107fa0 0x7f6e0819e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:11:32.653 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:31.237+0000 7f6e06800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e08107fa0 0x7f6e0819e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39170/0 (socket says 172.21.15.117:39170) 2024-06-03T03:11:32.653 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:31.237+0000 7f6e06800700 1 -- 172.21.15.117:0/4222503213 learned_addr learned my addr 172.21.15.117:0/4222503213 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:11:32.656 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:31.237+0000 7f6e06800700 1 -- 172.21.15.117:0/4222503213 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e0819e610 con 0x7f6e08107fa0 2024-06-03T03:11:32.656 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:31.237+0000 7f6e06800700 1 --2- 172.21.15.117:0/4222503213 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e08107fa0 0x7f6e0819e0d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6df80117e0 tx=0x7f6df8011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:11:32.656 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:31.238+0000 7f6dff400700 1 -- 172.21.15.117:0/4222503213 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6df8003bf0 con 0x7f6e08107fa0 2024-06-03T03:11:32.656 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:31.238+0000 7f6dff400700 1 -- 172.21.15.117:0/4222503213 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6df801a910 con 0x7f6e08107fa0 2024-06-03T03:11:32.656 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:31.238+0000 7f6dff400700 1 -- 172.21.15.117:0/4222503213 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6df8024970 con 0x7f6e08107fa0 2024-06-03T03:11:32.656 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:31.238+0000 7f6e0d400700 1 -- 172.21.15.117:0/4222503213 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e0819e8a0 con 0x7f6e08107fa0 2024-06-03T03:11:32.656 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:31.238+0000 7f6e0d400700 1 -- 172.21.15.117:0/4222503213 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e0819ed00 con 0x7f6e08107fa0 2024-06-03T03:11:32.656 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:31.239+0000 7f6e0d400700 1 -- 172.21.15.117:0/4222503213 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e08063af0 con 0x7f6e08107fa0 2024-06-03T03:11:32.656 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:31.239+0000 7f6dff400700 1 -- 172.21.15.117:0/4222503213 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f6df8022020 con 0x7f6e08107fa0 2024-06-03T03:11:32.656 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:31.240+0000 7f6dff400700 1 --2- 172.21.15.117:0/4222503213 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f6dec039100 0x7f6dec03b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:11:32.656 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:31.240+0000 7f6dff400700 1 -- 172.21.15.117:0/4222503213 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f6df804dbb0 con 0x7f6e08107fa0 2024-06-03T03:11:32.657 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:31.240+0000 7f6e05e00700 1 --2- 172.21.15.117:0/4222503213 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f6dec039100 0x7f6dec03b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:11:32.657 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:31.241+0000 7f6e05e00700 1 --2- 172.21.15.117:0/4222503213 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f6dec039100 0x7f6dec03b5c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6df4006fd0 tx=0x7f6df4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:11:32.657 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:31.245+0000 7f6dff400700 1 -- 172.21.15.117:0/4222503213 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6df8029060 con 0x7f6e08107fa0 2024-06-03T03:11:32.657 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:31.489+0000 7f6e0d400700 1 -- 172.21.15.117:0/4222503213 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f6e081a1a00 con 0x7f6e08107fa0 2024-06-03T03:11:32.657 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:31.490+0000 7f6dff400700 1 -- 172.21.15.117:0/4222503213 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f6df801ab70 con 0x7f6e08107fa0 2024-06-03T03:11:32.657 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:31.493+0000 7f6e0d400700 1 -- 172.21.15.117:0/4222503213 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f6dec039100 msgr2=0x7f6dec03b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:11:32.657 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:31.493+0000 7f6e0d400700 1 --2- 172.21.15.117:0/4222503213 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f6dec039100 0x7f6dec03b5c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6df4006fd0 tx=0x7f6df4006e40 comp rx=0 tx=0).stop 2024-06-03T03:11:32.657 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:31.493+0000 7f6e0d400700 1 -- 172.21.15.117:0/4222503213 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e08107fa0 msgr2=0x7f6e0819e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:11:32.657 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:31.493+0000 7f6e0d400700 1 --2- 172.21.15.117:0/4222503213 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e08107fa0 0x7f6e0819e0d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6df80117e0 tx=0x7f6df8011ba0 comp rx=0 tx=0).stop 2024-06-03T03:11:32.657 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:31.493+0000 7f6e0d400700 1 -- 172.21.15.117:0/4222503213 shutdown_connections 2024-06-03T03:11:32.657 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:31.493+0000 7f6e0d400700 1 --2- 172.21.15.117:0/4222503213 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f6dec039100 0x7f6dec03b5c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:11:32.657 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:31.493+0000 7f6e0d400700 1 --2- 172.21.15.117:0/4222503213 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e08107fa0 0x7f6e0819e0d0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:11:32.657 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:31.493+0000 7f6e0d400700 1 -- 172.21.15.117:0/4222503213 >> 172.21.15.117:0/4222503213 conn(0x7f6e081035b0 msgr2=0x7f6e08103fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:11:32.658 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:31.493+0000 7f6e0d400700 1 -- 172.21.15.117:0/4222503213 shutdown_connections 2024-06-03T03:11:32.658 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:31.493+0000 7f6e0d400700 1 -- 172.21.15.117:0/4222503213 wait complete. 2024-06-03T03:11:32.683 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-06-03T03:11:32.683 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stdout disabled 2024-06-03T03:11:32.705 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-06-03T03:11:32.705 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stdout inactive 2024-06-03T03:11:32.705 INFO:teuthology.orchestra.run.smithi117.stdout:firewalld.service is not enabled 2024-06-03T03:11:32.705 INFO:teuthology.orchestra.run.smithi117.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-06-03T03:11:32.708 INFO:teuthology.orchestra.run.smithi117.stdout:Ceph Dashboard is now available at: 2024-06-03T03:11:32.708 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T03:11:32.708 INFO:teuthology.orchestra.run.smithi117.stdout: URL: https://smithi117.front.sepia.ceph.com:8443/ 2024-06-03T03:11:32.708 INFO:teuthology.orchestra.run.smithi117.stdout: User: admin 2024-06-03T03:11:32.708 INFO:teuthology.orchestra.run.smithi117.stdout: Password: lp8qyvzg01 2024-06-03T03:11:32.708 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T03:11:32.708 INFO:teuthology.orchestra.run.smithi117.stdout:Saving cluster configuration to /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/config directory 2024-06-03T03:11:32.709 INFO:teuthology.orchestra.run.smithi117.stdout:Enabling autotune for osd_memory_target 2024-06-03T03:11:32.878 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:32 smithi117 ceph-mon[31649]: mgrmap e13: x(active, since 6s) 2024-06-03T03:11:34.520 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:33.646+0000 7f7be4c00700 1 Processor -- start 2024-06-03T03:11:34.520 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:33.646+0000 7f7be4c00700 1 -- start start 2024-06-03T03:11:34.520 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:33.647+0000 7f7be4c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7be0106870 0x7f7be0106c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:11:34.520 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:33.647+0000 7f7be4c00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7be007be30 con 0x7f7be0106870 2024-06-03T03:11:34.520 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:33.647+0000 7f7bdde00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7be0106870 0x7f7be0106c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:11:34.521 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:33.647+0000 7f7bdde00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7be0106870 0x7f7be0106c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39176/0 (socket says 172.21.15.117:39176) 2024-06-03T03:11:34.521 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:33.647+0000 7f7bdde00700 1 -- 172.21.15.117:0/1765998895 learned_addr learned my addr 172.21.15.117:0/1765998895 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:11:34.521 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:33.648+0000 7f7bdde00700 1 -- 172.21.15.117:0/1765998895 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7be01071d0 con 0x7f7be0106870 2024-06-03T03:11:34.521 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:33.648+0000 7f7bdde00700 1 --2- 172.21.15.117:0/1765998895 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7be0106870 0x7f7be0106c90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7bd00098a0 tx=0x7f7bd0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7bc7b0d6f773a3e server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:11:34.521 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:33.648+0000 7f7bdca00700 1 -- 172.21.15.117:0/1765998895 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7bd0004160 con 0x7f7be0106870 2024-06-03T03:11:34.521 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:33.648+0000 7f7bdca00700 1 -- 172.21.15.117:0/1765998895 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bd0004300 con 0x7f7be0106870 2024-06-03T03:11:34.521 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:33.648+0000 7f7bdca00700 1 -- 172.21.15.117:0/1765998895 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7bd0011b20 con 0x7f7be0106870 2024-06-03T03:11:34.521 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:33.649+0000 7f7be4c00700 1 -- 172.21.15.117:0/1765998895 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7be0106870 msgr2=0x7f7be0106c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:11:34.521 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:33.649+0000 7f7be4c00700 1 --2- 172.21.15.117:0/1765998895 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7be0106870 0x7f7be0106c90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7bd00098a0 tx=0x7f7bd0009bb0 comp rx=0 tx=0).stop 2024-06-03T03:11:34.521 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:33.650+0000 7f7be4c00700 1 -- 172.21.15.117:0/1765998895 shutdown_connections 2024-06-03T03:11:34.521 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:33.650+0000 7f7be4c00700 1 --2- 172.21.15.117:0/1765998895 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7be0106870 0x7f7be0106c90 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:11:34.521 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:33.650+0000 7f7be4c00700 1 -- 172.21.15.117:0/1765998895 >> 172.21.15.117:0/1765998895 conn(0x7f7be0102450 msgr2=0x7f7be0104870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:11:34.521 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:33.650+0000 7f7be4c00700 1 -- 172.21.15.117:0/1765998895 shutdown_connections 2024-06-03T03:11:34.521 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:33.650+0000 7f7be4c00700 1 -- 172.21.15.117:0/1765998895 wait complete. 2024-06-03T03:11:34.522 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:33.651+0000 7f7be4c00700 1 Processor -- start 2024-06-03T03:11:34.522 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:33.651+0000 7f7be4c00700 1 -- start start 2024-06-03T03:11:34.522 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:33.651+0000 7f7be4c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7be0106870 0x7f7be019e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:11:34.522 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:33.651+0000 7f7be4c00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7be01082a0 con 0x7f7be0106870 2024-06-03T03:11:34.522 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:33.651+0000 7f7bdde00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7be0106870 0x7f7be019e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:11:34.522 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:33.651+0000 7f7bdde00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7be0106870 0x7f7be019e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39182/0 (socket says 172.21.15.117:39182) 2024-06-03T03:11:34.522 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:33.651+0000 7f7bdde00700 1 -- 172.21.15.117:0/1591047258 learned_addr learned my addr 172.21.15.117:0/1591047258 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:11:34.522 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:33.652+0000 7f7bdde00700 1 -- 172.21.15.117:0/1591047258 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7be019e5c0 con 0x7f7be0106870 2024-06-03T03:11:34.522 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:33.652+0000 7f7bdde00700 1 --2- 172.21.15.117:0/1591047258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7be0106870 0x7f7be019e080 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7bd00117e0 tx=0x7f7bd0011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:11:34.522 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:33.652+0000 7f7bd6a00700 1 -- 172.21.15.117:0/1591047258 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7bd0003ba0 con 0x7f7be0106870 2024-06-03T03:11:34.522 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:33.652+0000 7f7bd6a00700 1 -- 172.21.15.117:0/1591047258 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bd001a910 con 0x7f7be0106870 2024-06-03T03:11:34.522 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:33.652+0000 7f7be4c00700 1 -- 172.21.15.117:0/1591047258 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7be019e850 con 0x7f7be0106870 2024-06-03T03:11:34.522 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:33.652+0000 7f7bd6a00700 1 -- 172.21.15.117:0/1591047258 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7bd00248f0 con 0x7f7be0106870 2024-06-03T03:11:34.523 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:33.653+0000 7f7be4c00700 1 -- 172.21.15.117:0/1591047258 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7be019ecb0 con 0x7f7be0106870 2024-06-03T03:11:34.523 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:33.654+0000 7f7bd6a00700 1 -- 172.21.15.117:0/1591047258 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f7bd0022020 con 0x7f7be0106870 2024-06-03T03:11:34.523 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:33.654+0000 7f7be4c00700 1 -- 172.21.15.117:0/1591047258 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7be0063af0 con 0x7f7be0106870 2024-06-03T03:11:34.523 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:33.654+0000 7f7bd6a00700 1 --2- 172.21.15.117:0/1591047258 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f7bc40391a0 0x7f7bc403b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:11:34.523 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:33.654+0000 7f7bd6a00700 1 -- 172.21.15.117:0/1591047258 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f7bd004d740 con 0x7f7be0106870 2024-06-03T03:11:34.523 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:33.654+0000 7f7bdd400700 1 --2- 172.21.15.117:0/1591047258 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f7bc40391a0 0x7f7bc403b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:11:34.523 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:33.655+0000 7f7bdd400700 1 --2- 172.21.15.117:0/1591047258 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f7bc40391a0 0x7f7bc403b660 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc006fd0 tx=0x7f7bcc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:11:34.523 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:33.660+0000 7f7bd6a00700 1 -- 172.21.15.117:0/1591047258 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7bd000b3e0 con 0x7f7be0106870 2024-06-03T03:11:34.524 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:33.880+0000 7f7be4c00700 1 -- 172.21.15.117:0/1591047258 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f7be00630a0 con 0x7f7be0106870 2024-06-03T03:11:34.524 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:33.881+0000 7f7bd6a00700 1 -- 172.21.15.117:0/1591047258 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f7bd00172b0 con 0x7f7be0106870 2024-06-03T03:11:34.524 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:33.884+0000 7f7be4c00700 1 -- 172.21.15.117:0/1591047258 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f7bc40391a0 msgr2=0x7f7bc403b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:11:34.524 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:33.884+0000 7f7be4c00700 1 --2- 172.21.15.117:0/1591047258 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f7bc40391a0 0x7f7bc403b660 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc006fd0 tx=0x7f7bcc006e40 comp rx=0 tx=0).stop 2024-06-03T03:11:34.524 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:33.884+0000 7f7be4c00700 1 -- 172.21.15.117:0/1591047258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7be0106870 msgr2=0x7f7be019e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:11:34.524 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:33.884+0000 7f7be4c00700 1 --2- 172.21.15.117:0/1591047258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7be0106870 0x7f7be019e080 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7bd00117e0 tx=0x7f7bd0011ba0 comp rx=0 tx=0).stop 2024-06-03T03:11:34.525 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:33.884+0000 7f7be4c00700 1 -- 172.21.15.117:0/1591047258 shutdown_connections 2024-06-03T03:11:34.525 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:33.884+0000 7f7be4c00700 1 --2- 172.21.15.117:0/1591047258 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f7bc40391a0 0x7f7bc403b660 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:11:34.525 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:33.884+0000 7f7be4c00700 1 --2- 172.21.15.117:0/1591047258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7be0106870 0x7f7be019e080 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:11:34.525 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:33.884+0000 7f7be4c00700 1 -- 172.21.15.117:0/1591047258 >> 172.21.15.117:0/1591047258 conn(0x7f7be0102450 msgr2=0x7f7be0194ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:11:34.525 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:33.884+0000 7f7be4c00700 1 -- 172.21.15.117:0/1591047258 shutdown_connections 2024-06-03T03:11:34.525 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:33.884+0000 7f7be4c00700 1 -- 172.21.15.117:0/1591047258 wait complete. 2024-06-03T03:11:36.131 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:35.087+0000 7f9e3ca00700 1 Processor -- start 2024-06-03T03:11:36.131 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:35.087+0000 7f9e3ca00700 1 -- start start 2024-06-03T03:11:36.131 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:35.088+0000 7f9e3ca00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e3810a1a0 0x7f9e3810a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:11:36.131 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:35.088+0000 7f9e3ca00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9e3810ab90 con 0x7f9e3810a1a0 2024-06-03T03:11:36.131 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:35.088+0000 7f9e35e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e3810a1a0 0x7f9e3810a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:11:36.131 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:35.089+0000 7f9e35e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e3810a1a0 0x7f9e3810a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39196/0 (socket says 172.21.15.117:39196) 2024-06-03T03:11:36.131 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:35.089+0000 7f9e35e00700 1 -- 172.21.15.117:0/633513960 learned_addr learned my addr 172.21.15.117:0/633513960 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:11:36.131 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:35.089+0000 7f9e35e00700 1 -- 172.21.15.117:0/633513960 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e3810b390 con 0x7f9e3810a1a0 2024-06-03T03:11:36.132 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:35.089+0000 7f9e35e00700 1 --2- 172.21.15.117:0/633513960 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e3810a1a0 0x7f9e3810a5c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9e280098a0 tx=0x7f9e28009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=834439225af73c04 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:11:36.132 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:35.090+0000 7f9e34a00700 1 -- 172.21.15.117:0/633513960 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e28004160 con 0x7f9e3810a1a0 2024-06-03T03:11:36.132 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:35.090+0000 7f9e34a00700 1 -- 172.21.15.117:0/633513960 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e28004300 con 0x7f9e3810a1a0 2024-06-03T03:11:36.132 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:35.090+0000 7f9e34a00700 1 -- 172.21.15.117:0/633513960 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e28011b20 con 0x7f9e3810a1a0 2024-06-03T03:11:36.132 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:35.090+0000 7f9e3ca00700 1 -- 172.21.15.117:0/633513960 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e3810a1a0 msgr2=0x7f9e3810a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:11:36.132 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:35.090+0000 7f9e3ca00700 1 --2- 172.21.15.117:0/633513960 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e3810a1a0 0x7f9e3810a5c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9e280098a0 tx=0x7f9e28009bb0 comp rx=0 tx=0).stop 2024-06-03T03:11:36.132 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:35.091+0000 7f9e3ca00700 1 -- 172.21.15.117:0/633513960 shutdown_connections 2024-06-03T03:11:36.132 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:35.091+0000 7f9e3ca00700 1 --2- 172.21.15.117:0/633513960 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e3810a1a0 0x7f9e3810a5c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:11:36.136 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:35.091+0000 7f9e3ca00700 1 -- 172.21.15.117:0/633513960 >> 172.21.15.117:0/633513960 conn(0x7f9e3807d710 msgr2=0x7f9e3807db60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:11:36.136 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:35.091+0000 7f9e3ca00700 1 -- 172.21.15.117:0/633513960 shutdown_connections 2024-06-03T03:11:36.136 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:35.091+0000 7f9e3ca00700 1 -- 172.21.15.117:0/633513960 wait complete. 2024-06-03T03:11:36.136 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:35.092+0000 7f9e3ca00700 1 Processor -- start 2024-06-03T03:11:36.137 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:35.092+0000 7f9e3ca00700 1 -- start start 2024-06-03T03:11:36.137 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:35.092+0000 7f9e3ca00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e3810a1a0 0x7f9e3819e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:11:36.137 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:35.092+0000 7f9e3ca00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9e3810e740 con 0x7f9e3810a1a0 2024-06-03T03:11:36.137 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:35.092+0000 7f9e35e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e3810a1a0 0x7f9e3819e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:11:36.137 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:35.093+0000 7f9e35e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e3810a1a0 0x7f9e3819e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39202/0 (socket says 172.21.15.117:39202) 2024-06-03T03:11:36.137 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:35.093+0000 7f9e35e00700 1 -- 172.21.15.117:0/1131233315 learned_addr learned my addr 172.21.15.117:0/1131233315 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:11:36.137 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:35.093+0000 7f9e35e00700 1 -- 172.21.15.117:0/1131233315 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e3819e620 con 0x7f9e3810a1a0 2024-06-03T03:11:36.137 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:35.093+0000 7f9e35e00700 1 --2- 172.21.15.117:0/1131233315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e3810a1a0 0x7f9e3819e0e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9e280117e0 tx=0x7f9e28011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:11:36.137 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:35.093+0000 7f9e2ea00700 1 -- 172.21.15.117:0/1131233315 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e28003bf0 con 0x7f9e3810a1a0 2024-06-03T03:11:36.137 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:35.093+0000 7f9e2ea00700 1 -- 172.21.15.117:0/1131233315 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e2801a910 con 0x7f9e3810a1a0 2024-06-03T03:11:36.137 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:35.093+0000 7f9e3ca00700 1 -- 172.21.15.117:0/1131233315 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e3819e8b0 con 0x7f9e3810a1a0 2024-06-03T03:11:36.137 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:35.094+0000 7f9e2ea00700 1 -- 172.21.15.117:0/1131233315 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e28024970 con 0x7f9e3810a1a0 2024-06-03T03:11:36.138 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:35.094+0000 7f9e3ca00700 1 -- 172.21.15.117:0/1131233315 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e3819ed10 con 0x7f9e3810a1a0 2024-06-03T03:11:36.138 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:35.095+0000 7f9e2ea00700 1 -- 172.21.15.117:0/1131233315 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f9e28022020 con 0x7f9e3810a1a0 2024-06-03T03:11:36.138 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:35.095+0000 7f9e3ca00700 1 -- 172.21.15.117:0/1131233315 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e38063af0 con 0x7f9e3810a1a0 2024-06-03T03:11:36.138 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:35.095+0000 7f9e2ea00700 1 --2- 172.21.15.117:0/1131233315 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f9e1c039140 0x7f9e1c03b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:11:36.138 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:35.095+0000 7f9e2ea00700 1 -- 172.21.15.117:0/1131233315 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f9e2804db00 con 0x7f9e3810a1a0 2024-06-03T03:11:36.138 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:35.095+0000 7f9e35400700 1 --2- 172.21.15.117:0/1131233315 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f9e1c039140 0x7f9e1c03b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:11:36.138 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:35.096+0000 7f9e35400700 1 --2- 172.21.15.117:0/1131233315 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f9e1c039140 0x7f9e1c03b600 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9e24006fd0 tx=0x7f9e24006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:11:36.138 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:35.101+0000 7f9e2ea00700 1 -- 172.21.15.117:0/1131233315 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9e2802d910 con 0x7f9e3810a1a0 2024-06-03T03:11:36.138 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:35.444+0000 7f9e3ca00700 1 -- 172.21.15.117:0/1131233315 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f9e381a16e0 con 0x7f9e3810a1a0 2024-06-03T03:11:36.138 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:35.450+0000 7f9e2ea00700 1 -- 172.21.15.117:0/1131233315 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) ==== 153+0+0 (secure 0 0 0) 0x7f9e2804de20 con 0x7f9e3810a1a0 2024-06-03T03:11:36.138 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:35.453+0000 7f9e3ca00700 1 -- 172.21.15.117:0/1131233315 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f9e1c039140 msgr2=0x7f9e1c03b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:11:36.138 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:35.453+0000 7f9e3ca00700 1 --2- 172.21.15.117:0/1131233315 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f9e1c039140 0x7f9e1c03b600 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9e24006fd0 tx=0x7f9e24006e40 comp rx=0 tx=0).stop 2024-06-03T03:11:36.138 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:35.453+0000 7f9e3ca00700 1 -- 172.21.15.117:0/1131233315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e3810a1a0 msgr2=0x7f9e3819e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:11:36.139 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:35.453+0000 7f9e3ca00700 1 --2- 172.21.15.117:0/1131233315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e3810a1a0 0x7f9e3819e0e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9e280117e0 tx=0x7f9e28011ba0 comp rx=0 tx=0).stop 2024-06-03T03:11:36.139 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:35.453+0000 7f9e3ca00700 1 -- 172.21.15.117:0/1131233315 shutdown_connections 2024-06-03T03:11:36.139 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:35.453+0000 7f9e3ca00700 1 --2- 172.21.15.117:0/1131233315 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f9e1c039140 0x7f9e1c03b600 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:11:36.139 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:35.453+0000 7f9e3ca00700 1 --2- 172.21.15.117:0/1131233315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e3810a1a0 0x7f9e3819e0e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:11:36.139 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:35.453+0000 7f9e3ca00700 1 -- 172.21.15.117:0/1131233315 >> 172.21.15.117:0/1131233315 conn(0x7f9e3807d710 msgr2=0x7f9e38107c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:11:36.139 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:35.453+0000 7f9e3ca00700 1 -- 172.21.15.117:0/1131233315 shutdown_connections 2024-06-03T03:11:36.139 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-06-03T03:11:35.453+0000 7f9e3ca00700 1 -- 172.21.15.117:0/1131233315 wait complete. 2024-06-03T03:11:36.139 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-06-03T03:11:36.139 INFO:teuthology.orchestra.run.smithi117.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-06-03T03:11:36.139 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T03:11:36.139 INFO:teuthology.orchestra.run.smithi117.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-06-03T03:11:36.139 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T03:11:36.140 INFO:teuthology.orchestra.run.smithi117.stdout:Or, if you are only running a single cluster on this host: 2024-06-03T03:11:36.140 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T03:11:36.140 INFO:teuthology.orchestra.run.smithi117.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-06-03T03:11:36.140 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T03:11:36.140 INFO:teuthology.orchestra.run.smithi117.stdout:Please consider enabling telemetry to help improve Ceph: 2024-06-03T03:11:36.140 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T03:11:36.140 INFO:teuthology.orchestra.run.smithi117.stdout: ceph telemetry on 2024-06-03T03:11:36.140 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T03:11:36.140 INFO:teuthology.orchestra.run.smithi117.stdout:For more information see: 2024-06-03T03:11:36.140 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T03:11:36.140 INFO:teuthology.orchestra.run.smithi117.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-06-03T03:11:36.140 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T03:11:36.140 INFO:teuthology.orchestra.run.smithi117.stdout:Bootstrap complete. 2024-06-03T03:11:36.229 INFO:tasks.cephadm:Fetching config... 2024-06-03T03:11:36.230 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-06-03T03:11:36.230 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-06-03T03:11:36.280 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-06-03T03:11:36.280 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-06-03T03:11:36.280 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-06-03T03:11:36.363 INFO:tasks.cephadm:Fetching mon keyring... 2024-06-03T03:11:36.363 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-06-03T03:11:36.363 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/keyring of=/dev/stdout 2024-06-03T03:11:36.485 INFO:tasks.cephadm:Fetching pub ssh key... 2024-06-03T03:11:36.485 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-06-03T03:11:36.485 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-06-03T03:11:36.534 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-06-03T03:11:36.535 DEBUG:teuthology.orchestra.run.smithi117:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDk9chwckqyNL+exGwktePMvfnjZu1PoS16S7g4tpE8O9j8b18USig1upkmhoI1nasKfujRnMxCFlRCKNN2DWYx3xkXfrwvLyjxQxEDiVlj88IhkIU03sTTMEfkghw6jUNEWznlWExf1VRiEdPd8GS7Uqeo2KogRGTDwUkjOaB3nvBu2Ot5ljE0kC0tTK41kT8oqgyT6L+4kAnPAKs+H5KalftCVqpdDVBB+xa1Uuj4UNdiBKYSqdCII5Ch92sOrWTkFykOinkvUju2Pge3FwJJ+m9CC1OQT0IgCZCU8DobO6Kg66Pu/CNsm2Z/QZwE4c9P6Bt3BTAwLt7Ne74/P9iLAuNdr4BoD5aPuHiMwJnm6vwuD1/KP6zKxzlHFTFZnaK9MO20FPGpZHZxW3HjGyVBx3apmDhoeIb3mZSZqaP5SdltmHxT+Nne4D8RjkJxIjNwi1cbQXUaQ6GDibL53PKb+3sEKE9B/nUEbfCQWvIro63q4Y4CcaYbH01iVaCenGc= ceph-a0e8473a-2156-11ef-bc9b-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-06-03T03:11:36.669 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:36 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/1131233315' entity='client.admin' 2024-06-03T03:11:36.704 INFO:teuthology.orchestra.run.smithi117.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDk9chwckqyNL+exGwktePMvfnjZu1PoS16S7g4tpE8O9j8b18USig1upkmhoI1nasKfujRnMxCFlRCKNN2DWYx3xkXfrwvLyjxQxEDiVlj88IhkIU03sTTMEfkghw6jUNEWznlWExf1VRiEdPd8GS7Uqeo2KogRGTDwUkjOaB3nvBu2Ot5ljE0kC0tTK41kT8oqgyT6L+4kAnPAKs+H5KalftCVqpdDVBB+xa1Uuj4UNdiBKYSqdCII5Ch92sOrWTkFykOinkvUju2Pge3FwJJ+m9CC1OQT0IgCZCU8DobO6Kg66Pu/CNsm2Z/QZwE4c9P6Bt3BTAwLt7Ne74/P9iLAuNdr4BoD5aPuHiMwJnm6vwuD1/KP6zKxzlHFTFZnaK9MO20FPGpZHZxW3HjGyVBx3apmDhoeIb3mZSZqaP5SdltmHxT+Nne4D8RjkJxIjNwi1cbQXUaQ6GDibL53PKb+3sEKE9B/nUEbfCQWvIro63q4Y4CcaYbH01iVaCenGc= ceph-a0e8473a-2156-11ef-bc9b-c7b262605968 2024-06-03T03:11:36.745 DEBUG:teuthology.orchestra.run.smithi151:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDk9chwckqyNL+exGwktePMvfnjZu1PoS16S7g4tpE8O9j8b18USig1upkmhoI1nasKfujRnMxCFlRCKNN2DWYx3xkXfrwvLyjxQxEDiVlj88IhkIU03sTTMEfkghw6jUNEWznlWExf1VRiEdPd8GS7Uqeo2KogRGTDwUkjOaB3nvBu2Ot5ljE0kC0tTK41kT8oqgyT6L+4kAnPAKs+H5KalftCVqpdDVBB+xa1Uuj4UNdiBKYSqdCII5Ch92sOrWTkFykOinkvUju2Pge3FwJJ+m9CC1OQT0IgCZCU8DobO6Kg66Pu/CNsm2Z/QZwE4c9P6Bt3BTAwLt7Ne74/P9iLAuNdr4BoD5aPuHiMwJnm6vwuD1/KP6zKxzlHFTFZnaK9MO20FPGpZHZxW3HjGyVBx3apmDhoeIb3mZSZqaP5SdltmHxT+Nne4D8RjkJxIjNwi1cbQXUaQ6GDibL53PKb+3sEKE9B/nUEbfCQWvIro63q4Y4CcaYbH01iVaCenGc= ceph-a0e8473a-2156-11ef-bc9b-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-06-03T03:11:36.787 INFO:teuthology.orchestra.run.smithi151.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDk9chwckqyNL+exGwktePMvfnjZu1PoS16S7g4tpE8O9j8b18USig1upkmhoI1nasKfujRnMxCFlRCKNN2DWYx3xkXfrwvLyjxQxEDiVlj88IhkIU03sTTMEfkghw6jUNEWznlWExf1VRiEdPd8GS7Uqeo2KogRGTDwUkjOaB3nvBu2Ot5ljE0kC0tTK41kT8oqgyT6L+4kAnPAKs+H5KalftCVqpdDVBB+xa1Uuj4UNdiBKYSqdCII5Ch92sOrWTkFykOinkvUju2Pge3FwJJ+m9CC1OQT0IgCZCU8DobO6Kg66Pu/CNsm2Z/QZwE4c9P6Bt3BTAwLt7Ne74/P9iLAuNdr4BoD5aPuHiMwJnm6vwuD1/KP6zKxzlHFTFZnaK9MO20FPGpZHZxW3HjGyVBx3apmDhoeIb3mZSZqaP5SdltmHxT+Nne4D8RjkJxIjNwi1cbQXUaQ6GDibL53PKb+3sEKE9B/nUEbfCQWvIro63q4Y4CcaYbH01iVaCenGc= ceph-a0e8473a-2156-11ef-bc9b-c7b262605968 2024-06-03T03:11:36.799 DEBUG:teuthology.orchestra.run.smithi183:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDk9chwckqyNL+exGwktePMvfnjZu1PoS16S7g4tpE8O9j8b18USig1upkmhoI1nasKfujRnMxCFlRCKNN2DWYx3xkXfrwvLyjxQxEDiVlj88IhkIU03sTTMEfkghw6jUNEWznlWExf1VRiEdPd8GS7Uqeo2KogRGTDwUkjOaB3nvBu2Ot5ljE0kC0tTK41kT8oqgyT6L+4kAnPAKs+H5KalftCVqpdDVBB+xa1Uuj4UNdiBKYSqdCII5Ch92sOrWTkFykOinkvUju2Pge3FwJJ+m9CC1OQT0IgCZCU8DobO6Kg66Pu/CNsm2Z/QZwE4c9P6Bt3BTAwLt7Ne74/P9iLAuNdr4BoD5aPuHiMwJnm6vwuD1/KP6zKxzlHFTFZnaK9MO20FPGpZHZxW3HjGyVBx3apmDhoeIb3mZSZqaP5SdltmHxT+Nne4D8RjkJxIjNwi1cbQXUaQ6GDibL53PKb+3sEKE9B/nUEbfCQWvIro63q4Y4CcaYbH01iVaCenGc= ceph-a0e8473a-2156-11ef-bc9b-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-06-03T03:11:36.841 INFO:teuthology.orchestra.run.smithi183.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDk9chwckqyNL+exGwktePMvfnjZu1PoS16S7g4tpE8O9j8b18USig1upkmhoI1nasKfujRnMxCFlRCKNN2DWYx3xkXfrwvLyjxQxEDiVlj88IhkIU03sTTMEfkghw6jUNEWznlWExf1VRiEdPd8GS7Uqeo2KogRGTDwUkjOaB3nvBu2Ot5ljE0kC0tTK41kT8oqgyT6L+4kAnPAKs+H5KalftCVqpdDVBB+xa1Uuj4UNdiBKYSqdCII5Ch92sOrWTkFykOinkvUju2Pge3FwJJ+m9CC1OQT0IgCZCU8DobO6Kg66Pu/CNsm2Z/QZwE4c9P6Bt3BTAwLt7Ne74/P9iLAuNdr4BoD5aPuHiMwJnm6vwuD1/KP6zKxzlHFTFZnaK9MO20FPGpZHZxW3HjGyVBx3apmDhoeIb3mZSZqaP5SdltmHxT+Nne4D8RjkJxIjNwi1cbQXUaQ6GDibL53PKb+3sEKE9B/nUEbfCQWvIro63q4Y4CcaYbH01iVaCenGc= ceph-a0e8473a-2156-11ef-bc9b-c7b262605968 2024-06-03T03:11:36.853 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-06-03T03:11:37.285 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:11:39.831 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:39.830+0000 7ffac2a00700 1 -- 172.21.15.117:0/113337927 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffabc101ea0 msgr2=0x7ffabc1022c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:11:39.831 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:39.830+0000 7ffac2a00700 1 --2- 172.21.15.117:0/113337927 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffabc101ea0 0x7ffabc1022c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ffaac009910 tx=0x7ffaac009c20 comp rx=0 tx=0).stop 2024-06-03T03:11:39.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:39.830+0000 7ffac2a00700 1 -- 172.21.15.117:0/113337927 shutdown_connections 2024-06-03T03:11:39.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:39.830+0000 7ffac2a00700 1 --2- 172.21.15.117:0/113337927 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffabc101ea0 0x7ffabc1022c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:11:39.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:39.830+0000 7ffac2a00700 1 -- 172.21.15.117:0/113337927 >> 172.21.15.117:0/113337927 conn(0x7ffabc0fd4b0 msgr2=0x7ffabc0ff8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:11:39.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:39.830+0000 7ffac2a00700 1 -- 172.21.15.117:0/113337927 shutdown_connections 2024-06-03T03:11:39.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:39.830+0000 7ffac2a00700 1 -- 172.21.15.117:0/113337927 wait complete. 2024-06-03T03:11:39.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:39.832+0000 7ffac2a00700 1 Processor -- start 2024-06-03T03:11:39.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:39.832+0000 7ffac2a00700 1 -- start start 2024-06-03T03:11:39.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:39.832+0000 7ffac2a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffabc101ea0 0x7ffabc1975d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:11:39.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:39.832+0000 7ffac2a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ffabc107ef0 con 0x7ffabc101ea0 2024-06-03T03:11:39.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:39.832+0000 7ffabbe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffabc101ea0 0x7ffabc1975d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:11:39.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:39.832+0000 7ffabbe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffabc101ea0 0x7ffabc1975d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41042/0 (socket says 172.21.15.117:41042) 2024-06-03T03:11:39.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:39.832+0000 7ffabbe00700 1 -- 172.21.15.117:0/578269353 learned_addr learned my addr 172.21.15.117:0/578269353 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:11:39.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:39.833+0000 7ffabbe00700 1 -- 172.21.15.117:0/578269353 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffabc197b10 con 0x7ffabc101ea0 2024-06-03T03:11:39.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:39.833+0000 7ffabbe00700 1 --2- 172.21.15.117:0/578269353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffabc101ea0 0x7ffabc1975d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ffaac005850 tx=0x7ffaac005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:11:39.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:39.833+0000 7ffab8c00700 1 -- 172.21.15.117:0/578269353 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffaac026070 con 0x7ffabc101ea0 2024-06-03T03:11:39.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:39.833+0000 7ffab8c00700 1 -- 172.21.15.117:0/578269353 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffaac0044a0 con 0x7ffabc101ea0 2024-06-03T03:11:39.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:39.833+0000 7ffac2a00700 1 -- 172.21.15.117:0/578269353 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffabc197da0 con 0x7ffabc101ea0 2024-06-03T03:11:39.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:39.833+0000 7ffab8c00700 1 -- 172.21.15.117:0/578269353 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffaac003ab0 con 0x7ffabc101ea0 2024-06-03T03:11:39.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:39.833+0000 7ffac2a00700 1 -- 172.21.15.117:0/578269353 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffabc19aa90 con 0x7ffabc101ea0 2024-06-03T03:11:39.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:39.835+0000 7ffab8c00700 1 -- 172.21.15.117:0/578269353 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7ffaac029070 con 0x7ffabc101ea0 2024-06-03T03:11:39.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:39.834+0000 7ffac2a00700 1 -- 172.21.15.117:0/578269353 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffabc108670 con 0x7ffabc101ea0 2024-06-03T03:11:39.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:39.835+0000 7ffab8c00700 1 --2- 172.21.15.117:0/578269353 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7ffaa00390f0 0x7ffaa003b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:11:39.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:39.835+0000 7ffab8c00700 1 -- 172.21.15.117:0/578269353 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ffaac01e070 con 0x7ffabc101ea0 2024-06-03T03:11:39.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:39.835+0000 7ffabb400700 1 --2- 172.21.15.117:0/578269353 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7ffaa00390f0 0x7ffaa003b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:11:39.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:39.836+0000 7ffabb400700 1 --2- 172.21.15.117:0/578269353 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7ffaa00390f0 0x7ffaa003b5b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ffaa8006fd0 tx=0x7ffaa8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:11:39.841 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:39.841+0000 7ffab8c00700 1 -- 172.21.15.117:0/578269353 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ffaac013070 con 0x7ffabc101ea0 2024-06-03T03:11:40.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:40.067+0000 7ffac2a00700 1 -- 172.21.15.117:0/578269353 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7ffabc02ddb0 con 0x7ffabc101ea0 2024-06-03T03:11:40.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:40.072+0000 7ffab8c00700 1 -- 172.21.15.117:0/578269353 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7ffaac030020 con 0x7ffabc101ea0 2024-06-03T03:11:40.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:40.075+0000 7ffac2a00700 1 -- 172.21.15.117:0/578269353 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7ffaa00390f0 msgr2=0x7ffaa003b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:11:40.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:40.075+0000 7ffac2a00700 1 --2- 172.21.15.117:0/578269353 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7ffaa00390f0 0x7ffaa003b5b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ffaa8006fd0 tx=0x7ffaa8006e40 comp rx=0 tx=0).stop 2024-06-03T03:11:40.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:40.075+0000 7ffac2a00700 1 -- 172.21.15.117:0/578269353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffabc101ea0 msgr2=0x7ffabc1975d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:11:40.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:40.075+0000 7ffac2a00700 1 --2- 172.21.15.117:0/578269353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffabc101ea0 0x7ffabc1975d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ffaac005850 tx=0x7ffaac005030 comp rx=0 tx=0).stop 2024-06-03T03:11:40.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:40.075+0000 7ffac2a00700 1 -- 172.21.15.117:0/578269353 shutdown_connections 2024-06-03T03:11:40.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:40.075+0000 7ffac2a00700 1 --2- 172.21.15.117:0/578269353 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7ffaa00390f0 0x7ffaa003b5b0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:11:40.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:40.075+0000 7ffac2a00700 1 --2- 172.21.15.117:0/578269353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffabc101ea0 0x7ffabc1975d0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:11:40.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:40.075+0000 7ffac2a00700 1 -- 172.21.15.117:0/578269353 >> 172.21.15.117:0/578269353 conn(0x7ffabc0fd4b0 msgr2=0x7ffabc0fded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:11:40.077 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:40.075+0000 7ffac2a00700 1 -- 172.21.15.117:0/578269353 shutdown_connections 2024-06-03T03:11:40.077 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:40.076+0000 7ffac2a00700 1 -- 172.21.15.117:0/578269353 wait complete. 2024-06-03T03:11:40.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:40 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:11:40.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:40 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:11:40.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:40 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi117", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:11:40.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:40 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:11:40.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:40 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:11:40.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:40 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:11:40.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:40 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/578269353' entity='client.admin' 2024-06-03T03:11:40.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:40 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:11:40.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:40 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:11:40.827 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-06-03T03:11:40.828 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-06-03T03:11:41.250 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:11:43.306 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:43.305+0000 7f3ed4200700 1 -- 172.21.15.117:0/2225234861 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ecc0764c0 msgr2=0x7f3ecc074920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:11:43.307 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:43.305+0000 7f3ed4200700 1 --2- 172.21.15.117:0/2225234861 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ecc0764c0 0x7f3ecc074920 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3ec0009910 tx=0x7f3ec0009c20 comp rx=0 tx=0).stop 2024-06-03T03:11:43.307 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:43.305+0000 7f3ed4200700 1 -- 172.21.15.117:0/2225234861 shutdown_connections 2024-06-03T03:11:43.307 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:43.306+0000 7f3ed4200700 1 --2- 172.21.15.117:0/2225234861 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ecc0764c0 0x7f3ecc074920 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:11:43.307 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:43.306+0000 7f3ed4200700 1 -- 172.21.15.117:0/2225234861 >> 172.21.15.117:0/2225234861 conn(0x7f3ecc0fd480 msgr2=0x7f3ecc0ff8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:11:43.307 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:43.306+0000 7f3ed4200700 1 -- 172.21.15.117:0/2225234861 shutdown_connections 2024-06-03T03:11:43.307 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:43.306+0000 7f3ed4200700 1 -- 172.21.15.117:0/2225234861 wait complete. 2024-06-03T03:11:43.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:43.307+0000 7f3ed4200700 1 Processor -- start 2024-06-03T03:11:43.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:43.307+0000 7f3ed4200700 1 -- start start 2024-06-03T03:11:43.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:43.307+0000 7f3ed4200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ecc0764c0 0x7f3ecc199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:11:43.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:43.307+0000 7f3ed4200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3ecc10a170 con 0x7f3ecc0764c0 2024-06-03T03:11:43.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:43.307+0000 7f3ed1800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ecc0764c0 0x7f3ecc199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:11:43.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:43.307+0000 7f3ed1800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ecc0764c0 0x7f3ecc199820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41056/0 (socket says 172.21.15.117:41056) 2024-06-03T03:11:43.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:43.308+0000 7f3ed1800700 1 -- 172.21.15.117:0/344778515 learned_addr learned my addr 172.21.15.117:0/344778515 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:11:43.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:43.308+0000 7f3ed1800700 1 -- 172.21.15.117:0/344778515 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ecc199d60 con 0x7f3ecc0764c0 2024-06-03T03:11:43.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:43.308+0000 7f3ed1800700 1 --2- 172.21.15.117:0/344778515 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ecc0764c0 0x7f3ecc199820 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3ec0006010 tx=0x7f3ec0005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:11:43.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:43.308+0000 7f3eca000700 1 -- 172.21.15.117:0/344778515 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ec0027070 con 0x7f3ecc0764c0 2024-06-03T03:11:43.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:43.308+0000 7f3eca000700 1 -- 172.21.15.117:0/344778515 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ec000be40 con 0x7f3ecc0764c0 2024-06-03T03:11:43.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:43.308+0000 7f3ed4200700 1 -- 172.21.15.117:0/344778515 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ecc199ff0 con 0x7f3ecc0764c0 2024-06-03T03:11:43.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:43.308+0000 7f3eca000700 1 -- 172.21.15.117:0/344778515 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ec000f460 con 0x7f3ecc0764c0 2024-06-03T03:11:43.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:43.309+0000 7f3ed4200700 1 -- 172.21.15.117:0/344778515 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ecc19cce0 con 0x7f3ecc0764c0 2024-06-03T03:11:43.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:43.310+0000 7f3eca000700 1 -- 172.21.15.117:0/344778515 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f3ec002a070 con 0x7f3ecc0764c0 2024-06-03T03:11:43.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:43.310+0000 7f3ed4200700 1 -- 172.21.15.117:0/344778515 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ecc10a8f0 con 0x7f3ecc0764c0 2024-06-03T03:11:43.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:43.310+0000 7f3eca000700 1 --2- 172.21.15.117:0/344778515 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f3eb4039150 0x7f3eb403b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:11:43.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:43.310+0000 7f3eca000700 1 -- 172.21.15.117:0/344778515 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f3ec001e030 con 0x7f3ecc0764c0 2024-06-03T03:11:43.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:43.310+0000 7f3ed0e00700 1 --2- 172.21.15.117:0/344778515 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f3eb4039150 0x7f3eb403b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:11:43.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:43.311+0000 7f3ed0e00700 1 --2- 172.21.15.117:0/344778515 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f3eb4039150 0x7f3eb403b610 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3eb8006fd0 tx=0x7f3eb8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:11:43.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:43.316+0000 7f3eca000700 1 -- 172.21.15.117:0/344778515 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3ec0013070 con 0x7f3ecc0764c0 2024-06-03T03:11:43.547 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:43.547+0000 7f3ed4200700 1 -- 172.21.15.117:0/344778515 --> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f3ecc02ddb0 con 0x7f3eb4039150 2024-06-03T03:11:43.553 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:43.553+0000 7f3eca000700 1 -- 172.21.15.117:0/344778515 <== mgr.14152 v2:172.21.15.117:6800/1398160887 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3ecc02ddb0 con 0x7f3eb4039150 2024-06-03T03:11:43.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:43.556+0000 7f3ed4200700 1 -- 172.21.15.117:0/344778515 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f3eb4039150 msgr2=0x7f3eb403b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:11:43.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:43.556+0000 7f3ed4200700 1 --2- 172.21.15.117:0/344778515 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f3eb4039150 0x7f3eb403b610 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3eb8006fd0 tx=0x7f3eb8006e40 comp rx=0 tx=0).stop 2024-06-03T03:11:43.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:43.556+0000 7f3ed4200700 1 -- 172.21.15.117:0/344778515 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ecc0764c0 msgr2=0x7f3ecc199820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:11:43.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:43.556+0000 7f3ed4200700 1 --2- 172.21.15.117:0/344778515 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ecc0764c0 0x7f3ecc199820 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3ec0006010 tx=0x7f3ec0005050 comp rx=0 tx=0).stop 2024-06-03T03:11:43.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:43.556+0000 7f3ed4200700 1 -- 172.21.15.117:0/344778515 shutdown_connections 2024-06-03T03:11:43.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:43.556+0000 7f3ed4200700 1 --2- 172.21.15.117:0/344778515 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f3eb4039150 0x7f3eb403b610 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:11:43.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:43.556+0000 7f3ed4200700 1 --2- 172.21.15.117:0/344778515 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ecc0764c0 0x7f3ecc199820 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:11:43.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:43.556+0000 7f3ed4200700 1 -- 172.21.15.117:0/344778515 >> 172.21.15.117:0/344778515 conn(0x7f3ecc0fd480 msgr2=0x7f3ecc0fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:11:43.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:43.556+0000 7f3ed4200700 1 -- 172.21.15.117:0/344778515 shutdown_connections 2024-06-03T03:11:43.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:43.556+0000 7f3ed4200700 1 -- 172.21.15.117:0/344778515 wait complete. 2024-06-03T03:11:44.326 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi151 2024-06-03T03:11:44.326 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-06-03T03:11:44.327 DEBUG:teuthology.orchestra.run.smithi151:> dd of=/etc/ceph/ceph.conf 2024-06-03T03:11:44.345 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-06-03T03:11:44.345 DEBUG:teuthology.orchestra.run.smithi151:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-03T03:11:44.402 INFO:tasks.cephadm:Adding host smithi151 to orchestrator... 2024-06-03T03:11:44.402 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph orch host add smithi151 2024-06-03T03:11:44.798 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:11:44.943 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:44 smithi117 ceph-mon[31649]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:11:44.943 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:44 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:11:44.943 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:44 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:11:44.943 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:44 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:11:44.943 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:44 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:11:44.943 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:44 smithi117 ceph-mon[31649]: Updating smithi117:/etc/ceph/ceph.conf 2024-06-03T03:11:45.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:45 smithi117 ceph-mon[31649]: Updating smithi117:/var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-03T03:11:45.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:45 smithi117 ceph-mon[31649]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:11:45.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:45 smithi117 ceph-mon[31649]: Updating smithi117:/etc/ceph/ceph.client.admin.keyring 2024-06-03T03:11:46.776 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:46.775+0000 7ff7da800700 1 -- 172.21.15.117:0/951793300 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7d410e4f0 msgr2=0x7ff7d410e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:11:46.777 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:46.775+0000 7ff7da800700 1 --2- 172.21.15.117:0/951793300 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7d410e4f0 0x7ff7d410e8d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff7c4009910 tx=0x7ff7c4009c20 comp rx=0 tx=0).stop 2024-06-03T03:11:46.777 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:46.775+0000 7ff7da800700 1 -- 172.21.15.117:0/951793300 shutdown_connections 2024-06-03T03:11:46.777 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:46.775+0000 7ff7da800700 1 --2- 172.21.15.117:0/951793300 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7d410e4f0 0x7ff7d410e8d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:11:46.777 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:46.775+0000 7ff7da800700 1 -- 172.21.15.117:0/951793300 >> 172.21.15.117:0/951793300 conn(0x7ff7d4077c00 msgr2=0x7ff7d407a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:11:46.777 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:46.775+0000 7ff7da800700 1 -- 172.21.15.117:0/951793300 shutdown_connections 2024-06-03T03:11:46.777 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:46.775+0000 7ff7da800700 1 -- 172.21.15.117:0/951793300 wait complete. 2024-06-03T03:11:46.778 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:46.777+0000 7ff7da800700 1 Processor -- start 2024-06-03T03:11:46.778 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:46.777+0000 7ff7da800700 1 -- start start 2024-06-03T03:11:46.778 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:46.777+0000 7ff7da800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7d410e4f0 0x7ff7d419e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:11:46.778 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:46.777+0000 7ff7da800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d4112aa0 con 0x7ff7d410e4f0 2024-06-03T03:11:46.778 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:46.778+0000 7ff7d3400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7d410e4f0 0x7ff7d419e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:11:46.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:46.778+0000 7ff7d3400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7d410e4f0 0x7ff7d419e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41068/0 (socket says 172.21.15.117:41068) 2024-06-03T03:11:46.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:46.778+0000 7ff7d3400700 1 -- 172.21.15.117:0/760141775 learned_addr learned my addr 172.21.15.117:0/760141775 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:11:46.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:46.778+0000 7ff7d3400700 1 -- 172.21.15.117:0/760141775 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7d419e570 con 0x7ff7d410e4f0 2024-06-03T03:11:46.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:46.778+0000 7ff7d3400700 1 --2- 172.21.15.117:0/760141775 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7d410e4f0 0x7ff7d419e030 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff7c4005850 tx=0x7ff7c4005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:11:46.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:46.779+0000 7ff7cbe00700 1 -- 172.21.15.117:0/760141775 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff7c4026070 con 0x7ff7d410e4f0 2024-06-03T03:11:46.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:46.779+0000 7ff7cbe00700 1 -- 172.21.15.117:0/760141775 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7c400bbd0 con 0x7ff7d410e4f0 2024-06-03T03:11:46.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:46.779+0000 7ff7da800700 1 -- 172.21.15.117:0/760141775 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7d419e860 con 0x7ff7d410e4f0 2024-06-03T03:11:46.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:46.779+0000 7ff7cbe00700 1 -- 172.21.15.117:0/760141775 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff7c4003970 con 0x7ff7d410e4f0 2024-06-03T03:11:46.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:46.779+0000 7ff7da800700 1 -- 172.21.15.117:0/760141775 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7d41a2900 con 0x7ff7d410e4f0 2024-06-03T03:11:46.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:46.780+0000 7ff7cbe00700 1 -- 172.21.15.117:0/760141775 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7ff7c4029070 con 0x7ff7d410e4f0 2024-06-03T03:11:46.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:46.780+0000 7ff7da800700 1 -- 172.21.15.117:0/760141775 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7d4109ba0 con 0x7ff7d410e4f0 2024-06-03T03:11:46.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:46.780+0000 7ff7cbe00700 1 --2- 172.21.15.117:0/760141775 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7ff7b80390f0 0x7ff7b803b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:11:46.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:46.780+0000 7ff7cbe00700 1 -- 172.21.15.117:0/760141775 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff7c401e070 con 0x7ff7d410e4f0 2024-06-03T03:11:46.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:46.781+0000 7ff7d2a00700 1 --2- 172.21.15.117:0/760141775 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7ff7b80390f0 0x7ff7b803b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:11:46.782 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:46.782+0000 7ff7d2a00700 1 --2- 172.21.15.117:0/760141775 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7ff7b80390f0 0x7ff7b803b5b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff7c0006fd0 tx=0x7ff7c0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:11:46.787 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:46.786+0000 7ff7cbe00700 1 -- 172.21.15.117:0/760141775 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff7c4013070 con 0x7ff7d410e4f0 2024-06-03T03:11:46.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:46 smithi117 ceph-mon[31649]: Updating smithi117:/var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-03T03:11:46.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:46 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:11:46.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:46 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:11:46.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:46 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:11:47.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:47.018+0000 7ff7da800700 1 -- 172.21.15.117:0/760141775 --> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi151", "target": ["mon-mgr", ""]}) -- 0x7ff7d4063af0 con 0x7ff7b80390f0 2024-06-03T03:11:47.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:47 smithi117 ceph-mon[31649]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:11:48.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:48 smithi117 ceph-mon[31649]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi151", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:11:49.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:49 smithi117 ceph-mon[31649]: Deploying cephadm binary to smithi151 2024-06-03T03:11:49.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:49 smithi117 ceph-mon[31649]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:11:51.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:51.941+0000 7ff7cbe00700 1 -- 172.21.15.117:0/760141775 <== mgr.14152 v2:172.21.15.117:6800/1398160887 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7ff7d4063af0 con 0x7ff7b80390f0 2024-06-03T03:11:51.944 INFO:teuthology.orchestra.run.smithi117.stdout:Added host 'smithi151' with addr '172.21.15.151' 2024-06-03T03:11:51.947 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:51.945+0000 7ff7da800700 1 -- 172.21.15.117:0/760141775 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7ff7b80390f0 msgr2=0x7ff7b803b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:11:51.947 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:51.945+0000 7ff7da800700 1 --2- 172.21.15.117:0/760141775 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7ff7b80390f0 0x7ff7b803b5b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff7c0006fd0 tx=0x7ff7c0006e40 comp rx=0 tx=0).stop 2024-06-03T03:11:51.947 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:51.945+0000 7ff7da800700 1 -- 172.21.15.117:0/760141775 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7d410e4f0 msgr2=0x7ff7d419e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:11:51.947 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:51.945+0000 7ff7da800700 1 --2- 172.21.15.117:0/760141775 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7d410e4f0 0x7ff7d419e030 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff7c4005850 tx=0x7ff7c4005030 comp rx=0 tx=0).stop 2024-06-03T03:11:51.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:51.945+0000 7ff7da800700 1 -- 172.21.15.117:0/760141775 shutdown_connections 2024-06-03T03:11:51.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:51.945+0000 7ff7da800700 1 --2- 172.21.15.117:0/760141775 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7ff7b80390f0 0x7ff7b803b5b0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:11:51.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:51.946+0000 7ff7da800700 1 --2- 172.21.15.117:0/760141775 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7d410e4f0 0x7ff7d419e030 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:11:51.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:51.946+0000 7ff7da800700 1 -- 172.21.15.117:0/760141775 >> 172.21.15.117:0/760141775 conn(0x7ff7d4077c00 msgr2=0x7ff7d4110010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:11:51.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:51.946+0000 7ff7da800700 1 -- 172.21.15.117:0/760141775 shutdown_connections 2024-06-03T03:11:51.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:51.946+0000 7ff7da800700 1 -- 172.21.15.117:0/760141775 wait complete. 2024-06-03T03:11:52.016 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:51 smithi117 ceph-mon[31649]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:11:52.693 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph orch host ls --format=json 2024-06-03T03:11:53.116 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:11:53.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:52 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:11:53.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:52 smithi117 ceph-mon[31649]: Added host smithi151 2024-06-03T03:11:53.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:52 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:11:53.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:52 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:11:53.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:52 smithi117 ceph-mon[31649]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:11:54.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:54 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:11:55.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:55.082+0000 7f5166600700 1 -- 172.21.15.117:0/418054850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51601020e0 msgr2=0x7f51601024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:11:55.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:55.082+0000 7f5166600700 1 --2- 172.21.15.117:0/418054850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51601020e0 0x7f51601024c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5150009910 tx=0x7f5150009c20 comp rx=0 tx=0).stop 2024-06-03T03:11:55.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:55.082+0000 7f5166600700 1 -- 172.21.15.117:0/418054850 shutdown_connections 2024-06-03T03:11:55.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:55.082+0000 7f5166600700 1 --2- 172.21.15.117:0/418054850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51601020e0 0x7f51601024c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:11:55.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:55.082+0000 7f5166600700 1 -- 172.21.15.117:0/418054850 >> 172.21.15.117:0/418054850 conn(0x7f51600fd9b0 msgr2=0x7f51600ffdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:11:55.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:55.082+0000 7f5166600700 1 -- 172.21.15.117:0/418054850 shutdown_connections 2024-06-03T03:11:55.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:55.082+0000 7f5166600700 1 -- 172.21.15.117:0/418054850 wait complete. 2024-06-03T03:11:55.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:55.083+0000 7f5166600700 1 Processor -- start 2024-06-03T03:11:55.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:55.084+0000 7f5166600700 1 -- start start 2024-06-03T03:11:55.087 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:55.084+0000 7f5166600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51601020e0 0x7f5160074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:11:55.087 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:55.084+0000 7f5166600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f516010c4b0 con 0x7f51601020e0 2024-06-03T03:11:55.087 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:55.084+0000 7f515f400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51601020e0 0x7f5160074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:11:55.087 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:55.084+0000 7f515f400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51601020e0 0x7f5160074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:59460/0 (socket says 172.21.15.117:59460) 2024-06-03T03:11:55.087 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:55.084+0000 7f515f400700 1 -- 172.21.15.117:0/2305094060 learned_addr learned my addr 172.21.15.117:0/2305094060 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:11:55.087 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:55.085+0000 7f515f400700 1 -- 172.21.15.117:0/2305094060 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5160076680 con 0x7f51601020e0 2024-06-03T03:11:55.087 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:55.085+0000 7f515f400700 1 --2- 172.21.15.117:0/2305094060 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51601020e0 0x7f5160074790 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5150006010 tx=0x7f5150005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:11:55.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:55.085+0000 7f5157e00700 1 -- 172.21.15.117:0/2305094060 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5150026070 con 0x7f51601020e0 2024-06-03T03:11:55.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:55.085+0000 7f5157e00700 1 -- 172.21.15.117:0/2305094060 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f515000bc30 con 0x7f51601020e0 2024-06-03T03:11:55.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:55.085+0000 7f5157e00700 1 -- 172.21.15.117:0/2305094060 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f51500039d0 con 0x7f51601020e0 2024-06-03T03:11:55.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:55.085+0000 7f5166600700 1 -- 172.21.15.117:0/2305094060 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5160074d30 con 0x7f51601020e0 2024-06-03T03:11:55.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:55.085+0000 7f5166600700 1 -- 172.21.15.117:0/2305094060 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51600751f0 con 0x7f51601020e0 2024-06-03T03:11:55.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:55.086+0000 7f5166600700 1 -- 172.21.15.117:0/2305094060 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51601035c0 con 0x7f51601020e0 2024-06-03T03:11:55.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:55.087+0000 7f5157e00700 1 -- 172.21.15.117:0/2305094060 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f5150029070 con 0x7f51601020e0 2024-06-03T03:11:55.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:55.087+0000 7f5157e00700 1 --2- 172.21.15.117:0/2305094060 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f51440390f0 0x7f514403b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:11:55.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:55.087+0000 7f5157e00700 1 -- 172.21.15.117:0/2305094060 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f515001e070 con 0x7f51601020e0 2024-06-03T03:11:55.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:55.087+0000 7f515ea00700 1 --2- 172.21.15.117:0/2305094060 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f51440390f0 0x7f514403b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:11:55.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:55.088+0000 7f515ea00700 1 --2- 172.21.15.117:0/2305094060 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f51440390f0 0x7f514403b5b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f514c006fd0 tx=0x7f514c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:11:55.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:55.093+0000 7f5157e00700 1 -- 172.21.15.117:0/2305094060 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5150013070 con 0x7f51601020e0 2024-06-03T03:11:55.322 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:55.321+0000 7f5166600700 1 -- 172.21.15.117:0/2305094060 --> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5160075cc0 con 0x7f51440390f0 2024-06-03T03:11:55.324 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:55.323+0000 7f5157e00700 1 -- 172.21.15.117:0/2305094060 <== mgr.14152 v2:172.21.15.117:6800/1398160887 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+161 (secure 0 0 0) 0x7f5160075cc0 con 0x7f51440390f0 2024-06-03T03:11:55.325 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T03:11:55.325 INFO:teuthology.orchestra.run.smithi117.stdout:[{"addr": "172.21.15.117", "hostname": "smithi117", "labels": [], "status": ""}, {"addr": "172.21.15.151", "hostname": "smithi151", "labels": [], "status": ""}] 2024-06-03T03:11:55.328 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:55.326+0000 7f5166600700 1 -- 172.21.15.117:0/2305094060 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f51440390f0 msgr2=0x7f514403b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:11:55.328 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:55.326+0000 7f5166600700 1 --2- 172.21.15.117:0/2305094060 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f51440390f0 0x7f514403b5b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f514c006fd0 tx=0x7f514c006e40 comp rx=0 tx=0).stop 2024-06-03T03:11:55.328 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:55.326+0000 7f5166600700 1 -- 172.21.15.117:0/2305094060 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51601020e0 msgr2=0x7f5160074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:11:55.328 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:55.326+0000 7f5166600700 1 --2- 172.21.15.117:0/2305094060 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51601020e0 0x7f5160074790 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5150006010 tx=0x7f5150005030 comp rx=0 tx=0).stop 2024-06-03T03:11:55.328 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:55.326+0000 7f5166600700 1 -- 172.21.15.117:0/2305094060 shutdown_connections 2024-06-03T03:11:55.328 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:55.326+0000 7f5166600700 1 --2- 172.21.15.117:0/2305094060 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f51440390f0 0x7f514403b5b0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:11:55.328 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:55.326+0000 7f5166600700 1 --2- 172.21.15.117:0/2305094060 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51601020e0 0x7f5160074790 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:11:55.328 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:55.326+0000 7f5166600700 1 -- 172.21.15.117:0/2305094060 >> 172.21.15.117:0/2305094060 conn(0x7f51600fd9b0 msgr2=0x7f5160103ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:11:55.328 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:55.326+0000 7f5166600700 1 -- 172.21.15.117:0/2305094060 shutdown_connections 2024-06-03T03:11:55.328 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:55.326+0000 7f5166600700 1 -- 172.21.15.117:0/2305094060 wait complete. 2024-06-03T03:11:56.082 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:55 smithi117 ceph-mon[31649]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:11:56.083 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:55 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:11:56.139 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi183 2024-06-03T03:11:56.139 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-06-03T03:11:56.139 DEBUG:teuthology.orchestra.run.smithi183:> dd of=/etc/ceph/ceph.conf 2024-06-03T03:11:56.158 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-06-03T03:11:56.158 DEBUG:teuthology.orchestra.run.smithi183:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-03T03:11:56.215 INFO:tasks.cephadm:Adding host smithi183 to orchestrator... 2024-06-03T03:11:56.215 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph orch host add smithi183 2024-06-03T03:11:56.649 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:11:56.972 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:56 smithi117 ceph-mon[31649]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-03T03:11:57.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:57 smithi117 ceph-mon[31649]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:11:58.788 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:58.786+0000 7ffbaa000700 1 -- 172.21.15.117:0/2880726957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffba4101f30 msgr2=0x7ffba4102310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:11:58.789 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:58.786+0000 7ffbaa000700 1 --2- 172.21.15.117:0/2880726957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffba4101f30 0x7ffba4102310 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ffb98009910 tx=0x7ffb98009c20 comp rx=0 tx=0).stop 2024-06-03T03:11:58.789 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:58.786+0000 7ffbaa000700 1 -- 172.21.15.117:0/2880726957 shutdown_connections 2024-06-03T03:11:58.789 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:58.786+0000 7ffbaa000700 1 --2- 172.21.15.117:0/2880726957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffba4101f30 0x7ffba4102310 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:11:58.789 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:58.786+0000 7ffbaa000700 1 -- 172.21.15.117:0/2880726957 >> 172.21.15.117:0/2880726957 conn(0x7ffba40fd820 msgr2=0x7ffba40ffc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:11:58.789 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:58.786+0000 7ffbaa000700 1 -- 172.21.15.117:0/2880726957 shutdown_connections 2024-06-03T03:11:58.789 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:58.786+0000 7ffbaa000700 1 -- 172.21.15.117:0/2880726957 wait complete. 2024-06-03T03:11:58.789 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:58.787+0000 7ffbaa000700 1 Processor -- start 2024-06-03T03:11:58.789 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:58.787+0000 7ffbaa000700 1 -- start start 2024-06-03T03:11:58.790 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:58.787+0000 7ffbaa000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffba4101f30 0x7ffba4074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:11:58.791 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:58.787+0000 7ffbaa000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ffba410c300 con 0x7ffba4101f30 2024-06-03T03:11:58.791 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:58.788+0000 7ffba8c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffba4101f30 0x7ffba4074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:11:58.791 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:58.788+0000 7ffba8c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffba4101f30 0x7ffba4074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50210/0 (socket says 172.21.15.117:50210) 2024-06-03T03:11:58.791 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:58.788+0000 7ffba8c00700 1 -- 172.21.15.117:0/1189964332 learned_addr learned my addr 172.21.15.117:0/1189964332 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:11:58.791 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:58.788+0000 7ffba8c00700 1 -- 172.21.15.117:0/1189964332 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffba4076680 con 0x7ffba4101f30 2024-06-03T03:11:58.791 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:58.789+0000 7ffba8c00700 1 --2- 172.21.15.117:0/1189964332 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffba4101f30 0x7ffba4074790 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ffb98006010 tx=0x7ffb98005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:11:58.791 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:58.789+0000 7ffba1600700 1 -- 172.21.15.117:0/1189964332 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb98026070 con 0x7ffba4101f30 2024-06-03T03:11:58.791 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:58.789+0000 7ffba1600700 1 -- 172.21.15.117:0/1189964332 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb9800bc30 con 0x7ffba4101f30 2024-06-03T03:11:58.791 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:58.789+0000 7ffbaa000700 1 -- 172.21.15.117:0/1189964332 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffba4074d30 con 0x7ffba4101f30 2024-06-03T03:11:58.791 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:58.789+0000 7ffba1600700 1 -- 172.21.15.117:0/1189964332 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb980039d0 con 0x7ffba4101f30 2024-06-03T03:11:58.791 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:58.789+0000 7ffbaa000700 1 -- 172.21.15.117:0/1189964332 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffba40751f0 con 0x7ffba4101f30 2024-06-03T03:11:58.792 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:58.790+0000 7ffba1600700 1 -- 172.21.15.117:0/1189964332 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7ffb98029070 con 0x7ffba4101f30 2024-06-03T03:11:58.792 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:58.790+0000 7ffbaa000700 1 -- 172.21.15.117:0/1189964332 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffba4103410 con 0x7ffba4101f30 2024-06-03T03:11:58.792 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:58.790+0000 7ffba1600700 1 --2- 172.21.15.117:0/1189964332 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7ffb8c039150 0x7ffb8c03b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:11:58.792 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:58.791+0000 7ffba1600700 1 -- 172.21.15.117:0/1189964332 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ffb9801e070 con 0x7ffba4101f30 2024-06-03T03:11:58.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:58.791+0000 7ffba3e00700 1 --2- 172.21.15.117:0/1189964332 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7ffb8c039150 0x7ffb8c03b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:11:58.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:58.791+0000 7ffba3e00700 1 --2- 172.21.15.117:0/1189964332 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7ffb8c039150 0x7ffb8c03b610 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ffb90006fd0 tx=0x7ffb90006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:11:58.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:58.798+0000 7ffba1600700 1 -- 172.21.15.117:0/1189964332 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ffb98050030 con 0x7ffba4101f30 2024-06-03T03:11:59.034 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:11:59.032+0000 7ffbaa000700 1 -- 172.21.15.117:0/1189964332 --> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi183", "target": ["mon-mgr", ""]}) -- 0x7ffba4075cc0 con 0x7ffb8c039150 2024-06-03T03:11:59.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:58 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:11:59.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:58 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:11:59.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:58 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:11:59.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:58 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:11:59.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:58 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi151", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:11:59.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:58 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:11:59.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:58 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:11:59.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:58 smithi117 ceph-mon[31649]: Updating smithi151:/etc/ceph/ceph.conf 2024-06-03T03:11:59.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:58 smithi117 ceph-mon[31649]: Updating smithi151:/var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-03T03:11:59.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:58 smithi117 ceph-mon[31649]: Updating smithi151:/etc/ceph/ceph.client.admin.keyring 2024-06-03T03:11:59.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:58 smithi117 ceph-mon[31649]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:11:59.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:11:58 smithi117 ceph-mon[31649]: Updating smithi151:/var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-03T03:12:00.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:00 smithi117 ceph-mon[31649]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi183", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:12:00.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:00 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:00.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:00 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:00.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:00 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:00.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:00 smithi117 ceph-mon[31649]: Deploying cephadm binary to smithi183 2024-06-03T03:12:01.696 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:01 smithi117 ceph-mon[31649]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:03.690 INFO:teuthology.orchestra.run.smithi117.stdout:Added host 'smithi183' with addr '172.21.15.183' 2024-06-03T03:12:03.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:03.689+0000 7ffba1600700 1 -- 172.21.15.117:0/1189964332 <== mgr.14152 v2:172.21.15.117:6800/1398160887 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7ffba4075cc0 con 0x7ffb8c039150 2024-06-03T03:12:03.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:03.693+0000 7ffbaa000700 1 -- 172.21.15.117:0/1189964332 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7ffb8c039150 msgr2=0x7ffb8c03b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:12:03.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:03.693+0000 7ffbaa000700 1 --2- 172.21.15.117:0/1189964332 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7ffb8c039150 0x7ffb8c03b610 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ffb90006fd0 tx=0x7ffb90006e40 comp rx=0 tx=0).stop 2024-06-03T03:12:03.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:03.693+0000 7ffbaa000700 1 -- 172.21.15.117:0/1189964332 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffba4101f30 msgr2=0x7ffba4074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:12:03.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:03.693+0000 7ffbaa000700 1 --2- 172.21.15.117:0/1189964332 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffba4101f30 0x7ffba4074790 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ffb98006010 tx=0x7ffb98005030 comp rx=0 tx=0).stop 2024-06-03T03:12:03.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:03.693+0000 7ffbaa000700 1 -- 172.21.15.117:0/1189964332 shutdown_connections 2024-06-03T03:12:03.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:03.693+0000 7ffbaa000700 1 --2- 172.21.15.117:0/1189964332 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7ffb8c039150 0x7ffb8c03b610 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:12:03.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:03.693+0000 7ffbaa000700 1 --2- 172.21.15.117:0/1189964332 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffba4101f30 0x7ffba4074790 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:12:03.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:03.693+0000 7ffbaa000700 1 -- 172.21.15.117:0/1189964332 >> 172.21.15.117:0/1189964332 conn(0x7ffba40fd820 msgr2=0x7ffba4104d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:12:03.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:03.693+0000 7ffbaa000700 1 -- 172.21.15.117:0/1189964332 shutdown_connections 2024-06-03T03:12:03.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:03.693+0000 7ffbaa000700 1 -- 172.21.15.117:0/1189964332 wait complete. 2024-06-03T03:12:03.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:03 smithi117 ceph-mon[31649]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:03.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:03 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:03.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:03 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:12:04.478 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph orch host ls --format=json 2024-06-03T03:12:04.913 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:12:05.053 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:04 smithi117 ceph-mon[31649]: Added host smithi183 2024-06-03T03:12:05.053 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:04 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:06.030 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:05 smithi117 ceph-mon[31649]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:06.031 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:05 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:06.887 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:06.886+0000 7f5483e00700 1 -- 172.21.15.117:0/141079179 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f547c103090 msgr2=0x7f547c103470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:12:06.888 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:06.886+0000 7f5483e00700 1 --2- 172.21.15.117:0/141079179 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f547c103090 0x7f547c103470 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5470009910 tx=0x7f5470009c20 comp rx=0 tx=0).stop 2024-06-03T03:12:06.888 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:06.886+0000 7f5483e00700 1 -- 172.21.15.117:0/141079179 shutdown_connections 2024-06-03T03:12:06.888 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:06.886+0000 7f5483e00700 1 --2- 172.21.15.117:0/141079179 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f547c103090 0x7f547c103470 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:12:06.888 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:06.886+0000 7f5483e00700 1 -- 172.21.15.117:0/141079179 >> 172.21.15.117:0/141079179 conn(0x7f547c077a60 msgr2=0x7f547c0fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:12:06.888 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:06.887+0000 7f5483e00700 1 -- 172.21.15.117:0/141079179 shutdown_connections 2024-06-03T03:12:06.888 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:06.887+0000 7f5483e00700 1 -- 172.21.15.117:0/141079179 wait complete. 2024-06-03T03:12:06.889 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:06.888+0000 7f5483e00700 1 Processor -- start 2024-06-03T03:12:06.889 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:06.888+0000 7f5483e00700 1 -- start start 2024-06-03T03:12:06.889 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:06.888+0000 7f5483e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f547c103090 0x7f547c100cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:12:06.889 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:06.888+0000 7f5483e00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f547c10c510 con 0x7f547c103090 2024-06-03T03:12:06.890 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:06.888+0000 7f5481400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f547c103090 0x7f547c100cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:12:06.890 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:06.888+0000 7f5481400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f547c103090 0x7f547c100cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40040/0 (socket says 172.21.15.117:40040) 2024-06-03T03:12:06.890 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:06.888+0000 7f5481400700 1 -- 172.21.15.117:0/2034209358 learned_addr learned my addr 172.21.15.117:0/2034209358 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:12:06.890 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:06.889+0000 7f5481400700 1 -- 172.21.15.117:0/2034209358 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f547c102be0 con 0x7f547c103090 2024-06-03T03:12:06.891 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:06.889+0000 7f5481400700 1 --2- 172.21.15.117:0/2034209358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f547c103090 0x7f547c100cf0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5470006010 tx=0x7f5470005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:12:06.891 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:06.889+0000 7f547a000700 1 -- 172.21.15.117:0/2034209358 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5470026070 con 0x7f547c103090 2024-06-03T03:12:06.891 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:06.889+0000 7f547a000700 1 -- 172.21.15.117:0/2034209358 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f547000bc30 con 0x7f547c103090 2024-06-03T03:12:06.891 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:06.889+0000 7f5483e00700 1 -- 172.21.15.117:0/2034209358 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f547c101230 con 0x7f547c103090 2024-06-03T03:12:06.891 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:06.890+0000 7f547a000700 1 -- 172.21.15.117:0/2034209358 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f54700039a0 con 0x7f547c103090 2024-06-03T03:12:06.891 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:06.890+0000 7f5483e00700 1 -- 172.21.15.117:0/2034209358 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f547c1016f0 con 0x7f547c103090 2024-06-03T03:12:06.891 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:06.891+0000 7f5483e00700 1 -- 172.21.15.117:0/2034209358 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f547c06a140 con 0x7f547c103090 2024-06-03T03:12:06.891 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:06.891+0000 7f547a000700 1 -- 172.21.15.117:0/2034209358 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f5470029070 con 0x7f547c103090 2024-06-03T03:12:06.892 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:06.891+0000 7f547a000700 1 --2- 172.21.15.117:0/2034209358 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f54640390f0 0x7f546403b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:12:06.892 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:06.891+0000 7f547a000700 1 -- 172.21.15.117:0/2034209358 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f547001e070 con 0x7f547c103090 2024-06-03T03:12:06.892 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:06.892+0000 7f5480a00700 1 --2- 172.21.15.117:0/2034209358 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f54640390f0 0x7f546403b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:12:06.893 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:06.892+0000 7f5480a00700 1 --2- 172.21.15.117:0/2034209358 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f54640390f0 0x7f546403b5b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5468006fd0 tx=0x7f5468006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:12:06.897 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:06.897+0000 7f547a000700 1 -- 172.21.15.117:0/2034209358 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5470013070 con 0x7f547c103090 2024-06-03T03:12:07.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:07.125+0000 7f5483e00700 1 -- 172.21.15.117:0/2034209358 --> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f547c102220 con 0x7f54640390f0 2024-06-03T03:12:07.128 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:07.127+0000 7f547a000700 1 -- 172.21.15.117:0/2034209358 <== mgr.14152 v2:172.21.15.117:6800/1398160887 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+241 (secure 0 0 0) 0x7f547c102220 con 0x7f54640390f0 2024-06-03T03:12:07.128 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T03:12:07.128 INFO:teuthology.orchestra.run.smithi117.stdout:[{"addr": "172.21.15.117", "hostname": "smithi117", "labels": [], "status": ""}, {"addr": "172.21.15.151", "hostname": "smithi151", "labels": [], "status": ""}, {"addr": "172.21.15.183", "hostname": "smithi183", "labels": [], "status": ""}] 2024-06-03T03:12:07.132 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:07.130+0000 7f5483e00700 1 -- 172.21.15.117:0/2034209358 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f54640390f0 msgr2=0x7f546403b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:12:07.132 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:07.130+0000 7f5483e00700 1 --2- 172.21.15.117:0/2034209358 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f54640390f0 0x7f546403b5b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5468006fd0 tx=0x7f5468006e40 comp rx=0 tx=0).stop 2024-06-03T03:12:07.132 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:07.131+0000 7f5483e00700 1 -- 172.21.15.117:0/2034209358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f547c103090 msgr2=0x7f547c100cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:12:07.132 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:07.131+0000 7f5483e00700 1 --2- 172.21.15.117:0/2034209358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f547c103090 0x7f547c100cf0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5470006010 tx=0x7f5470005030 comp rx=0 tx=0).stop 2024-06-03T03:12:07.132 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:07.131+0000 7f5483e00700 1 -- 172.21.15.117:0/2034209358 shutdown_connections 2024-06-03T03:12:07.132 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:07.131+0000 7f5483e00700 1 --2- 172.21.15.117:0/2034209358 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f54640390f0 0x7f546403b5b0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:12:07.132 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:07.131+0000 7f5483e00700 1 --2- 172.21.15.117:0/2034209358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f547c103090 0x7f547c100cf0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:12:07.132 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:07.131+0000 7f5483e00700 1 -- 172.21.15.117:0/2034209358 >> 172.21.15.117:0/2034209358 conn(0x7f547c077a60 msgr2=0x7f547c103e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:12:07.132 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:07.131+0000 7f5483e00700 1 -- 172.21.15.117:0/2034209358 shutdown_connections 2024-06-03T03:12:07.132 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:07.131+0000 7f5483e00700 1 -- 172.21.15.117:0/2034209358 wait complete. 2024-06-03T03:12:07.696 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:07 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:07.696 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:07 smithi117 ceph-mon[31649]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:07.946 INFO:tasks.cephadm:Setting crush tunables to default 2024-06-03T03:12:07.947 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph osd crush tunables default 2024-06-03T03:12:08.381 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:12:08.696 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:08 smithi117 ceph-mon[31649]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-03T03:12:09.759 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:09 smithi117 ceph-mon[31649]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:10.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:10.633+0000 7fbc25a00700 1 -- 172.21.15.117:0/3207509429 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc201042e0 msgr2=0x7fbc201046c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:12:10.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:10.633+0000 7fbc25a00700 1 --2- 172.21.15.117:0/3207509429 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc201042e0 0x7fbc201046c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbc10009910 tx=0x7fbc10009c20 comp rx=0 tx=0).stop 2024-06-03T03:12:10.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:10.633+0000 7fbc25a00700 1 -- 172.21.15.117:0/3207509429 shutdown_connections 2024-06-03T03:12:10.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:10.633+0000 7fbc25a00700 1 --2- 172.21.15.117:0/3207509429 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc201042e0 0x7fbc201046c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:12:10.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:10.634+0000 7fbc25a00700 1 -- 172.21.15.117:0/3207509429 >> 172.21.15.117:0/3207509429 conn(0x7fbc200ffbb0 msgr2=0x7fbc20101fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:12:10.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:10.634+0000 7fbc25a00700 1 -- 172.21.15.117:0/3207509429 shutdown_connections 2024-06-03T03:12:10.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:10.634+0000 7fbc25a00700 1 -- 172.21.15.117:0/3207509429 wait complete. 2024-06-03T03:12:10.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:10.635+0000 7fbc25a00700 1 Processor -- start 2024-06-03T03:12:10.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:10.635+0000 7fbc25a00700 1 -- start start 2024-06-03T03:12:10.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:10.635+0000 7fbc25a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc201042e0 0x7fbc2019f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:12:10.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:10.635+0000 7fbc25a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbc2010e6b0 con 0x7fbc201042e0 2024-06-03T03:12:10.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:10.636+0000 7fbc1ea00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc201042e0 0x7fbc2019f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:12:10.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:10.636+0000 7fbc1ea00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc201042e0 0x7fbc2019f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40062/0 (socket says 172.21.15.117:40062) 2024-06-03T03:12:10.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:10.636+0000 7fbc1ea00700 1 -- 172.21.15.117:0/20444315 learned_addr learned my addr 172.21.15.117:0/20444315 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:12:10.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:10.636+0000 7fbc1ea00700 1 -- 172.21.15.117:0/20444315 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc2019fe00 con 0x7fbc201042e0 2024-06-03T03:12:10.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:10.636+0000 7fbc1ea00700 1 --2- 172.21.15.117:0/20444315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc201042e0 0x7fbc2019f8c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbc10006010 tx=0x7fbc10005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:12:10.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:10.637+0000 7fbc17e00700 1 -- 172.21.15.117:0/20444315 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc10026070 con 0x7fbc201042e0 2024-06-03T03:12:10.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:10.637+0000 7fbc17e00700 1 -- 172.21.15.117:0/20444315 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc1000bbd0 con 0x7fbc201042e0 2024-06-03T03:12:10.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:10.637+0000 7fbc25a00700 1 -- 172.21.15.117:0/20444315 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc201a00f0 con 0x7fbc201042e0 2024-06-03T03:12:10.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:10.637+0000 7fbc17e00700 1 -- 172.21.15.117:0/20444315 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc10003910 con 0x7fbc201042e0 2024-06-03T03:12:10.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:10.637+0000 7fbc25a00700 1 -- 172.21.15.117:0/20444315 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc20199a80 con 0x7fbc201042e0 2024-06-03T03:12:10.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:10.638+0000 7fbc17e00700 1 -- 172.21.15.117:0/20444315 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fbc10029070 con 0x7fbc201042e0 2024-06-03T03:12:10.639 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:10.638+0000 7fbc25a00700 1 -- 172.21.15.117:0/20444315 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc201057c0 con 0x7fbc201042e0 2024-06-03T03:12:10.639 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:10.639+0000 7fbc17e00700 1 --2- 172.21.15.117:0/20444315 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fbc040390f0 0x7fbc0403b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:12:10.639 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:10.639+0000 7fbc17e00700 1 -- 172.21.15.117:0/20444315 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fbc1001e070 con 0x7fbc201042e0 2024-06-03T03:12:10.639 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:10.639+0000 7fbc16000700 1 --2- 172.21.15.117:0/20444315 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fbc040390f0 0x7fbc0403b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:12:10.640 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:10.640+0000 7fbc16000700 1 --2- 172.21.15.117:0/20444315 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fbc040390f0 0x7fbc0403b5b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fbc08006fd0 tx=0x7fbc08006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:12:10.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:10.644+0000 7fbc17e00700 1 -- 172.21.15.117:0/20444315 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fbc10013070 con 0x7fbc201042e0 2024-06-03T03:12:10.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:10.876+0000 7fbc25a00700 1 -- 172.21.15.117:0/20444315 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fbc2019afb0 con 0x7fbc201042e0 2024-06-03T03:12:10.877 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:10 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:10.878 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:10 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:10.878 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:10 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:10.878 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:10 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:10.878 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:10 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi183", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:12:10.878 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:10 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:12:10.878 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:10 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:12:10.878 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:10 smithi117 ceph-mon[31649]: Updating smithi183:/etc/ceph/ceph.conf 2024-06-03T03:12:11.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:11.680+0000 7fbc17e00700 1 -- 172.21.15.117:0/20444315 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7fbc10030090 con 0x7fbc201042e0 2024-06-03T03:12:11.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:11.684+0000 7fbc25a00700 1 -- 172.21.15.117:0/20444315 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fbc040390f0 msgr2=0x7fbc0403b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:12:11.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:11.684+0000 7fbc25a00700 1 --2- 172.21.15.117:0/20444315 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fbc040390f0 0x7fbc0403b5b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fbc08006fd0 tx=0x7fbc08006e40 comp rx=0 tx=0).stop 2024-06-03T03:12:11.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:11.684+0000 7fbc25a00700 1 -- 172.21.15.117:0/20444315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc201042e0 msgr2=0x7fbc2019f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:12:11.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:11.684+0000 7fbc25a00700 1 --2- 172.21.15.117:0/20444315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc201042e0 0x7fbc2019f8c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbc10006010 tx=0x7fbc10005030 comp rx=0 tx=0).stop 2024-06-03T03:12:11.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:11.684+0000 7fbc25a00700 1 -- 172.21.15.117:0/20444315 shutdown_connections 2024-06-03T03:12:11.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:11.684+0000 7fbc25a00700 1 --2- 172.21.15.117:0/20444315 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fbc040390f0 0x7fbc0403b5b0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:12:11.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:11.684+0000 7fbc25a00700 1 --2- 172.21.15.117:0/20444315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc201042e0 0x7fbc2019f8c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:12:11.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:11.684+0000 7fbc25a00700 1 -- 172.21.15.117:0/20444315 >> 172.21.15.117:0/20444315 conn(0x7fbc200ffbb0 msgr2=0x7fbc20107110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:12:11.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:11.684+0000 7fbc25a00700 1 -- 172.21.15.117:0/20444315 shutdown_connections 2024-06-03T03:12:11.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:11.684+0000 7fbc25a00700 1 -- 172.21.15.117:0/20444315 wait complete. 2024-06-03T03:12:11.686 INFO:teuthology.orchestra.run.smithi117.stderr:adjusted tunables profile to default 2024-06-03T03:12:11.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:11 smithi117 ceph-mon[31649]: Updating smithi183:/var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-03T03:12:11.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:11 smithi117 ceph-mon[31649]: Updating smithi183:/etc/ceph/ceph.client.admin.keyring 2024-06-03T03:12:11.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:11 smithi117 ceph-mon[31649]: Updating smithi183:/var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-03T03:12:11.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:11 smithi117 ceph-mon[31649]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:11.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:11 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/20444315' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-06-03T03:12:11.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:11 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:11.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:11 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:11.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:11 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:12.395 INFO:tasks.cephadm:Adding mon.a on smithi117 2024-06-03T03:12:12.395 INFO:tasks.cephadm:Adding mon.b on smithi151 2024-06-03T03:12:12.395 INFO:tasks.cephadm:Adding mon.c on smithi183 2024-06-03T03:12:12.395 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph orch apply mon '3;smithi117:172.21.15.117=a;smithi151:172.21.15.151=b;smithi183:172.21.15.183=c' 2024-06-03T03:12:12.632 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-03T03:12:12.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:12 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/20444315' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-06-03T03:12:12.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:12 smithi117 ceph-mon[31649]: osdmap e4: 0 total, 0 up, 0 in 2024-06-03T03:12:13.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:13 smithi117 ceph-mon[31649]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:14.132 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:14.131+0000 7f7a54941700 1 -- 172.21.15.183:0/3740700518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a501020d0 msgr2=0x7f7a501024b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:12:14.132 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:14.131+0000 7f7a54941700 1 --2- 172.21.15.183:0/3740700518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a501020d0 0x7f7a501024b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7a40009910 tx=0x7f7a40009c20 comp rx=0 tx=0).stop 2024-06-03T03:12:14.133 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:14.131+0000 7f7a54941700 1 -- 172.21.15.183:0/3740700518 shutdown_connections 2024-06-03T03:12:14.133 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:14.131+0000 7f7a54941700 1 --2- 172.21.15.183:0/3740700518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a501020d0 0x7f7a501024b0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:12:14.133 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:14.131+0000 7f7a54941700 1 -- 172.21.15.183:0/3740700518 >> 172.21.15.183:0/3740700518 conn(0x7f7a500fd9a0 msgr2=0x7f7a500ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:12:14.133 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:14.131+0000 7f7a54941700 1 -- 172.21.15.183:0/3740700518 shutdown_connections 2024-06-03T03:12:14.133 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:14.131+0000 7f7a54941700 1 -- 172.21.15.183:0/3740700518 wait complete. 2024-06-03T03:12:14.134 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:14.132+0000 7f7a54941700 1 Processor -- start 2024-06-03T03:12:14.134 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:14.132+0000 7f7a54941700 1 -- start start 2024-06-03T03:12:14.134 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:14.132+0000 7f7a54941700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a501020d0 0x7f7a50074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:12:14.134 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:14.132+0000 7f7a54941700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7a5010c4a0 con 0x7f7a501020d0 2024-06-03T03:12:14.134 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:14.132+0000 7f7a4e59c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a501020d0 0x7f7a50074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:12:14.134 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:14.133+0000 7f7a4e59c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a501020d0 0x7f7a50074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.183:40130/0 (socket says 172.21.15.183:40130) 2024-06-03T03:12:14.134 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:14.133+0000 7f7a4e59c700 1 -- 172.21.15.183:0/1871965207 learned_addr learned my addr 172.21.15.183:0/1871965207 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-06-03T03:12:14.135 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:14.133+0000 7f7a4e59c700 1 -- 172.21.15.183:0/1871965207 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a50076680 con 0x7f7a501020d0 2024-06-03T03:12:14.135 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:14.133+0000 7f7a4e59c700 1 --2- 172.21.15.183:0/1871965207 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a501020d0 0x7f7a50074790 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7a40005850 tx=0x7f7a40005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:12:14.135 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:14.134+0000 7f7a3f7fe700 1 -- 172.21.15.183:0/1871965207 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a40026070 con 0x7f7a501020d0 2024-06-03T03:12:14.136 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:14.134+0000 7f7a54941700 1 -- 172.21.15.183:0/1871965207 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a50074d30 con 0x7f7a501020d0 2024-06-03T03:12:14.136 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:14.134+0000 7f7a54941700 1 -- 172.21.15.183:0/1871965207 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a500751f0 con 0x7f7a501020d0 2024-06-03T03:12:14.136 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:14.134+0000 7f7a3f7fe700 1 -- 172.21.15.183:0/1871965207 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a4000b6f0 con 0x7f7a501020d0 2024-06-03T03:12:14.136 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:14.134+0000 7f7a3f7fe700 1 -- 172.21.15.183:0/1871965207 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a40030060 con 0x7f7a501020d0 2024-06-03T03:12:14.136 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:14.135+0000 7f7a54941700 1 -- 172.21.15.183:0/1871965207 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a30005330 con 0x7f7a501020d0 2024-06-03T03:12:14.137 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:14.135+0000 7f7a3f7fe700 1 -- 172.21.15.183:0/1871965207 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f7a40029070 con 0x7f7a501020d0 2024-06-03T03:12:14.137 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:14.136+0000 7f7a3f7fe700 1 --2- 172.21.15.183:0/1871965207 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f7a380390a0 0x7f7a3803b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:12:14.137 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:14.136+0000 7f7a3f7fe700 1 -- 172.21.15.183:0/1871965207 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f7a4001e070 con 0x7f7a501020d0 2024-06-03T03:12:14.137 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:14.136+0000 7f7a4dd9b700 1 --2- 172.21.15.183:0/1871965207 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f7a380390a0 0x7f7a3803b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:12:14.138 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:14.137+0000 7f7a4dd9b700 1 --2- 172.21.15.183:0/1871965207 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f7a380390a0 0x7f7a3803b560 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7a44006fd0 tx=0x7f7a44006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:12:14.152 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:14.151+0000 7f7a3f7fe700 1 -- 172.21.15.183:0/1871965207 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7a40013070 con 0x7f7a501020d0 2024-06-03T03:12:14.358 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:14.356+0000 7f7a54941700 1 -- 172.21.15.183:0/1871965207 --> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi117:172.21.15.117=a;smithi151:172.21.15.151=b;smithi183:172.21.15.183=c", "target": ["mon-mgr", ""]}) -- 0x7f7a30000c80 con 0x7f7a380390a0 2024-06-03T03:12:14.372 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:14.371+0000 7f7a3f7fe700 1 -- 172.21.15.183:0/1871965207 <== mgr.14152 v2:172.21.15.117:6800/1398160887 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f7a30000c80 con 0x7f7a380390a0 2024-06-03T03:12:14.372 INFO:teuthology.orchestra.run.smithi183.stdout:Scheduled mon update... 2024-06-03T03:12:14.374 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:14.373+0000 7f7a54941700 1 -- 172.21.15.183:0/1871965207 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f7a380390a0 msgr2=0x7f7a3803b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:12:14.375 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:14.373+0000 7f7a54941700 1 --2- 172.21.15.183:0/1871965207 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f7a380390a0 0x7f7a3803b560 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7a44006fd0 tx=0x7f7a44006e40 comp rx=0 tx=0).stop 2024-06-03T03:12:14.375 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:14.373+0000 7f7a54941700 1 -- 172.21.15.183:0/1871965207 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a501020d0 msgr2=0x7f7a50074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:12:14.375 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:14.373+0000 7f7a54941700 1 --2- 172.21.15.183:0/1871965207 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a501020d0 0x7f7a50074790 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7a40005850 tx=0x7f7a40005030 comp rx=0 tx=0).stop 2024-06-03T03:12:14.375 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:14.373+0000 7f7a54941700 1 -- 172.21.15.183:0/1871965207 shutdown_connections 2024-06-03T03:12:14.375 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:14.373+0000 7f7a54941700 1 --2- 172.21.15.183:0/1871965207 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f7a380390a0 0x7f7a3803b560 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:12:14.375 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:14.373+0000 7f7a54941700 1 --2- 172.21.15.183:0/1871965207 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a501020d0 0x7f7a50074790 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:12:14.375 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:14.373+0000 7f7a54941700 1 -- 172.21.15.183:0/1871965207 >> 172.21.15.183:0/1871965207 conn(0x7f7a500fd9a0 msgr2=0x7f7a50104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:12:14.375 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:14.373+0000 7f7a54941700 1 -- 172.21.15.183:0/1871965207 shutdown_connections 2024-06-03T03:12:14.375 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:14.373+0000 7f7a54941700 1 -- 172.21.15.183:0/1871965207 wait complete. 2024-06-03T03:12:14.967 DEBUG:teuthology.orchestra.run.smithi151:mon.b> sudo journalctl -f -n 0 -u ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@mon.b.service 2024-06-03T03:12:14.969 DEBUG:teuthology.orchestra.run.smithi183:mon.c> sudo journalctl -f -n 0 -u ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@mon.c.service 2024-06-03T03:12:14.972 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-06-03T03:12:14.972 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-06-03T03:12:15.687 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-03T03:12:15.697 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:15 smithi117 ceph-mon[31649]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi117:172.21.15.117=a;smithi151:172.21.15.151=b;smithi183:172.21.15.183=c", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:12:15.697 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:15 smithi117 ceph-mon[31649]: Saving service mon spec with placement smithi117:172.21.15.117=a;smithi151:172.21.15.151=b;smithi183:172.21.15.183=c;count:3 2024-06-03T03:12:15.697 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:15 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:15.697 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:15 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:12:15.697 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:15 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:12:15.697 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:15 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:12:15.697 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:15 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:15.697 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:15 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-03T03:12:15.697 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:15 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:12:15.697 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:15 smithi117 ceph-mon[31649]: Deploying daemon mon.c on smithi183 2024-06-03T03:12:15.697 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:15 smithi117 ceph-mon[31649]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:18.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:17 smithi117 ceph-mon[31649]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:18.571 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:18.549+0000 7f75f9e3b700 1 -- 172.21.15.183:0/3071752397 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f75f4100630 msgr2=0x7f75f4100a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:12:18.571 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:18.549+0000 7f75f9e3b700 1 --2- 172.21.15.183:0/3071752397 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f75f4100630 0x7f75f4100a10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f75e4009910 tx=0x7f75e4009c20 comp rx=0 tx=0).stop 2024-06-03T03:12:18.571 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:18.549+0000 7f75f9e3b700 1 -- 172.21.15.183:0/3071752397 shutdown_connections 2024-06-03T03:12:18.571 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:18.549+0000 7f75f9e3b700 1 --2- 172.21.15.183:0/3071752397 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f75f4100630 0x7f75f4100a10 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:12:18.571 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:18.549+0000 7f75f9e3b700 1 -- 172.21.15.183:0/3071752397 >> 172.21.15.183:0/3071752397 conn(0x7f75f40fbfd0 msgr2=0x7f75f40fe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:12:18.571 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:18.549+0000 7f75f9e3b700 1 -- 172.21.15.183:0/3071752397 shutdown_connections 2024-06-03T03:12:18.572 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:18.549+0000 7f75f9e3b700 1 -- 172.21.15.183:0/3071752397 wait complete. 2024-06-03T03:12:18.572 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:18.549+0000 7f75f9e3b700 1 Processor -- start 2024-06-03T03:12:18.572 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:18.550+0000 7f75f9e3b700 1 -- start start 2024-06-03T03:12:18.572 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:18.550+0000 7f75f9e3b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f75f4100630 0x7f75f41015b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:12:18.572 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:18.550+0000 7f75f9e3b700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f75f410c4c0 con 0x7f75f4100630 2024-06-03T03:12:18.572 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:18.550+0000 7f75f37fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f75f4100630 0x7f75f41015b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:12:18.572 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:18.550+0000 7f75f37fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f75f4100630 0x7f75f41015b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.183:40144/0 (socket says 172.21.15.183:40144) 2024-06-03T03:12:18.572 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:18.550+0000 7f75f37fe700 1 -- 172.21.15.183:0/129012457 learned_addr learned my addr 172.21.15.183:0/129012457 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-06-03T03:12:18.572 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:18.551+0000 7f75f37fe700 1 -- 172.21.15.183:0/129012457 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75f41034a0 con 0x7f75f4100630 2024-06-03T03:12:18.572 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:18.551+0000 7f75f37fe700 1 --2- 172.21.15.183:0/129012457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f75f4100630 0x7f75f41015b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f75e4006010 tx=0x7f75e4005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:12:18.572 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:18.552+0000 7f75f0ff9700 1 -- 172.21.15.183:0/129012457 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f75e4026070 con 0x7f75f4100630 2024-06-03T03:12:18.572 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:18.552+0000 7f75f9e3b700 1 -- 172.21.15.183:0/129012457 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75f4101b50 con 0x7f75f4100630 2024-06-03T03:12:18.572 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:18.552+0000 7f75f9e3b700 1 -- 172.21.15.183:0/129012457 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75f4102010 con 0x7f75f4100630 2024-06-03T03:12:18.573 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:18.552+0000 7f75f0ff9700 1 -- 172.21.15.183:0/129012457 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75e400b6f0 con 0x7f75f4100630 2024-06-03T03:12:18.573 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:18.552+0000 7f75f0ff9700 1 -- 172.21.15.183:0/129012457 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f75e4030060 con 0x7f75f4100630 2024-06-03T03:12:18.573 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:18.552+0000 7f75f9e3b700 1 -- 172.21.15.183:0/129012457 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75e0005330 con 0x7f75f4100630 2024-06-03T03:12:18.573 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:18.553+0000 7f75f0ff9700 1 -- 172.21.15.183:0/129012457 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f75e4029070 con 0x7f75f4100630 2024-06-03T03:12:18.573 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:18.553+0000 7f75f0ff9700 1 --2- 172.21.15.183:0/129012457 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f75dc039150 0x7f75dc03b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:12:18.573 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:18.553+0000 7f75f0ff9700 1 -- 172.21.15.183:0/129012457 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f75e401e070 con 0x7f75f4100630 2024-06-03T03:12:18.573 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:18.554+0000 7f75f2ffd700 1 --2- 172.21.15.183:0/129012457 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f75dc039150 0x7f75dc03b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:12:18.573 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:18.555+0000 7f75f2ffd700 1 --2- 172.21.15.183:0/129012457 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f75dc039150 0x7f75dc03b610 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f75e8006fd0 tx=0x7f75e8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:12:18.573 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:18.567+0000 7f75f0ff9700 1 -- 172.21.15.183:0/129012457 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f75e4013070 con 0x7f75f4100630 2024-06-03T03:12:18.878 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:18.876+0000 7f75f9e3b700 1 -- 172.21.15.183:0/129012457 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f75e00058f0 con 0x7f75f4100630 2024-06-03T03:12:18.878 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:18.877+0000 7f75f0ff9700 1 -- 172.21.15.183:0/129012457 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+749 (secure 0 0 0) 0x7f75e400e840 con 0x7f75f4100630 2024-06-03T03:12:18.878 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-06-03T03:12:18.879 INFO:teuthology.orchestra.run.smithi183.stdout:{"epoch":1,"fsid":"a0e8473a-2156-11ef-bc9b-c7b262605968","modified":"2024-06-03T03:10:04.002179Z","created":"2024-06-03T03:10:04.002179Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:3300","nonce":0},{"type":"v1","addr":"172.21.15.117:6789","nonce":0}]},"addr":"172.21.15.117:6789/0","public_addr":"172.21.15.117:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-06-03T03:12:18.881 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:18.879+0000 7f75f9e3b700 1 -- 172.21.15.183:0/129012457 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f75dc039150 msgr2=0x7f75dc03b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:12:18.881 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:18.879+0000 7f75f9e3b700 1 --2- 172.21.15.183:0/129012457 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f75dc039150 0x7f75dc03b610 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f75e8006fd0 tx=0x7f75e8006e40 comp rx=0 tx=0).stop 2024-06-03T03:12:18.881 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:18.879+0000 7f75f9e3b700 1 -- 172.21.15.183:0/129012457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f75f4100630 msgr2=0x7f75f41015b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:12:18.881 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:18.879+0000 7f75f9e3b700 1 --2- 172.21.15.183:0/129012457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f75f4100630 0x7f75f41015b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f75e4006010 tx=0x7f75e4005030 comp rx=0 tx=0).stop 2024-06-03T03:12:18.881 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:18.879+0000 7f75f9e3b700 1 -- 172.21.15.183:0/129012457 shutdown_connections 2024-06-03T03:12:18.881 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:18.879+0000 7f75f9e3b700 1 --2- 172.21.15.183:0/129012457 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f75dc039150 0x7f75dc03b610 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:12:18.881 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:18.879+0000 7f75f9e3b700 1 --2- 172.21.15.183:0/129012457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f75f4100630 0x7f75f41015b0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:12:18.881 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:18.879+0000 7f75f9e3b700 1 -- 172.21.15.183:0/129012457 >> 172.21.15.183:0/129012457 conn(0x7f75f40fbfd0 msgr2=0x7f75f4109a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:12:18.881 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:18.880+0000 7f75f9e3b700 1 -- 172.21.15.183:0/129012457 shutdown_connections 2024-06-03T03:12:18.881 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:18.880+0000 7f75f9e3b700 1 -- 172.21.15.183:0/129012457 wait complete. 2024-06-03T03:12:18.882 INFO:teuthology.orchestra.run.smithi183.stderr:dumped monmap epoch 1 2024-06-03T03:12:19.840 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: mon.c@-1(synchronizing).mds e1 new map 2024-06-03T03:12:19.841 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: mon.c@-1(synchronizing).mds e1 print_map 2024-06-03T03:12:19.841 INFO:journalctl@ceph.mon.c.smithi183.stdout: e1 2024-06-03T03:12:19.841 INFO:journalctl@ceph.mon.c.smithi183.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-06-03T03:12:19.841 INFO:journalctl@ceph.mon.c.smithi183.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-06-03T03:12:19.841 INFO:journalctl@ceph.mon.c.smithi183.stdout: legacy client fscid: -1 2024-06-03T03:12:19.841 INFO:journalctl@ceph.mon.c.smithi183.stdout: 2024-06-03T03:12:19.841 INFO:journalctl@ceph.mon.c.smithi183.stdout: No filesystems configured 2024-06-03T03:12:19.841 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-06-03T03:12:19.841 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-06-03T03:12:20.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:19 smithi117 ceph-mon[31649]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:20.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:19 smithi117 ceph-mon[31649]: from='client.? 172.21.15.183:0/129012457' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-03T03:12:20.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-06-03T03:12:20.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-06-03T03:12:20.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-06-03T03:12:20.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-06-03T03:12:20.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-06-03T03:12:20.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-06-03T03:12:20.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-06-03T03:12:20.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-06-03T03:12:20.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:20.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi151", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:12:20.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: Deploying cephadm binary to smithi151 2024-06-03T03:12:20.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:20.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:20.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:20.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: Added host smithi151 2024-06-03T03:12:20.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:12:20.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:20.234 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:20.234 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:20.234 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:20.234 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:20.234 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-03T03:12:20.234 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:20.234 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:20.234 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:20.234 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:20.234 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:20.234 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi151", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:12:20.234 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:12:20.234 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:12:20.235 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: Updating smithi151:/etc/ceph/ceph.conf 2024-06-03T03:12:20.235 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: Updating smithi151:/var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-03T03:12:20.235 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: Updating smithi151:/etc/ceph/ceph.client.admin.keyring 2024-06-03T03:12:20.235 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:20.235 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: Updating smithi151:/var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-03T03:12:20.235 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi183", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:12:20.235 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:20.236 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:20.236 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:20.236 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: Deploying cephadm binary to smithi183 2024-06-03T03:12:20.236 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:20.236 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:20.236 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:20.236 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:12:20.236 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: Added host smithi183 2024-06-03T03:12:20.236 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:20.236 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:20.236 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:20.236 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:20.237 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:20.237 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-03T03:12:20.237 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:20.237 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:20.237 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:20.237 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:20.237 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:20.237 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi183", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:12:20.237 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:12:20.237 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:12:20.237 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: Updating smithi183:/etc/ceph/ceph.conf 2024-06-03T03:12:20.237 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: Updating smithi183:/var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-03T03:12:20.237 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: Updating smithi183:/etc/ceph/ceph.client.admin.keyring 2024-06-03T03:12:20.237 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: Updating smithi183:/var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-03T03:12:20.238 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:20.238 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/20444315' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-06-03T03:12:20.238 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:20.238 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:20.238 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:20.238 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/20444315' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-06-03T03:12:20.238 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: osdmap e4: 0 total, 0 up, 0 in 2024-06-03T03:12:20.238 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:20.238 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi117:172.21.15.117=a;smithi151:172.21.15.151=b;smithi183:172.21.15.183=c", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:12:20.238 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: Saving service mon spec with placement smithi117:172.21.15.117=a;smithi151:172.21.15.151=b;smithi183:172.21.15.183=c;count:3 2024-06-03T03:12:20.238 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:20.238 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:12:20.238 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:12:20.239 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:12:20.239 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:20.239 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-03T03:12:20.239 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:12:20.239 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: Deploying daemon mon.c on smithi183 2024-06-03T03:12:20.239 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:20.239 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:20.239 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:20.239 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: from='client.? 172.21.15.183:0/129012457' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-03T03:12:20.239 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:19 smithi183 ceph-mon[37326]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-06-03T03:12:20.427 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-06-03T03:12:20.427 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-06-03T03:12:20.663 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.c/config 2024-06-03T03:12:25.189 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:25.188+0000 7fa92cd08700 1 -- 172.21.15.183:0/1783887441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa928102210 msgr2=0x7fa910005610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:12:25.190 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:25.188+0000 7fa92cd08700 1 --2- 172.21.15.183:0/1783887441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa928102210 0x7fa910005610 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa920009bf0 tx=0x7fa920009f00 comp rx=0 tx=0).stop 2024-06-03T03:12:25.190 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:25.188+0000 7fa92cd08700 1 -- 172.21.15.183:0/1783887441 shutdown_connections 2024-06-03T03:12:25.190 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:25.188+0000 7fa92cd08700 1 --2- 172.21.15.183:0/1783887441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa928102210 0x7fa910005610 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:12:25.190 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:25.188+0000 7fa92cd08700 1 -- 172.21.15.183:0/1783887441 >> 172.21.15.183:0/1783887441 conn(0x7fa9280fdaa0 msgr2=0x7fa9280ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:12:25.190 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:25.188+0000 7fa92cd08700 1 -- 172.21.15.183:0/1783887441 shutdown_connections 2024-06-03T03:12:25.190 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:25.188+0000 7fa92cd08700 1 -- 172.21.15.183:0/1783887441 wait complete. 2024-06-03T03:12:25.190 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:24 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-03T03:12:25.191 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:24 smithi183 ceph-mon[37326]: mon.a calling monitor election 2024-06-03T03:12:25.191 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:24 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T03:12:25.191 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:24 smithi183 ceph-mon[37326]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:25.191 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:24 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T03:12:25.191 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:24 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T03:12:25.191 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:24 smithi183 ceph-mon[37326]: mon.c calling monitor election 2024-06-03T03:12:25.191 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:24 smithi183 ceph-mon[37326]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:25.191 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:24 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T03:12:25.191 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:24 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T03:12:25.191 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:24 smithi183 ceph-mon[37326]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:25.191 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:24 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T03:12:25.191 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:24 smithi183 ceph-mon[37326]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-06-03T03:12:25.192 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:24 smithi183 ceph-mon[37326]: monmap epoch 2 2024-06-03T03:12:25.192 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:24 smithi183 ceph-mon[37326]: fsid a0e8473a-2156-11ef-bc9b-c7b262605968 2024-06-03T03:12:25.192 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:24 smithi183 ceph-mon[37326]: last_changed 2024-06-03T03:12:19.848370+0000 2024-06-03T03:12:25.192 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:24 smithi183 ceph-mon[37326]: created 2024-06-03T03:10:04.002179+0000 2024-06-03T03:12:25.192 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:24 smithi183 ceph-mon[37326]: min_mon_release 19 (squid) 2024-06-03T03:12:25.192 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:24 smithi183 ceph-mon[37326]: election_strategy: 1 2024-06-03T03:12:25.192 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:24 smithi183 ceph-mon[37326]: 0: [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] mon.a 2024-06-03T03:12:25.192 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:24 smithi183 ceph-mon[37326]: 1: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.c 2024-06-03T03:12:25.192 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:24 smithi183 ceph-mon[37326]: fsmap 2024-06-03T03:12:25.192 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:24 smithi183 ceph-mon[37326]: osdmap e4: 0 total, 0 up, 0 in 2024-06-03T03:12:25.192 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:24 smithi183 ceph-mon[37326]: mgrmap e13: x(active, since 60s) 2024-06-03T03:12:25.192 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:24 smithi183 ceph-mon[37326]: overall HEALTH_OK 2024-06-03T03:12:25.192 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:24 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:25.192 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:24 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:25.193 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:24 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:25.193 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:24 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-03T03:12:25.194 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:25.189+0000 7fa92cd08700 1 Processor -- start 2024-06-03T03:12:25.195 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:25.189+0000 7fa92cd08700 1 -- start start 2024-06-03T03:12:25.195 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:25.189+0000 7fa92cd08700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa928102210 0x7fa92819d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:12:25.195 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:25.189+0000 7fa92cd08700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa92819dc90 0x7fa928197960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:12:25.195 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:25.189+0000 7fa92cd08700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa92810a040 con 0x7fa928102210 2024-06-03T03:12:25.195 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:25.189+0000 7fa92cd08700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa928109ec0 con 0x7fa92819dc90 2024-06-03T03:12:25.195 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:25.189+0000 7fa925d9b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa92819dc90 0x7fa928197960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:12:25.195 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:25.189+0000 7fa925d9b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa92819dc90 0x7fa928197960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:48896/0 (socket says 172.21.15.183:48896) 2024-06-03T03:12:25.195 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:25.189+0000 7fa925d9b700 1 -- 172.21.15.183:0/723901051 learned_addr learned my addr 172.21.15.183:0/723901051 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-06-03T03:12:25.195 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:25.189+0000 7fa925d9b700 1 -- 172.21.15.183:0/723901051 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa92819dc90 msgr2=0x7fa928197960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-06-03T03:12:25.195 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:25.189+0000 7fa925d9b700 1 -- 172.21.15.183:0/723901051 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa92819dc90 msgr2=0x7fa928197960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-06-03T03:12:25.195 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:25.189+0000 7fa925d9b700 1 --2- 172.21.15.183:0/723901051 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa92819dc90 0x7fa928197960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-06-03T03:12:25.196 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:25.189+0000 7fa925d9b700 1 --2- 172.21.15.183:0/723901051 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa92819dc90 0x7fa928197960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-03T03:12:25.196 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:25.190+0000 7fa92659c700 1 --2- 172.21.15.183:0/723901051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa928102210 0x7fa92819d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:12:25.196 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:25.190+0000 7fa92659c700 1 -- 172.21.15.183:0/723901051 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa92819dc90 msgr2=0x7fa928197960 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-03T03:12:25.196 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:25.190+0000 7fa92659c700 1 --2- 172.21.15.183:0/723901051 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa92819dc90 0x7fa928197960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:12:25.196 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:25.190+0000 7fa92659c700 1 -- 172.21.15.183:0/723901051 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa92819e2d0 con 0x7fa928102210 2024-06-03T03:12:25.196 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:25.190+0000 7fa92659c700 1 --2- 172.21.15.183:0/723901051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa928102210 0x7fa92819d750 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa920009bf0 tx=0x7fa920003e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:12:25.196 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:25.191+0000 7fa9177fe700 1 -- 172.21.15.183:0/723901051 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fa920018c30 con 0x7fa928102210 2024-06-03T03:12:25.196 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:25.191+0000 7fa92cd08700 1 -- 172.21.15.183:0/723901051 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa928197ea0 con 0x7fa928102210 2024-06-03T03:12:25.196 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:25.191+0000 7fa9177fe700 1 -- 172.21.15.183:0/723901051 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa920018dd0 con 0x7fa928102210 2024-06-03T03:12:25.196 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:25.191+0000 7fa92cd08700 1 -- 172.21.15.183:0/723901051 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa928198430 con 0x7fa928102210 2024-06-03T03:12:25.196 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:25.191+0000 7fa9177fe700 1 -- 172.21.15.183:0/723901051 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fa920020a90 con 0x7fa928102210 2024-06-03T03:12:25.196 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:25.192+0000 7fa92cd08700 1 -- 172.21.15.183:0/723901051 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9281036f0 con 0x7fa928102210 2024-06-03T03:12:25.197 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:25.192+0000 7fa9177fe700 1 -- 172.21.15.183:0/723901051 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fa92001e070 con 0x7fa928102210 2024-06-03T03:12:25.197 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:25.193+0000 7fa9177fe700 1 --2- 172.21.15.183:0/723901051 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fa908038e90 0x7fa90803b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:12:25.197 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:25.193+0000 7fa9177fe700 1 -- 172.21.15.183:0/723901051 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fa92001c070 con 0x7fa928102210 2024-06-03T03:12:25.197 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:25.193+0000 7fa925d9b700 1 --2- 172.21.15.183:0/723901051 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fa908038e90 0x7fa90803b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:12:25.197 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:25.194+0000 7fa925d9b700 1 --2- 172.21.15.183:0/723901051 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fa908038e90 0x7fa90803b350 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa91c009180 tx=0x7fa91c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:12:25.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:24 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-03T03:12:25.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:24 smithi117 ceph-mon[31649]: mon.a calling monitor election 2024-06-03T03:12:25.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:24 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T03:12:25.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:24 smithi117 ceph-mon[31649]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:25.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:24 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T03:12:25.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:24 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T03:12:25.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:24 smithi117 ceph-mon[31649]: mon.c calling monitor election 2024-06-03T03:12:25.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:24 smithi117 ceph-mon[31649]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:25.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:24 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T03:12:25.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:24 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T03:12:25.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:24 smithi117 ceph-mon[31649]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:25.199 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:24 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T03:12:25.199 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:24 smithi117 ceph-mon[31649]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-06-03T03:12:25.199 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:24 smithi117 ceph-mon[31649]: monmap epoch 2 2024-06-03T03:12:25.199 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:24 smithi117 ceph-mon[31649]: fsid a0e8473a-2156-11ef-bc9b-c7b262605968 2024-06-03T03:12:25.199 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:24 smithi117 ceph-mon[31649]: last_changed 2024-06-03T03:12:19.848370+0000 2024-06-03T03:12:25.199 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:24 smithi117 ceph-mon[31649]: created 2024-06-03T03:10:04.002179+0000 2024-06-03T03:12:25.199 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:24 smithi117 ceph-mon[31649]: min_mon_release 19 (squid) 2024-06-03T03:12:25.199 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:24 smithi117 ceph-mon[31649]: election_strategy: 1 2024-06-03T03:12:25.199 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:24 smithi117 ceph-mon[31649]: 0: [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] mon.a 2024-06-03T03:12:25.199 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:24 smithi117 ceph-mon[31649]: 1: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.c 2024-06-03T03:12:25.199 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:24 smithi117 ceph-mon[31649]: fsmap 2024-06-03T03:12:25.199 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:24 smithi117 ceph-mon[31649]: osdmap e4: 0 total, 0 up, 0 in 2024-06-03T03:12:25.199 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:24 smithi117 ceph-mon[31649]: mgrmap e13: x(active, since 60s) 2024-06-03T03:12:25.200 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:24 smithi117 ceph-mon[31649]: overall HEALTH_OK 2024-06-03T03:12:25.200 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:24 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:25.200 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:24 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:25.200 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:24 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:25.200 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:24 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-03T03:12:25.210 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:25.210+0000 7fa9177fe700 1 -- 172.21.15.183:0/723901051 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa92001ede0 con 0x7fa928102210 2024-06-03T03:12:25.483 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:25.481+0000 7fa92cd08700 1 -- 172.21.15.183:0/723901051 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fa928198e40 con 0x7fa928102210 2024-06-03T03:12:25.483 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:25.482+0000 7fa9177fe700 1 -- 172.21.15.183:0/723901051 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1019 (secure 0 0 0) 0x7fa92004c9c0 con 0x7fa928102210 2024-06-03T03:12:25.484 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-06-03T03:12:25.484 INFO:teuthology.orchestra.run.smithi183.stdout:{"epoch":2,"fsid":"a0e8473a-2156-11ef-bc9b-c7b262605968","modified":"2024-06-03T03:12:19.848370Z","created":"2024-06-03T03:10:04.002179Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:3300","nonce":0},{"type":"v1","addr":"172.21.15.117:6789","nonce":0}]},"addr":"172.21.15.117:6789/0","public_addr":"172.21.15.117:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:3300","nonce":0},{"type":"v1","addr":"172.21.15.183:6789","nonce":0}]},"addr":"172.21.15.183:6789/0","public_addr":"172.21.15.183:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-06-03T03:12:25.487 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:25.485+0000 7fa92cd08700 1 -- 172.21.15.183:0/723901051 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fa908038e90 msgr2=0x7fa90803b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:12:25.487 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:25.485+0000 7fa92cd08700 1 --2- 172.21.15.183:0/723901051 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fa908038e90 0x7fa90803b350 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa91c009180 tx=0x7fa91c008040 comp rx=0 tx=0).stop 2024-06-03T03:12:25.487 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:25.485+0000 7fa92cd08700 1 -- 172.21.15.183:0/723901051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa928102210 msgr2=0x7fa92819d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:12:25.487 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:25.485+0000 7fa92cd08700 1 --2- 172.21.15.183:0/723901051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa928102210 0x7fa92819d750 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa920009bf0 tx=0x7fa920003e00 comp rx=0 tx=0).stop 2024-06-03T03:12:25.488 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:25.485+0000 7fa92cd08700 1 -- 172.21.15.183:0/723901051 shutdown_connections 2024-06-03T03:12:25.488 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:25.485+0000 7fa92cd08700 1 --2- 172.21.15.183:0/723901051 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fa908038e90 0x7fa90803b350 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:12:25.488 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:25.485+0000 7fa92cd08700 1 --2- 172.21.15.183:0/723901051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa928102210 0x7fa92819d750 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:12:25.488 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:25.485+0000 7fa92cd08700 1 --2- 172.21.15.183:0/723901051 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa92819dc90 0x7fa928197960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:12:25.488 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:25.485+0000 7fa92cd08700 1 -- 172.21.15.183:0/723901051 >> 172.21.15.183:0/723901051 conn(0x7fa9280fdaa0 msgr2=0x7fa928100e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:12:25.488 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:25.485+0000 7fa92cd08700 1 -- 172.21.15.183:0/723901051 shutdown_connections 2024-06-03T03:12:25.489 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:25.485+0000 7fa92cd08700 1 -- 172.21.15.183:0/723901051 wait complete. 2024-06-03T03:12:25.489 INFO:teuthology.orchestra.run.smithi183.stderr:dumped monmap epoch 2 2024-06-03T03:12:26.057 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:25 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:12:26.057 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:25 smithi183 ceph-mon[37326]: Deploying daemon mon.b on smithi151 2024-06-03T03:12:26.058 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:25 smithi183 ceph-mon[37326]: from='client.? 172.21.15.183:0/723901051' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-03T03:12:26.058 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:25 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T03:12:26.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:25 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:12:26.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:25 smithi117 ceph-mon[31649]: Deploying daemon mon.b on smithi151 2024-06-03T03:12:26.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:25 smithi117 ceph-mon[31649]: from='client.? 172.21.15.183:0/723901051' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-03T03:12:26.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:25 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T03:12:27.059 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-06-03T03:12:27.059 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-06-03T03:12:27.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:26 smithi117 ceph-mon[31649]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:27.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:26 smithi183 ceph-mon[37326]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:27.301 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.c/config 2024-06-03T03:12:28.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 podman[37023]: 2024-06-03 03:12:28.616855107 +0000 UTC m=+0.122072135 container create 193aee5b1d93f2a5ea2f4c162941a9064cb59f5c91830f5f31f1c59d0414fa4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mon-b, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-06-03T03:12:28.781 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:28.781+0000 7f7214e7a700 1 -- 172.21.15.183:0/1726803656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7210101100 msgr2=0x7f72101014e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:12:28.782 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:28.781+0000 7f7214e7a700 1 --2- 172.21.15.183:0/1726803656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7210101100 0x7f72101014e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f720800b0a0 tx=0x7f720800b3b0 comp rx=0 tx=0).stop 2024-06-03T03:12:28.782 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:28.781+0000 7f7214e7a700 1 -- 172.21.15.183:0/1726803656 shutdown_connections 2024-06-03T03:12:28.782 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:28.781+0000 7f7214e7a700 1 --2- 172.21.15.183:0/1726803656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7210101100 0x7f72101014e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:12:28.782 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:28.781+0000 7f7214e7a700 1 -- 172.21.15.183:0/1726803656 >> 172.21.15.183:0/1726803656 conn(0x7f72100fc990 msgr2=0x7f72100fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:12:28.782 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:28.781+0000 7f7214e7a700 1 -- 172.21.15.183:0/1726803656 shutdown_connections 2024-06-03T03:12:28.782 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:28.781+0000 7f7214e7a700 1 -- 172.21.15.183:0/1726803656 wait complete. 2024-06-03T03:12:28.783 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:28.782+0000 7f7214e7a700 1 Processor -- start 2024-06-03T03:12:28.783 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:28.782+0000 7f7214e7a700 1 -- start start 2024-06-03T03:12:28.783 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:28.782+0000 7f7214e7a700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7210101100 0x7f7210198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:12:28.783 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:28.782+0000 7f7214e7a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7210199040 0x7f72101994c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:12:28.784 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:28.782+0000 7f7214e7a700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f721010b4d0 con 0x7f7210199040 2024-06-03T03:12:28.784 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:28.782+0000 7f7214e7a700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f721010b350 con 0x7f7210101100 2024-06-03T03:12:28.784 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:28.782+0000 7f720e59c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7210101100 0x7f7210198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:12:28.784 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:28.782+0000 7f720e59c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7210101100 0x7f7210198b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:48902/0 (socket says 172.21.15.183:48902) 2024-06-03T03:12:28.784 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:28.782+0000 7f720e59c700 1 -- 172.21.15.183:0/4083681676 learned_addr learned my addr 172.21.15.183:0/4083681676 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-06-03T03:12:28.784 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:28.782+0000 7f720e59c700 1 -- 172.21.15.183:0/4083681676 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7210101100 msgr2=0x7f7210198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-06-03T03:12:28.784 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:28.782+0000 7f720e59c700 1 -- 172.21.15.183:0/4083681676 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7210101100 msgr2=0x7f7210198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-06-03T03:12:28.784 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:28.782+0000 7f720e59c700 1 --2- 172.21.15.183:0/4083681676 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7210101100 0x7f7210198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-06-03T03:12:28.784 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:28.782+0000 7f720e59c700 1 --2- 172.21.15.183:0/4083681676 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7210101100 0x7f7210198b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-03T03:12:28.784 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:28.783+0000 7f720dd9b700 1 --2- 172.21.15.183:0/4083681676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7210199040 0x7f72101994c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:12:28.784 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:28.783+0000 7f720dd9b700 1 -- 172.21.15.183:0/4083681676 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7210101100 msgr2=0x7f7210198b00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-03T03:12:28.784 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:28.783+0000 7f720dd9b700 1 --2- 172.21.15.183:0/4083681676 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7210101100 0x7f7210198b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:12:28.785 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:28.783+0000 7f720dd9b700 1 -- 172.21.15.183:0/4083681676 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f721006b5e0 con 0x7f7210199040 2024-06-03T03:12:28.786 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:28.784+0000 7f720dd9b700 1 --2- 172.21.15.183:0/4083681676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7210199040 0x7f72101994c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f720400b4d0 tx=0x7f720400b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:12:28.786 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:28.784+0000 7f71ff7fe700 1 -- 172.21.15.183:0/4083681676 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f7204011630 con 0x7f7210199040 2024-06-03T03:12:28.786 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:28.784+0000 7f7214e7a700 1 -- 172.21.15.183:0/4083681676 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72101a1c90 con 0x7f7210199040 2024-06-03T03:12:28.786 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:28.784+0000 7f7214e7a700 1 -- 172.21.15.183:0/4083681676 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72101a2220 con 0x7f7210199040 2024-06-03T03:12:28.786 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:28.784+0000 7f71ff7fe700 1 -- 172.21.15.183:0/4083681676 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7204011cb0 con 0x7f7210199040 2024-06-03T03:12:28.786 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:28.784+0000 7f71ff7fe700 1 -- 172.21.15.183:0/4083681676 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f720401ae40 con 0x7f7210199040 2024-06-03T03:12:28.786 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:28.785+0000 7f7214e7a700 1 -- 172.21.15.183:0/4083681676 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71f0005330 con 0x7f7210199040 2024-06-03T03:12:28.786 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:28.786+0000 7f71ff7fe700 1 -- 172.21.15.183:0/4083681676 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f7204018020 con 0x7f7210199040 2024-06-03T03:12:28.787 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:28.786+0000 7f71ff7fe700 1 --2- 172.21.15.183:0/4083681676 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f71f8038e90 0x7f71f803b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:12:28.787 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:28.786+0000 7f71ff7fe700 1 -- 172.21.15.183:0/4083681676 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f720404bf80 con 0x7f7210199040 2024-06-03T03:12:28.787 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:28.786+0000 7f720e59c700 1 --2- 172.21.15.183:0/4083681676 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f71f8038e90 0x7f71f803b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:12:28.787 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:28.787+0000 7f720e59c700 1 --2- 172.21.15.183:0/4083681676 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f71f8038e90 0x7f71f803b350 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7208009e60 tx=0x7f7208006230 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:12:28.800 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:28.800+0000 7f71ff7fe700 1 -- 172.21.15.183:0/4083681676 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f720401f020 con 0x7f7210199040 2024-06-03T03:12:28.851 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:28.851+0000 7f71ff7fe700 1 -- 172.21.15.183:0/4083681676 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72040117d0 con 0x7f7210199040 2024-06-03T03:12:29.077 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:29.077+0000 7f7214e7a700 1 -- 172.21.15.183:0/4083681676 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f71f00058f0 con 0x7f7210199040 2024-06-03T03:12:29.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 podman[37023]: 2024-06-03 03:12:28.794179557 +0000 UTC m=+0.299396554 container init 193aee5b1d93f2a5ea2f4c162941a9064cb59f5c91830f5f31f1c59d0414fa4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mon-b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-06-03T03:12:29.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 podman[37023]: 2024-06-03 03:12:28.797115144 +0000 UTC m=+0.302332142 container start 193aee5b1d93f2a5ea2f4c162941a9064cb59f5c91830f5f31f1c59d0414fa4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mon-b, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-06-03T03:12:29.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: set uid:gid to 167:167 (ceph:ceph) 2024-06-03T03:12:29.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc), process ceph-mon, pid 2 2024-06-03T03:12:29.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: pidfile_write: ignore empty --pid-file 2024-06-03T03:12:29.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: load: jerasure load: lrc 2024-06-03T03:12:29.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: RocksDB version: 7.9.2 2024-06-03T03:12:29.158 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Git sha 0 2024-06-03T03:12:29.158 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Compile date 2024-05-30 15:42:47 2024-06-03T03:12:29.158 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: DB SUMMARY 2024-06-03T03:12:29.158 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: DB Session ID: LTYV3F8K0BQE8MTO6VHX 2024-06-03T03:12:29.158 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: CURRENT file: CURRENT 2024-06-03T03:12:29.158 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: IDENTITY file: IDENTITY 2024-06-03T03:12:29.158 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-06-03T03:12:29.158 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-06-03T03:12:29.158 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-06-03T03:12:29.158 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.error_if_exists: 0 2024-06-03T03:12:29.158 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.create_if_missing: 0 2024-06-03T03:12:29.159 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.paranoid_checks: 1 2024-06-03T03:12:29.159 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.flush_verify_memtable_count: 1 2024-06-03T03:12:29.159 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-06-03T03:12:29.159 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-06-03T03:12:29.159 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.env: 0x556e17356c60 2024-06-03T03:12:29.159 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.fs: PosixFileSystem 2024-06-03T03:12:29.159 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.info_log: 0x556e19167b20 2024-06-03T03:12:29.159 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.max_file_opening_threads: 16 2024-06-03T03:12:29.161 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.statistics: (nil) 2024-06-03T03:12:29.161 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.use_fsync: 0 2024-06-03T03:12:29.161 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.max_log_file_size: 0 2024-06-03T03:12:29.161 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-06-03T03:12:29.161 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.log_file_time_to_roll: 0 2024-06-03T03:12:29.161 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.keep_log_file_num: 1000 2024-06-03T03:12:29.162 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.recycle_log_file_num: 0 2024-06-03T03:12:29.162 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.allow_fallocate: 1 2024-06-03T03:12:29.162 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.allow_mmap_reads: 0 2024-06-03T03:12:29.162 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.allow_mmap_writes: 0 2024-06-03T03:12:29.162 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.use_direct_reads: 0 2024-06-03T03:12:29.162 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-06-03T03:12:29.162 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.create_missing_column_families: 0 2024-06-03T03:12:29.162 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.db_log_dir: 2024-06-03T03:12:29.162 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.wal_dir: 2024-06-03T03:12:29.162 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.table_cache_numshardbits: 6 2024-06-03T03:12:29.162 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.WAL_ttl_seconds: 0 2024-06-03T03:12:29.162 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.WAL_size_limit_MB: 0 2024-06-03T03:12:29.163 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-06-03T03:12:29.163 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-06-03T03:12:29.163 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.is_fd_close_on_exec: 1 2024-06-03T03:12:29.163 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.advise_random_on_open: 1 2024-06-03T03:12:29.163 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.db_write_buffer_size: 0 2024-06-03T03:12:29.163 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.write_buffer_manager: 0x556e191381e0 2024-06-03T03:12:29.163 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-06-03T03:12:29.163 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-06-03T03:12:29.163 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.use_adaptive_mutex: 0 2024-06-03T03:12:29.163 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.rate_limiter: (nil) 2024-06-03T03:12:29.163 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-06-03T03:12:29.163 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.wal_recovery_mode: 2 2024-06-03T03:12:29.163 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.enable_thread_tracking: 0 2024-06-03T03:12:29.164 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.enable_pipelined_write: 0 2024-06-03T03:12:29.164 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.unordered_write: 0 2024-06-03T03:12:29.164 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-06-03T03:12:29.164 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-06-03T03:12:29.164 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-06-03T03:12:29.164 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-06-03T03:12:29.164 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.row_cache: None 2024-06-03T03:12:29.164 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.wal_filter: None 2024-06-03T03:12:29.164 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-06-03T03:12:29.164 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.allow_ingest_behind: 0 2024-06-03T03:12:29.164 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.two_write_queues: 0 2024-06-03T03:12:29.164 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.manual_wal_flush: 0 2024-06-03T03:12:29.164 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.wal_compression: 0 2024-06-03T03:12:29.164 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.atomic_flush: 0 2024-06-03T03:12:29.165 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-06-03T03:12:29.165 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.persist_stats_to_disk: 0 2024-06-03T03:12:29.165 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.write_dbid_to_manifest: 0 2024-06-03T03:12:29.165 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.log_readahead_size: 0 2024-06-03T03:12:29.165 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-06-03T03:12:29.165 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.best_efforts_recovery: 0 2024-06-03T03:12:29.165 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-06-03T03:12:29.165 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-06-03T03:12:29.165 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.allow_data_in_errors: 0 2024-06-03T03:12:29.165 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.db_host_id: __hostname__ 2024-06-03T03:12:29.165 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.enforce_single_del_contracts: true 2024-06-03T03:12:29.165 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.max_background_jobs: 2 2024-06-03T03:12:29.165 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.max_background_compactions: -1 2024-06-03T03:12:29.166 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.max_subcompactions: 1 2024-06-03T03:12:29.166 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-06-03T03:12:29.166 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-06-03T03:12:29.166 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.delayed_write_rate : 16777216 2024-06-03T03:12:29.166 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.max_total_wal_size: 0 2024-06-03T03:12:29.166 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-06-03T03:12:29.166 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.stats_dump_period_sec: 600 2024-06-03T03:12:29.166 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.stats_persist_period_sec: 600 2024-06-03T03:12:29.166 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-06-03T03:12:29.166 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.max_open_files: -1 2024-06-03T03:12:29.166 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.bytes_per_sync: 0 2024-06-03T03:12:29.166 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.wal_bytes_per_sync: 0 2024-06-03T03:12:29.166 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.strict_bytes_per_sync: 0 2024-06-03T03:12:29.167 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.compaction_readahead_size: 0 2024-06-03T03:12:29.167 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.max_background_flushes: -1 2024-06-03T03:12:29.167 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Compression algorithms supported: 2024-06-03T03:12:29.167 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-06-03T03:12:29.167 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: kZSTD supported: 0 2024-06-03T03:12:29.167 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: kXpressCompression supported: 0 2024-06-03T03:12:29.167 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: kLZ4HCCompression supported: 1 2024-06-03T03:12:29.167 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: kZlibCompression supported: 1 2024-06-03T03:12:29.167 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: kSnappyCompression supported: 1 2024-06-03T03:12:29.167 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: kLZ4Compression supported: 1 2024-06-03T03:12:29.167 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: kBZip2Compression supported: 0 2024-06-03T03:12:29.167 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Fast CRC32 supported: Supported on x86 2024-06-03T03:12:29.167 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: DMutex implementation: pthread_mutex_t 2024-06-03T03:12:29.167 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-06-03T03:12:29.168 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-06-03T03:12:29.168 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-06-03T03:12:29.168 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.merge_operator: 2024-06-03T03:12:29.168 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.compaction_filter: None 2024-06-03T03:12:29.168 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.compaction_filter_factory: None 2024-06-03T03:12:29.168 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.sst_partitioner_factory: None 2024-06-03T03:12:29.168 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.memtable_factory: SkipListFactory 2024-06-03T03:12:29.168 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.table_factory: BlockBasedTable 2024-06-03T03:12:29.168 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x556e19167c40) 2024-06-03T03:12:29.168 INFO:journalctl@ceph.mon.b.smithi151.stdout: cache_index_and_filter_blocks: 1 2024-06-03T03:12:29.168 INFO:journalctl@ceph.mon.b.smithi151.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-06-03T03:12:29.168 INFO:journalctl@ceph.mon.b.smithi151.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-06-03T03:12:29.168 INFO:journalctl@ceph.mon.b.smithi151.stdout: pin_top_level_index_and_filter: 1 2024-06-03T03:12:29.168 INFO:journalctl@ceph.mon.b.smithi151.stdout: index_type: 0 2024-06-03T03:12:29.169 INFO:journalctl@ceph.mon.b.smithi151.stdout: data_block_index_type: 0 2024-06-03T03:12:29.169 INFO:journalctl@ceph.mon.b.smithi151.stdout: index_shortening: 1 2024-06-03T03:12:29.169 INFO:journalctl@ceph.mon.b.smithi151.stdout: data_block_hash_table_util_ratio: 0.750000 2024-06-03T03:12:29.169 INFO:journalctl@ceph.mon.b.smithi151.stdout: checksum: 4 2024-06-03T03:12:29.169 INFO:journalctl@ceph.mon.b.smithi151.stdout: no_block_cache: 0 2024-06-03T03:12:29.169 INFO:journalctl@ceph.mon.b.smithi151.stdout: block_cache: 0x556e19169090 2024-06-03T03:12:29.169 INFO:journalctl@ceph.mon.b.smithi151.stdout: block_cache_name: BinnedLRUCache 2024-06-03T03:12:29.169 INFO:journalctl@ceph.mon.b.smithi151.stdout: block_cache_options: 2024-06-03T03:12:29.169 INFO:journalctl@ceph.mon.b.smithi151.stdout: capacity : 536870912 2024-06-03T03:12:29.169 INFO:journalctl@ceph.mon.b.smithi151.stdout: num_shard_bits : 4 2024-06-03T03:12:29.169 INFO:journalctl@ceph.mon.b.smithi151.stdout: strict_capacity_limit : 0 2024-06-03T03:12:29.169 INFO:journalctl@ceph.mon.b.smithi151.stdout: high_pri_pool_ratio: 0.000 2024-06-03T03:12:29.169 INFO:journalctl@ceph.mon.b.smithi151.stdout: block_cache_compressed: (nil) 2024-06-03T03:12:29.170 INFO:journalctl@ceph.mon.b.smithi151.stdout: persistent_cache: (nil) 2024-06-03T03:12:29.170 INFO:journalctl@ceph.mon.b.smithi151.stdout: block_size: 4096 2024-06-03T03:12:29.170 INFO:journalctl@ceph.mon.b.smithi151.stdout: block_size_deviation: 10 2024-06-03T03:12:29.170 INFO:journalctl@ceph.mon.b.smithi151.stdout: block_restart_interval: 16 2024-06-03T03:12:29.170 INFO:journalctl@ceph.mon.b.smithi151.stdout: index_block_restart_interval: 1 2024-06-03T03:12:29.170 INFO:journalctl@ceph.mon.b.smithi151.stdout: metadata_block_size: 4096 2024-06-03T03:12:29.170 INFO:journalctl@ceph.mon.b.smithi151.stdout: partition_filters: 0 2024-06-03T03:12:29.170 INFO:journalctl@ceph.mon.b.smithi151.stdout: use_delta_encoding: 1 2024-06-03T03:12:29.170 INFO:journalctl@ceph.mon.b.smithi151.stdout: filter_policy: bloomfilter 2024-06-03T03:12:29.170 INFO:journalctl@ceph.mon.b.smithi151.stdout: whole_key_filtering: 1 2024-06-03T03:12:29.170 INFO:journalctl@ceph.mon.b.smithi151.stdout: verify_compression: 0 2024-06-03T03:12:29.170 INFO:journalctl@ceph.mon.b.smithi151.stdout: read_amp_bytes_per_bit: 0 2024-06-03T03:12:29.170 INFO:journalctl@ceph.mon.b.smithi151.stdout: format_version: 5 2024-06-03T03:12:29.170 INFO:journalctl@ceph.mon.b.smithi151.stdout: enable_index_compression: 1 2024-06-03T03:12:29.170 INFO:journalctl@ceph.mon.b.smithi151.stdout: block_align: 0 2024-06-03T03:12:29.171 INFO:journalctl@ceph.mon.b.smithi151.stdout: max_auto_readahead_size: 262144 2024-06-03T03:12:29.171 INFO:journalctl@ceph.mon.b.smithi151.stdout: prepopulate_block_cache: 0 2024-06-03T03:12:29.171 INFO:journalctl@ceph.mon.b.smithi151.stdout: initial_auto_readahead_size: 8192 2024-06-03T03:12:29.171 INFO:journalctl@ceph.mon.b.smithi151.stdout: num_file_reads_for_auto_readahead: 2 2024-06-03T03:12:29.171 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.write_buffer_size: 33554432 2024-06-03T03:12:29.171 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.max_write_buffer_number: 2 2024-06-03T03:12:29.171 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.compression: NoCompression 2024-06-03T03:12:29.171 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.bottommost_compression: Disabled 2024-06-03T03:12:29.171 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.prefix_extractor: nullptr 2024-06-03T03:12:29.171 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-06-03T03:12:29.171 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.num_levels: 7 2024-06-03T03:12:29.171 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-06-03T03:12:29.171 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-06-03T03:12:29.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-06-03T03:12:29.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-06-03T03:12:29.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-06-03T03:12:29.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-06-03T03:12:29.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-06-03T03:12:29.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-06-03T03:12:29.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-06-03T03:12:29.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-06-03T03:12:29.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-06-03T03:12:29.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-06-03T03:12:29.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.compression_opts.window_bits: -14 2024-06-03T03:12:29.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.compression_opts.level: 32767 2024-06-03T03:12:29.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.compression_opts.strategy: 0 2024-06-03T03:12:29.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-06-03T03:12:29.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-06-03T03:12:29.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-06-03T03:12:29.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-06-03T03:12:29.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.compression_opts.enabled: false 2024-06-03T03:12:29.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-06-03T03:12:29.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-06-03T03:12:29.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-06-03T03:12:29.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-06-03T03:12:29.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.target_file_size_base: 67108864 2024-06-03T03:12:29.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.target_file_size_multiplier: 1 2024-06-03T03:12:29.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-06-03T03:12:29.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-06-03T03:12:29.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-06-03T03:12:29.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-06-03T03:12:29.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-06-03T03:12:29.174 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-06-03T03:12:29.174 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-06-03T03:12:29.174 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-06-03T03:12:29.174 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-06-03T03:12:29.174 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-06-03T03:12:29.174 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-06-03T03:12:29.174 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-06-03T03:12:29.174 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-06-03T03:12:29.174 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.arena_block_size: 1048576 2024-06-03T03:12:29.174 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-06-03T03:12:29.174 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-06-03T03:12:29.174 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.disable_auto_compactions: 0 2024-06-03T03:12:29.174 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-06-03T03:12:29.175 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-06-03T03:12:29.175 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-06-03T03:12:29.175 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-06-03T03:12:29.175 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-06-03T03:12:29.175 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-06-03T03:12:29.175 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-06-03T03:12:29.175 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-06-03T03:12:29.175 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-06-03T03:12:29.175 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-06-03T03:12:29.175 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.table_properties_collectors: 2024-06-03T03:12:29.175 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.inplace_update_support: 0 2024-06-03T03:12:29.175 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.inplace_update_num_locks: 10000 2024-06-03T03:12:29.175 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-06-03T03:12:29.175 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-06-03T03:12:29.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.memtable_huge_page_size: 0 2024-06-03T03:12:29.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.bloom_locality: 0 2024-06-03T03:12:29.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.max_successive_merges: 0 2024-06-03T03:12:29.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.optimize_filters_for_hits: 0 2024-06-03T03:12:29.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.paranoid_file_checks: 0 2024-06-03T03:12:29.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.force_consistency_checks: 1 2024-06-03T03:12:29.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.report_bg_io_stats: 0 2024-06-03T03:12:29.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.ttl: 2592000 2024-06-03T03:12:29.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.periodic_compaction_seconds: 0 2024-06-03T03:12:29.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-06-03T03:12:29.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-06-03T03:12:29.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.enable_blob_files: false 2024-06-03T03:12:29.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.min_blob_size: 0 2024-06-03T03:12:29.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.blob_file_size: 268435456 2024-06-03T03:12:29.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.blob_compression_type: NoCompression 2024-06-03T03:12:29.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.enable_blob_garbage_collection: false 2024-06-03T03:12:29.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-06-03T03:12:29.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-06-03T03:12:29.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-06-03T03:12:29.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.blob_file_starting_level: 0 2024-06-03T03:12:29.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-06-03T03:12:29.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-06-03T03:12:29.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-06-03T03:12:29.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: da0b01d3-582f-4537-9e1d-b15ee026421d 2024-06-03T03:12:29.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717384348826526, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-06-03T03:12:29.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-06-03T03:12:29.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717384348826959, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1717384348, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "da0b01d3-582f-4537-9e1d-b15ee026421d", "db_session_id": "LTYV3F8K0BQE8MTO6VHX", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-06-03T03:12:29.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717384348827047, "job": 1, "event": "recovery_finished"} 2024-06-03T03:12:29.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-06-03T03:12:29.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-06-03T03:12:29.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x556e19258000 2024-06-03T03:12:29.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: DB pointer 0x556e19242000 2024-06-03T03:12:29.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-06-03T03:12:29.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-06-03T03:12:29.178 INFO:journalctl@ceph.mon.b.smithi151.stdout: ** DB Stats ** 2024-06-03T03:12:29.178 INFO:journalctl@ceph.mon.b.smithi151.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-03T03:12:29.178 INFO:journalctl@ceph.mon.b.smithi151.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-06-03T03:12:29.178 INFO:journalctl@ceph.mon.b.smithi151.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-03T03:12:29.178 INFO:journalctl@ceph.mon.b.smithi151.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-03T03:12:29.178 INFO:journalctl@ceph.mon.b.smithi151.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-06-03T03:12:29.179 INFO:journalctl@ceph.mon.b.smithi151.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-03T03:12:29.179 INFO:journalctl@ceph.mon.b.smithi151.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-03T03:12:29.179 INFO:journalctl@ceph.mon.b.smithi151.stdout: 2024-06-03T03:12:29.179 INFO:journalctl@ceph.mon.b.smithi151.stdout: ** Compaction Stats [default] ** 2024-06-03T03:12:29.179 INFO:journalctl@ceph.mon.b.smithi151.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-06-03T03:12:29.179 INFO:journalctl@ceph.mon.b.smithi151.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-06-03T03:12:29.179 INFO:journalctl@ceph.mon.b.smithi151.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-06-03T03:12:29.179 INFO:journalctl@ceph.mon.b.smithi151.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-06-03T03:12:29.179 INFO:journalctl@ceph.mon.b.smithi151.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-06-03T03:12:29.179 INFO:journalctl@ceph.mon.b.smithi151.stdout: 2024-06-03T03:12:29.179 INFO:journalctl@ceph.mon.b.smithi151.stdout: ** Compaction Stats [default] ** 2024-06-03T03:12:29.179 INFO:journalctl@ceph.mon.b.smithi151.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-06-03T03:12:29.179 INFO:journalctl@ceph.mon.b.smithi151.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-06-03T03:12:29.179 INFO:journalctl@ceph.mon.b.smithi151.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-06-03T03:12:29.180 INFO:journalctl@ceph.mon.b.smithi151.stdout: 2024-06-03T03:12:29.180 INFO:journalctl@ceph.mon.b.smithi151.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-06-03T03:12:29.180 INFO:journalctl@ceph.mon.b.smithi151.stdout: 2024-06-03T03:12:29.180 INFO:journalctl@ceph.mon.b.smithi151.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-03T03:12:29.180 INFO:journalctl@ceph.mon.b.smithi151.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-06-03T03:12:29.180 INFO:journalctl@ceph.mon.b.smithi151.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-06-03T03:12:29.180 INFO:journalctl@ceph.mon.b.smithi151.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-06-03T03:12:29.180 INFO:journalctl@ceph.mon.b.smithi151.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-06-03T03:12:29.180 INFO:journalctl@ceph.mon.b.smithi151.stdout: AddFile(Keys): cumulative 0, interval 0 2024-06-03T03:12:29.180 INFO:journalctl@ceph.mon.b.smithi151.stdout: Cumulative compaction: 0.00 GB write, 0.68 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-03T03:12:29.180 INFO:journalctl@ceph.mon.b.smithi151.stdout: Interval compaction: 0.00 GB write, 0.68 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-03T03:12:29.180 INFO:journalctl@ceph.mon.b.smithi151.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-06-03T03:12:29.180 INFO:journalctl@ceph.mon.b.smithi151.stdout: Block cache BinnedLRUCache@0x556e19169090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-06-03T03:12:29.181 INFO:journalctl@ceph.mon.b.smithi151.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-06-03T03:12:29.181 INFO:journalctl@ceph.mon.b.smithi151.stdout: 2024-06-03T03:12:29.181 INFO:journalctl@ceph.mon.b.smithi151.stdout: ** File Read Latency Histogram By Level [default] ** 2024-06-03T03:12:29.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-06-03T03:12:29.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: using public_addr v2:172.21.15.151:0/0 -> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] 2024-06-03T03:12:29.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: starting mon.b rank -1 at public addrs [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] at bind addrs [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid a0e8473a-2156-11ef-bc9b-c7b262605968 2024-06-03T03:12:29.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: mon.b@-1(???) e0 preinit fsid a0e8473a-2156-11ef-bc9b-c7b262605968 2024-06-03T03:12:29.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: mon.b@-1(synchronizing).mds e1 new map 2024-06-03T03:12:29.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: mon.b@-1(synchronizing).mds e1 print_map 2024-06-03T03:12:29.181 INFO:journalctl@ceph.mon.b.smithi151.stdout: e1 2024-06-03T03:12:29.181 INFO:journalctl@ceph.mon.b.smithi151.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-06-03T03:12:29.181 INFO:journalctl@ceph.mon.b.smithi151.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-06-03T03:12:29.181 INFO:journalctl@ceph.mon.b.smithi151.stdout: legacy client fscid: -1 2024-06-03T03:12:29.181 INFO:journalctl@ceph.mon.b.smithi151.stdout: 2024-06-03T03:12:29.181 INFO:journalctl@ceph.mon.b.smithi151.stdout: No filesystems configured 2024-06-03T03:12:29.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-06-03T03:12:29.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-06-03T03:12:29.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-06-03T03:12:29.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-06-03T03:12:29.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-06-03T03:12:29.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-06-03T03:12:29.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-06-03T03:12:29.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-06-03T03:12:29.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-06-03T03:12:29.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-06-03T03:12:29.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:29.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi151", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:12:29.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: Deploying cephadm binary to smithi151 2024-06-03T03:12:29.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:29.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:29.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:29.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: Added host smithi151 2024-06-03T03:12:29.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:12:29.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:29.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:29.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:29.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:29.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:29.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-03T03:12:29.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:29.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:29.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:29.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:29.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:29.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi151", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:12:29.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:12:29.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:12:29.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: Updating smithi151:/etc/ceph/ceph.conf 2024-06-03T03:12:29.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: Updating smithi151:/var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-03T03:12:29.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: Updating smithi151:/etc/ceph/ceph.client.admin.keyring 2024-06-03T03:12:29.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:29.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: Updating smithi151:/var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-03T03:12:29.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi183", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:12:29.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:29.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:29.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:29.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: Deploying cephadm binary to smithi183 2024-06-03T03:12:29.185 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:29.185 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:29.185 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:29.185 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:12:29.185 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: Added host smithi183 2024-06-03T03:12:29.185 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:29.185 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:29.185 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:29.185 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:29.185 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:29.185 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-03T03:12:29.185 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:29.185 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:29.186 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:29.186 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:29.186 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:29.186 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi183", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:12:29.186 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:12:29.186 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:12:29.186 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: Updating smithi183:/etc/ceph/ceph.conf 2024-06-03T03:12:29.186 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: Updating smithi183:/var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-03T03:12:29.186 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: Updating smithi183:/etc/ceph/ceph.client.admin.keyring 2024-06-03T03:12:29.186 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: Updating smithi183:/var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-03T03:12:29.186 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:29.186 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/20444315' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-06-03T03:12:29.186 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:29.186 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:29.187 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:29.187 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/20444315' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-06-03T03:12:29.187 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: osdmap e4: 0 total, 0 up, 0 in 2024-06-03T03:12:29.187 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:29.187 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi117:172.21.15.117=a;smithi151:172.21.15.151=b;smithi183:172.21.15.183=c", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:12:29.187 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: Saving service mon spec with placement smithi117:172.21.15.117=a;smithi151:172.21.15.151=b;smithi183:172.21.15.183=c;count:3 2024-06-03T03:12:29.187 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:29.187 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:12:29.187 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:12:29.187 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:12:29.187 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:29.187 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-03T03:12:29.187 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:12:29.187 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: Deploying daemon mon.c on smithi183 2024-06-03T03:12:29.188 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:29.188 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:29.188 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:29.188 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: from='client.? 172.21.15.183:0/129012457' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-03T03:12:29.188 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-03T03:12:29.188 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: mon.a calling monitor election 2024-06-03T03:12:29.188 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T03:12:29.188 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:29.188 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T03:12:29.188 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T03:12:29.188 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: mon.c calling monitor election 2024-06-03T03:12:29.188 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:29.188 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T03:12:29.188 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T03:12:29.189 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:29.189 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T03:12:29.189 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-06-03T03:12:29.189 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: monmap epoch 2 2024-06-03T03:12:29.189 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: fsid a0e8473a-2156-11ef-bc9b-c7b262605968 2024-06-03T03:12:29.189 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: last_changed 2024-06-03T03:12:19.848370+0000 2024-06-03T03:12:29.189 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: created 2024-06-03T03:10:04.002179+0000 2024-06-03T03:12:29.189 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: min_mon_release 19 (squid) 2024-06-03T03:12:29.189 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: election_strategy: 1 2024-06-03T03:12:29.189 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: 0: [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] mon.a 2024-06-03T03:12:29.189 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: 1: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.c 2024-06-03T03:12:29.189 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: fsmap 2024-06-03T03:12:29.189 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: osdmap e4: 0 total, 0 up, 0 in 2024-06-03T03:12:29.189 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: mgrmap e13: x(active, since 60s) 2024-06-03T03:12:29.190 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: overall HEALTH_OK 2024-06-03T03:12:29.190 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:29.190 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:29.190 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:29.190 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-03T03:12:29.190 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:12:29.190 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: Deploying daemon mon.b on smithi151 2024-06-03T03:12:29.190 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: from='client.? 172.21.15.183:0/723901051' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-03T03:12:29.190 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T03:12:29.190 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:29.190 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 ceph-mon[37043]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-06-03T03:12:29.190 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 bash[37023]: 193aee5b1d93f2a5ea2f4c162941a9064cb59f5c91830f5f31f1c59d0414fa4e 2024-06-03T03:12:29.190 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:28 smithi151 systemd[1]: Started Ceph mon.b for a0e8473a-2156-11ef-bc9b-c7b262605968. 2024-06-03T03:12:33.871 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:33.870+0000 7f71ff7fe700 1 -- 172.21.15.183:0/4083681676 <== mon.0 v2:172.21.15.117:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1287 (secure 0 0 0) 0x7f7204021300 con 0x7f7210199040 2024-06-03T03:12:33.872 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-06-03T03:12:33.872 INFO:teuthology.orchestra.run.smithi183.stdout:{"epoch":3,"fsid":"a0e8473a-2156-11ef-bc9b-c7b262605968","modified":"2024-06-03T03:12:28.849342Z","created":"2024-06-03T03:10:04.002179Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:3300","nonce":0},{"type":"v1","addr":"172.21.15.117:6789","nonce":0}]},"addr":"172.21.15.117:6789/0","public_addr":"172.21.15.117:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:3300","nonce":0},{"type":"v1","addr":"172.21.15.183:6789","nonce":0}]},"addr":"172.21.15.183:6789/0","public_addr":"172.21.15.183:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:3300","nonce":0},{"type":"v1","addr":"172.21.15.151:6789","nonce":0}]},"addr":"172.21.15.151:6789/0","public_addr":"172.21.15.151:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-06-03T03:12:33.874 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:33.874+0000 7f7214e7a700 1 -- 172.21.15.183:0/4083681676 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f71f8038e90 msgr2=0x7f71f803b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:12:33.875 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:33.874+0000 7f7214e7a700 1 --2- 172.21.15.183:0/4083681676 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f71f8038e90 0x7f71f803b350 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7208009e60 tx=0x7f7208006230 comp rx=0 tx=0).stop 2024-06-03T03:12:33.875 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:33.874+0000 7f7214e7a700 1 -- 172.21.15.183:0/4083681676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7210199040 msgr2=0x7f72101994c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:12:33.875 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:33.874+0000 7f7214e7a700 1 --2- 172.21.15.183:0/4083681676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7210199040 0x7f72101994c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f720400b4d0 tx=0x7f720400b890 comp rx=0 tx=0).stop 2024-06-03T03:12:33.875 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:33.874+0000 7f7214e7a700 1 -- 172.21.15.183:0/4083681676 shutdown_connections 2024-06-03T03:12:33.875 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:33.874+0000 7f7214e7a700 1 --2- 172.21.15.183:0/4083681676 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f71f8038e90 0x7f71f803b350 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:12:33.875 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:33.874+0000 7f7214e7a700 1 --2- 172.21.15.183:0/4083681676 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7210101100 0x7f7210198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:12:33.876 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:33.874+0000 7f7214e7a700 1 --2- 172.21.15.183:0/4083681676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7210199040 0x7f72101994c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:12:33.876 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:33.874+0000 7f7214e7a700 1 -- 172.21.15.183:0/4083681676 >> 172.21.15.183:0/4083681676 conn(0x7f72100fc990 msgr2=0x7f7210102ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:12:33.876 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:33.874+0000 7f7214e7a700 1 -- 172.21.15.183:0/4083681676 shutdown_connections 2024-06-03T03:12:33.876 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:33.874+0000 7f7214e7a700 1 -- 172.21.15.183:0/4083681676 wait complete. 2024-06-03T03:12:33.877 INFO:teuthology.orchestra.run.smithi183.stderr:dumped monmap epoch 3 2024-06-03T03:12:34.191 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:33 smithi183 ceph-mon[37326]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:34.191 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:33 smithi183 ceph-mon[37326]: mon.c calling monitor election 2024-06-03T03:12:34.191 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:33 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-03T03:12:34.191 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:33 smithi183 ceph-mon[37326]: mon.a calling monitor election 2024-06-03T03:12:34.191 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:33 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T03:12:34.192 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:33 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T03:12:34.192 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:33 smithi183 ceph-mon[37326]: from='client.? 172.21.15.183:0/4083681676' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-03T03:12:34.192 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:33 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T03:12:34.192 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:33 smithi183 ceph-mon[37326]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:34.192 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:33 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T03:12:34.192 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:33 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T03:12:34.192 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:33 smithi183 ceph-mon[37326]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:34.192 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:33 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T03:12:34.192 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:33 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T03:12:34.192 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:33 smithi183 ceph-mon[37326]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-06-03T03:12:34.192 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:33 smithi183 ceph-mon[37326]: monmap epoch 3 2024-06-03T03:12:34.192 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:33 smithi183 ceph-mon[37326]: fsid a0e8473a-2156-11ef-bc9b-c7b262605968 2024-06-03T03:12:34.193 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:33 smithi183 ceph-mon[37326]: last_changed 2024-06-03T03:12:28.849342+0000 2024-06-03T03:12:34.193 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:33 smithi183 ceph-mon[37326]: created 2024-06-03T03:10:04.002179+0000 2024-06-03T03:12:34.193 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:33 smithi183 ceph-mon[37326]: min_mon_release 19 (squid) 2024-06-03T03:12:34.193 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:33 smithi183 ceph-mon[37326]: election_strategy: 1 2024-06-03T03:12:34.193 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:33 smithi183 ceph-mon[37326]: 0: [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] mon.a 2024-06-03T03:12:34.193 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:33 smithi183 ceph-mon[37326]: 1: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.c 2024-06-03T03:12:34.193 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:33 smithi183 ceph-mon[37326]: 2: [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] mon.b 2024-06-03T03:12:34.193 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:33 smithi183 ceph-mon[37326]: fsmap 2024-06-03T03:12:34.193 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:33 smithi183 ceph-mon[37326]: osdmap e4: 0 total, 0 up, 0 in 2024-06-03T03:12:34.193 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:33 smithi183 ceph-mon[37326]: mgrmap e13: x(active, since 69s) 2024-06-03T03:12:34.193 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:33 smithi183 ceph-mon[37326]: overall HEALTH_OK 2024-06-03T03:12:34.193 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:33 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:34.193 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:33 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:34.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:33 smithi117 ceph-mon[31649]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:34.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:33 smithi117 ceph-mon[31649]: mon.c calling monitor election 2024-06-03T03:12:34.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:33 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-03T03:12:34.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:33 smithi117 ceph-mon[31649]: mon.a calling monitor election 2024-06-03T03:12:34.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:33 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T03:12:34.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:33 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T03:12:34.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:33 smithi117 ceph-mon[31649]: from='client.? 172.21.15.183:0/4083681676' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-03T03:12:34.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:33 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T03:12:34.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:33 smithi117 ceph-mon[31649]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:34.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:33 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T03:12:34.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:33 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T03:12:34.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:33 smithi117 ceph-mon[31649]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:34.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:33 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T03:12:34.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:33 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T03:12:34.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:33 smithi117 ceph-mon[31649]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-06-03T03:12:34.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:33 smithi117 ceph-mon[31649]: monmap epoch 3 2024-06-03T03:12:34.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:33 smithi117 ceph-mon[31649]: fsid a0e8473a-2156-11ef-bc9b-c7b262605968 2024-06-03T03:12:34.199 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:33 smithi117 ceph-mon[31649]: last_changed 2024-06-03T03:12:28.849342+0000 2024-06-03T03:12:34.199 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:33 smithi117 ceph-mon[31649]: created 2024-06-03T03:10:04.002179+0000 2024-06-03T03:12:34.199 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:33 smithi117 ceph-mon[31649]: min_mon_release 19 (squid) 2024-06-03T03:12:34.199 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:33 smithi117 ceph-mon[31649]: election_strategy: 1 2024-06-03T03:12:34.199 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:33 smithi117 ceph-mon[31649]: 0: [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] mon.a 2024-06-03T03:12:34.199 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:33 smithi117 ceph-mon[31649]: 1: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.c 2024-06-03T03:12:34.199 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:33 smithi117 ceph-mon[31649]: 2: [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] mon.b 2024-06-03T03:12:34.199 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:33 smithi117 ceph-mon[31649]: fsmap 2024-06-03T03:12:34.199 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:33 smithi117 ceph-mon[31649]: osdmap e4: 0 total, 0 up, 0 in 2024-06-03T03:12:34.199 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:33 smithi117 ceph-mon[31649]: mgrmap e13: x(active, since 69s) 2024-06-03T03:12:34.199 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:33 smithi117 ceph-mon[31649]: overall HEALTH_OK 2024-06-03T03:12:34.199 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:33 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:34.199 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:33 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:34.485 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-06-03T03:12:34.486 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph config generate-minimal-conf 2024-06-03T03:12:34.915 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:12:35.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:34 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:35.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:34 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:35.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:34 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:12:35.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:34 smithi117 ceph-mon[31649]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:35.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:34 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:35.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:34 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T03:12:35.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:34 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:35.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:34 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:35.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:34 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:12:35.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:34 smithi183 ceph-mon[37326]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:35.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:34 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:35.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:34 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T03:12:36.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:35 smithi151 ceph-mon[37043]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:36.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:35 smithi151 ceph-mon[37043]: mon.c calling monitor election 2024-06-03T03:12:36.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:35 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-03T03:12:36.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:35 smithi151 ceph-mon[37043]: mon.a calling monitor election 2024-06-03T03:12:36.158 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:35 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T03:12:36.158 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:35 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T03:12:36.158 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:35 smithi151 ceph-mon[37043]: from='client.? 172.21.15.183:0/4083681676' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-03T03:12:36.158 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:35 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T03:12:36.158 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:35 smithi151 ceph-mon[37043]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:36.158 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:35 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T03:12:36.158 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:35 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T03:12:36.158 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:35 smithi151 ceph-mon[37043]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:36.158 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:35 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T03:12:36.158 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:35 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T03:12:36.158 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:35 smithi151 ceph-mon[37043]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-06-03T03:12:36.158 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:35 smithi151 ceph-mon[37043]: monmap epoch 3 2024-06-03T03:12:36.158 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:35 smithi151 ceph-mon[37043]: fsid a0e8473a-2156-11ef-bc9b-c7b262605968 2024-06-03T03:12:36.158 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:35 smithi151 ceph-mon[37043]: last_changed 2024-06-03T03:12:28.849342+0000 2024-06-03T03:12:36.159 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:35 smithi151 ceph-mon[37043]: created 2024-06-03T03:10:04.002179+0000 2024-06-03T03:12:36.159 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:35 smithi151 ceph-mon[37043]: min_mon_release 19 (squid) 2024-06-03T03:12:36.159 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:35 smithi151 ceph-mon[37043]: election_strategy: 1 2024-06-03T03:12:36.159 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:35 smithi151 ceph-mon[37043]: 0: [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] mon.a 2024-06-03T03:12:36.159 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:35 smithi151 ceph-mon[37043]: 1: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.c 2024-06-03T03:12:36.159 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:35 smithi151 ceph-mon[37043]: 2: [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] mon.b 2024-06-03T03:12:36.159 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:35 smithi151 ceph-mon[37043]: fsmap 2024-06-03T03:12:36.159 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:35 smithi151 ceph-mon[37043]: osdmap e4: 0 total, 0 up, 0 in 2024-06-03T03:12:36.159 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:35 smithi151 ceph-mon[37043]: mgrmap e13: x(active, since 69s) 2024-06-03T03:12:36.159 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:35 smithi151 ceph-mon[37043]: overall HEALTH_OK 2024-06-03T03:12:36.159 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:35 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:36.159 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:35 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:36.159 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:35 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:36.160 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:35 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:36.160 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:35 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:12:36.160 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:35 smithi151 ceph-mon[37043]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:36.160 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:35 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:36.160 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:35 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T03:12:36.650 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:36 smithi117 ceph-mon[31649]: mon.b calling monitor election 2024-06-03T03:12:36.650 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:36 smithi117 ceph-mon[31649]: Updating smithi117:/etc/ceph/ceph.conf 2024-06-03T03:12:36.650 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:36 smithi117 ceph-mon[31649]: Updating smithi151:/etc/ceph/ceph.conf 2024-06-03T03:12:36.650 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:36 smithi117 ceph-mon[31649]: Updating smithi183:/etc/ceph/ceph.conf 2024-06-03T03:12:36.650 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:36 smithi117 ceph-mon[31649]: Updating smithi151:/var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-03T03:12:36.650 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:36 smithi117 ceph-mon[31649]: Updating smithi183:/var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-03T03:12:36.650 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:36 smithi117 ceph-mon[31649]: mon.b calling monitor election 2024-06-03T03:12:36.650 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:36 smithi117 ceph-mon[31649]: mon.c calling monitor election 2024-06-03T03:12:36.651 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:36 smithi117 ceph-mon[31649]: mon.a calling monitor election 2024-06-03T03:12:36.651 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:36 smithi117 ceph-mon[31649]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-06-03T03:12:36.651 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:36 smithi117 ceph-mon[31649]: monmap epoch 3 2024-06-03T03:12:36.651 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:36 smithi117 ceph-mon[31649]: fsid a0e8473a-2156-11ef-bc9b-c7b262605968 2024-06-03T03:12:36.651 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:36 smithi117 ceph-mon[31649]: last_changed 2024-06-03T03:12:28.849342+0000 2024-06-03T03:12:36.651 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:36 smithi117 ceph-mon[31649]: created 2024-06-03T03:10:04.002179+0000 2024-06-03T03:12:36.651 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:36 smithi117 ceph-mon[31649]: min_mon_release 19 (squid) 2024-06-03T03:12:36.651 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:36 smithi117 ceph-mon[31649]: election_strategy: 1 2024-06-03T03:12:36.651 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:36 smithi117 ceph-mon[31649]: 0: [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] mon.a 2024-06-03T03:12:36.651 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:36 smithi117 ceph-mon[31649]: 1: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.c 2024-06-03T03:12:36.651 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:36 smithi117 ceph-mon[31649]: 2: [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] mon.b 2024-06-03T03:12:36.651 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:36 smithi117 ceph-mon[31649]: fsmap 2024-06-03T03:12:36.651 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:36 smithi117 ceph-mon[31649]: osdmap e4: 0 total, 0 up, 0 in 2024-06-03T03:12:36.652 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:36 smithi117 ceph-mon[31649]: mgrmap e13: x(active, since 71s) 2024-06-03T03:12:36.652 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:36 smithi117 ceph-mon[31649]: overall HEALTH_OK 2024-06-03T03:12:36.652 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:36 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:36.652 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:36 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:36.652 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:36 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:36.652 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:36 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:36.807 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:36.806+0000 7ff2e6200700 1 -- 172.21.15.117:0/413145604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2e0100e10 msgr2=0x7ff2e01011f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:12:36.807 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:36.806+0000 7ff2e6200700 1 --2- 172.21.15.117:0/413145604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2e0100e10 0x7ff2e01011f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff2d400b0a0 tx=0x7ff2d400b3b0 comp rx=0 tx=0).stop 2024-06-03T03:12:36.807 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:36.806+0000 7ff2e6200700 1 -- 172.21.15.117:0/413145604 shutdown_connections 2024-06-03T03:12:36.807 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:36.806+0000 7ff2e6200700 1 --2- 172.21.15.117:0/413145604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2e0100e10 0x7ff2e01011f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:12:36.808 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:36.806+0000 7ff2e6200700 1 -- 172.21.15.117:0/413145604 >> 172.21.15.117:0/413145604 conn(0x7ff2e0076730 msgr2=0x7ff2e0076b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:12:36.808 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:36.806+0000 7ff2e6200700 1 -- 172.21.15.117:0/413145604 shutdown_connections 2024-06-03T03:12:36.808 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:36.806+0000 7ff2e6200700 1 -- 172.21.15.117:0/413145604 wait complete. 2024-06-03T03:12:36.808 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:36.808+0000 7ff2e6200700 1 Processor -- start 2024-06-03T03:12:36.808 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:36.808+0000 7ff2e6200700 1 -- start start 2024-06-03T03:12:36.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:36.808+0000 7ff2e6200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff2e0100e10 0x7ff2e019f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:12:36.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:36.808+0000 7ff2e6200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2e019fe60 0x7ff2e0199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:12:36.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:36.808+0000 7ff2e6200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2e019a020 0x7ff2e019a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:12:36.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:36.809+0000 7ff2e6200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e010b1a0 con 0x7ff2e019fe60 2024-06-03T03:12:36.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:36.809+0000 7ff2e6200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e010b020 con 0x7ff2e0100e10 2024-06-03T03:12:36.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:36.809+0000 7ff2e6200700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e010b320 con 0x7ff2e019a020 2024-06-03T03:12:36.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:36.809+0000 7ff2dfe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2e019fe60 0x7ff2e0199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:12:36.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:36.809+0000 7ff2dfe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2e019fe60 0x7ff2e0199ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:43366/0 (socket says 172.21.15.117:43366) 2024-06-03T03:12:36.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:36.809+0000 7ff2e4e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff2e0100e10 0x7ff2e019f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:12:36.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:36.809+0000 7ff2dfe00700 1 -- 172.21.15.117:0/3788997978 learned_addr learned my addr 172.21.15.117:0/3788997978 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:12:36.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:36.809+0000 7ff2e5800700 1 --2- 172.21.15.117:0/3788997978 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2e019a020 0x7ff2e019a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:12:36.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:36.809+0000 7ff2e5800700 1 -- 172.21.15.117:0/3788997978 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff2e0100e10 msgr2=0x7ff2e019f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:12:36.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:36.809+0000 7ff2e5800700 1 --2- 172.21.15.117:0/3788997978 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff2e0100e10 0x7ff2e019f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:12:36.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:36.809+0000 7ff2e5800700 1 -- 172.21.15.117:0/3788997978 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2e019fe60 msgr2=0x7ff2e0199ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:12:36.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:36.809+0000 7ff2e5800700 1 --2- 172.21.15.117:0/3788997978 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2e019fe60 0x7ff2e0199ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:12:36.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:36.809+0000 7ff2e5800700 1 -- 172.21.15.117:0/3788997978 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2e01a67f0 con 0x7ff2e019a020 2024-06-03T03:12:36.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:36.809+0000 7ff2e4e00700 1 --2- 172.21.15.117:0/3788997978 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff2e0100e10 0x7ff2e019f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:12:36.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:36.809+0000 7ff2dfe00700 1 --2- 172.21.15.117:0/3788997978 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2e019fe60 0x7ff2e0199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:12:36.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:36.810+0000 7ff2e5800700 1 --2- 172.21.15.117:0/3788997978 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2e019a020 0x7ff2e019a4d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff2d8006fd0 tx=0x7ff2d800b730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:12:36.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:36.810+0000 7ff2dd600700 1 -- 172.21.15.117:0/3788997978 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d8015020 con 0x7ff2e019a020 2024-06-03T03:12:36.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:36.810+0000 7ff2dd600700 1 -- 172.21.15.117:0/3788997978 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2d800be30 con 0x7ff2e019a020 2024-06-03T03:12:36.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:36.810+0000 7ff2dd600700 1 -- 172.21.15.117:0/3788997978 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d8010cc0 con 0x7ff2e019a020 2024-06-03T03:12:36.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:36.810+0000 7ff2e6200700 1 -- 172.21.15.117:0/3788997978 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2e01a6a20 con 0x7ff2e019a020 2024-06-03T03:12:36.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:36.810+0000 7ff2e6200700 1 -- 172.21.15.117:0/3788997978 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2e01a6f60 con 0x7ff2e019a020 2024-06-03T03:12:36.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:36.811+0000 7ff2e6200700 1 -- 172.21.15.117:0/3788997978 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2e0063af0 con 0x7ff2e019a020 2024-06-03T03:12:36.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:36.812+0000 7ff2dd600700 1 -- 172.21.15.117:0/3788997978 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7ff2d8018020 con 0x7ff2e019a020 2024-06-03T03:12:36.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:36.812+0000 7ff2dd600700 1 --2- 172.21.15.117:0/3788997978 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7ff2c8039580 0x7ff2c803ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:12:36.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:36.812+0000 7ff2dd600700 1 -- 172.21.15.117:0/3788997978 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ff2d804bb00 con 0x7ff2e019a020 2024-06-03T03:12:36.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:36.812+0000 7ff2e4e00700 1 --2- 172.21.15.117:0/3788997978 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7ff2c8039580 0x7ff2c803ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:12:36.814 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:36.813+0000 7ff2e4e00700 1 --2- 172.21.15.117:0/3788997978 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7ff2c8039580 0x7ff2c803ba40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff2d40062a0 tx=0x7ff2d4005040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:12:36.818 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:36.817+0000 7ff2dd600700 1 -- 172.21.15.117:0/3788997978 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff2d801f020 con 0x7ff2e019a020 2024-06-03T03:12:36.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:36 smithi151 ceph-mon[37043]: mon.b calling monitor election 2024-06-03T03:12:36.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:36 smithi151 ceph-mon[37043]: Updating smithi117:/etc/ceph/ceph.conf 2024-06-03T03:12:36.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:36 smithi151 ceph-mon[37043]: Updating smithi151:/etc/ceph/ceph.conf 2024-06-03T03:12:36.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:36 smithi151 ceph-mon[37043]: Updating smithi183:/etc/ceph/ceph.conf 2024-06-03T03:12:36.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:36 smithi151 ceph-mon[37043]: Updating smithi151:/var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-03T03:12:36.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:36 smithi151 ceph-mon[37043]: Updating smithi183:/var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-03T03:12:36.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:36 smithi151 ceph-mon[37043]: mon.b calling monitor election 2024-06-03T03:12:36.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:36 smithi151 ceph-mon[37043]: mon.c calling monitor election 2024-06-03T03:12:36.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:36 smithi151 ceph-mon[37043]: mon.a calling monitor election 2024-06-03T03:12:36.908 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:36 smithi151 ceph-mon[37043]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-06-03T03:12:36.908 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:36 smithi151 ceph-mon[37043]: monmap epoch 3 2024-06-03T03:12:36.908 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:36 smithi151 ceph-mon[37043]: fsid a0e8473a-2156-11ef-bc9b-c7b262605968 2024-06-03T03:12:36.908 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:36 smithi151 ceph-mon[37043]: last_changed 2024-06-03T03:12:28.849342+0000 2024-06-03T03:12:36.908 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:36 smithi151 ceph-mon[37043]: created 2024-06-03T03:10:04.002179+0000 2024-06-03T03:12:36.908 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:36 smithi151 ceph-mon[37043]: min_mon_release 19 (squid) 2024-06-03T03:12:36.908 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:36 smithi151 ceph-mon[37043]: election_strategy: 1 2024-06-03T03:12:36.908 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:36 smithi151 ceph-mon[37043]: 0: [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] mon.a 2024-06-03T03:12:36.908 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:36 smithi151 ceph-mon[37043]: 1: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.c 2024-06-03T03:12:36.908 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:36 smithi151 ceph-mon[37043]: 2: [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] mon.b 2024-06-03T03:12:36.908 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:36 smithi151 ceph-mon[37043]: fsmap 2024-06-03T03:12:36.908 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:36 smithi151 ceph-mon[37043]: osdmap e4: 0 total, 0 up, 0 in 2024-06-03T03:12:36.908 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:36 smithi151 ceph-mon[37043]: mgrmap e13: x(active, since 71s) 2024-06-03T03:12:36.908 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:36 smithi151 ceph-mon[37043]: overall HEALTH_OK 2024-06-03T03:12:36.909 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:36 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:36.909 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:36 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:36.909 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:36 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:36.909 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:36 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:36.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:36 smithi183 ceph-mon[37326]: mon.b calling monitor election 2024-06-03T03:12:36.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:36 smithi183 ceph-mon[37326]: Updating smithi117:/etc/ceph/ceph.conf 2024-06-03T03:12:36.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:36 smithi183 ceph-mon[37326]: Updating smithi151:/etc/ceph/ceph.conf 2024-06-03T03:12:36.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:36 smithi183 ceph-mon[37326]: Updating smithi183:/etc/ceph/ceph.conf 2024-06-03T03:12:36.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:36 smithi183 ceph-mon[37326]: Updating smithi151:/var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-03T03:12:36.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:36 smithi183 ceph-mon[37326]: Updating smithi183:/var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-03T03:12:36.983 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:36 smithi183 ceph-mon[37326]: mon.b calling monitor election 2024-06-03T03:12:36.983 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:36 smithi183 ceph-mon[37326]: mon.c calling monitor election 2024-06-03T03:12:36.983 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:36 smithi183 ceph-mon[37326]: mon.a calling monitor election 2024-06-03T03:12:36.983 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:36 smithi183 ceph-mon[37326]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-06-03T03:12:36.983 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:36 smithi183 ceph-mon[37326]: monmap epoch 3 2024-06-03T03:12:36.983 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:36 smithi183 ceph-mon[37326]: fsid a0e8473a-2156-11ef-bc9b-c7b262605968 2024-06-03T03:12:36.983 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:36 smithi183 ceph-mon[37326]: last_changed 2024-06-03T03:12:28.849342+0000 2024-06-03T03:12:36.983 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:36 smithi183 ceph-mon[37326]: created 2024-06-03T03:10:04.002179+0000 2024-06-03T03:12:36.983 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:36 smithi183 ceph-mon[37326]: min_mon_release 19 (squid) 2024-06-03T03:12:36.983 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:36 smithi183 ceph-mon[37326]: election_strategy: 1 2024-06-03T03:12:36.983 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:36 smithi183 ceph-mon[37326]: 0: [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] mon.a 2024-06-03T03:12:36.983 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:36 smithi183 ceph-mon[37326]: 1: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.c 2024-06-03T03:12:36.983 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:36 smithi183 ceph-mon[37326]: 2: [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] mon.b 2024-06-03T03:12:36.983 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:36 smithi183 ceph-mon[37326]: fsmap 2024-06-03T03:12:36.983 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:36 smithi183 ceph-mon[37326]: osdmap e4: 0 total, 0 up, 0 in 2024-06-03T03:12:36.984 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:36 smithi183 ceph-mon[37326]: mgrmap e13: x(active, since 71s) 2024-06-03T03:12:36.984 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:36 smithi183 ceph-mon[37326]: overall HEALTH_OK 2024-06-03T03:12:36.984 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:36 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:36.984 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:36 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:36.984 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:36 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:36.984 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:36 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:37.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:37.048+0000 7ff2e6200700 1 -- 172.21.15.117:0/3788997978 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7ff2e01a72a0 con 0x7ff2e019a020 2024-06-03T03:12:37.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:37.049+0000 7ff2dd600700 1 -- 172.21.15.117:0/3788997978 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+277 (secure 0 0 0) 0x7ff2d8010e80 con 0x7ff2e019a020 2024-06-03T03:12:37.050 INFO:teuthology.orchestra.run.smithi117.stdout:# minimal ceph.conf for a0e8473a-2156-11ef-bc9b-c7b262605968 2024-06-03T03:12:37.050 INFO:teuthology.orchestra.run.smithi117.stdout:[global] 2024-06-03T03:12:37.050 INFO:teuthology.orchestra.run.smithi117.stdout: fsid = a0e8473a-2156-11ef-bc9b-c7b262605968 2024-06-03T03:12:37.050 INFO:teuthology.orchestra.run.smithi117.stdout: mon_host = [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] 2024-06-03T03:12:37.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:37.052+0000 7ff2e6200700 1 -- 172.21.15.117:0/3788997978 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7ff2c8039580 msgr2=0x7ff2c803ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:12:37.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:37.052+0000 7ff2e6200700 1 --2- 172.21.15.117:0/3788997978 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7ff2c8039580 0x7ff2c803ba40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff2d40062a0 tx=0x7ff2d4005040 comp rx=0 tx=0).stop 2024-06-03T03:12:37.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:37.052+0000 7ff2e6200700 1 -- 172.21.15.117:0/3788997978 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2e019a020 msgr2=0x7ff2e019a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:12:37.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:37.052+0000 7ff2e6200700 1 --2- 172.21.15.117:0/3788997978 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2e019a020 0x7ff2e019a4d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff2d8006fd0 tx=0x7ff2d800b730 comp rx=0 tx=0).stop 2024-06-03T03:12:37.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:37.052+0000 7ff2e6200700 1 -- 172.21.15.117:0/3788997978 shutdown_connections 2024-06-03T03:12:37.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:37.052+0000 7ff2e6200700 1 --2- 172.21.15.117:0/3788997978 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7ff2c8039580 0x7ff2c803ba40 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:12:37.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:37.052+0000 7ff2e6200700 1 --2- 172.21.15.117:0/3788997978 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2e019a020 0x7ff2e019a4d0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:12:37.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:37.052+0000 7ff2e6200700 1 --2- 172.21.15.117:0/3788997978 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff2e0100e10 0x7ff2e019f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:12:37.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:37.052+0000 7ff2e6200700 1 --2- 172.21.15.117:0/3788997978 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2e019fe60 0x7ff2e0199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:12:37.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:37.052+0000 7ff2e6200700 1 -- 172.21.15.117:0/3788997978 >> 172.21.15.117:0/3788997978 conn(0x7ff2e0076730 msgr2=0x7ff2e0108a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:12:37.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:37.053+0000 7ff2e6200700 1 -- 172.21.15.117:0/3788997978 shutdown_connections 2024-06-03T03:12:37.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:37.053+0000 7ff2e6200700 1 -- 172.21.15.117:0/3788997978 wait complete. 2024-06-03T03:12:37.773 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-06-03T03:12:37.774 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-06-03T03:12:37.774 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/ceph/ceph.conf 2024-06-03T03:12:37.844 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-06-03T03:12:37.844 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-03T03:12:37.958 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-06-03T03:12:37.958 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/etc/ceph/ceph.conf 2024-06-03T03:12:37.981 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:37 smithi151 ceph-mon[37043]: Updating smithi117:/var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-03T03:12:37.982 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:37 smithi151 ceph-mon[37043]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:37.982 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:37 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:37.982 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:37 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:37.982 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:37 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:37.982 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:37 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T03:12:37.982 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:37 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/3788997978' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:12:37.982 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:37 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:37.982 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:37 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:37.982 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:37 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:37.982 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:37 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:37.982 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:37 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-03T03:12:37.982 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:37 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-03T03:12:37.982 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:37 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:12:37.986 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-06-03T03:12:37.986 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-03T03:12:38.051 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-06-03T03:12:38.051 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/ceph/ceph.conf 2024-06-03T03:12:38.074 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:37 smithi183 ceph-mon[37326]: Updating smithi117:/var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-03T03:12:38.074 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:37 smithi183 ceph-mon[37326]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:38.074 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:37 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:38.074 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:37 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:38.074 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:37 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:38.074 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:37 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T03:12:38.074 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:37 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/3788997978' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:12:38.074 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:37 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:38.074 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:37 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:38.074 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:37 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:38.075 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:37 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:38.075 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:37 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-03T03:12:38.075 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:37 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-03T03:12:38.075 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:37 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:12:38.077 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-06-03T03:12:38.077 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-03T03:12:38.131 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:37 smithi117 ceph-mon[31649]: Updating smithi117:/var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-03T03:12:38.132 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:37 smithi117 ceph-mon[31649]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:38.132 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:37 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:38.132 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:37 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:38.132 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:37 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:38.132 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:37 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T03:12:38.132 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:37 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/3788997978' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:12:38.132 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:37 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:38.132 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:37 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:38.132 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:37 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:38.132 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:37 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:38.132 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:37 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-03T03:12:38.132 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:37 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-03T03:12:38.133 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:37 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:12:38.142 INFO:tasks.cephadm:Adding mgr.x on smithi117 2024-06-03T03:12:38.142 INFO:tasks.cephadm:Adding mgr.y on smithi151 2024-06-03T03:12:38.142 INFO:tasks.cephadm:Adding mgr.z on smithi183 2024-06-03T03:12:38.143 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph orch apply mgr '3;smithi117=x;smithi151=y;smithi183=z' 2024-06-03T03:12:38.417 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.c/config 2024-06-03T03:12:38.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:38 smithi117 ceph-mon[31649]: Reconfiguring mon.a (unknown last config time)... 2024-06-03T03:12:38.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:38 smithi117 ceph-mon[31649]: Reconfiguring daemon mon.a on smithi117 2024-06-03T03:12:38.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:38 smithi183 ceph-mon[37326]: Reconfiguring mon.a (unknown last config time)... 2024-06-03T03:12:38.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:38 smithi183 ceph-mon[37326]: Reconfiguring daemon mon.a on smithi117 2024-06-03T03:12:39.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:38 smithi151 ceph-mon[37043]: Reconfiguring mon.a (unknown last config time)... 2024-06-03T03:12:39.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:38 smithi151 ceph-mon[37043]: Reconfiguring daemon mon.a on smithi117 2024-06-03T03:12:39.812 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:39.809+0000 7fd87d03c700 1 -- 172.21.15.183:0/411303144 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd878100f10 msgr2=0x7fd878101330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:12:39.812 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:39.809+0000 7fd87d03c700 1 --2- 172.21.15.183:0/411303144 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd878100f10 0x7fd878101330 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd868009910 tx=0x7fd868009c20 comp rx=0 tx=0).stop 2024-06-03T03:12:39.812 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:39.809+0000 7fd87d03c700 1 -- 172.21.15.183:0/411303144 shutdown_connections 2024-06-03T03:12:39.812 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:39.809+0000 7fd87d03c700 1 --2- 172.21.15.183:0/411303144 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd878100f10 0x7fd878101330 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:12:39.812 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:39.809+0000 7fd87d03c700 1 -- 172.21.15.183:0/411303144 >> 172.21.15.183:0/411303144 conn(0x7fd8780fc4d0 msgr2=0x7fd8780fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:12:39.813 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:39.809+0000 7fd87d03c700 1 -- 172.21.15.183:0/411303144 shutdown_connections 2024-06-03T03:12:39.813 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:39.809+0000 7fd87d03c700 1 -- 172.21.15.183:0/411303144 wait complete. 2024-06-03T03:12:39.813 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:39.810+0000 7fd87d03c700 1 Processor -- start 2024-06-03T03:12:39.813 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:39.810+0000 7fd87d03c700 1 -- start start 2024-06-03T03:12:39.813 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:39.811+0000 7fd87d03c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd878100f10 0x7fd8781977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:12:39.813 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:39.811+0000 7fd87d03c700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd878197d20 0x7fd87819cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:12:39.813 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:39.811+0000 7fd87d03c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8781981a0 0x7fd87819f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:12:39.813 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:39.811+0000 7fd87d03c700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd878106ed0 con 0x7fd8781981a0 2024-06-03T03:12:39.813 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:39.811+0000 7fd87d03c700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd878106d50 con 0x7fd878197d20 2024-06-03T03:12:39.813 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:39.811+0000 7fd87d03c700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd878107050 con 0x7fd878100f10 2024-06-03T03:12:39.813 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:39.811+0000 7fd876d9d700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd878100f10 0x7fd8781977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:12:39.813 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:39.811+0000 7fd876d9d700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd878100f10 0x7fd8781977e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:38468/0 (socket says 172.21.15.183:38468) 2024-06-03T03:12:39.814 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:39.811+0000 7fd876d9d700 1 -- 172.21.15.183:0/2331945901 learned_addr learned my addr 172.21.15.183:0/2331945901 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-06-03T03:12:39.814 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:39.811+0000 7fd87659c700 1 --2- 172.21.15.183:0/2331945901 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd878197d20 0x7fd87819cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:12:39.814 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:39.811+0000 7fd876d9d700 1 -- 172.21.15.183:0/2331945901 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd878197d20 msgr2=0x7fd87819cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:12:39.814 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:39.811+0000 7fd876d9d700 1 --2- 172.21.15.183:0/2331945901 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd878197d20 0x7fd87819cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:12:39.814 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:39.811+0000 7fd876d9d700 1 -- 172.21.15.183:0/2331945901 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8781981a0 msgr2=0x7fd87819f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:12:39.814 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:39.811+0000 7fd876d9d700 1 --2- 172.21.15.183:0/2331945901 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8781981a0 0x7fd87819f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:12:39.814 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:39.811+0000 7fd876d9d700 1 -- 172.21.15.183:0/2331945901 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd87819fbf0 con 0x7fd878100f10 2024-06-03T03:12:39.814 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:39.811+0000 7fd876d9d700 1 --2- 172.21.15.183:0/2331945901 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd878100f10 0x7fd8781977e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd868005000 tx=0x7fd868005030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:12:39.815 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:39.811+0000 7fd85ffff700 1 -- 172.21.15.183:0/2331945901 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd868026070 con 0x7fd878100f10 2024-06-03T03:12:39.815 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:39.811+0000 7fd85ffff700 1 -- 172.21.15.183:0/2331945901 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd868005dd0 con 0x7fd878100f10 2024-06-03T03:12:39.816 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:39.811+0000 7fd87d03c700 1 -- 172.21.15.183:0/2331945901 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd87819fe80 con 0x7fd878100f10 2024-06-03T03:12:39.816 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:39.811+0000 7fd85ffff700 1 -- 172.21.15.183:0/2331945901 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd868017440 con 0x7fd878100f10 2024-06-03T03:12:39.816 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:39.811+0000 7fd87d03c700 1 -- 172.21.15.183:0/2331945901 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8781a0360 con 0x7fd878100f10 2024-06-03T03:12:39.816 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:39.812+0000 7fd85ffff700 1 -- 172.21.15.183:0/2331945901 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fd868029070 con 0x7fd878100f10 2024-06-03T03:12:39.816 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:39.812+0000 7fd87d03c700 1 -- 172.21.15.183:0/2331945901 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd878063af0 con 0x7fd878100f10 2024-06-03T03:12:39.816 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:39.813+0000 7fd85ffff700 1 --2- 172.21.15.183:0/2331945901 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fd860039580 0x7fd86003ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:12:39.816 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:39.813+0000 7fd85ffff700 1 -- 172.21.15.183:0/2331945901 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fd86804d260 con 0x7fd878100f10 2024-06-03T03:12:39.816 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:39.813+0000 7fd87659c700 1 --2- 172.21.15.183:0/2331945901 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fd860039580 0x7fd86003ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:12:39.816 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:39.814+0000 7fd87659c700 1 --2- 172.21.15.183:0/2331945901 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fd860039580 0x7fd86003ba40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd86c009ad0 tx=0x7fd86c009210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:12:39.819 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:39.818+0000 7fd85ffff700 1 -- 172.21.15.183:0/2331945901 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd868013070 con 0x7fd878100f10 2024-06-03T03:12:39.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:39 smithi183 ceph-mon[37326]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:39.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:39 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:39.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:39 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:39.983 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:39 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-03T03:12:39.983 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:39 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-03T03:12:39.983 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:39 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:12:40.026 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:40.025+0000 7fd87d03c700 1 -- 172.21.15.183:0/2331945901 --> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi117=x;smithi151=y;smithi183=z", "target": ["mon-mgr", ""]}) -- 0x7fd878062960 con 0x7fd860039580 2024-06-03T03:12:40.039 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:40.038+0000 7fd85ffff700 1 -- 172.21.15.183:0/2331945901 <== mgr.14152 v2:172.21.15.117:6800/1398160887 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fd878062960 con 0x7fd860039580 2024-06-03T03:12:40.039 INFO:teuthology.orchestra.run.smithi183.stdout:Scheduled mgr update... 2024-06-03T03:12:40.042 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:40.040+0000 7fd87d03c700 1 -- 172.21.15.183:0/2331945901 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fd860039580 msgr2=0x7fd86003ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:12:40.042 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:40.040+0000 7fd87d03c700 1 --2- 172.21.15.183:0/2331945901 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fd860039580 0x7fd86003ba40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd86c009ad0 tx=0x7fd86c009210 comp rx=0 tx=0).stop 2024-06-03T03:12:40.042 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:40.040+0000 7fd87d03c700 1 -- 172.21.15.183:0/2331945901 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd878100f10 msgr2=0x7fd8781977e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:12:40.042 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:40.040+0000 7fd87d03c700 1 --2- 172.21.15.183:0/2331945901 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd878100f10 0x7fd8781977e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd868005000 tx=0x7fd868005030 comp rx=0 tx=0).stop 2024-06-03T03:12:40.042 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:40.040+0000 7fd87d03c700 1 -- 172.21.15.183:0/2331945901 shutdown_connections 2024-06-03T03:12:40.042 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:40.040+0000 7fd87d03c700 1 --2- 172.21.15.183:0/2331945901 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fd860039580 0x7fd86003ba40 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:12:40.042 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:40.040+0000 7fd87d03c700 1 --2- 172.21.15.183:0/2331945901 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8781981a0 0x7fd87819f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:12:40.042 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:40.040+0000 7fd87d03c700 1 --2- 172.21.15.183:0/2331945901 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd878100f10 0x7fd8781977e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:12:40.042 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:40.040+0000 7fd87d03c700 1 --2- 172.21.15.183:0/2331945901 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd878197d20 0x7fd87819cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:12:40.042 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:40.040+0000 7fd87d03c700 1 -- 172.21.15.183:0/2331945901 >> 172.21.15.183:0/2331945901 conn(0x7fd8780fc4d0 msgr2=0x7fd87806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:12:40.043 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:40.041+0000 7fd87d03c700 1 -- 172.21.15.183:0/2331945901 shutdown_connections 2024-06-03T03:12:40.043 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:12:40.041+0000 7fd87d03c700 1 -- 172.21.15.183:0/2331945901 wait complete. 2024-06-03T03:12:40.100 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:39 smithi151 ceph-mon[37043]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:40.100 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:39 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:40.100 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:39 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:40.100 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:39 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-03T03:12:40.100 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:39 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-03T03:12:40.100 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:39 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:12:40.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:39 smithi117 ceph-mon[31649]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:40.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:39 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:40.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:39 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:40.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:39 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-03T03:12:40.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:39 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-03T03:12:40.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:39 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:12:40.623 DEBUG:teuthology.orchestra.run.smithi151:mgr.y> sudo journalctl -f -n 0 -u ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@mgr.y.service 2024-06-03T03:12:40.626 DEBUG:teuthology.orchestra.run.smithi183:mgr.z> sudo journalctl -f -n 0 -u ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@mgr.z.service 2024-06-03T03:12:40.628 INFO:tasks.cephadm:Deploying OSDs... 2024-06-03T03:12:40.628 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-06-03T03:12:40.629 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/scratch_devs of=/dev/stdout 2024-06-03T03:12:40.684 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-06-03T03:12:40.685 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_1 2024-06-03T03:12:40.736 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-06-03T03:12:40.736 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-03T03:12:40.736 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 6h/6d Inode: 639 Links: 1 2024-06-03T03:12:40.736 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-03T03:12:40.736 INFO:teuthology.orchestra.run.smithi117.stdout:Context: system_u:object_r:device_t:s0 2024-06-03T03:12:40.736 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2024-06-03 03:11:37.918232974 +0000 2024-06-03T03:12:40.736 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-06-03 03:06:13.018299126 +0000 2024-06-03T03:12:40.736 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-06-03 03:06:13.018299126 +0000 2024-06-03T03:12:40.736 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: 2024-06-03 03:06:13.018299126 +0000 2024-06-03T03:12:40.737 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-06-03T03:12:40.815 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2024-06-03T03:12:40.816 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2024-06-03T03:12:40.816 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000368906 s, 1.4 MB/s 2024-06-03T03:12:40.819 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-06-03T03:12:40.872 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_2 2024-06-03T03:12:40.906 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:40 smithi151 ceph-mon[37043]: Reconfiguring mon.b (monmap changed)... 2024-06-03T03:12:40.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:40 smithi151 ceph-mon[37043]: Reconfiguring daemon mon.b on smithi151 2024-06-03T03:12:40.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:40 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:40.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:40 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:40.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:40 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:40.916 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-06-03T03:12:40.916 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-03T03:12:40.916 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 6h/6d Inode: 659 Links: 1 2024-06-03T03:12:40.916 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-03T03:12:40.916 INFO:teuthology.orchestra.run.smithi117.stdout:Context: system_u:object_r:device_t:s0 2024-06-03T03:12:40.916 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2024-06-03 03:11:37.988231619 +0000 2024-06-03T03:12:40.916 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-06-03 03:06:13.601287670 +0000 2024-06-03T03:12:40.916 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-06-03 03:06:13.601287670 +0000 2024-06-03T03:12:40.917 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: 2024-06-03 03:06:13.601287670 +0000 2024-06-03T03:12:40.917 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-06-03T03:12:41.035 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2024-06-03T03:12:41.035 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2024-06-03T03:12:41.035 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000326502 s, 1.6 MB/s 2024-06-03T03:12:41.040 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-06-03T03:12:41.092 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_3 2024-06-03T03:12:41.140 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-06-03T03:12:41.140 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-03T03:12:41.140 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 6h/6d Inode: 677 Links: 1 2024-06-03T03:12:41.140 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-03T03:12:41.140 INFO:teuthology.orchestra.run.smithi117.stdout:Context: system_u:object_r:device_t:s0 2024-06-03T03:12:41.140 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2024-06-03 03:11:38.068230069 +0000 2024-06-03T03:12:41.140 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-06-03 03:06:14.192276056 +0000 2024-06-03T03:12:41.140 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-06-03 03:06:14.192276056 +0000 2024-06-03T03:12:41.140 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: 2024-06-03 03:06:14.192276056 +0000 2024-06-03T03:12:41.141 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-06-03T03:12:41.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:40 smithi117 ceph-mon[31649]: Reconfiguring mon.b (monmap changed)... 2024-06-03T03:12:41.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:40 smithi117 ceph-mon[31649]: Reconfiguring daemon mon.b on smithi151 2024-06-03T03:12:41.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:40 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:41.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:40 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:41.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:40 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:41.209 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2024-06-03T03:12:41.209 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2024-06-03T03:12:41.210 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000312747 s, 1.6 MB/s 2024-06-03T03:12:41.212 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-06-03T03:12:41.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:40 smithi183 ceph-mon[37326]: Reconfiguring mon.b (monmap changed)... 2024-06-03T03:12:41.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:40 smithi183 ceph-mon[37326]: Reconfiguring daemon mon.b on smithi151 2024-06-03T03:12:41.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:40 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:41.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:40 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:41.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:40 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:41.264 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_4 2024-06-03T03:12:41.314 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-06-03T03:12:41.314 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-03T03:12:41.314 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 6h/6d Inode: 698 Links: 1 2024-06-03T03:12:41.314 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-03T03:12:41.314 INFO:teuthology.orchestra.run.smithi117.stdout:Context: system_u:object_r:device_t:s0 2024-06-03T03:12:41.314 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2024-06-03 03:11:38.158228326 +0000 2024-06-03T03:12:41.314 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-06-03 03:06:14.780264501 +0000 2024-06-03T03:12:41.314 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-06-03 03:06:14.780264501 +0000 2024-06-03T03:12:41.315 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: 2024-06-03 03:06:14.780264501 +0000 2024-06-03T03:12:41.315 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-06-03T03:12:41.389 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2024-06-03T03:12:41.389 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2024-06-03T03:12:41.389 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000360192 s, 1.4 MB/s 2024-06-03T03:12:41.393 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-06-03T03:12:41.449 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-06-03T03:12:41.450 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/scratch_devs of=/dev/stdout 2024-06-03T03:12:41.469 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-06-03T03:12:41.469 DEBUG:teuthology.orchestra.run.smithi151:> stat /dev/vg_nvme/lv_1 2024-06-03T03:12:41.526 INFO:teuthology.orchestra.run.smithi151.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-06-03T03:12:41.526 INFO:teuthology.orchestra.run.smithi151.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-03T03:12:41.526 INFO:teuthology.orchestra.run.smithi151.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-06-03T03:12:41.526 INFO:teuthology.orchestra.run.smithi151.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-03T03:12:41.526 INFO:teuthology.orchestra.run.smithi151.stdout:Context: system_u:object_r:device_t:s0 2024-06-03T03:12:41.527 INFO:teuthology.orchestra.run.smithi151.stdout:Access: 2024-06-03 03:12:28.309036296 +0000 2024-06-03T03:12:41.527 INFO:teuthology.orchestra.run.smithi151.stdout:Modify: 2024-06-03 03:04:35.227542973 +0000 2024-06-03T03:12:41.527 INFO:teuthology.orchestra.run.smithi151.stdout:Change: 2024-06-03 03:04:35.227542973 +0000 2024-06-03T03:12:41.527 INFO:teuthology.orchestra.run.smithi151.stdout: Birth: 2024-06-03 03:04:35.227542973 +0000 2024-06-03T03:12:41.527 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-06-03T03:12:41.593 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records in 2024-06-03T03:12:41.593 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records out 2024-06-03T03:12:41.593 INFO:teuthology.orchestra.run.smithi151.stderr:512 bytes copied, 0.000368819 s, 1.4 MB/s 2024-06-03T03:12:41.595 DEBUG:teuthology.orchestra.run.smithi151:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-06-03T03:12:41.653 DEBUG:teuthology.orchestra.run.smithi151:> stat /dev/vg_nvme/lv_2 2024-06-03T03:12:41.709 INFO:teuthology.orchestra.run.smithi151.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-06-03T03:12:41.709 INFO:teuthology.orchestra.run.smithi151.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-03T03:12:41.709 INFO:teuthology.orchestra.run.smithi151.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-06-03T03:12:41.709 INFO:teuthology.orchestra.run.smithi151.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-03T03:12:41.709 INFO:teuthology.orchestra.run.smithi151.stdout:Context: system_u:object_r:device_t:s0 2024-06-03T03:12:41.709 INFO:teuthology.orchestra.run.smithi151.stdout:Access: 2024-06-03 03:12:28.309036296 +0000 2024-06-03T03:12:41.709 INFO:teuthology.orchestra.run.smithi151.stdout:Modify: 2024-06-03 03:04:35.546538678 +0000 2024-06-03T03:12:41.709 INFO:teuthology.orchestra.run.smithi151.stdout:Change: 2024-06-03 03:04:35.546538678 +0000 2024-06-03T03:12:41.710 INFO:teuthology.orchestra.run.smithi151.stdout: Birth: 2024-06-03 03:04:35.546538678 +0000 2024-06-03T03:12:41.710 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-06-03T03:12:41.775 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records in 2024-06-03T03:12:41.775 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records out 2024-06-03T03:12:41.775 INFO:teuthology.orchestra.run.smithi151.stderr:512 bytes copied, 0.000246514 s, 2.1 MB/s 2024-06-03T03:12:41.776 DEBUG:teuthology.orchestra.run.smithi151:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-06-03T03:12:41.835 DEBUG:teuthology.orchestra.run.smithi151:> stat /dev/vg_nvme/lv_3 2024-06-03T03:12:41.892 INFO:teuthology.orchestra.run.smithi151.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-06-03T03:12:41.892 INFO:teuthology.orchestra.run.smithi151.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-03T03:12:41.892 INFO:teuthology.orchestra.run.smithi151.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-06-03T03:12:41.893 INFO:teuthology.orchestra.run.smithi151.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-03T03:12:41.893 INFO:teuthology.orchestra.run.smithi151.stdout:Context: system_u:object_r:device_t:s0 2024-06-03T03:12:41.893 INFO:teuthology.orchestra.run.smithi151.stdout:Access: 2024-06-03 03:12:28.310036282 +0000 2024-06-03T03:12:41.893 INFO:teuthology.orchestra.run.smithi151.stdout:Modify: 2024-06-03 03:04:35.897533953 +0000 2024-06-03T03:12:41.893 INFO:teuthology.orchestra.run.smithi151.stdout:Change: 2024-06-03 03:04:35.897533953 +0000 2024-06-03T03:12:41.893 INFO:teuthology.orchestra.run.smithi151.stdout: Birth: 2024-06-03 03:04:35.897533953 +0000 2024-06-03T03:12:41.893 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-06-03T03:12:41.906 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:41 smithi151 ceph-mon[37043]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi117=x;smithi151=y;smithi183=z", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:12:41.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:41 smithi151 ceph-mon[37043]: Saving service mgr spec with placement smithi117=x;smithi151=y;smithi183=z;count:3 2024-06-03T03:12:41.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:41 smithi151 ceph-mon[37043]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:41.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:41 smithi151 ceph-mon[37043]: Reconfiguring mon.c (monmap changed)... 2024-06-03T03:12:41.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:41 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-03T03:12:41.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:41 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-03T03:12:41.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:41 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:12:41.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:41 smithi151 ceph-mon[37043]: Reconfiguring daemon mon.c on smithi183 2024-06-03T03:12:41.931 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records in 2024-06-03T03:12:41.931 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records out 2024-06-03T03:12:41.931 INFO:teuthology.orchestra.run.smithi151.stderr:512 bytes copied, 0.000216923 s, 2.4 MB/s 2024-06-03T03:12:41.932 DEBUG:teuthology.orchestra.run.smithi151:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-06-03T03:12:41.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:41 smithi183 ceph-mon[37326]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi117=x;smithi151=y;smithi183=z", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:12:41.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:41 smithi183 ceph-mon[37326]: Saving service mgr spec with placement smithi117=x;smithi151=y;smithi183=z;count:3 2024-06-03T03:12:41.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:41 smithi183 ceph-mon[37326]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:41.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:41 smithi183 ceph-mon[37326]: Reconfiguring mon.c (monmap changed)... 2024-06-03T03:12:41.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:41 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-03T03:12:41.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:41 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-03T03:12:41.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:41 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:12:41.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:41 smithi183 ceph-mon[37326]: Reconfiguring daemon mon.c on smithi183 2024-06-03T03:12:41.990 DEBUG:teuthology.orchestra.run.smithi151:> stat /dev/vg_nvme/lv_4 2024-06-03T03:12:42.045 INFO:teuthology.orchestra.run.smithi151.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-06-03T03:12:42.045 INFO:teuthology.orchestra.run.smithi151.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-03T03:12:42.045 INFO:teuthology.orchestra.run.smithi151.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-06-03T03:12:42.045 INFO:teuthology.orchestra.run.smithi151.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-03T03:12:42.045 INFO:teuthology.orchestra.run.smithi151.stdout:Context: system_u:object_r:device_t:s0 2024-06-03T03:12:42.045 INFO:teuthology.orchestra.run.smithi151.stdout:Access: 2024-06-03 03:12:28.310036282 +0000 2024-06-03T03:12:42.045 INFO:teuthology.orchestra.run.smithi151.stdout:Modify: 2024-06-03 03:04:36.236529389 +0000 2024-06-03T03:12:42.046 INFO:teuthology.orchestra.run.smithi151.stdout:Change: 2024-06-03 03:04:36.236529389 +0000 2024-06-03T03:12:42.046 INFO:teuthology.orchestra.run.smithi151.stdout: Birth: 2024-06-03 03:04:36.236529389 +0000 2024-06-03T03:12:42.046 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-06-03T03:12:42.109 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records in 2024-06-03T03:12:42.109 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records out 2024-06-03T03:12:42.109 INFO:teuthology.orchestra.run.smithi151.stderr:512 bytes copied, 0.000255642 s, 2.0 MB/s 2024-06-03T03:12:42.111 DEBUG:teuthology.orchestra.run.smithi151:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-06-03T03:12:42.167 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-06-03T03:12:42.167 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/scratch_devs of=/dev/stdout 2024-06-03T03:12:42.183 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-06-03T03:12:42.183 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_1 2024-06-03T03:12:42.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:41 smithi117 ceph-mon[31649]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi117=x;smithi151=y;smithi183=z", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:12:42.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:41 smithi117 ceph-mon[31649]: Saving service mgr spec with placement smithi117=x;smithi151=y;smithi183=z;count:3 2024-06-03T03:12:42.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:41 smithi117 ceph-mon[31649]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:42.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:41 smithi117 ceph-mon[31649]: Reconfiguring mon.c (monmap changed)... 2024-06-03T03:12:42.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:41 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-03T03:12:42.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:41 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-03T03:12:42.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:41 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:12:42.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:41 smithi117 ceph-mon[31649]: Reconfiguring daemon mon.c on smithi183 2024-06-03T03:12:42.239 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-06-03T03:12:42.240 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-03T03:12:42.240 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-06-03T03:12:42.240 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-03T03:12:42.240 INFO:teuthology.orchestra.run.smithi183.stdout:Context: system_u:object_r:device_t:s0 2024-06-03T03:12:42.240 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2024-06-03 03:12:18.880434574 +0000 2024-06-03T03:12:42.240 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2024-06-03 03:04:35.223062108 +0000 2024-06-03T03:12:42.240 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2024-06-03 03:04:35.223062108 +0000 2024-06-03T03:12:42.240 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: 2024-06-03 03:04:35.223062108 +0000 2024-06-03T03:12:42.240 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-06-03T03:12:42.303 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2024-06-03T03:12:42.303 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2024-06-03T03:12:42.303 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000360007 s, 1.4 MB/s 2024-06-03T03:12:42.304 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-06-03T03:12:42.360 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_2 2024-06-03T03:12:42.415 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-06-03T03:12:42.415 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-03T03:12:42.415 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-06-03T03:12:42.415 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-03T03:12:42.415 INFO:teuthology.orchestra.run.smithi183.stdout:Context: system_u:object_r:device_t:s0 2024-06-03T03:12:42.415 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2024-06-03 03:12:18.881434548 +0000 2024-06-03T03:12:42.415 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2024-06-03 03:04:35.547053269 +0000 2024-06-03T03:12:42.415 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2024-06-03 03:04:35.547053269 +0000 2024-06-03T03:12:42.415 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: 2024-06-03 03:04:35.547053269 +0000 2024-06-03T03:12:42.415 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-06-03T03:12:42.477 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2024-06-03T03:12:42.477 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2024-06-03T03:12:42.477 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000211625 s, 2.4 MB/s 2024-06-03T03:12:42.478 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-06-03T03:12:42.533 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_3 2024-06-03T03:12:42.589 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-06-03T03:12:42.589 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-03T03:12:42.589 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-06-03T03:12:42.589 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-03T03:12:42.589 INFO:teuthology.orchestra.run.smithi183.stdout:Context: system_u:object_r:device_t:s0 2024-06-03T03:12:42.589 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2024-06-03 03:12:18.881434548 +0000 2024-06-03T03:12:42.590 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2024-06-03 03:04:35.894043803 +0000 2024-06-03T03:12:42.590 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2024-06-03 03:04:35.894043803 +0000 2024-06-03T03:12:42.590 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: 2024-06-03 03:04:35.894043803 +0000 2024-06-03T03:12:42.590 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-06-03T03:12:42.653 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2024-06-03T03:12:42.653 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2024-06-03T03:12:42.653 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.00018217 s, 2.8 MB/s 2024-06-03T03:12:42.654 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-06-03T03:12:42.709 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_4 2024-06-03T03:12:42.764 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-06-03T03:12:42.765 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-03T03:12:42.765 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-06-03T03:12:42.765 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-03T03:12:42.765 INFO:teuthology.orchestra.run.smithi183.stdout:Context: system_u:object_r:device_t:s0 2024-06-03T03:12:42.765 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2024-06-03 03:12:18.881434548 +0000 2024-06-03T03:12:42.765 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2024-06-03 03:04:36.230034637 +0000 2024-06-03T03:12:42.765 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2024-06-03 03:04:36.230034637 +0000 2024-06-03T03:12:42.765 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: 2024-06-03 03:04:36.230034637 +0000 2024-06-03T03:12:42.765 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-06-03T03:12:42.828 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2024-06-03T03:12:42.828 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2024-06-03T03:12:42.828 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000192174 s, 2.7 MB/s 2024-06-03T03:12:42.829 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-06-03T03:12:42.886 INFO:tasks.cephadm:Deploying osd.0 on smithi117 with /dev/vg_nvme/lv_4... 2024-06-03T03:12:42.886 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-06-03T03:12:43.310 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:12:43.654 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:43 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:43.655 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:43 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:43.655 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:43 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:12:43.655 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:43 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:12:43.655 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:43 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:12:43.655 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:43 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:43.655 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:43 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-03T03:12:43.655 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:43 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-03T03:12:43.655 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:43 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-03T03:12:43.655 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:43 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:12:43.655 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:43 smithi117 ceph-mon[31649]: Deploying daemon mgr.z on smithi183 2024-06-03T03:12:43.655 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:43 smithi117 ceph-mon[31649]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:43.656 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:43 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:43.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:43 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:43.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:43 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:12:43.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:43 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:12:43.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:43 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:12:43.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:43 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:43.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:43 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-03T03:12:43.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:43 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-03T03:12:43.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:43 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-03T03:12:43.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:43 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:12:43.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:43 smithi151 ceph-mon[37043]: Deploying daemon mgr.z on smithi183 2024-06-03T03:12:43.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:43 smithi151 ceph-mon[37043]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:43.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:43 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:43.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:43 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:43.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:43 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:12:43.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:43 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:12:43.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:43 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:12:43.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:43 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:43.733 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:43 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-03T03:12:43.733 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:43 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-03T03:12:43.733 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:43 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-03T03:12:43.733 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:43 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:12:43.733 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:43 smithi183 ceph-mon[37326]: Deploying daemon mgr.z on smithi183 2024-06-03T03:12:43.733 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:43 smithi183 ceph-mon[37326]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:44.732 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Jun 03 03:12:44 smithi183 podman[39153]: 2024-06-03 03:12:44.652695018 +0000 UTC m=+0.123243506 container create 275464c5a73749c329419ea95dff4897704b80b3175c39fae5c47a83490f765e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-z, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, ceph=True, org.label-schema.schema-version=1.0) 2024-06-03T03:12:45.035 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Jun 03 03:12:44 smithi183 podman[39153]: 2024-06-03 03:12:44.880102258 +0000 UTC m=+0.350650713 container init 275464c5a73749c329419ea95dff4897704b80b3175c39fae5c47a83490f765e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-06-03T03:12:45.035 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Jun 03 03:12:44 smithi183 podman[39153]: 2024-06-03 03:12:44.882619141 +0000 UTC m=+0.353167607 container start 275464c5a73749c329419ea95dff4897704b80b3175c39fae5c47a83490f765e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-z, ceph=True, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS) 2024-06-03T03:12:45.036 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Jun 03 03:12:44 smithi183 ceph-mgr[39172]: -- 172.21.15.183:0/387891152 <== mon.1 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564bbf8f25a0 con 0x564bbf8c6c00 2024-06-03T03:12:45.036 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Jun 03 03:12:44 smithi183 bash[39153]: 275464c5a73749c329419ea95dff4897704b80b3175c39fae5c47a83490f765e 2024-06-03T03:12:45.036 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Jun 03 03:12:44 smithi183 systemd[1]: Started Ceph mgr.z for a0e8473a-2156-11ef-bc9b-c7b262605968. 2024-06-03T03:12:45.319 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Jun 03 03:12:45 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-z[39168]: 2024-06-03T03:12:45.034+0000 7f5497b9f200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-03T03:12:45.320 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Jun 03 03:12:45 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-z[39168]: 2024-06-03T03:12:45.217+0000 7f5497b9f200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-03T03:12:45.320 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Jun 03 03:12:45 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-z[39168]: 2024-06-03T03:12:45.318+0000 7f5497b9f200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-03T03:12:45.982 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Jun 03 03:12:45 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-z[39168]: 2024-06-03T03:12:45.624+0000 7f5497b9f200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-03T03:12:45.982 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Jun 03 03:12:45 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-z[39168]: 2024-06-03T03:12:45.708+0000 7f5497b9f200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-03T03:12:45.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:45 smithi183 ceph-mon[37326]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:45.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:45 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:45.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:45 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:45.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:45 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:45.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:45 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-03T03:12:45.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:45 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-03T03:12:45.983 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:45 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-03T03:12:45.983 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:45 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:12:46.030 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:45 smithi117 ceph-mon[31649]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:46.031 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:45 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:46.031 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:45 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:46.031 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:45 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:46.031 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:45 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-03T03:12:46.031 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:45 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-03T03:12:46.031 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:45 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-03T03:12:46.031 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:45 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:12:46.031 INFO:teuthology.orchestra.run.smithi117.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-06-03T03:12:46.031 INFO:teuthology.orchestra.run.smithi117.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-06-03T03:12:46.032 INFO:teuthology.orchestra.run.smithi117.stderr: stderr: 10+0 records in 2024-06-03T03:12:46.032 INFO:teuthology.orchestra.run.smithi117.stderr:10+0 records out 2024-06-03T03:12:46.032 INFO:teuthology.orchestra.run.smithi117.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.035014 s, 299 MB/s 2024-06-03T03:12:46.032 INFO:teuthology.orchestra.run.smithi117.stderr:--> Zapping successful for: 2024-06-03T03:12:46.038 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:45 smithi151 ceph-mon[37043]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:46.039 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:45 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:46.039 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:45 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:46.039 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:45 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:46.039 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:45 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-03T03:12:46.039 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:45 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-03T03:12:46.039 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:45 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-03T03:12:46.039 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:45 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:12:46.786 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi117:vg_nvme/lv_4 2024-06-03T03:12:47.060 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:46 smithi151 ceph-mon[37043]: Deploying daemon mgr.y on smithi151 2024-06-03T03:12:47.060 INFO:journalctl@ceph.mgr.y.smithi151.stdout:Jun 03 03:12:46 smithi151 systemd[1]: Starting Ceph mgr.y for a0e8473a-2156-11ef-bc9b-c7b262605968... 2024-06-03T03:12:47.072 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:46 smithi183 ceph-mon[37326]: Deploying daemon mgr.y on smithi151 2024-06-03T03:12:47.072 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Jun 03 03:12:46 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-z[39168]: 2024-06-03T03:12:46.905+0000 7f5497b9f200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-03T03:12:47.073 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Jun 03 03:12:47 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-z[39168]: 2024-06-03T03:12:47.071+0000 7f5497b9f200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-03T03:12:47.145 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:46 smithi117 ceph-mon[31649]: Deploying daemon mgr.y on smithi151 2024-06-03T03:12:47.221 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:12:47.328 INFO:journalctl@ceph.mgr.y.smithi151.stdout:Jun 03 03:12:47 smithi151 podman[38399]: 2024-06-03 03:12:47.059967893 +0000 UTC m=+0.107393853 container create 11784dd35c97acae90e740bd4df326c57b9cf9902d84e9d60baf47df567740da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-y, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-03T03:12:47.329 INFO:journalctl@ceph.mgr.y.smithi151.stdout:Jun 03 03:12:47 smithi151 podman[38399]: 2024-06-03 03:12:46.969859741 +0000 UTC m=+0.017285720 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T03:12:47.329 INFO:journalctl@ceph.mgr.y.smithi151.stdout:Jun 03 03:12:47 smithi151 podman[38399]: 2024-06-03 03:12:47.253732802 +0000 UTC m=+0.301158770 container init 11784dd35c97acae90e740bd4df326c57b9cf9902d84e9d60baf47df567740da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240528, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-06-03T03:12:47.329 INFO:journalctl@ceph.mgr.y.smithi151.stdout:Jun 03 03:12:47 smithi151 podman[38399]: 2024-06-03 03:12:47.256346036 +0000 UTC m=+0.303772017 container start 11784dd35c97acae90e740bd4df326c57b9cf9902d84e9d60baf47df567740da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-y, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, RELEASE=squid-3d89acc) 2024-06-03T03:12:47.329 INFO:journalctl@ceph.mgr.y.smithi151.stdout:Jun 03 03:12:47 smithi151 ceph-mgr[38418]: -- 172.21.15.151:0/688697533 <== mon.2 v2:172.21.15.151:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c9933d65a0 con 0x55c9933aa800 2024-06-03T03:12:47.329 INFO:journalctl@ceph.mgr.y.smithi151.stdout:Jun 03 03:12:47 smithi151 bash[38399]: 11784dd35c97acae90e740bd4df326c57b9cf9902d84e9d60baf47df567740da 2024-06-03T03:12:47.482 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Jun 03 03:12:47 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-z[39168]: 2024-06-03T03:12:47.241+0000 7f5497b9f200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-03T03:12:47.598 INFO:journalctl@ceph.mgr.y.smithi151.stdout:Jun 03 03:12:47 smithi151 systemd[1]: Started Ceph mgr.y for a0e8473a-2156-11ef-bc9b-c7b262605968. 2024-06-03T03:12:47.598 INFO:journalctl@ceph.mgr.y.smithi151.stdout:Jun 03 03:12:47 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-y[38414]: 2024-06-03T03:12:47.410+0000 7f7befb90200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-03T03:12:47.598 INFO:journalctl@ceph.mgr.y.smithi151.stdout:Jun 03 03:12:47 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-y[38414]: 2024-06-03T03:12:47.597+0000 7f7befb90200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-03T03:12:47.772 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:47 smithi117 ceph-mon[31649]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:47.773 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:47 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:47.773 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:47 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:47.773 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:47 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:47.773 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:47 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:47.773 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:47 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:12:47.832 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Jun 03 03:12:47 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-z[39168]: 2024-06-03T03:12:47.529+0000 7f5497b9f200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-03T03:12:47.832 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Jun 03 03:12:47 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-z[39168]: 2024-06-03T03:12:47.619+0000 7f5497b9f200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-03T03:12:47.832 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Jun 03 03:12:47 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-z[39168]: 2024-06-03T03:12:47.830+0000 7f5497b9f200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-03T03:12:47.832 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:47 smithi183 ceph-mon[37326]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:47.833 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:47 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:47.833 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:47 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:47.833 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:47 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:47.833 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:47 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:47.833 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:47 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:12:47.906 INFO:journalctl@ceph.mgr.y.smithi151.stdout:Jun 03 03:12:47 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-y[38414]: 2024-06-03T03:12:47.702+0000 7f7befb90200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-03T03:12:47.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:47 smithi151 ceph-mon[37043]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:47.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:47 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:47.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:47 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:47.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:47 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:47.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:47 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:47.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:47 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:12:48.232 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Jun 03 03:12:48 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-z[39168]: 2024-06-03T03:12:48.026+0000 7f5497b9f200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-03T03:12:48.406 INFO:journalctl@ceph.mgr.y.smithi151.stdout:Jun 03 03:12:48 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-y[38414]: 2024-06-03T03:12:48.032+0000 7f7befb90200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-03T03:12:48.407 INFO:journalctl@ceph.mgr.y.smithi151.stdout:Jun 03 03:12:48 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-y[38414]: 2024-06-03T03:12:48.121+0000 7f7befb90200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-03T03:12:48.982 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Jun 03 03:12:48 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-z[39168]: 2024-06-03T03:12:48.669+0000 7f5497b9f200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-03T03:12:49.391 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:49.389+0000 7fce56e00700 1 -- 172.21.15.117:0/3964757072 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce50074ef0 msgr2=0x7fce50075370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:12:49.391 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:49.389+0000 7fce56e00700 1 --2- 172.21.15.117:0/3964757072 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce50074ef0 0x7fce50075370 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fce3c009990 tx=0x7fce3c009ca0 comp rx=0 tx=0).stop 2024-06-03T03:12:49.392 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:49.389+0000 7fce56e00700 1 -- 172.21.15.117:0/3964757072 shutdown_connections 2024-06-03T03:12:49.392 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:49.389+0000 7fce56e00700 1 --2- 172.21.15.117:0/3964757072 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce5010eae0 0x7fce50110ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:12:49.392 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:49.389+0000 7fce56e00700 1 --2- 172.21.15.117:0/3964757072 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce500764c0 0x7fce50074920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:12:49.392 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:49.389+0000 7fce56e00700 1 --2- 172.21.15.117:0/3964757072 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce50074ef0 0x7fce50075370 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:12:49.392 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:49.389+0000 7fce56e00700 1 -- 172.21.15.117:0/3964757072 >> 172.21.15.117:0/3964757072 conn(0x7fce500fda60 msgr2=0x7fce500ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:12:49.392 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:49.389+0000 7fce56e00700 1 -- 172.21.15.117:0/3964757072 shutdown_connections 2024-06-03T03:12:49.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:49.389+0000 7fce56e00700 1 -- 172.21.15.117:0/3964757072 wait complete. 2024-06-03T03:12:49.397 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:49.390+0000 7fce56e00700 1 Processor -- start 2024-06-03T03:12:49.397 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:49.390+0000 7fce56e00700 1 -- start start 2024-06-03T03:12:49.397 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:49.391+0000 7fce56e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce50074ef0 0x7fce5019eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:12:49.398 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:49.391+0000 7fce56e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce500764c0 0x7fce5019f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:12:49.398 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:49.391+0000 7fce56e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce5010eae0 0x7fce501a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:12:49.398 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:49.391+0000 7fce56e00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fce50114700 con 0x7fce50074ef0 2024-06-03T03:12:49.398 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:49.391+0000 7fce56e00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fce50114580 con 0x7fce500764c0 2024-06-03T03:12:49.398 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:49.391+0000 7fce56e00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fce50114880 con 0x7fce5010eae0 2024-06-03T03:12:49.398 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:49.391+0000 7fce4fe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce50074ef0 0x7fce5019eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:12:49.399 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:49.391+0000 7fce4f400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce500764c0 0x7fce5019f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:12:49.399 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:49.391+0000 7fce4f400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce500764c0 0x7fce5019f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:58822/0 (socket says 172.21.15.117:58822) 2024-06-03T03:12:49.399 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:49.391+0000 7fce4f400700 1 -- 172.21.15.117:0/1126053018 learned_addr learned my addr 172.21.15.117:0/1126053018 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:12:49.399 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:49.391+0000 7fce54e00700 1 --2- 172.21.15.117:0/1126053018 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce5010eae0 0x7fce501a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:12:49.399 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:49.392+0000 7fce4f400700 1 -- 172.21.15.117:0/1126053018 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce5010eae0 msgr2=0x7fce501a6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:12:49.399 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:49.392+0000 7fce4f400700 1 --2- 172.21.15.117:0/1126053018 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce5010eae0 0x7fce501a6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:12:49.399 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:49.392+0000 7fce4f400700 1 -- 172.21.15.117:0/1126053018 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce50074ef0 msgr2=0x7fce5019eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:12:49.399 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:49.392+0000 7fce4f400700 1 --2- 172.21.15.117:0/1126053018 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce50074ef0 0x7fce5019eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:12:49.399 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:49.392+0000 7fce4f400700 1 -- 172.21.15.117:0/1126053018 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce501a69f0 con 0x7fce500764c0 2024-06-03T03:12:49.399 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:49.392+0000 7fce4f400700 1 --2- 172.21.15.117:0/1126053018 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce500764c0 0x7fce5019f2e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fce3c00b2e0 tx=0x7fce3c00bb90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:12:49.399 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:49.392+0000 7fce4cc00700 1 -- 172.21.15.117:0/1126053018 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce3c027070 con 0x7fce500764c0 2024-06-03T03:12:49.399 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:49.392+0000 7fce4cc00700 1 -- 172.21.15.117:0/1126053018 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce3c022470 con 0x7fce500764c0 2024-06-03T03:12:49.399 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:49.392+0000 7fce56e00700 1 -- 172.21.15.117:0/1126053018 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce501a6c80 con 0x7fce500764c0 2024-06-03T03:12:49.400 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:49.392+0000 7fce4cc00700 1 -- 172.21.15.117:0/1126053018 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce3c00f6c0 con 0x7fce500764c0 2024-06-03T03:12:49.400 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:49.393+0000 7fce56e00700 1 -- 172.21.15.117:0/1126053018 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce501a7130 con 0x7fce500764c0 2024-06-03T03:12:49.400 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:49.394+0000 7fce56e00700 1 -- 172.21.15.117:0/1126053018 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce50069180 con 0x7fce500764c0 2024-06-03T03:12:49.400 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:49.394+0000 7fce4cc00700 1 -- 172.21.15.117:0/1126053018 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fce3c02a070 con 0x7fce500764c0 2024-06-03T03:12:49.400 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:49.394+0000 7fce4cc00700 1 --2- 172.21.15.117:0/1126053018 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fce34039330 0x7fce3403b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:12:49.400 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:49.394+0000 7fce4cc00700 1 -- 172.21.15.117:0/1126053018 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fce3c04d530 con 0x7fce500764c0 2024-06-03T03:12:49.400 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:49.395+0000 7fce4fe00700 1 --2- 172.21.15.117:0/1126053018 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fce34039330 0x7fce3403b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:12:49.400 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:49.395+0000 7fce4fe00700 1 --2- 172.21.15.117:0/1126053018 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fce34039330 0x7fce3403b7f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fce501a0280 tx=0x7fce40009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:12:49.400 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Jun 03 03:12:49 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-z[39168]: 2024-06-03T03:12:49.100+0000 7f5497b9f200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-03T03:12:49.401 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Jun 03 03:12:49 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-z[39168]: 2024-06-03T03:12:49.195+0000 7f5497b9f200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-03T03:12:49.401 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Jun 03 03:12:49 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-z[39168]: 2024-06-03T03:12:49.398+0000 7f5497b9f200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-03T03:12:49.401 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:49.400+0000 7fce4cc00700 1 -- 172.21.15.117:0/1126053018 <== mon.2 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fce3c013070 con 0x7fce500764c0 2024-06-03T03:12:49.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:49.635+0000 7fce56e00700 1 -- 172.21.15.117:0/1126053018 --> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fce500628d0 con 0x7fce34039330 2024-06-03T03:12:49.656 INFO:journalctl@ceph.mgr.y.smithi151.stdout:Jun 03 03:12:49 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-y[38414]: 2024-06-03T03:12:49.393+0000 7f7befb90200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-03T03:12:49.656 INFO:journalctl@ceph.mgr.y.smithi151.stdout:Jun 03 03:12:49 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-y[38414]: 2024-06-03T03:12:49.563+0000 7f7befb90200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-03T03:12:50.022 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Jun 03 03:12:49 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-z[39168]: 2024-06-03T03:12:49.929+0000 7f5497b9f200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-03T03:12:50.022 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Jun 03 03:12:50 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-z[39168]: 2024-06-03T03:12:50.019+0000 7f5497b9f200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-03T03:12:50.022 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:49 smithi183 ceph-mon[37326]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:50.022 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:49 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:50.022 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:49 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:50.023 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:49 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:50.023 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:49 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:50.023 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:49 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:12:50.023 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:49 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:12:50.023 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:49 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:50.023 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:49 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-03T03:12:50.023 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:49 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-03T03:12:50.023 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:49 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:12:50.023 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:49 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T03:12:50.023 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:49 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T03:12:50.023 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:49 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:12:50.027 INFO:journalctl@ceph.mgr.y.smithi151.stdout:Jun 03 03:12:49 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-y[38414]: 2024-06-03T03:12:49.734+0000 7f7befb90200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-03T03:12:50.027 INFO:journalctl@ceph.mgr.y.smithi151.stdout:Jun 03 03:12:50 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-y[38414]: 2024-06-03T03:12:50.026+0000 7f7befb90200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-03T03:12:50.027 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:49 smithi151 ceph-mon[37043]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:50.027 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:49 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:50.027 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:49 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:50.028 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:49 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:50.028 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:49 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:50.028 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:49 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:12:50.028 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:49 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:12:50.028 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:49 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:50.028 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:49 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-03T03:12:50.028 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:49 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-03T03:12:50.028 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:49 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:12:50.028 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:49 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T03:12:50.028 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:49 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T03:12:50.028 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:49 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:12:50.138 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:49 smithi117 ceph-mon[31649]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:50.139 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:49 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:50.139 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:49 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:50.139 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:49 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:50.139 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:49 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:50.139 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:49 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:12:50.139 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:49 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:12:50.139 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:49 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:50.139 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:49 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-03T03:12:50.139 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:49 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-03T03:12:50.139 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:49 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:12:50.139 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:49 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T03:12:50.139 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:49 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T03:12:50.140 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:49 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:12:50.326 INFO:journalctl@ceph.mgr.y.smithi151.stdout:Jun 03 03:12:50 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-y[38414]: 2024-06-03T03:12:50.119+0000 7f7befb90200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-03T03:12:50.326 INFO:journalctl@ceph.mgr.y.smithi151.stdout:Jun 03 03:12:50 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-y[38414]: 2024-06-03T03:12:50.325+0000 7f7befb90200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-03T03:12:50.391 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Jun 03 03:12:50 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-z[39168]: 2024-06-03T03:12:50.258+0000 7f5497b9f200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-03T03:12:50.391 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Jun 03 03:12:50 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-z[39168]: 2024-06-03T03:12:50.390+0000 7f5497b9f200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-03T03:12:50.656 INFO:journalctl@ceph.mgr.y.smithi151.stdout:Jun 03 03:12:50 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-y[38414]: 2024-06-03T03:12:50.517+0000 7f7befb90200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-03T03:12:50.732 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Jun 03 03:12:50 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-z[39168]: 2024-06-03T03:12:50.483+0000 7f5497b9f200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-03T03:12:50.794 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Jun 03 03:12:50 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-z[39168]: 2024-06-03T03:12:50.598+0000 7f5497b9f200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-03T03:12:50.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:50 smithi117 ceph-mon[31649]: Reconfiguring mgr.x (unknown last config time)... 2024-06-03T03:12:50.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:50 smithi117 ceph-mon[31649]: Reconfiguring daemon mgr.x on smithi117 2024-06-03T03:12:50.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:50 smithi117 ceph-mon[31649]: from='client.24119 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:12:50.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:50 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:51.127 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:50 smithi151 ceph-mon[37043]: Reconfiguring mgr.x (unknown last config time)... 2024-06-03T03:12:51.127 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:50 smithi151 ceph-mon[37043]: Reconfiguring daemon mgr.x on smithi117 2024-06-03T03:12:51.127 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:50 smithi151 ceph-mon[37043]: from='client.24119 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:12:51.127 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:50 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:51.232 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Jun 03 03:12:50 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-z[39168]: 2024-06-03T03:12:50.897+0000 7f5497b9f200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-03T03:12:51.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:50 smithi183 ceph-mon[37326]: Reconfiguring mgr.x (unknown last config time)... 2024-06-03T03:12:51.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:50 smithi183 ceph-mon[37326]: Reconfiguring daemon mgr.x on smithi117 2024-06-03T03:12:51.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:50 smithi183 ceph-mon[37326]: from='client.24119 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:12:51.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:50 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:51.406 INFO:journalctl@ceph.mgr.y.smithi151.stdout:Jun 03 03:12:51 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-y[38414]: 2024-06-03T03:12:51.126+0000 7f7befb90200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-03T03:12:51.764 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Jun 03 03:12:51 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-z[39168]: 2024-06-03T03:12:51.483+0000 7f5497b9f200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-03T03:12:51.764 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Jun 03 03:12:51 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-z[39168]: 2024-06-03T03:12:51.697+0000 7f5497b9f200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-03T03:12:51.860 INFO:journalctl@ceph.mgr.y.smithi151.stdout:Jun 03 03:12:51 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-y[38414]: 2024-06-03T03:12:51.555+0000 7f7befb90200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-03T03:12:51.861 INFO:journalctl@ceph.mgr.y.smithi151.stdout:Jun 03 03:12:51 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-y[38414]: 2024-06-03T03:12:51.653+0000 7f7befb90200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-03T03:12:51.861 INFO:journalctl@ceph.mgr.y.smithi151.stdout:Jun 03 03:12:51 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-y[38414]: 2024-06-03T03:12:51.859+0000 7f7befb90200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-03T03:12:51.861 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:51 smithi151 ceph-mon[37043]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:51.861 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:51 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:51.861 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:51 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:51.861 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:51 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:12:51.861 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:51 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:12:51.861 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:51 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:12:51.861 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:51 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:51.862 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:51 smithi151 ceph-mon[37043]: from='mgr.? 172.21.15.183:0/2007675128' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-06-03T03:12:51.862 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:51 smithi151 ceph-mon[37043]: from='mgr.? 172.21.15.183:0/2007675128' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-03T03:12:51.862 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:51 smithi151 ceph-mon[37043]: Standby manager daemon z started 2024-06-03T03:12:51.862 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:51 smithi151 ceph-mon[37043]: from='mgr.? 172.21.15.183:0/2007675128' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-06-03T03:12:51.862 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:51 smithi151 ceph-mon[37043]: from='mgr.? 172.21.15.183:0/2007675128' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-03T03:12:51.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:51 smithi117 ceph-mon[31649]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:51.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:51 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:51.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:51 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:51.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:51 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:12:51.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:51 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:12:51.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:51 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:12:51.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:51 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:51.948 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:51 smithi117 ceph-mon[31649]: from='mgr.? 172.21.15.183:0/2007675128' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-06-03T03:12:51.948 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:51 smithi117 ceph-mon[31649]: from='mgr.? 172.21.15.183:0/2007675128' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-03T03:12:51.948 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:51 smithi117 ceph-mon[31649]: Standby manager daemon z started 2024-06-03T03:12:51.948 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:51 smithi117 ceph-mon[31649]: from='mgr.? 172.21.15.183:0/2007675128' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-06-03T03:12:51.948 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:51 smithi117 ceph-mon[31649]: from='mgr.? 172.21.15.183:0/2007675128' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-03T03:12:52.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:51 smithi183 ceph-mon[37326]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:52.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:51 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:52.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:51 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:52.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:51 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:12:52.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:51 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:12:52.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:51 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:12:52.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:51 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:12:52.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:51 smithi183 ceph-mon[37326]: from='mgr.? 172.21.15.183:0/2007675128' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-06-03T03:12:52.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:51 smithi183 ceph-mon[37326]: from='mgr.? 172.21.15.183:0/2007675128' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-03T03:12:52.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:51 smithi183 ceph-mon[37326]: Standby manager daemon z started 2024-06-03T03:12:52.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:51 smithi183 ceph-mon[37326]: from='mgr.? 172.21.15.183:0/2007675128' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-06-03T03:12:52.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:51 smithi183 ceph-mon[37326]: from='mgr.? 172.21.15.183:0/2007675128' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-03T03:12:52.553 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:52.551+0000 7fce4cc00700 1 -- 172.21.15.117:0/1126053018 <== mon.2 v2:172.21.15.151:3300/0 7 ==== mgrmap(e 14) ==== 92061+0+0 (secure 0 0 0) 0x7fce3c009d10 con 0x7fce500764c0 2024-06-03T03:12:52.656 INFO:journalctl@ceph.mgr.y.smithi151.stdout:Jun 03 03:12:52 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-y[38414]: 2024-06-03T03:12:52.391+0000 7f7befb90200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-03T03:12:52.656 INFO:journalctl@ceph.mgr.y.smithi151.stdout:Jun 03 03:12:52 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-y[38414]: 2024-06-03T03:12:52.484+0000 7f7befb90200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-03T03:12:53.076 INFO:journalctl@ceph.mgr.y.smithi151.stdout:Jun 03 03:12:52 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-y[38414]: 2024-06-03T03:12:52.728+0000 7f7befb90200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-03T03:12:53.076 INFO:journalctl@ceph.mgr.y.smithi151.stdout:Jun 03 03:12:52 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-y[38414]: 2024-06-03T03:12:52.863+0000 7f7befb90200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-03T03:12:53.076 INFO:journalctl@ceph.mgr.y.smithi151.stdout:Jun 03 03:12:52 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-y[38414]: 2024-06-03T03:12:52.958+0000 7f7befb90200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-03T03:12:53.076 INFO:journalctl@ceph.mgr.y.smithi151.stdout:Jun 03 03:12:53 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-y[38414]: 2024-06-03T03:12:53.075+0000 7f7befb90200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-03T03:12:53.656 INFO:journalctl@ceph.mgr.y.smithi151.stdout:Jun 03 03:12:53 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-y[38414]: 2024-06-03T03:12:53.378+0000 7f7befb90200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-03T03:12:53.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:53 smithi151 ceph-mon[37043]: mgrmap e14: x(active, since 87s), standbys: z 2024-06-03T03:12:53.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:53 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-06-03T03:12:53.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:53 smithi151 ceph-mon[37043]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:53.697 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:53 smithi117 ceph-mon[31649]: mgrmap e14: x(active, since 87s), standbys: z 2024-06-03T03:12:53.697 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:53 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-06-03T03:12:53.697 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:53 smithi117 ceph-mon[31649]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:53.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:53 smithi183 ceph-mon[37326]: mgrmap e14: x(active, since 87s), standbys: z 2024-06-03T03:12:53.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:53 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-06-03T03:12:53.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:53 smithi183 ceph-mon[37326]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:54.406 INFO:journalctl@ceph.mgr.y.smithi151.stdout:Jun 03 03:12:53 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-y[38414]: 2024-06-03T03:12:53.981+0000 7f7befb90200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-03T03:12:54.407 INFO:journalctl@ceph.mgr.y.smithi151.stdout:Jun 03 03:12:54 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-y[38414]: 2024-06-03T03:12:54.198+0000 7f7befb90200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-03T03:12:54.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:12:54.582+0000 7fce4cc00700 1 -- 172.21.15.117:0/1126053018 <== mon.2 v2:172.21.15.151:3300/0 8 ==== mgrmap(e 15) ==== 137778+0+0 (secure 0 0 0) 0x7fce3c00e3e0 con 0x7fce500764c0 2024-06-03T03:12:54.906 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:54 smithi151 ceph-mon[37043]: Standby manager daemon y started 2024-06-03T03:12:54.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:54 smithi151 ceph-mon[37043]: from='mgr.? 172.21.15.151:0/1110123708' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-03T03:12:54.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:54 smithi151 ceph-mon[37043]: from='mgr.? 172.21.15.151:0/1110123708' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-03T03:12:54.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:54 smithi151 ceph-mon[37043]: from='mgr.? 172.21.15.151:0/1110123708' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-03T03:12:54.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:54 smithi151 ceph-mon[37043]: from='mgr.? 172.21.15.151:0/1110123708' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-03T03:12:54.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:54 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/970898666' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8fefa089-9401-4a1c-b9fb-fe30c63e6a1d"}]: dispatch 2024-06-03T03:12:54.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:54 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/970898666' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8fefa089-9401-4a1c-b9fb-fe30c63e6a1d"}]': finished 2024-06-03T03:12:54.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:54 smithi151 ceph-mon[37043]: osdmap e5: 1 total, 0 up, 1 in 2024-06-03T03:12:54.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:54 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-03T03:12:54.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:54 smithi117 ceph-mon[31649]: Standby manager daemon y started 2024-06-03T03:12:54.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:54 smithi117 ceph-mon[31649]: from='mgr.? 172.21.15.151:0/1110123708' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-03T03:12:54.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:54 smithi117 ceph-mon[31649]: from='mgr.? 172.21.15.151:0/1110123708' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-03T03:12:54.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:54 smithi117 ceph-mon[31649]: from='mgr.? 172.21.15.151:0/1110123708' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-03T03:12:54.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:54 smithi117 ceph-mon[31649]: from='mgr.? 172.21.15.151:0/1110123708' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-03T03:12:54.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:54 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/970898666' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8fefa089-9401-4a1c-b9fb-fe30c63e6a1d"}]: dispatch 2024-06-03T03:12:54.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:54 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/970898666' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8fefa089-9401-4a1c-b9fb-fe30c63e6a1d"}]': finished 2024-06-03T03:12:54.948 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:54 smithi117 ceph-mon[31649]: osdmap e5: 1 total, 0 up, 1 in 2024-06-03T03:12:54.948 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:54 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-03T03:12:54.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:54 smithi183 ceph-mon[37326]: Standby manager daemon y started 2024-06-03T03:12:54.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:54 smithi183 ceph-mon[37326]: from='mgr.? 172.21.15.151:0/1110123708' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-03T03:12:54.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:54 smithi183 ceph-mon[37326]: from='mgr.? 172.21.15.151:0/1110123708' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-03T03:12:54.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:54 smithi183 ceph-mon[37326]: from='mgr.? 172.21.15.151:0/1110123708' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-03T03:12:54.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:54 smithi183 ceph-mon[37326]: from='mgr.? 172.21.15.151:0/1110123708' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-03T03:12:54.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:54 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/970898666' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8fefa089-9401-4a1c-b9fb-fe30c63e6a1d"}]: dispatch 2024-06-03T03:12:54.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:54 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/970898666' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8fefa089-9401-4a1c-b9fb-fe30c63e6a1d"}]': finished 2024-06-03T03:12:54.983 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:54 smithi183 ceph-mon[37326]: osdmap e5: 1 total, 0 up, 1 in 2024-06-03T03:12:54.983 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:54 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-03T03:12:55.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:55 smithi117 ceph-mon[31649]: mgrmap e15: x(active, since 89s), standbys: z, y 2024-06-03T03:12:55.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:55 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-03T03:12:55.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:55 smithi117 ceph-mon[31649]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:55.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:55 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/1167243227' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T03:12:55.906 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:55 smithi151 ceph-mon[37043]: mgrmap e15: x(active, since 89s), standbys: z, y 2024-06-03T03:12:55.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:55 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-03T03:12:55.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:55 smithi151 ceph-mon[37043]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:55.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:55 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/1167243227' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T03:12:55.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:55 smithi183 ceph-mon[37326]: mgrmap e15: x(active, since 89s), standbys: z, y 2024-06-03T03:12:55.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:55 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-03T03:12:55.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:55 smithi183 ceph-mon[37326]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:55.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:55 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/1167243227' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T03:12:58.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:57 smithi151 ceph-mon[37043]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:58.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:57 smithi117 ceph-mon[31649]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:58.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:57 smithi183 ceph-mon[37326]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:12:59.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:12:59 smithi117 ceph-mon[31649]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:13:00.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:12:59 smithi151 ceph-mon[37043]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:13:00.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:12:59 smithi183 ceph-mon[37326]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:13:01.846 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:01 smithi117 ceph-mon[31649]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:13:01.906 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:01 smithi151 ceph-mon[37043]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:13:02.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:01 smithi183 ceph-mon[37326]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:13:04.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:03 smithi151 ceph-mon[37043]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:13:04.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:03 smithi117 ceph-mon[31649]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:13:04.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:03 smithi183 ceph-mon[37326]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:13:05.025 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:04 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-06-03T03:13:05.026 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:04 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:13:05.026 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:04 smithi117 ceph-mon[31649]: Deploying daemon osd.0 on smithi117 2024-06-03T03:13:05.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:04 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-06-03T03:13:05.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:04 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:13:05.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:04 smithi151 ceph-mon[37043]: Deploying daemon osd.0 on smithi117 2024-06-03T03:13:05.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:04 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-06-03T03:13:05.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:04 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:13:05.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:04 smithi183 ceph-mon[37326]: Deploying daemon osd.0 on smithi117 2024-06-03T03:13:06.025 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:05 smithi117 ceph-mon[31649]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:13:06.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:05 smithi151 ceph-mon[37043]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:13:06.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:05 smithi183 ceph-mon[37326]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:13:08.037 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:07 smithi183 ceph-mon[37326]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:13:08.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:07 smithi151 ceph-mon[37043]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:13:08.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:07 smithi117 ceph-mon[31649]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:13:09.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:09 smithi117 ceph-mon[31649]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:13:10.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:09 smithi151 ceph-mon[37043]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:13:10.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:09 smithi183 ceph-mon[37326]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:13:12.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:11 smithi151 ceph-mon[37043]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:13:12.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:11 smithi117 ceph-mon[31649]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:13:12.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:11 smithi183 ceph-mon[37326]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:13:13.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:13 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:13:13.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:13 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:13:13.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:13 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:13:13.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:13 smithi117 ceph-mon[31649]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:13:13.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:13 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:13:13.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:13 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:13:13.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:13 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:13:13.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:13 smithi151 ceph-mon[37043]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:13:13.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:13 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:13:13.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:13 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:13:13.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:13 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:13:13.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:13 smithi183 ceph-mon[37326]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:13:16.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:15 smithi151 ceph-mon[37043]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:13:16.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:15 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:13:16.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:15 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:13:16.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:15 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:13:16.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:15 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:13:16.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:15 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:13:16.169 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:15 smithi117 ceph-mon[31649]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:13:16.169 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:15 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:13:16.169 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:15 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:13:16.169 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:15 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:13:16.169 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:15 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:13:16.170 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:15 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:13:16.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:15 smithi183 ceph-mon[37326]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:13:16.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:15 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:13:16.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:15 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:13:16.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:15 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:13:16.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:15 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:13:16.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:15 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:13:16.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:16 smithi117 ceph-mon[31649]: from='osd.0 [v2:172.21.15.117:6802/4286644492,v1:172.21.15.117:6803/4286644492]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-03T03:13:17.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:13:17.134+0000 7fce4cc00700 1 -- 172.21.15.117:0/1126053018 <== mgr.14152 v2:172.21.15.117:6800/1398160887 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fce500628d0 con 0x7fce34039330 2024-06-03T03:13:17.135 INFO:teuthology.orchestra.run.smithi117.stdout:Created osd(s) 0 on host 'smithi117' 2024-06-03T03:13:17.138 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:13:17.137+0000 7fce56e00700 1 -- 172.21.15.117:0/1126053018 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fce34039330 msgr2=0x7fce3403b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:13:17.138 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:13:17.137+0000 7fce56e00700 1 --2- 172.21.15.117:0/1126053018 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fce34039330 0x7fce3403b7f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fce501a0280 tx=0x7fce40009260 comp rx=0 tx=0).stop 2024-06-03T03:13:17.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:13:17.137+0000 7fce56e00700 1 -- 172.21.15.117:0/1126053018 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce500764c0 msgr2=0x7fce5019f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:13:17.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:13:17.137+0000 7fce56e00700 1 --2- 172.21.15.117:0/1126053018 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce500764c0 0x7fce5019f2e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fce3c00b2e0 tx=0x7fce3c00bb90 comp rx=0 tx=0).stop 2024-06-03T03:13:17.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:13:17.137+0000 7fce56e00700 1 -- 172.21.15.117:0/1126053018 shutdown_connections 2024-06-03T03:13:17.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:13:17.137+0000 7fce56e00700 1 --2- 172.21.15.117:0/1126053018 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fce34039330 0x7fce3403b7f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:13:17.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:13:17.137+0000 7fce56e00700 1 --2- 172.21.15.117:0/1126053018 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce5010eae0 0x7fce501a6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:13:17.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:13:17.137+0000 7fce56e00700 1 --2- 172.21.15.117:0/1126053018 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce50074ef0 0x7fce5019eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:13:17.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:13:17.137+0000 7fce56e00700 1 --2- 172.21.15.117:0/1126053018 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce500764c0 0x7fce5019f2e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:13:17.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:13:17.137+0000 7fce56e00700 1 -- 172.21.15.117:0/1126053018 >> 172.21.15.117:0/1126053018 conn(0x7fce500fda60 msgr2=0x7fce50101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:13:17.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:13:17.138+0000 7fce56e00700 1 -- 172.21.15.117:0/1126053018 shutdown_connections 2024-06-03T03:13:17.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:13:17.138+0000 7fce56e00700 1 -- 172.21.15.117:0/1126053018 wait complete. 2024-06-03T03:13:17.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:16 smithi151 ceph-mon[37043]: from='osd.0 [v2:172.21.15.117:6802/4286644492,v1:172.21.15.117:6803/4286644492]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-03T03:13:17.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:16 smithi183 ceph-mon[37326]: from='osd.0 [v2:172.21.15.117:6802/4286644492,v1:172.21.15.117:6803/4286644492]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-03T03:13:17.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:17 smithi117 ceph-mon[31649]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:13:17.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:17 smithi117 ceph-mon[31649]: from='osd.0 [v2:172.21.15.117:6802/4286644492,v1:172.21.15.117:6803/4286644492]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-06-03T03:13:17.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:17 smithi117 ceph-mon[31649]: osdmap e6: 1 total, 0 up, 1 in 2024-06-03T03:13:17.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:17 smithi117 ceph-mon[31649]: from='osd.0 [v2:172.21.15.117:6802/4286644492,v1:172.21.15.117:6803/4286644492]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-06-03T03:13:17.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:17 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-03T03:13:17.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:17 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:13:17.778 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:17 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:13:17.778 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:17 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:13:17.923 DEBUG:teuthology.orchestra.run.smithi117:osd.0> sudo journalctl -f -n 0 -u ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@osd.0.service 2024-06-03T03:13:17.927 INFO:tasks.cephadm:Deploying osd.1 on smithi151 with /dev/vg_nvme/lv_4... 2024-06-03T03:13:17.927 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-06-03T03:13:18.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:17 smithi151 ceph-mon[37043]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:13:18.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:17 smithi151 ceph-mon[37043]: from='osd.0 [v2:172.21.15.117:6802/4286644492,v1:172.21.15.117:6803/4286644492]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-06-03T03:13:18.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:17 smithi151 ceph-mon[37043]: osdmap e6: 1 total, 0 up, 1 in 2024-06-03T03:13:18.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:17 smithi151 ceph-mon[37043]: from='osd.0 [v2:172.21.15.117:6802/4286644492,v1:172.21.15.117:6803/4286644492]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-06-03T03:13:18.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:17 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-03T03:13:18.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:17 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:13:18.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:17 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:13:18.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:17 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:13:18.170 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.b/config 2024-06-03T03:13:18.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:17 smithi183 ceph-mon[37326]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:13:18.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:17 smithi183 ceph-mon[37326]: from='osd.0 [v2:172.21.15.117:6802/4286644492,v1:172.21.15.117:6803/4286644492]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-06-03T03:13:18.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:17 smithi183 ceph-mon[37326]: osdmap e6: 1 total, 0 up, 1 in 2024-06-03T03:13:18.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:17 smithi183 ceph-mon[37326]: from='osd.0 [v2:172.21.15.117:6802/4286644492,v1:172.21.15.117:6803/4286644492]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-06-03T03:13:18.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:17 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-03T03:13:18.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:17 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:13:18.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:17 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:13:18.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:17 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:13:18.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:18 smithi151 ceph-mon[37043]: from='osd.0 [v2:172.21.15.117:6802/4286644492,v1:172.21.15.117:6803/4286644492]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-06-03T03:13:18.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:18 smithi151 ceph-mon[37043]: osdmap e7: 1 total, 0 up, 1 in 2024-06-03T03:13:18.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:18 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-03T03:13:18.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:18 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-03T03:13:19.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:18 smithi117 ceph-mon[31649]: from='osd.0 [v2:172.21.15.117:6802/4286644492,v1:172.21.15.117:6803/4286644492]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-06-03T03:13:19.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:18 smithi117 ceph-mon[31649]: osdmap e7: 1 total, 0 up, 1 in 2024-06-03T03:13:19.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:18 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-03T03:13:19.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:18 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-03T03:13:19.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:18 smithi183 ceph-mon[37326]: from='osd.0 [v2:172.21.15.117:6802/4286644492,v1:172.21.15.117:6803/4286644492]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-06-03T03:13:19.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:18 smithi183 ceph-mon[37326]: osdmap e7: 1 total, 0 up, 1 in 2024-06-03T03:13:19.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:18 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-03T03:13:19.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:18 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-03T03:13:19.923 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:19 smithi117 ceph-mon[31649]: purged_snaps scrub starts 2024-06-03T03:13:19.924 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:19 smithi117 ceph-mon[31649]: purged_snaps scrub ok 2024-06-03T03:13:19.924 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:19 smithi117 ceph-mon[31649]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:13:19.924 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:19 smithi117 ceph-mon[31649]: osd.0 [v2:172.21.15.117:6802/4286644492,v1:172.21.15.117:6803/4286644492] boot 2024-06-03T03:13:19.924 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:19 smithi117 ceph-mon[31649]: osdmap e8: 1 total, 1 up, 1 in 2024-06-03T03:13:19.924 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:19 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-03T03:13:20.042 INFO:teuthology.orchestra.run.smithi151.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-06-03T03:13:20.043 INFO:teuthology.orchestra.run.smithi151.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-06-03T03:13:20.043 INFO:teuthology.orchestra.run.smithi151.stderr: stderr: 10+0 records in 2024-06-03T03:13:20.043 INFO:teuthology.orchestra.run.smithi151.stderr:10+0 records out 2024-06-03T03:13:20.043 INFO:teuthology.orchestra.run.smithi151.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115176 s, 910 MB/s 2024-06-03T03:13:20.043 INFO:teuthology.orchestra.run.smithi151.stderr:--> Zapping successful for: 2024-06-03T03:13:20.043 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:19 smithi151 ceph-mon[37043]: purged_snaps scrub starts 2024-06-03T03:13:20.043 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:19 smithi151 ceph-mon[37043]: purged_snaps scrub ok 2024-06-03T03:13:20.043 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:19 smithi151 ceph-mon[37043]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:13:20.043 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:19 smithi151 ceph-mon[37043]: osd.0 [v2:172.21.15.117:6802/4286644492,v1:172.21.15.117:6803/4286644492] boot 2024-06-03T03:13:20.043 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:19 smithi151 ceph-mon[37043]: osdmap e8: 1 total, 1 up, 1 in 2024-06-03T03:13:20.043 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:19 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-03T03:13:20.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:19 smithi183 ceph-mon[37326]: purged_snaps scrub starts 2024-06-03T03:13:20.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:19 smithi183 ceph-mon[37326]: purged_snaps scrub ok 2024-06-03T03:13:20.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:19 smithi183 ceph-mon[37326]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T03:13:20.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:19 smithi183 ceph-mon[37326]: osd.0 [v2:172.21.15.117:6802/4286644492,v1:172.21.15.117:6803/4286644492] boot 2024-06-03T03:13:20.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:19 smithi183 ceph-mon[37326]: osdmap e8: 1 total, 1 up, 1 in 2024-06-03T03:13:20.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:19 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-03T03:13:20.620 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi151:vg_nvme/lv_4 2024-06-03T03:13:20.861 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.b/config 2024-06-03T03:13:20.906 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:20 smithi151 ceph-mon[37043]: osdmap e9: 1 total, 1 up, 1 in 2024-06-03T03:13:21.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:20 smithi117 ceph-mon[31649]: osdmap e9: 1 total, 1 up, 1 in 2024-06-03T03:13:21.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:20 smithi183 ceph-mon[37326]: osdmap e9: 1 total, 1 up, 1 in 2024-06-03T03:13:22.042 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:21 smithi151 ceph-mon[37043]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T03:13:22.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:21 smithi117 ceph-mon[31649]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T03:13:22.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:21 smithi183 ceph-mon[37326]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T03:13:22.316 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:13:22.311+0000 7f1e4da75700 1 -- 172.21.15.151:0/3161308763 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e48102fa0 msgr2=0x7f1e48109850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:13:22.316 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:13:22.311+0000 7f1e4da75700 1 --2- 172.21.15.151:0/3161308763 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e48102fa0 0x7f1e48109850 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1e4000b0d0 tx=0x7f1e4000b3e0 comp rx=0 tx=0).stop 2024-06-03T03:13:22.317 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:13:22.312+0000 7f1e4da75700 1 -- 172.21.15.151:0/3161308763 shutdown_connections 2024-06-03T03:13:22.317 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:13:22.312+0000 7f1e4da75700 1 --2- 172.21.15.151:0/3161308763 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e48102fa0 0x7f1e48109850 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:13:22.317 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:13:22.312+0000 7f1e4da75700 1 --2- 172.21.15.151:0/3161308763 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e48101400 0x7f1e48101820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:13:22.317 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:13:22.312+0000 7f1e4da75700 1 --2- 172.21.15.151:0/3161308763 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e48102600 0x7f1e48102a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:13:22.317 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:13:22.312+0000 7f1e4da75700 1 -- 172.21.15.151:0/3161308763 >> 172.21.15.151:0/3161308763 conn(0x7f1e480fc9c0 msgr2=0x7f1e480fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:13:22.317 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:13:22.312+0000 7f1e4da75700 1 -- 172.21.15.151:0/3161308763 shutdown_connections 2024-06-03T03:13:22.317 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:13:22.312+0000 7f1e4da75700 1 -- 172.21.15.151:0/3161308763 wait complete. 2024-06-03T03:13:22.317 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:13:22.312+0000 7f1e4da75700 1 Processor -- start 2024-06-03T03:13:22.317 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:13:22.313+0000 7f1e4da75700 1 -- start start 2024-06-03T03:13:22.318 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:13:22.313+0000 7f1e4da75700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e48101400 0x7f1e4806b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:13:22.318 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:13:22.313+0000 7f1e4da75700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e48102600 0x7f1e4819d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:13:22.318 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:13:22.313+0000 7f1e4da75700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e48102fa0 0x7f1e4819fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:13:22.318 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:13:22.313+0000 7f1e4da75700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4810d040 con 0x7f1e48102fa0 2024-06-03T03:13:22.318 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:13:22.313+0000 7f1e4da75700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4810cec0 con 0x7f1e48101400 2024-06-03T03:13:22.318 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:13:22.313+0000 7f1e4da75700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4810d1c0 con 0x7f1e48102600 2024-06-03T03:13:22.318 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:13:22.313+0000 7f1e477fe700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e48101400 0x7f1e4806b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:13:22.319 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:13:22.313+0000 7f1e477fe700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e48101400 0x7f1e4806b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:34640/0 (socket says 172.21.15.151:34640) 2024-06-03T03:13:22.319 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:13:22.313+0000 7f1e477fe700 1 -- 172.21.15.151:0/1961482139 learned_addr learned my addr 172.21.15.151:0/1961482139 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-06-03T03:13:22.319 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:13:22.313+0000 7f1e46ffd700 1 --2- 172.21.15.151:0/1961482139 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e48102600 0x7f1e4819d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:13:22.319 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:13:22.313+0000 7f1e477fe700 1 -- 172.21.15.151:0/1961482139 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e48102600 msgr2=0x7f1e4819d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:13:22.319 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:13:22.313+0000 7f1e477fe700 1 --2- 172.21.15.151:0/1961482139 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e48102600 0x7f1e4819d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:13:22.319 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:13:22.313+0000 7f1e477fe700 1 -- 172.21.15.151:0/1961482139 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e48102fa0 msgr2=0x7f1e4819fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:13:22.319 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:13:22.313+0000 7f1e477fe700 1 --2- 172.21.15.151:0/1961482139 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e48102fa0 0x7f1e4819fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:13:22.320 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:13:22.313+0000 7f1e477fe700 1 -- 172.21.15.151:0/1961482139 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e481a0390 con 0x7f1e48101400 2024-06-03T03:13:22.320 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:13:22.313+0000 7f1e477fe700 1 --2- 172.21.15.151:0/1961482139 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e48101400 0x7f1e4806b740 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1e3800e970 tx=0x7f1e3800ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:13:22.320 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:13:22.314+0000 7f1e44ff9700 1 -- 172.21.15.151:0/1961482139 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3800cbd0 con 0x7f1e48101400 2024-06-03T03:13:22.320 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:13:22.314+0000 7f1e44ff9700 1 -- 172.21.15.151:0/1961482139 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e38004500 con 0x7f1e48101400 2024-06-03T03:13:22.320 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:13:22.314+0000 7f1e4da75700 1 -- 172.21.15.151:0/1961482139 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e481a0680 con 0x7f1e48101400 2024-06-03T03:13:22.320 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:13:22.314+0000 7f1e4da75700 1 -- 172.21.15.151:0/1961482139 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e481a0bc0 con 0x7f1e48101400 2024-06-03T03:13:22.320 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:13:22.314+0000 7f1e44ff9700 1 -- 172.21.15.151:0/1961482139 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e38003e30 con 0x7f1e48101400 2024-06-03T03:13:22.320 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:13:22.315+0000 7f1e4da75700 1 -- 172.21.15.151:0/1961482139 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e48069180 con 0x7f1e48101400 2024-06-03T03:13:22.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:13:22.316+0000 7f1e44ff9700 1 -- 172.21.15.151:0/1961482139 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 15) ==== 137778+0+0 (secure 0 0 0) 0x7f1e38004020 con 0x7f1e48101400 2024-06-03T03:13:22.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:13:22.316+0000 7f1e44ff9700 1 --2- 172.21.15.151:0/1961482139 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f1e300a2b90 0x7f1e300a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:13:22.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:13:22.316+0000 7f1e44ff9700 1 -- 172.21.15.151:0/1961482139 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f1e380cd8c0 con 0x7f1e48101400 2024-06-03T03:13:22.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:13:22.317+0000 7f1e46ffd700 1 --2- 172.21.15.151:0/1961482139 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f1e300a2b90 0x7f1e300a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:13:22.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:13:22.317+0000 7f1e46ffd700 1 --2- 172.21.15.151:0/1961482139 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f1e300a2b90 0x7f1e300a5050 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1e4806bcc0 tx=0x7f1e3c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:13:22.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:13:22.321+0000 7f1e44ff9700 1 -- 172.21.15.151:0/1961482139 <== mon.2 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1e3800b2c0 con 0x7f1e48101400 2024-06-03T03:13:22.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:13:22.523+0000 7f1e4da75700 1 -- 172.21.15.151:0/1961482139 --> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f1e480628d0 con 0x7f1e300a2b90 2024-06-03T03:13:22.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:22 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T03:13:22.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:22 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T03:13:22.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:22 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:13:22.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:22 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:13:22.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:22 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:13:22.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:22 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:13:23.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:22 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T03:13:23.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:22 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T03:13:23.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:22 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:13:23.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:22 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:13:23.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:22 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:13:23.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:22 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:13:23.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:22 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T03:13:23.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:22 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T03:13:23.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:22 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:13:23.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:22 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:13:23.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:22 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:13:23.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:22 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:13:24.107 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:23 smithi151 ceph-mon[37043]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:13:24.107 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:23 smithi151 ceph-mon[37043]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T03:13:24.107 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:23 smithi151 ceph-mon[37043]: Detected new or changed devices on smithi117 2024-06-03T03:13:24.107 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:23 smithi151 ceph-mon[37043]: Adjusting osd_memory_target on smithi117 to 17089M 2024-06-03T03:13:24.107 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:23 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:13:24.108 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:23 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:13:24.108 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:23 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:13:24.108 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:23 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:13:24.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:23 smithi117 ceph-mon[31649]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:13:24.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:23 smithi117 ceph-mon[31649]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T03:13:24.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:23 smithi117 ceph-mon[31649]: Detected new or changed devices on smithi117 2024-06-03T03:13:24.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:23 smithi117 ceph-mon[31649]: Adjusting osd_memory_target on smithi117 to 17089M 2024-06-03T03:13:24.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:23 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:13:24.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:23 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:13:24.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:23 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:13:24.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:23 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:13:24.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:23 smithi183 ceph-mon[37326]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:13:24.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:23 smithi183 ceph-mon[37326]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T03:13:24.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:23 smithi183 ceph-mon[37326]: Detected new or changed devices on smithi117 2024-06-03T03:13:24.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:23 smithi183 ceph-mon[37326]: Adjusting osd_memory_target on smithi117 to 17089M 2024-06-03T03:13:24.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:23 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:13:24.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:23 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:13:24.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:23 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:13:24.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:23 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:13:26.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:25 smithi151 ceph-mon[37043]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T03:13:26.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:25 smithi151 ceph-mon[37043]: from='client.? 172.21.15.151:0/3070638015' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5b12b371-f155-4b67-a470-25cbe59a2999"}]: dispatch 2024-06-03T03:13:26.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:25 smithi151 ceph-mon[37043]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5b12b371-f155-4b67-a470-25cbe59a2999"}]: dispatch 2024-06-03T03:13:26.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:25 smithi151 ceph-mon[37043]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5b12b371-f155-4b67-a470-25cbe59a2999"}]': finished 2024-06-03T03:13:26.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:25 smithi151 ceph-mon[37043]: osdmap e10: 2 total, 1 up, 2 in 2024-06-03T03:13:26.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:25 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-03T03:13:26.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:25 smithi117 ceph-mon[31649]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T03:13:26.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:25 smithi117 ceph-mon[31649]: from='client.? 172.21.15.151:0/3070638015' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5b12b371-f155-4b67-a470-25cbe59a2999"}]: dispatch 2024-06-03T03:13:26.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:25 smithi117 ceph-mon[31649]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5b12b371-f155-4b67-a470-25cbe59a2999"}]: dispatch 2024-06-03T03:13:26.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:25 smithi117 ceph-mon[31649]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5b12b371-f155-4b67-a470-25cbe59a2999"}]': finished 2024-06-03T03:13:26.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:25 smithi117 ceph-mon[31649]: osdmap e10: 2 total, 1 up, 2 in 2024-06-03T03:13:26.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:25 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-03T03:13:26.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:25 smithi183 ceph-mon[37326]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T03:13:26.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:25 smithi183 ceph-mon[37326]: from='client.? 172.21.15.151:0/3070638015' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5b12b371-f155-4b67-a470-25cbe59a2999"}]: dispatch 2024-06-03T03:13:26.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:25 smithi183 ceph-mon[37326]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5b12b371-f155-4b67-a470-25cbe59a2999"}]: dispatch 2024-06-03T03:13:26.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:25 smithi183 ceph-mon[37326]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5b12b371-f155-4b67-a470-25cbe59a2999"}]': finished 2024-06-03T03:13:26.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:25 smithi183 ceph-mon[37326]: osdmap e10: 2 total, 1 up, 2 in 2024-06-03T03:13:26.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:25 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-03T03:13:27.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:26 smithi151 ceph-mon[37043]: from='client.? 172.21.15.151:0/326035758' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T03:13:27.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:26 smithi117 ceph-mon[31649]: from='client.? 172.21.15.151:0/326035758' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T03:13:27.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:26 smithi183 ceph-mon[37326]: from='client.? 172.21.15.151:0/326035758' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T03:13:27.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:27 smithi117 ceph-mon[31649]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T03:13:28.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:27 smithi151 ceph-mon[37043]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T03:13:28.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:27 smithi183 ceph-mon[37326]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T03:13:30.154 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:29 smithi151 ceph-mon[37043]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T03:13:30.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:29 smithi117 ceph-mon[31649]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T03:13:30.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:29 smithi183 ceph-mon[37326]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T03:13:31.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:31 smithi151 ceph-mon[37043]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T03:13:32.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:31 smithi117 ceph-mon[31649]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T03:13:32.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:31 smithi183 ceph-mon[37326]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T03:13:33.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:32 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-06-03T03:13:33.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:32 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:13:33.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:32 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-06-03T03:13:33.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:32 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:13:33.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:32 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-06-03T03:13:33.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:32 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:13:33.906 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:33 smithi151 ceph-mon[37043]: Deploying daemon osd.1 on smithi151 2024-06-03T03:13:33.906 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:33 smithi151 ceph-mon[37043]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T03:13:34.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:33 smithi117 ceph-mon[31649]: Deploying daemon osd.1 on smithi151 2024-06-03T03:13:34.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:33 smithi117 ceph-mon[31649]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T03:13:34.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:33 smithi183 ceph-mon[37326]: Deploying daemon osd.1 on smithi151 2024-06-03T03:13:34.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:33 smithi183 ceph-mon[37326]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T03:13:36.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:35 smithi151 ceph-mon[37043]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T03:13:36.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:35 smithi117 ceph-mon[31649]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T03:13:36.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:35 smithi183 ceph-mon[37326]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T03:13:37.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:37 smithi117 ceph-mon[31649]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T03:13:37.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:37 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:13:37.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:37 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:13:37.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:37 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:13:38.065 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:37 smithi151 ceph-mon[37043]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T03:13:38.065 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:37 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:13:38.066 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:37 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:13:38.066 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:37 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:13:38.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:37 smithi183 ceph-mon[37326]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T03:13:38.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:37 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:13:38.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:37 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:13:38.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:37 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:13:40.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:39 smithi151 ceph-mon[37043]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T03:13:40.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:39 smithi117 ceph-mon[31649]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T03:13:40.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:39 smithi183 ceph-mon[37326]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T03:13:41.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:13:41.007+0000 7f1e44ff9700 1 -- 172.21.15.151:0/1961482139 <== mgr.14152 v2:172.21.15.117:6800/1398160887 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1e480628d0 con 0x7f1e300a2b90 2024-06-03T03:13:41.008 INFO:teuthology.orchestra.run.smithi151.stdout:Created osd(s) 1 on host 'smithi151' 2024-06-03T03:13:41.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:13:41.009+0000 7f1e4da75700 1 -- 172.21.15.151:0/1961482139 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f1e300a2b90 msgr2=0x7f1e300a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:13:41.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:13:41.009+0000 7f1e4da75700 1 --2- 172.21.15.151:0/1961482139 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f1e300a2b90 0x7f1e300a5050 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1e4806bcc0 tx=0x7f1e3c009190 comp rx=0 tx=0).stop 2024-06-03T03:13:41.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:13:41.009+0000 7f1e4da75700 1 -- 172.21.15.151:0/1961482139 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e48101400 msgr2=0x7f1e4806b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:13:41.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:13:41.009+0000 7f1e4da75700 1 --2- 172.21.15.151:0/1961482139 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e48101400 0x7f1e4806b740 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1e3800e970 tx=0x7f1e3800ec80 comp rx=0 tx=0).stop 2024-06-03T03:13:41.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:13:41.010+0000 7f1e4da75700 1 -- 172.21.15.151:0/1961482139 shutdown_connections 2024-06-03T03:13:41.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:13:41.010+0000 7f1e4da75700 1 --2- 172.21.15.151:0/1961482139 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f1e300a2b90 0x7f1e300a5050 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:13:41.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:13:41.010+0000 7f1e4da75700 1 --2- 172.21.15.151:0/1961482139 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e48102fa0 0x7f1e4819fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:13:41.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:13:41.010+0000 7f1e4da75700 1 --2- 172.21.15.151:0/1961482139 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e48101400 0x7f1e4806b740 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:13:41.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:13:41.010+0000 7f1e4da75700 1 --2- 172.21.15.151:0/1961482139 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e48102600 0x7f1e4819d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:13:41.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:13:41.010+0000 7f1e4da75700 1 -- 172.21.15.151:0/1961482139 >> 172.21.15.151:0/1961482139 conn(0x7f1e480fc9c0 msgr2=0x7f1e48105900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:13:41.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:13:41.010+0000 7f1e4da75700 1 -- 172.21.15.151:0/1961482139 shutdown_connections 2024-06-03T03:13:41.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:13:41.010+0000 7f1e4da75700 1 -- 172.21.15.151:0/1961482139 wait complete. 2024-06-03T03:13:41.280 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:41 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:13:41.281 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:41 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:13:41.281 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:41 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:13:41.281 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:41 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:13:41.281 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:41 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:13:41.281 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:41 smithi151 ceph-mon[37043]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T03:13:41.281 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:41 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:13:41.281 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:41 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:13:41.281 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:41 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:13:41.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:41 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:13:41.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:41 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:13:41.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:41 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:13:41.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:41 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:13:41.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:41 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:13:41.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:41 smithi117 ceph-mon[31649]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T03:13:41.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:41 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:13:41.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:41 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:13:41.448 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:41 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:13:41.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:41 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:13:41.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:41 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:13:41.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:41 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:13:41.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:41 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:13:41.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:41 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:13:41.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:41 smithi183 ceph-mon[37326]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T03:13:41.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:41 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:13:41.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:41 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:13:41.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:41 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:13:41.702 DEBUG:teuthology.orchestra.run.smithi151:osd.1> sudo journalctl -f -n 0 -u ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@osd.1.service 2024-06-03T03:13:41.704 INFO:tasks.cephadm:Deploying osd.2 on smithi183 with /dev/vg_nvme/lv_4... 2024-06-03T03:13:41.704 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-06-03T03:13:41.947 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.c/config 2024-06-03T03:13:42.397 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:42 smithi183 ceph-mon[37326]: from='osd.1 [v2:172.21.15.151:6800/262491158,v1:172.21.15.151:6801/262491158]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-03T03:13:42.397 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:42 smithi183 ceph-mon[37326]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-03T03:13:42.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:42 smithi151 ceph-mon[37043]: from='osd.1 [v2:172.21.15.151:6800/262491158,v1:172.21.15.151:6801/262491158]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-03T03:13:42.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:42 smithi151 ceph-mon[37043]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-03T03:13:42.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:42 smithi117 ceph-mon[31649]: from='osd.1 [v2:172.21.15.151:6800/262491158,v1:172.21.15.151:6801/262491158]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-03T03:13:42.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:42 smithi117 ceph-mon[31649]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-03T03:13:43.405 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:43 smithi183 ceph-mon[37326]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-06-03T03:13:43.405 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:43 smithi183 ceph-mon[37326]: osdmap e11: 2 total, 1 up, 2 in 2024-06-03T03:13:43.406 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:43 smithi183 ceph-mon[37326]: from='osd.1 [v2:172.21.15.151:6800/262491158,v1:172.21.15.151:6801/262491158]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-06-03T03:13:43.406 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:43 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-03T03:13:43.406 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:43 smithi183 ceph-mon[37326]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-06-03T03:13:43.406 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:43 smithi183 ceph-mon[37326]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T03:13:43.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:43 smithi151 ceph-mon[37043]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-06-03T03:13:43.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:43 smithi151 ceph-mon[37043]: osdmap e11: 2 total, 1 up, 2 in 2024-06-03T03:13:43.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:43 smithi151 ceph-mon[37043]: from='osd.1 [v2:172.21.15.151:6800/262491158,v1:172.21.15.151:6801/262491158]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-06-03T03:13:43.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:43 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-03T03:13:43.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:43 smithi151 ceph-mon[37043]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-06-03T03:13:43.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:43 smithi151 ceph-mon[37043]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T03:13:43.407 INFO:journalctl@ceph.osd.1.smithi151.stdout:Jun 03 03:13:43 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-1[41582]: 2024-06-03T03:13:43.183+0000 7f8047468700 -1 osd.1 0 waiting for initial osdmap 2024-06-03T03:13:43.407 INFO:journalctl@ceph.osd.1.smithi151.stdout:Jun 03 03:13:43 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-1[41582]: 2024-06-03T03:13:43.188+0000 7f804325b700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-03T03:13:43.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:43 smithi117 ceph-mon[31649]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-06-03T03:13:43.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:43 smithi117 ceph-mon[31649]: osdmap e11: 2 total, 1 up, 2 in 2024-06-03T03:13:43.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:43 smithi117 ceph-mon[31649]: from='osd.1 [v2:172.21.15.151:6800/262491158,v1:172.21.15.151:6801/262491158]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-06-03T03:13:43.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:43 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-03T03:13:43.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:43 smithi117 ceph-mon[31649]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-06-03T03:13:43.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:43 smithi117 ceph-mon[31649]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T03:13:43.934 INFO:teuthology.orchestra.run.smithi183.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-06-03T03:13:43.934 INFO:teuthology.orchestra.run.smithi183.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-06-03T03:13:43.935 INFO:teuthology.orchestra.run.smithi183.stderr: stderr: 10+0 records in 2024-06-03T03:13:43.935 INFO:teuthology.orchestra.run.smithi183.stderr:10+0 records out 2024-06-03T03:13:43.935 INFO:teuthology.orchestra.run.smithi183.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0122608 s, 855 MB/s 2024-06-03T03:13:43.935 INFO:teuthology.orchestra.run.smithi183.stderr:--> Zapping successful for: 2024-06-03T03:13:44.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:44 smithi151 ceph-mon[37043]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-06-03T03:13:44.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:44 smithi151 ceph-mon[37043]: osdmap e12: 2 total, 1 up, 2 in 2024-06-03T03:13:44.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:44 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-03T03:13:44.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:44 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-03T03:13:44.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:44 smithi183 ceph-mon[37326]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-06-03T03:13:44.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:44 smithi183 ceph-mon[37326]: osdmap e12: 2 total, 1 up, 2 in 2024-06-03T03:13:44.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:44 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-03T03:13:44.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:44 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-03T03:13:44.535 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi183:vg_nvme/lv_4 2024-06-03T03:13:44.696 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:44 smithi117 ceph-mon[31649]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-06-03T03:13:44.697 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:44 smithi117 ceph-mon[31649]: osdmap e12: 2 total, 1 up, 2 in 2024-06-03T03:13:44.697 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:44 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-03T03:13:44.697 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:44 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-03T03:13:44.773 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.c/config 2024-06-03T03:13:45.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:45 smithi183 ceph-mon[37326]: purged_snaps scrub starts 2024-06-03T03:13:45.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:45 smithi183 ceph-mon[37326]: purged_snaps scrub ok 2024-06-03T03:13:45.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:45 smithi183 ceph-mon[37326]: osd.1 [v2:172.21.15.151:6800/262491158,v1:172.21.15.151:6801/262491158] boot 2024-06-03T03:13:45.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:45 smithi183 ceph-mon[37326]: osdmap e13: 2 total, 2 up, 2 in 2024-06-03T03:13:45.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:45 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-03T03:13:45.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:45 smithi183 ceph-mon[37326]: Detected new or changed devices on smithi151 2024-06-03T03:13:45.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:45 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:13:45.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:45 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:13:45.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:45 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:13:45.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:45 smithi183 ceph-mon[37326]: Adjusting osd_memory_target on smithi151 to 17088M 2024-06-03T03:13:45.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:45 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:13:45.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:45 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:13:45.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:45 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:13:45.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:45 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:13:45.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:45 smithi183 ceph-mon[37326]: pgmap v73: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T03:13:45.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:45 smithi151 ceph-mon[37043]: purged_snaps scrub starts 2024-06-03T03:13:45.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:45 smithi151 ceph-mon[37043]: purged_snaps scrub ok 2024-06-03T03:13:45.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:45 smithi151 ceph-mon[37043]: osd.1 [v2:172.21.15.151:6800/262491158,v1:172.21.15.151:6801/262491158] boot 2024-06-03T03:13:45.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:45 smithi151 ceph-mon[37043]: osdmap e13: 2 total, 2 up, 2 in 2024-06-03T03:13:45.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:45 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-03T03:13:45.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:45 smithi151 ceph-mon[37043]: Detected new or changed devices on smithi151 2024-06-03T03:13:45.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:45 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:13:45.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:45 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:13:45.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:45 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:13:45.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:45 smithi151 ceph-mon[37043]: Adjusting osd_memory_target on smithi151 to 17088M 2024-06-03T03:13:45.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:45 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:13:45.658 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:45 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:13:45.658 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:45 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:13:45.658 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:45 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:13:45.658 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:45 smithi151 ceph-mon[37043]: pgmap v73: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T03:13:45.697 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:45 smithi117 ceph-mon[31649]: purged_snaps scrub starts 2024-06-03T03:13:45.697 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:45 smithi117 ceph-mon[31649]: purged_snaps scrub ok 2024-06-03T03:13:45.697 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:45 smithi117 ceph-mon[31649]: osd.1 [v2:172.21.15.151:6800/262491158,v1:172.21.15.151:6801/262491158] boot 2024-06-03T03:13:45.697 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:45 smithi117 ceph-mon[31649]: osdmap e13: 2 total, 2 up, 2 in 2024-06-03T03:13:45.697 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:45 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-03T03:13:45.697 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:45 smithi117 ceph-mon[31649]: Detected new or changed devices on smithi151 2024-06-03T03:13:45.697 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:45 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:13:45.697 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:45 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:13:45.698 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:45 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:13:45.698 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:45 smithi117 ceph-mon[31649]: Adjusting osd_memory_target on smithi151 to 17088M 2024-06-03T03:13:45.698 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:45 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:13:45.698 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:45 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:13:45.698 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:45 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:13:45.698 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:45 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:13:45.698 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:45 smithi117 ceph-mon[31649]: pgmap v73: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-03T03:13:46.323 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:13:46.316+0000 7fe11bf17700 1 -- 172.21.15.183:0/3763355002 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe114104050 msgr2=0x7fe11410a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:13:46.323 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:13:46.316+0000 7fe11bf17700 1 --2- 172.21.15.183:0/3763355002 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe114104050 0x7fe11410a900 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe10c00b0a0 tx=0x7fe10c00b3b0 comp rx=0 tx=0).stop 2024-06-03T03:13:46.323 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:13:46.316+0000 7fe11bf17700 1 -- 172.21.15.183:0/3763355002 shutdown_connections 2024-06-03T03:13:46.323 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:13:46.316+0000 7fe11bf17700 1 --2- 172.21.15.183:0/3763355002 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe114104050 0x7fe11410a900 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:13:46.323 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:13:46.316+0000 7fe11bf17700 1 --2- 172.21.15.183:0/3763355002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe1141024b0 0x7fe1141028d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:13:46.323 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:13:46.316+0000 7fe11bf17700 1 --2- 172.21.15.183:0/3763355002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1141036b0 0x7fe114103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:13:46.323 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:13:46.316+0000 7fe11bf17700 1 -- 172.21.15.183:0/3763355002 >> 172.21.15.183:0/3763355002 conn(0x7fe1140fda50 msgr2=0x7fe1140ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:13:46.323 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:13:46.316+0000 7fe11bf17700 1 -- 172.21.15.183:0/3763355002 shutdown_connections 2024-06-03T03:13:46.323 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:13:46.316+0000 7fe11bf17700 1 -- 172.21.15.183:0/3763355002 wait complete. 2024-06-03T03:13:46.323 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:13:46.317+0000 7fe11bf17700 1 Processor -- start 2024-06-03T03:13:46.324 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:13:46.317+0000 7fe11bf17700 1 -- start start 2024-06-03T03:13:46.324 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:13:46.317+0000 7fe11bf17700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe1141024b0 0x7fe114076230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:13:46.324 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:13:46.317+0000 7fe11bf17700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe1141036b0 0x7fe114074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:13:46.324 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:13:46.317+0000 7fe11bf17700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe114104050 0x7fe114074dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:13:46.324 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:13:46.317+0000 7fe11bf17700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe11410e1e0 con 0x7fe114104050 2024-06-03T03:13:46.324 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:13:46.317+0000 7fe11bf17700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe11410e060 con 0x7fe1141024b0 2024-06-03T03:13:46.324 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:13:46.317+0000 7fe11bf17700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe11410e360 con 0x7fe1141036b0 2024-06-03T03:13:46.324 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:13:46.317+0000 7fe1194b2700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe1141036b0 0x7fe114074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:13:46.324 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:13:46.317+0000 7fe1194b2700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe1141036b0 0x7fe114074880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:41816/0 (socket says 172.21.15.183:41816) 2024-06-03T03:13:46.324 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:13:46.317+0000 7fe1194b2700 1 -- 172.21.15.183:0/3336510888 learned_addr learned my addr 172.21.15.183:0/3336510888 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-06-03T03:13:46.325 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:13:46.318+0000 7fe119cb3700 1 --2- 172.21.15.183:0/3336510888 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe1141024b0 0x7fe114076230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:13:46.325 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:13:46.318+0000 7fe1194b2700 1 -- 172.21.15.183:0/3336510888 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe1141024b0 msgr2=0x7fe114076230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:13:46.325 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:13:46.318+0000 7fe1194b2700 1 --2- 172.21.15.183:0/3336510888 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe1141024b0 0x7fe114076230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:13:46.325 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:13:46.318+0000 7fe1194b2700 1 -- 172.21.15.183:0/3336510888 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe114104050 msgr2=0x7fe114074dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:13:46.325 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:13:46.318+0000 7fe1194b2700 1 --2- 172.21.15.183:0/3336510888 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe114104050 0x7fe114074dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:13:46.325 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:13:46.318+0000 7fe1194b2700 1 -- 172.21.15.183:0/3336510888 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe114075670 con 0x7fe1141036b0 2024-06-03T03:13:46.325 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:13:46.318+0000 7fe1194b2700 1 --2- 172.21.15.183:0/3336510888 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe1141036b0 0x7fe114074880 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe10400b530 tx=0x7fe10400b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:13:46.325 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:13:46.318+0000 7fe10affd700 1 -- 172.21.15.183:0/3336510888 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe104015020 con 0x7fe1141036b0 2024-06-03T03:13:46.325 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:13:46.318+0000 7fe10affd700 1 -- 172.21.15.183:0/3336510888 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe104011d10 con 0x7fe1141036b0 2024-06-03T03:13:46.325 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:13:46.318+0000 7fe11bf17700 1 -- 172.21.15.183:0/3336510888 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1141a29b0 con 0x7fe1141036b0 2024-06-03T03:13:46.325 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:13:46.318+0000 7fe10affd700 1 -- 172.21.15.183:0/3336510888 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe104010e50 con 0x7fe1141036b0 2024-06-03T03:13:46.325 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:13:46.318+0000 7fe11bf17700 1 -- 172.21.15.183:0/3336510888 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1141a2f40 con 0x7fe1141036b0 2024-06-03T03:13:46.325 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:13:46.319+0000 7fe11bf17700 1 -- 172.21.15.183:0/3336510888 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe114069180 con 0x7fe1141036b0 2024-06-03T03:13:46.326 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:13:46.320+0000 7fe10affd700 1 -- 172.21.15.183:0/3336510888 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 15) ==== 137778+0+0 (secure 0 0 0) 0x7fe104018020 con 0x7fe1141036b0 2024-06-03T03:13:46.326 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:13:46.320+0000 7fe10affd700 1 --2- 172.21.15.183:0/3336510888 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fe1000a2d30 0x7fe1000a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:13:46.326 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:13:46.321+0000 7fe10affd700 1 -- 172.21.15.183:0/3336510888 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7fe1040cd030 con 0x7fe1141036b0 2024-06-03T03:13:46.326 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:13:46.321+0000 7fe119cb3700 1 --2- 172.21.15.183:0/3336510888 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fe1000a2d30 0x7fe1000a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:13:46.326 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:13:46.322+0000 7fe119cb3700 1 --2- 172.21.15.183:0/3336510888 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fe1000a2d30 0x7fe1000a51f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe110006fd0 tx=0x7fe110008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:13:46.326 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:13:46.325+0000 7fe10affd700 1 -- 172.21.15.183:0/3336510888 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe10401f020 con 0x7fe1141036b0 2024-06-03T03:13:46.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:46 smithi183 ceph-mon[37326]: osdmap e14: 2 total, 2 up, 2 in 2024-06-03T03:13:46.527 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:13:46.527+0000 7fe11bf17700 1 -- 172.21.15.183:0/3336510888 --> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fe1140628d0 con 0x7fe1000a2d30 2024-06-03T03:13:46.656 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:46 smithi151 ceph-mon[37043]: osdmap e14: 2 total, 2 up, 2 in 2024-06-03T03:13:46.696 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:46 smithi117 ceph-mon[31649]: osdmap e14: 2 total, 2 up, 2 in 2024-06-03T03:13:47.451 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:47 smithi183 ceph-mon[37326]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:13:47.451 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:47 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T03:13:47.451 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:47 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T03:13:47.451 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:47 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:13:47.451 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:47 smithi183 ceph-mon[37326]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T03:13:47.656 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:47 smithi151 ceph-mon[37043]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:13:47.656 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:47 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T03:13:47.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:47 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T03:13:47.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:47 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:13:47.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:47 smithi151 ceph-mon[37043]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T03:13:47.696 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:47 smithi117 ceph-mon[31649]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:13:47.697 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:47 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T03:13:47.697 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:47 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T03:13:47.697 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:47 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:13:47.697 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:47 smithi117 ceph-mon[31649]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T03:13:49.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:49 smithi183 ceph-mon[37326]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T03:13:49.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:49 smithi183 ceph-mon[37326]: from='client.? 172.21.15.183:0/1537139146' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dbc17818-c54c-4ecf-b656-03578b95d8c4"}]: dispatch 2024-06-03T03:13:49.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:49 smithi183 ceph-mon[37326]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dbc17818-c54c-4ecf-b656-03578b95d8c4"}]: dispatch 2024-06-03T03:13:49.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:49 smithi183 ceph-mon[37326]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dbc17818-c54c-4ecf-b656-03578b95d8c4"}]': finished 2024-06-03T03:13:49.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:49 smithi183 ceph-mon[37326]: osdmap e15: 3 total, 2 up, 3 in 2024-06-03T03:13:49.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:49 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-03T03:13:50.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:49 smithi151 ceph-mon[37043]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T03:13:50.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:49 smithi151 ceph-mon[37043]: from='client.? 172.21.15.183:0/1537139146' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dbc17818-c54c-4ecf-b656-03578b95d8c4"}]: dispatch 2024-06-03T03:13:50.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:49 smithi151 ceph-mon[37043]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dbc17818-c54c-4ecf-b656-03578b95d8c4"}]: dispatch 2024-06-03T03:13:50.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:49 smithi151 ceph-mon[37043]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dbc17818-c54c-4ecf-b656-03578b95d8c4"}]': finished 2024-06-03T03:13:50.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:49 smithi151 ceph-mon[37043]: osdmap e15: 3 total, 2 up, 3 in 2024-06-03T03:13:50.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:49 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-03T03:13:50.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:49 smithi117 ceph-mon[31649]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T03:13:50.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:49 smithi117 ceph-mon[31649]: from='client.? 172.21.15.183:0/1537139146' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dbc17818-c54c-4ecf-b656-03578b95d8c4"}]: dispatch 2024-06-03T03:13:50.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:49 smithi117 ceph-mon[31649]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dbc17818-c54c-4ecf-b656-03578b95d8c4"}]: dispatch 2024-06-03T03:13:50.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:49 smithi117 ceph-mon[31649]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dbc17818-c54c-4ecf-b656-03578b95d8c4"}]': finished 2024-06-03T03:13:50.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:49 smithi117 ceph-mon[31649]: osdmap e15: 3 total, 2 up, 3 in 2024-06-03T03:13:50.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:49 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-03T03:13:51.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:50 smithi151 ceph-mon[37043]: from='client.? 172.21.15.183:0/767939699' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T03:13:51.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:50 smithi117 ceph-mon[31649]: from='client.? 172.21.15.183:0/767939699' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T03:13:51.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:50 smithi183 ceph-mon[37326]: from='client.? 172.21.15.183:0/767939699' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T03:13:52.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:51 smithi151 ceph-mon[37043]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T03:13:52.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:51 smithi117 ceph-mon[31649]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T03:13:52.231 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:51 smithi183 ceph-mon[37326]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T03:13:54.078 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:53 smithi183 ceph-mon[37326]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T03:13:54.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:53 smithi151 ceph-mon[37043]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T03:13:54.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:53 smithi117 ceph-mon[31649]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T03:13:55.838 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:55 smithi183 ceph-mon[37326]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T03:13:56.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:55 smithi151 ceph-mon[37043]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T03:13:56.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:55 smithi117 ceph-mon[31649]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T03:13:57.076 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:56 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-06-03T03:13:57.076 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:56 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:13:57.098 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:56 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-06-03T03:13:57.099 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:56 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:13:57.152 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:56 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-06-03T03:13:57.152 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:56 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:13:57.898 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:57 smithi183 ceph-mon[37326]: Deploying daemon osd.2 on smithi183 2024-06-03T03:13:57.898 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:57 smithi183 ceph-mon[37326]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T03:13:58.129 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:57 smithi117 ceph-mon[31649]: Deploying daemon osd.2 on smithi183 2024-06-03T03:13:58.130 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:57 smithi117 ceph-mon[31649]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T03:13:58.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:57 smithi151 ceph-mon[37043]: Deploying daemon osd.2 on smithi183 2024-06-03T03:13:58.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:57 smithi151 ceph-mon[37043]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T03:13:59.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:13:59 smithi117 ceph-mon[31649]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T03:14:00.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:13:59 smithi151 ceph-mon[37043]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T03:14:00.178 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:13:59 smithi183 ceph-mon[37326]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T03:14:01.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:01 smithi183 ceph-mon[37326]: pgmap v83: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T03:14:01.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:01 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:14:01.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:01 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:14:01.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:01 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:14:02.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:01 smithi151 ceph-mon[37043]: pgmap v83: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T03:14:02.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:01 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:14:02.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:01 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:14:02.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:01 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:14:02.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:01 smithi117 ceph-mon[31649]: pgmap v83: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T03:14:02.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:01 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:14:02.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:01 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:14:02.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:01 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:14:03.841 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:03 smithi183 ceph-mon[37326]: pgmap v84: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T03:14:03.841 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:03 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:14:03.841 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:03 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:14:03.841 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:03 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:14:03.842 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:03 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:14:03.842 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:03 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:14:04.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:03 smithi151 ceph-mon[37043]: pgmap v84: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T03:14:04.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:03 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:14:04.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:03 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:14:04.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:03 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:14:04.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:03 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:14:04.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:03 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:14:04.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:03 smithi117 ceph-mon[31649]: pgmap v84: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T03:14:04.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:03 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:14:04.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:03 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:14:04.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:03 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:14:04.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:03 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:14:04.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:03 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:14:05.106 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:14:05.105+0000 7fe10affd700 1 -- 172.21.15.183:0/3336510888 <== mgr.14152 v2:172.21.15.117:6800/1398160887 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe1140628d0 con 0x7fe1000a2d30 2024-06-03T03:14:05.106 INFO:teuthology.orchestra.run.smithi183.stdout:Created osd(s) 2 on host 'smithi183' 2024-06-03T03:14:05.108 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:14:05.107+0000 7fe11bf17700 1 -- 172.21.15.183:0/3336510888 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fe1000a2d30 msgr2=0x7fe1000a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:14:05.108 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:14:05.107+0000 7fe11bf17700 1 --2- 172.21.15.183:0/3336510888 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fe1000a2d30 0x7fe1000a51f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe110006fd0 tx=0x7fe110008040 comp rx=0 tx=0).stop 2024-06-03T03:14:05.109 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:14:05.107+0000 7fe11bf17700 1 -- 172.21.15.183:0/3336510888 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe1141036b0 msgr2=0x7fe114074880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:14:05.109 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:14:05.107+0000 7fe11bf17700 1 --2- 172.21.15.183:0/3336510888 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe1141036b0 0x7fe114074880 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe10400b530 tx=0x7fe10400b8f0 comp rx=0 tx=0).stop 2024-06-03T03:14:05.109 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:14:05.107+0000 7fe11bf17700 1 -- 172.21.15.183:0/3336510888 shutdown_connections 2024-06-03T03:14:05.109 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:14:05.107+0000 7fe11bf17700 1 --2- 172.21.15.183:0/3336510888 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fe1000a2d30 0x7fe1000a51f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:14:05.109 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:14:05.107+0000 7fe11bf17700 1 --2- 172.21.15.183:0/3336510888 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe114104050 0x7fe114074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:14:05.109 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:14:05.107+0000 7fe11bf17700 1 --2- 172.21.15.183:0/3336510888 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe1141024b0 0x7fe114076230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:14:05.109 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:14:05.107+0000 7fe11bf17700 1 --2- 172.21.15.183:0/3336510888 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe1141036b0 0x7fe114074880 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:14:05.109 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:14:05.107+0000 7fe11bf17700 1 -- 172.21.15.183:0/3336510888 >> 172.21.15.183:0/3336510888 conn(0x7fe1140fda50 msgr2=0x7fe114106a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:14:05.109 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:14:05.107+0000 7fe11bf17700 1 -- 172.21.15.183:0/3336510888 shutdown_connections 2024-06-03T03:14:05.109 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:14:05.107+0000 7fe11bf17700 1 -- 172.21.15.183:0/3336510888 wait complete. 2024-06-03T03:14:05.789 DEBUG:teuthology.orchestra.run.smithi183:osd.2> sudo journalctl -f -n 0 -u ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@osd.2.service 2024-06-03T03:14:05.792 INFO:tasks.cephadm:Deploying osd.3 on smithi117 with /dev/vg_nvme/lv_3... 2024-06-03T03:14:05.792 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-06-03T03:14:06.039 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:05 smithi183 ceph-mon[37326]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T03:14:06.039 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:05 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:14:06.039 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:05 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:14:06.039 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:05 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:14:06.039 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:05 smithi183 ceph-mon[37326]: from='osd.2 [v2:172.21.15.183:6800/3542453623,v1:172.21.15.183:6801/3542453623]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-03T03:14:06.039 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:05 smithi183 ceph-mon[37326]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-03T03:14:06.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:05 smithi151 ceph-mon[37043]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T03:14:06.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:05 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:14:06.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:05 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:14:06.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:05 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:14:06.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:05 smithi151 ceph-mon[37043]: from='osd.2 [v2:172.21.15.183:6800/3542453623,v1:172.21.15.183:6801/3542453623]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-03T03:14:06.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:05 smithi151 ceph-mon[37043]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-03T03:14:06.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:05 smithi117 ceph-mon[31649]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T03:14:06.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:05 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:14:06.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:05 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:14:06.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:05 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:14:06.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:05 smithi117 ceph-mon[31649]: from='osd.2 [v2:172.21.15.183:6800/3542453623,v1:172.21.15.183:6801/3542453623]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-03T03:14:06.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:05 smithi117 ceph-mon[31649]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-03T03:14:06.240 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:14:07.322 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:07 smithi183 ceph-mon[37326]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-06-03T03:14:07.322 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:07 smithi183 ceph-mon[37326]: osdmap e16: 3 total, 2 up, 3 in 2024-06-03T03:14:07.322 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:07 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-03T03:14:07.323 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:07 smithi183 ceph-mon[37326]: from='osd.2 [v2:172.21.15.183:6800/3542453623,v1:172.21.15.183:6801/3542453623]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-06-03T03:14:07.323 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:07 smithi183 ceph-mon[37326]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-06-03T03:14:07.323 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:07 smithi183 ceph-mon[37326]: pgmap v87: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T03:14:07.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:07 smithi151 ceph-mon[37043]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-06-03T03:14:07.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:07 smithi151 ceph-mon[37043]: osdmap e16: 3 total, 2 up, 3 in 2024-06-03T03:14:07.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:07 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-03T03:14:07.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:07 smithi151 ceph-mon[37043]: from='osd.2 [v2:172.21.15.183:6800/3542453623,v1:172.21.15.183:6801/3542453623]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-06-03T03:14:07.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:07 smithi151 ceph-mon[37043]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-06-03T03:14:07.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:07 smithi151 ceph-mon[37043]: pgmap v87: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T03:14:07.410 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:07 smithi117 ceph-mon[31649]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-06-03T03:14:07.410 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:07 smithi117 ceph-mon[31649]: osdmap e16: 3 total, 2 up, 3 in 2024-06-03T03:14:07.410 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:07 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-03T03:14:07.410 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:07 smithi117 ceph-mon[31649]: from='osd.2 [v2:172.21.15.183:6800/3542453623,v1:172.21.15.183:6801/3542453623]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-06-03T03:14:07.410 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:07 smithi117 ceph-mon[31649]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-06-03T03:14:07.411 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:07 smithi117 ceph-mon[31649]: pgmap v87: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T03:14:07.732 INFO:journalctl@ceph.osd.2.smithi183.stdout:Jun 03 03:14:07 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-2[42419]: 2024-06-03T03:14:07.550+0000 7ff9570ff700 -1 osd.2 0 waiting for initial osdmap 2024-06-03T03:14:07.732 INFO:journalctl@ceph.osd.2.smithi183.stdout:Jun 03 03:14:07 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-2[42419]: 2024-06-03T03:14:07.554+0000 7ff94eeea700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-03T03:14:08.157 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:08 smithi117 ceph-mon[31649]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-06-03T03:14:08.157 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:08 smithi117 ceph-mon[31649]: osdmap e17: 3 total, 2 up, 3 in 2024-06-03T03:14:08.157 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:08 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-03T03:14:08.157 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:08 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-03T03:14:08.157 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:08 smithi117 ceph-mon[31649]: from='osd.2 ' entity='osd.2' 2024-06-03T03:14:08.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:08 smithi151 ceph-mon[37043]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-06-03T03:14:08.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:08 smithi151 ceph-mon[37043]: osdmap e17: 3 total, 2 up, 3 in 2024-06-03T03:14:08.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:08 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-03T03:14:08.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:08 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-03T03:14:08.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:08 smithi151 ceph-mon[37043]: from='osd.2 ' entity='osd.2' 2024-06-03T03:14:08.443 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:08 smithi183 ceph-mon[37326]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-06-03T03:14:08.443 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:08 smithi183 ceph-mon[37326]: osdmap e17: 3 total, 2 up, 3 in 2024-06-03T03:14:08.443 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:08 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-03T03:14:08.443 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:08 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-03T03:14:08.443 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:08 smithi183 ceph-mon[37326]: from='osd.2 ' entity='osd.2' 2024-06-03T03:14:09.155 INFO:teuthology.orchestra.run.smithi117.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-06-03T03:14:09.155 INFO:teuthology.orchestra.run.smithi117.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-06-03T03:14:09.155 INFO:teuthology.orchestra.run.smithi117.stderr: stderr: 10+0 records in 2024-06-03T03:14:09.156 INFO:teuthology.orchestra.run.smithi117.stderr:10+0 records out 2024-06-03T03:14:09.156 INFO:teuthology.orchestra.run.smithi117.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0338098 s, 310 MB/s 2024-06-03T03:14:09.156 INFO:teuthology.orchestra.run.smithi117.stderr:--> Zapping successful for: 2024-06-03T03:14:09.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:09 smithi117 ceph-mon[31649]: purged_snaps scrub starts 2024-06-03T03:14:09.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:09 smithi117 ceph-mon[31649]: purged_snaps scrub ok 2024-06-03T03:14:09.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:09 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-03T03:14:09.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:09 smithi117 ceph-mon[31649]: osd.2 [v2:172.21.15.183:6800/3542453623,v1:172.21.15.183:6801/3542453623] boot 2024-06-03T03:14:09.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:09 smithi117 ceph-mon[31649]: osdmap e18: 3 total, 3 up, 3 in 2024-06-03T03:14:09.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:09 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-03T03:14:09.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:09 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-06-03T03:14:09.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:09 smithi151 ceph-mon[37043]: purged_snaps scrub starts 2024-06-03T03:14:09.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:09 smithi151 ceph-mon[37043]: purged_snaps scrub ok 2024-06-03T03:14:09.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:09 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-03T03:14:09.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:09 smithi151 ceph-mon[37043]: osd.2 [v2:172.21.15.183:6800/3542453623,v1:172.21.15.183:6801/3542453623] boot 2024-06-03T03:14:09.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:09 smithi151 ceph-mon[37043]: osdmap e18: 3 total, 3 up, 3 in 2024-06-03T03:14:09.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:09 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-03T03:14:09.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:09 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-06-03T03:14:09.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:09 smithi183 ceph-mon[37326]: purged_snaps scrub starts 2024-06-03T03:14:09.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:09 smithi183 ceph-mon[37326]: purged_snaps scrub ok 2024-06-03T03:14:09.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:09 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-03T03:14:09.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:09 smithi183 ceph-mon[37326]: osd.2 [v2:172.21.15.183:6800/3542453623,v1:172.21.15.183:6801/3542453623] boot 2024-06-03T03:14:09.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:09 smithi183 ceph-mon[37326]: osdmap e18: 3 total, 3 up, 3 in 2024-06-03T03:14:09.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:09 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-03T03:14:09.733 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:09 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-06-03T03:14:10.057 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi117:vg_nvme/lv_3 2024-06-03T03:14:10.496 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:14:10.648 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:10 smithi117 ceph-mon[31649]: pgmap v90: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T03:14:10.648 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:10 smithi117 ceph-mon[31649]: Detected new or changed devices on smithi183 2024-06-03T03:14:10.648 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:10 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:14:10.648 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:10 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:14:10.648 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:10 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:14:10.648 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:10 smithi117 ceph-mon[31649]: Adjusting osd_memory_target on smithi183 to 17088M 2024-06-03T03:14:10.648 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:10 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:14:10.648 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:10 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:14:10.648 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:10 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:14:10.649 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:10 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:14:10.649 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:10 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-06-03T03:14:10.649 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:10 smithi117 ceph-mon[31649]: osdmap e19: 3 total, 3 up, 3 in 2024-06-03T03:14:10.649 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:10 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-06-03T03:14:10.656 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:10 smithi151 ceph-mon[37043]: pgmap v90: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T03:14:10.656 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:10 smithi151 ceph-mon[37043]: Detected new or changed devices on smithi183 2024-06-03T03:14:10.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:10 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:14:10.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:10 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:14:10.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:10 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:14:10.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:10 smithi151 ceph-mon[37043]: Adjusting osd_memory_target on smithi183 to 17088M 2024-06-03T03:14:10.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:10 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:14:10.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:10 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:14:10.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:10 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:14:10.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:10 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:14:10.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:10 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-06-03T03:14:10.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:10 smithi151 ceph-mon[37043]: osdmap e19: 3 total, 3 up, 3 in 2024-06-03T03:14:10.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:10 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-06-03T03:14:10.674 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:10 smithi183 ceph-mon[37326]: pgmap v90: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-03T03:14:10.674 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:10 smithi183 ceph-mon[37326]: Detected new or changed devices on smithi183 2024-06-03T03:14:10.674 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:10 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:14:10.674 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:10 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:14:10.675 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:10 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:14:10.675 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:10 smithi183 ceph-mon[37326]: Adjusting osd_memory_target on smithi183 to 17088M 2024-06-03T03:14:10.675 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:10 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:14:10.675 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:10 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:14:10.675 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:10 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:14:10.675 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:10 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:14:10.675 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:10 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-06-03T03:14:10.675 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:10 smithi183 ceph-mon[37326]: osdmap e19: 3 total, 3 up, 3 in 2024-06-03T03:14:10.675 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:10 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-06-03T03:14:11.586 INFO:journalctl@ceph.osd.0.smithi117.stdout:Jun 03 03:14:11 smithi117 sudo[48703]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-03T03:14:11.586 INFO:journalctl@ceph.osd.0.smithi117.stdout:Jun 03 03:14:11 smithi117 sudo[48703]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-03T03:14:11.586 INFO:journalctl@ceph.osd.0.smithi117.stdout:Jun 03 03:14:11 smithi117 sudo[48703]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-03T03:14:11.586 INFO:journalctl@ceph.osd.0.smithi117.stdout:Jun 03 03:14:11 smithi117 sudo[48703]: pam_unix(sudo:session): session closed for user root 2024-06-03T03:14:11.850 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:11 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-06-03T03:14:11.850 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:11 smithi183 ceph-mon[37326]: osdmap e20: 3 total, 3 up, 3 in 2024-06-03T03:14:11.850 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:11 smithi183 ceph-mon[37326]: pgmap v93: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-03T03:14:11.862 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:11 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-06-03T03:14:11.862 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:11 smithi117 ceph-mon[31649]: osdmap e20: 3 total, 3 up, 3 in 2024-06-03T03:14:11.862 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:11 smithi117 ceph-mon[31649]: pgmap v93: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-03T03:14:11.906 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:11 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-06-03T03:14:11.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:11 smithi151 ceph-mon[37043]: osdmap e20: 3 total, 3 up, 3 in 2024-06-03T03:14:11.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:11 smithi151 ceph-mon[37043]: pgmap v93: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-03T03:14:12.163 INFO:journalctl@ceph.osd.0.smithi117.stdout:Jun 03 03:14:11 smithi117 sudo[48707]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-03T03:14:12.163 INFO:journalctl@ceph.osd.0.smithi117.stdout:Jun 03 03:14:11 smithi117 sudo[48707]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-03T03:14:12.163 INFO:journalctl@ceph.osd.0.smithi117.stdout:Jun 03 03:14:11 smithi117 sudo[48707]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-03T03:14:12.163 INFO:journalctl@ceph.osd.0.smithi117.stdout:Jun 03 03:14:11 smithi117 sudo[48707]: pam_unix(sudo:session): session closed for user root 2024-06-03T03:14:12.406 INFO:journalctl@ceph.osd.1.smithi151.stdout:Jun 03 03:14:12 smithi151 sudo[45266]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-03T03:14:12.407 INFO:journalctl@ceph.osd.1.smithi151.stdout:Jun 03 03:14:12 smithi151 sudo[45266]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-03T03:14:12.407 INFO:journalctl@ceph.osd.1.smithi151.stdout:Jun 03 03:14:12 smithi151 sudo[45266]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-03T03:14:12.407 INFO:journalctl@ceph.osd.1.smithi151.stdout:Jun 03 03:14:12 smithi151 sudo[45266]: pam_unix(sudo:session): session closed for user root 2024-06-03T03:14:12.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:14:12.693+0000 7f46e2400700 1 -- 172.21.15.117:0/3293501932 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f46dc105830 msgr2=0x7f46dc105c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:14:12.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:14:12.693+0000 7f46e2400700 1 --2- 172.21.15.117:0/3293501932 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f46dc105830 0x7f46dc105c90 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f46cc0099c0 tx=0x7f46cc009cd0 comp rx=0 tx=0).stop 2024-06-03T03:14:12.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:14:12.693+0000 7f46e2400700 1 -- 172.21.15.117:0/3293501932 shutdown_connections 2024-06-03T03:14:12.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:14:12.693+0000 7f46e2400700 1 --2- 172.21.15.117:0/3293501932 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46dc1061d0 0x7f46dc10ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:14:12.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:14:12.693+0000 7f46e2400700 1 --2- 172.21.15.117:0/3293501932 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f46dc104630 0x7f46dc104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:14:12.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:14:12.693+0000 7f46e2400700 1 --2- 172.21.15.117:0/3293501932 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f46dc105830 0x7f46dc105c90 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:14:12.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:14:12.693+0000 7f46e2400700 1 -- 172.21.15.117:0/3293501932 >> 172.21.15.117:0/3293501932 conn(0x7f46dc0ffbd0 msgr2=0x7f46dc102010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:14:12.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:14:12.693+0000 7f46e2400700 1 -- 172.21.15.117:0/3293501932 shutdown_connections 2024-06-03T03:14:12.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:14:12.693+0000 7f46e2400700 1 -- 172.21.15.117:0/3293501932 wait complete. 2024-06-03T03:14:12.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:14:12.694+0000 7f46e2400700 1 Processor -- start 2024-06-03T03:14:12.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:14:12.694+0000 7f46e2400700 1 -- start start 2024-06-03T03:14:12.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:14:12.695+0000 7f46e2400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f46dc104630 0x7f46dc19a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:14:12.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:14:12.695+0000 7f46e2400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f46dc105830 0x7f46dc19ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:14:12.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:14:12.695+0000 7f46e2400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46dc1061d0 0x7f46dc1a1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:14:12.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:14:12.695+0000 7f46e2400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f46dc110250 con 0x7f46dc1061d0 2024-06-03T03:14:12.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:14:12.695+0000 7f46e2400700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f46dc1100d0 con 0x7f46dc104630 2024-06-03T03:14:12.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:14:12.695+0000 7f46e2400700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f46dc1103d0 con 0x7f46dc105830 2024-06-03T03:14:12.697 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:14:12.695+0000 7f46e1000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f46dc104630 0x7f46dc19a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:14:12.697 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:14:12.695+0000 7f46dbe00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f46dc105830 0x7f46dc19ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:14:12.697 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:14:12.695+0000 7f46e1000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f46dc104630 0x7f46dc19a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:44794/0 (socket says 172.21.15.117:44794) 2024-06-03T03:14:12.697 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:14:12.695+0000 7f46dbe00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f46dc105830 0x7f46dc19ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:45038/0 (socket says 172.21.15.117:45038) 2024-06-03T03:14:12.697 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:14:12.695+0000 7f46e1000700 1 -- 172.21.15.117:0/310836624 learned_addr learned my addr 172.21.15.117:0/310836624 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:14:12.697 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:14:12.695+0000 7f46e1a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46dc1061d0 0x7f46dc1a1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:14:12.697 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:14:12.695+0000 7f46e1000700 1 -- 172.21.15.117:0/310836624 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f46dc105830 msgr2=0x7f46dc19ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:14:12.697 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:14:12.695+0000 7f46e1000700 1 --2- 172.21.15.117:0/310836624 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f46dc105830 0x7f46dc19ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:14:12.697 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:14:12.695+0000 7f46e1000700 1 -- 172.21.15.117:0/310836624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46dc1061d0 msgr2=0x7f46dc1a1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:14:12.697 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:14:12.695+0000 7f46e1000700 1 --2- 172.21.15.117:0/310836624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46dc1061d0 0x7f46dc1a1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:14:12.697 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:14:12.695+0000 7f46e1000700 1 -- 172.21.15.117:0/310836624 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46dc1a24a0 con 0x7f46dc104630 2024-06-03T03:14:12.698 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:14:12.696+0000 7f46e1000700 1 --2- 172.21.15.117:0/310836624 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f46dc104630 0x7f46dc19a840 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f46d000d980 tx=0x7f46d000dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:14:12.698 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:14:12.696+0000 7f46d9600700 1 -- 172.21.15.117:0/310836624 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d0017030 con 0x7f46dc104630 2024-06-03T03:14:12.698 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:14:12.696+0000 7f46d9600700 1 -- 172.21.15.117:0/310836624 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46d0004500 con 0x7f46dc104630 2024-06-03T03:14:12.698 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:14:12.696+0000 7f46e2400700 1 -- 172.21.15.117:0/310836624 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46dc1a2790 con 0x7f46dc104630 2024-06-03T03:14:12.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:14:12.696+0000 7f46d9600700 1 -- 172.21.15.117:0/310836624 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d00039e0 con 0x7f46dc104630 2024-06-03T03:14:12.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:14:12.696+0000 7f46e2400700 1 -- 172.21.15.117:0/310836624 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46dc1a2d20 con 0x7f46dc104630 2024-06-03T03:14:12.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:14:12.697+0000 7f46e2400700 1 -- 172.21.15.117:0/310836624 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46dc069180 con 0x7f46dc104630 2024-06-03T03:14:12.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:14:12.699+0000 7f46d9600700 1 -- 172.21.15.117:0/310836624 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f46d001a070 con 0x7f46dc104630 2024-06-03T03:14:12.701 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:14:12.700+0000 7f46d9600700 1 --2- 172.21.15.117:0/310836624 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f46c40a2ca0 0x7f46c40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:14:12.701 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:14:12.700+0000 7f46d9600700 1 -- 172.21.15.117:0/310836624 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 3237+0+0 (secure 0 0 0) 0x7f46d00cd560 con 0x7f46dc104630 2024-06-03T03:14:12.701 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:14:12.700+0000 7f46dbe00700 1 --2- 172.21.15.117:0/310836624 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f46c40a2ca0 0x7f46c40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:14:12.701 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:14:12.701+0000 7f46dbe00700 1 --2- 172.21.15.117:0/310836624 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f46c40a2ca0 0x7f46c40a5160 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f46cc000c00 tx=0x7f46cc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:14:12.705 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:14:12.704+0000 7f46d9600700 1 -- 172.21.15.117:0/310836624 <== mon.2 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f46d009b7f0 con 0x7f46dc104630 2024-06-03T03:14:12.713 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:12 smithi151 ceph-mon[37043]: osdmap e21: 3 total, 3 up, 3 in 2024-06-03T03:14:12.713 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:12 smithi151 ceph-mon[37043]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-06-03T03:14:12.713 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:12 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:14:12.713 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:12 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:14:12.713 INFO:journalctl@ceph.osd.1.smithi151.stdout:Jun 03 03:14:12 smithi151 sudo[45269]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-03T03:14:12.713 INFO:journalctl@ceph.osd.1.smithi151.stdout:Jun 03 03:14:12 smithi151 sudo[45269]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-03T03:14:12.713 INFO:journalctl@ceph.osd.1.smithi151.stdout:Jun 03 03:14:12 smithi151 sudo[45269]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-03T03:14:12.713 INFO:journalctl@ceph.osd.1.smithi151.stdout:Jun 03 03:14:12 smithi151 sudo[45269]: pam_unix(sudo:session): session closed for user root 2024-06-03T03:14:12.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:14:12.938+0000 7f46e2400700 1 -- 172.21.15.117:0/310836624 --> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f46dc0628d0 con 0x7f46c40a2ca0 2024-06-03T03:14:12.940 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:12 smithi117 ceph-mon[31649]: osdmap e21: 3 total, 3 up, 3 in 2024-06-03T03:14:12.940 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:12 smithi117 ceph-mon[31649]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-06-03T03:14:12.940 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:12 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:14:12.940 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:12 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:14:12.945 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:12 smithi183 ceph-mon[37326]: osdmap e21: 3 total, 3 up, 3 in 2024-06-03T03:14:12.945 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:12 smithi183 ceph-mon[37326]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-06-03T03:14:12.945 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:12 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:14:12.945 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:12 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:14:12.945 INFO:journalctl@ceph.osd.2.smithi183.stdout:Jun 03 03:14:12 smithi183 sudo[46185]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-03T03:14:12.945 INFO:journalctl@ceph.osd.2.smithi183.stdout:Jun 03 03:14:12 smithi183 sudo[46185]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-03T03:14:12.945 INFO:journalctl@ceph.osd.2.smithi183.stdout:Jun 03 03:14:12 smithi183 sudo[46185]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-03T03:14:12.945 INFO:journalctl@ceph.osd.2.smithi183.stdout:Jun 03 03:14:12 smithi183 sudo[46185]: pam_unix(sudo:session): session closed for user root 2024-06-03T03:14:13.232 INFO:journalctl@ceph.osd.2.smithi183.stdout:Jun 03 03:14:12 smithi183 sudo[46188]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-03T03:14:13.232 INFO:journalctl@ceph.osd.2.smithi183.stdout:Jun 03 03:14:12 smithi183 sudo[46188]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-03T03:14:13.232 INFO:journalctl@ceph.osd.2.smithi183.stdout:Jun 03 03:14:12 smithi183 sudo[46188]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-03T03:14:13.232 INFO:journalctl@ceph.osd.2.smithi183.stdout:Jun 03 03:14:12 smithi183 sudo[46188]: pam_unix(sudo:session): session closed for user root 2024-06-03T03:14:13.840 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:13 smithi117 sudo[48824]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-03T03:14:13.840 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:13 smithi117 sudo[48824]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-03T03:14:13.840 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:13 smithi117 sudo[48824]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-03T03:14:13.840 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:13 smithi117 sudo[48824]: pam_unix(sudo:session): session closed for user root 2024-06-03T03:14:14.090 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:14 smithi117 sudo[48886]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-03T03:14:14.342 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:14 smithi117 sudo[48886]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-03T03:14:14.342 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:14 smithi117 sudo[48886]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-03T03:14:14.343 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:14 smithi117 sudo[48886]: pam_unix(sudo:session): session closed for user root 2024-06-03T03:14:14.343 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:14 smithi117 ceph-mon[31649]: pgmap v95: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-03T03:14:14.343 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:14 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T03:14:14.343 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:14 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T03:14:14.343 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:14 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:14:14.343 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:14 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:14:14.343 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:14 smithi117 ceph-mon[31649]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-03T03:14:14.361 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:14 smithi183 ceph-mon[37326]: pgmap v95: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-03T03:14:14.361 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:14 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T03:14:14.361 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:14 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T03:14:14.361 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:14 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:14:14.361 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:14 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:14:14.361 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:14 smithi183 ceph-mon[37326]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-03T03:14:14.361 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:14 smithi183 sudo[46191]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-03T03:14:14.361 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:14 smithi183 sudo[46191]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-03T03:14:14.362 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:14 smithi183 sudo[46191]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-03T03:14:14.362 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:14 smithi183 sudo[46191]: pam_unix(sudo:session): session closed for user root 2024-06-03T03:14:14.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:14 smithi151 ceph-mon[37043]: pgmap v95: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-03T03:14:14.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:14 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T03:14:14.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:14 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T03:14:14.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:14 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:14:14.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:14 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:14:14.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:14 smithi151 ceph-mon[37043]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-03T03:14:14.731 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:14 smithi183 sudo[46194]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-03T03:14:14.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:14 smithi183 sudo[46194]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-03T03:14:14.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:14 smithi183 sudo[46194]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-03T03:14:14.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:14 smithi183 sudo[46194]: pam_unix(sudo:session): session closed for user root 2024-06-03T03:14:15.092 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:14 smithi151 sudo[45272]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-03T03:14:15.092 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:14 smithi151 sudo[45272]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-03T03:14:15.092 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:14 smithi151 sudo[45272]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-03T03:14:15.092 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:14 smithi151 sudo[45272]: pam_unix(sudo:session): session closed for user root 2024-06-03T03:14:15.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:15 smithi151 sudo[45275]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-03T03:14:15.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:15 smithi151 sudo[45275]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-03T03:14:15.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:15 smithi151 sudo[45275]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-03T03:14:15.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:15 smithi151 sudo[45275]: pam_unix(sudo:session): session closed for user root 2024-06-03T03:14:15.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:15 smithi151 ceph-mon[37043]: from='client.24194 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:14:15.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:15 smithi151 ceph-mon[37043]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-03T03:14:15.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:15 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-03T03:14:15.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:15 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T03:14:15.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:15 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T03:14:15.408 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:15 smithi151 ceph-mon[37043]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-03T03:14:15.408 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:15 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-03T03:14:15.408 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:15 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T03:14:15.408 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:15 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T03:14:15.408 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:15 smithi151 ceph-mon[37043]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-03T03:14:15.408 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:15 smithi151 ceph-mon[37043]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-03T03:14:15.408 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:15 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-03T03:14:15.408 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:15 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T03:14:15.408 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:15 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T03:14:15.408 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:15 smithi151 ceph-mon[37043]: pgmap v96: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-03T03:14:15.408 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:15 smithi151 ceph-mon[37043]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-03T03:14:15.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:15 smithi117 ceph-mon[31649]: from='client.24194 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:14:15.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:15 smithi117 ceph-mon[31649]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-03T03:14:15.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:15 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-03T03:14:15.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:15 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T03:14:15.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:15 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T03:14:15.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:15 smithi117 ceph-mon[31649]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-03T03:14:15.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:15 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-03T03:14:15.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:15 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T03:14:15.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:15 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T03:14:15.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:15 smithi117 ceph-mon[31649]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-03T03:14:15.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:15 smithi117 ceph-mon[31649]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-03T03:14:15.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:15 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-03T03:14:15.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:15 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T03:14:15.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:15 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T03:14:15.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:15 smithi117 ceph-mon[31649]: pgmap v96: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-03T03:14:15.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:15 smithi117 ceph-mon[31649]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-03T03:14:15.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:15 smithi183 ceph-mon[37326]: from='client.24194 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:14:15.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:15 smithi183 ceph-mon[37326]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-03T03:14:15.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:15 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-03T03:14:15.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:15 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T03:14:15.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:15 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T03:14:15.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:15 smithi183 ceph-mon[37326]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-03T03:14:15.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:15 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-03T03:14:15.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:15 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T03:14:15.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:15 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T03:14:15.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:15 smithi183 ceph-mon[37326]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-03T03:14:15.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:15 smithi183 ceph-mon[37326]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-03T03:14:15.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:15 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-03T03:14:15.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:15 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T03:14:15.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:15 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T03:14:15.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:15 smithi183 ceph-mon[37326]: pgmap v96: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-03T03:14:15.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:15 smithi183 ceph-mon[37326]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-03T03:14:17.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:17 smithi117 ceph-mon[31649]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-03T03:14:17.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:17 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/2221863556' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ca08d2ac-b1d6-4a49-9ae8-e3ec146899a7"}]: dispatch 2024-06-03T03:14:17.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:17 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/2221863556' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ca08d2ac-b1d6-4a49-9ae8-e3ec146899a7"}]': finished 2024-06-03T03:14:17.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:17 smithi117 ceph-mon[31649]: osdmap e22: 4 total, 3 up, 4 in 2024-06-03T03:14:17.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:17 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-03T03:14:18.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:17 smithi151 ceph-mon[37043]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-03T03:14:18.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:17 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/2221863556' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ca08d2ac-b1d6-4a49-9ae8-e3ec146899a7"}]: dispatch 2024-06-03T03:14:18.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:17 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/2221863556' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ca08d2ac-b1d6-4a49-9ae8-e3ec146899a7"}]': finished 2024-06-03T03:14:18.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:17 smithi151 ceph-mon[37043]: osdmap e22: 4 total, 3 up, 4 in 2024-06-03T03:14:18.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:17 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-03T03:14:18.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:17 smithi183 ceph-mon[37326]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-03T03:14:18.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:17 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/2221863556' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ca08d2ac-b1d6-4a49-9ae8-e3ec146899a7"}]: dispatch 2024-06-03T03:14:18.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:17 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/2221863556' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ca08d2ac-b1d6-4a49-9ae8-e3ec146899a7"}]': finished 2024-06-03T03:14:18.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:17 smithi183 ceph-mon[37326]: osdmap e22: 4 total, 3 up, 4 in 2024-06-03T03:14:18.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:17 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-03T03:14:19.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:18 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/1415596676' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T03:14:19.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:18 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/1415596676' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T03:14:19.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:18 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/1415596676' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T03:14:20.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:19 smithi151 ceph-mon[37043]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-03T03:14:20.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:19 smithi117 ceph-mon[31649]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-03T03:14:20.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:19 smithi183 ceph-mon[37326]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-03T03:14:22.135 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:21 smithi117 ceph-mon[31649]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-03T03:14:22.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:21 smithi151 ceph-mon[37043]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-03T03:14:22.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:21 smithi183 ceph-mon[37326]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-03T03:14:24.039 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:23 smithi117 ceph-mon[31649]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-03T03:14:24.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:23 smithi151 ceph-mon[37043]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-03T03:14:24.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:23 smithi183 ceph-mon[37326]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-03T03:14:25.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:25 smithi117 ceph-mon[31649]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-03T03:14:26.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:25 smithi151 ceph-mon[37043]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-03T03:14:26.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:25 smithi183 ceph-mon[37326]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-03T03:14:27.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:26 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-06-03T03:14:27.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:26 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:14:27.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:26 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-06-03T03:14:27.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:26 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:14:27.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:26 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-06-03T03:14:27.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:26 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:14:28.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:27 smithi151 ceph-mon[37043]: Deploying daemon osd.3 on smithi117 2024-06-03T03:14:28.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:27 smithi151 ceph-mon[37043]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-03T03:14:28.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:27 smithi117 ceph-mon[31649]: Deploying daemon osd.3 on smithi117 2024-06-03T03:14:28.199 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:27 smithi117 ceph-mon[31649]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-03T03:14:28.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:27 smithi183 ceph-mon[37326]: Deploying daemon osd.3 on smithi117 2024-06-03T03:14:28.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:27 smithi183 ceph-mon[37326]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-03T03:14:30.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:29 smithi151 ceph-mon[37043]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-03T03:14:30.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:29 smithi117 ceph-mon[31649]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-03T03:14:30.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:29 smithi183 ceph-mon[37326]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-03T03:14:31.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:31 smithi117 ceph-mon[31649]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-03T03:14:32.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:31 smithi151 ceph-mon[37043]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-03T03:14:32.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:31 smithi183 ceph-mon[37326]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-03T03:14:33.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:33 smithi117 ceph-mon[31649]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-03T03:14:34.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:33 smithi151 ceph-mon[37043]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-03T03:14:34.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:33 smithi183 ceph-mon[37326]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-03T03:14:36.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:35 smithi117 ceph-mon[31649]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-03T03:14:36.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:35 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:14:36.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:35 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:14:36.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:35 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:14:36.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:35 smithi183 ceph-mon[37326]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-03T03:14:36.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:35 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:14:36.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:35 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:14:36.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:35 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:14:36.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:35 smithi151 ceph-mon[37043]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-03T03:14:36.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:35 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:14:36.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:35 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:14:36.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:35 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:14:38.165 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:37 smithi117 ceph-mon[31649]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-03T03:14:38.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:37 smithi183 ceph-mon[37326]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-03T03:14:38.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:37 smithi151 ceph-mon[37043]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-03T03:14:39.890 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:39 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:14:39.890 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:39 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:14:39.891 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:39 smithi117 ceph-mon[31649]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-03T03:14:39.891 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:39 smithi117 ceph-mon[31649]: from='osd.3 [v2:172.21.15.117:6810/589012315,v1:172.21.15.117:6811/589012315]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-03T03:14:39.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:39 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:14:39.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:39 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:14:39.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:39 smithi183 ceph-mon[37326]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-03T03:14:39.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:39 smithi183 ceph-mon[37326]: from='osd.3 [v2:172.21.15.117:6810/589012315,v1:172.21.15.117:6811/589012315]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-03T03:14:39.997 INFO:teuthology.orchestra.run.smithi117.stdout:Created osd(s) 3 on host 'smithi117' 2024-06-03T03:14:39.997 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:14:39.996+0000 7f46d9600700 1 -- 172.21.15.117:0/310836624 <== mgr.14152 v2:172.21.15.117:6800/1398160887 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f46dc0628d0 con 0x7f46c40a2ca0 2024-06-03T03:14:40.000 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:14:39.999+0000 7f46e2400700 1 -- 172.21.15.117:0/310836624 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f46c40a2ca0 msgr2=0x7f46c40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:14:40.000 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:14:39.999+0000 7f46e2400700 1 --2- 172.21.15.117:0/310836624 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f46c40a2ca0 0x7f46c40a5160 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f46cc000c00 tx=0x7f46cc005fb0 comp rx=0 tx=0).stop 2024-06-03T03:14:40.000 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:14:39.999+0000 7f46e2400700 1 -- 172.21.15.117:0/310836624 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f46dc104630 msgr2=0x7f46dc19a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:14:40.000 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:14:39.999+0000 7f46e2400700 1 --2- 172.21.15.117:0/310836624 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f46dc104630 0x7f46dc19a840 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f46d000d980 tx=0x7f46d000dc90 comp rx=0 tx=0).stop 2024-06-03T03:14:40.000 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:14:39.999+0000 7f46e2400700 1 -- 172.21.15.117:0/310836624 shutdown_connections 2024-06-03T03:14:40.000 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:14:39.999+0000 7f46e2400700 1 --2- 172.21.15.117:0/310836624 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f46c40a2ca0 0x7f46c40a5160 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:14:40.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:14:39.999+0000 7f46e2400700 1 --2- 172.21.15.117:0/310836624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46dc1061d0 0x7f46dc1a1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:14:40.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:14:39.999+0000 7f46e2400700 1 --2- 172.21.15.117:0/310836624 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f46dc104630 0x7f46dc19a840 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:14:40.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:14:39.999+0000 7f46e2400700 1 --2- 172.21.15.117:0/310836624 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f46dc105830 0x7f46dc19ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:14:40.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:14:39.999+0000 7f46e2400700 1 -- 172.21.15.117:0/310836624 >> 172.21.15.117:0/310836624 conn(0x7f46dc0ffbd0 msgr2=0x7f46dc108b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:14:40.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:14:39.999+0000 7f46e2400700 1 -- 172.21.15.117:0/310836624 shutdown_connections 2024-06-03T03:14:40.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:14:39.999+0000 7f46e2400700 1 -- 172.21.15.117:0/310836624 wait complete. 2024-06-03T03:14:40.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:39 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:14:40.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:39 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:14:40.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:39 smithi151 ceph-mon[37043]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-03T03:14:40.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:39 smithi151 ceph-mon[37043]: from='osd.3 [v2:172.21.15.117:6810/589012315,v1:172.21.15.117:6811/589012315]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-03T03:14:40.864 DEBUG:teuthology.orchestra.run.smithi117:osd.3> sudo journalctl -f -n 0 -u ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@osd.3.service 2024-06-03T03:14:40.869 INFO:tasks.cephadm:Deploying osd.4 on smithi151 with /dev/vg_nvme/lv_3... 2024-06-03T03:14:40.869 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-06-03T03:14:40.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:40 smithi117 ceph-mon[31649]: from='osd.3 [v2:172.21.15.117:6810/589012315,v1:172.21.15.117:6811/589012315]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-06-03T03:14:40.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:40 smithi117 ceph-mon[31649]: osdmap e23: 4 total, 3 up, 4 in 2024-06-03T03:14:40.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:40 smithi117 ceph-mon[31649]: from='osd.3 [v2:172.21.15.117:6810/589012315,v1:172.21.15.117:6811/589012315]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-06-03T03:14:40.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:40 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-03T03:14:40.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:40 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:14:40.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:40 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:14:40.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:40 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:14:40.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:40 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:14:40.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:40 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:14:40.948 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:40 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:14:40.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:40 smithi183 ceph-mon[37326]: from='osd.3 [v2:172.21.15.117:6810/589012315,v1:172.21.15.117:6811/589012315]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-06-03T03:14:40.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:40 smithi183 ceph-mon[37326]: osdmap e23: 4 total, 3 up, 4 in 2024-06-03T03:14:40.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:40 smithi183 ceph-mon[37326]: from='osd.3 [v2:172.21.15.117:6810/589012315,v1:172.21.15.117:6811/589012315]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-06-03T03:14:40.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:40 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-03T03:14:40.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:40 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:14:40.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:40 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:14:40.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:40 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:14:40.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:40 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:14:40.983 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:40 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:14:40.983 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:40 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:14:41.113 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.b/config 2024-06-03T03:14:41.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:40 smithi151 ceph-mon[37043]: from='osd.3 [v2:172.21.15.117:6810/589012315,v1:172.21.15.117:6811/589012315]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-06-03T03:14:41.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:40 smithi151 ceph-mon[37043]: osdmap e23: 4 total, 3 up, 4 in 2024-06-03T03:14:41.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:40 smithi151 ceph-mon[37043]: from='osd.3 [v2:172.21.15.117:6810/589012315,v1:172.21.15.117:6811/589012315]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-06-03T03:14:41.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:40 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-03T03:14:41.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:40 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:14:41.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:40 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:14:41.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:40 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:14:41.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:40 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:14:41.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:40 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:14:41.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:40 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:14:41.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:41 smithi151 ceph-mon[37043]: from='osd.3 [v2:172.21.15.117:6810/589012315,v1:172.21.15.117:6811/589012315]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-06-03T03:14:41.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:41 smithi151 ceph-mon[37043]: osdmap e24: 4 total, 3 up, 4 in 2024-06-03T03:14:41.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:41 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-03T03:14:41.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:41 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-03T03:14:41.908 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:41 smithi151 ceph-mon[37043]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-03T03:14:41.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:41 smithi117 ceph-mon[31649]: from='osd.3 [v2:172.21.15.117:6810/589012315,v1:172.21.15.117:6811/589012315]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-06-03T03:14:41.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:41 smithi117 ceph-mon[31649]: osdmap e24: 4 total, 3 up, 4 in 2024-06-03T03:14:41.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:41 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-03T03:14:41.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:41 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-03T03:14:41.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:41 smithi117 ceph-mon[31649]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-03T03:14:41.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:41 smithi183 ceph-mon[37326]: from='osd.3 [v2:172.21.15.117:6810/589012315,v1:172.21.15.117:6811/589012315]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-06-03T03:14:41.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:41 smithi183 ceph-mon[37326]: osdmap e24: 4 total, 3 up, 4 in 2024-06-03T03:14:41.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:41 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-03T03:14:41.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:41 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-03T03:14:41.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:41 smithi183 ceph-mon[37326]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-03T03:14:42.778 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:42 smithi117 ceph-mon[31649]: purged_snaps scrub starts 2024-06-03T03:14:42.778 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:42 smithi117 ceph-mon[31649]: purged_snaps scrub ok 2024-06-03T03:14:42.779 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:42 smithi117 ceph-mon[31649]: osd.3 [v2:172.21.15.117:6810/589012315,v1:172.21.15.117:6811/589012315] boot 2024-06-03T03:14:42.779 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:42 smithi117 ceph-mon[31649]: osdmap e25: 4 total, 4 up, 4 in 2024-06-03T03:14:42.779 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:42 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-03T03:14:42.779 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:42 smithi117 ceph-mon[31649]: osdmap e26: 4 total, 4 up, 4 in 2024-06-03T03:14:42.892 INFO:teuthology.orchestra.run.smithi151.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-06-03T03:14:42.892 INFO:teuthology.orchestra.run.smithi151.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-06-03T03:14:42.892 INFO:teuthology.orchestra.run.smithi151.stderr: stderr: 10+0 records in 2024-06-03T03:14:42.892 INFO:teuthology.orchestra.run.smithi151.stderr:10+0 records out 2024-06-03T03:14:42.892 INFO:teuthology.orchestra.run.smithi151.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115092 s, 911 MB/s 2024-06-03T03:14:42.892 INFO:teuthology.orchestra.run.smithi151.stderr:--> Zapping successful for: 2024-06-03T03:14:42.916 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:42 smithi151 ceph-mon[37043]: purged_snaps scrub starts 2024-06-03T03:14:42.917 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:42 smithi151 ceph-mon[37043]: purged_snaps scrub ok 2024-06-03T03:14:42.917 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:42 smithi151 ceph-mon[37043]: osd.3 [v2:172.21.15.117:6810/589012315,v1:172.21.15.117:6811/589012315] boot 2024-06-03T03:14:42.917 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:42 smithi151 ceph-mon[37043]: osdmap e25: 4 total, 4 up, 4 in 2024-06-03T03:14:42.917 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:42 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-03T03:14:42.917 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:42 smithi151 ceph-mon[37043]: osdmap e26: 4 total, 4 up, 4 in 2024-06-03T03:14:42.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:42 smithi183 ceph-mon[37326]: purged_snaps scrub starts 2024-06-03T03:14:42.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:42 smithi183 ceph-mon[37326]: purged_snaps scrub ok 2024-06-03T03:14:42.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:42 smithi183 ceph-mon[37326]: osd.3 [v2:172.21.15.117:6810/589012315,v1:172.21.15.117:6811/589012315] boot 2024-06-03T03:14:42.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:42 smithi183 ceph-mon[37326]: osdmap e25: 4 total, 4 up, 4 in 2024-06-03T03:14:42.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:42 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-03T03:14:42.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:42 smithi183 ceph-mon[37326]: osdmap e26: 4 total, 4 up, 4 in 2024-06-03T03:14:43.477 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi151:vg_nvme/lv_3 2024-06-03T03:14:43.713 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.b/config 2024-06-03T03:14:43.834 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:43 smithi151 ceph-mon[37043]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-06-03T03:14:43.835 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:43 smithi151 ceph-mon[37043]: osdmap e27: 4 total, 4 up, 4 in 2024-06-03T03:14:43.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:43 smithi183 ceph-mon[37326]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-06-03T03:14:43.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:43 smithi183 ceph-mon[37326]: osdmap e27: 4 total, 4 up, 4 in 2024-06-03T03:14:44.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:43 smithi117 ceph-mon[31649]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-06-03T03:14:44.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:43 smithi117 ceph-mon[31649]: osdmap e27: 4 total, 4 up, 4 in 2024-06-03T03:14:45.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:14:45.174+0000 7f82cb97c700 1 -- 172.21.15.151:0/3794744765 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f82c4104090 msgr2=0x7f82c410a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:14:45.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:14:45.174+0000 7f82cb97c700 1 --2- 172.21.15.151:0/3794744765 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f82c4104090 0x7f82c410a940 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f82bc00b0d0 tx=0x7f82bc00b3e0 comp rx=0 tx=0).stop 2024-06-03T03:14:45.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:14:45.174+0000 7f82cb97c700 1 -- 172.21.15.151:0/3794744765 shutdown_connections 2024-06-03T03:14:45.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:14:45.174+0000 7f82cb97c700 1 --2- 172.21.15.151:0/3794744765 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f82c4104090 0x7f82c410a940 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:14:45.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:14:45.174+0000 7f82cb97c700 1 --2- 172.21.15.151:0/3794744765 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82c41024f0 0x7f82c4102910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:14:45.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:14:45.174+0000 7f82cb97c700 1 --2- 172.21.15.151:0/3794744765 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82c41036f0 0x7f82c4103b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:14:45.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:14:45.174+0000 7f82cb97c700 1 -- 172.21.15.151:0/3794744765 >> 172.21.15.151:0/3794744765 conn(0x7f82c40fda90 msgr2=0x7f82c40ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:14:45.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:14:45.174+0000 7f82cb97c700 1 -- 172.21.15.151:0/3794744765 shutdown_connections 2024-06-03T03:14:45.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:14:45.174+0000 7f82cb97c700 1 -- 172.21.15.151:0/3794744765 wait complete. 2024-06-03T03:14:45.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:14:45.175+0000 7f82cb97c700 1 Processor -- start 2024-06-03T03:14:45.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:14:45.175+0000 7f82cb97c700 1 -- start start 2024-06-03T03:14:45.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:14:45.175+0000 7f82cb97c700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f82c41024f0 0x7f82c41986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:14:45.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:14:45.175+0000 7f82cb97c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82c41036f0 0x7f82c4198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:14:45.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:14:45.175+0000 7f82cb97c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82c4104090 0x7f82c419fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:14:45.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:14:45.175+0000 7f82cb97c700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f82c410e150 con 0x7f82c4104090 2024-06-03T03:14:45.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:14:45.175+0000 7f82cb97c700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f82c410dfd0 con 0x7f82c41024f0 2024-06-03T03:14:45.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:14:45.175+0000 7f82cb97c700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f82c410e2d0 con 0x7f82c41036f0 2024-06-03T03:14:45.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:14:45.175+0000 7f82c9718700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f82c41024f0 0x7f82c41986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:14:45.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:14:45.175+0000 7f82c9718700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f82c41024f0 0x7f82c41986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:38902/0 (socket says 172.21.15.151:38902) 2024-06-03T03:14:45.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:14:45.175+0000 7f82c9718700 1 -- 172.21.15.151:0/1537717464 learned_addr learned my addr 172.21.15.151:0/1537717464 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-06-03T03:14:45.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:14:45.175+0000 7f82c8f17700 1 --2- 172.21.15.151:0/1537717464 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82c41036f0 0x7f82c4198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:14:45.193 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:14:45.175+0000 7f82c9718700 1 -- 172.21.15.151:0/1537717464 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82c41036f0 msgr2=0x7f82c4198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:14:45.193 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:14:45.175+0000 7f82c9718700 1 --2- 172.21.15.151:0/1537717464 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82c41036f0 0x7f82c4198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:14:45.193 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:14:45.175+0000 7f82c9718700 1 -- 172.21.15.151:0/1537717464 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82c4104090 msgr2=0x7f82c419fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:14:45.193 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:14:45.175+0000 7f82c9718700 1 --2- 172.21.15.151:0/1537717464 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82c4104090 0x7f82c419fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:14:45.193 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:14:45.175+0000 7f82c9718700 1 -- 172.21.15.151:0/1537717464 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82c41a0340 con 0x7f82c41024f0 2024-06-03T03:14:45.193 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:14:45.175+0000 7f82c9718700 1 --2- 172.21.15.151:0/1537717464 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f82c41024f0 0x7f82c41986e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f82b400d980 tx=0x7f82b400dd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:14:45.193 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:14:45.175+0000 7f82ba7fc700 1 -- 172.21.15.151:0/1537717464 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82b4017070 con 0x7f82c41024f0 2024-06-03T03:14:45.193 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:14:45.175+0000 7f82ba7fc700 1 -- 172.21.15.151:0/1537717464 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82b4009b40 con 0x7f82c41024f0 2024-06-03T03:14:45.193 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:14:45.175+0000 7f82cb97c700 1 -- 172.21.15.151:0/1537717464 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82c41a0630 con 0x7f82c41024f0 2024-06-03T03:14:45.193 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:14:45.175+0000 7f82ba7fc700 1 -- 172.21.15.151:0/1537717464 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82b400f620 con 0x7f82c41024f0 2024-06-03T03:14:45.193 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:14:45.176+0000 7f82cb97c700 1 -- 172.21.15.151:0/1537717464 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82c41a0ba0 con 0x7f82c41024f0 2024-06-03T03:14:45.193 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:14:45.176+0000 7f82cb97c700 1 -- 172.21.15.151:0/1537717464 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82c4069180 con 0x7f82c41024f0 2024-06-03T03:14:45.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:14:45.177+0000 7f82ba7fc700 1 -- 172.21.15.151:0/1537717464 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f82b4010460 con 0x7f82c41024f0 2024-06-03T03:14:45.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:14:45.178+0000 7f82ba7fc700 1 --2- 172.21.15.151:0/1537717464 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f82b00a2c20 0x7f82b00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:14:45.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:14:45.178+0000 7f82ba7fc700 1 -- 172.21.15.151:0/1537717464 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 3669+0+0 (secure 0 0 0) 0x7f82b40cdac0 con 0x7f82c41024f0 2024-06-03T03:14:45.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:14:45.179+0000 7f82c8f17700 1 --2- 172.21.15.151:0/1537717464 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f82b00a2c20 0x7f82b00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:14:45.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:14:45.180+0000 7f82c8f17700 1 --2- 172.21.15.151:0/1537717464 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f82b00a2c20 0x7f82b00a50e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f82c0006fd0 tx=0x7f82c0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:14:45.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:14:45.183+0000 7f82ba7fc700 1 -- 172.21.15.151:0/1537717464 <== mon.2 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f82b409bc40 con 0x7f82c41024f0 2024-06-03T03:14:45.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:14:45.387+0000 7f82cb97c700 1 -- 172.21.15.151:0/1537717464 --> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f82c40628d0 con 0x7f82b00a2c20 2024-06-03T03:14:45.656 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:45 smithi151 ceph-mon[37043]: osdmap e28: 4 total, 4 up, 4 in 2024-06-03T03:14:45.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:45 smithi151 ceph-mon[37043]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-06-03T03:14:45.696 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:45 smithi117 ceph-mon[31649]: osdmap e28: 4 total, 4 up, 4 in 2024-06-03T03:14:45.697 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:45 smithi117 ceph-mon[31649]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-06-03T03:14:45.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:45 smithi183 ceph-mon[37326]: osdmap e28: 4 total, 4 up, 4 in 2024-06-03T03:14:45.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:45 smithi183 ceph-mon[37326]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-06-03T03:14:46.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:46 smithi151 ceph-mon[37043]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:14:46.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:46 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T03:14:46.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:46 smithi151 ceph-mon[37043]: osdmap e29: 4 total, 4 up, 4 in 2024-06-03T03:14:46.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:46 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T03:14:46.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:46 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:14:46.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:46 smithi151 ceph-mon[37043]: Detected new or changed devices on smithi117 2024-06-03T03:14:46.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:46 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:14:46.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:46 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:14:46.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:46 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:14:46.658 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:46 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:14:46.658 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:46 smithi151 ceph-mon[37043]: Adjusting osd_memory_target on smithi117 to 8544M 2024-06-03T03:14:46.658 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:46 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:14:46.658 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:46 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:14:46.658 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:46 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:14:46.658 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:46 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:14:46.697 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:46 smithi117 ceph-mon[31649]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:14:46.697 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:46 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T03:14:46.698 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:46 smithi117 ceph-mon[31649]: osdmap e29: 4 total, 4 up, 4 in 2024-06-03T03:14:46.698 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:46 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T03:14:46.698 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:46 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:14:46.698 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:46 smithi117 ceph-mon[31649]: Detected new or changed devices on smithi117 2024-06-03T03:14:46.698 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:46 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:14:46.698 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:46 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:14:46.698 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:46 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:14:46.698 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:46 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:14:46.698 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:46 smithi117 ceph-mon[31649]: Adjusting osd_memory_target on smithi117 to 8544M 2024-06-03T03:14:46.698 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:46 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:14:46.699 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:46 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:14:46.699 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:46 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:14:46.699 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:46 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:14:46.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:46 smithi183 ceph-mon[37326]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:14:46.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:46 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T03:14:46.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:46 smithi183 ceph-mon[37326]: osdmap e29: 4 total, 4 up, 4 in 2024-06-03T03:14:46.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:46 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T03:14:46.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:46 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:14:46.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:46 smithi183 ceph-mon[37326]: Detected new or changed devices on smithi117 2024-06-03T03:14:46.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:46 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:14:46.733 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:46 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:14:46.733 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:46 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:14:46.733 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:46 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:14:46.733 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:46 smithi183 ceph-mon[37326]: Adjusting osd_memory_target on smithi117 to 8544M 2024-06-03T03:14:46.733 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:46 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:14:46.733 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:46 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:14:46.733 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:46 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:14:46.733 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:46 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:14:47.656 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:47 smithi151 ceph-mon[37043]: pgmap v120: 1 pgs: 1 peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-06-03T03:14:47.696 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:47 smithi117 ceph-mon[31649]: pgmap v120: 1 pgs: 1 peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-06-03T03:14:47.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:47 smithi183 ceph-mon[37326]: pgmap v120: 1 pgs: 1 peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-06-03T03:14:48.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:48 smithi117 ceph-mon[31649]: from='client.? 172.21.15.151:0/4277708852' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "810d6c7a-023f-4930-82af-bc2bbfd647da"}]: dispatch 2024-06-03T03:14:48.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:48 smithi117 ceph-mon[31649]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "810d6c7a-023f-4930-82af-bc2bbfd647da"}]: dispatch 2024-06-03T03:14:48.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:48 smithi117 ceph-mon[31649]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "810d6c7a-023f-4930-82af-bc2bbfd647da"}]': finished 2024-06-03T03:14:48.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:48 smithi117 ceph-mon[31649]: osdmap e30: 5 total, 4 up, 5 in 2024-06-03T03:14:48.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:48 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-03T03:14:48.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:48 smithi151 ceph-mon[37043]: from='client.? 172.21.15.151:0/4277708852' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "810d6c7a-023f-4930-82af-bc2bbfd647da"}]: dispatch 2024-06-03T03:14:48.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:48 smithi151 ceph-mon[37043]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "810d6c7a-023f-4930-82af-bc2bbfd647da"}]: dispatch 2024-06-03T03:14:48.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:48 smithi151 ceph-mon[37043]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "810d6c7a-023f-4930-82af-bc2bbfd647da"}]': finished 2024-06-03T03:14:48.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:48 smithi151 ceph-mon[37043]: osdmap e30: 5 total, 4 up, 5 in 2024-06-03T03:14:48.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:48 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-03T03:14:48.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:48 smithi183 ceph-mon[37326]: from='client.? 172.21.15.151:0/4277708852' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "810d6c7a-023f-4930-82af-bc2bbfd647da"}]: dispatch 2024-06-03T03:14:48.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:48 smithi183 ceph-mon[37326]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "810d6c7a-023f-4930-82af-bc2bbfd647da"}]: dispatch 2024-06-03T03:14:48.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:48 smithi183 ceph-mon[37326]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "810d6c7a-023f-4930-82af-bc2bbfd647da"}]': finished 2024-06-03T03:14:48.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:48 smithi183 ceph-mon[37326]: osdmap e30: 5 total, 4 up, 5 in 2024-06-03T03:14:48.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:48 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-03T03:14:49.696 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:49 smithi117 ceph-mon[31649]: from='client.? 172.21.15.151:0/2979432849' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T03:14:49.697 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:49 smithi117 ceph-mon[31649]: pgmap v122: 1 pgs: 1 peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-06-03T03:14:49.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:49 smithi183 ceph-mon[37326]: from='client.? 172.21.15.151:0/2979432849' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T03:14:49.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:49 smithi183 ceph-mon[37326]: pgmap v122: 1 pgs: 1 peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-06-03T03:14:49.906 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:49 smithi151 ceph-mon[37043]: from='client.? 172.21.15.151:0/2979432849' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T03:14:49.906 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:49 smithi151 ceph-mon[37043]: pgmap v122: 1 pgs: 1 peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-06-03T03:14:52.049 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:51 smithi151 ceph-mon[37043]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-06-03T03:14:52.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:51 smithi117 ceph-mon[31649]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-06-03T03:14:52.231 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:51 smithi183 ceph-mon[37326]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-06-03T03:14:54.115 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:53 smithi151 ceph-mon[37043]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-03T03:14:54.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:53 smithi117 ceph-mon[31649]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-03T03:14:54.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:53 smithi183 ceph-mon[37326]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-03T03:14:56.129 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:55 smithi151 ceph-mon[37043]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-06-03T03:14:56.130 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:55 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-06-03T03:14:56.130 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:55 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:14:56.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:55 smithi117 ceph-mon[31649]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-06-03T03:14:56.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:55 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-06-03T03:14:56.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:55 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:14:56.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:55 smithi183 ceph-mon[37326]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-06-03T03:14:56.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:55 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-06-03T03:14:56.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:55 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:14:57.098 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:56 smithi117 ceph-mon[31649]: Deploying daemon osd.4 on smithi151 2024-06-03T03:14:57.112 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:56 smithi151 ceph-mon[37043]: Deploying daemon osd.4 on smithi151 2024-06-03T03:14:57.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:56 smithi183 ceph-mon[37326]: Deploying daemon osd.4 on smithi151 2024-06-03T03:14:58.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:57 smithi151 ceph-mon[37043]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-06-03T03:14:58.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:57 smithi117 ceph-mon[31649]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-06-03T03:14:58.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:57 smithi183 ceph-mon[37326]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-06-03T03:15:00.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:14:59 smithi151 ceph-mon[37043]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-06-03T03:15:00.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:14:59 smithi117 ceph-mon[31649]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-06-03T03:15:00.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:14:59 smithi183 ceph-mon[37326]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-06-03T03:15:02.059 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:01 smithi151 ceph-mon[37043]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-06-03T03:15:02.060 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:01 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:15:02.060 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:01 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:15:02.060 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:01 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:15:02.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:01 smithi117 ceph-mon[31649]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-06-03T03:15:02.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:01 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:15:02.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:01 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:15:02.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:01 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:15:02.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:01 smithi183 ceph-mon[37326]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-06-03T03:15:02.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:01 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:15:02.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:01 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:15:02.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:01 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:15:04.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:03 smithi151 ceph-mon[37043]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-06-03T03:15:04.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:03 smithi117 ceph-mon[31649]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-06-03T03:15:04.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:03 smithi183 ceph-mon[37326]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-06-03T03:15:04.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:15:04.272+0000 7f82ba7fc700 1 -- 172.21.15.151:0/1537717464 <== mgr.14152 v2:172.21.15.117:6800/1398160887 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f82c40628d0 con 0x7f82b00a2c20 2024-06-03T03:15:04.275 INFO:teuthology.orchestra.run.smithi151.stdout:Created osd(s) 4 on host 'smithi151' 2024-06-03T03:15:04.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:15:04.275+0000 7f82cb97c700 1 -- 172.21.15.151:0/1537717464 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f82b00a2c20 msgr2=0x7f82b00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:15:04.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:15:04.275+0000 7f82cb97c700 1 --2- 172.21.15.151:0/1537717464 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f82b00a2c20 0x7f82b00a50e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f82c0006fd0 tx=0x7f82c0008040 comp rx=0 tx=0).stop 2024-06-03T03:15:04.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:15:04.275+0000 7f82cb97c700 1 -- 172.21.15.151:0/1537717464 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f82c41024f0 msgr2=0x7f82c41986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:15:04.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:15:04.275+0000 7f82cb97c700 1 --2- 172.21.15.151:0/1537717464 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f82c41024f0 0x7f82c41986e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f82b400d980 tx=0x7f82b400dd40 comp rx=0 tx=0).stop 2024-06-03T03:15:04.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:15:04.275+0000 7f82cb97c700 1 -- 172.21.15.151:0/1537717464 shutdown_connections 2024-06-03T03:15:04.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:15:04.275+0000 7f82cb97c700 1 --2- 172.21.15.151:0/1537717464 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f82b00a2c20 0x7f82b00a50e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:15:04.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:15:04.275+0000 7f82cb97c700 1 --2- 172.21.15.151:0/1537717464 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82c4104090 0x7f82c419fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:15:04.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:15:04.275+0000 7f82cb97c700 1 --2- 172.21.15.151:0/1537717464 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f82c41024f0 0x7f82c41986e0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:15:04.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:15:04.275+0000 7f82cb97c700 1 --2- 172.21.15.151:0/1537717464 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82c41036f0 0x7f82c4198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:15:04.278 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:15:04.275+0000 7f82cb97c700 1 -- 172.21.15.151:0/1537717464 >> 172.21.15.151:0/1537717464 conn(0x7f82c40fda90 msgr2=0x7f82c41069f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:15:04.278 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:15:04.275+0000 7f82cb97c700 1 -- 172.21.15.151:0/1537717464 shutdown_connections 2024-06-03T03:15:04.278 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:15:04.275+0000 7f82cb97c700 1 -- 172.21.15.151:0/1537717464 wait complete. 2024-06-03T03:15:05.075 DEBUG:teuthology.orchestra.run.smithi151:osd.4> sudo journalctl -f -n 0 -u ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@osd.4.service 2024-06-03T03:15:05.077 INFO:tasks.cephadm:Deploying osd.5 on smithi183 with /dev/vg_nvme/lv_3... 2024-06-03T03:15:05.078 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-06-03T03:15:05.322 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.c/config 2024-06-03T03:15:05.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:05 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:15:05.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:05 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:15:05.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:05 smithi151 ceph-mon[37043]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-06-03T03:15:05.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:05 smithi151 ceph-mon[37043]: from='osd.4 [v2:172.21.15.151:6808/1239048647,v1:172.21.15.151:6809/1239048647]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-03T03:15:05.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:05 smithi151 ceph-mon[37043]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-03T03:15:05.429 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:05 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:15:05.430 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:05 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:15:05.430 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:05 smithi183 ceph-mon[37326]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-06-03T03:15:05.430 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:05 smithi183 ceph-mon[37326]: from='osd.4 [v2:172.21.15.151:6808/1239048647,v1:172.21.15.151:6809/1239048647]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-03T03:15:05.430 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:05 smithi183 ceph-mon[37326]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-03T03:15:05.697 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:05 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:15:05.697 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:05 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:15:05.697 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:05 smithi117 ceph-mon[31649]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-06-03T03:15:05.697 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:05 smithi117 ceph-mon[31649]: from='osd.4 [v2:172.21.15.151:6808/1239048647,v1:172.21.15.151:6809/1239048647]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-03T03:15:05.697 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:05 smithi117 ceph-mon[31649]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-03T03:15:06.477 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:06 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:15:06.477 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:06 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:15:06.477 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:06 smithi183 ceph-mon[37326]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-06-03T03:15:06.477 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:06 smithi183 ceph-mon[37326]: from='osd.4 [v2:172.21.15.151:6808/1239048647,v1:172.21.15.151:6809/1239048647]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-06-03T03:15:06.478 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:06 smithi183 ceph-mon[37326]: osdmap e31: 5 total, 4 up, 5 in 2024-06-03T03:15:06.478 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:06 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-03T03:15:06.478 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:06 smithi183 ceph-mon[37326]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-06-03T03:15:06.560 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:06 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:15:06.561 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:06 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:15:06.561 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:06 smithi151 ceph-mon[37043]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-06-03T03:15:06.561 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:06 smithi151 ceph-mon[37043]: from='osd.4 [v2:172.21.15.151:6808/1239048647,v1:172.21.15.151:6809/1239048647]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-06-03T03:15:06.561 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:06 smithi151 ceph-mon[37043]: osdmap e31: 5 total, 4 up, 5 in 2024-06-03T03:15:06.561 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:06 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-03T03:15:06.561 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:06 smithi151 ceph-mon[37043]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-06-03T03:15:06.562 INFO:journalctl@ceph.osd.4.smithi151.stdout:Jun 03 03:15:06 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-4[48065]: 2024-06-03T03:15:06.252+0000 7fc8aad79700 -1 osd.4 0 waiting for initial osdmap 2024-06-03T03:15:06.562 INFO:journalctl@ceph.osd.4.smithi151.stdout:Jun 03 03:15:06 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-4[48065]: 2024-06-03T03:15:06.259+0000 7fc8a6b6c700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-03T03:15:06.697 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:06 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:15:06.697 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:06 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:15:06.697 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:06 smithi117 ceph-mon[31649]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-06-03T03:15:06.697 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:06 smithi117 ceph-mon[31649]: from='osd.4 [v2:172.21.15.151:6808/1239048647,v1:172.21.15.151:6809/1239048647]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-06-03T03:15:06.697 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:06 smithi117 ceph-mon[31649]: osdmap e31: 5 total, 4 up, 5 in 2024-06-03T03:15:06.697 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:06 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-03T03:15:06.697 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:06 smithi117 ceph-mon[31649]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-06-03T03:15:07.423 INFO:teuthology.orchestra.run.smithi183.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-06-03T03:15:07.423 INFO:teuthology.orchestra.run.smithi183.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-06-03T03:15:07.423 INFO:teuthology.orchestra.run.smithi183.stderr: stderr: 10+0 records in 2024-06-03T03:15:07.423 INFO:teuthology.orchestra.run.smithi183.stderr:10+0 records out 2024-06-03T03:15:07.423 INFO:teuthology.orchestra.run.smithi183.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0135479 s, 774 MB/s 2024-06-03T03:15:07.423 INFO:teuthology.orchestra.run.smithi183.stderr:--> Zapping successful for: 2024-06-03T03:15:07.522 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:07 smithi151 ceph-mon[37043]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-06-03T03:15:07.522 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:07 smithi151 ceph-mon[37043]: osdmap e32: 5 total, 4 up, 5 in 2024-06-03T03:15:07.522 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:07 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-03T03:15:07.522 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:07 smithi151 ceph-mon[37043]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-06-03T03:15:07.697 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:07 smithi117 ceph-mon[31649]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-06-03T03:15:07.697 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:07 smithi117 ceph-mon[31649]: osdmap e32: 5 total, 4 up, 5 in 2024-06-03T03:15:07.697 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:07 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-03T03:15:07.697 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:07 smithi117 ceph-mon[31649]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-06-03T03:15:07.718 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:07 smithi183 ceph-mon[37326]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-06-03T03:15:07.718 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:07 smithi183 ceph-mon[37326]: osdmap e32: 5 total, 4 up, 5 in 2024-06-03T03:15:07.718 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:07 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-03T03:15:07.718 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:07 smithi183 ceph-mon[37326]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-06-03T03:15:08.133 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi183:vg_nvme/lv_3 2024-06-03T03:15:08.368 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.c/config 2024-06-03T03:15:08.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:08 smithi151 ceph-mon[37043]: purged_snaps scrub starts 2024-06-03T03:15:08.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:08 smithi151 ceph-mon[37043]: purged_snaps scrub ok 2024-06-03T03:15:08.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:08 smithi151 ceph-mon[37043]: osd.4 [v2:172.21.15.151:6808/1239048647,v1:172.21.15.151:6809/1239048647] boot 2024-06-03T03:15:08.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:08 smithi151 ceph-mon[37043]: osdmap e33: 5 total, 5 up, 5 in 2024-06-03T03:15:08.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:08 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-03T03:15:08.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:08 smithi117 ceph-mon[31649]: purged_snaps scrub starts 2024-06-03T03:15:08.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:08 smithi117 ceph-mon[31649]: purged_snaps scrub ok 2024-06-03T03:15:08.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:08 smithi117 ceph-mon[31649]: osd.4 [v2:172.21.15.151:6808/1239048647,v1:172.21.15.151:6809/1239048647] boot 2024-06-03T03:15:08.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:08 smithi117 ceph-mon[31649]: osdmap e33: 5 total, 5 up, 5 in 2024-06-03T03:15:08.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:08 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-03T03:15:08.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:08 smithi183 ceph-mon[37326]: purged_snaps scrub starts 2024-06-03T03:15:08.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:08 smithi183 ceph-mon[37326]: purged_snaps scrub ok 2024-06-03T03:15:08.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:08 smithi183 ceph-mon[37326]: osd.4 [v2:172.21.15.151:6808/1239048647,v1:172.21.15.151:6809/1239048647] boot 2024-06-03T03:15:08.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:08 smithi183 ceph-mon[37326]: osdmap e33: 5 total, 5 up, 5 in 2024-06-03T03:15:08.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:08 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-03T03:15:09.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:09 smithi183 ceph-mon[37326]: osdmap e34: 5 total, 5 up, 5 in 2024-06-03T03:15:09.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:09 smithi183 ceph-mon[37326]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 447 GiB / 447 GiB avail 2024-06-03T03:15:09.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:09 smithi183 ceph-mon[37326]: Detected new or changed devices on smithi151 2024-06-03T03:15:09.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:09 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:15:09.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:09 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:15:09.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:09 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:15:09.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:09 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:15:09.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:09 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:15:09.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:09 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:15:09.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:09 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:15:09.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:09 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:15:09.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:09 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:15:09.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:09 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:15:09.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:09 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:15:09.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:09 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:15:09.656 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:09 smithi151 ceph-mon[37043]: osdmap e34: 5 total, 5 up, 5 in 2024-06-03T03:15:09.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:09 smithi151 ceph-mon[37043]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 447 GiB / 447 GiB avail 2024-06-03T03:15:09.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:09 smithi151 ceph-mon[37043]: Detected new or changed devices on smithi151 2024-06-03T03:15:09.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:09 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:15:09.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:09 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:15:09.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:09 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:15:09.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:09 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:15:09.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:09 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:15:09.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:09 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:15:09.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:09 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:15:09.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:09 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:15:09.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:09 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:15:09.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:09 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:15:09.658 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:09 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:15:09.658 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:09 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:15:09.697 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:09 smithi117 ceph-mon[31649]: osdmap e34: 5 total, 5 up, 5 in 2024-06-03T03:15:09.697 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:09 smithi117 ceph-mon[31649]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 447 GiB / 447 GiB avail 2024-06-03T03:15:09.697 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:09 smithi117 ceph-mon[31649]: Detected new or changed devices on smithi151 2024-06-03T03:15:09.697 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:09 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:15:09.698 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:09 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:15:09.698 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:09 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:15:09.698 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:09 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:15:09.698 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:09 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:15:09.698 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:09 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:15:09.698 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:09 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:15:09.698 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:09 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:15:09.698 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:09 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:15:09.698 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:09 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:15:09.698 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:09 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:15:09.698 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:09 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:15:10.115 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:15:10.112+0000 7f6e57615700 1 -- 172.21.15.183:0/2626902804 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6e50104080 msgr2=0x7f6e5010a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:15:10.115 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:15:10.112+0000 7f6e57615700 1 --2- 172.21.15.183:0/2626902804 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6e50104080 0x7f6e5010a930 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f6e48009990 tx=0x7f6e48009ca0 comp rx=0 tx=0).stop 2024-06-03T03:15:10.115 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:15:10.112+0000 7f6e57615700 1 -- 172.21.15.183:0/2626902804 shutdown_connections 2024-06-03T03:15:10.115 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:15:10.112+0000 7f6e57615700 1 --2- 172.21.15.183:0/2626902804 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6e50104080 0x7f6e5010a930 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:15:10.115 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:15:10.112+0000 7f6e57615700 1 --2- 172.21.15.183:0/2626902804 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6e501024e0 0x7f6e50102900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:15:10.115 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:15:10.112+0000 7f6e57615700 1 --2- 172.21.15.183:0/2626902804 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e501036e0 0x7f6e50103b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:15:10.115 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:15:10.112+0000 7f6e57615700 1 -- 172.21.15.183:0/2626902804 >> 172.21.15.183:0/2626902804 conn(0x7f6e500fda80 msgr2=0x7f6e500ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:15:10.115 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:15:10.112+0000 7f6e57615700 1 -- 172.21.15.183:0/2626902804 shutdown_connections 2024-06-03T03:15:10.115 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:15:10.112+0000 7f6e57615700 1 -- 172.21.15.183:0/2626902804 wait complete. 2024-06-03T03:15:10.115 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:15:10.113+0000 7f6e57615700 1 Processor -- start 2024-06-03T03:15:10.116 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:15:10.113+0000 7f6e57615700 1 -- start start 2024-06-03T03:15:10.116 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:15:10.113+0000 7f6e57615700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6e501024e0 0x7f6e50076230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:15:10.116 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:15:10.113+0000 7f6e57615700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e501036e0 0x7f6e50074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:15:10.116 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:15:10.113+0000 7f6e57615700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6e50104080 0x7f6e50074dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:15:10.116 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:15:10.113+0000 7f6e57615700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6e5010e210 con 0x7f6e501036e0 2024-06-03T03:15:10.116 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:15:10.113+0000 7f6e57615700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6e5010e090 con 0x7f6e50104080 2024-06-03T03:15:10.116 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:15:10.113+0000 7f6e57615700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6e5010e390 con 0x7f6e501024e0 2024-06-03T03:15:10.116 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:15:10.113+0000 7f6e553b1700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6e501024e0 0x7f6e50076230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:15:10.116 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:15:10.113+0000 7f6e553b1700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6e501024e0 0x7f6e50076230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:54006/0 (socket says 172.21.15.183:54006) 2024-06-03T03:15:10.116 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:15:10.113+0000 7f6e553b1700 1 -- 172.21.15.183:0/2844643291 learned_addr learned my addr 172.21.15.183:0/2844643291 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-06-03T03:15:10.116 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:15:10.113+0000 7f6e553b1700 1 -- 172.21.15.183:0/2844643291 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6e50104080 msgr2=0x7f6e50074dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:15:10.116 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:15:10.113+0000 7f6e553b1700 1 --2- 172.21.15.183:0/2844643291 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6e50104080 0x7f6e50074dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:15:10.116 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:15:10.113+0000 7f6e553b1700 1 -- 172.21.15.183:0/2844643291 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e501036e0 msgr2=0x7f6e50074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:15:10.117 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:15:10.113+0000 7f6e553b1700 1 --2- 172.21.15.183:0/2844643291 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e501036e0 0x7f6e50074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:15:10.117 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:15:10.113+0000 7f6e553b1700 1 -- 172.21.15.183:0/2844643291 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e50075670 con 0x7f6e501024e0 2024-06-03T03:15:10.117 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:15:10.113+0000 7f6e553b1700 1 --2- 172.21.15.183:0/2844643291 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6e501024e0 0x7f6e50076230 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c00b9c0 tx=0x7f6e4c00bd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:15:10.117 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:15:10.113+0000 7f6e467fc700 1 -- 172.21.15.183:0/2844643291 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c019070 con 0x7f6e501024e0 2024-06-03T03:15:10.117 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:15:10.113+0000 7f6e467fc700 1 -- 172.21.15.183:0/2844643291 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e4c014440 con 0x7f6e501024e0 2024-06-03T03:15:10.117 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:15:10.113+0000 7f6e57615700 1 -- 172.21.15.183:0/2844643291 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e501a29e0 con 0x7f6e501024e0 2024-06-03T03:15:10.117 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:15:10.113+0000 7f6e467fc700 1 -- 172.21.15.183:0/2844643291 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c0125a0 con 0x7f6e501024e0 2024-06-03T03:15:10.117 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:15:10.114+0000 7f6e57615700 1 -- 172.21.15.183:0/2844643291 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e501a2f20 con 0x7f6e501024e0 2024-06-03T03:15:10.118 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:15:10.114+0000 7f6e57615700 1 -- 172.21.15.183:0/2844643291 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e50069180 con 0x7f6e501024e0 2024-06-03T03:15:10.118 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:15:10.115+0000 7f6e467fc700 1 -- 172.21.15.183:0/2844643291 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f6e4c01c070 con 0x7f6e501024e0 2024-06-03T03:15:10.118 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:15:10.116+0000 7f6e467fc700 1 --2- 172.21.15.183:0/2844643291 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f6e3c0a2c20 0x7f6e3c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:15:10.118 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:15:10.116+0000 7f6e467fc700 1 -- 172.21.15.183:0/2844643291 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4101+0+0 (secure 0 0 0) 0x7f6e4c0cd950 con 0x7f6e501024e0 2024-06-03T03:15:10.119 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:15:10.116+0000 7f6e54bb0700 1 --2- 172.21.15.183:0/2844643291 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f6e3c0a2c20 0x7f6e3c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:15:10.119 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:15:10.117+0000 7f6e54bb0700 1 --2- 172.21.15.183:0/2844643291 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f6e3c0a2c20 0x7f6e3c0a50e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6e400097b0 tx=0x7f6e40006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:15:10.121 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:15:10.120+0000 7f6e467fc700 1 -- 172.21.15.183:0/2844643291 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6e4c09b920 con 0x7f6e501024e0 2024-06-03T03:15:10.330 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:15:10.329+0000 7f6e57615700 1 -- 172.21.15.183:0/2844643291 --> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f6e500628d0 con 0x7f6e3c0a2c20 2024-06-03T03:15:10.656 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:10 smithi151 ceph-mon[37043]: Adjusting osd_memory_target on smithi151 to 8544M 2024-06-03T03:15:10.696 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:10 smithi117 ceph-mon[31649]: Adjusting osd_memory_target on smithi151 to 8544M 2024-06-03T03:15:10.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:10 smithi183 ceph-mon[37326]: Adjusting osd_memory_target on smithi151 to 8544M 2024-06-03T03:15:11.656 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:11 smithi151 ceph-mon[37043]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:15:11.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:11 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T03:15:11.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:11 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T03:15:11.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:11 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:15:11.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:11 smithi151 ceph-mon[37043]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 447 GiB / 447 GiB avail 2024-06-03T03:15:11.696 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:11 smithi117 ceph-mon[31649]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:15:11.697 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:11 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T03:15:11.697 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:11 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T03:15:11.697 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:11 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:15:11.697 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:11 smithi117 ceph-mon[31649]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 447 GiB / 447 GiB avail 2024-06-03T03:15:11.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:11 smithi183 ceph-mon[37326]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:15:11.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:11 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T03:15:11.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:11 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T03:15:11.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:11 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:15:11.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:11 smithi183 ceph-mon[37326]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 447 GiB / 447 GiB avail 2024-06-03T03:15:14.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:13 smithi151 ceph-mon[37043]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-06-03T03:15:14.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:13 smithi151 ceph-mon[37043]: from='client.? 172.21.15.183:0/352983711' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8ab6a5f0-34bb-48d8-9471-a7dc836e486e"}]: dispatch 2024-06-03T03:15:14.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:13 smithi151 ceph-mon[37043]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8ab6a5f0-34bb-48d8-9471-a7dc836e486e"}]: dispatch 2024-06-03T03:15:14.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:13 smithi151 ceph-mon[37043]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8ab6a5f0-34bb-48d8-9471-a7dc836e486e"}]': finished 2024-06-03T03:15:14.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:13 smithi151 ceph-mon[37043]: osdmap e35: 6 total, 5 up, 6 in 2024-06-03T03:15:14.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:13 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-03T03:15:14.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:13 smithi151 ceph-mon[37043]: from='client.? 172.21.15.183:0/1800659047' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T03:15:14.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:13 smithi117 ceph-mon[31649]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-06-03T03:15:14.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:13 smithi117 ceph-mon[31649]: from='client.? 172.21.15.183:0/352983711' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8ab6a5f0-34bb-48d8-9471-a7dc836e486e"}]: dispatch 2024-06-03T03:15:14.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:13 smithi117 ceph-mon[31649]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8ab6a5f0-34bb-48d8-9471-a7dc836e486e"}]: dispatch 2024-06-03T03:15:14.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:13 smithi117 ceph-mon[31649]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8ab6a5f0-34bb-48d8-9471-a7dc836e486e"}]': finished 2024-06-03T03:15:14.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:13 smithi117 ceph-mon[31649]: osdmap e35: 6 total, 5 up, 6 in 2024-06-03T03:15:14.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:13 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-03T03:15:14.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:13 smithi117 ceph-mon[31649]: from='client.? 172.21.15.183:0/1800659047' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T03:15:14.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:13 smithi183 ceph-mon[37326]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-06-03T03:15:14.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:13 smithi183 ceph-mon[37326]: from='client.? 172.21.15.183:0/352983711' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8ab6a5f0-34bb-48d8-9471-a7dc836e486e"}]: dispatch 2024-06-03T03:15:14.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:13 smithi183 ceph-mon[37326]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8ab6a5f0-34bb-48d8-9471-a7dc836e486e"}]: dispatch 2024-06-03T03:15:14.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:13 smithi183 ceph-mon[37326]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8ab6a5f0-34bb-48d8-9471-a7dc836e486e"}]': finished 2024-06-03T03:15:14.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:13 smithi183 ceph-mon[37326]: osdmap e35: 6 total, 5 up, 6 in 2024-06-03T03:15:14.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:13 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-03T03:15:14.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:13 smithi183 ceph-mon[37326]: from='client.? 172.21.15.183:0/1800659047' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T03:15:16.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:15 smithi151 ceph-mon[37043]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-06-03T03:15:16.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:15 smithi117 ceph-mon[31649]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-06-03T03:15:16.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:15 smithi183 ceph-mon[37326]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-06-03T03:15:18.092 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:17 smithi183 ceph-mon[37326]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-06-03T03:15:18.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:17 smithi151 ceph-mon[37043]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-06-03T03:15:18.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:17 smithi117 ceph-mon[31649]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-06-03T03:15:19.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:19 smithi183 ceph-mon[37326]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-06-03T03:15:20.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:19 smithi151 ceph-mon[37043]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-06-03T03:15:20.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:19 smithi117 ceph-mon[31649]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-06-03T03:15:21.070 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:20 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-06-03T03:15:21.070 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:20 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:15:21.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:20 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-06-03T03:15:21.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:20 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:15:21.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:20 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-06-03T03:15:21.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:20 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:15:22.098 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:21 smithi183 ceph-mon[37326]: Deploying daemon osd.5 on smithi183 2024-06-03T03:15:22.098 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:21 smithi183 ceph-mon[37326]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-06-03T03:15:22.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:21 smithi151 ceph-mon[37043]: Deploying daemon osd.5 on smithi183 2024-06-03T03:15:22.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:21 smithi151 ceph-mon[37043]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-06-03T03:15:22.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:21 smithi117 ceph-mon[31649]: Deploying daemon osd.5 on smithi183 2024-06-03T03:15:22.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:21 smithi117 ceph-mon[31649]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-06-03T03:15:24.083 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:23 smithi183 ceph-mon[37326]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-06-03T03:15:24.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:23 smithi151 ceph-mon[37043]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-06-03T03:15:24.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:23 smithi117 ceph-mon[31649]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-06-03T03:15:26.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:25 smithi151 ceph-mon[37043]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-06-03T03:15:26.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:25 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:15:26.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:25 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:15:26.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:25 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:15:26.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:25 smithi117 ceph-mon[31649]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-06-03T03:15:26.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:25 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:15:26.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:25 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:15:26.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:25 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:15:26.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:25 smithi183 ceph-mon[37326]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-06-03T03:15:26.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:25 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:15:26.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:25 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:15:26.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:25 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:15:27.850 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:27 smithi183 ceph-mon[37326]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-06-03T03:15:28.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:27 smithi151 ceph-mon[37043]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-06-03T03:15:28.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:27 smithi117 ceph-mon[31649]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-06-03T03:15:29.417 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:15:29.416+0000 7f6e467fc700 1 -- 172.21.15.183:0/2844643291 <== mgr.14152 v2:172.21.15.117:6800/1398160887 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6e500628d0 con 0x7f6e3c0a2c20 2024-06-03T03:15:29.418 INFO:teuthology.orchestra.run.smithi183.stdout:Created osd(s) 5 on host 'smithi183' 2024-06-03T03:15:29.421 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:15:29.418+0000 7f6e57615700 1 -- 172.21.15.183:0/2844643291 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f6e3c0a2c20 msgr2=0x7f6e3c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:15:29.421 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:15:29.418+0000 7f6e57615700 1 --2- 172.21.15.183:0/2844643291 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f6e3c0a2c20 0x7f6e3c0a50e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6e400097b0 tx=0x7f6e40006d20 comp rx=0 tx=0).stop 2024-06-03T03:15:29.421 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:15:29.418+0000 7f6e57615700 1 -- 172.21.15.183:0/2844643291 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6e501024e0 msgr2=0x7f6e50076230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:15:29.421 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:15:29.418+0000 7f6e57615700 1 --2- 172.21.15.183:0/2844643291 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6e501024e0 0x7f6e50076230 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c00b9c0 tx=0x7f6e4c00bd80 comp rx=0 tx=0).stop 2024-06-03T03:15:29.421 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:15:29.418+0000 7f6e57615700 1 -- 172.21.15.183:0/2844643291 shutdown_connections 2024-06-03T03:15:29.421 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:15:29.418+0000 7f6e57615700 1 --2- 172.21.15.183:0/2844643291 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f6e3c0a2c20 0x7f6e3c0a50e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:15:29.421 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:15:29.418+0000 7f6e57615700 1 --2- 172.21.15.183:0/2844643291 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6e50104080 0x7f6e50074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:15:29.421 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:15:29.418+0000 7f6e57615700 1 --2- 172.21.15.183:0/2844643291 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6e501024e0 0x7f6e50076230 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:15:29.421 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:15:29.418+0000 7f6e57615700 1 --2- 172.21.15.183:0/2844643291 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e501036e0 0x7f6e50074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:15:29.421 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:15:29.418+0000 7f6e57615700 1 -- 172.21.15.183:0/2844643291 >> 172.21.15.183:0/2844643291 conn(0x7f6e500fda80 msgr2=0x7f6e50106a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:15:29.422 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:15:29.418+0000 7f6e57615700 1 -- 172.21.15.183:0/2844643291 shutdown_connections 2024-06-03T03:15:29.422 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:15:29.418+0000 7f6e57615700 1 -- 172.21.15.183:0/2844643291 wait complete. 2024-06-03T03:15:29.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:29 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:15:29.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:29 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:15:29.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:29 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:15:29.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:29 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:15:29.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:29 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:15:29.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:29 smithi183 ceph-mon[37326]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-06-03T03:15:29.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:29 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:15:29.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:29 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:15:29.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:29 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:15:29.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:29 smithi183 ceph-mon[37326]: from='osd.5 [v2:172.21.15.183:6808/2985874686,v1:172.21.15.183:6809/2985874686]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-03T03:15:29.733 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:29 smithi183 ceph-mon[37326]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-03T03:15:29.906 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:29 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:15:29.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:29 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:15:29.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:29 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:15:29.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:29 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:15:29.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:29 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:15:29.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:29 smithi151 ceph-mon[37043]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-06-03T03:15:29.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:29 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:15:29.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:29 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:15:29.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:29 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:15:29.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:29 smithi151 ceph-mon[37043]: from='osd.5 [v2:172.21.15.183:6808/2985874686,v1:172.21.15.183:6809/2985874686]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-03T03:15:29.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:29 smithi151 ceph-mon[37043]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-03T03:15:29.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:29 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:15:29.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:29 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:15:29.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:29 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:15:29.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:29 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:15:29.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:29 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:15:29.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:29 smithi117 ceph-mon[31649]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-06-03T03:15:29.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:29 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:15:29.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:29 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:15:29.948 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:29 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:15:29.948 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:29 smithi117 ceph-mon[31649]: from='osd.5 [v2:172.21.15.183:6808/2985874686,v1:172.21.15.183:6809/2985874686]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-03T03:15:29.948 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:29 smithi117 ceph-mon[31649]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-03T03:15:30.071 DEBUG:teuthology.orchestra.run.smithi183:osd.5> sudo journalctl -f -n 0 -u ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@osd.5.service 2024-06-03T03:15:30.073 INFO:tasks.cephadm:Deploying osd.6 on smithi117 with /dev/vg_nvme/lv_2... 2024-06-03T03:15:30.073 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-06-03T03:15:30.521 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:15:31.469 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:31 smithi183 ceph-mon[37326]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-06-03T03:15:31.470 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:31 smithi183 ceph-mon[37326]: from='osd.5 [v2:172.21.15.183:6808/2985874686,v1:172.21.15.183:6809/2985874686]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-06-03T03:15:31.470 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:31 smithi183 ceph-mon[37326]: osdmap e36: 6 total, 5 up, 6 in 2024-06-03T03:15:31.470 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:31 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-03T03:15:31.470 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:31 smithi183 ceph-mon[37326]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-06-03T03:15:31.470 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:31 smithi183 ceph-mon[37326]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-06-03T03:15:31.656 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:31 smithi151 ceph-mon[37043]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-06-03T03:15:31.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:31 smithi151 ceph-mon[37043]: from='osd.5 [v2:172.21.15.183:6808/2985874686,v1:172.21.15.183:6809/2985874686]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-06-03T03:15:31.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:31 smithi151 ceph-mon[37043]: osdmap e36: 6 total, 5 up, 6 in 2024-06-03T03:15:31.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:31 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-03T03:15:31.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:31 smithi151 ceph-mon[37043]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-06-03T03:15:31.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:31 smithi151 ceph-mon[37043]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-06-03T03:15:31.697 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:31 smithi117 ceph-mon[31649]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-06-03T03:15:31.697 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:31 smithi117 ceph-mon[31649]: from='osd.5 [v2:172.21.15.183:6808/2985874686,v1:172.21.15.183:6809/2985874686]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-06-03T03:15:31.697 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:31 smithi117 ceph-mon[31649]: osdmap e36: 6 total, 5 up, 6 in 2024-06-03T03:15:31.697 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:31 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-03T03:15:31.697 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:31 smithi117 ceph-mon[31649]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-06-03T03:15:31.697 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:31 smithi117 ceph-mon[31649]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-06-03T03:15:32.233 INFO:journalctl@ceph.osd.5.smithi183.stdout:Jun 03 03:15:31 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-5[49050]: 2024-06-03T03:15:31.857+0000 7f780a364700 -1 osd.5 0 waiting for initial osdmap 2024-06-03T03:15:32.233 INFO:journalctl@ceph.osd.5.smithi183.stdout:Jun 03 03:15:31 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-5[49050]: 2024-06-03T03:15:31.864+0000 7f780214f700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-03T03:15:32.592 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:32 smithi183 ceph-mon[37326]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-06-03T03:15:32.592 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:32 smithi183 ceph-mon[37326]: osdmap e37: 6 total, 5 up, 6 in 2024-06-03T03:15:32.592 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:32 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-03T03:15:32.592 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:32 smithi183 ceph-mon[37326]: from='osd.5 ' entity='osd.5' 2024-06-03T03:15:32.696 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:32 smithi117 ceph-mon[31649]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-06-03T03:15:32.697 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:32 smithi117 ceph-mon[31649]: osdmap e37: 6 total, 5 up, 6 in 2024-06-03T03:15:32.697 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:32 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-03T03:15:32.697 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:32 smithi117 ceph-mon[31649]: from='osd.5 ' entity='osd.5' 2024-06-03T03:15:32.906 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:32 smithi151 ceph-mon[37043]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-06-03T03:15:32.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:32 smithi151 ceph-mon[37043]: osdmap e37: 6 total, 5 up, 6 in 2024-06-03T03:15:32.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:32 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-03T03:15:32.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:32 smithi151 ceph-mon[37043]: from='osd.5 ' entity='osd.5' 2024-06-03T03:15:33.585 INFO:teuthology.orchestra.run.smithi117.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-06-03T03:15:33.585 INFO:teuthology.orchestra.run.smithi117.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-06-03T03:15:33.586 INFO:teuthology.orchestra.run.smithi117.stderr: stderr: 10+0 records in 2024-06-03T03:15:33.586 INFO:teuthology.orchestra.run.smithi117.stderr:10+0 records out 2024-06-03T03:15:33.586 INFO:teuthology.orchestra.run.smithi117.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0335938 s, 312 MB/s 2024-06-03T03:15:33.586 INFO:teuthology.orchestra.run.smithi117.stderr:--> Zapping successful for: 2024-06-03T03:15:33.695 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:33 smithi117 ceph-mon[31649]: purged_snaps scrub starts 2024-06-03T03:15:33.695 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:33 smithi117 ceph-mon[31649]: purged_snaps scrub ok 2024-06-03T03:15:33.695 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:33 smithi117 ceph-mon[31649]: osd.5 [v2:172.21.15.183:6808/2985874686,v1:172.21.15.183:6809/2985874686] boot 2024-06-03T03:15:33.695 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:33 smithi117 ceph-mon[31649]: osdmap e38: 6 total, 6 up, 6 in 2024-06-03T03:15:33.695 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:33 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-03T03:15:33.695 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:33 smithi117 ceph-mon[31649]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-06-03T03:15:33.695 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:33 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:15:33.695 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:33 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:15:33.695 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:33 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:15:33.695 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:33 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:15:33.696 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:33 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:15:33.696 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:33 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:15:33.696 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:33 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:15:33.696 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:33 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:15:33.696 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:33 smithi117 ceph-mon[31649]: osdmap e39: 6 total, 6 up, 6 in 2024-06-03T03:15:33.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:33 smithi183 ceph-mon[37326]: purged_snaps scrub starts 2024-06-03T03:15:33.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:33 smithi183 ceph-mon[37326]: purged_snaps scrub ok 2024-06-03T03:15:33.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:33 smithi183 ceph-mon[37326]: osd.5 [v2:172.21.15.183:6808/2985874686,v1:172.21.15.183:6809/2985874686] boot 2024-06-03T03:15:33.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:33 smithi183 ceph-mon[37326]: osdmap e38: 6 total, 6 up, 6 in 2024-06-03T03:15:33.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:33 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-03T03:15:33.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:33 smithi183 ceph-mon[37326]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-06-03T03:15:33.733 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:33 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:15:33.733 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:33 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:15:33.733 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:33 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:15:33.733 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:33 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:15:33.733 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:33 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:15:33.733 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:33 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:15:33.733 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:33 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:15:33.733 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:33 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:15:33.733 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:33 smithi183 ceph-mon[37326]: osdmap e39: 6 total, 6 up, 6 in 2024-06-03T03:15:33.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:33 smithi151 ceph-mon[37043]: purged_snaps scrub starts 2024-06-03T03:15:33.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:33 smithi151 ceph-mon[37043]: purged_snaps scrub ok 2024-06-03T03:15:33.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:33 smithi151 ceph-mon[37043]: osd.5 [v2:172.21.15.183:6808/2985874686,v1:172.21.15.183:6809/2985874686] boot 2024-06-03T03:15:33.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:33 smithi151 ceph-mon[37043]: osdmap e38: 6 total, 6 up, 6 in 2024-06-03T03:15:33.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:33 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-03T03:15:33.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:33 smithi151 ceph-mon[37043]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-06-03T03:15:33.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:33 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:15:33.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:33 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:15:33.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:33 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:15:33.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:33 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:15:33.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:33 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:15:33.908 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:33 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:15:33.908 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:33 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:15:33.908 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:33 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:15:33.908 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:33 smithi151 ceph-mon[37043]: osdmap e39: 6 total, 6 up, 6 in 2024-06-03T03:15:34.622 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi117:vg_nvme/lv_2 2024-06-03T03:15:34.696 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:34 smithi117 ceph-mon[31649]: Detected new or changed devices on smithi183 2024-06-03T03:15:34.696 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:34 smithi117 ceph-mon[31649]: Adjusting osd_memory_target on smithi183 to 8544M 2024-06-03T03:15:34.696 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:34 smithi117 ceph-mon[31649]: osdmap e40: 6 total, 6 up, 6 in 2024-06-03T03:15:34.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:34 smithi183 ceph-mon[37326]: Detected new or changed devices on smithi183 2024-06-03T03:15:34.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:34 smithi183 ceph-mon[37326]: Adjusting osd_memory_target on smithi183 to 8544M 2024-06-03T03:15:34.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:34 smithi183 ceph-mon[37326]: osdmap e40: 6 total, 6 up, 6 in 2024-06-03T03:15:34.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:34 smithi151 ceph-mon[37043]: Detected new or changed devices on smithi183 2024-06-03T03:15:34.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:34 smithi151 ceph-mon[37043]: Adjusting osd_memory_target on smithi183 to 8544M 2024-06-03T03:15:34.908 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:34 smithi151 ceph-mon[37043]: osdmap e40: 6 total, 6 up, 6 in 2024-06-03T03:15:35.074 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:15:35.560 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:35 smithi117 ceph-mon[31649]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-06-03T03:15:35.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:35 smithi183 ceph-mon[37326]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-06-03T03:15:35.906 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:35 smithi151 ceph-mon[37043]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-06-03T03:15:36.696 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:36 smithi117 ceph-mon[31649]: osdmap e41: 6 total, 6 up, 6 in 2024-06-03T03:15:36.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:36 smithi183 ceph-mon[37326]: osdmap e41: 6 total, 6 up, 6 in 2024-06-03T03:15:36.906 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:36 smithi151 ceph-mon[37043]: osdmap e41: 6 total, 6 up, 6 in 2024-06-03T03:15:37.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:15:37.395+0000 7fadd6400700 1 -- 172.21.15.117:0/3722193542 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadd0102f80 msgr2=0x7fadd0105370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:15:37.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:15:37.395+0000 7fadd6400700 1 --2- 172.21.15.117:0/3722193542 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadd0102f80 0x7fadd0105370 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fadc0009990 tx=0x7fadc0009ca0 comp rx=0 tx=0).stop 2024-06-03T03:15:37.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:15:37.395+0000 7fadd6400700 1 -- 172.21.15.117:0/3722193542 shutdown_connections 2024-06-03T03:15:37.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:15:37.395+0000 7fadd6400700 1 --2- 172.21.15.117:0/3722193542 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fadd01081e0 0x7fadd010a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:15:37.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:15:37.395+0000 7fadd6400700 1 --2- 172.21.15.117:0/3722193542 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadd0102f80 0x7fadd0105370 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:15:37.397 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:15:37.395+0000 7fadd6400700 1 --2- 172.21.15.117:0/3722193542 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fadd01058b0 0x7fadd0107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:15:37.397 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:15:37.395+0000 7fadd6400700 1 -- 172.21.15.117:0/3722193542 >> 172.21.15.117:0/3722193542 conn(0x7fadd00fc990 msgr2=0x7fadd00fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:15:37.397 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:15:37.395+0000 7fadd6400700 1 -- 172.21.15.117:0/3722193542 shutdown_connections 2024-06-03T03:15:37.397 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:15:37.395+0000 7fadd6400700 1 -- 172.21.15.117:0/3722193542 wait complete. 2024-06-03T03:15:37.398 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:15:37.396+0000 7fadd6400700 1 Processor -- start 2024-06-03T03:15:37.398 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:15:37.396+0000 7fadd6400700 1 -- start start 2024-06-03T03:15:37.398 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:15:37.397+0000 7fadd6400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fadd0102f80 0x7fadd019a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:15:37.398 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:15:37.397+0000 7fadd6400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fadd01058b0 0x7fadd019af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:15:37.398 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:15:37.397+0000 7fadd6400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadd01081e0 0x7fadd01a1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:15:37.398 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:15:37.397+0000 7fadd6400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fadd010e150 con 0x7fadd01081e0 2024-06-03T03:15:37.398 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:15:37.397+0000 7fadd6400700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fadd010dfd0 con 0x7fadd0102f80 2024-06-03T03:15:37.398 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:15:37.397+0000 7fadd6400700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fadd010e2d0 con 0x7fadd01058b0 2024-06-03T03:15:37.398 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:15:37.397+0000 7fadcf400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fadd0102f80 0x7fadd019a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:15:37.398 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:15:37.397+0000 7fadcfe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadd01081e0 0x7fadd01a1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:15:37.398 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:15:37.397+0000 7fadcea00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fadd01058b0 0x7fadd019af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:15:37.399 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:15:37.397+0000 7fadcf400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fadd0102f80 0x7fadd019a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:56392/0 (socket says 172.21.15.117:56392) 2024-06-03T03:15:37.399 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:15:37.397+0000 7fadcf400700 1 -- 172.21.15.117:0/565912203 learned_addr learned my addr 172.21.15.117:0/565912203 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:15:37.399 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:15:37.397+0000 7fadcea00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fadd01058b0 0x7fadd019af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:59884/0 (socket says 172.21.15.117:59884) 2024-06-03T03:15:37.400 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:15:37.397+0000 7fadcf400700 1 -- 172.21.15.117:0/565912203 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fadd01058b0 msgr2=0x7fadd019af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:15:37.400 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:15:37.397+0000 7fadcf400700 1 --2- 172.21.15.117:0/565912203 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fadd01058b0 0x7fadd019af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:15:37.400 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:15:37.397+0000 7fadcf400700 1 -- 172.21.15.117:0/565912203 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadd01081e0 msgr2=0x7fadd01a1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:15:37.400 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:15:37.397+0000 7fadcf400700 1 --2- 172.21.15.117:0/565912203 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadd01081e0 0x7fadd01a1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:15:37.400 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:15:37.397+0000 7fadcf400700 1 -- 172.21.15.117:0/565912203 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadd01a2630 con 0x7fadd0102f80 2024-06-03T03:15:37.400 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:15:37.398+0000 7fadcf400700 1 --2- 172.21.15.117:0/565912203 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fadd0102f80 0x7fadd019a9d0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fadc0005910 tx=0x7fadc000bbb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:15:37.400 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:15:37.398+0000 7fadc7e00700 1 -- 172.21.15.117:0/565912203 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc0026070 con 0x7fadd0102f80 2024-06-03T03:15:37.400 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:15:37.398+0000 7fadd6400700 1 -- 172.21.15.117:0/565912203 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadd01a28c0 con 0x7fadd0102f80 2024-06-03T03:15:37.401 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:15:37.398+0000 7fadc7e00700 1 -- 172.21.15.117:0/565912203 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadc0003d70 con 0x7fadd0102f80 2024-06-03T03:15:37.401 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:15:37.398+0000 7fadc7e00700 1 -- 172.21.15.117:0/565912203 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc0021410 con 0x7fadd0102f80 2024-06-03T03:15:37.401 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:15:37.398+0000 7fadd6400700 1 -- 172.21.15.117:0/565912203 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadd01a2d70 con 0x7fadd0102f80 2024-06-03T03:15:37.401 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:15:37.399+0000 7fadd6400700 1 -- 172.21.15.117:0/565912203 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadd0069180 con 0x7fadd0102f80 2024-06-03T03:15:37.402 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:15:37.401+0000 7fadc7e00700 1 -- 172.21.15.117:0/565912203 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fadc0029070 con 0x7fadd0102f80 2024-06-03T03:15:37.403 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:15:37.403+0000 7fadc7e00700 1 --2- 172.21.15.117:0/565912203 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fadb40a2c20 0x7fadb40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:15:37.404 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:15:37.403+0000 7fadc7e00700 1 -- 172.21.15.117:0/565912203 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4533+0+0 (secure 0 0 0) 0x7fadc00ceef0 con 0x7fadd0102f80 2024-06-03T03:15:37.404 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:15:37.403+0000 7fadcea00700 1 --2- 172.21.15.117:0/565912203 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fadb40a2c20 0x7fadb40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:15:37.404 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:15:37.404+0000 7fadcea00700 1 --2- 172.21.15.117:0/565912203 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fadb40a2c20 0x7fadb40a50e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fadbc009c60 tx=0x7fadbc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:15:37.406 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:15:37.405+0000 7fadc7e00700 1 -- 172.21.15.117:0/565912203 <== mon.2 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fadc0013070 con 0x7fadd0102f80 2024-06-03T03:15:37.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:15:37.645+0000 7fadd6400700 1 -- 172.21.15.117:0/565912203 --> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fadd00628d0 con 0x7fadb40a2c20 2024-06-03T03:15:37.646 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:37 smithi117 ceph-mon[31649]: osdmap e42: 6 total, 6 up, 6 in 2024-06-03T03:15:37.646 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:37 smithi117 ceph-mon[31649]: pgmap v158: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-06-03T03:15:37.656 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:37 smithi151 ceph-mon[37043]: osdmap e42: 6 total, 6 up, 6 in 2024-06-03T03:15:37.656 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:37 smithi151 ceph-mon[37043]: pgmap v158: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-06-03T03:15:37.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:37 smithi183 ceph-mon[37326]: osdmap e42: 6 total, 6 up, 6 in 2024-06-03T03:15:37.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:37 smithi183 ceph-mon[37326]: pgmap v158: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-06-03T03:15:38.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:38 smithi183 ceph-mon[37326]: from='client.24260 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:15:38.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:38 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T03:15:38.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:38 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T03:15:38.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:38 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:15:38.794 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:38 smithi117 ceph-mon[31649]: from='client.24260 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:15:38.794 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:38 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T03:15:38.794 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:38 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T03:15:38.794 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:38 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:15:38.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:38 smithi151 ceph-mon[37043]: from='client.24260 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:15:38.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:38 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T03:15:38.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:38 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T03:15:38.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:38 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:15:39.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:39 smithi183 ceph-mon[37326]: pgmap v159: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 108 KiB/s, 0 objects/s recovering 2024-06-03T03:15:39.906 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:39 smithi151 ceph-mon[37043]: pgmap v159: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 108 KiB/s, 0 objects/s recovering 2024-06-03T03:15:39.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:39 smithi117 ceph-mon[31649]: pgmap v159: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 108 KiB/s, 0 objects/s recovering 2024-06-03T03:15:42.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:41 smithi151 ceph-mon[37043]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-06-03T03:15:42.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:41 smithi117 ceph-mon[31649]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-06-03T03:15:42.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:41 smithi183 ceph-mon[37326]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-06-03T03:15:43.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:42 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/2083614239' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "78d15739-e9e0-4f19-ae5f-ee8527a5b527"}]: dispatch 2024-06-03T03:15:43.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:42 smithi151 ceph-mon[37043]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "78d15739-e9e0-4f19-ae5f-ee8527a5b527"}]: dispatch 2024-06-03T03:15:43.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:42 smithi151 ceph-mon[37043]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "78d15739-e9e0-4f19-ae5f-ee8527a5b527"}]': finished 2024-06-03T03:15:43.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:42 smithi151 ceph-mon[37043]: osdmap e43: 7 total, 6 up, 7 in 2024-06-03T03:15:43.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:42 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-03T03:15:43.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:42 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/2211845901' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T03:15:43.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:42 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/2083614239' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "78d15739-e9e0-4f19-ae5f-ee8527a5b527"}]: dispatch 2024-06-03T03:15:43.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:42 smithi117 ceph-mon[31649]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "78d15739-e9e0-4f19-ae5f-ee8527a5b527"}]: dispatch 2024-06-03T03:15:43.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:42 smithi117 ceph-mon[31649]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "78d15739-e9e0-4f19-ae5f-ee8527a5b527"}]': finished 2024-06-03T03:15:43.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:42 smithi117 ceph-mon[31649]: osdmap e43: 7 total, 6 up, 7 in 2024-06-03T03:15:43.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:42 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-03T03:15:43.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:42 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/2211845901' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T03:15:43.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:42 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/2083614239' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "78d15739-e9e0-4f19-ae5f-ee8527a5b527"}]: dispatch 2024-06-03T03:15:43.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:42 smithi183 ceph-mon[37326]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "78d15739-e9e0-4f19-ae5f-ee8527a5b527"}]: dispatch 2024-06-03T03:15:43.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:42 smithi183 ceph-mon[37326]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "78d15739-e9e0-4f19-ae5f-ee8527a5b527"}]': finished 2024-06-03T03:15:43.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:42 smithi183 ceph-mon[37326]: osdmap e43: 7 total, 6 up, 7 in 2024-06-03T03:15:43.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:42 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-03T03:15:43.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:42 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/2211845901' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T03:15:44.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:43 smithi151 ceph-mon[37043]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-06-03T03:15:44.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:43 smithi117 ceph-mon[31649]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-06-03T03:15:44.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:43 smithi183 ceph-mon[37326]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-06-03T03:15:46.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:45 smithi151 ceph-mon[37043]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-06-03T03:15:46.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:45 smithi117 ceph-mon[31649]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-06-03T03:15:46.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:45 smithi183 ceph-mon[37326]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-06-03T03:15:48.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:47 smithi151 ceph-mon[37043]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-06-03T03:15:48.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:47 smithi117 ceph-mon[31649]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-06-03T03:15:48.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:47 smithi183 ceph-mon[37326]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-06-03T03:15:49.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:49 smithi117 ceph-mon[31649]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-06-03T03:15:50.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:49 smithi151 ceph-mon[37043]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-06-03T03:15:50.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:49 smithi183 ceph-mon[37326]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-06-03T03:15:52.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:51 smithi151 ceph-mon[37043]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-06-03T03:15:52.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:51 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-06-03T03:15:52.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:51 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:15:52.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:51 smithi117 ceph-mon[31649]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-06-03T03:15:52.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:51 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-06-03T03:15:52.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:51 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:15:52.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:51 smithi183 ceph-mon[37326]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-06-03T03:15:52.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:51 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-06-03T03:15:52.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:51 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:15:53.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:52 smithi117 ceph-mon[31649]: Deploying daemon osd.6 on smithi117 2024-06-03T03:15:53.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:52 smithi183 ceph-mon[37326]: Deploying daemon osd.6 on smithi117 2024-06-03T03:15:53.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:52 smithi151 ceph-mon[37043]: Deploying daemon osd.6 on smithi117 2024-06-03T03:15:54.173 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:53 smithi117 ceph-mon[31649]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-06-03T03:15:54.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:53 smithi183 ceph-mon[37326]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-06-03T03:15:54.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:53 smithi151 ceph-mon[37043]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-06-03T03:15:56.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:55 smithi117 ceph-mon[31649]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-06-03T03:15:56.209 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:55 smithi151 ceph-mon[37043]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-06-03T03:15:56.231 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:55 smithi183 ceph-mon[37326]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-06-03T03:15:58.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:57 smithi117 ceph-mon[31649]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-06-03T03:15:58.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:57 smithi183 ceph-mon[37326]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-06-03T03:15:58.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:57 smithi151 ceph-mon[37043]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-06-03T03:15:59.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:15:59 smithi117 ceph-mon[31649]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-06-03T03:16:00.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:15:59 smithi183 ceph-mon[37326]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-06-03T03:16:00.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:15:59 smithi151 ceph-mon[37043]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-06-03T03:16:01.756 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:01 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:16:01.756 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:01 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:16:01.756 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:01 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:16:01.756 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:01 smithi117 ceph-mon[31649]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-06-03T03:16:01.906 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:01 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:16:01.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:01 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:16:01.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:01 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:16:01.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:01 smithi151 ceph-mon[37043]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-06-03T03:16:01.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:01 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:16:01.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:01 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:16:01.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:01 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:16:01.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:01 smithi183 ceph-mon[37326]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-06-03T03:16:04.011 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:03 smithi117 ceph-mon[31649]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-06-03T03:16:04.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:03 smithi151 ceph-mon[37043]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-06-03T03:16:04.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:03 smithi183 ceph-mon[37326]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-06-03T03:16:05.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:05 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:16:05.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:05 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:16:05.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:05 smithi117 ceph-mon[31649]: from='osd.6 [v2:172.21.15.117:6818/4250583193,v1:172.21.15.117:6819/4250583193]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-03T03:16:05.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:05 smithi117 ceph-mon[31649]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-03T03:16:05.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:05 smithi117 ceph-mon[31649]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-06-03T03:16:05.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:05 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:16:05.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:05 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:16:05.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:05 smithi183 ceph-mon[37326]: from='osd.6 [v2:172.21.15.117:6818/4250583193,v1:172.21.15.117:6819/4250583193]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-03T03:16:05.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:05 smithi183 ceph-mon[37326]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-03T03:16:05.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:05 smithi183 ceph-mon[37326]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-06-03T03:16:05.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:05 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:16:05.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:05 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:16:05.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:05 smithi151 ceph-mon[37043]: from='osd.6 [v2:172.21.15.117:6818/4250583193,v1:172.21.15.117:6819/4250583193]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-03T03:16:05.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:05 smithi151 ceph-mon[37043]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-03T03:16:05.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:05 smithi151 ceph-mon[37043]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-06-03T03:16:05.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:16:05.809+0000 7fadc7e00700 1 -- 172.21.15.117:0/565912203 <== mgr.14152 v2:172.21.15.117:6800/1398160887 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fadd00628d0 con 0x7fadb40a2c20 2024-06-03T03:16:05.812 INFO:teuthology.orchestra.run.smithi117.stdout:Created osd(s) 6 on host 'smithi117' 2024-06-03T03:16:05.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:16:05.813+0000 7fadd6400700 1 -- 172.21.15.117:0/565912203 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fadb40a2c20 msgr2=0x7fadb40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:16:05.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:16:05.813+0000 7fadd6400700 1 --2- 172.21.15.117:0/565912203 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fadb40a2c20 0x7fadb40a50e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fadbc009c60 tx=0x7fadbc009260 comp rx=0 tx=0).stop 2024-06-03T03:16:05.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:16:05.813+0000 7fadd6400700 1 -- 172.21.15.117:0/565912203 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fadd0102f80 msgr2=0x7fadd019a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:16:05.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:16:05.813+0000 7fadd6400700 1 --2- 172.21.15.117:0/565912203 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fadd0102f80 0x7fadd019a9d0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fadc0005910 tx=0x7fadc000bbb0 comp rx=0 tx=0).stop 2024-06-03T03:16:05.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:16:05.814+0000 7fadd6400700 1 -- 172.21.15.117:0/565912203 shutdown_connections 2024-06-03T03:16:05.816 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:16:05.814+0000 7fadd6400700 1 --2- 172.21.15.117:0/565912203 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fadb40a2c20 0x7fadb40a50e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:16:05.816 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:16:05.814+0000 7fadd6400700 1 --2- 172.21.15.117:0/565912203 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadd01081e0 0x7fadd01a1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:16:05.816 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:16:05.814+0000 7fadd6400700 1 --2- 172.21.15.117:0/565912203 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fadd0102f80 0x7fadd019a9d0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:16:05.816 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:16:05.814+0000 7fadd6400700 1 --2- 172.21.15.117:0/565912203 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fadd01058b0 0x7fadd019af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:16:05.816 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:16:05.814+0000 7fadd6400700 1 -- 172.21.15.117:0/565912203 >> 172.21.15.117:0/565912203 conn(0x7fadd00fc990 msgr2=0x7fadd0102730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:16:05.816 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:16:05.814+0000 7fadd6400700 1 -- 172.21.15.117:0/565912203 shutdown_connections 2024-06-03T03:16:05.817 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:16:05.814+0000 7fadd6400700 1 -- 172.21.15.117:0/565912203 wait complete. 2024-06-03T03:16:06.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:06 smithi183 ceph-mon[37326]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-06-03T03:16:06.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:06 smithi183 ceph-mon[37326]: from='osd.6 [v2:172.21.15.117:6818/4250583193,v1:172.21.15.117:6819/4250583193]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-06-03T03:16:06.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:06 smithi183 ceph-mon[37326]: osdmap e44: 7 total, 6 up, 7 in 2024-06-03T03:16:06.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:06 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-03T03:16:06.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:06 smithi183 ceph-mon[37326]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-06-03T03:16:06.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:06 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:16:06.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:06 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:16:06.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:06 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:16:06.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:06 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:16:06.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:06 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:16:06.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:06 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:16:06.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:06 smithi151 ceph-mon[37043]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-06-03T03:16:06.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:06 smithi151 ceph-mon[37043]: from='osd.6 [v2:172.21.15.117:6818/4250583193,v1:172.21.15.117:6819/4250583193]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-06-03T03:16:06.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:06 smithi151 ceph-mon[37043]: osdmap e44: 7 total, 6 up, 7 in 2024-06-03T03:16:06.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:06 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-03T03:16:06.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:06 smithi151 ceph-mon[37043]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-06-03T03:16:06.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:06 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:16:06.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:06 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:16:06.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:06 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:16:06.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:06 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:16:06.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:06 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:16:06.658 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:06 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:16:06.697 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:06 smithi117 ceph-mon[31649]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-06-03T03:16:06.697 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:06 smithi117 ceph-mon[31649]: from='osd.6 [v2:172.21.15.117:6818/4250583193,v1:172.21.15.117:6819/4250583193]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-06-03T03:16:06.697 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:06 smithi117 ceph-mon[31649]: osdmap e44: 7 total, 6 up, 7 in 2024-06-03T03:16:06.697 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:06 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-03T03:16:06.697 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:06 smithi117 ceph-mon[31649]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-06-03T03:16:06.697 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:06 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:16:06.697 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:06 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:16:06.698 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:06 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:16:06.698 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:06 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:16:06.698 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:06 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:16:06.698 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:06 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:16:06.869 DEBUG:teuthology.orchestra.run.smithi117:osd.6> sudo journalctl -f -n 0 -u ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@osd.6.service 2024-06-03T03:16:06.876 INFO:tasks.cephadm:Deploying osd.7 on smithi151 with /dev/vg_nvme/lv_2... 2024-06-03T03:16:06.876 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-06-03T03:16:07.119 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.b/config 2024-06-03T03:16:07.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:07 smithi117 ceph-mon[31649]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-06-03T03:16:07.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:07 smithi117 ceph-mon[31649]: osdmap e45: 7 total, 6 up, 7 in 2024-06-03T03:16:07.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:07 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-03T03:16:07.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:07 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-03T03:16:07.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:07 smithi117 ceph-mon[31649]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-06-03T03:16:07.494 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:07 smithi151 ceph-mon[37043]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-06-03T03:16:07.494 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:07 smithi151 ceph-mon[37043]: osdmap e45: 7 total, 6 up, 7 in 2024-06-03T03:16:07.494 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:07 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-03T03:16:07.494 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:07 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-03T03:16:07.495 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:07 smithi151 ceph-mon[37043]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-06-03T03:16:07.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:07 smithi183 ceph-mon[37326]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-06-03T03:16:07.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:07 smithi183 ceph-mon[37326]: osdmap e45: 7 total, 6 up, 7 in 2024-06-03T03:16:07.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:07 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-03T03:16:07.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:07 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-03T03:16:07.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:07 smithi183 ceph-mon[37326]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-06-03T03:16:08.339 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:08 smithi117 ceph-mon[31649]: purged_snaps scrub starts 2024-06-03T03:16:08.340 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:08 smithi117 ceph-mon[31649]: purged_snaps scrub ok 2024-06-03T03:16:08.340 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:08 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-03T03:16:08.340 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:08 smithi117 ceph-mon[31649]: osd.6 [v2:172.21.15.117:6818/4250583193,v1:172.21.15.117:6819/4250583193] boot 2024-06-03T03:16:08.340 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:08 smithi117 ceph-mon[31649]: osdmap e46: 7 total, 7 up, 7 in 2024-06-03T03:16:08.340 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:08 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-03T03:16:08.558 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:08 smithi151 ceph-mon[37043]: purged_snaps scrub starts 2024-06-03T03:16:08.558 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:08 smithi151 ceph-mon[37043]: purged_snaps scrub ok 2024-06-03T03:16:08.558 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:08 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-03T03:16:08.558 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:08 smithi151 ceph-mon[37043]: osd.6 [v2:172.21.15.117:6818/4250583193,v1:172.21.15.117:6819/4250583193] boot 2024-06-03T03:16:08.558 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:08 smithi151 ceph-mon[37043]: osdmap e46: 7 total, 7 up, 7 in 2024-06-03T03:16:08.558 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:08 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-03T03:16:08.633 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:08 smithi183 ceph-mon[37326]: purged_snaps scrub starts 2024-06-03T03:16:08.633 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:08 smithi183 ceph-mon[37326]: purged_snaps scrub ok 2024-06-03T03:16:08.634 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:08 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-03T03:16:08.634 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:08 smithi183 ceph-mon[37326]: osd.6 [v2:172.21.15.117:6818/4250583193,v1:172.21.15.117:6819/4250583193] boot 2024-06-03T03:16:08.634 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:08 smithi183 ceph-mon[37326]: osdmap e46: 7 total, 7 up, 7 in 2024-06-03T03:16:08.634 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:08 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-03T03:16:09.129 INFO:teuthology.orchestra.run.smithi151.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-06-03T03:16:09.130 INFO:teuthology.orchestra.run.smithi151.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-06-03T03:16:09.130 INFO:teuthology.orchestra.run.smithi151.stderr: stderr: 10+0 records in 2024-06-03T03:16:09.130 INFO:teuthology.orchestra.run.smithi151.stderr:10+0 records out 2024-06-03T03:16:09.130 INFO:teuthology.orchestra.run.smithi151.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113073 s, 927 MB/s 2024-06-03T03:16:09.130 INFO:teuthology.orchestra.run.smithi151.stderr:--> Zapping successful for: 2024-06-03T03:16:09.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:09 smithi151 ceph-mon[37043]: osdmap e47: 7 total, 7 up, 7 in 2024-06-03T03:16:09.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:09 smithi151 ceph-mon[37043]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-03T03:16:09.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:09 smithi117 ceph-mon[31649]: osdmap e47: 7 total, 7 up, 7 in 2024-06-03T03:16:09.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:09 smithi117 ceph-mon[31649]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-03T03:16:09.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:09 smithi183 ceph-mon[37326]: osdmap e47: 7 total, 7 up, 7 in 2024-06-03T03:16:09.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:09 smithi183 ceph-mon[37326]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-03T03:16:09.875 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi151:vg_nvme/lv_2 2024-06-03T03:16:10.118 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.b/config 2024-06-03T03:16:11.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:16:11.783+0000 7f5b721dd700 1 -- 172.21.15.151:0/2337820574 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5b6c108240 msgr2=0x7f5b6c10a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:16:11.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:16:11.783+0000 7f5b721dd700 1 --2- 172.21.15.151:0/2337820574 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5b6c108240 0x7f5b6c10a650 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5b6400b0a0 tx=0x7f5b6400b3b0 comp rx=0 tx=0).stop 2024-06-03T03:16:11.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:16:11.783+0000 7f5b721dd700 1 -- 172.21.15.151:0/2337820574 shutdown_connections 2024-06-03T03:16:11.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:16:11.783+0000 7f5b721dd700 1 --2- 172.21.15.151:0/2337820574 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5b6c108240 0x7f5b6c10a650 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:16:11.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:16:11.783+0000 7f5b721dd700 1 --2- 172.21.15.151:0/2337820574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b6c102fe0 0x7f5b6c1053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:16:11.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:16:11.783+0000 7f5b721dd700 1 --2- 172.21.15.151:0/2337820574 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b6c105910 0x7f5b6c107d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:16:11.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:16:11.783+0000 7f5b721dd700 1 -- 172.21.15.151:0/2337820574 >> 172.21.15.151:0/2337820574 conn(0x7f5b6c0fc990 msgr2=0x7f5b6c0fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:16:11.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:16:11.783+0000 7f5b721dd700 1 -- 172.21.15.151:0/2337820574 shutdown_connections 2024-06-03T03:16:11.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:16:11.783+0000 7f5b721dd700 1 -- 172.21.15.151:0/2337820574 wait complete. 2024-06-03T03:16:11.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:16:11.784+0000 7f5b721dd700 1 Processor -- start 2024-06-03T03:16:11.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:16:11.784+0000 7f5b721dd700 1 -- start start 2024-06-03T03:16:11.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:16:11.784+0000 7f5b721dd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b6c102fe0 0x7f5b6c102940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:16:11.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:16:11.784+0000 7f5b721dd700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5b6c105910 0x7f5b6c100f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:16:11.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:16:11.784+0000 7f5b721dd700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b6c108240 0x7f5b6c1014d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:16:11.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:16:11.784+0000 7f5b721dd700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6c10e140 con 0x7f5b6c102fe0 2024-06-03T03:16:11.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:16:11.784+0000 7f5b721dd700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6c10dfc0 con 0x7f5b6c105910 2024-06-03T03:16:11.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:16:11.784+0000 7f5b721dd700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6c10e2c0 con 0x7f5b6c108240 2024-06-03T03:16:11.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:16:11.784+0000 7f5b62ffd700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5b6c105910 0x7f5b6c100f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:16:11.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:16:11.784+0000 7f5b62ffd700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5b6c105910 0x7f5b6c100f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:60108/0 (socket says 172.21.15.151:60108) 2024-06-03T03:16:11.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:16:11.784+0000 7f5b62ffd700 1 -- 172.21.15.151:0/4261001962 learned_addr learned my addr 172.21.15.151:0/4261001962 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-06-03T03:16:11.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:16:11.784+0000 7f5b6bfff700 1 --2- 172.21.15.151:0/4261001962 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b6c108240 0x7f5b6c1014d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:16:11.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:16:11.784+0000 7f5b62ffd700 1 -- 172.21.15.151:0/4261001962 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b6c108240 msgr2=0x7f5b6c1014d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:16:11.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:16:11.784+0000 7f5b62ffd700 1 --2- 172.21.15.151:0/4261001962 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b6c108240 0x7f5b6c1014d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:16:11.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:16:11.784+0000 7f5b62ffd700 1 -- 172.21.15.151:0/4261001962 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b6c102fe0 msgr2=0x7f5b6c102940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:16:11.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:16:11.784+0000 7f5b62ffd700 1 --2- 172.21.15.151:0/4261001962 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b6c102fe0 0x7f5b6c102940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:16:11.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:16:11.784+0000 7f5b62ffd700 1 -- 172.21.15.151:0/4261001962 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b6c101d80 con 0x7f5b6c105910 2024-06-03T03:16:11.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:16:11.785+0000 7f5b62ffd700 1 --2- 172.21.15.151:0/4261001962 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5b6c105910 0x7f5b6c100f90 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5b5400d980 tx=0x7f5b5400dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:16:11.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:16:11.785+0000 7f5b697fa700 1 -- 172.21.15.151:0/4261001962 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b54017070 con 0x7f5b6c105910 2024-06-03T03:16:11.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:16:11.785+0000 7f5b697fa700 1 -- 172.21.15.151:0/4261001962 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b54005020 con 0x7f5b6c105910 2024-06-03T03:16:11.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:16:11.785+0000 7f5b697fa700 1 -- 172.21.15.151:0/4261001962 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b54009bc0 con 0x7f5b6c105910 2024-06-03T03:16:11.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:16:11.785+0000 7f5b721dd700 1 -- 172.21.15.151:0/4261001962 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b6c1a2910 con 0x7f5b6c105910 2024-06-03T03:16:11.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:16:11.785+0000 7f5b721dd700 1 -- 172.21.15.151:0/4261001962 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b6c1a2dd0 con 0x7f5b6c105910 2024-06-03T03:16:11.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:16:11.786+0000 7f5b721dd700 1 -- 172.21.15.151:0/4261001962 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b6c069180 con 0x7f5b6c105910 2024-06-03T03:16:11.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:16:11.787+0000 7f5b697fa700 1 -- 172.21.15.151:0/4261001962 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f5b540051d0 con 0x7f5b6c105910 2024-06-03T03:16:11.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:16:11.787+0000 7f5b697fa700 1 --2- 172.21.15.151:0/4261001962 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f5b580a2c20 0x7f5b580a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:16:11.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:16:11.788+0000 7f5b697fa700 1 -- 172.21.15.151:0/4261001962 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 4965+0+0 (secure 0 0 0) 0x7f5b540ce260 con 0x7f5b6c105910 2024-06-03T03:16:11.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:16:11.788+0000 7f5b6b7fe700 1 --2- 172.21.15.151:0/4261001962 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f5b580a2c20 0x7f5b580a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:16:11.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:16:11.790+0000 7f5b6b7fe700 1 --2- 172.21.15.151:0/4261001962 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f5b580a2c20 0x7f5b580a50e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c0097b0 tx=0x7f5b5c006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:16:11.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:16:11.792+0000 7f5b697fa700 1 -- 172.21.15.151:0/4261001962 <== mon.2 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5b54020020 con 0x7f5b6c105910 2024-06-03T03:16:11.906 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:11 smithi151 ceph-mon[37043]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-03T03:16:12.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:16:12.001+0000 7f5b721dd700 1 -- 172.21.15.151:0/4261001962 --> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f5b6c062990 con 0x7f5b580a2c20 2024-06-03T03:16:12.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:11 smithi117 ceph-mon[31649]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-03T03:16:12.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:11 smithi183 ceph-mon[37326]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-03T03:16:12.903 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:12 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T03:16:12.903 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:12 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T03:16:12.904 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:12 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:16:12.904 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:12 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:16:12.904 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:12 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:16:12.904 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:12 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:16:12.904 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:12 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:16:12.904 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:12 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:16:12.904 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:12 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:16:12.904 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:12 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:16:12.904 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:12 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:16:12.904 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:12 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:16:13.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:12 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T03:16:13.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:12 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T03:16:13.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:12 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:16:13.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:12 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:16:13.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:12 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:16:13.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:12 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:16:13.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:12 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:16:13.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:12 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:16:13.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:12 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:16:13.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:12 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:16:13.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:12 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:16:13.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:12 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:16:13.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:12 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T03:16:13.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:12 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T03:16:13.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:12 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:16:13.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:12 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:16:13.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:12 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:16:13.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:12 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:16:13.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:12 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:16:13.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:12 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:16:13.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:12 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:16:13.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:12 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:16:13.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:12 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:16:13.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:12 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:16:13.906 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:13 smithi151 ceph-mon[37043]: from='client.24287 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:16:13.906 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:13 smithi151 ceph-mon[37043]: Detected new or changed devices on smithi117 2024-06-03T03:16:13.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:13 smithi151 ceph-mon[37043]: Adjusting osd_memory_target on smithi117 to 5696M 2024-06-03T03:16:13.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:13 smithi151 ceph-mon[37043]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-03T03:16:14.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:13 smithi117 ceph-mon[31649]: from='client.24287 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:16:14.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:13 smithi117 ceph-mon[31649]: Detected new or changed devices on smithi117 2024-06-03T03:16:14.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:13 smithi117 ceph-mon[31649]: Adjusting osd_memory_target on smithi117 to 5696M 2024-06-03T03:16:14.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:13 smithi117 ceph-mon[31649]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-03T03:16:14.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:13 smithi183 ceph-mon[37326]: from='client.24287 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:16:14.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:13 smithi183 ceph-mon[37326]: Detected new or changed devices on smithi117 2024-06-03T03:16:14.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:13 smithi183 ceph-mon[37326]: Adjusting osd_memory_target on smithi117 to 5696M 2024-06-03T03:16:14.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:13 smithi183 ceph-mon[37326]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-03T03:16:16.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:15 smithi151 ceph-mon[37043]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-03T03:16:16.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:15 smithi151 ceph-mon[37043]: from='client.? 172.21.15.151:0/412129467' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3724fe1f-f4d7-443b-aea8-ff3ab02bfdc6"}]: dispatch 2024-06-03T03:16:16.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:15 smithi151 ceph-mon[37043]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3724fe1f-f4d7-443b-aea8-ff3ab02bfdc6"}]: dispatch 2024-06-03T03:16:16.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:15 smithi151 ceph-mon[37043]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3724fe1f-f4d7-443b-aea8-ff3ab02bfdc6"}]': finished 2024-06-03T03:16:16.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:15 smithi151 ceph-mon[37043]: osdmap e48: 8 total, 7 up, 8 in 2024-06-03T03:16:16.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:15 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-03T03:16:16.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:15 smithi151 ceph-mon[37043]: from='client.? 172.21.15.151:0/220126312' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T03:16:16.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:15 smithi117 ceph-mon[31649]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-03T03:16:16.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:15 smithi117 ceph-mon[31649]: from='client.? 172.21.15.151:0/412129467' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3724fe1f-f4d7-443b-aea8-ff3ab02bfdc6"}]: dispatch 2024-06-03T03:16:16.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:15 smithi117 ceph-mon[31649]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3724fe1f-f4d7-443b-aea8-ff3ab02bfdc6"}]: dispatch 2024-06-03T03:16:16.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:15 smithi117 ceph-mon[31649]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3724fe1f-f4d7-443b-aea8-ff3ab02bfdc6"}]': finished 2024-06-03T03:16:16.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:15 smithi117 ceph-mon[31649]: osdmap e48: 8 total, 7 up, 8 in 2024-06-03T03:16:16.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:15 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-03T03:16:16.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:15 smithi117 ceph-mon[31649]: from='client.? 172.21.15.151:0/220126312' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T03:16:16.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:15 smithi183 ceph-mon[37326]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-03T03:16:16.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:15 smithi183 ceph-mon[37326]: from='client.? 172.21.15.151:0/412129467' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3724fe1f-f4d7-443b-aea8-ff3ab02bfdc6"}]: dispatch 2024-06-03T03:16:16.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:15 smithi183 ceph-mon[37326]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3724fe1f-f4d7-443b-aea8-ff3ab02bfdc6"}]: dispatch 2024-06-03T03:16:16.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:15 smithi183 ceph-mon[37326]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3724fe1f-f4d7-443b-aea8-ff3ab02bfdc6"}]': finished 2024-06-03T03:16:16.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:15 smithi183 ceph-mon[37326]: osdmap e48: 8 total, 7 up, 8 in 2024-06-03T03:16:16.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:15 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-03T03:16:16.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:15 smithi183 ceph-mon[37326]: from='client.? 172.21.15.151:0/220126312' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T03:16:18.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:17 smithi151 ceph-mon[37043]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-03T03:16:18.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:17 smithi117 ceph-mon[31649]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-03T03:16:18.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:17 smithi183 ceph-mon[37326]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-03T03:16:20.093 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:19 smithi151 ceph-mon[37043]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-03T03:16:20.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:19 smithi117 ceph-mon[31649]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-03T03:16:20.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:19 smithi183 ceph-mon[37326]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-03T03:16:21.898 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:21 smithi151 ceph-mon[37043]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-03T03:16:22.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:21 smithi117 ceph-mon[31649]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-03T03:16:22.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:21 smithi183 ceph-mon[37326]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-03T03:16:23.145 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:22 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-06-03T03:16:23.145 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:22 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:16:23.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:22 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-06-03T03:16:23.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:22 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:16:23.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:22 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-06-03T03:16:23.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:22 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:16:24.098 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:23 smithi151 ceph-mon[37043]: Deploying daemon osd.7 on smithi151 2024-06-03T03:16:24.099 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:23 smithi151 ceph-mon[37043]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-03T03:16:24.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:23 smithi117 ceph-mon[31649]: Deploying daemon osd.7 on smithi151 2024-06-03T03:16:24.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:23 smithi117 ceph-mon[31649]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-03T03:16:24.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:23 smithi183 ceph-mon[37326]: Deploying daemon osd.7 on smithi151 2024-06-03T03:16:24.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:23 smithi183 ceph-mon[37326]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-03T03:16:25.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:25 smithi151 ceph-mon[37043]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-03T03:16:26.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:25 smithi117 ceph-mon[31649]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-03T03:16:26.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:25 smithi183 ceph-mon[37326]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-03T03:16:27.886 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:27 smithi151 ceph-mon[37043]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-03T03:16:28.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:27 smithi117 ceph-mon[31649]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-03T03:16:28.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:27 smithi183 ceph-mon[37326]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-03T03:16:29.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:29 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:16:29.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:29 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:16:29.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:29 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:16:29.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:29 smithi151 ceph-mon[37043]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-03T03:16:29.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:29 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:16:29.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:29 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:16:29.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:29 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:16:29.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:29 smithi117 ceph-mon[31649]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-03T03:16:29.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:29 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:16:29.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:29 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:16:29.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:29 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:16:29.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:29 smithi183 ceph-mon[37326]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-03T03:16:31.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:31 smithi151 ceph-mon[37043]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-03T03:16:31.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:31 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:16:31.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:31 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:16:31.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:31 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:16:31.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:31 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:16:31.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:31 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:16:32.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:16:32.037+0000 7f5b697fa700 1 -- 172.21.15.151:0/4261001962 <== mgr.14152 v2:172.21.15.117:6800/1398160887 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5b6c062990 con 0x7f5b580a2c20 2024-06-03T03:16:32.039 INFO:teuthology.orchestra.run.smithi151.stdout:Created osd(s) 7 on host 'smithi151' 2024-06-03T03:16:32.041 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:16:32.040+0000 7f5b721dd700 1 -- 172.21.15.151:0/4261001962 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f5b580a2c20 msgr2=0x7f5b580a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:16:32.041 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:16:32.040+0000 7f5b721dd700 1 --2- 172.21.15.151:0/4261001962 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f5b580a2c20 0x7f5b580a50e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c0097b0 tx=0x7f5b5c006d20 comp rx=0 tx=0).stop 2024-06-03T03:16:32.041 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:16:32.040+0000 7f5b721dd700 1 -- 172.21.15.151:0/4261001962 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5b6c105910 msgr2=0x7f5b6c100f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:16:32.041 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:16:32.040+0000 7f5b721dd700 1 --2- 172.21.15.151:0/4261001962 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5b6c105910 0x7f5b6c100f90 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5b5400d980 tx=0x7f5b5400dc90 comp rx=0 tx=0).stop 2024-06-03T03:16:32.041 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:16:32.040+0000 7f5b721dd700 1 -- 172.21.15.151:0/4261001962 shutdown_connections 2024-06-03T03:16:32.041 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:16:32.040+0000 7f5b721dd700 1 --2- 172.21.15.151:0/4261001962 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f5b580a2c20 0x7f5b580a50e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:16:32.041 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:16:32.040+0000 7f5b721dd700 1 --2- 172.21.15.151:0/4261001962 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b6c108240 0x7f5b6c1014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:16:32.041 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:16:32.040+0000 7f5b721dd700 1 --2- 172.21.15.151:0/4261001962 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b6c102fe0 0x7f5b6c102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:16:32.041 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:16:32.040+0000 7f5b721dd700 1 --2- 172.21.15.151:0/4261001962 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5b6c105910 0x7f5b6c100f90 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:16:32.042 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:16:32.040+0000 7f5b721dd700 1 -- 172.21.15.151:0/4261001962 >> 172.21.15.151:0/4261001962 conn(0x7f5b6c0fc990 msgr2=0x7f5b6c103940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:16:32.042 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:16:32.040+0000 7f5b721dd700 1 -- 172.21.15.151:0/4261001962 shutdown_connections 2024-06-03T03:16:32.042 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:16:32.040+0000 7f5b721dd700 1 -- 172.21.15.151:0/4261001962 wait complete. 2024-06-03T03:16:32.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:31 smithi117 ceph-mon[31649]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-03T03:16:32.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:31 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:16:32.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:31 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:16:32.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:31 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:16:32.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:31 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:16:32.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:31 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:16:32.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:31 smithi183 ceph-mon[37326]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-03T03:16:32.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:31 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:16:32.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:31 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:16:32.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:31 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:16:32.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:31 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:16:32.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:31 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:16:32.636 DEBUG:teuthology.orchestra.run.smithi151:osd.7> sudo journalctl -f -n 0 -u ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@osd.7.service 2024-06-03T03:16:32.643 INFO:tasks.cephadm:Deploying osd.8 on smithi183 with /dev/vg_nvme/lv_2... 2024-06-03T03:16:32.643 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-06-03T03:16:32.889 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.c/config 2024-06-03T03:16:32.906 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:32 smithi151 ceph-mon[37043]: from='osd.7 [v2:172.21.15.151:6816/386094645,v1:172.21.15.151:6817/386094645]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-03T03:16:32.906 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:32 smithi151 ceph-mon[37043]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-03T03:16:32.906 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:32 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:16:32.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:32 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:16:32.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:32 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:16:32.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:32 smithi183 ceph-mon[37326]: from='osd.7 [v2:172.21.15.151:6816/386094645,v1:172.21.15.151:6817/386094645]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-03T03:16:32.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:32 smithi183 ceph-mon[37326]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-03T03:16:32.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:32 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:16:32.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:32 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:16:32.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:32 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:16:33.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:32 smithi117 ceph-mon[31649]: from='osd.7 [v2:172.21.15.151:6816/386094645,v1:172.21.15.151:6817/386094645]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-03T03:16:33.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:32 smithi117 ceph-mon[31649]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-03T03:16:33.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:32 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:16:33.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:32 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:16:33.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:32 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:16:33.975 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:33 smithi183 ceph-mon[37326]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-03T03:16:34.062 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:33 smithi183 ceph-mon[37326]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-06-03T03:16:34.062 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:33 smithi183 ceph-mon[37326]: from='osd.7 [v2:172.21.15.151:6816/386094645,v1:172.21.15.151:6817/386094645]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-06-03T03:16:34.062 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:33 smithi183 ceph-mon[37326]: osdmap e49: 8 total, 7 up, 8 in 2024-06-03T03:16:34.062 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:33 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-03T03:16:34.062 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:33 smithi183 ceph-mon[37326]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-06-03T03:16:34.158 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:33 smithi151 ceph-mon[37043]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-03T03:16:34.158 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:33 smithi151 ceph-mon[37043]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-06-03T03:16:34.158 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:33 smithi151 ceph-mon[37043]: from='osd.7 [v2:172.21.15.151:6816/386094645,v1:172.21.15.151:6817/386094645]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-06-03T03:16:34.159 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:33 smithi151 ceph-mon[37043]: osdmap e49: 8 total, 7 up, 8 in 2024-06-03T03:16:34.159 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:33 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-03T03:16:34.159 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:33 smithi151 ceph-mon[37043]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-06-03T03:16:34.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:33 smithi117 ceph-mon[31649]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-03T03:16:34.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:33 smithi117 ceph-mon[31649]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-06-03T03:16:34.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:33 smithi117 ceph-mon[31649]: from='osd.7 [v2:172.21.15.151:6816/386094645,v1:172.21.15.151:6817/386094645]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-06-03T03:16:34.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:33 smithi117 ceph-mon[31649]: osdmap e49: 8 total, 7 up, 8 in 2024-06-03T03:16:34.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:33 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-03T03:16:34.199 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:33 smithi117 ceph-mon[31649]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-06-03T03:16:34.657 INFO:journalctl@ceph.osd.7.smithi151.stdout:Jun 03 03:16:34 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-7[55035]: 2024-06-03T03:16:34.253+0000 7f8e6493d700 -1 osd.7 0 waiting for initial osdmap 2024-06-03T03:16:34.657 INFO:journalctl@ceph.osd.7.smithi151.stdout:Jun 03 03:16:34 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-7[55035]: 2024-06-03T03:16:34.264+0000 7f8e5c728700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-03T03:16:34.763 INFO:teuthology.orchestra.run.smithi183.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-06-03T03:16:34.764 INFO:teuthology.orchestra.run.smithi183.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-06-03T03:16:34.764 INFO:teuthology.orchestra.run.smithi183.stderr: stderr: 10+0 records in 2024-06-03T03:16:34.764 INFO:teuthology.orchestra.run.smithi183.stderr:10+0 records out 2024-06-03T03:16:34.764 INFO:teuthology.orchestra.run.smithi183.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0121753 s, 861 MB/s 2024-06-03T03:16:34.764 INFO:teuthology.orchestra.run.smithi183.stderr:--> Zapping successful for: 2024-06-03T03:16:35.144 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:34 smithi183 ceph-mon[37326]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-06-03T03:16:35.144 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:34 smithi183 ceph-mon[37326]: osdmap e50: 8 total, 7 up, 8 in 2024-06-03T03:16:35.144 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:34 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-03T03:16:35.144 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:34 smithi183 ceph-mon[37326]: from='osd.7 ' entity='osd.7' 2024-06-03T03:16:35.145 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:34 smithi183 ceph-mon[37326]: osd.7 [v2:172.21.15.151:6816/386094645,v1:172.21.15.151:6817/386094645] boot 2024-06-03T03:16:35.145 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:34 smithi183 ceph-mon[37326]: osdmap e51: 8 total, 8 up, 8 in 2024-06-03T03:16:35.145 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:34 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-03T03:16:35.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:34 smithi151 ceph-mon[37043]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-06-03T03:16:35.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:34 smithi151 ceph-mon[37043]: osdmap e50: 8 total, 7 up, 8 in 2024-06-03T03:16:35.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:34 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-03T03:16:35.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:34 smithi151 ceph-mon[37043]: from='osd.7 ' entity='osd.7' 2024-06-03T03:16:35.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:34 smithi151 ceph-mon[37043]: osd.7 [v2:172.21.15.151:6816/386094645,v1:172.21.15.151:6817/386094645] boot 2024-06-03T03:16:35.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:34 smithi151 ceph-mon[37043]: osdmap e51: 8 total, 8 up, 8 in 2024-06-03T03:16:35.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:34 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-03T03:16:35.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:34 smithi117 ceph-mon[31649]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-06-03T03:16:35.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:34 smithi117 ceph-mon[31649]: osdmap e50: 8 total, 7 up, 8 in 2024-06-03T03:16:35.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:34 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-03T03:16:35.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:34 smithi117 ceph-mon[31649]: from='osd.7 ' entity='osd.7' 2024-06-03T03:16:35.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:34 smithi117 ceph-mon[31649]: osd.7 [v2:172.21.15.151:6816/386094645,v1:172.21.15.151:6817/386094645] boot 2024-06-03T03:16:35.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:34 smithi117 ceph-mon[31649]: osdmap e51: 8 total, 8 up, 8 in 2024-06-03T03:16:35.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:34 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-03T03:16:35.469 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi183:vg_nvme/lv_2 2024-06-03T03:16:35.710 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.c/config 2024-06-03T03:16:36.026 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:35 smithi151 ceph-mon[37043]: purged_snaps scrub starts 2024-06-03T03:16:36.026 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:35 smithi151 ceph-mon[37043]: purged_snaps scrub ok 2024-06-03T03:16:36.026 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:35 smithi151 ceph-mon[37043]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-03T03:16:36.026 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:35 smithi151 ceph-mon[37043]: osdmap e52: 8 total, 8 up, 8 in 2024-06-03T03:16:36.035 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:35 smithi183 ceph-mon[37326]: purged_snaps scrub starts 2024-06-03T03:16:36.035 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:35 smithi183 ceph-mon[37326]: purged_snaps scrub ok 2024-06-03T03:16:36.035 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:35 smithi183 ceph-mon[37326]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-03T03:16:36.035 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:35 smithi183 ceph-mon[37326]: osdmap e52: 8 total, 8 up, 8 in 2024-06-03T03:16:36.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:35 smithi117 ceph-mon[31649]: purged_snaps scrub starts 2024-06-03T03:16:36.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:35 smithi117 ceph-mon[31649]: purged_snaps scrub ok 2024-06-03T03:16:36.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:35 smithi117 ceph-mon[31649]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-03T03:16:36.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:35 smithi117 ceph-mon[31649]: osdmap e52: 8 total, 8 up, 8 in 2024-06-03T03:16:37.282 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:16:37.278+0000 7fe3f3409700 1 -- 172.21.15.183:0/3781540770 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe3ec104090 msgr2=0x7fe3ec10a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:16:37.282 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:16:37.278+0000 7fe3f3409700 1 --2- 172.21.15.183:0/3781540770 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe3ec104090 0x7fe3ec10a940 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe3e400b0a0 tx=0x7fe3e400b3b0 comp rx=0 tx=0).stop 2024-06-03T03:16:37.282 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:16:37.278+0000 7fe3f3409700 1 -- 172.21.15.183:0/3781540770 shutdown_connections 2024-06-03T03:16:37.282 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:16:37.278+0000 7fe3f3409700 1 --2- 172.21.15.183:0/3781540770 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe3ec104090 0x7fe3ec10a940 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:16:37.282 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:16:37.278+0000 7fe3f3409700 1 --2- 172.21.15.183:0/3781540770 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3ec1024f0 0x7fe3ec102910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:16:37.282 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:16:37.278+0000 7fe3f3409700 1 --2- 172.21.15.183:0/3781540770 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe3ec1036f0 0x7fe3ec103b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:16:37.282 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:16:37.278+0000 7fe3f3409700 1 -- 172.21.15.183:0/3781540770 >> 172.21.15.183:0/3781540770 conn(0x7fe3ec0fda90 msgr2=0x7fe3ec0ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:16:37.283 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:16:37.279+0000 7fe3f3409700 1 -- 172.21.15.183:0/3781540770 shutdown_connections 2024-06-03T03:16:37.283 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:16:37.279+0000 7fe3f3409700 1 -- 172.21.15.183:0/3781540770 wait complete. 2024-06-03T03:16:37.283 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:16:37.279+0000 7fe3f3409700 1 Processor -- start 2024-06-03T03:16:37.283 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:16:37.279+0000 7fe3f3409700 1 -- start start 2024-06-03T03:16:37.283 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:16:37.279+0000 7fe3f3409700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3ec1024f0 0x7fe3ec198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:16:37.283 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:16:37.279+0000 7fe3f3409700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe3ec1036f0 0x7fe3ec198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:16:37.283 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:16:37.279+0000 7fe3f3409700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe3ec104090 0x7fe3ec19fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:16:37.283 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:16:37.279+0000 7fe3f3409700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe3ec10e160 con 0x7fe3ec1024f0 2024-06-03T03:16:37.283 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:16:37.279+0000 7fe3f3409700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe3ec10dfe0 con 0x7fe3ec104090 2024-06-03T03:16:37.283 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:16:37.279+0000 7fe3f3409700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe3ec10e2e0 con 0x7fe3ec1036f0 2024-06-03T03:16:37.283 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:16:37.279+0000 7fe3f09a4700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe3ec1036f0 0x7fe3ec198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:16:37.283 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:16:37.279+0000 7fe3f09a4700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe3ec1036f0 0x7fe3ec198c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:42414/0 (socket says 172.21.15.183:42414) 2024-06-03T03:16:37.283 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:16:37.279+0000 7fe3f09a4700 1 -- 172.21.15.183:0/3383197738 learned_addr learned my addr 172.21.15.183:0/3383197738 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-06-03T03:16:37.284 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:16:37.279+0000 7fe3f19a6700 1 --2- 172.21.15.183:0/3383197738 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe3ec104090 0x7fe3ec19fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:16:37.284 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:16:37.280+0000 7fe3f09a4700 1 -- 172.21.15.183:0/3383197738 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe3ec104090 msgr2=0x7fe3ec19fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:16:37.284 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:16:37.280+0000 7fe3f09a4700 1 --2- 172.21.15.183:0/3383197738 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe3ec104090 0x7fe3ec19fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:16:37.284 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:16:37.280+0000 7fe3f09a4700 1 -- 172.21.15.183:0/3383197738 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3ec1024f0 msgr2=0x7fe3ec198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:16:37.284 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:16:37.280+0000 7fe3f09a4700 1 --2- 172.21.15.183:0/3383197738 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3ec1024f0 0x7fe3ec198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:16:37.284 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:16:37.280+0000 7fe3f09a4700 1 -- 172.21.15.183:0/3383197738 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3ec1a0370 con 0x7fe3ec1036f0 2024-06-03T03:16:37.284 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:16:37.280+0000 7fe3f09a4700 1 --2- 172.21.15.183:0/3383197738 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe3ec1036f0 0x7fe3ec198c50 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe3e800d980 tx=0x7fe3e800dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:16:37.284 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:16:37.280+0000 7fe3e27fc700 1 -- 172.21.15.183:0/3383197738 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3e8016070 con 0x7fe3ec1036f0 2024-06-03T03:16:37.284 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:16:37.280+0000 7fe3e27fc700 1 -- 172.21.15.183:0/3383197738 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3e800de70 con 0x7fe3ec1036f0 2024-06-03T03:16:37.284 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:16:37.280+0000 7fe3f3409700 1 -- 172.21.15.183:0/3383197738 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3ec1a0660 con 0x7fe3ec1036f0 2024-06-03T03:16:37.284 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:16:37.280+0000 7fe3e27fc700 1 -- 172.21.15.183:0/3383197738 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3e800f3c0 con 0x7fe3ec1036f0 2024-06-03T03:16:37.284 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:16:37.280+0000 7fe3f3409700 1 -- 172.21.15.183:0/3383197738 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3ec1a0c20 con 0x7fe3ec1036f0 2024-06-03T03:16:37.285 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:16:37.281+0000 7fe3f3409700 1 -- 172.21.15.183:0/3383197738 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3ec069180 con 0x7fe3ec1036f0 2024-06-03T03:16:37.285 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:16:37.282+0000 7fe3e27fc700 1 -- 172.21.15.183:0/3383197738 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fe3e8019070 con 0x7fe3ec1036f0 2024-06-03T03:16:37.285 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:16:37.283+0000 7fe3e27fc700 1 --2- 172.21.15.183:0/3383197738 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fe3d80a2c20 0x7fe3d80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:16:37.285 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:16:37.283+0000 7fe3e27fc700 1 -- 172.21.15.183:0/3383197738 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5426+0+0 (secure 0 0 0) 0x7fe3e80ce090 con 0x7fe3ec1036f0 2024-06-03T03:16:37.285 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:16:37.283+0000 7fe3f11a5700 1 --2- 172.21.15.183:0/3383197738 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fe3d80a2c20 0x7fe3d80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:16:37.286 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:16:37.284+0000 7fe3f11a5700 1 --2- 172.21.15.183:0/3383197738 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fe3d80a2c20 0x7fe3d80a50e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec103550 tx=0x7fe3dc006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:16:37.288 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:16:37.287+0000 7fe3e27fc700 1 -- 172.21.15.183:0/3383197738 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe3e809bb30 con 0x7fe3ec1036f0 2024-06-03T03:16:37.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:37 smithi151 ceph-mon[37043]: Detected new or changed devices on smithi151 2024-06-03T03:16:37.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:37 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:16:37.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:37 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:16:37.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:37 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:16:37.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:37 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:16:37.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:37 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:16:37.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:37 smithi151 ceph-mon[37043]: Adjusting osd_memory_target on smithi151 to 5696M 2024-06-03T03:16:37.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:37 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:16:37.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:37 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:16:37.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:37 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:16:37.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:37 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:16:37.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:37 smithi151 ceph-mon[37043]: osdmap e53: 8 total, 8 up, 8 in 2024-06-03T03:16:37.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:37 smithi151 ceph-mon[37043]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-06-03T03:16:37.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:37 smithi117 ceph-mon[31649]: Detected new or changed devices on smithi151 2024-06-03T03:16:37.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:37 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:16:37.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:37 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:16:37.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:37 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:16:37.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:37 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:16:37.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:37 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:16:37.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:37 smithi117 ceph-mon[31649]: Adjusting osd_memory_target on smithi151 to 5696M 2024-06-03T03:16:37.448 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:37 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:16:37.448 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:37 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:16:37.448 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:37 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:16:37.448 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:37 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:16:37.448 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:37 smithi117 ceph-mon[31649]: osdmap e53: 8 total, 8 up, 8 in 2024-06-03T03:16:37.448 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:37 smithi117 ceph-mon[31649]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-06-03T03:16:37.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:37 smithi183 ceph-mon[37326]: Detected new or changed devices on smithi151 2024-06-03T03:16:37.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:37 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:16:37.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:37 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:16:37.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:37 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:16:37.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:37 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:16:37.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:37 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:16:37.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:37 smithi183 ceph-mon[37326]: Adjusting osd_memory_target on smithi151 to 5696M 2024-06-03T03:16:37.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:37 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:16:37.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:37 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:16:37.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:37 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:16:37.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:37 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:16:37.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:37 smithi183 ceph-mon[37326]: osdmap e53: 8 total, 8 up, 8 in 2024-06-03T03:16:37.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:37 smithi183 ceph-mon[37326]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-06-03T03:16:37.496 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:16:37.495+0000 7fe3f3409700 1 -- 172.21.15.183:0/3383197738 --> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fe3ec0628d0 con 0x7fe3d80a2c20 2024-06-03T03:16:38.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:38 smithi183 ceph-mon[37326]: osdmap e54: 8 total, 8 up, 8 in 2024-06-03T03:16:38.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:38 smithi183 ceph-mon[37326]: from='client.24322 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:16:38.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:38 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T03:16:38.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:38 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T03:16:38.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:38 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:16:38.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:38 smithi151 ceph-mon[37043]: osdmap e54: 8 total, 8 up, 8 in 2024-06-03T03:16:38.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:38 smithi151 ceph-mon[37043]: from='client.24322 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:16:38.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:38 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T03:16:38.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:38 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T03:16:38.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:38 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:16:38.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:38 smithi117 ceph-mon[31649]: osdmap e54: 8 total, 8 up, 8 in 2024-06-03T03:16:38.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:38 smithi117 ceph-mon[31649]: from='client.24322 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:16:38.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:38 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T03:16:38.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:38 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T03:16:38.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:38 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:16:39.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:39 smithi183 ceph-mon[37326]: osdmap e55: 8 total, 8 up, 8 in 2024-06-03T03:16:39.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:39 smithi183 ceph-mon[37326]: pgmap v202: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-06-03T03:16:39.906 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:39 smithi151 ceph-mon[37043]: osdmap e55: 8 total, 8 up, 8 in 2024-06-03T03:16:39.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:39 smithi151 ceph-mon[37043]: pgmap v202: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-06-03T03:16:39.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:39 smithi117 ceph-mon[31649]: osdmap e55: 8 total, 8 up, 8 in 2024-06-03T03:16:39.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:39 smithi117 ceph-mon[31649]: pgmap v202: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-06-03T03:16:40.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:40 smithi183 ceph-mon[37326]: from='client.? 172.21.15.183:0/3089265870' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "006ac092-f015-448e-a0c0-f7b0c4597a18"}]: dispatch 2024-06-03T03:16:40.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:40 smithi183 ceph-mon[37326]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "006ac092-f015-448e-a0c0-f7b0c4597a18"}]: dispatch 2024-06-03T03:16:40.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:40 smithi183 ceph-mon[37326]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "006ac092-f015-448e-a0c0-f7b0c4597a18"}]': finished 2024-06-03T03:16:40.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:40 smithi183 ceph-mon[37326]: osdmap e56: 9 total, 8 up, 9 in 2024-06-03T03:16:40.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:40 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-03T03:16:40.906 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:40 smithi151 ceph-mon[37043]: from='client.? 172.21.15.183:0/3089265870' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "006ac092-f015-448e-a0c0-f7b0c4597a18"}]: dispatch 2024-06-03T03:16:40.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:40 smithi151 ceph-mon[37043]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "006ac092-f015-448e-a0c0-f7b0c4597a18"}]: dispatch 2024-06-03T03:16:40.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:40 smithi151 ceph-mon[37043]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "006ac092-f015-448e-a0c0-f7b0c4597a18"}]': finished 2024-06-03T03:16:40.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:40 smithi151 ceph-mon[37043]: osdmap e56: 9 total, 8 up, 9 in 2024-06-03T03:16:40.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:40 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-03T03:16:40.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:40 smithi117 ceph-mon[31649]: from='client.? 172.21.15.183:0/3089265870' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "006ac092-f015-448e-a0c0-f7b0c4597a18"}]: dispatch 2024-06-03T03:16:40.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:40 smithi117 ceph-mon[31649]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "006ac092-f015-448e-a0c0-f7b0c4597a18"}]: dispatch 2024-06-03T03:16:40.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:40 smithi117 ceph-mon[31649]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "006ac092-f015-448e-a0c0-f7b0c4597a18"}]': finished 2024-06-03T03:16:40.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:40 smithi117 ceph-mon[31649]: osdmap e56: 9 total, 8 up, 9 in 2024-06-03T03:16:40.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:40 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-03T03:16:41.803 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:41 smithi117 ceph-mon[31649]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-06-03T03:16:41.803 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:41 smithi117 ceph-mon[31649]: from='client.? 172.21.15.183:0/1275427605' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T03:16:41.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:41 smithi151 ceph-mon[37043]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-06-03T03:16:41.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:41 smithi151 ceph-mon[37043]: from='client.? 172.21.15.183:0/1275427605' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T03:16:41.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:41 smithi183 ceph-mon[37326]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-06-03T03:16:41.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:41 smithi183 ceph-mon[37326]: from='client.? 172.21.15.183:0/1275427605' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T03:16:44.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:43 smithi151 ceph-mon[37043]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-06-03T03:16:44.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:43 smithi117 ceph-mon[31649]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-06-03T03:16:44.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:43 smithi183 ceph-mon[37326]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-06-03T03:16:45.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:45 smithi183 ceph-mon[37326]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-06-03T03:16:46.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:45 smithi151 ceph-mon[37043]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-06-03T03:16:46.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:45 smithi117 ceph-mon[31649]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-06-03T03:16:48.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:47 smithi151 ceph-mon[37043]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-06-03T03:16:48.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:47 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-06-03T03:16:48.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:47 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:16:48.176 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:47 smithi183 ceph-mon[37326]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-06-03T03:16:48.177 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:47 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-06-03T03:16:48.177 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:47 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:16:48.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:47 smithi117 ceph-mon[31649]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-06-03T03:16:48.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:47 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-06-03T03:16:48.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:47 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:16:49.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:48 smithi151 ceph-mon[37043]: Deploying daemon osd.8 on smithi183 2024-06-03T03:16:49.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:48 smithi117 ceph-mon[31649]: Deploying daemon osd.8 on smithi183 2024-06-03T03:16:49.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:48 smithi183 ceph-mon[37326]: Deploying daemon osd.8 on smithi183 2024-06-03T03:16:49.937 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:49 smithi183 ceph-mon[37326]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-06-03T03:16:50.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:49 smithi151 ceph-mon[37043]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-06-03T03:16:50.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:49 smithi117 ceph-mon[31649]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-06-03T03:16:52.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:51 smithi151 ceph-mon[37043]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-06-03T03:16:52.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:51 smithi117 ceph-mon[31649]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-06-03T03:16:52.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:51 smithi183 ceph-mon[37326]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-06-03T03:16:53.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:53 smithi183 ceph-mon[37326]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-06-03T03:16:53.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:53 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:16:53.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:53 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:16:53.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:53 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:16:54.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:53 smithi151 ceph-mon[37043]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-06-03T03:16:54.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:53 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:16:54.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:53 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:16:54.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:53 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:16:54.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:53 smithi117 ceph-mon[31649]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-06-03T03:16:54.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:53 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:16:54.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:53 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:16:54.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:53 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:16:56.124 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:55 smithi183 ceph-mon[37326]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-06-03T03:16:56.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:55 smithi151 ceph-mon[37043]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-06-03T03:16:56.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:55 smithi117 ceph-mon[31649]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-06-03T03:16:56.879 INFO:teuthology.orchestra.run.smithi183.stdout:Created osd(s) 8 on host 'smithi183' 2024-06-03T03:16:56.879 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:16:56.877+0000 7fe3e27fc700 1 -- 172.21.15.183:0/3383197738 <== mgr.14152 v2:172.21.15.117:6800/1398160887 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe3ec0628d0 con 0x7fe3d80a2c20 2024-06-03T03:16:56.881 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:16:56.880+0000 7fe3f3409700 1 -- 172.21.15.183:0/3383197738 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fe3d80a2c20 msgr2=0x7fe3d80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:16:56.881 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:16:56.880+0000 7fe3f3409700 1 --2- 172.21.15.183:0/3383197738 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fe3d80a2c20 0x7fe3d80a50e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec103550 tx=0x7fe3dc006d20 comp rx=0 tx=0).stop 2024-06-03T03:16:56.881 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:16:56.880+0000 7fe3f3409700 1 -- 172.21.15.183:0/3383197738 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe3ec1036f0 msgr2=0x7fe3ec198c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:16:56.881 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:16:56.880+0000 7fe3f3409700 1 --2- 172.21.15.183:0/3383197738 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe3ec1036f0 0x7fe3ec198c50 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe3e800d980 tx=0x7fe3e800dc90 comp rx=0 tx=0).stop 2024-06-03T03:16:56.882 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:16:56.880+0000 7fe3f3409700 1 -- 172.21.15.183:0/3383197738 shutdown_connections 2024-06-03T03:16:56.882 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:16:56.880+0000 7fe3f3409700 1 --2- 172.21.15.183:0/3383197738 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fe3d80a2c20 0x7fe3d80a50e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:16:56.882 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:16:56.880+0000 7fe3f3409700 1 --2- 172.21.15.183:0/3383197738 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe3ec104090 0x7fe3ec19fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:16:56.882 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:16:56.880+0000 7fe3f3409700 1 --2- 172.21.15.183:0/3383197738 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3ec1024f0 0x7fe3ec198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:16:56.882 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:16:56.880+0000 7fe3f3409700 1 --2- 172.21.15.183:0/3383197738 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe3ec1036f0 0x7fe3ec198c50 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:16:56.882 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:16:56.880+0000 7fe3f3409700 1 -- 172.21.15.183:0/3383197738 >> 172.21.15.183:0/3383197738 conn(0x7fe3ec0fda90 msgr2=0x7fe3ec106990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:16:56.882 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:16:56.880+0000 7fe3f3409700 1 -- 172.21.15.183:0/3383197738 shutdown_connections 2024-06-03T03:16:56.882 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:16:56.880+0000 7fe3f3409700 1 -- 172.21.15.183:0/3383197738 wait complete. 2024-06-03T03:16:57.583 DEBUG:teuthology.orchestra.run.smithi183:osd.8> sudo journalctl -f -n 0 -u ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@osd.8.service 2024-06-03T03:16:57.586 INFO:tasks.cephadm:Deploying osd.9 on smithi117 with /dev/vg_nvme/lv_1... 2024-06-03T03:16:57.586 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-06-03T03:16:57.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:57 smithi117 ceph-mon[31649]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-06-03T03:16:57.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:57 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:16:57.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:57 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:16:57.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:57 smithi117 ceph-mon[31649]: from='osd.8 [v2:172.21.15.183:6816/1829748222,v1:172.21.15.183:6817/1829748222]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-03T03:16:57.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:57 smithi117 ceph-mon[31649]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-03T03:16:57.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:57 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:16:57.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:57 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:16:58.023 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:16:58.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:57 smithi151 ceph-mon[37043]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-06-03T03:16:58.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:57 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:16:58.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:57 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:16:58.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:57 smithi151 ceph-mon[37043]: from='osd.8 [v2:172.21.15.183:6816/1829748222,v1:172.21.15.183:6817/1829748222]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-03T03:16:58.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:57 smithi151 ceph-mon[37043]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-03T03:16:58.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:57 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:16:58.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:57 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:16:58.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:57 smithi183 ceph-mon[37326]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-06-03T03:16:58.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:57 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:16:58.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:57 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:16:58.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:57 smithi183 ceph-mon[37326]: from='osd.8 [v2:172.21.15.183:6816/1829748222,v1:172.21.15.183:6817/1829748222]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-03T03:16:58.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:57 smithi183 ceph-mon[37326]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-03T03:16:58.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:57 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:16:58.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:57 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:16:59.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:58 smithi151 ceph-mon[37043]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-06-03T03:16:59.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:58 smithi151 ceph-mon[37043]: from='osd.8 [v2:172.21.15.183:6816/1829748222,v1:172.21.15.183:6817/1829748222]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-06-03T03:16:59.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:58 smithi151 ceph-mon[37043]: osdmap e57: 9 total, 8 up, 9 in 2024-06-03T03:16:59.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:58 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-03T03:16:59.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:58 smithi151 ceph-mon[37043]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-06-03T03:16:59.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:58 smithi117 ceph-mon[31649]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-06-03T03:16:59.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:58 smithi117 ceph-mon[31649]: from='osd.8 [v2:172.21.15.183:6816/1829748222,v1:172.21.15.183:6817/1829748222]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-06-03T03:16:59.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:58 smithi117 ceph-mon[31649]: osdmap e57: 9 total, 8 up, 9 in 2024-06-03T03:16:59.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:58 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-03T03:16:59.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:58 smithi117 ceph-mon[31649]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-06-03T03:16:59.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:58 smithi183 ceph-mon[37326]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-06-03T03:16:59.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:58 smithi183 ceph-mon[37326]: from='osd.8 [v2:172.21.15.183:6816/1829748222,v1:172.21.15.183:6817/1829748222]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-06-03T03:16:59.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:58 smithi183 ceph-mon[37326]: osdmap e57: 9 total, 8 up, 9 in 2024-06-03T03:16:59.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:58 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-03T03:16:59.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:58 smithi183 ceph-mon[37326]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-06-03T03:16:59.575 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:16:59 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:16:59.287+0000 7f4168e7c700 -1 osd.8 0 waiting for initial osdmap 2024-06-03T03:16:59.575 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:16:59 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:16:59.299+0000 7f4160466700 -1 osd.8 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-03T03:17:00.101 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:59 smithi117 ceph-mon[31649]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-06-03T03:17:00.101 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:59 smithi117 ceph-mon[31649]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-06-03T03:17:00.101 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:59 smithi117 ceph-mon[31649]: osdmap e58: 9 total, 8 up, 9 in 2024-06-03T03:17:00.101 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:59 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-03T03:17:00.102 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:59 smithi117 ceph-mon[31649]: from='osd.8 ' entity='osd.8' 2024-06-03T03:17:00.102 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:59 smithi117 ceph-mon[31649]: osd.8 [v2:172.21.15.183:6816/1829748222,v1:172.21.15.183:6817/1829748222] boot 2024-06-03T03:17:00.102 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:59 smithi117 ceph-mon[31649]: osdmap e59: 9 total, 9 up, 9 in 2024-06-03T03:17:00.102 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:16:59 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-03T03:17:00.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:59 smithi151 ceph-mon[37043]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-06-03T03:17:00.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:59 smithi151 ceph-mon[37043]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-06-03T03:17:00.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:59 smithi151 ceph-mon[37043]: osdmap e58: 9 total, 8 up, 9 in 2024-06-03T03:17:00.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:59 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-03T03:17:00.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:59 smithi151 ceph-mon[37043]: from='osd.8 ' entity='osd.8' 2024-06-03T03:17:00.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:59 smithi151 ceph-mon[37043]: osd.8 [v2:172.21.15.183:6816/1829748222,v1:172.21.15.183:6817/1829748222] boot 2024-06-03T03:17:00.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:59 smithi151 ceph-mon[37043]: osdmap e59: 9 total, 9 up, 9 in 2024-06-03T03:17:00.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:16:59 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-03T03:17:00.231 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:59 smithi183 ceph-mon[37326]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-06-03T03:17:00.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:59 smithi183 ceph-mon[37326]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-06-03T03:17:00.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:59 smithi183 ceph-mon[37326]: osdmap e58: 9 total, 8 up, 9 in 2024-06-03T03:17:00.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:59 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-03T03:17:00.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:59 smithi183 ceph-mon[37326]: from='osd.8 ' entity='osd.8' 2024-06-03T03:17:00.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:59 smithi183 ceph-mon[37326]: osd.8 [v2:172.21.15.183:6816/1829748222,v1:172.21.15.183:6817/1829748222] boot 2024-06-03T03:17:00.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:59 smithi183 ceph-mon[37326]: osdmap e59: 9 total, 9 up, 9 in 2024-06-03T03:17:00.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:16:59 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-03T03:17:01.142 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:00 smithi117 ceph-mon[31649]: purged_snaps scrub starts 2024-06-03T03:17:01.142 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:00 smithi117 ceph-mon[31649]: purged_snaps scrub ok 2024-06-03T03:17:01.142 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:00 smithi117 ceph-mon[31649]: osdmap e60: 9 total, 9 up, 9 in 2024-06-03T03:17:01.142 INFO:teuthology.orchestra.run.smithi117.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-06-03T03:17:01.142 INFO:teuthology.orchestra.run.smithi117.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-06-03T03:17:01.142 INFO:teuthology.orchestra.run.smithi117.stderr: stderr: 10+0 records in 2024-06-03T03:17:01.142 INFO:teuthology.orchestra.run.smithi117.stderr:10+0 records out 2024-06-03T03:17:01.142 INFO:teuthology.orchestra.run.smithi117.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0326769 s, 321 MB/s 2024-06-03T03:17:01.143 INFO:teuthology.orchestra.run.smithi117.stderr:--> Zapping successful for: 2024-06-03T03:17:01.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:00 smithi151 ceph-mon[37043]: purged_snaps scrub starts 2024-06-03T03:17:01.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:00 smithi151 ceph-mon[37043]: purged_snaps scrub ok 2024-06-03T03:17:01.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:00 smithi151 ceph-mon[37043]: osdmap e60: 9 total, 9 up, 9 in 2024-06-03T03:17:01.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:00 smithi183 ceph-mon[37326]: purged_snaps scrub starts 2024-06-03T03:17:01.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:00 smithi183 ceph-mon[37326]: purged_snaps scrub ok 2024-06-03T03:17:01.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:00 smithi183 ceph-mon[37326]: osdmap e60: 9 total, 9 up, 9 in 2024-06-03T03:17:02.058 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi117:vg_nvme/lv_1 2024-06-03T03:17:02.135 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:01 smithi117 ceph-mon[31649]: pgmap v218: 1 pgs: 1 remapped+peering; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail 2024-06-03T03:17:02.135 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:01 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:17:02.135 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:01 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:17:02.135 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:01 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:17:02.135 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:01 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:17:02.135 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:01 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:17:02.135 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:01 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:17:02.135 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:01 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:17:02.136 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:01 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:17:02.136 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:01 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:17:02.136 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:01 smithi117 ceph-mon[31649]: osdmap e61: 9 total, 9 up, 9 in 2024-06-03T03:17:02.136 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:01 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:17:02.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:01 smithi151 ceph-mon[37043]: pgmap v218: 1 pgs: 1 remapped+peering; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail 2024-06-03T03:17:02.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:01 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:17:02.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:01 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:17:02.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:01 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:17:02.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:01 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:17:02.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:01 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:17:02.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:01 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:17:02.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:01 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:17:02.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:01 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:17:02.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:01 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:17:02.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:01 smithi151 ceph-mon[37043]: osdmap e61: 9 total, 9 up, 9 in 2024-06-03T03:17:02.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:01 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:17:02.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:01 smithi183 ceph-mon[37326]: pgmap v218: 1 pgs: 1 remapped+peering; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail 2024-06-03T03:17:02.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:01 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:17:02.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:01 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:17:02.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:01 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:17:02.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:01 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:17:02.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:01 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:17:02.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:01 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:17:02.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:01 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:17:02.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:01 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:17:02.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:01 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:17:02.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:01 smithi183 ceph-mon[37326]: osdmap e61: 9 total, 9 up, 9 in 2024-06-03T03:17:02.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:01 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:17:02.504 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:17:02.913 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:02 smithi117 ceph-mon[31649]: Detected new or changed devices on smithi183 2024-06-03T03:17:02.932 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:02 smithi117 ceph-mon[31649]: Adjusting osd_memory_target on smithi183 to 5696M 2024-06-03T03:17:02.933 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:02 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:17:02.933 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:02 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:17:02.933 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:02 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:17:02.933 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:02 smithi117 ceph-mon[31649]: osdmap e62: 9 total, 9 up, 9 in 2024-06-03T03:17:03.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:02 smithi151 ceph-mon[37043]: Detected new or changed devices on smithi183 2024-06-03T03:17:03.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:02 smithi151 ceph-mon[37043]: Adjusting osd_memory_target on smithi183 to 5696M 2024-06-03T03:17:03.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:02 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:17:03.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:02 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:17:03.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:02 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:17:03.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:02 smithi151 ceph-mon[37043]: osdmap e62: 9 total, 9 up, 9 in 2024-06-03T03:17:03.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:02 smithi183 ceph-mon[37326]: Detected new or changed devices on smithi183 2024-06-03T03:17:03.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:02 smithi183 ceph-mon[37326]: Adjusting osd_memory_target on smithi183 to 5696M 2024-06-03T03:17:03.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:02 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:17:03.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:02 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:17:03.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:02 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:17:03.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:02 smithi183 ceph-mon[37326]: osdmap e62: 9 total, 9 up, 9 in 2024-06-03T03:17:04.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:03 smithi151 ceph-mon[37043]: pgmap v221: 1 pgs: 1 remapped+peering; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail 2024-06-03T03:17:04.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:03 smithi151 ceph-mon[37043]: osdmap e63: 9 total, 9 up, 9 in 2024-06-03T03:17:04.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:03 smithi117 ceph-mon[31649]: pgmap v221: 1 pgs: 1 remapped+peering; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail 2024-06-03T03:17:04.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:03 smithi117 ceph-mon[31649]: osdmap e63: 9 total, 9 up, 9 in 2024-06-03T03:17:04.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:03 smithi183 ceph-mon[37326]: pgmap v221: 1 pgs: 1 remapped+peering; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail 2024-06-03T03:17:04.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:03 smithi183 ceph-mon[37326]: osdmap e63: 9 total, 9 up, 9 in 2024-06-03T03:17:04.888 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:17:04.885+0000 7f54c2e00700 1 -- 172.21.15.117:0/989444590 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f54bc074ef0 msgr2=0x7f54bc075370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:17:04.889 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:17:04.885+0000 7f54c2e00700 1 --2- 172.21.15.117:0/989444590 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f54bc074ef0 0x7f54bc075370 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f54a8009990 tx=0x7f54a8009ca0 comp rx=0 tx=0).stop 2024-06-03T03:17:04.889 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:17:04.885+0000 7f54c2e00700 1 -- 172.21.15.117:0/989444590 shutdown_connections 2024-06-03T03:17:04.889 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:17:04.885+0000 7f54c2e00700 1 --2- 172.21.15.117:0/989444590 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54bc10ea00 0x7f54bc110df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:17:04.889 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:17:04.885+0000 7f54c2e00700 1 --2- 172.21.15.117:0/989444590 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54bc0764c0 0x7f54bc074920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:17:04.890 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:17:04.885+0000 7f54c2e00700 1 --2- 172.21.15.117:0/989444590 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f54bc074ef0 0x7f54bc075370 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:17:04.890 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:17:04.885+0000 7f54c2e00700 1 -- 172.21.15.117:0/989444590 >> 172.21.15.117:0/989444590 conn(0x7f54bc0fd9c0 msgr2=0x7f54bc0ffe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:17:04.890 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:17:04.885+0000 7f54c2e00700 1 -- 172.21.15.117:0/989444590 shutdown_connections 2024-06-03T03:17:04.890 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:17:04.885+0000 7f54c2e00700 1 -- 172.21.15.117:0/989444590 wait complete. 2024-06-03T03:17:04.890 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:17:04.887+0000 7f54c2e00700 1 Processor -- start 2024-06-03T03:17:04.890 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:17:04.887+0000 7f54c2e00700 1 -- start start 2024-06-03T03:17:04.892 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:17:04.887+0000 7f54c2e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f54bc074ef0 0x7f54bc19ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:17:04.892 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:17:04.887+0000 7f54c2e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54bc0764c0 0x7f54bc19f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:17:04.892 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:17:04.887+0000 7f54c2e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54bc10ea00 0x7f54bc1a6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:17:04.892 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:17:04.887+0000 7f54c2e00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f54bc114670 con 0x7f54bc0764c0 2024-06-03T03:17:04.892 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:17:04.887+0000 7f54c2e00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f54bc1144f0 con 0x7f54bc074ef0 2024-06-03T03:17:04.892 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:17:04.887+0000 7f54c2e00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f54bc1147f0 con 0x7f54bc10ea00 2024-06-03T03:17:04.893 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:17:04.887+0000 7f54c1000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54bc0764c0 0x7f54bc19f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:17:04.893 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:17:04.888+0000 7f54c1000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54bc0764c0 0x7f54bc19f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39506/0 (socket says 172.21.15.117:39506) 2024-06-03T03:17:04.893 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:17:04.888+0000 7f54c1000700 1 -- 172.21.15.117:0/3671921970 learned_addr learned my addr 172.21.15.117:0/3671921970 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:17:04.893 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:17:04.888+0000 7f54c1a00700 1 --2- 172.21.15.117:0/3671921970 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f54bc074ef0 0x7f54bc19ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:17:04.893 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:17:04.888+0000 7f54c2400700 1 --2- 172.21.15.117:0/3671921970 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54bc10ea00 0x7f54bc1a6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:17:04.893 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:17:04.888+0000 7f54c1a00700 1 -- 172.21.15.117:0/3671921970 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54bc10ea00 msgr2=0x7f54bc1a6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:17:04.893 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:17:04.888+0000 7f54c1a00700 1 --2- 172.21.15.117:0/3671921970 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54bc10ea00 0x7f54bc1a6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:17:04.893 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:17:04.888+0000 7f54c1a00700 1 -- 172.21.15.117:0/3671921970 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54bc0764c0 msgr2=0x7f54bc19f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:17:04.893 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:17:04.888+0000 7f54c1a00700 1 --2- 172.21.15.117:0/3671921970 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54bc0764c0 0x7f54bc19f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:17:04.893 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:17:04.888+0000 7f54c1a00700 1 -- 172.21.15.117:0/3671921970 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54bc1a68f0 con 0x7f54bc074ef0 2024-06-03T03:17:04.893 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:17:04.888+0000 7f54c1a00700 1 --2- 172.21.15.117:0/3671921970 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f54bc074ef0 0x7f54bc19ec70 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f54b000e9c0 tx=0x7f54b000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:17:04.893 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:17:04.889+0000 7f54ba000700 1 -- 172.21.15.117:0/3671921970 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b0004220 con 0x7f54bc074ef0 2024-06-03T03:17:04.894 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:17:04.889+0000 7f54ba000700 1 -- 172.21.15.117:0/3671921970 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54b0004d10 con 0x7f54bc074ef0 2024-06-03T03:17:04.894 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:17:04.889+0000 7f54c2e00700 1 -- 172.21.15.117:0/3671921970 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54bc1a6be0 con 0x7f54bc074ef0 2024-06-03T03:17:04.894 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:17:04.889+0000 7f54ba000700 1 -- 172.21.15.117:0/3671921970 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b0005020 con 0x7f54bc074ef0 2024-06-03T03:17:04.894 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:17:04.889+0000 7f54c2e00700 1 -- 172.21.15.117:0/3671921970 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54bc1a7120 con 0x7f54bc074ef0 2024-06-03T03:17:04.894 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:17:04.890+0000 7f54c2e00700 1 -- 172.21.15.117:0/3671921970 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54bc069180 con 0x7f54bc074ef0 2024-06-03T03:17:04.895 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:17:04.892+0000 7f54ba000700 1 -- 172.21.15.117:0/3671921970 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f54b0004750 con 0x7f54bc074ef0 2024-06-03T03:17:04.895 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:17:04.893+0000 7f54ba000700 1 --2- 172.21.15.117:0/3671921970 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f54a40a2ca0 0x7f54a40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:17:04.895 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:17:04.893+0000 7f54ba000700 1 -- 172.21.15.117:0/3671921970 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5829+0+0 (secure 0 0 0) 0x7f54b00ce580 con 0x7f54bc074ef0 2024-06-03T03:17:04.895 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:17:04.893+0000 7f54c1000700 1 --2- 172.21.15.117:0/3671921970 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f54a40a2ca0 0x7f54a40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:17:04.895 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:17:04.894+0000 7f54c1000700 1 --2- 172.21.15.117:0/3671921970 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f54a40a2ca0 0x7f54a40a5160 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f54bc1a0180 tx=0x7f54a8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:17:04.899 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:17:04.897+0000 7f54ba000700 1 -- 172.21.15.117:0/3671921970 <== mon.2 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f54b009bdf0 con 0x7f54bc074ef0 2024-06-03T03:17:05.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:17:05.138+0000 7f54c2e00700 1 -- 172.21.15.117:0/3671921970 --> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f54bc0628d0 con 0x7f54a40a2ca0 2024-06-03T03:17:06.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:05 smithi151 ceph-mon[37043]: pgmap v223: 1 pgs: 1 remapped+peering; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail 2024-06-03T03:17:06.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:05 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T03:17:06.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:05 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T03:17:06.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:05 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:17:06.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:05 smithi117 ceph-mon[31649]: pgmap v223: 1 pgs: 1 remapped+peering; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail 2024-06-03T03:17:06.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:05 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T03:17:06.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:05 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T03:17:06.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:05 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:17:06.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:05 smithi183 ceph-mon[37326]: pgmap v223: 1 pgs: 1 remapped+peering; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail 2024-06-03T03:17:06.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:05 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T03:17:06.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:05 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T03:17:06.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:05 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:17:07.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:06 smithi151 ceph-mon[37043]: from='client.24329 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:17:07.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:06 smithi117 ceph-mon[31649]: from='client.24329 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:17:07.231 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:06 smithi183 ceph-mon[37326]: from='client.24329 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:17:08.029 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:07 smithi117 ceph-mon[31649]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-03T03:17:08.148 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:07 smithi183 ceph-mon[37326]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-03T03:17:08.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:07 smithi151 ceph-mon[37043]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-03T03:17:09.923 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:09 smithi117 ceph-mon[31649]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-06-03T03:17:09.924 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:09 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/2877290006' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "00a970e6-2502-459e-975c-badac93c9bd0"}]: dispatch 2024-06-03T03:17:09.924 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:09 smithi117 ceph-mon[31649]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "00a970e6-2502-459e-975c-badac93c9bd0"}]: dispatch 2024-06-03T03:17:09.924 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:09 smithi117 ceph-mon[31649]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "00a970e6-2502-459e-975c-badac93c9bd0"}]': finished 2024-06-03T03:17:09.924 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:09 smithi117 ceph-mon[31649]: osdmap e64: 10 total, 9 up, 10 in 2024-06-03T03:17:10.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:09 smithi151 ceph-mon[37043]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-06-03T03:17:10.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:09 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/2877290006' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "00a970e6-2502-459e-975c-badac93c9bd0"}]: dispatch 2024-06-03T03:17:10.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:09 smithi151 ceph-mon[37043]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "00a970e6-2502-459e-975c-badac93c9bd0"}]: dispatch 2024-06-03T03:17:10.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:09 smithi151 ceph-mon[37043]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "00a970e6-2502-459e-975c-badac93c9bd0"}]': finished 2024-06-03T03:17:10.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:09 smithi151 ceph-mon[37043]: osdmap e64: 10 total, 9 up, 10 in 2024-06-03T03:17:10.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:09 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-03T03:17:10.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:09 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-03T03:17:10.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:09 smithi183 ceph-mon[37326]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-06-03T03:17:10.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:09 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/2877290006' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "00a970e6-2502-459e-975c-badac93c9bd0"}]: dispatch 2024-06-03T03:17:10.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:09 smithi183 ceph-mon[37326]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "00a970e6-2502-459e-975c-badac93c9bd0"}]: dispatch 2024-06-03T03:17:10.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:09 smithi183 ceph-mon[37326]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "00a970e6-2502-459e-975c-badac93c9bd0"}]': finished 2024-06-03T03:17:10.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:09 smithi183 ceph-mon[37326]: osdmap e64: 10 total, 9 up, 10 in 2024-06-03T03:17:10.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:09 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-03T03:17:11.184 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:10 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/403743941' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T03:17:11.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:10 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/403743941' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T03:17:11.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:10 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/403743941' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T03:17:12.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:11 smithi117 ceph-mon[31649]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-03T03:17:12.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:11 smithi183 ceph-mon[37326]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-03T03:17:12.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:11 smithi151 ceph-mon[37043]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-03T03:17:14.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:13 smithi117 ceph-mon[31649]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-06-03T03:17:14.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:13 smithi183 ceph-mon[37326]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-06-03T03:17:14.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:13 smithi151 ceph-mon[37043]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-06-03T03:17:16.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:15 smithi117 ceph-mon[31649]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-03T03:17:16.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:15 smithi183 ceph-mon[37326]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-03T03:17:16.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:15 smithi151 ceph-mon[37043]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-03T03:17:18.193 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:17 smithi117 ceph-mon[31649]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-06-03T03:17:18.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:17 smithi183 ceph-mon[37326]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-06-03T03:17:18.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:17 smithi151 ceph-mon[37043]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-06-03T03:17:20.192 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:19 smithi117 ceph-mon[31649]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-06-03T03:17:20.192 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:19 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-06-03T03:17:20.192 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:19 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:17:20.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:19 smithi183 ceph-mon[37326]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-06-03T03:17:20.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:19 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-06-03T03:17:20.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:19 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:17:20.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:19 smithi151 ceph-mon[37043]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-06-03T03:17:20.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:19 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-06-03T03:17:20.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:19 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:17:21.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:20 smithi117 ceph-mon[31649]: Deploying daemon osd.9 on smithi117 2024-06-03T03:17:21.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:20 smithi183 ceph-mon[37326]: Deploying daemon osd.9 on smithi117 2024-06-03T03:17:21.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:20 smithi151 ceph-mon[37043]: Deploying daemon osd.9 on smithi117 2024-06-03T03:17:22.090 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:21 smithi117 ceph-mon[31649]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-06-03T03:17:22.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:21 smithi183 ceph-mon[37326]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-06-03T03:17:22.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:21 smithi151 ceph-mon[37043]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-06-03T03:17:23.326 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:22 smithi117 ceph-mon[31649]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-06-03T03:17:23.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:23 smithi151 ceph-mon[37043]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-06-03T03:17:23.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:22 smithi183 ceph-mon[37326]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-06-03T03:17:26.082 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:25 smithi117 ceph-mon[31649]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-06-03T03:17:26.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:25 smithi151 ceph-mon[37043]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-06-03T03:17:26.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:25 smithi183 ceph-mon[37326]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-06-03T03:17:27.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:27 smithi117 ceph-mon[31649]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-06-03T03:17:28.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:27 smithi151 ceph-mon[37043]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-06-03T03:17:28.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:27 smithi183 ceph-mon[37326]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-06-03T03:17:29.157 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:28 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:17:29.157 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:28 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:17:29.157 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:28 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:17:29.157 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:28 smithi117 ceph-mon[31649]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-06-03T03:17:29.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:28 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:17:29.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:28 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:17:29.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:28 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:17:29.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:28 smithi183 ceph-mon[37326]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-06-03T03:17:29.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:28 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:17:29.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:28 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:17:29.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:28 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:17:29.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:28 smithi151 ceph-mon[37043]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-06-03T03:17:32.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:31 smithi151 ceph-mon[37043]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-06-03T03:17:32.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:31 smithi117 ceph-mon[31649]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-06-03T03:17:32.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:31 smithi183 ceph-mon[37326]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-06-03T03:17:33.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:32 smithi151 ceph-mon[37043]: from='osd.9 [v2:172.21.15.117:6826/2538315141,v1:172.21.15.117:6827/2538315141]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-06-03T03:17:33.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:32 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:17:33.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:32 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:17:33.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:32 smithi117 ceph-mon[31649]: from='osd.9 [v2:172.21.15.117:6826/2538315141,v1:172.21.15.117:6827/2538315141]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-06-03T03:17:33.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:32 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:17:33.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:32 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:17:33.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:32 smithi183 ceph-mon[37326]: from='osd.9 [v2:172.21.15.117:6826/2538315141,v1:172.21.15.117:6827/2538315141]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-06-03T03:17:33.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:32 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:17:33.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:32 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:17:33.492 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:17:33.490+0000 7f54ba000700 1 -- 172.21.15.117:0/3671921970 <== mgr.14152 v2:172.21.15.117:6800/1398160887 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f54bc0628d0 con 0x7f54a40a2ca0 2024-06-03T03:17:33.492 INFO:teuthology.orchestra.run.smithi117.stdout:Created osd(s) 9 on host 'smithi117' 2024-06-03T03:17:33.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:17:33.494+0000 7f54c2e00700 1 -- 172.21.15.117:0/3671921970 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f54a40a2ca0 msgr2=0x7f54a40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:17:33.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:17:33.494+0000 7f54c2e00700 1 --2- 172.21.15.117:0/3671921970 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f54a40a2ca0 0x7f54a40a5160 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f54bc1a0180 tx=0x7f54a8005fb0 comp rx=0 tx=0).stop 2024-06-03T03:17:33.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:17:33.494+0000 7f54c2e00700 1 -- 172.21.15.117:0/3671921970 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f54bc074ef0 msgr2=0x7f54bc19ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:17:33.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:17:33.494+0000 7f54c2e00700 1 --2- 172.21.15.117:0/3671921970 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f54bc074ef0 0x7f54bc19ec70 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f54b000e9c0 tx=0x7f54b000ecd0 comp rx=0 tx=0).stop 2024-06-03T03:17:33.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:17:33.494+0000 7f54c2e00700 1 -- 172.21.15.117:0/3671921970 shutdown_connections 2024-06-03T03:17:33.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:17:33.494+0000 7f54c2e00700 1 --2- 172.21.15.117:0/3671921970 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f54a40a2ca0 0x7f54a40a5160 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:17:33.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:17:33.494+0000 7f54c2e00700 1 --2- 172.21.15.117:0/3671921970 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54bc10ea00 0x7f54bc1a6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:17:33.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:17:33.494+0000 7f54c2e00700 1 --2- 172.21.15.117:0/3671921970 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f54bc074ef0 0x7f54bc19ec70 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:17:33.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:17:33.494+0000 7f54c2e00700 1 --2- 172.21.15.117:0/3671921970 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54bc0764c0 0x7f54bc19f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:17:33.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:17:33.494+0000 7f54c2e00700 1 -- 172.21.15.117:0/3671921970 >> 172.21.15.117:0/3671921970 conn(0x7f54bc0fd9c0 msgr2=0x7f54bc101da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:17:33.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:17:33.494+0000 7f54c2e00700 1 -- 172.21.15.117:0/3671921970 shutdown_connections 2024-06-03T03:17:33.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:17:33.494+0000 7f54c2e00700 1 -- 172.21.15.117:0/3671921970 wait complete. 2024-06-03T03:17:34.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:33 smithi151 ceph-mon[37043]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-06-03T03:17:34.161 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:33 smithi151 ceph-mon[37043]: from='osd.9 [v2:172.21.15.117:6826/2538315141,v1:172.21.15.117:6827/2538315141]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-06-03T03:17:34.161 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:33 smithi151 ceph-mon[37043]: osdmap e65: 10 total, 9 up, 10 in 2024-06-03T03:17:34.161 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:33 smithi151 ceph-mon[37043]: from='osd.9 [v2:172.21.15.117:6826/2538315141,v1:172.21.15.117:6827/2538315141]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-06-03T03:17:34.162 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:33 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-03T03:17:34.162 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:33 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:17:34.162 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:33 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:17:34.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:33 smithi117 ceph-mon[31649]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-06-03T03:17:34.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:33 smithi117 ceph-mon[31649]: from='osd.9 [v2:172.21.15.117:6826/2538315141,v1:172.21.15.117:6827/2538315141]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-06-03T03:17:34.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:33 smithi117 ceph-mon[31649]: osdmap e65: 10 total, 9 up, 10 in 2024-06-03T03:17:34.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:33 smithi117 ceph-mon[31649]: from='osd.9 [v2:172.21.15.117:6826/2538315141,v1:172.21.15.117:6827/2538315141]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-06-03T03:17:34.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:33 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-03T03:17:34.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:33 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:17:34.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:33 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:17:34.234 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:33 smithi183 ceph-mon[37326]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-06-03T03:17:34.234 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:33 smithi183 ceph-mon[37326]: from='osd.9 [v2:172.21.15.117:6826/2538315141,v1:172.21.15.117:6827/2538315141]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-06-03T03:17:34.234 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:33 smithi183 ceph-mon[37326]: osdmap e65: 10 total, 9 up, 10 in 2024-06-03T03:17:34.234 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:33 smithi183 ceph-mon[37326]: from='osd.9 [v2:172.21.15.117:6826/2538315141,v1:172.21.15.117:6827/2538315141]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-06-03T03:17:34.234 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:33 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-03T03:17:34.234 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:33 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:17:34.234 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:33 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:17:34.358 DEBUG:teuthology.orchestra.run.smithi117:osd.9> sudo journalctl -f -n 0 -u ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@osd.9.service 2024-06-03T03:17:34.365 INFO:tasks.cephadm:Deploying osd.10 on smithi151 with /dev/vg_nvme/lv_1... 2024-06-03T03:17:34.365 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-06-03T03:17:34.616 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.b/config 2024-06-03T03:17:34.617 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:17:34 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:17:34.437+0000 7f91ad600700 -1 osd.9 0 waiting for initial osdmap 2024-06-03T03:17:34.618 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:17:34 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:17:34.469+0000 7f91a3400700 -1 osd.9 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-03T03:17:34.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:34 smithi117 ceph-mon[31649]: from='osd.9 [v2:172.21.15.117:6826/2538315141,v1:172.21.15.117:6827/2538315141]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-06-03T03:17:34.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:34 smithi117 ceph-mon[31649]: osdmap e66: 10 total, 9 up, 10 in 2024-06-03T03:17:34.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:34 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-03T03:17:34.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:34 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-03T03:17:34.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:34 smithi117 ceph-mon[31649]: from='osd.9 [v2:172.21.15.117:6826/2538315141,v1:172.21.15.117:6827/2538315141]' entity='osd.9' 2024-06-03T03:17:35.059 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:34 smithi151 ceph-mon[37043]: from='osd.9 [v2:172.21.15.117:6826/2538315141,v1:172.21.15.117:6827/2538315141]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-06-03T03:17:35.059 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:34 smithi151 ceph-mon[37043]: osdmap e66: 10 total, 9 up, 10 in 2024-06-03T03:17:35.059 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:34 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-03T03:17:35.059 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:34 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-03T03:17:35.059 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:34 smithi151 ceph-mon[37043]: from='osd.9 [v2:172.21.15.117:6826/2538315141,v1:172.21.15.117:6827/2538315141]' entity='osd.9' 2024-06-03T03:17:35.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:34 smithi183 ceph-mon[37326]: from='osd.9 [v2:172.21.15.117:6826/2538315141,v1:172.21.15.117:6827/2538315141]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-06-03T03:17:35.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:34 smithi183 ceph-mon[37326]: osdmap e66: 10 total, 9 up, 10 in 2024-06-03T03:17:35.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:34 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-03T03:17:35.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:34 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-03T03:17:35.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:34 smithi183 ceph-mon[37326]: from='osd.9 [v2:172.21.15.117:6826/2538315141,v1:172.21.15.117:6827/2538315141]' entity='osd.9' 2024-06-03T03:17:36.031 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:35 smithi151 ceph-mon[37043]: purged_snaps scrub starts 2024-06-03T03:17:36.031 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:35 smithi151 ceph-mon[37043]: purged_snaps scrub ok 2024-06-03T03:17:36.032 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:35 smithi151 ceph-mon[37043]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-06-03T03:17:36.032 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:35 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-03T03:17:36.032 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:35 smithi151 ceph-mon[37043]: osd.9 [v2:172.21.15.117:6826/2538315141,v1:172.21.15.117:6827/2538315141] boot 2024-06-03T03:17:36.032 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:35 smithi151 ceph-mon[37043]: osdmap e67: 10 total, 10 up, 10 in 2024-06-03T03:17:36.032 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:35 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-03T03:17:36.126 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:35 smithi117 ceph-mon[31649]: purged_snaps scrub starts 2024-06-03T03:17:36.126 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:35 smithi117 ceph-mon[31649]: purged_snaps scrub ok 2024-06-03T03:17:36.126 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:35 smithi117 ceph-mon[31649]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-06-03T03:17:36.126 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:35 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-03T03:17:36.126 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:35 smithi117 ceph-mon[31649]: osd.9 [v2:172.21.15.117:6826/2538315141,v1:172.21.15.117:6827/2538315141] boot 2024-06-03T03:17:36.126 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:35 smithi117 ceph-mon[31649]: osdmap e67: 10 total, 10 up, 10 in 2024-06-03T03:17:36.126 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:35 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-03T03:17:36.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:35 smithi183 ceph-mon[37326]: purged_snaps scrub starts 2024-06-03T03:17:36.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:35 smithi183 ceph-mon[37326]: purged_snaps scrub ok 2024-06-03T03:17:36.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:35 smithi183 ceph-mon[37326]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-06-03T03:17:36.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:35 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-03T03:17:36.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:35 smithi183 ceph-mon[37326]: osd.9 [v2:172.21.15.117:6826/2538315141,v1:172.21.15.117:6827/2538315141] boot 2024-06-03T03:17:36.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:35 smithi183 ceph-mon[37326]: osdmap e67: 10 total, 10 up, 10 in 2024-06-03T03:17:36.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:35 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-03T03:17:36.710 INFO:teuthology.orchestra.run.smithi151.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-06-03T03:17:36.711 INFO:teuthology.orchestra.run.smithi151.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-06-03T03:17:36.711 INFO:teuthology.orchestra.run.smithi151.stderr: stderr: 10+0 records in 2024-06-03T03:17:36.711 INFO:teuthology.orchestra.run.smithi151.stderr:10+0 records out 2024-06-03T03:17:36.711 INFO:teuthology.orchestra.run.smithi151.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114204 s, 918 MB/s 2024-06-03T03:17:36.711 INFO:teuthology.orchestra.run.smithi151.stderr:--> Zapping successful for: 2024-06-03T03:17:37.360 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi151:vg_nvme/lv_1 2024-06-03T03:17:37.596 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.b/config 2024-06-03T03:17:38.026 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:37 smithi151 ceph-mon[37043]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-06-03T03:17:38.027 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:37 smithi151 ceph-mon[37043]: osdmap e68: 10 total, 10 up, 10 in 2024-06-03T03:17:38.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:37 smithi117 ceph-mon[31649]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-06-03T03:17:38.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:37 smithi117 ceph-mon[31649]: osdmap e68: 10 total, 10 up, 10 in 2024-06-03T03:17:38.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:37 smithi183 ceph-mon[37326]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-06-03T03:17:38.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:37 smithi183 ceph-mon[37326]: osdmap e68: 10 total, 10 up, 10 in 2024-06-03T03:17:39.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:17:39.238+0000 7f9468806700 1 -- 172.21.15.151:0/2838591548 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9464102fa0 msgr2=0x7f9464109850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:17:39.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:17:39.238+0000 7f9468806700 1 --2- 172.21.15.151:0/2838591548 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9464102fa0 0x7f9464109850 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f945c00b0d0 tx=0x7f945c00b3e0 comp rx=0 tx=0).stop 2024-06-03T03:17:39.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:17:39.238+0000 7f9468806700 1 -- 172.21.15.151:0/2838591548 shutdown_connections 2024-06-03T03:17:39.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:17:39.238+0000 7f9468806700 1 --2- 172.21.15.151:0/2838591548 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9464102fa0 0x7f9464109850 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:17:39.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:17:39.238+0000 7f9468806700 1 --2- 172.21.15.151:0/2838591548 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9464101400 0x7f9464101820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:17:39.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:17:39.238+0000 7f9468806700 1 --2- 172.21.15.151:0/2838591548 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9464102600 0x7f9464102a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:17:39.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:17:39.239+0000 7f9468806700 1 -- 172.21.15.151:0/2838591548 >> 172.21.15.151:0/2838591548 conn(0x7f94640fc9c0 msgr2=0x7f94640fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:17:39.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:17:39.239+0000 7f9468806700 1 -- 172.21.15.151:0/2838591548 shutdown_connections 2024-06-03T03:17:39.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:17:39.239+0000 7f9468806700 1 -- 172.21.15.151:0/2838591548 wait complete. 2024-06-03T03:17:39.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:17:39.239+0000 7f9468806700 1 Processor -- start 2024-06-03T03:17:39.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:17:39.239+0000 7f9468806700 1 -- start start 2024-06-03T03:17:39.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:17:39.239+0000 7f9468806700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9464101400 0x7f9464197660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:17:39.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:17:39.239+0000 7f9468806700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9464102600 0x7f9464197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:17:39.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:17:39.239+0000 7f9468806700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9464102fa0 0x7f946419ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:17:39.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:17:39.239+0000 7f9468806700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f946410d080 con 0x7f9464101400 2024-06-03T03:17:39.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:17:39.239+0000 7f9468806700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f946410cf00 con 0x7f9464102600 2024-06-03T03:17:39.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:17:39.239+0000 7f9468806700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f946410d200 con 0x7f9464102fa0 2024-06-03T03:17:39.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:17:39.240+0000 7f9461d9b700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9464102600 0x7f9464197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:17:39.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:17:39.240+0000 7f9461d9b700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9464102600 0x7f9464197ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:55500/0 (socket says 172.21.15.151:55500) 2024-06-03T03:17:39.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:17:39.240+0000 7f9461d9b700 1 -- 172.21.15.151:0/2811170634 learned_addr learned my addr 172.21.15.151:0/2811170634 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-06-03T03:17:39.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:17:39.240+0000 7f9462d9d700 1 --2- 172.21.15.151:0/2811170634 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9464102fa0 0x7f946419ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:17:39.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:17:39.240+0000 7f9461d9b700 1 -- 172.21.15.151:0/2811170634 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9464102fa0 msgr2=0x7f946419ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:17:39.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:17:39.240+0000 7f9461d9b700 1 --2- 172.21.15.151:0/2811170634 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9464102fa0 0x7f946419ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:17:39.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:17:39.240+0000 7f9461d9b700 1 -- 172.21.15.151:0/2811170634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9464101400 msgr2=0x7f9464197660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:17:39.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:17:39.240+0000 7f9461d9b700 1 --2- 172.21.15.151:0/2811170634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9464101400 0x7f9464197660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:17:39.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:17:39.240+0000 7f9461d9b700 1 -- 172.21.15.151:0/2811170634 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f946419f2c0 con 0x7f9464102600 2024-06-03T03:17:39.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:17:39.240+0000 7f9461d9b700 1 --2- 172.21.15.151:0/2811170634 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9464102600 0x7f9464197ba0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f94540098f0 tx=0x7f9454009cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:17:39.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:17:39.240+0000 7f94537fe700 1 -- 172.21.15.151:0/2811170634 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f945400d020 con 0x7f9464102600 2024-06-03T03:17:39.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:17:39.240+0000 7f9468806700 1 -- 172.21.15.151:0/2811170634 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f946406ac40 con 0x7f9464102600 2024-06-03T03:17:39.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:17:39.240+0000 7f94537fe700 1 -- 172.21.15.151:0/2811170634 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f945400d1c0 con 0x7f9464102600 2024-06-03T03:17:39.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:17:39.240+0000 7f94537fe700 1 -- 172.21.15.151:0/2811170634 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94540076b0 con 0x7f9464102600 2024-06-03T03:17:39.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:17:39.240+0000 7f9468806700 1 -- 172.21.15.151:0/2811170634 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f946406b1b0 con 0x7f9464102600 2024-06-03T03:17:39.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:17:39.241+0000 7f9468806700 1 -- 172.21.15.151:0/2811170634 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9464069180 con 0x7f9464102600 2024-06-03T03:17:39.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:17:39.242+0000 7f94537fe700 1 -- 172.21.15.151:0/2811170634 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f945401f020 con 0x7f9464102600 2024-06-03T03:17:39.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:17:39.243+0000 7f94537fe700 1 --2- 172.21.15.151:0/2811170634 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f944c0a2c20 0x7f944c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:17:39.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:17:39.243+0000 7f94537fe700 1 -- 172.21.15.151:0/2811170634 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6261+0+0 (secure 0 0 0) 0x7f94540cf9a0 con 0x7f9464102600 2024-06-03T03:17:39.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:17:39.243+0000 7f946259c700 1 --2- 172.21.15.151:0/2811170634 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f944c0a2c20 0x7f944c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:17:39.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:17:39.244+0000 7f946259c700 1 --2- 172.21.15.151:0/2811170634 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f944c0a2c20 0x7f944c0a50e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9464102460 tx=0x7f9458006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:17:39.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:17:39.248+0000 7f94537fe700 1 -- 172.21.15.151:0/2811170634 <== mon.2 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f945409d100 con 0x7f9464102600 2024-06-03T03:17:39.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:17:39.457+0000 7f9468806700 1 -- 172.21.15.151:0/2811170634 --> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f946406b410 con 0x7f944c0a2c20 2024-06-03T03:17:40.110 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:39 smithi151 ceph-mon[37043]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-06-03T03:17:40.110 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:39 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:17:40.111 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:39 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:17:40.111 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:39 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:17:40.111 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:39 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:17:40.111 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:39 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:17:40.111 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:39 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:17:40.111 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:39 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:17:40.111 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:39 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:17:40.111 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:39 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:17:40.111 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:39 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:17:40.111 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:39 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T03:17:40.111 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:39 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T03:17:40.111 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:39 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:17:40.112 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:39 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:17:40.112 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:39 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:17:40.112 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:39 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:17:40.112 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:39 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:17:40.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:39 smithi117 ceph-mon[31649]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-06-03T03:17:40.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:39 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:17:40.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:39 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:17:40.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:39 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:17:40.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:39 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:17:40.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:39 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:17:40.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:39 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:17:40.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:39 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:17:40.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:39 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:17:40.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:39 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:17:40.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:39 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:17:40.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:39 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T03:17:40.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:39 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T03:17:40.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:39 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:17:40.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:39 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:17:40.199 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:39 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:17:40.199 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:39 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:17:40.199 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:39 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:17:40.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:39 smithi183 ceph-mon[37326]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-06-03T03:17:40.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:39 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:17:40.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:39 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:17:40.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:39 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:17:40.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:39 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:17:40.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:39 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:17:40.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:39 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:17:40.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:39 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:17:40.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:39 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:17:40.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:39 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:17:40.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:39 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:17:40.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:39 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T03:17:40.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:39 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T03:17:40.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:39 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:17:40.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:39 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:17:40.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:39 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:17:40.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:39 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:17:40.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:39 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:17:41.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:40 smithi151 ceph-mon[37043]: Detected new or changed devices on smithi117 2024-06-03T03:17:41.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:40 smithi151 ceph-mon[37043]: Adjusting osd_memory_target on smithi117 to 4272M 2024-06-03T03:17:41.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:40 smithi151 ceph-mon[37043]: from='client.24356 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:17:41.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:40 smithi117 ceph-mon[31649]: Detected new or changed devices on smithi117 2024-06-03T03:17:41.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:40 smithi117 ceph-mon[31649]: Adjusting osd_memory_target on smithi117 to 4272M 2024-06-03T03:17:41.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:40 smithi117 ceph-mon[31649]: from='client.24356 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:17:41.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:40 smithi183 ceph-mon[37326]: Detected new or changed devices on smithi117 2024-06-03T03:17:41.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:40 smithi183 ceph-mon[37326]: Adjusting osd_memory_target on smithi117 to 4272M 2024-06-03T03:17:41.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:40 smithi183 ceph-mon[37326]: from='client.24356 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:17:42.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:41 smithi151 ceph-mon[37043]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-06-03T03:17:42.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:41 smithi117 ceph-mon[31649]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-06-03T03:17:42.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:41 smithi183 ceph-mon[37326]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-06-03T03:17:43.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:42 smithi151 ceph-mon[37043]: from='client.? 172.21.15.151:0/1933119498' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ba2333cd-32a3-41ef-9ea5-287c707f512b"}]: dispatch 2024-06-03T03:17:43.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:42 smithi151 ceph-mon[37043]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ba2333cd-32a3-41ef-9ea5-287c707f512b"}]: dispatch 2024-06-03T03:17:43.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:42 smithi151 ceph-mon[37043]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ba2333cd-32a3-41ef-9ea5-287c707f512b"}]': finished 2024-06-03T03:17:43.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:42 smithi151 ceph-mon[37043]: osdmap e69: 11 total, 10 up, 11 in 2024-06-03T03:17:43.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:42 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-03T03:17:43.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:42 smithi117 ceph-mon[31649]: from='client.? 172.21.15.151:0/1933119498' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ba2333cd-32a3-41ef-9ea5-287c707f512b"}]: dispatch 2024-06-03T03:17:43.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:42 smithi117 ceph-mon[31649]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ba2333cd-32a3-41ef-9ea5-287c707f512b"}]: dispatch 2024-06-03T03:17:43.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:42 smithi117 ceph-mon[31649]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ba2333cd-32a3-41ef-9ea5-287c707f512b"}]': finished 2024-06-03T03:17:43.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:42 smithi117 ceph-mon[31649]: osdmap e69: 11 total, 10 up, 11 in 2024-06-03T03:17:43.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:42 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-03T03:17:43.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:42 smithi183 ceph-mon[37326]: from='client.? 172.21.15.151:0/1933119498' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ba2333cd-32a3-41ef-9ea5-287c707f512b"}]: dispatch 2024-06-03T03:17:43.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:42 smithi183 ceph-mon[37326]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ba2333cd-32a3-41ef-9ea5-287c707f512b"}]: dispatch 2024-06-03T03:17:43.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:42 smithi183 ceph-mon[37326]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ba2333cd-32a3-41ef-9ea5-287c707f512b"}]': finished 2024-06-03T03:17:43.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:42 smithi183 ceph-mon[37326]: osdmap e69: 11 total, 10 up, 11 in 2024-06-03T03:17:43.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:42 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-03T03:17:44.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:43 smithi151 ceph-mon[37043]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-06-03T03:17:44.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:43 smithi151 ceph-mon[37043]: from='client.? 172.21.15.151:0/3326275536' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T03:17:44.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:43 smithi117 ceph-mon[31649]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-06-03T03:17:44.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:43 smithi117 ceph-mon[31649]: from='client.? 172.21.15.151:0/3326275536' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T03:17:44.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:43 smithi183 ceph-mon[37326]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-06-03T03:17:44.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:43 smithi183 ceph-mon[37326]: from='client.? 172.21.15.151:0/3326275536' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T03:17:46.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:45 smithi151 ceph-mon[37043]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-06-03T03:17:46.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:45 smithi117 ceph-mon[31649]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-06-03T03:17:46.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:45 smithi183 ceph-mon[37326]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-06-03T03:17:48.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:47 smithi151 ceph-mon[37043]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-06-03T03:17:48.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:47 smithi117 ceph-mon[31649]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-06-03T03:17:48.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:47 smithi183 ceph-mon[37326]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-06-03T03:17:50.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:49 smithi151 ceph-mon[37043]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-06-03T03:17:50.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:49 smithi117 ceph-mon[31649]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-06-03T03:17:50.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:49 smithi183 ceph-mon[37326]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-06-03T03:17:51.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:50 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-06-03T03:17:51.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:50 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:17:51.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:50 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-06-03T03:17:51.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:50 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:17:51.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:50 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-06-03T03:17:51.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:50 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:17:51.906 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:51 smithi151 ceph-mon[37043]: Deploying daemon osd.10 on smithi151 2024-06-03T03:17:51.906 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:51 smithi151 ceph-mon[37043]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-06-03T03:17:52.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:51 smithi117 ceph-mon[31649]: Deploying daemon osd.10 on smithi151 2024-06-03T03:17:52.199 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:51 smithi117 ceph-mon[31649]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-06-03T03:17:52.231 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:51 smithi183 ceph-mon[37326]: Deploying daemon osd.10 on smithi151 2024-06-03T03:17:52.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:51 smithi183 ceph-mon[37326]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-06-03T03:17:54.088 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:53 smithi151 ceph-mon[37043]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-06-03T03:17:54.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:53 smithi117 ceph-mon[31649]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-06-03T03:17:54.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:53 smithi183 ceph-mon[37326]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-06-03T03:17:56.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:55 smithi151 ceph-mon[37043]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-06-03T03:17:56.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:55 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:17:56.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:55 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:17:56.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:55 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:17:56.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:55 smithi117 ceph-mon[31649]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-06-03T03:17:56.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:55 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:17:56.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:55 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:17:56.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:55 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:17:56.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:55 smithi183 ceph-mon[37326]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-06-03T03:17:56.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:55 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:17:56.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:55 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:17:56.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:55 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:17:58.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:57 smithi151 ceph-mon[37043]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-06-03T03:17:58.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:57 smithi117 ceph-mon[31649]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-06-03T03:17:58.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:57 smithi183 ceph-mon[37326]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-06-03T03:17:59.946 INFO:teuthology.orchestra.run.smithi151.stdout:Created osd(s) 10 on host 'smithi151' 2024-06-03T03:17:59.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:17:59.945+0000 7f94537fe700 1 -- 172.21.15.151:0/2811170634 <== mgr.14152 v2:172.21.15.117:6800/1398160887 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f946406b410 con 0x7f944c0a2c20 2024-06-03T03:17:59.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:17:59.948+0000 7f9468806700 1 -- 172.21.15.151:0/2811170634 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f944c0a2c20 msgr2=0x7f944c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:17:59.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:17:59.948+0000 7f9468806700 1 --2- 172.21.15.151:0/2811170634 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f944c0a2c20 0x7f944c0a50e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9464102460 tx=0x7f9458006d20 comp rx=0 tx=0).stop 2024-06-03T03:17:59.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:17:59.948+0000 7f9468806700 1 -- 172.21.15.151:0/2811170634 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9464102600 msgr2=0x7f9464197ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:17:59.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:17:59.948+0000 7f9468806700 1 --2- 172.21.15.151:0/2811170634 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9464102600 0x7f9464197ba0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f94540098f0 tx=0x7f9454009cb0 comp rx=0 tx=0).stop 2024-06-03T03:17:59.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:17:59.948+0000 7f9468806700 1 -- 172.21.15.151:0/2811170634 shutdown_connections 2024-06-03T03:17:59.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:17:59.948+0000 7f9468806700 1 --2- 172.21.15.151:0/2811170634 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f944c0a2c20 0x7f944c0a50e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:17:59.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:17:59.948+0000 7f9468806700 1 --2- 172.21.15.151:0/2811170634 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9464102fa0 0x7f946419ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:17:59.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:17:59.948+0000 7f9468806700 1 --2- 172.21.15.151:0/2811170634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9464101400 0x7f9464197660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:17:59.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:17:59.948+0000 7f9468806700 1 --2- 172.21.15.151:0/2811170634 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9464102600 0x7f9464197ba0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:17:59.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:17:59.948+0000 7f9468806700 1 -- 172.21.15.151:0/2811170634 >> 172.21.15.151:0/2811170634 conn(0x7f94640fc9c0 msgr2=0x7f9464105900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:17:59.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:17:59.948+0000 7f9468806700 1 -- 172.21.15.151:0/2811170634 shutdown_connections 2024-06-03T03:17:59.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-06-03T03:17:59.948+0000 7f9468806700 1 -- 172.21.15.151:0/2811170634 wait complete. 2024-06-03T03:18:00.158 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:59 smithi151 ceph-mon[37043]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-06-03T03:18:00.158 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:59 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:00.158 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:59 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:00.158 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:59 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:18:00.159 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:59 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:18:00.159 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:59 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:00.159 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:59 smithi151 ceph-mon[37043]: from='osd.10 [v2:172.21.15.151:6824/3801045133,v1:172.21.15.151:6825/3801045133]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-03T03:18:00.159 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:59 smithi151 ceph-mon[37043]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-03T03:18:00.159 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:59 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:00.159 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:59 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:18:00.159 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:17:59 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:00.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:59 smithi117 ceph-mon[31649]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-06-03T03:18:00.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:59 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:00.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:59 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:00.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:59 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:18:00.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:59 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:18:00.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:59 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:00.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:59 smithi117 ceph-mon[31649]: from='osd.10 [v2:172.21.15.151:6824/3801045133,v1:172.21.15.151:6825/3801045133]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-03T03:18:00.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:59 smithi117 ceph-mon[31649]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-03T03:18:00.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:59 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:00.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:59 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:18:00.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:17:59 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:00.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:59 smithi183 ceph-mon[37326]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-06-03T03:18:00.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:59 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:00.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:59 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:00.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:59 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:18:00.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:59 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:18:00.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:59 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:00.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:59 smithi183 ceph-mon[37326]: from='osd.10 [v2:172.21.15.151:6824/3801045133,v1:172.21.15.151:6825/3801045133]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-03T03:18:00.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:59 smithi183 ceph-mon[37326]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-03T03:18:00.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:59 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:00.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:59 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:18:00.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:17:59 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:00.584 DEBUG:teuthology.orchestra.run.smithi151:osd.10> sudo journalctl -f -n 0 -u ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@osd.10.service 2024-06-03T03:18:00.588 INFO:tasks.cephadm:Deploying osd.11 on smithi183 with /dev/vg_nvme/lv_1... 2024-06-03T03:18:00.588 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-06-03T03:18:00.834 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.c/config 2024-06-03T03:18:01.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:01 smithi151 ceph-mon[37043]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-06-03T03:18:01.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:01 smithi151 ceph-mon[37043]: from='osd.10 [v2:172.21.15.151:6824/3801045133,v1:172.21.15.151:6825/3801045133]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-06-03T03:18:01.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:01 smithi151 ceph-mon[37043]: osdmap e70: 11 total, 10 up, 11 in 2024-06-03T03:18:01.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:01 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-03T03:18:01.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:01 smithi151 ceph-mon[37043]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-06-03T03:18:01.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:01 smithi151 ceph-mon[37043]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-06-03T03:18:01.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:01 smithi183 ceph-mon[37326]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-06-03T03:18:01.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:01 smithi183 ceph-mon[37326]: from='osd.10 [v2:172.21.15.151:6824/3801045133,v1:172.21.15.151:6825/3801045133]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-06-03T03:18:01.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:01 smithi183 ceph-mon[37326]: osdmap e70: 11 total, 10 up, 11 in 2024-06-03T03:18:01.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:01 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-03T03:18:01.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:01 smithi183 ceph-mon[37326]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-06-03T03:18:01.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:01 smithi183 ceph-mon[37326]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-06-03T03:18:01.375 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:01 smithi117 ceph-mon[31649]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-06-03T03:18:01.375 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:01 smithi117 ceph-mon[31649]: from='osd.10 [v2:172.21.15.151:6824/3801045133,v1:172.21.15.151:6825/3801045133]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-06-03T03:18:01.375 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:01 smithi117 ceph-mon[31649]: osdmap e70: 11 total, 10 up, 11 in 2024-06-03T03:18:01.375 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:01 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-03T03:18:01.375 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:01 smithi117 ceph-mon[31649]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-06-03T03:18:01.376 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:01 smithi117 ceph-mon[31649]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-06-03T03:18:01.808 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:18:01 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:18:01.435+0000 7fd514170700 -1 osd.10 0 waiting for initial osdmap 2024-06-03T03:18:01.808 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:18:01 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:18:01.448+0000 7fd50bf5b700 -1 osd.10 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-03T03:18:02.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:02 smithi151 ceph-mon[37043]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-06-03T03:18:02.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:02 smithi151 ceph-mon[37043]: osdmap e71: 11 total, 10 up, 11 in 2024-06-03T03:18:02.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:02 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-03T03:18:02.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:02 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-03T03:18:02.116 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:02 smithi151 ceph-mon[37043]: from='osd.10 ' entity='osd.10' 2024-06-03T03:18:02.336 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:02 smithi183 ceph-mon[37326]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-06-03T03:18:02.336 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:02 smithi183 ceph-mon[37326]: osdmap e71: 11 total, 10 up, 11 in 2024-06-03T03:18:02.336 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:02 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-03T03:18:02.337 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:02 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-03T03:18:02.337 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:02 smithi183 ceph-mon[37326]: from='osd.10 ' entity='osd.10' 2024-06-03T03:18:02.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:02 smithi117 ceph-mon[31649]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-06-03T03:18:02.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:02 smithi117 ceph-mon[31649]: osdmap e71: 11 total, 10 up, 11 in 2024-06-03T03:18:02.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:02 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-03T03:18:02.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:02 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-03T03:18:02.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:02 smithi117 ceph-mon[31649]: from='osd.10 ' entity='osd.10' 2024-06-03T03:18:02.906 INFO:teuthology.orchestra.run.smithi183.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-06-03T03:18:02.907 INFO:teuthology.orchestra.run.smithi183.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-06-03T03:18:02.907 INFO:teuthology.orchestra.run.smithi183.stderr: stderr: 10+0 records in 2024-06-03T03:18:02.907 INFO:teuthology.orchestra.run.smithi183.stderr:10+0 records out 2024-06-03T03:18:02.907 INFO:teuthology.orchestra.run.smithi183.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0122539 s, 856 MB/s 2024-06-03T03:18:02.907 INFO:teuthology.orchestra.run.smithi183.stderr:--> Zapping successful for: 2024-06-03T03:18:03.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:03 smithi151 ceph-mon[37043]: purged_snaps scrub starts 2024-06-03T03:18:03.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:03 smithi151 ceph-mon[37043]: purged_snaps scrub ok 2024-06-03T03:18:03.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:03 smithi151 ceph-mon[37043]: osd.10 [v2:172.21.15.151:6824/3801045133,v1:172.21.15.151:6825/3801045133] boot 2024-06-03T03:18:03.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:03 smithi151 ceph-mon[37043]: osdmap e72: 11 total, 11 up, 11 in 2024-06-03T03:18:03.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:03 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-03T03:18:03.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:03 smithi151 ceph-mon[37043]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-06-03T03:18:03.293 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:03 smithi183 ceph-mon[37326]: purged_snaps scrub starts 2024-06-03T03:18:03.293 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:03 smithi183 ceph-mon[37326]: purged_snaps scrub ok 2024-06-03T03:18:03.293 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:03 smithi183 ceph-mon[37326]: osd.10 [v2:172.21.15.151:6824/3801045133,v1:172.21.15.151:6825/3801045133] boot 2024-06-03T03:18:03.294 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:03 smithi183 ceph-mon[37326]: osdmap e72: 11 total, 11 up, 11 in 2024-06-03T03:18:03.294 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:03 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-03T03:18:03.294 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:03 smithi183 ceph-mon[37326]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-06-03T03:18:03.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:03 smithi117 ceph-mon[31649]: purged_snaps scrub starts 2024-06-03T03:18:03.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:03 smithi117 ceph-mon[31649]: purged_snaps scrub ok 2024-06-03T03:18:03.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:03 smithi117 ceph-mon[31649]: osd.10 [v2:172.21.15.151:6824/3801045133,v1:172.21.15.151:6825/3801045133] boot 2024-06-03T03:18:03.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:03 smithi117 ceph-mon[31649]: osdmap e72: 11 total, 11 up, 11 in 2024-06-03T03:18:03.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:03 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-03T03:18:03.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:03 smithi117 ceph-mon[31649]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-06-03T03:18:03.623 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi183:vg_nvme/lv_1 2024-06-03T03:18:03.867 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.c/config 2024-06-03T03:18:04.223 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:04 smithi183 ceph-mon[37326]: osdmap e73: 11 total, 11 up, 11 in 2024-06-03T03:18:04.223 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:04 smithi183 ceph-mon[37326]: Detected new or changed devices on smithi151 2024-06-03T03:18:04.223 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:04 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:04.223 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:04 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:04.223 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:04 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:18:04.223 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:04 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:18:04.224 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:04 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:18:04.224 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:04 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:18:04.224 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:04 smithi183 ceph-mon[37326]: Adjusting osd_memory_target on smithi151 to 4272M 2024-06-03T03:18:04.224 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:04 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:04.224 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:04 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:18:04.224 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:04 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:18:04.224 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:04 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:04.224 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:04 smithi183 ceph-mon[37326]: osdmap e74: 11 total, 11 up, 11 in 2024-06-03T03:18:04.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:04 smithi151 ceph-mon[37043]: osdmap e73: 11 total, 11 up, 11 in 2024-06-03T03:18:04.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:04 smithi151 ceph-mon[37043]: Detected new or changed devices on smithi151 2024-06-03T03:18:04.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:04 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:04.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:04 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:04.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:04 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:18:04.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:04 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:18:04.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:04 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:18:04.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:04 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:18:04.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:04 smithi151 ceph-mon[37043]: Adjusting osd_memory_target on smithi151 to 4272M 2024-06-03T03:18:04.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:04 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:04.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:04 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:18:04.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:04 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:18:04.408 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:04 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:04.408 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:04 smithi151 ceph-mon[37043]: osdmap e74: 11 total, 11 up, 11 in 2024-06-03T03:18:04.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:04 smithi117 ceph-mon[31649]: osdmap e73: 11 total, 11 up, 11 in 2024-06-03T03:18:04.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:04 smithi117 ceph-mon[31649]: Detected new or changed devices on smithi151 2024-06-03T03:18:04.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:04 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:04.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:04 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:04.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:04 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:18:04.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:04 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:18:04.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:04 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:18:04.448 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:04 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:18:04.448 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:04 smithi117 ceph-mon[31649]: Adjusting osd_memory_target on smithi151 to 4272M 2024-06-03T03:18:04.448 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:04 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:04.448 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:04 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:18:04.448 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:04 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:18:04.448 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:04 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:04.448 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:04 smithi117 ceph-mon[31649]: osdmap e74: 11 total, 11 up, 11 in 2024-06-03T03:18:05.503 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:05.500+0000 7fede11d2700 1 -- 172.21.15.183:0/2855108141 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feddc1029d0 msgr2=0x7feddc102df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:18:05.503 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:05.500+0000 7fede11d2700 1 --2- 172.21.15.183:0/2855108141 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feddc1029d0 0x7feddc102df0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fedcc0099c0 tx=0x7fedcc009cd0 comp rx=0 tx=0).stop 2024-06-03T03:18:05.503 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:05.500+0000 7fede11d2700 1 -- 172.21.15.183:0/2855108141 shutdown_connections 2024-06-03T03:18:05.503 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:05.500+0000 7fede11d2700 1 --2- 172.21.15.183:0/2855108141 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feddc1020b0 0x7feddc109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:18:05.503 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:05.500+0000 7fede11d2700 1 --2- 172.21.15.183:0/2855108141 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feddc1029d0 0x7feddc102df0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:18:05.503 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:05.500+0000 7fede11d2700 1 --2- 172.21.15.183:0/2855108141 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feddc101710 0x7feddc101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:18:05.504 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:05.500+0000 7fede11d2700 1 -- 172.21.15.183:0/2855108141 >> 172.21.15.183:0/2855108141 conn(0x7feddc0fc160 msgr2=0x7feddc0fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:18:05.504 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:05.500+0000 7fede11d2700 1 -- 172.21.15.183:0/2855108141 shutdown_connections 2024-06-03T03:18:05.504 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:05.501+0000 7fede11d2700 1 -- 172.21.15.183:0/2855108141 wait complete. 2024-06-03T03:18:05.504 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:05.501+0000 7fede11d2700 1 Processor -- start 2024-06-03T03:18:05.504 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:05.501+0000 7fede11d2700 1 -- start start 2024-06-03T03:18:05.504 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:05.502+0000 7fede11d2700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feddc101710 0x7feddc10f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:18:05.504 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:05.502+0000 7fede11d2700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feddc1020b0 0x7feddc10dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:18:05.504 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:05.502+0000 7fede11d2700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feddc1029d0 0x7feddc10e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:18:05.504 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:05.502+0000 7fede11d2700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7feddc10d0a0 con 0x7feddc101710 2024-06-03T03:18:05.504 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:05.502+0000 7fede11d2700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7feddc10cf20 con 0x7feddc1029d0 2024-06-03T03:18:05.504 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:05.502+0000 7fede11d2700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7feddc10d220 con 0x7feddc1020b0 2024-06-03T03:18:05.504 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:05.502+0000 7fedda59c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feddc1020b0 0x7feddc10dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:18:05.505 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:05.502+0000 7fedda59c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feddc1020b0 0x7feddc10dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:48482/0 (socket says 172.21.15.183:48482) 2024-06-03T03:18:05.505 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:05.502+0000 7fedda59c700 1 -- 172.21.15.183:0/961831139 learned_addr learned my addr 172.21.15.183:0/961831139 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-06-03T03:18:05.505 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:05.502+0000 7feddb59e700 1 --2- 172.21.15.183:0/961831139 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feddc1029d0 0x7feddc10e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:18:05.505 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:05.502+0000 7fedda59c700 1 -- 172.21.15.183:0/961831139 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feddc1029d0 msgr2=0x7feddc10e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:18:05.505 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:05.502+0000 7fedda59c700 1 --2- 172.21.15.183:0/961831139 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feddc1029d0 0x7feddc10e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:18:05.505 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:05.502+0000 7fedda59c700 1 -- 172.21.15.183:0/961831139 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feddc101710 msgr2=0x7feddc10f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:18:05.505 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:05.502+0000 7fedda59c700 1 --2- 172.21.15.183:0/961831139 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feddc101710 0x7feddc10f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:18:05.505 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:05.502+0000 7fedda59c700 1 -- 172.21.15.183:0/961831139 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feddc10ea20 con 0x7feddc1020b0 2024-06-03T03:18:05.505 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:05.502+0000 7fedda59c700 1 --2- 172.21.15.183:0/961831139 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feddc1020b0 0x7feddc10dc30 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fedd000d980 tx=0x7fedd000dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:18:05.505 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:05.502+0000 7fedc3fff700 1 -- 172.21.15.183:0/961831139 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd0017070 con 0x7feddc1020b0 2024-06-03T03:18:05.505 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:05.502+0000 7fedc3fff700 1 -- 172.21.15.183:0/961831139 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedd0005020 con 0x7feddc1020b0 2024-06-03T03:18:05.505 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:05.502+0000 7fede11d2700 1 -- 172.21.15.183:0/961831139 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feddc1a2910 con 0x7feddc1020b0 2024-06-03T03:18:05.505 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:05.502+0000 7fedc3fff700 1 -- 172.21.15.183:0/961831139 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd0009bc0 con 0x7feddc1020b0 2024-06-03T03:18:05.505 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:05.503+0000 7fede11d2700 1 -- 172.21.15.183:0/961831139 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feddc1a2ea0 con 0x7feddc1020b0 2024-06-03T03:18:05.506 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:05.504+0000 7fede11d2700 1 -- 172.21.15.183:0/961831139 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feddc1a3070 con 0x7feddc1020b0 2024-06-03T03:18:05.506 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:05.505+0000 7fedc3fff700 1 -- 172.21.15.183:0/961831139 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fedd00051d0 con 0x7feddc1020b0 2024-06-03T03:18:05.506 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:05.505+0000 7fedc3fff700 1 --2- 172.21.15.183:0/961831139 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fedc40a2ca0 0x7fedc40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:18:05.506 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:05.505+0000 7fedc3fff700 1 -- 172.21.15.183:0/961831139 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6694+0+0 (secure 0 0 0) 0x7fedd00ce8b0 con 0x7feddc1020b0 2024-06-03T03:18:05.507 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:05.506+0000 7feddad9d700 1 --2- 172.21.15.183:0/961831139 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fedc40a2ca0 0x7fedc40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:18:05.507 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:05.507+0000 7feddad9d700 1 --2- 172.21.15.183:0/961831139 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fedc40a2ca0 0x7fedc40a5160 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fedcc003890 tx=0x7fedcc000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:18:05.510 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:05.510+0000 7fedc3fff700 1 -- 172.21.15.183:0/961831139 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fedd0020020 con 0x7feddc1020b0 2024-06-03T03:18:05.716 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:05.716+0000 7fede11d2700 1 -- 172.21.15.183:0/961831139 --> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7feddc0fe330 con 0x7fedc40a2ca0 2024-06-03T03:18:05.716 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:05 smithi183 ceph-mon[37326]: osdmap e75: 11 total, 11 up, 11 in 2024-06-03T03:18:05.716 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:05 smithi183 ceph-mon[37326]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-06-03T03:18:05.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:05 smithi151 ceph-mon[37043]: osdmap e75: 11 total, 11 up, 11 in 2024-06-03T03:18:05.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:05 smithi151 ceph-mon[37043]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-06-03T03:18:05.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:05 smithi117 ceph-mon[31649]: osdmap e75: 11 total, 11 up, 11 in 2024-06-03T03:18:05.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:05 smithi117 ceph-mon[31649]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-06-03T03:18:06.675 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:06 smithi183 ceph-mon[37326]: osdmap e76: 11 total, 11 up, 11 in 2024-06-03T03:18:06.675 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:06 smithi183 ceph-mon[37326]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:18:06.675 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:06 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T03:18:06.675 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:06 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T03:18:06.675 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:06 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:18:06.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:06 smithi151 ceph-mon[37043]: osdmap e76: 11 total, 11 up, 11 in 2024-06-03T03:18:06.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:06 smithi151 ceph-mon[37043]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:18:06.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:06 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T03:18:06.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:06 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T03:18:06.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:06 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:18:06.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:06 smithi117 ceph-mon[31649]: osdmap e76: 11 total, 11 up, 11 in 2024-06-03T03:18:06.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:06 smithi117 ceph-mon[31649]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:18:06.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:06 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T03:18:06.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:06 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T03:18:06.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:06 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:18:07.650 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:07 smithi183 ceph-mon[37326]: pgmap v267: 1 pgs: 1 peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-06-03T03:18:07.906 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:07 smithi151 ceph-mon[37043]: pgmap v267: 1 pgs: 1 peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-06-03T03:18:07.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:07 smithi117 ceph-mon[31649]: pgmap v267: 1 pgs: 1 peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-06-03T03:18:08.866 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:08 smithi183 ceph-mon[37326]: from='client.? 172.21.15.183:0/1823610645' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1556d654-e6fa-42c1-b399-aa53f6837a1e"}]: dispatch 2024-06-03T03:18:08.866 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:08 smithi183 ceph-mon[37326]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1556d654-e6fa-42c1-b399-aa53f6837a1e"}]: dispatch 2024-06-03T03:18:08.866 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:08 smithi183 ceph-mon[37326]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1556d654-e6fa-42c1-b399-aa53f6837a1e"}]': finished 2024-06-03T03:18:08.866 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:08 smithi183 ceph-mon[37326]: osdmap e77: 12 total, 11 up, 12 in 2024-06-03T03:18:08.866 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:08 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-03T03:18:08.897 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:08 smithi151 ceph-mon[37043]: from='client.? 172.21.15.183:0/1823610645' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1556d654-e6fa-42c1-b399-aa53f6837a1e"}]: dispatch 2024-06-03T03:18:08.897 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:08 smithi151 ceph-mon[37043]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1556d654-e6fa-42c1-b399-aa53f6837a1e"}]: dispatch 2024-06-03T03:18:08.897 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:08 smithi151 ceph-mon[37043]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1556d654-e6fa-42c1-b399-aa53f6837a1e"}]': finished 2024-06-03T03:18:08.897 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:08 smithi151 ceph-mon[37043]: osdmap e77: 12 total, 11 up, 12 in 2024-06-03T03:18:08.897 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:08 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-03T03:18:08.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:08 smithi117 ceph-mon[31649]: from='client.? 172.21.15.183:0/1823610645' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1556d654-e6fa-42c1-b399-aa53f6837a1e"}]: dispatch 2024-06-03T03:18:08.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:08 smithi117 ceph-mon[31649]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1556d654-e6fa-42c1-b399-aa53f6837a1e"}]: dispatch 2024-06-03T03:18:08.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:08 smithi117 ceph-mon[31649]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1556d654-e6fa-42c1-b399-aa53f6837a1e"}]': finished 2024-06-03T03:18:08.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:08 smithi117 ceph-mon[31649]: osdmap e77: 12 total, 11 up, 12 in 2024-06-03T03:18:08.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:08 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-03T03:18:09.906 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:09 smithi151 ceph-mon[37043]: pgmap v269: 1 pgs: 1 peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-06-03T03:18:09.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:09 smithi151 ceph-mon[37043]: from='client.? 172.21.15.183:0/1534157721' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T03:18:09.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:09 smithi117 ceph-mon[31649]: pgmap v269: 1 pgs: 1 peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-06-03T03:18:09.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:09 smithi117 ceph-mon[31649]: from='client.? 172.21.15.183:0/1534157721' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T03:18:09.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:09 smithi183 ceph-mon[37326]: pgmap v269: 1 pgs: 1 peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-06-03T03:18:09.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:09 smithi183 ceph-mon[37326]: from='client.? 172.21.15.183:0/1534157721' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T03:18:12.135 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:11 smithi183 ceph-mon[37326]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-06-03T03:18:12.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:11 smithi151 ceph-mon[37043]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-06-03T03:18:12.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:11 smithi117 ceph-mon[31649]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-06-03T03:18:13.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:13 smithi183 ceph-mon[37326]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-03T03:18:14.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:13 smithi151 ceph-mon[37043]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-03T03:18:14.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:13 smithi117 ceph-mon[31649]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-03T03:18:16.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:15 smithi151 ceph-mon[37043]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-06-03T03:18:16.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:15 smithi117 ceph-mon[31649]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-06-03T03:18:16.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:15 smithi183 ceph-mon[37326]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-06-03T03:18:17.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:16 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-06-03T03:18:17.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:16 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:18:17.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:16 smithi151 ceph-mon[37043]: Deploying daemon osd.11 on smithi183 2024-06-03T03:18:17.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:16 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-06-03T03:18:17.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:16 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:18:17.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:16 smithi117 ceph-mon[31649]: Deploying daemon osd.11 on smithi183 2024-06-03T03:18:17.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:16 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-06-03T03:18:17.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:16 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:18:17.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:16 smithi183 ceph-mon[37326]: Deploying daemon osd.11 on smithi183 2024-06-03T03:18:18.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:17 smithi151 ceph-mon[37043]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-06-03T03:18:18.177 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:17 smithi183 ceph-mon[37326]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-06-03T03:18:18.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:17 smithi117 ceph-mon[31649]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-06-03T03:18:20.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:19 smithi151 ceph-mon[37043]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-06-03T03:18:20.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:19 smithi117 ceph-mon[31649]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-06-03T03:18:20.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:19 smithi183 ceph-mon[37326]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-06-03T03:18:22.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:21 smithi151 ceph-mon[37043]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-06-03T03:18:22.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:21 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:22.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:21 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:22.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:21 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:18:22.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:21 smithi117 ceph-mon[31649]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-06-03T03:18:22.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:21 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:22.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:21 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:22.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:21 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:18:22.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:21 smithi183 ceph-mon[37326]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-06-03T03:18:22.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:21 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:22.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:21 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:22.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:21 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:18:24.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:23 smithi151 ceph-mon[37043]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-06-03T03:18:24.158 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:23 smithi183 ceph-mon[37326]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-06-03T03:18:24.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:23 smithi117 ceph-mon[31649]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-06-03T03:18:25.316 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:25.314+0000 7fedc3fff700 1 -- 172.21.15.183:0/961831139 <== mgr.14152 v2:172.21.15.117:6800/1398160887 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7feddc0fe330 con 0x7fedc40a2ca0 2024-06-03T03:18:25.317 INFO:teuthology.orchestra.run.smithi183.stdout:Created osd(s) 11 on host 'smithi183' 2024-06-03T03:18:25.318 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:25.317+0000 7fede11d2700 1 -- 172.21.15.183:0/961831139 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fedc40a2ca0 msgr2=0x7fedc40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:18:25.318 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:25.317+0000 7fede11d2700 1 --2- 172.21.15.183:0/961831139 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fedc40a2ca0 0x7fedc40a5160 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fedcc003890 tx=0x7fedcc000bc0 comp rx=0 tx=0).stop 2024-06-03T03:18:25.319 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:25.317+0000 7fede11d2700 1 -- 172.21.15.183:0/961831139 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feddc1020b0 msgr2=0x7feddc10dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:18:25.319 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:25.317+0000 7fede11d2700 1 --2- 172.21.15.183:0/961831139 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feddc1020b0 0x7feddc10dc30 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fedd000d980 tx=0x7fedd000dc90 comp rx=0 tx=0).stop 2024-06-03T03:18:25.319 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:25.317+0000 7fede11d2700 1 -- 172.21.15.183:0/961831139 shutdown_connections 2024-06-03T03:18:25.319 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:25.317+0000 7fede11d2700 1 --2- 172.21.15.183:0/961831139 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fedc40a2ca0 0x7fedc40a5160 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:18:25.319 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:25.317+0000 7fede11d2700 1 --2- 172.21.15.183:0/961831139 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feddc1029d0 0x7feddc10e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:18:25.319 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:25.317+0000 7fede11d2700 1 --2- 172.21.15.183:0/961831139 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feddc101710 0x7feddc10f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:18:25.319 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:25.317+0000 7fede11d2700 1 --2- 172.21.15.183:0/961831139 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feddc1020b0 0x7feddc10dc30 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:18:25.319 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:25.317+0000 7fede11d2700 1 -- 172.21.15.183:0/961831139 >> 172.21.15.183:0/961831139 conn(0x7feddc0fc160 msgr2=0x7feddc0fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:18:25.319 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:25.317+0000 7fede11d2700 1 -- 172.21.15.183:0/961831139 shutdown_connections 2024-06-03T03:18:25.319 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:25.317+0000 7fede11d2700 1 -- 172.21.15.183:0/961831139 wait complete. 2024-06-03T03:18:25.906 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:25 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:25.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:25 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:25.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:25 smithi151 ceph-mon[37043]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-06-03T03:18:25.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:25 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:25.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:25 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:25.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:25 smithi151 ceph-mon[37043]: from='osd.11 [v2:172.21.15.183:6824/3513899696,v1:172.21.15.183:6825/3513899696]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-03T03:18:25.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:25 smithi151 ceph-mon[37043]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-03T03:18:25.950 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:25 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:25.950 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:25 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:25.950 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:25 smithi117 ceph-mon[31649]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-06-03T03:18:25.950 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:25 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:25.950 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:25 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:25.950 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:25 smithi117 ceph-mon[31649]: from='osd.11 [v2:172.21.15.183:6824/3513899696,v1:172.21.15.183:6825/3513899696]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-03T03:18:25.951 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:25 smithi117 ceph-mon[31649]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-03T03:18:25.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:25 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:25.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:25 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:25.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:25 smithi183 ceph-mon[37326]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-06-03T03:18:25.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:25 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:25.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:25 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:25.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:25 smithi183 ceph-mon[37326]: from='osd.11 [v2:172.21.15.183:6824/3513899696,v1:172.21.15.183:6825/3513899696]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-03T03:18:25.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:25 smithi183 ceph-mon[37326]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-03T03:18:26.112 DEBUG:teuthology.orchestra.run.smithi183:osd.11> sudo journalctl -f -n 0 -u ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@osd.11.service 2024-06-03T03:18:26.114 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-06-03T03:18:26.114 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph osd stat -f json 2024-06-03T03:18:26.563 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:18:27.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:27 smithi117 ceph-mon[31649]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-06-03T03:18:27.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:27 smithi117 ceph-mon[31649]: from='osd.11 [v2:172.21.15.183:6824/3513899696,v1:172.21.15.183:6825/3513899696]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-06-03T03:18:27.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:27 smithi117 ceph-mon[31649]: osdmap e78: 12 total, 11 up, 12 in 2024-06-03T03:18:27.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:27 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-03T03:18:27.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:27 smithi117 ceph-mon[31649]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-06-03T03:18:27.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:27 smithi117 ceph-mon[31649]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-06-03T03:18:27.564 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:27 smithi183 ceph-mon[37326]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-06-03T03:18:27.565 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:27 smithi183 ceph-mon[37326]: from='osd.11 [v2:172.21.15.183:6824/3513899696,v1:172.21.15.183:6825/3513899696]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-06-03T03:18:27.565 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:27 smithi183 ceph-mon[37326]: osdmap e78: 12 total, 11 up, 12 in 2024-06-03T03:18:27.565 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:27 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-03T03:18:27.565 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:27 smithi183 ceph-mon[37326]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-06-03T03:18:27.565 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:27 smithi183 ceph-mon[37326]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-06-03T03:18:27.656 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:27 smithi151 ceph-mon[37043]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-06-03T03:18:27.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:27 smithi151 ceph-mon[37043]: from='osd.11 [v2:172.21.15.183:6824/3513899696,v1:172.21.15.183:6825/3513899696]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-06-03T03:18:27.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:27 smithi151 ceph-mon[37043]: osdmap e78: 12 total, 11 up, 12 in 2024-06-03T03:18:27.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:27 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-03T03:18:27.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:27 smithi151 ceph-mon[37043]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-06-03T03:18:27.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:27 smithi151 ceph-mon[37043]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-06-03T03:18:27.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:18:27 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:18:27.731+0000 7fd8be28c700 -1 osd.11 0 waiting for initial osdmap 2024-06-03T03:18:27.983 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:18:27 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:18:27.746+0000 7fd8b6077700 -1 osd.11 79 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-03T03:18:28.625 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:28 smithi117 ceph-mon[31649]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-06-03T03:18:28.625 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:28 smithi117 ceph-mon[31649]: osdmap e79: 12 total, 11 up, 12 in 2024-06-03T03:18:28.625 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:28 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-03T03:18:28.625 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:28 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-03T03:18:28.625 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:28 smithi117 ceph-mon[31649]: from='osd.11 ' entity='osd.11' 2024-06-03T03:18:28.625 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:28 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-03T03:18:28.656 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:28 smithi151 ceph-mon[37043]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-06-03T03:18:28.656 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:28 smithi151 ceph-mon[37043]: osdmap e79: 12 total, 11 up, 12 in 2024-06-03T03:18:28.656 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:28 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-03T03:18:28.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:28 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-03T03:18:28.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:28 smithi151 ceph-mon[37043]: from='osd.11 ' entity='osd.11' 2024-06-03T03:18:28.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:28 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-03T03:18:28.692 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:28 smithi183 ceph-mon[37326]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-06-03T03:18:28.692 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:28 smithi183 ceph-mon[37326]: osdmap e79: 12 total, 11 up, 12 in 2024-06-03T03:18:28.692 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:28 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-03T03:18:28.692 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:28 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-03T03:18:28.692 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:28 smithi183 ceph-mon[37326]: from='osd.11 ' entity='osd.11' 2024-06-03T03:18:28.692 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:28 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-03T03:18:28.980 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:28.979+0000 7f4697400700 1 -- 172.21.15.117:0/1578593297 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4690109d30 msgr2=0x7f469010a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:18:28.980 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:28.979+0000 7f4697400700 1 --2- 172.21.15.117:0/1578593297 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4690109d30 0x7f469010a190 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f46800099c0 tx=0x7f4680009cd0 comp rx=0 tx=0).stop 2024-06-03T03:18:28.980 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:28.979+0000 7f4697400700 1 -- 172.21.15.117:0/1578593297 shutdown_connections 2024-06-03T03:18:28.980 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:28.979+0000 7f4697400700 1 --2- 172.21.15.117:0/1578593297 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f469010a6d0 0x7f4690110f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:18:28.981 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:28.979+0000 7f4697400700 1 --2- 172.21.15.117:0/1578593297 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4690108b30 0x7f4690108f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:18:28.981 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:28.979+0000 7f4697400700 1 --2- 172.21.15.117:0/1578593297 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4690109d30 0x7f469010a190 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:18:28.981 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:28.979+0000 7f4697400700 1 -- 172.21.15.117:0/1578593297 >> 172.21.15.117:0/1578593297 conn(0x7f4690077b00 msgr2=0x7f4690079f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:18:28.981 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:28.979+0000 7f4697400700 1 -- 172.21.15.117:0/1578593297 shutdown_connections 2024-06-03T03:18:28.981 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:28.979+0000 7f4697400700 1 -- 172.21.15.117:0/1578593297 wait complete. 2024-06-03T03:18:28.981 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:28.981+0000 7f4697400700 1 Processor -- start 2024-06-03T03:18:28.982 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:28.981+0000 7f4697400700 1 -- start start 2024-06-03T03:18:28.983 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:28.982+0000 7f4697400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4690108b30 0x7f469019ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:18:28.983 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:28.982+0000 7f4697400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4690109d30 0x7f469019f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:18:28.983 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:28.982+0000 7f4697400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f469010a6d0 0x7f46901a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:18:28.983 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:28.982+0000 7f4697400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4690114740 con 0x7f469010a6d0 2024-06-03T03:18:28.983 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:28.982+0000 7f4697400700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f46901145c0 con 0x7f4690108b30 2024-06-03T03:18:28.983 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:28.982+0000 7f4697400700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f46901148c0 con 0x7f4690109d30 2024-06-03T03:18:28.983 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:28.982+0000 7f4694a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4690108b30 0x7f469019ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:18:28.983 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:28.982+0000 7f468fe00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4690109d30 0x7f469019f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:18:28.983 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:28.982+0000 7f4695400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f469010a6d0 0x7f46901a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:18:28.983 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:28.982+0000 7f4694a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4690108b30 0x7f469019ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:52650/0 (socket says 172.21.15.117:52650) 2024-06-03T03:18:28.984 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:28.982+0000 7f468fe00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4690109d30 0x7f469019f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:50470/0 (socket says 172.21.15.117:50470) 2024-06-03T03:18:28.984 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:28.982+0000 7f4694a00700 1 -- 172.21.15.117:0/2611619870 learned_addr learned my addr 172.21.15.117:0/2611619870 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:18:28.984 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:28.983+0000 7f4694a00700 1 -- 172.21.15.117:0/2611619870 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4690109d30 msgr2=0x7f469019f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:18:28.985 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:28.983+0000 7f4694a00700 1 --2- 172.21.15.117:0/2611619870 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4690109d30 0x7f469019f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:18:28.985 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:28.983+0000 7f4694a00700 1 -- 172.21.15.117:0/2611619870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f469010a6d0 msgr2=0x7f46901a6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:18:28.985 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:28.983+0000 7f4694a00700 1 --2- 172.21.15.117:0/2611619870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f469010a6d0 0x7f46901a6290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:18:28.985 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:28.983+0000 7f4694a00700 1 -- 172.21.15.117:0/2611619870 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46901a6960 con 0x7f4690108b30 2024-06-03T03:18:28.985 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:28.983+0000 7f4694a00700 1 --2- 172.21.15.117:0/2611619870 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4690108b30 0x7f469019ed10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f468400e970 tx=0x7f468400ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:18:28.985 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:28.983+0000 7f468d600700 1 -- 172.21.15.117:0/2611619870 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f468400c9e0 con 0x7f4690108b30 2024-06-03T03:18:28.985 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:28.983+0000 7f4697400700 1 -- 172.21.15.117:0/2611619870 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46901a6c50 con 0x7f4690108b30 2024-06-03T03:18:28.985 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:28.983+0000 7f468d600700 1 -- 172.21.15.117:0/2611619870 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f468400cb80 con 0x7f4690108b30 2024-06-03T03:18:28.985 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:28.983+0000 7f468d600700 1 -- 172.21.15.117:0/2611619870 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4684018680 con 0x7f4690108b30 2024-06-03T03:18:28.985 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:28.983+0000 7f4697400700 1 -- 172.21.15.117:0/2611619870 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46901a7190 con 0x7f4690108b30 2024-06-03T03:18:28.986 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:28.985+0000 7f4697400700 1 -- 172.21.15.117:0/2611619870 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4690069180 con 0x7f4690108b30 2024-06-03T03:18:28.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:28.986+0000 7f468d600700 1 -- 172.21.15.117:0/2611619870 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f468401f020 con 0x7f4690108b30 2024-06-03T03:18:28.988 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:28.988+0000 7f468d600700 1 --2- 172.21.15.117:0/2611619870 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f46780a2c20 0x7f46780a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:18:28.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:28.988+0000 7f468d600700 1 -- 172.21.15.117:0/2611619870 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7127+0+0 (secure 0 0 0) 0x7f46840cea20 con 0x7f4690108b30 2024-06-03T03:18:28.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:28.988+0000 7f468fe00700 1 --2- 172.21.15.117:0/2611619870 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f46780a2c20 0x7f46780a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:18:28.990 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:28.989+0000 7f468fe00700 1 --2- 172.21.15.117:0/2611619870 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f46780a2c20 0x7f46780a50e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4680000c00 tx=0x7f4680005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:18:28.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:28.992+0000 7f468d600700 1 -- 172.21.15.117:0/2611619870 <== mon.2 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f468409bd80 con 0x7f4690108b30 2024-06-03T03:18:29.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:29.227+0000 7f4697400700 1 -- 172.21.15.117:0/2611619870 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f46900630a0 con 0x7f4690108b30 2024-06-03T03:18:29.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:29.228+0000 7f468d600700 1 -- 172.21.15.117:0/2611619870 <== mon.2 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v80) ==== 74+0+133 (secure 0 0 0) 0x7f468409b680 con 0x7f4690108b30 2024-06-03T03:18:29.229 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T03:18:29.232 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:29.230+0000 7f4697400700 1 -- 172.21.15.117:0/2611619870 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f46780a2c20 msgr2=0x7f46780a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:18:29.232 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:29.230+0000 7f4697400700 1 --2- 172.21.15.117:0/2611619870 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f46780a2c20 0x7f46780a50e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4680000c00 tx=0x7f4680005fb0 comp rx=0 tx=0).stop 2024-06-03T03:18:29.232 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:29.230+0000 7f4697400700 1 -- 172.21.15.117:0/2611619870 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4690108b30 msgr2=0x7f469019ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:18:29.232 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:29.230+0000 7f4697400700 1 --2- 172.21.15.117:0/2611619870 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4690108b30 0x7f469019ed10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f468400e970 tx=0x7f468400ec80 comp rx=0 tx=0).stop 2024-06-03T03:18:29.232 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:29.231+0000 7f4697400700 1 -- 172.21.15.117:0/2611619870 shutdown_connections 2024-06-03T03:18:29.232 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:29.231+0000 7f4697400700 1 --2- 172.21.15.117:0/2611619870 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f46780a2c20 0x7f46780a50e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:18:29.232 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:29.231+0000 7f4697400700 1 --2- 172.21.15.117:0/2611619870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f469010a6d0 0x7f46901a6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:18:29.232 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:29.231+0000 7f4697400700 1 --2- 172.21.15.117:0/2611619870 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4690108b30 0x7f469019ed10 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:18:29.232 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:29.231+0000 7f4697400700 1 --2- 172.21.15.117:0/2611619870 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4690109d30 0x7f469019f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:18:29.232 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:29.231+0000 7f4697400700 1 -- 172.21.15.117:0/2611619870 >> 172.21.15.117:0/2611619870 conn(0x7f4690077b00 msgr2=0x7f469010d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:18:29.232 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:29.231+0000 7f4697400700 1 -- 172.21.15.117:0/2611619870 shutdown_connections 2024-06-03T03:18:29.232 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:29.231+0000 7f4697400700 1 -- 172.21.15.117:0/2611619870 wait complete. 2024-06-03T03:18:29.513 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:29 smithi117 ceph-mon[31649]: purged_snaps scrub starts 2024-06-03T03:18:29.514 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:29 smithi117 ceph-mon[31649]: purged_snaps scrub ok 2024-06-03T03:18:29.514 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:29 smithi117 ceph-mon[31649]: osd.11 [v2:172.21.15.183:6824/3513899696,v1:172.21.15.183:6825/3513899696] boot 2024-06-03T03:18:29.514 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:29 smithi117 ceph-mon[31649]: osdmap e80: 12 total, 12 up, 12 in 2024-06-03T03:18:29.514 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:29 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-03T03:18:29.514 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:29 smithi117 ceph-mon[31649]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-06-03T03:18:29.514 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:29 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:29.514 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:29 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:29.514 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:29 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:18:29.514 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:29 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:18:29.514 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:29 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:18:29.514 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:29 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:18:29.514 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:29 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:29.514 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:29 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:18:29.515 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:29 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:18:29.515 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:29 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:29.515 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:29 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/2611619870' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-03T03:18:29.656 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:29 smithi151 ceph-mon[37043]: purged_snaps scrub starts 2024-06-03T03:18:29.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:29 smithi151 ceph-mon[37043]: purged_snaps scrub ok 2024-06-03T03:18:29.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:29 smithi151 ceph-mon[37043]: osd.11 [v2:172.21.15.183:6824/3513899696,v1:172.21.15.183:6825/3513899696] boot 2024-06-03T03:18:29.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:29 smithi151 ceph-mon[37043]: osdmap e80: 12 total, 12 up, 12 in 2024-06-03T03:18:29.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:29 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-03T03:18:29.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:29 smithi151 ceph-mon[37043]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-06-03T03:18:29.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:29 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:29.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:29 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:29.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:29 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:18:29.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:29 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:18:29.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:29 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:18:29.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:29 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:18:29.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:29 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:29.658 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:29 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:18:29.658 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:29 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:18:29.658 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:29 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:29.658 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:29 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/2611619870' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-03T03:18:29.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:29 smithi183 ceph-mon[37326]: purged_snaps scrub starts 2024-06-03T03:18:29.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:29 smithi183 ceph-mon[37326]: purged_snaps scrub ok 2024-06-03T03:18:29.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:29 smithi183 ceph-mon[37326]: osd.11 [v2:172.21.15.183:6824/3513899696,v1:172.21.15.183:6825/3513899696] boot 2024-06-03T03:18:29.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:29 smithi183 ceph-mon[37326]: osdmap e80: 12 total, 12 up, 12 in 2024-06-03T03:18:29.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:29 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-03T03:18:29.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:29 smithi183 ceph-mon[37326]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-06-03T03:18:29.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:29 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:29.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:29 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:29.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:29 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:18:29.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:29 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:18:29.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:29 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:18:29.733 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:29 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:18:29.733 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:29 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:29.733 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:29 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:18:29.733 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:29 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:18:29.733 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:29 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:29.733 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:29 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/2611619870' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-03T03:18:30.070 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":80,"num_osds":12,"num_up_osds":12,"osd_up_since":1717384708,"num_in_osds":12,"osd_in_since":1717384688,"num_remapped_pgs":0} 2024-06-03T03:18:30.070 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-03T03:18:30.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:30 smithi117 ceph-mon[31649]: Detected new or changed devices on smithi183 2024-06-03T03:18:30.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:30 smithi117 ceph-mon[31649]: Adjusting osd_memory_target on smithi183 to 4272M 2024-06-03T03:18:30.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:30 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:18:30.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:30 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:18:30.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:30 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:18:30.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:30 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:30.506 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:18:30.656 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:30 smithi151 ceph-mon[37043]: Detected new or changed devices on smithi183 2024-06-03T03:18:30.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:30 smithi151 ceph-mon[37043]: Adjusting osd_memory_target on smithi183 to 4272M 2024-06-03T03:18:30.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:30 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:18:30.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:30 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:18:30.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:30 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:18:30.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:30 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:30.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:30 smithi183 ceph-mon[37326]: Detected new or changed devices on smithi183 2024-06-03T03:18:30.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:30 smithi183 ceph-mon[37326]: Adjusting osd_memory_target on smithi183 to 4272M 2024-06-03T03:18:30.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:30 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:18:30.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:30 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:18:30.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:30 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:18:30.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:30 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:31.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:31 smithi117 ceph-mon[31649]: osdmap e81: 12 total, 12 up, 12 in 2024-06-03T03:18:31.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:31 smithi117 ceph-mon[31649]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:18:31.656 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:31 smithi151 ceph-mon[37043]: osdmap e81: 12 total, 12 up, 12 in 2024-06-03T03:18:31.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:31 smithi151 ceph-mon[37043]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:18:31.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:31 smithi183 ceph-mon[37326]: osdmap e81: 12 total, 12 up, 12 in 2024-06-03T03:18:31.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:31 smithi183 ceph-mon[37326]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:18:32.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:32.903+0000 7fc479c00700 1 -- 172.21.15.117:0/2312117809 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc4741025e0 msgr2=0x7fc4741029c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:18:32.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:32.903+0000 7fc479c00700 1 --2- 172.21.15.117:0/2312117809 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc4741025e0 0x7fc4741029c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc464009990 tx=0x7fc464009ca0 comp rx=0 tx=0).stop 2024-06-03T03:18:32.905 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:32.903+0000 7fc479c00700 1 -- 172.21.15.117:0/2312117809 shutdown_connections 2024-06-03T03:18:32.905 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:32.903+0000 7fc479c00700 1 --2- 172.21.15.117:0/2312117809 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc4741076c0 0x7fc47410f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:18:32.905 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:32.903+0000 7fc479c00700 1 --2- 172.21.15.117:0/2312117809 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc4741025e0 0x7fc4741029c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:18:32.905 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:32.903+0000 7fc479c00700 1 --2- 172.21.15.117:0/2312117809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc474102f90 0x7fc474106fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:18:32.905 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:32.903+0000 7fc479c00700 1 -- 172.21.15.117:0/2312117809 >> 172.21.15.117:0/2312117809 conn(0x7fc4740fdeb0 msgr2=0x7fc4741002d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:18:32.905 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:32.903+0000 7fc479c00700 1 -- 172.21.15.117:0/2312117809 shutdown_connections 2024-06-03T03:18:32.905 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:32.903+0000 7fc479c00700 1 -- 172.21.15.117:0/2312117809 wait complete. 2024-06-03T03:18:32.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:32.904+0000 7fc479c00700 1 Processor -- start 2024-06-03T03:18:32.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:32.905+0000 7fc479c00700 1 -- start start 2024-06-03T03:18:32.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:32.905+0000 7fc479c00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc4741025e0 0x7fc47419adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:18:32.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:32.905+0000 7fc479c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc474102f90 0x7fc47419b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:18:32.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:32.905+0000 7fc479c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4741076c0 0x7fc47419f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:18:32.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:32.905+0000 7fc479c00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc474112720 con 0x7fc4741076c0 2024-06-03T03:18:32.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:32.905+0000 7fc479c00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc4741125a0 con 0x7fc474102f90 2024-06-03T03:18:32.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:32.905+0000 7fc479c00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc4741128a0 con 0x7fc4741025e0 2024-06-03T03:18:32.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:32.906+0000 7fc479200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4741076c0 0x7fc47419f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:18:32.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:32.906+0000 7fc479200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4741076c0 0x7fc47419f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:38202/0 (socket says 172.21.15.117:38202) 2024-06-03T03:18:32.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:32.906+0000 7fc473e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc4741025e0 0x7fc47419adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:18:32.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:32.906+0000 7fc479200700 1 -- 172.21.15.117:0/3100633081 learned_addr learned my addr 172.21.15.117:0/3100633081 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:18:32.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:32.906+0000 7fc473400700 1 --2- 172.21.15.117:0/3100633081 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc474102f90 0x7fc47419b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:18:32.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:32.906+0000 7fc473e00700 1 -- 172.21.15.117:0/3100633081 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc474102f90 msgr2=0x7fc47419b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:18:32.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:32.906+0000 7fc473e00700 1 --2- 172.21.15.117:0/3100633081 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc474102f90 0x7fc47419b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:18:32.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:32.906+0000 7fc473e00700 1 -- 172.21.15.117:0/3100633081 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4741076c0 msgr2=0x7fc47419f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:18:32.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:32.906+0000 7fc473e00700 1 --2- 172.21.15.117:0/3100633081 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4741076c0 0x7fc47419f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:18:32.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:32.906+0000 7fc473e00700 1 -- 172.21.15.117:0/3100633081 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc47419fe00 con 0x7fc4741025e0 2024-06-03T03:18:32.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:32.906+0000 7fc479200700 1 --2- 172.21.15.117:0/3100633081 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4741076c0 0x7fc47419f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:18:32.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:32.906+0000 7fc473e00700 1 --2- 172.21.15.117:0/3100633081 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc4741025e0 0x7fc47419adf0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc4640098e0 tx=0x7fc46400b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:18:32.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:32.907+0000 7fc470c00700 1 -- 172.21.15.117:0/3100633081 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc464027070 con 0x7fc4741025e0 2024-06-03T03:18:32.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:32.907+0000 7fc470c00700 1 -- 172.21.15.117:0/3100633081 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc46400fd10 con 0x7fc4741025e0 2024-06-03T03:18:32.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:32.907+0000 7fc479c00700 1 -- 172.21.15.117:0/3100633081 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4741a0090 con 0x7fc4741025e0 2024-06-03T03:18:32.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:32.907+0000 7fc470c00700 1 -- 172.21.15.117:0/3100633081 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4640173f0 con 0x7fc4741025e0 2024-06-03T03:18:32.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:32.907+0000 7fc479c00700 1 -- 172.21.15.117:0/3100633081 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4741a7970 con 0x7fc4741025e0 2024-06-03T03:18:32.910 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:32.908+0000 7fc479c00700 1 -- 172.21.15.117:0/3100633081 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc474069180 con 0x7fc4741025e0 2024-06-03T03:18:32.910 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:32.910+0000 7fc470c00700 1 -- 172.21.15.117:0/3100633081 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fc46402a070 con 0x7fc4741025e0 2024-06-03T03:18:32.912 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:32.911+0000 7fc470c00700 1 --2- 172.21.15.117:0/3100633081 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fc4580a2ca0 0x7fc4580a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:18:32.912 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:32.912+0000 7fc470c00700 1 -- 172.21.15.117:0/3100633081 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fc4640cfbe0 con 0x7fc4741025e0 2024-06-03T03:18:32.912 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:32.912+0000 7fc473400700 1 --2- 172.21.15.117:0/3100633081 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fc4580a2ca0 0x7fc4580a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:18:32.913 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:32.912+0000 7fc473400700 1 --2- 172.21.15.117:0/3100633081 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fc4580a2ca0 0x7fc4580a5160 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc4740ff5d0 tx=0x7fc460009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:18:32.915 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:32.914+0000 7fc470c00700 1 -- 172.21.15.117:0/3100633081 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc464031080 con 0x7fc4741025e0 2024-06-03T03:18:33.138 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:33.137+0000 7fc479c00700 1 -- 172.21.15.117:0/3100633081 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc4741a7b40 con 0x7fc4741025e0 2024-06-03T03:18:33.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:33.138+0000 7fc470c00700 1 -- 172.21.15.117:0/3100633081 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18515 (secure 0 0 0) 0x7fc464013070 con 0x7fc4741025e0 2024-06-03T03:18:33.140 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T03:18:33.140 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":81,"fsid":"a0e8473a-2156-11ef-bc9b-c7b262605968","created":"2024-06-03T03:10:08.446859+0000","modified":"2024-06-03T03:18:30.388105+0000","last_up_change":"2024-06-03T03:18:28.736904+0000","last_in_change":"2024-06-03T03:18:08.485711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-03T03:14:09.370986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8fefa089-9401-4a1c-b9fb-fe30c63e6a1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":4286644492},{"type":"v1","addr":"172.21.15.117:6803","nonce":4286644492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":4286644492},{"type":"v1","addr":"172.21.15.117:6805","nonce":4286644492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":4286644492},{"type":"v1","addr":"172.21.15.117:6809","nonce":4286644492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":4286644492},{"type":"v1","addr":"172.21.15.117:6807","nonce":4286644492}]},"public_addr":"172.21.15.117:6803/4286644492","cluster_addr":"172.21.15.117:6805/4286644492","heartbeat_back_addr":"172.21.15.117:6809/4286644492","heartbeat_front_addr":"172.21.15.117:6807/4286644492","state":["exists","up"]},{"osd":1,"uuid":"5b12b371-f155-4b67-a470-25cbe59a2999","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":262491158},{"type":"v1","addr":"172.21.15.151:6801","nonce":262491158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":262491158},{"type":"v1","addr":"172.21.15.151:6803","nonce":262491158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":262491158},{"type":"v1","addr":"172.21.15.151:6807","nonce":262491158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":262491158},{"type":"v1","addr":"172.21.15.151:6805","nonce":262491158}]},"public_addr":"172.21.15.151:6801/262491158","cluster_addr":"172.21.15.151:6803/262491158","heartbeat_back_addr":"172.21.15.151:6807/262491158","heartbeat_front_addr":"172.21.15.151:6805/262491158","state":["exists","up"]},{"osd":2,"uuid":"dbc17818-c54c-4ecf-b656-03578b95d8c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":3542453623},{"type":"v1","addr":"172.21.15.183:6801","nonce":3542453623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":3542453623},{"type":"v1","addr":"172.21.15.183:6803","nonce":3542453623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":3542453623},{"type":"v1","addr":"172.21.15.183:6807","nonce":3542453623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":3542453623},{"type":"v1","addr":"172.21.15.183:6805","nonce":3542453623}]},"public_addr":"172.21.15.183:6801/3542453623","cluster_addr":"172.21.15.183:6803/3542453623","heartbeat_back_addr":"172.21.15.183:6807/3542453623","heartbeat_front_addr":"172.21.15.183:6805/3542453623","state":["exists","up"]},{"osd":3,"uuid":"ca08d2ac-b1d6-4a49-9ae8-e3ec146899a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":589012315},{"type":"v1","addr":"172.21.15.117:6811","nonce":589012315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":589012315},{"type":"v1","addr":"172.21.15.117:6813","nonce":589012315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":589012315},{"type":"v1","addr":"172.21.15.117:6817","nonce":589012315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":589012315},{"type":"v1","addr":"172.21.15.117:6815","nonce":589012315}]},"public_addr":"172.21.15.117:6811/589012315","cluster_addr":"172.21.15.117:6813/589012315","heartbeat_back_addr":"172.21.15.117:6817/589012315","heartbeat_front_addr":"172.21.15.117:6815/589012315","state":["exists","up"]},{"osd":4,"uuid":"810d6c7a-023f-4930-82af-bc2bbfd647da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":1239048647},{"type":"v1","addr":"172.21.15.151:6809","nonce":1239048647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":1239048647},{"type":"v1","addr":"172.21.15.151:6811","nonce":1239048647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":1239048647},{"type":"v1","addr":"172.21.15.151:6815","nonce":1239048647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":1239048647},{"type":"v1","addr":"172.21.15.151:6813","nonce":1239048647}]},"public_addr":"172.21.15.151:6809/1239048647","cluster_addr":"172.21.15.151:6811/1239048647","heartbeat_back_addr":"172.21.15.151:6815/1239048647","heartbeat_front_addr":"172.21.15.151:6813/1239048647","state":["exists","up"]},{"osd":5,"uuid":"8ab6a5f0-34bb-48d8-9471-a7dc836e486e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":2985874686},{"type":"v1","addr":"172.21.15.183:6809","nonce":2985874686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":2985874686},{"type":"v1","addr":"172.21.15.183:6811","nonce":2985874686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":2985874686},{"type":"v1","addr":"172.21.15.183:6815","nonce":2985874686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":2985874686},{"type":"v1","addr":"172.21.15.183:6813","nonce":2985874686}]},"public_addr":"172.21.15.183:6809/2985874686","cluster_addr":"172.21.15.183:6811/2985874686","heartbeat_back_addr":"172.21.15.183:6815/2985874686","heartbeat_front_addr":"172.21.15.183:6813/2985874686","state":["exists","up"]},{"osd":6,"uuid":"78d15739-e9e0-4f19-ae5f-ee8527a5b527","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4250583193},{"type":"v1","addr":"172.21.15.117:6819","nonce":4250583193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":4250583193},{"type":"v1","addr":"172.21.15.117:6821","nonce":4250583193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":4250583193},{"type":"v1","addr":"172.21.15.117:6825","nonce":4250583193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4250583193},{"type":"v1","addr":"172.21.15.117:6823","nonce":4250583193}]},"public_addr":"172.21.15.117:6819/4250583193","cluster_addr":"172.21.15.117:6821/4250583193","heartbeat_back_addr":"172.21.15.117:6825/4250583193","heartbeat_front_addr":"172.21.15.117:6823/4250583193","state":["exists","up"]},{"osd":7,"uuid":"3724fe1f-f4d7-443b-aea8-ff3ab02bfdc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":386094645},{"type":"v1","addr":"172.21.15.151:6817","nonce":386094645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":386094645},{"type":"v1","addr":"172.21.15.151:6819","nonce":386094645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":386094645},{"type":"v1","addr":"172.21.15.151:6823","nonce":386094645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":386094645},{"type":"v1","addr":"172.21.15.151:6821","nonce":386094645}]},"public_addr":"172.21.15.151:6817/386094645","cluster_addr":"172.21.15.151:6819/386094645","heartbeat_back_addr":"172.21.15.151:6823/386094645","heartbeat_front_addr":"172.21.15.151:6821/386094645","state":["exists","up"]},{"osd":8,"uuid":"006ac092-f015-448e-a0c0-f7b0c4597a18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1829748222},{"type":"v1","addr":"172.21.15.183:6817","nonce":1829748222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1829748222},{"type":"v1","addr":"172.21.15.183:6819","nonce":1829748222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1829748222},{"type":"v1","addr":"172.21.15.183:6823","nonce":1829748222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1829748222},{"type":"v1","addr":"172.21.15.183:6821","nonce":1829748222}]},"public_addr":"172.21.15.183:6817/1829748222","cluster_addr":"172.21.15.183:6819/1829748222","heartbeat_back_addr":"172.21.15.183:6823/1829748222","heartbeat_front_addr":"172.21.15.183:6821/1829748222","state":["exists","up"]},{"osd":9,"uuid":"00a970e6-2502-459e-975c-badac93c9bd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2538315141},{"type":"v1","addr":"172.21.15.117:6827","nonce":2538315141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2538315141},{"type":"v1","addr":"172.21.15.117:6829","nonce":2538315141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2538315141},{"type":"v1","addr":"172.21.15.117:6833","nonce":2538315141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2538315141},{"type":"v1","addr":"172.21.15.117:6831","nonce":2538315141}]},"public_addr":"172.21.15.117:6827/2538315141","cluster_addr":"172.21.15.117:6829/2538315141","heartbeat_back_addr":"172.21.15.117:6833/2538315141","heartbeat_front_addr":"172.21.15.117:6831/2538315141","state":["exists","up"]},{"osd":10,"uuid":"ba2333cd-32a3-41ef-9ea5-287c707f512b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":3801045133},{"type":"v1","addr":"172.21.15.151:6825","nonce":3801045133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3801045133},{"type":"v1","addr":"172.21.15.151:6827","nonce":3801045133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3801045133},{"type":"v1","addr":"172.21.15.151:6831","nonce":3801045133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3801045133},{"type":"v1","addr":"172.21.15.151:6829","nonce":3801045133}]},"public_addr":"172.21.15.151:6825/3801045133","cluster_addr":"172.21.15.151:6827/3801045133","heartbeat_back_addr":"172.21.15.151:6831/3801045133","heartbeat_front_addr":"172.21.15.151:6829/3801045133","state":["exists","up"]},{"osd":11,"uuid":"1556d654-e6fa-42c1-b399-aa53f6837a1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3513899696},{"type":"v1","addr":"172.21.15.183:6825","nonce":3513899696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3513899696},{"type":"v1","addr":"172.21.15.183:6827","nonce":3513899696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3513899696},{"type":"v1","addr":"172.21.15.183:6831","nonce":3513899696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3513899696},{"type":"v1","addr":"172.21.15.183:6829","nonce":3513899696}]},"public_addr":"172.21.15.183:6825/3513899696","cluster_addr":"172.21.15.183:6827/3513899696","heartbeat_back_addr":"172.21.15.183:6831/3513899696","heartbeat_front_addr":"172.21.15.183:6829/3513899696","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:13:17.174041+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:13:42.237001+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:14:06.273315+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:14:39.912564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:15:06.063202+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:15:30.424818+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:16:05.623554+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:16:32.926393+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:16:58.165899+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:17:32.897477+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:18:00.720866+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:18:26.474441+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.117:0/186309008":"2024-06-04T03:11:24.655516+0000","172.21.15.117:6801/2142313590":"2024-06-04T03:11:24.655516+0000","172.21.15.117:6800/2142313590":"2024-06-04T03:11:24.655516+0000","172.21.15.117:0/2399725156":"2024-06-04T03:11:24.655516+0000","172.21.15.117:0/1453232061":"2024-06-04T03:10:50.552899+0000","172.21.15.117:0/4281674974":"2024-06-04T03:10:50.552899+0000","172.21.15.117:6800/2424554985":"2024-06-04T03:10:50.552899+0000","172.21.15.117:0/3946365225":"2024-06-04T03:11:24.655516+0000","172.21.15.117:0/1146162354":"2024-06-04T03:10:50.552899+0000","172.21.15.117:6801/2424554985":"2024-06-04T03:10:50.552899+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-03T03:18:33.142 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:33.142+0000 7fc479c00700 1 -- 172.21.15.117:0/3100633081 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fc4580a2ca0 msgr2=0x7fc4580a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:18:33.142 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:33.142+0000 7fc479c00700 1 --2- 172.21.15.117:0/3100633081 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fc4580a2ca0 0x7fc4580a5160 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc4740ff5d0 tx=0x7fc460009260 comp rx=0 tx=0).stop 2024-06-03T03:18:33.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:33.142+0000 7fc479c00700 1 -- 172.21.15.117:0/3100633081 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc4741025e0 msgr2=0x7fc47419adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:18:33.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:33.142+0000 7fc479c00700 1 --2- 172.21.15.117:0/3100633081 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc4741025e0 0x7fc47419adf0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc4640098e0 tx=0x7fc46400b840 comp rx=0 tx=0).stop 2024-06-03T03:18:33.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:33.142+0000 7fc479c00700 1 -- 172.21.15.117:0/3100633081 shutdown_connections 2024-06-03T03:18:33.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:33.142+0000 7fc479c00700 1 --2- 172.21.15.117:0/3100633081 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fc4580a2ca0 0x7fc4580a5160 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:18:33.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:33.142+0000 7fc479c00700 1 --2- 172.21.15.117:0/3100633081 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4741076c0 0x7fc47419f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:18:33.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:33.142+0000 7fc479c00700 1 --2- 172.21.15.117:0/3100633081 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc4741025e0 0x7fc47419adf0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:18:33.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:33.142+0000 7fc479c00700 1 --2- 172.21.15.117:0/3100633081 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc474102f90 0x7fc47419b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:18:33.144 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:33.142+0000 7fc479c00700 1 -- 172.21.15.117:0/3100633081 >> 172.21.15.117:0/3100633081 conn(0x7fc4740fdeb0 msgr2=0x7fc47410e560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:18:33.144 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:33.142+0000 7fc479c00700 1 -- 172.21.15.117:0/3100633081 shutdown_connections 2024-06-03T03:18:33.144 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:33.142+0000 7fc479c00700 1 -- 172.21.15.117:0/3100633081 wait complete. 2024-06-03T03:18:34.012 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-06-03T03:14:09.370986+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-06-03T03:18:34.012 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-06-03T03:18:34.087 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:33 smithi117 ceph-mon[31649]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:18:34.087 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:33 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/3100633081' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T03:18:34.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:33 smithi151 ceph-mon[37043]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:18:34.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:33 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/3100633081' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T03:18:34.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:33 smithi183 ceph-mon[37326]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:18:34.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:33 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/3100633081' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T03:18:34.441 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:18:36.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:35 smithi151 ceph-mon[37043]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:18:36.192 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:35 smithi117 ceph-mon[31649]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:18:36.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:35 smithi183 ceph-mon[37326]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:18:36.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:36.751+0000 7f3529600700 1 -- 172.21.15.117:0/3101477366 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35240751e0 msgr2=0x7f352410e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:18:36.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:36.751+0000 7f3529600700 1 --2- 172.21.15.117:0/3101477366 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35240751e0 0x7f352410e6c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f350c009990 tx=0x7f350c009ca0 comp rx=0 tx=0).stop 2024-06-03T03:18:36.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:36.752+0000 7f3529600700 1 -- 172.21.15.117:0/3101477366 shutdown_connections 2024-06-03T03:18:36.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:36.752+0000 7f3529600700 1 --2- 172.21.15.117:0/3101477366 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f352410ec00 0x7f3524111070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:18:36.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:36.752+0000 7f3529600700 1 --2- 172.21.15.117:0/3101477366 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f35240748c0 0x7f3524074ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:18:36.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:36.752+0000 7f3529600700 1 --2- 172.21.15.117:0/3101477366 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35240751e0 0x7f352410e6c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:18:36.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:36.752+0000 7f3529600700 1 -- 172.21.15.117:0/3101477366 >> 172.21.15.117:0/3101477366 conn(0x7f35240fdf10 msgr2=0x7f3524100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:18:36.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:36.752+0000 7f3529600700 1 -- 172.21.15.117:0/3101477366 shutdown_connections 2024-06-03T03:18:36.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:36.752+0000 7f3529600700 1 -- 172.21.15.117:0/3101477366 wait complete. 2024-06-03T03:18:36.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:36.753+0000 7f3529600700 1 Processor -- start 2024-06-03T03:18:36.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:36.754+0000 7f3529600700 1 -- start start 2024-06-03T03:18:36.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:36.754+0000 7f3529600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35240748c0 0x7f352419ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:18:36.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:36.754+0000 7f3529600700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f35240751e0 0x7f352419b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:18:36.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:36.754+0000 7f3529600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f352410ec00 0x7f352419f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:18:36.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:36.754+0000 7f3529600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f35241148d0 con 0x7f35240748c0 2024-06-03T03:18:36.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:36.754+0000 7f3529600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3524114750 con 0x7f352410ec00 2024-06-03T03:18:36.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:36.754+0000 7f3529600700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3524114a50 con 0x7f35240751e0 2024-06-03T03:18:36.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:36.754+0000 7f3523e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35240748c0 0x7f352419ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:18:36.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:36.754+0000 7f3523e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35240748c0 0x7f352419ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:38222/0 (socket says 172.21.15.117:38222) 2024-06-03T03:18:36.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:36.754+0000 7f3523e00700 1 -- 172.21.15.117:0/2470072259 learned_addr learned my addr 172.21.15.117:0/2470072259 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:18:36.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:36.754+0000 7f351b400700 1 --2- 172.21.15.117:0/2470072259 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f35240751e0 0x7f352419b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:18:36.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:36.754+0000 7f3528c00700 1 --2- 172.21.15.117:0/2470072259 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f352410ec00 0x7f352419f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:18:36.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:36.755+0000 7f3528c00700 1 -- 172.21.15.117:0/2470072259 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f35240751e0 msgr2=0x7f352419b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:18:36.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:36.755+0000 7f3528c00700 1 --2- 172.21.15.117:0/2470072259 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f35240751e0 0x7f352419b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:18:36.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:36.755+0000 7f3528c00700 1 -- 172.21.15.117:0/2470072259 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35240748c0 msgr2=0x7f352419ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:18:36.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:36.755+0000 7f3528c00700 1 --2- 172.21.15.117:0/2470072259 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35240748c0 0x7f352419ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:18:36.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:36.755+0000 7f3528c00700 1 -- 172.21.15.117:0/2470072259 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f352419fd40 con 0x7f352410ec00 2024-06-03T03:18:36.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:36.755+0000 7f3523e00700 1 --2- 172.21.15.117:0/2470072259 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35240748c0 0x7f352419ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:18:36.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:36.755+0000 7f3528c00700 1 --2- 172.21.15.117:0/2470072259 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f352410ec00 0x7f352419f600 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f351c00e970 tx=0x7f351c00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:18:36.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:36.755+0000 7f3521600700 1 -- 172.21.15.117:0/2470072259 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f351c004220 con 0x7f352410ec00 2024-06-03T03:18:36.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:36.755+0000 7f3521600700 1 -- 172.21.15.117:0/2470072259 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f351c004d10 con 0x7f352410ec00 2024-06-03T03:18:36.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:36.755+0000 7f3529600700 1 -- 172.21.15.117:0/2470072259 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35241a0030 con 0x7f352410ec00 2024-06-03T03:18:36.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:36.755+0000 7f3521600700 1 -- 172.21.15.117:0/2470072259 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f351c010700 con 0x7f352410ec00 2024-06-03T03:18:36.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:36.756+0000 7f3529600700 1 -- 172.21.15.117:0/2470072259 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35241a78b0 con 0x7f352410ec00 2024-06-03T03:18:36.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:36.757+0000 7f3529600700 1 -- 172.21.15.117:0/2470072259 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3524069180 con 0x7f352410ec00 2024-06-03T03:18:36.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:36.759+0000 7f3521600700 1 -- 172.21.15.117:0/2470072259 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f351c004750 con 0x7f352410ec00 2024-06-03T03:18:36.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:36.760+0000 7f3521600700 1 --2- 172.21.15.117:0/2470072259 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f35080a2c20 0x7f35080a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:18:36.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:36.760+0000 7f3521600700 1 -- 172.21.15.117:0/2470072259 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f351c0ce9a0 con 0x7f352410ec00 2024-06-03T03:18:36.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:36.760+0000 7f3523e00700 1 --2- 172.21.15.117:0/2470072259 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f35080a2c20 0x7f35080a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:18:36.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:36.761+0000 7f3523e00700 1 --2- 172.21.15.117:0/2470072259 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f35080a2c20 0x7f35080a50e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3514009c10 tx=0x7f351400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:18:36.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:36.764+0000 7f3521600700 1 -- 172.21.15.117:0/2470072259 <== mon.2 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f351c01e300 con 0x7f352410ec00 2024-06-03T03:18:36.990 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:36.989+0000 7f3529600700 1 -- 172.21.15.117:0/2470072259 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f352419c090 con 0x7f352410ec00 2024-06-03T03:18:36.990 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:36.989+0000 7f3521600700 1 -- 172.21.15.117:0/2470072259 <== mon.2 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7f351c09bd00 con 0x7f352410ec00 2024-06-03T03:18:36.990 INFO:teuthology.orchestra.run.smithi117.stdout:pg_num: 1 2024-06-03T03:18:36.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:36.992+0000 7f3529600700 1 -- 172.21.15.117:0/2470072259 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f35080a2c20 msgr2=0x7f35080a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:18:36.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:36.992+0000 7f3529600700 1 --2- 172.21.15.117:0/2470072259 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f35080a2c20 0x7f35080a50e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3514009c10 tx=0x7f351400b040 comp rx=0 tx=0).stop 2024-06-03T03:18:36.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:36.992+0000 7f3529600700 1 -- 172.21.15.117:0/2470072259 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f352410ec00 msgr2=0x7f352419f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:18:36.994 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:36.992+0000 7f3529600700 1 --2- 172.21.15.117:0/2470072259 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f352410ec00 0x7f352419f600 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f351c00e970 tx=0x7f351c00ec80 comp rx=0 tx=0).stop 2024-06-03T03:18:36.994 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:36.993+0000 7f3529600700 1 -- 172.21.15.117:0/2470072259 shutdown_connections 2024-06-03T03:18:36.994 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:36.993+0000 7f3529600700 1 --2- 172.21.15.117:0/2470072259 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f35080a2c20 0x7f35080a50e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:18:36.994 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:36.993+0000 7f3529600700 1 --2- 172.21.15.117:0/2470072259 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f352410ec00 0x7f352419f600 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:18:36.994 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:36.993+0000 7f3529600700 1 --2- 172.21.15.117:0/2470072259 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35240748c0 0x7f352419ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:18:36.994 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:36.993+0000 7f3529600700 1 --2- 172.21.15.117:0/2470072259 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f35240751e0 0x7f352419b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:18:36.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:36.993+0000 7f3529600700 1 -- 172.21.15.117:0/2470072259 >> 172.21.15.117:0/2470072259 conn(0x7f35240fdf10 msgr2=0x7f35240febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:18:36.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:36.993+0000 7f3529600700 1 -- 172.21.15.117:0/2470072259 shutdown_connections 2024-06-03T03:18:36.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:36.993+0000 7f3529600700 1 -- 172.21.15.117:0/2470072259 wait complete. 2024-06-03T03:18:37.923 INFO:tasks.cephadm:Adding mds.a on smithi117 2024-06-03T03:18:37.923 INFO:tasks.cephadm:Adding mds.d on smithi117 2024-06-03T03:18:37.923 INFO:tasks.cephadm:Adding mds.g on smithi117 2024-06-03T03:18:37.923 INFO:tasks.cephadm:Adding mds.j on smithi117 2024-06-03T03:18:37.923 INFO:tasks.cephadm:Adding mds.b on smithi151 2024-06-03T03:18:37.923 INFO:tasks.cephadm:Adding mds.e on smithi151 2024-06-03T03:18:37.923 INFO:tasks.cephadm:Adding mds.h on smithi151 2024-06-03T03:18:37.923 INFO:tasks.cephadm:Adding mds.k on smithi151 2024-06-03T03:18:37.924 INFO:tasks.cephadm:Adding mds.c on smithi183 2024-06-03T03:18:37.924 INFO:tasks.cephadm:Adding mds.f on smithi183 2024-06-03T03:18:37.924 INFO:tasks.cephadm:Adding mds.i on smithi183 2024-06-03T03:18:37.924 INFO:tasks.cephadm:Adding mds.l on smithi183 2024-06-03T03:18:37.924 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph orch apply mds all '12;smithi117=a;smithi117=d;smithi117=g;smithi117=j;smithi151=b;smithi151=e;smithi151=h;smithi151=k;smithi183=c;smithi183=f;smithi183=i;smithi183=l' 2024-06-03T03:18:38.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:37 smithi151 ceph-mon[37043]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:18:38.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:37 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/2470072259' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-03T03:18:38.174 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.c/config 2024-06-03T03:18:38.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:37 smithi117 ceph-mon[31649]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:18:38.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:37 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/2470072259' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-03T03:18:38.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:37 smithi183 ceph-mon[37326]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:18:38.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:37 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/2470072259' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-03T03:18:39.869 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:39.868+0000 7f3e8c659700 1 -- 172.21.15.183:0/4073042866 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e841024e0 msgr2=0x7f3e84102900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:18:39.869 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:39.868+0000 7f3e8c659700 1 --2- 172.21.15.183:0/4073042866 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e841024e0 0x7f3e84102900 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3e80009990 tx=0x7f3e80009ca0 comp rx=0 tx=0).stop 2024-06-03T03:18:39.869 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:39.868+0000 7f3e8c659700 1 -- 172.21.15.183:0/4073042866 shutdown_connections 2024-06-03T03:18:39.869 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:39.868+0000 7f3e8c659700 1 --2- 172.21.15.183:0/4073042866 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3e84104080 0x7f3e8410a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:18:39.869 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:39.868+0000 7f3e8c659700 1 --2- 172.21.15.183:0/4073042866 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e841024e0 0x7f3e84102900 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:18:39.869 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:39.868+0000 7f3e8c659700 1 --2- 172.21.15.183:0/4073042866 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e841036e0 0x7f3e84103b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:18:39.869 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:39.868+0000 7f3e8c659700 1 -- 172.21.15.183:0/4073042866 >> 172.21.15.183:0/4073042866 conn(0x7f3e840fda80 msgr2=0x7f3e840ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:18:39.869 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:39.868+0000 7f3e8c659700 1 -- 172.21.15.183:0/4073042866 shutdown_connections 2024-06-03T03:18:39.870 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:39 smithi183 ceph-mon[37326]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:18:39.871 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:39.868+0000 7f3e8c659700 1 -- 172.21.15.183:0/4073042866 wait complete. 2024-06-03T03:18:39.872 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:39.869+0000 7f3e8c659700 1 Processor -- start 2024-06-03T03:18:39.872 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:39.869+0000 7f3e8c659700 1 -- start start 2024-06-03T03:18:39.872 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:39.869+0000 7f3e8c659700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3e841024e0 0x7f3e84076230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:18:39.872 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:39.869+0000 7f3e8c659700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e841036e0 0x7f3e84074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:18:39.872 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:39.869+0000 7f3e8c659700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e84104080 0x7f3e84074dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:18:39.872 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:39.869+0000 7f3e8c659700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3e8410e1a0 con 0x7f3e84104080 2024-06-03T03:18:39.872 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:39.869+0000 7f3e8c659700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3e8410e020 con 0x7f3e841024e0 2024-06-03T03:18:39.872 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:39.869+0000 7f3e8c659700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3e8410e320 con 0x7f3e841036e0 2024-06-03T03:18:39.872 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:39.869+0000 7f3e89bf4700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e841036e0 0x7f3e84074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:18:39.872 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:39.869+0000 7f3e89bf4700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e841036e0 0x7f3e84074880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:59684/0 (socket says 172.21.15.183:59684) 2024-06-03T03:18:39.872 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:39.869+0000 7f3e89bf4700 1 -- 172.21.15.183:0/71444582 learned_addr learned my addr 172.21.15.183:0/71444582 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-06-03T03:18:39.872 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:39.869+0000 7f3e8a3f5700 1 --2- 172.21.15.183:0/71444582 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3e841024e0 0x7f3e84076230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:18:39.873 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:39.869+0000 7f3e89bf4700 1 -- 172.21.15.183:0/71444582 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3e841024e0 msgr2=0x7f3e84076230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:18:39.873 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:39.869+0000 7f3e89bf4700 1 --2- 172.21.15.183:0/71444582 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3e841024e0 0x7f3e84076230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:18:39.873 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:39.869+0000 7f3e89bf4700 1 -- 172.21.15.183:0/71444582 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e84104080 msgr2=0x7f3e84074dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:18:39.873 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:39.869+0000 7f3e89bf4700 1 --2- 172.21.15.183:0/71444582 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e84104080 0x7f3e84074dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:18:39.873 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:39.869+0000 7f3e89bf4700 1 -- 172.21.15.183:0/71444582 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e84075670 con 0x7f3e841036e0 2024-06-03T03:18:39.873 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:39.870+0000 7f3e89bf4700 1 --2- 172.21.15.183:0/71444582 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e841036e0 0x7f3e84074880 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3e7400b530 tx=0x7f3e7400b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:18:39.873 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:39.870+0000 7f3e7b7fe700 1 -- 172.21.15.183:0/71444582 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e74015020 con 0x7f3e841036e0 2024-06-03T03:18:39.873 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:39.870+0000 7f3e8c659700 1 -- 172.21.15.183:0/71444582 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e841a29e0 con 0x7f3e841036e0 2024-06-03T03:18:39.873 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:39.870+0000 7f3e7b7fe700 1 -- 172.21.15.183:0/71444582 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e74011d10 con 0x7f3e841036e0 2024-06-03T03:18:39.873 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:39.870+0000 7f3e7b7fe700 1 -- 172.21.15.183:0/71444582 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e7400f2e0 con 0x7f3e841036e0 2024-06-03T03:18:39.873 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:39.870+0000 7f3e8c659700 1 -- 172.21.15.183:0/71444582 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e841a2fa0 con 0x7f3e841036e0 2024-06-03T03:18:39.873 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:39.871+0000 7f3e8c659700 1 -- 172.21.15.183:0/71444582 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e84069180 con 0x7f3e841036e0 2024-06-03T03:18:39.874 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:39.872+0000 7f3e7b7fe700 1 -- 172.21.15.183:0/71444582 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f3e74018020 con 0x7f3e841036e0 2024-06-03T03:18:39.874 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:39.873+0000 7f3e7b7fe700 1 --2- 172.21.15.183:0/71444582 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f3e700a2c20 0x7f3e700a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:18:39.874 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:39.873+0000 7f3e7b7fe700 1 -- 172.21.15.183:0/71444582 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f3e740ce400 con 0x7f3e841036e0 2024-06-03T03:18:39.874 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:39.873+0000 7f3e8a3f5700 1 --2- 172.21.15.183:0/71444582 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f3e700a2c20 0x7f3e700a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:18:39.875 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:39.874+0000 7f3e8a3f5700 1 --2- 172.21.15.183:0/71444582 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f3e700a2c20 0x7f3e700a50e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3e80003890 tx=0x7f3e80005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:18:39.878 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:39.878+0000 7f3e7b7fe700 1 -- 172.21.15.183:0/71444582 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3e7401f020 con 0x7f3e841036e0 2024-06-03T03:18:40.079 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:40.079+0000 7f3e8c659700 1 -- 172.21.15.183:0/71444582 --> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi117=a;smithi117=d;smithi117=g;smithi117=j;smithi151=b;smithi151=e;smithi151=h;smithi151=k;smithi183=c;smithi183=f;smithi183=i;smithi183=l", "target": ["mon-mgr", ""]}) -- 0x7f3e84062ad0 con 0x7f3e700a2c20 2024-06-03T03:18:40.097 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:40.097+0000 7f3e7b7fe700 1 -- 172.21.15.183:0/71444582 <== mgr.14152 v2:172.21.15.117:6800/1398160887 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f3e84062ad0 con 0x7f3e700a2c20 2024-06-03T03:18:40.097 INFO:teuthology.orchestra.run.smithi183.stdout:Scheduled mds.all update... 2024-06-03T03:18:40.100 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:40.099+0000 7f3e8c659700 1 -- 172.21.15.183:0/71444582 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f3e700a2c20 msgr2=0x7f3e700a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:18:40.100 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:40.099+0000 7f3e8c659700 1 --2- 172.21.15.183:0/71444582 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f3e700a2c20 0x7f3e700a50e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3e80003890 tx=0x7f3e80005fd0 comp rx=0 tx=0).stop 2024-06-03T03:18:40.100 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:40.099+0000 7f3e8c659700 1 -- 172.21.15.183:0/71444582 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e841036e0 msgr2=0x7f3e84074880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:18:40.100 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:40.099+0000 7f3e8c659700 1 --2- 172.21.15.183:0/71444582 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e841036e0 0x7f3e84074880 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3e7400b530 tx=0x7f3e7400b840 comp rx=0 tx=0).stop 2024-06-03T03:18:40.100 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:40.099+0000 7f3e8c659700 1 -- 172.21.15.183:0/71444582 shutdown_connections 2024-06-03T03:18:40.100 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:40.099+0000 7f3e8c659700 1 --2- 172.21.15.183:0/71444582 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f3e700a2c20 0x7f3e700a50e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:18:40.100 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:40.099+0000 7f3e8c659700 1 --2- 172.21.15.183:0/71444582 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e84104080 0x7f3e84074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:18:40.100 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:40.099+0000 7f3e8c659700 1 --2- 172.21.15.183:0/71444582 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3e841024e0 0x7f3e84076230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:18:40.100 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:40.099+0000 7f3e8c659700 1 --2- 172.21.15.183:0/71444582 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e841036e0 0x7f3e84074880 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:18:40.100 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:40.099+0000 7f3e8c659700 1 -- 172.21.15.183:0/71444582 >> 172.21.15.183:0/71444582 conn(0x7f3e840fda80 msgr2=0x7f3e84106b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:18:40.100 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:40.099+0000 7f3e8c659700 1 -- 172.21.15.183:0/71444582 shutdown_connections 2024-06-03T03:18:40.100 INFO:teuthology.orchestra.run.smithi183.stderr:2024-06-03T03:18:40.099+0000 7f3e8c659700 1 -- 172.21.15.183:0/71444582 wait complete. 2024-06-03T03:18:40.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:39 smithi151 ceph-mon[37043]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:18:40.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:39 smithi117 ceph-mon[31649]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:18:40.813 DEBUG:teuthology.orchestra.run.smithi117:mds.a> sudo journalctl -f -n 0 -u ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@mds.a.service 2024-06-03T03:18:40.817 DEBUG:teuthology.orchestra.run.smithi117:mds.d> sudo journalctl -f -n 0 -u ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@mds.d.service 2024-06-03T03:18:40.863 DEBUG:teuthology.orchestra.run.smithi117:mds.g> sudo journalctl -f -n 0 -u ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@mds.g.service 2024-06-03T03:18:40.909 DEBUG:teuthology.orchestra.run.smithi117:mds.j> sudo journalctl -f -n 0 -u ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@mds.j.service 2024-06-03T03:18:40.954 DEBUG:teuthology.orchestra.run.smithi151:mds.b> sudo journalctl -f -n 0 -u ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@mds.b.service 2024-06-03T03:18:40.957 DEBUG:teuthology.orchestra.run.smithi151:mds.e> sudo journalctl -f -n 0 -u ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@mds.e.service 2024-06-03T03:18:41.001 DEBUG:teuthology.orchestra.run.smithi151:mds.h> sudo journalctl -f -n 0 -u ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@mds.h.service 2024-06-03T03:18:41.043 DEBUG:teuthology.orchestra.run.smithi151:mds.k> sudo journalctl -f -n 0 -u ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@mds.k.service 2024-06-03T03:18:41.087 DEBUG:teuthology.orchestra.run.smithi183:mds.c> sudo journalctl -f -n 0 -u ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@mds.c.service 2024-06-03T03:18:41.090 DEBUG:teuthology.orchestra.run.smithi183:mds.f> sudo journalctl -f -n 0 -u ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@mds.f.service 2024-06-03T03:18:41.133 DEBUG:teuthology.orchestra.run.smithi183:mds.i> sudo journalctl -f -n 0 -u ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@mds.i.service 2024-06-03T03:18:41.176 DEBUG:teuthology.orchestra.run.smithi183:mds.l> sudo journalctl -f -n 0 -u ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@mds.l.service 2024-06-03T03:18:41.178 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:41 smithi117 ceph-mon[31649]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi117=a;smithi117=d;smithi117=g;smithi117=j;smithi151=b;smithi151=e;smithi151=h;smithi151=k;smithi183=c;smithi183=f;smithi183=i;smithi183=l", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:18:41.179 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:41 smithi117 ceph-mon[31649]: Saving service mds.all spec with placement smithi117=a;smithi117=d;smithi117=g;smithi117=j;smithi151=b;smithi151=e;smithi151=h;smithi151=k;smithi183=c;smithi183=f;smithi183=i;smithi183=l;count:12 2024-06-03T03:18:41.179 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:41 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:41.179 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:41 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:18:41.179 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:41 smithi117 ceph-mon[31649]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:18:41.219 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-06-03T03:18:41.219 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-03T03:18:41.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:41 smithi151 ceph-mon[37043]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi117=a;smithi117=d;smithi117=g;smithi117=j;smithi151=b;smithi151=e;smithi151=h;smithi151=k;smithi183=c;smithi183=f;smithi183=i;smithi183=l", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:18:41.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:41 smithi151 ceph-mon[37043]: Saving service mds.all spec with placement smithi117=a;smithi117=d;smithi117=g;smithi117=j;smithi151=b;smithi151=e;smithi151=h;smithi151=k;smithi183=c;smithi183=f;smithi183=i;smithi183=l;count:12 2024-06-03T03:18:41.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:41 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:41.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:41 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:18:41.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:41 smithi151 ceph-mon[37043]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:18:41.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:41 smithi183 ceph-mon[37326]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi117=a;smithi117=d;smithi117=g;smithi117=j;smithi151=b;smithi151=e;smithi151=h;smithi151=k;smithi183=c;smithi183=f;smithi183=i;smithi183=l", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:18:41.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:41 smithi183 ceph-mon[37326]: Saving service mds.all spec with placement smithi117=a;smithi117=d;smithi117=g;smithi117=j;smithi151=b;smithi151=e;smithi151=h;smithi151=k;smithi183=c;smithi183=f;smithi183=i;smithi183=l;count:12 2024-06-03T03:18:41.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:41 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:41.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:41 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:18:41.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:41 smithi183 ceph-mon[37326]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:18:41.658 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:18:42.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:42 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:18:42.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:42 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:18:42.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:42 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:42.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:42 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T03:18:42.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:42 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T03:18:42.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:42 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:18:42.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:42 smithi183 ceph-mon[37326]: Deploying daemon mds.l on smithi183 2024-06-03T03:18:42.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:42 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:18:42.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:42 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:18:42.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:42 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:42.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:42 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T03:18:42.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:42 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T03:18:42.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:42 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:18:42.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:42 smithi151 ceph-mon[37043]: Deploying daemon mds.l on smithi183 2024-06-03T03:18:42.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:42 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:18:42.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:42 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:18:42.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:42 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:42.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:42 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T03:18:42.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:42 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T03:18:42.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:42 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:18:42.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:42 smithi117 ceph-mon[31649]: Deploying daemon mds.l on smithi183 2024-06-03T03:18:43.312 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:43 smithi183 ceph-mon[37326]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:18:43.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:43 smithi151 ceph-mon[37043]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:18:43.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:43 smithi117 ceph-mon[31649]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:18:43.703 INFO:journalctl@ceph.mds.l.smithi183.stdout:Jun 03 03:18:43 smithi183 systemd[1]: Starting Ceph mds.l for a0e8473a-2156-11ef-bc9b-c7b262605968... 2024-06-03T03:18:43.982 INFO:journalctl@ceph.mds.l.smithi183.stdout:Jun 03 03:18:43 smithi183 podman[66973]: 2024-06-03 03:18:43.604995919 +0000 UTC m=+0.017538190 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T03:18:43.982 INFO:journalctl@ceph.mds.l.smithi183.stdout:Jun 03 03:18:43 smithi183 podman[66973]: 2024-06-03 03:18:43.712594458 +0000 UTC m=+0.125136761 container create 148d60d50a9a8a210164585fdeab13b122a2c65809cb072385fe4c0e7b55dd0d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-l, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, ceph=True) 2024-06-03T03:18:43.982 INFO:journalctl@ceph.mds.l.smithi183.stdout:Jun 03 03:18:43 smithi183 podman[66973]: 2024-06-03 03:18:43.981878552 +0000 UTC m=+0.394420810 container init 148d60d50a9a8a210164585fdeab13b122a2c65809cb072385fe4c0e7b55dd0d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-06-03T03:18:44.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:44.011+0000 7f9e98000700 1 -- 172.21.15.117:0/1728083327 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e901046b0 msgr2=0x7f9e90104b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:18:44.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:44.011+0000 7f9e98000700 1 --2- 172.21.15.117:0/1728083327 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e901046b0 0x7f9e90104b30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c009990 tx=0x7f9e7c009ca0 comp rx=0 tx=0).stop 2024-06-03T03:18:44.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:44.012+0000 7f9e98000700 1 -- 172.21.15.117:0/1728083327 shutdown_connections 2024-06-03T03:18:44.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:44.012+0000 7f9e98000700 1 --2- 172.21.15.117:0/1728083327 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e90105070 0x7f9e90111580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:18:44.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:44.012+0000 7f9e98000700 1 --2- 172.21.15.117:0/1728083327 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9e9010a6b0 0x7f9e9010aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:18:44.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:44.012+0000 7f9e98000700 1 --2- 172.21.15.117:0/1728083327 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e901046b0 0x7f9e90104b30 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:18:44.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:44.012+0000 7f9e98000700 1 -- 172.21.15.117:0/1728083327 >> 172.21.15.117:0/1728083327 conn(0x7f9e901001a0 msgr2=0x7f9e901025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:18:44.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:44.012+0000 7f9e98000700 1 -- 172.21.15.117:0/1728083327 shutdown_connections 2024-06-03T03:18:44.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:44.012+0000 7f9e98000700 1 -- 172.21.15.117:0/1728083327 wait complete. 2024-06-03T03:18:44.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:44.013+0000 7f9e98000700 1 Processor -- start 2024-06-03T03:18:44.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:44.013+0000 7f9e98000700 1 -- start start 2024-06-03T03:18:44.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:44.013+0000 7f9e98000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9e901046b0 0x7f9e9019af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:18:44.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:44.013+0000 7f9e98000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e90105070 0x7f9e9019b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:18:44.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:44.013+0000 7f9e98000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e9010a6b0 0x7f9e9019f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:18:44.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:44.013+0000 7f9e98000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9e90114a90 con 0x7f9e90105070 2024-06-03T03:18:44.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:44.013+0000 7f9e98000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9e90114910 con 0x7f9e901046b0 2024-06-03T03:18:44.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:44.013+0000 7f9e98000700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9e90114c10 con 0x7f9e9010a6b0 2024-06-03T03:18:44.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:44.014+0000 7f9e95600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9e901046b0 0x7f9e9019af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:18:44.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:44.014+0000 7f9e96000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e9010a6b0 0x7f9e9019f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:18:44.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:44.014+0000 7f9e95600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9e901046b0 0x7f9e9019af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:50146/0 (socket says 172.21.15.117:50146) 2024-06-03T03:18:44.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:44.014+0000 7f9e95600700 1 -- 172.21.15.117:0/1216526553 learned_addr learned my addr 172.21.15.117:0/1216526553 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:18:44.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:44.014+0000 7f9e96000700 1 --2- 172.21.15.117:0/1216526553 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e9010a6b0 0x7f9e9019f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:51550/0 (socket says 172.21.15.117:51550) 2024-06-03T03:18:44.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:44.014+0000 7f9e94c00700 1 --2- 172.21.15.117:0/1216526553 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e90105070 0x7f9e9019b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:18:44.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:44.014+0000 7f9e95600700 1 -- 172.21.15.117:0/1216526553 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e9010a6b0 msgr2=0x7f9e9019f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:18:44.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:44.014+0000 7f9e95600700 1 --2- 172.21.15.117:0/1216526553 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e9010a6b0 0x7f9e9019f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:18:44.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:44.014+0000 7f9e95600700 1 -- 172.21.15.117:0/1216526553 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e90105070 msgr2=0x7f9e9019b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:18:44.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:44.014+0000 7f9e95600700 1 --2- 172.21.15.117:0/1216526553 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e90105070 0x7f9e9019b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:18:44.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:44.014+0000 7f9e95600700 1 -- 172.21.15.117:0/1216526553 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e9019fec0 con 0x7f9e901046b0 2024-06-03T03:18:44.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:44.014+0000 7f9e96000700 1 --2- 172.21.15.117:0/1216526553 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e9010a6b0 0x7f9e9019f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:18:44.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:44.015+0000 7f9e95600700 1 --2- 172.21.15.117:0/1216526553 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9e901046b0 0x7f9e9019af00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9e8400d950 tx=0x7f9e8400dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:18:44.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:44.015+0000 7f9e8e000700 1 -- 172.21.15.117:0/1216526553 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e84016070 con 0x7f9e901046b0 2024-06-03T03:18:44.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:44.015+0000 7f9e98000700 1 -- 172.21.15.117:0/1216526553 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e901a01b0 con 0x7f9e901046b0 2024-06-03T03:18:44.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:44.015+0000 7f9e8e000700 1 -- 172.21.15.117:0/1216526553 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e8400fd70 con 0x7f9e901046b0 2024-06-03T03:18:44.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:44.015+0000 7f9e8e000700 1 -- 172.21.15.117:0/1216526553 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e8400e610 con 0x7f9e901046b0 2024-06-03T03:18:44.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:44.015+0000 7f9e98000700 1 -- 172.21.15.117:0/1216526553 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e901a7a30 con 0x7f9e901046b0 2024-06-03T03:18:44.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:44.016+0000 7f9e98000700 1 -- 172.21.15.117:0/1216526553 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e90069180 con 0x7f9e901046b0 2024-06-03T03:18:44.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:44.018+0000 7f9e8e000700 1 -- 172.21.15.117:0/1216526553 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f9e84019070 con 0x7f9e901046b0 2024-06-03T03:18:44.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:44.019+0000 7f9e8e000700 1 --2- 172.21.15.117:0/1216526553 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f9e780a2c20 0x7f9e780a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:18:44.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:44.019+0000 7f9e8e000700 1 -- 172.21.15.117:0/1216526553 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f9e840ce650 con 0x7f9e901046b0 2024-06-03T03:18:44.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:44.020+0000 7f9e94c00700 1 --2- 172.21.15.117:0/1216526553 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f9e780a2c20 0x7f9e780a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:18:44.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:44.020+0000 7f9e94c00700 1 --2- 172.21.15.117:0/1216526553 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f9e780a2c20 0x7f9e780a50e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9e90109d00 tx=0x7f9e7c00b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:18:44.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:44.024+0000 7f9e8e000700 1 -- 172.21.15.117:0/1216526553 <== mon.2 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9e8409b9b0 con 0x7f9e901046b0 2024-06-03T03:18:44.232 INFO:journalctl@ceph.mds.l.smithi183.stdout:Jun 03 03:18:43 smithi183 podman[66973]: 2024-06-03 03:18:43.984577603 +0000 UTC m=+0.397119862 container start 148d60d50a9a8a210164585fdeab13b122a2c65809cb072385fe4c0e7b55dd0d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-l, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-06-03T03:18:44.232 INFO:journalctl@ceph.mds.l.smithi183.stdout:Jun 03 03:18:44 smithi183 ceph-mds[66992]: -- 172.21.15.183:0/3586292820 <== mon.1 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e0bd775a40 con 0x55e0bd749000 2024-06-03T03:18:44.232 INFO:journalctl@ceph.mds.l.smithi183.stdout:Jun 03 03:18:44 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-l[66988]: starting mds.l at 2024-06-03T03:18:44.232 INFO:journalctl@ceph.mds.l.smithi183.stdout:Jun 03 03:18:44 smithi183 bash[66973]: 148d60d50a9a8a210164585fdeab13b122a2c65809cb072385fe4c0e7b55dd0d 2024-06-03T03:18:44.232 INFO:journalctl@ceph.mds.l.smithi183.stdout:Jun 03 03:18:44 smithi183 systemd[1]: Started Ceph mds.l for a0e8473a-2156-11ef-bc9b-c7b262605968. 2024-06-03T03:18:44.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:44.307+0000 7f9e98000700 1 -- 172.21.15.117:0/1216526553 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9e9019c210 con 0x7f9e901046b0 2024-06-03T03:18:44.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:44.308+0000 7f9e8e000700 1 -- 172.21.15.117:0/1216526553 <== mon.2 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7f9e84095e20 con 0x7f9e901046b0 2024-06-03T03:18:44.310 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T03:18:44.311 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/2429725085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2429725085},{"type":"v1","addr":"172.21.15.183:6833","nonce":2429725085}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-06-03T03:18:44.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:44.311+0000 7f9e98000700 1 -- 172.21.15.117:0/1216526553 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f9e780a2c20 msgr2=0x7f9e780a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:18:44.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:44.311+0000 7f9e98000700 1 --2- 172.21.15.117:0/1216526553 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f9e780a2c20 0x7f9e780a50e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9e90109d00 tx=0x7f9e7c00b260 comp rx=0 tx=0).stop 2024-06-03T03:18:44.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:44.311+0000 7f9e98000700 1 -- 172.21.15.117:0/1216526553 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9e901046b0 msgr2=0x7f9e9019af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:18:44.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:44.311+0000 7f9e98000700 1 --2- 172.21.15.117:0/1216526553 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9e901046b0 0x7f9e9019af00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9e8400d950 tx=0x7f9e8400dc60 comp rx=0 tx=0).stop 2024-06-03T03:18:44.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:44.311+0000 7f9e98000700 1 -- 172.21.15.117:0/1216526553 shutdown_connections 2024-06-03T03:18:44.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:44.311+0000 7f9e98000700 1 --2- 172.21.15.117:0/1216526553 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f9e780a2c20 0x7f9e780a50e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:18:44.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:44.311+0000 7f9e98000700 1 --2- 172.21.15.117:0/1216526553 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e9010a6b0 0x7f9e9019f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:18:44.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:44.311+0000 7f9e98000700 1 --2- 172.21.15.117:0/1216526553 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9e901046b0 0x7f9e9019af00 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:18:44.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:44.311+0000 7f9e98000700 1 --2- 172.21.15.117:0/1216526553 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e90105070 0x7f9e9019b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:18:44.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:44.311+0000 7f9e98000700 1 -- 172.21.15.117:0/1216526553 >> 172.21.15.117:0/1216526553 conn(0x7f9e901001a0 msgr2=0x7f9e90100760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:18:44.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:44.311+0000 7f9e98000700 1 -- 172.21.15.117:0/1216526553 shutdown_connections 2024-06-03T03:18:44.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:44.311+0000 7f9e98000700 1 -- 172.21.15.117:0/1216526553 wait complete. 2024-06-03T03:18:44.314 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 2 2024-06-03T03:18:45.062 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:45 smithi151 ceph-mon[37043]: mds.? [v2:172.21.15.183:6832/2429725085,v1:172.21.15.183:6833/2429725085] up:boot 2024-06-03T03:18:45.062 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:45 smithi151 ceph-mon[37043]: fsmap 1 up:standby 2024-06-03T03:18:45.062 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:45 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-03T03:18:45.062 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:45 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:45.062 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:45 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:45.062 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:45 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:45.063 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:45 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T03:18:45.063 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:45 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T03:18:45.063 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:45 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:18:45.063 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:45 smithi151 ceph-mon[37043]: Deploying daemon mds.k on smithi151 2024-06-03T03:18:45.063 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:45 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/1216526553' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T03:18:45.063 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:45 smithi151 ceph-mon[37043]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:18:45.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:45 smithi117 ceph-mon[31649]: mds.? [v2:172.21.15.183:6832/2429725085,v1:172.21.15.183:6833/2429725085] up:boot 2024-06-03T03:18:45.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:45 smithi117 ceph-mon[31649]: fsmap 1 up:standby 2024-06-03T03:18:45.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:45 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-03T03:18:45.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:45 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:45.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:45 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:45.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:45 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:45.448 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:45 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T03:18:45.448 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:45 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T03:18:45.448 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:45 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:18:45.448 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:45 smithi117 ceph-mon[31649]: Deploying daemon mds.k on smithi151 2024-06-03T03:18:45.448 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:45 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/1216526553' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T03:18:45.448 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:45 smithi117 ceph-mon[31649]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:18:45.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:45 smithi183 ceph-mon[37326]: mds.? [v2:172.21.15.183:6832/2429725085,v1:172.21.15.183:6833/2429725085] up:boot 2024-06-03T03:18:45.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:45 smithi183 ceph-mon[37326]: fsmap 1 up:standby 2024-06-03T03:18:45.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:45 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-03T03:18:45.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:45 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:45.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:45 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:45.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:45 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:45.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:45 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T03:18:45.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:45 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T03:18:45.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:45 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:18:45.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:45 smithi183 ceph-mon[37326]: Deploying daemon mds.k on smithi151 2024-06-03T03:18:45.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:45 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/1216526553' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T03:18:45.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:45 smithi183 ceph-mon[37326]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:18:46.407 INFO:journalctl@ceph.mds.k.smithi151.stdout:Jun 03 03:18:46 smithi151 systemd[1]: Starting Ceph mds.k for a0e8473a-2156-11ef-bc9b-c7b262605968... 2024-06-03T03:18:46.810 INFO:journalctl@ceph.mds.k.smithi151.stdout:Jun 03 03:18:46 smithi151 podman[66313]: 2024-06-03 03:18:46.433466163 +0000 UTC m=+0.017993598 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T03:18:46.810 INFO:journalctl@ceph.mds.k.smithi151.stdout:Jun 03 03:18:46 smithi151 podman[66313]: 2024-06-03 03:18:46.531948917 +0000 UTC m=+0.116476329 container create 8f37978860c8f3f84836688ea416e6287ef69b0dd92bcf246dab4998407f5195 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-k, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-06-03T03:18:47.108 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-03T03:18:47.156 INFO:journalctl@ceph.mds.k.smithi151.stdout:Jun 03 03:18:46 smithi151 podman[66313]: 2024-06-03 03:18:46.809965512 +0000 UTC m=+0.394492943 container init 8f37978860c8f3f84836688ea416e6287ef69b0dd92bcf246dab4998407f5195 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True) 2024-06-03T03:18:47.157 INFO:journalctl@ceph.mds.k.smithi151.stdout:Jun 03 03:18:46 smithi151 podman[66313]: 2024-06-03 03:18:46.813189531 +0000 UTC m=+0.397716955 container start 8f37978860c8f3f84836688ea416e6287ef69b0dd92bcf246dab4998407f5195 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-k, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-06-03T03:18:47.157 INFO:journalctl@ceph.mds.k.smithi151.stdout:Jun 03 03:18:46 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-k[66328]: starting mds.k at 2024-06-03T03:18:47.157 INFO:journalctl@ceph.mds.k.smithi151.stdout:Jun 03 03:18:46 smithi151 bash[66313]: 8f37978860c8f3f84836688ea416e6287ef69b0dd92bcf246dab4998407f5195 2024-06-03T03:18:47.157 INFO:journalctl@ceph.mds.k.smithi151.stdout:Jun 03 03:18:46 smithi151 systemd[1]: Started Ceph mds.k for a0e8473a-2156-11ef-bc9b-c7b262605968. 2024-06-03T03:18:47.550 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:18:47.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:47 smithi117 ceph-mon[31649]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:18:47.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:47 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:47.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:47 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:47.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:47 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:47.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:47 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T03:18:47.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:47 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T03:18:47.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:47 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:18:48.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:47 smithi151 ceph-mon[37043]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:18:48.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:47 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:48.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:47 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:48.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:47 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:48.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:47 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T03:18:48.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:47 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T03:18:48.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:47 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:18:48.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:47 smithi183 ceph-mon[37326]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:18:48.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:47 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:48.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:47 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:48.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:47 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:48.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:47 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T03:18:48.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:47 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T03:18:48.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:47 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:18:48.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:48 smithi117 ceph-mon[31649]: Deploying daemon mds.j on smithi117 2024-06-03T03:18:48.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:48 smithi117 ceph-mon[31649]: mds.? [v2:172.21.15.151:6832/3355662416,v1:172.21.15.151:6833/3355662416] up:boot 2024-06-03T03:18:48.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:48 smithi117 ceph-mon[31649]: fsmap 2 up:standby 2024-06-03T03:18:48.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:48 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-03T03:18:49.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:48 smithi151 ceph-mon[37043]: Deploying daemon mds.j on smithi117 2024-06-03T03:18:49.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:48 smithi151 ceph-mon[37043]: mds.? [v2:172.21.15.151:6832/3355662416,v1:172.21.15.151:6833/3355662416] up:boot 2024-06-03T03:18:49.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:48 smithi151 ceph-mon[37043]: fsmap 2 up:standby 2024-06-03T03:18:49.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:48 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-03T03:18:49.234 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:48 smithi183 ceph-mon[37326]: Deploying daemon mds.j on smithi117 2024-06-03T03:18:49.234 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:48 smithi183 ceph-mon[37326]: mds.? [v2:172.21.15.151:6832/3355662416,v1:172.21.15.151:6833/3355662416] up:boot 2024-06-03T03:18:49.235 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:48 smithi183 ceph-mon[37326]: fsmap 2 up:standby 2024-06-03T03:18:49.235 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:48 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-03T03:18:49.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:49 smithi117 ceph-mon[31649]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:18:50.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:49 smithi151 ceph-mon[37043]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:18:50.170 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:49 smithi183 ceph-mon[37326]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:18:50.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:50.943+0000 7fafc5400700 1 -- 172.21.15.117:0/4274329997 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fafc0102740 msgr2=0x7fafc0102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:18:50.946 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:50.943+0000 7fafc5400700 1 --2- 172.21.15.117:0/4274329997 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fafc0102740 0x7fafc0102b20 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fafb00099c0 tx=0x7fafb0009cd0 comp rx=0 tx=0).stop 2024-06-03T03:18:50.946 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:50.943+0000 7fafc5400700 1 -- 172.21.15.117:0/4274329997 shutdown_connections 2024-06-03T03:18:50.946 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:50.943+0000 7fafc5400700 1 --2- 172.21.15.117:0/4274329997 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fafc0107840 0x7fafc010f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:18:50.946 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:50.943+0000 7fafc5400700 1 --2- 172.21.15.117:0/4274329997 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fafc0102740 0x7fafc0102b20 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:18:50.946 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:50.943+0000 7fafc5400700 1 --2- 172.21.15.117:0/4274329997 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fafc01030f0 0x7fafc0107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:18:50.946 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:50.943+0000 7fafc5400700 1 -- 172.21.15.117:0/4274329997 >> 172.21.15.117:0/4274329997 conn(0x7fafc00795f0 msgr2=0x7fafc0079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:18:50.946 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:50.943+0000 7fafc5400700 1 -- 172.21.15.117:0/4274329997 shutdown_connections 2024-06-03T03:18:50.946 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:50.943+0000 7fafc5400700 1 -- 172.21.15.117:0/4274329997 wait complete. 2024-06-03T03:18:50.947 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:50.945+0000 7fafc5400700 1 Processor -- start 2024-06-03T03:18:50.947 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:50.945+0000 7fafc5400700 1 -- start start 2024-06-03T03:18:50.947 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:50.946+0000 7fafc5400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fafc0102740 0x7fafc019e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:18:50.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:50.946+0000 7fafc5400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fafc01030f0 0x7fafc019edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:18:50.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:50.946+0000 7fafc5400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fafc0107840 0x7fafc0198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:18:50.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:50.946+0000 7fafc5400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fafc01127d0 con 0x7fafc0102740 2024-06-03T03:18:50.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:50.946+0000 7fafc5400700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fafc0112650 con 0x7fafc01030f0 2024-06-03T03:18:50.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:50.946+0000 7fafc5400700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fafc0112950 con 0x7fafc0107840 2024-06-03T03:18:50.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:50.946+0000 7fafbe800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fafc0102740 0x7fafc019e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:18:50.949 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:50.946+0000 7fafbe800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fafc0102740 0x7fafc019e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:48268/0 (socket says 172.21.15.117:48268) 2024-06-03T03:18:50.949 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:50.946+0000 7fafbe800700 1 -- 172.21.15.117:0/2206577364 learned_addr learned my addr 172.21.15.117:0/2206577364 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:18:50.949 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:50.946+0000 7fafb7e00700 1 --2- 172.21.15.117:0/2206577364 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fafc01030f0 0x7fafc019edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:18:50.949 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:50.946+0000 7fafbf200700 1 --2- 172.21.15.117:0/2206577364 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fafc0107840 0x7fafc0198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:18:50.949 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:50.947+0000 7fafbe800700 1 -- 172.21.15.117:0/2206577364 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fafc0107840 msgr2=0x7fafc0198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:18:50.949 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:50.947+0000 7fafbe800700 1 --2- 172.21.15.117:0/2206577364 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fafc0107840 0x7fafc0198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:18:50.950 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:50.947+0000 7fafbe800700 1 -- 172.21.15.117:0/2206577364 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fafc01030f0 msgr2=0x7fafc019edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:18:50.950 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:50.947+0000 7fafbe800700 1 --2- 172.21.15.117:0/2206577364 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fafc01030f0 0x7fafc019edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:18:50.950 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:50.947+0000 7fafbe800700 1 -- 172.21.15.117:0/2206577364 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafc01991d0 con 0x7fafc0102740 2024-06-03T03:18:50.950 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:50.947+0000 7fafbe800700 1 --2- 172.21.15.117:0/2206577364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fafc0102740 0x7fafc019e8b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fafb0006010 tx=0x7fafb0003a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:18:50.950 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:50.947+0000 7fafb7400700 1 -- 172.21.15.117:0/2206577364 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb0003e50 con 0x7fafc0102740 2024-06-03T03:18:50.950 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:50.947+0000 7fafb7400700 1 -- 172.21.15.117:0/2206577364 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafb00055a0 con 0x7fafc0102740 2024-06-03T03:18:50.950 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:50.947+0000 7fafc5400700 1 -- 172.21.15.117:0/2206577364 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafc0199460 con 0x7fafc0102740 2024-06-03T03:18:50.950 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:50.947+0000 7fafb7400700 1 -- 172.21.15.117:0/2206577364 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb000fe30 con 0x7fafc0102740 2024-06-03T03:18:50.950 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:50.947+0000 7fafc5400700 1 -- 172.21.15.117:0/2206577364 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafc00746c0 con 0x7fafc0102740 2024-06-03T03:18:50.951 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:50.949+0000 7fafc5400700 1 -- 172.21.15.117:0/2206577364 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafc0103bd0 con 0x7fafc0102740 2024-06-03T03:18:50.952 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:50.951+0000 7fafb7400700 1 -- 172.21.15.117:0/2206577364 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fafb000f4f0 con 0x7fafc0102740 2024-06-03T03:18:50.955 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:50.953+0000 7fafb7400700 1 --2- 172.21.15.117:0/2206577364 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fafa40a2dc0 0x7fafa40a5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:18:50.955 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:50.953+0000 7fafb7400700 1 -- 172.21.15.117:0/2206577364 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fafb00cfa70 con 0x7fafc0102740 2024-06-03T03:18:50.955 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:50.953+0000 7fafb7e00700 1 --2- 172.21.15.117:0/2206577364 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fafa40a2dc0 0x7fafa40a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:18:50.956 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:50.954+0000 7fafb7e00700 1 --2- 172.21.15.117:0/2206577364 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fafa40a2dc0 0x7fafa40a5280 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fafc0100f60 tx=0x7fafa8012040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:18:50.958 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:50.956+0000 7fafb7400700 1 -- 172.21.15.117:0/2206577364 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fafb0013070 con 0x7fafc0102740 2024-06-03T03:18:51.248 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:51.246+0000 7fafc5400700 1 -- 172.21.15.117:0/2206577364 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fafc0199fa0 con 0x7fafc0102740 2024-06-03T03:18:51.248 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:51.246+0000 7fafb7400700 1 -- 172.21.15.117:0/2206577364 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2224 (secure 0 0 0) 0x7fafb009ce60 con 0x7fafc0102740 2024-06-03T03:18:51.248 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T03:18:51.249 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3355662416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3355662416},{"type":"v1","addr":"172.21.15.151:6833","nonce":3355662416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/2429725085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2429725085},{"type":"v1","addr":"172.21.15.183:6833","nonce":2429725085}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-06-03T03:18:51.251 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:51.249+0000 7fafc5400700 1 -- 172.21.15.117:0/2206577364 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fafa40a2dc0 msgr2=0x7fafa40a5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:18:51.252 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:51.249+0000 7fafc5400700 1 --2- 172.21.15.117:0/2206577364 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fafa40a2dc0 0x7fafa40a5280 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fafc0100f60 tx=0x7fafa8012040 comp rx=0 tx=0).stop 2024-06-03T03:18:51.252 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:51.249+0000 7fafc5400700 1 -- 172.21.15.117:0/2206577364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fafc0102740 msgr2=0x7fafc019e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:18:51.252 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:51.250+0000 7fafc5400700 1 --2- 172.21.15.117:0/2206577364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fafc0102740 0x7fafc019e8b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fafb0006010 tx=0x7fafb0003a90 comp rx=0 tx=0).stop 2024-06-03T03:18:51.252 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:51.250+0000 7fafc5400700 1 -- 172.21.15.117:0/2206577364 shutdown_connections 2024-06-03T03:18:51.252 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:51.250+0000 7fafc5400700 1 --2- 172.21.15.117:0/2206577364 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fafa40a2dc0 0x7fafa40a5280 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:18:51.252 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:51.250+0000 7fafc5400700 1 --2- 172.21.15.117:0/2206577364 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fafc0107840 0x7fafc0198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:18:51.252 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:51.250+0000 7fafc5400700 1 --2- 172.21.15.117:0/2206577364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fafc0102740 0x7fafc019e8b0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:18:51.252 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:51.250+0000 7fafc5400700 1 --2- 172.21.15.117:0/2206577364 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fafc01030f0 0x7fafc019edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:18:51.252 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:51.250+0000 7fafc5400700 1 -- 172.21.15.117:0/2206577364 >> 172.21.15.117:0/2206577364 conn(0x7fafc00795f0 msgr2=0x7fafc0101c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:18:51.253 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:51.250+0000 7fafc5400700 1 -- 172.21.15.117:0/2206577364 shutdown_connections 2024-06-03T03:18:51.253 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:51.250+0000 7fafc5400700 1 -- 172.21.15.117:0/2206577364 wait complete. 2024-06-03T03:18:51.253 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 3 2024-06-03T03:18:51.697 INFO:journalctl@ceph.mds.j.smithi117.stdout:Jun 03 03:18:51 smithi117 systemd[1]: Starting Ceph mds.j for a0e8473a-2156-11ef-bc9b-c7b262605968... 2024-06-03T03:18:52.146 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:51 smithi117 ceph-mon[31649]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:18:52.146 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:51 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/2206577364' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T03:18:52.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:51 smithi151 ceph-mon[37043]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:18:52.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:51 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/2206577364' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T03:18:52.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:51 smithi183 ceph-mon[37326]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:18:52.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:51 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/2206577364' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T03:18:52.447 INFO:journalctl@ceph.mds.j.smithi117.stdout:Jun 03 03:18:52 smithi117 podman[69594]: 2024-06-03 03:18:52.246680314 +0000 UTC m=+0.035344848 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T03:18:52.447 INFO:journalctl@ceph.mds.j.smithi117.stdout:Jun 03 03:18:52 smithi117 podman[69594]: 2024-06-03 03:18:52.446008175 +0000 UTC m=+0.234672743 container create acc516aa313c6e1f542a16c863765ce05cca030da9c0bc8585a8cf3d2dddb75f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-j, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc) 2024-06-03T03:18:53.196 INFO:journalctl@ceph.mds.j.smithi117.stdout:Jun 03 03:18:52 smithi117 podman[69594]: 2024-06-03 03:18:52.784303488 +0000 UTC m=+0.572968152 container init acc516aa313c6e1f542a16c863765ce05cca030da9c0bc8585a8cf3d2dddb75f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-j, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-06-03T03:18:53.197 INFO:journalctl@ceph.mds.j.smithi117.stdout:Jun 03 03:18:52 smithi117 podman[69594]: 2024-06-03 03:18:52.794082088 +0000 UTC m=+0.582746628 container start acc516aa313c6e1f542a16c863765ce05cca030da9c0bc8585a8cf3d2dddb75f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-j, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-06-03T03:18:53.197 INFO:journalctl@ceph.mds.j.smithi117.stdout:Jun 03 03:18:52 smithi117 ceph-mds[69612]: -- 172.21.15.117:0/3680952514 <== mon.1 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cbecc39a40 con 0x55cbecc0c800 2024-06-03T03:18:53.197 INFO:journalctl@ceph.mds.j.smithi117.stdout:Jun 03 03:18:52 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-j[69608]: starting mds.j at 2024-06-03T03:18:53.197 INFO:journalctl@ceph.mds.j.smithi117.stdout:Jun 03 03:18:52 smithi117 bash[69594]: acc516aa313c6e1f542a16c863765ce05cca030da9c0bc8585a8cf3d2dddb75f 2024-06-03T03:18:53.197 INFO:journalctl@ceph.mds.j.smithi117.stdout:Jun 03 03:18:52 smithi117 systemd[1]: Started Ceph mds.j for a0e8473a-2156-11ef-bc9b-c7b262605968. 2024-06-03T03:18:54.126 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:53 smithi117 ceph-mon[31649]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:18:54.126 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:53 smithi117 ceph-mon[31649]: mds.? [v2:172.21.15.117:6834/2884460088,v1:172.21.15.117:6835/2884460088] up:boot 2024-06-03T03:18:54.126 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:53 smithi117 ceph-mon[31649]: fsmap 3 up:standby 2024-06-03T03:18:54.126 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:53 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-06-03T03:18:54.126 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:53 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:54.126 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:53 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:54.126 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:53 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:54.126 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:53 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T03:18:54.126 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:53 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T03:18:54.126 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:53 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:18:54.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:53 smithi151 ceph-mon[37043]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:18:54.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:53 smithi151 ceph-mon[37043]: mds.? [v2:172.21.15.117:6834/2884460088,v1:172.21.15.117:6835/2884460088] up:boot 2024-06-03T03:18:54.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:53 smithi151 ceph-mon[37043]: fsmap 3 up:standby 2024-06-03T03:18:54.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:53 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-06-03T03:18:54.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:53 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:54.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:53 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:54.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:53 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:54.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:53 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T03:18:54.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:53 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T03:18:54.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:53 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:18:54.196 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-03T03:18:54.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:53 smithi183 ceph-mon[37326]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:18:54.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:53 smithi183 ceph-mon[37326]: mds.? [v2:172.21.15.117:6834/2884460088,v1:172.21.15.117:6835/2884460088] up:boot 2024-06-03T03:18:54.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:53 smithi183 ceph-mon[37326]: fsmap 3 up:standby 2024-06-03T03:18:54.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:53 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-06-03T03:18:54.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:53 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:54.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:53 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:54.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:53 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:54.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:53 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T03:18:54.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:53 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T03:18:54.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:53 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:18:55.138 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:54 smithi117 ceph-mon[31649]: Deploying daemon mds.d on smithi117 2024-06-03T03:18:55.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:54 smithi151 ceph-mon[37043]: Deploying daemon mds.d on smithi117 2024-06-03T03:18:55.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:54 smithi183 ceph-mon[37326]: Deploying daemon mds.d on smithi117 2024-06-03T03:18:55.498 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:18:55.949 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:55 smithi117 ceph-mon[31649]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:18:56.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:55 smithi151 ceph-mon[37043]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:18:56.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:55 smithi183 ceph-mon[37326]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:18:56.446 INFO:journalctl@ceph.mds.d.smithi117.stdout:Jun 03 03:18:56 smithi117 systemd[1]: Starting Ceph mds.d for a0e8473a-2156-11ef-bc9b-c7b262605968... 2024-06-03T03:18:57.447 INFO:journalctl@ceph.mds.d.smithi117.stdout:Jun 03 03:18:57 smithi117 podman[69991]: 2024-06-03 03:18:57.085870447 +0000 UTC m=+0.034953291 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T03:18:57.447 INFO:journalctl@ceph.mds.d.smithi117.stdout:Jun 03 03:18:57 smithi117 podman[69991]: 2024-06-03 03:18:57.327428574 +0000 UTC m=+0.276511439 container create d731bf2a4312788c35dd6b92708cd0ead80445a1bd50e4baec045dc39902dccf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-d, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-06-03T03:18:57.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:57 smithi117 ceph-mon[31649]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:18:57.947 INFO:journalctl@ceph.mds.d.smithi117.stdout:Jun 03 03:18:57 smithi117 podman[69991]: 2024-06-03 03:18:57.749714869 +0000 UTC m=+0.698797741 container init d731bf2a4312788c35dd6b92708cd0ead80445a1bd50e4baec045dc39902dccf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-d, RELEASE=squid-3d89acc, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-03T03:18:57.947 INFO:journalctl@ceph.mds.d.smithi117.stdout:Jun 03 03:18:57 smithi117 podman[69991]: 2024-06-03 03:18:57.763693399 +0000 UTC m=+0.712776233 container start d731bf2a4312788c35dd6b92708cd0ead80445a1bd50e4baec045dc39902dccf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-d, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-06-03T03:18:57.947 INFO:journalctl@ceph.mds.d.smithi117.stdout:Jun 03 03:18:57 smithi117 ceph-mds[70027]: -- 172.21.15.117:0/1583270911 <== mon.0 v2:172.21.15.117:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560a09551a40 con 0x560a09525000 2024-06-03T03:18:57.947 INFO:journalctl@ceph.mds.d.smithi117.stdout:Jun 03 03:18:57 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-d[70018]: starting mds.d at 2024-06-03T03:18:58.087 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:58.086+0000 7fc0fbc00700 1 -- 172.21.15.117:0/2187727410 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0f4105050 msgr2=0x7fc0f4111560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:18:58.087 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:58.086+0000 7fc0fbc00700 1 --2- 172.21.15.117:0/2187727410 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0f4105050 0x7fc0f4111560 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec00b0a0 tx=0x7fc0ec00b3b0 comp rx=0 tx=0).stop 2024-06-03T03:18:58.087 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:58.086+0000 7fc0fbc00700 1 -- 172.21.15.117:0/2187727410 shutdown_connections 2024-06-03T03:18:58.087 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:58.086+0000 7fc0fbc00700 1 --2- 172.21.15.117:0/2187727410 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0f4105050 0x7fc0f4111560 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:18:58.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:58.086+0000 7fc0fbc00700 1 --2- 172.21.15.117:0/2187727410 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc0f410a690 0x7fc0f410aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:18:58.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:58.086+0000 7fc0fbc00700 1 --2- 172.21.15.117:0/2187727410 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc0f4104690 0x7fc0f4104b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:18:58.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:58.086+0000 7fc0fbc00700 1 -- 172.21.15.117:0/2187727410 >> 172.21.15.117:0/2187727410 conn(0x7fc0f41001a0 msgr2=0x7fc0f41025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:18:58.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:58.086+0000 7fc0fbc00700 1 -- 172.21.15.117:0/2187727410 shutdown_connections 2024-06-03T03:18:58.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:58.086+0000 7fc0fbc00700 1 -- 172.21.15.117:0/2187727410 wait complete. 2024-06-03T03:18:58.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:58.087+0000 7fc0fbc00700 1 Processor -- start 2024-06-03T03:18:58.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:58.087+0000 7fc0fbc00700 1 -- start start 2024-06-03T03:18:58.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:58.088+0000 7fc0fbc00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0f4104690 0x7fc0f419ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:18:58.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:58.088+0000 7fc0fbc00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc0f4105050 0x7fc0f419b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:18:58.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:58.088+0000 7fc0fbc00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc0f410a690 0x7fc0f419f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:18:58.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:58.088+0000 7fc0fbc00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f41149c0 con 0x7fc0f4104690 2024-06-03T03:18:58.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:58.088+0000 7fc0fbc00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f4114840 con 0x7fc0f4105050 2024-06-03T03:18:58.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:58.088+0000 7fc0fbc00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f4114b40 con 0x7fc0f410a690 2024-06-03T03:18:58.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:58.088+0000 7fc0f3e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc0f4105050 0x7fc0f419b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:18:58.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:58.088+0000 7fc0f3e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc0f4105050 0x7fc0f419b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:33710/0 (socket says 172.21.15.117:33710) 2024-06-03T03:18:58.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:58.088+0000 7fc0f3e00700 1 -- 172.21.15.117:0/1996045358 learned_addr learned my addr 172.21.15.117:0/1996045358 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:18:58.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:58.088+0000 7fc0f9c00700 1 --2- 172.21.15.117:0/1996045358 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc0f410a690 0x7fc0f419f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:18:58.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:58.088+0000 7fc0f9200700 1 --2- 172.21.15.117:0/1996045358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0f4104690 0x7fc0f419ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:18:58.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:58.089+0000 7fc0f3e00700 1 -- 172.21.15.117:0/1996045358 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc0f410a690 msgr2=0x7fc0f419f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:18:58.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:58.089+0000 7fc0f3e00700 1 --2- 172.21.15.117:0/1996045358 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc0f410a690 0x7fc0f419f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:18:58.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:58.089+0000 7fc0f3e00700 1 -- 172.21.15.117:0/1996045358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0f4104690 msgr2=0x7fc0f419ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:18:58.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:58.089+0000 7fc0f3e00700 1 --2- 172.21.15.117:0/1996045358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0f4104690 0x7fc0f419ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:18:58.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:58.089+0000 7fc0f3e00700 1 -- 172.21.15.117:0/1996045358 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0f419fd80 con 0x7fc0f4105050 2024-06-03T03:18:58.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:58.089+0000 7fc0f9c00700 1 --2- 172.21.15.117:0/1996045358 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc0f410a690 0x7fc0f419f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:18:58.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:58.089+0000 7fc0f9200700 1 --2- 172.21.15.117:0/1996045358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0f4104690 0x7fc0f419ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T03:18:58.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:58.089+0000 7fc0f3e00700 1 --2- 172.21.15.117:0/1996045358 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc0f4105050 0x7fc0f419b2b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc0e000e9a0 tx=0x7fc0e000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:18:58.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:58.089+0000 7fc0f1600700 1 -- 172.21.15.117:0/1996045358 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e000cbd0 con 0x7fc0f4105050 2024-06-03T03:18:58.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:58.089+0000 7fc0f1600700 1 -- 172.21.15.117:0/1996045358 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0e000cd70 con 0x7fc0f4105050 2024-06-03T03:18:58.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:58.089+0000 7fc0fbc00700 1 -- 172.21.15.117:0/1996045358 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0f41a0070 con 0x7fc0f4105050 2024-06-03T03:18:58.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:58.089+0000 7fc0f1600700 1 -- 172.21.15.117:0/1996045358 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e0018750 con 0x7fc0f4105050 2024-06-03T03:18:58.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:58.089+0000 7fc0fbc00700 1 -- 172.21.15.117:0/1996045358 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0f41a78f0 con 0x7fc0f4105050 2024-06-03T03:18:58.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:58.091+0000 7fc0fbc00700 1 -- 172.21.15.117:0/1996045358 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0f4069180 con 0x7fc0f4105050 2024-06-03T03:18:58.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:58.092+0000 7fc0f1600700 1 -- 172.21.15.117:0/1996045358 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fc0e0010430 con 0x7fc0f4105050 2024-06-03T03:18:58.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:58.094+0000 7fc0f1600700 1 --2- 172.21.15.117:0/1996045358 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fc0dc0a2c20 0x7fc0dc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:18:58.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:58.094+0000 7fc0f1600700 1 -- 172.21.15.117:0/1996045358 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fc0e00cef20 con 0x7fc0f4105050 2024-06-03T03:18:58.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:58.094+0000 7fc0f9200700 1 --2- 172.21.15.117:0/1996045358 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fc0dc0a2c20 0x7fc0dc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:18:58.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:58.095+0000 7fc0f9200700 1 --2- 172.21.15.117:0/1996045358 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fc0dc0a2c20 0x7fc0dc0a50e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc0e8009bc0 tx=0x7fc0e8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:18:58.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:58.098+0000 7fc0f1600700 1 -- 172.21.15.117:0/1996045358 <== mon.2 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc0e009c280 con 0x7fc0f4105050 2024-06-03T03:18:58.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:57 smithi151 ceph-mon[37043]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:18:58.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:57 smithi183 ceph-mon[37326]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:18:58.387 INFO:journalctl@ceph.mds.d.smithi117.stdout:Jun 03 03:18:58 smithi117 bash[69991]: d731bf2a4312788c35dd6b92708cd0ead80445a1bd50e4baec045dc39902dccf 2024-06-03T03:18:58.387 INFO:journalctl@ceph.mds.d.smithi117.stdout:Jun 03 03:18:58 smithi117 systemd[1]: Started Ceph mds.d for a0e8473a-2156-11ef-bc9b-c7b262605968. 2024-06-03T03:18:58.387 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:58.385+0000 7fc0fbc00700 1 -- 172.21.15.117:0/1996045358 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc0f41a7ac0 con 0x7fc0f4105050 2024-06-03T03:18:58.388 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:58.386+0000 7fc0f1600700 1 -- 172.21.15.117:0/1996045358 <== mon.2 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3880 (secure 0 0 0) 0x7fc0e009bb80 con 0x7fc0f4105050 2024-06-03T03:18:58.388 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T03:18:58.388 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3353001119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3353001119},{"type":"v1","addr":"172.21.15.117:6837","nonce":3353001119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3355662416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3355662416},{"type":"v1","addr":"172.21.15.151:6833","nonce":3355662416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/2429725085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2429725085},{"type":"v1","addr":"172.21.15.183:6833","nonce":2429725085}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24466,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/2884460088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2884460088},{"type":"v1","addr":"172.21.15.117:6835","nonce":2884460088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[]} 2024-06-03T03:18:58.391 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:58.389+0000 7fc0fbc00700 1 -- 172.21.15.117:0/1996045358 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fc0dc0a2c20 msgr2=0x7fc0dc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:18:58.391 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:58.389+0000 7fc0fbc00700 1 --2- 172.21.15.117:0/1996045358 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fc0dc0a2c20 0x7fc0dc0a50e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc0e8009bc0 tx=0x7fc0e8009260 comp rx=0 tx=0).stop 2024-06-03T03:18:58.391 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:58.390+0000 7fc0fbc00700 1 -- 172.21.15.117:0/1996045358 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc0f4105050 msgr2=0x7fc0f419b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:18:58.391 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:58.390+0000 7fc0fbc00700 1 --2- 172.21.15.117:0/1996045358 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc0f4105050 0x7fc0f419b2b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc0e000e9a0 tx=0x7fc0e000ecb0 comp rx=0 tx=0).stop 2024-06-03T03:18:58.391 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:58.390+0000 7fc0fbc00700 1 -- 172.21.15.117:0/1996045358 shutdown_connections 2024-06-03T03:18:58.391 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:58.390+0000 7fc0fbc00700 1 --2- 172.21.15.117:0/1996045358 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fc0dc0a2c20 0x7fc0dc0a50e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:18:58.391 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:58.390+0000 7fc0fbc00700 1 --2- 172.21.15.117:0/1996045358 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc0f410a690 0x7fc0f419f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:18:58.392 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:58.390+0000 7fc0fbc00700 1 --2- 172.21.15.117:0/1996045358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0f4104690 0x7fc0f419ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:18:58.392 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:58.390+0000 7fc0fbc00700 1 --2- 172.21.15.117:0/1996045358 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc0f4105050 0x7fc0f419b2b0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:18:58.392 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:58.390+0000 7fc0fbc00700 1 -- 172.21.15.117:0/1996045358 >> 172.21.15.117:0/1996045358 conn(0x7fc0f41001a0 msgr2=0x7fc0f4100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:18:58.392 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:58.390+0000 7fc0fbc00700 1 -- 172.21.15.117:0/1996045358 shutdown_connections 2024-06-03T03:18:58.392 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:18:58.390+0000 7fc0fbc00700 1 -- 172.21.15.117:0/1996045358 wait complete. 2024-06-03T03:18:58.393 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 5 2024-06-03T03:18:59.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:58 smithi151 ceph-mon[37043]: mds.? [v2:172.21.15.117:6836/3353001119,v1:172.21.15.117:6837/3353001119] up:boot 2024-06-03T03:18:59.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:58 smithi151 ceph-mon[37043]: fsmap 4 up:standby 2024-06-03T03:18:59.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:58 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-06-03T03:18:59.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:58 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:59.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:58 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:59.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:58 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/1996045358' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T03:18:59.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:58 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:59.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:58 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T03:18:59.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:58 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T03:18:59.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:58 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:18:59.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:58 smithi117 ceph-mon[31649]: mds.? [v2:172.21.15.117:6836/3353001119,v1:172.21.15.117:6837/3353001119] up:boot 2024-06-03T03:18:59.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:58 smithi117 ceph-mon[31649]: fsmap 4 up:standby 2024-06-03T03:18:59.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:58 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-06-03T03:18:59.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:58 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:59.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:58 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:59.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:58 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/1996045358' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T03:18:59.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:58 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:59.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:58 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T03:18:59.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:58 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T03:18:59.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:58 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:18:59.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:58 smithi183 ceph-mon[37326]: mds.? [v2:172.21.15.117:6836/3353001119,v1:172.21.15.117:6837/3353001119] up:boot 2024-06-03T03:18:59.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:58 smithi183 ceph-mon[37326]: fsmap 4 up:standby 2024-06-03T03:18:59.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:58 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-06-03T03:18:59.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:58 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:59.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:58 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:59.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:58 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/1996045358' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T03:18:59.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:58 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:18:59.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:58 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T03:18:59.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:58 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T03:18:59.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:58 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:19:00.009 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:59 smithi151 ceph-mon[37043]: Deploying daemon mds.e on smithi151 2024-06-03T03:19:00.009 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:18:59 smithi151 ceph-mon[37043]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:19:00.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:59 smithi117 ceph-mon[31649]: Deploying daemon mds.e on smithi151 2024-06-03T03:19:00.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:18:59 smithi117 ceph-mon[31649]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:19:00.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:59 smithi183 ceph-mon[37326]: Deploying daemon mds.e on smithi151 2024-06-03T03:19:00.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:18:59 smithi183 ceph-mon[37326]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:19:00.657 INFO:journalctl@ceph.mds.e.smithi151.stdout:Jun 03 03:19:00 smithi151 systemd[1]: Starting Ceph mds.e for a0e8473a-2156-11ef-bc9b-c7b262605968... 2024-06-03T03:19:00.956 INFO:journalctl@ceph.mds.e.smithi151.stdout:Jun 03 03:19:00 smithi151 podman[66624]: 2024-06-03 03:19:00.603514076 +0000 UTC m=+0.017752576 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T03:19:00.956 INFO:journalctl@ceph.mds.e.smithi151.stdout:Jun 03 03:19:00 smithi151 podman[66624]: 2024-06-03 03:19:00.702966825 +0000 UTC m=+0.117205316 container create 427fca5011668a65822af965d667dc996432a1fc724a1baf615b3b0f59178bdf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-e, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, ceph=True) 2024-06-03T03:19:00.956 INFO:journalctl@ceph.mds.e.smithi151.stdout:Jun 03 03:19:00 smithi151 podman[66624]: 2024-06-03 03:19:00.955542969 +0000 UTC m=+0.369781457 container init 427fca5011668a65822af965d667dc996432a1fc724a1baf615b3b0f59178bdf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-e, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD) 2024-06-03T03:19:01.270 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-03T03:19:01.406 INFO:journalctl@ceph.mds.e.smithi151.stdout:Jun 03 03:19:00 smithi151 podman[66624]: 2024-06-03 03:19:00.958386407 +0000 UTC m=+0.372624904 container start 427fca5011668a65822af965d667dc996432a1fc724a1baf615b3b0f59178bdf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-e, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5) 2024-06-03T03:19:01.407 INFO:journalctl@ceph.mds.e.smithi151.stdout:Jun 03 03:19:00 smithi151 ceph-mds[66643]: -- 172.21.15.151:0/576005532 <== mon.2 v2:172.21.15.151:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556066ceda40 con 0x556066cc1000 2024-06-03T03:19:01.407 INFO:journalctl@ceph.mds.e.smithi151.stdout:Jun 03 03:19:00 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-e[66639]: starting mds.e at 2024-06-03T03:19:01.407 INFO:journalctl@ceph.mds.e.smithi151.stdout:Jun 03 03:19:01 smithi151 bash[66624]: 427fca5011668a65822af965d667dc996432a1fc724a1baf615b3b0f59178bdf 2024-06-03T03:19:01.407 INFO:journalctl@ceph.mds.e.smithi151.stdout:Jun 03 03:19:01 smithi151 systemd[1]: Started Ceph mds.e for a0e8473a-2156-11ef-bc9b-c7b262605968. 2024-06-03T03:19:01.718 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:19:02.116 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:01 smithi117 ceph-mon[31649]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:19:02.117 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:01 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:02.117 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:01 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:02.117 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:01 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:02.117 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:01 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T03:19:02.117 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:01 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T03:19:02.117 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:01 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:19:02.118 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:01 smithi183 ceph-mon[37326]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:19:02.118 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:01 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:02.118 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:01 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:02.118 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:01 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:02.118 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:01 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T03:19:02.119 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:01 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T03:19:02.119 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:01 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:19:02.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:01 smithi151 ceph-mon[37043]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:19:02.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:01 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:02.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:01 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:02.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:01 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:02.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:01 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T03:19:02.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:01 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T03:19:02.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:01 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:19:03.154 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:02 smithi183 ceph-mon[37326]: Deploying daemon mds.f on smithi183 2024-06-03T03:19:03.154 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:02 smithi183 ceph-mon[37326]: mds.? [v2:172.21.15.151:6834/31153579,v1:172.21.15.151:6835/31153579] up:boot 2024-06-03T03:19:03.154 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:02 smithi183 ceph-mon[37326]: fsmap 5 up:standby 2024-06-03T03:19:03.154 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:02 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-03T03:19:03.156 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:02 smithi117 ceph-mon[31649]: Deploying daemon mds.f on smithi183 2024-06-03T03:19:03.156 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:02 smithi117 ceph-mon[31649]: mds.? [v2:172.21.15.151:6834/31153579,v1:172.21.15.151:6835/31153579] up:boot 2024-06-03T03:19:03.156 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:02 smithi117 ceph-mon[31649]: fsmap 5 up:standby 2024-06-03T03:19:03.157 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:02 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-03T03:19:03.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:02 smithi151 ceph-mon[37043]: Deploying daemon mds.f on smithi183 2024-06-03T03:19:03.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:02 smithi151 ceph-mon[37043]: mds.? [v2:172.21.15.151:6834/31153579,v1:172.21.15.151:6835/31153579] up:boot 2024-06-03T03:19:03.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:02 smithi151 ceph-mon[37043]: fsmap 5 up:standby 2024-06-03T03:19:03.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:02 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-03T03:19:03.482 INFO:journalctl@ceph.mds.f.smithi183.stdout:Jun 03 03:19:03 smithi183 systemd[1]: Starting Ceph mds.f for a0e8473a-2156-11ef-bc9b-c7b262605968... 2024-06-03T03:19:03.790 INFO:journalctl@ceph.mds.f.smithi183.stdout:Jun 03 03:19:03 smithi183 podman[67278]: 2024-06-03 03:19:03.440708183 +0000 UTC m=+0.016708977 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T03:19:03.790 INFO:journalctl@ceph.mds.f.smithi183.stdout:Jun 03 03:19:03 smithi183 podman[67278]: 2024-06-03 03:19:03.551532535 +0000 UTC m=+0.127533330 container create 9477330b41945ed7fd5ed49deb4a6f497382a7dcfcea2a04570a233589b6b131 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-f, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=squid-3d89acc, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2) 2024-06-03T03:19:03.790 INFO:journalctl@ceph.mds.f.smithi183.stdout:Jun 03 03:19:03 smithi183 podman[67278]: 2024-06-03 03:19:03.787365257 +0000 UTC m=+0.363366042 container init 9477330b41945ed7fd5ed49deb4a6f497382a7dcfcea2a04570a233589b6b131 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-f, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-06-03T03:19:03.790 INFO:journalctl@ceph.mds.f.smithi183.stdout:Jun 03 03:19:03 smithi183 podman[67278]: 2024-06-03 03:19:03.789796286 +0000 UTC m=+0.365797082 container start 9477330b41945ed7fd5ed49deb4a6f497382a7dcfcea2a04570a233589b6b131 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-f, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-06-03T03:19:03.993 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:03 smithi117 ceph-mon[31649]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:19:04.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:04.051+0000 7fbe0b600700 1 -- 172.21.15.117:0/1310105463 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe04105070 msgr2=0x7fbe04111580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:19:04.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:04.051+0000 7fbe0b600700 1 --2- 172.21.15.117:0/1310105463 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe04105070 0x7fbe04111580 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc00b0a0 tx=0x7fbdfc00b3b0 comp rx=0 tx=0).stop 2024-06-03T03:19:04.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:04.051+0000 7fbe0b600700 1 -- 172.21.15.117:0/1310105463 shutdown_connections 2024-06-03T03:19:04.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:04.051+0000 7fbe0b600700 1 --2- 172.21.15.117:0/1310105463 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe04105070 0x7fbe04111580 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:04.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:04.051+0000 7fbe0b600700 1 --2- 172.21.15.117:0/1310105463 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe0410a6b0 0x7fbe0410aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:04.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:04.051+0000 7fbe0b600700 1 --2- 172.21.15.117:0/1310105463 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbe041046b0 0x7fbe04104b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:04.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:04.051+0000 7fbe0b600700 1 -- 172.21.15.117:0/1310105463 >> 172.21.15.117:0/1310105463 conn(0x7fbe041001a0 msgr2=0x7fbe041025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:19:04.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:04.051+0000 7fbe0b600700 1 -- 172.21.15.117:0/1310105463 shutdown_connections 2024-06-03T03:19:04.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:04.051+0000 7fbe0b600700 1 -- 172.21.15.117:0/1310105463 wait complete. 2024-06-03T03:19:04.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:04.053+0000 7fbe0b600700 1 Processor -- start 2024-06-03T03:19:04.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:04.053+0000 7fbe0b600700 1 -- start start 2024-06-03T03:19:04.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:04.053+0000 7fbe0b600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe041046b0 0x7fbe0419ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:19:04.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:04.053+0000 7fbe0b600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbe04105070 0x7fbe0419b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:19:04.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:04.053+0000 7fbe0b600700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe0410a6b0 0x7fbe0419f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:19:04.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:04.053+0000 7fbe0b600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbe04114a40 con 0x7fbe041046b0 2024-06-03T03:19:04.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:04.053+0000 7fbe0b600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbe041148c0 con 0x7fbe04105070 2024-06-03T03:19:04.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:04.053+0000 7fbe0b600700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbe04114bc0 con 0x7fbe0410a6b0 2024-06-03T03:19:04.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:04.054+0000 7fbe08c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe041046b0 0x7fbe0419ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:19:04.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:04.054+0000 7fbe09600700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe0410a6b0 0x7fbe0419f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:19:04.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:04.054+0000 7fbe09600700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe0410a6b0 0x7fbe0419f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:51696/0 (socket says 172.21.15.117:51696) 2024-06-03T03:19:04.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:04.054+0000 7fbe08c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe041046b0 0x7fbe0419ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41734/0 (socket says 172.21.15.117:41734) 2024-06-03T03:19:04.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:04.054+0000 7fbe09600700 1 -- 172.21.15.117:0/4227058590 learned_addr learned my addr 172.21.15.117:0/4227058590 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:19:04.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:04.054+0000 7fbe03e00700 1 --2- 172.21.15.117:0/4227058590 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbe04105070 0x7fbe0419b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:19:04.056 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:04.054+0000 7fbe08c00700 1 -- 172.21.15.117:0/4227058590 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe0410a6b0 msgr2=0x7fbe0419f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:19:04.057 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:04.054+0000 7fbe08c00700 1 --2- 172.21.15.117:0/4227058590 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe0410a6b0 0x7fbe0419f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:04.057 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:04.054+0000 7fbe08c00700 1 -- 172.21.15.117:0/4227058590 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbe04105070 msgr2=0x7fbe0419b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:19:04.057 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:04.054+0000 7fbe08c00700 1 --2- 172.21.15.117:0/4227058590 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbe04105070 0x7fbe0419b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:04.057 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:04.054+0000 7fbe08c00700 1 -- 172.21.15.117:0/4227058590 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe0419fdf0 con 0x7fbe041046b0 2024-06-03T03:19:04.057 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:04.055+0000 7fbe08c00700 1 --2- 172.21.15.117:0/4227058590 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe041046b0 0x7fbe0419ae30 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fbdf800d750 tx=0x7fbdf800da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:19:04.057 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:04.055+0000 7fbe01600700 1 -- 172.21.15.117:0/4227058590 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf8014030 con 0x7fbe041046b0 2024-06-03T03:19:04.057 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:04.055+0000 7fbe01600700 1 -- 172.21.15.117:0/4227058590 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbdf8004500 con 0x7fbe041046b0 2024-06-03T03:19:04.057 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:04.055+0000 7fbe0b600700 1 -- 172.21.15.117:0/4227058590 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe041a00e0 con 0x7fbe041046b0 2024-06-03T03:19:04.057 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:04.055+0000 7fbe01600700 1 -- 172.21.15.117:0/4227058590 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf8005420 con 0x7fbe041046b0 2024-06-03T03:19:04.057 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:04.055+0000 7fbe0b600700 1 -- 172.21.15.117:0/4227058590 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe041a7960 con 0x7fbe041046b0 2024-06-03T03:19:04.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:04.056+0000 7fbe0b600700 1 -- 172.21.15.117:0/4227058590 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe04069180 con 0x7fbe041046b0 2024-06-03T03:19:04.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:04.058+0000 7fbe01600700 1 -- 172.21.15.117:0/4227058590 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fbdf8004750 con 0x7fbe041046b0 2024-06-03T03:19:04.060 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:04.060+0000 7fbe01600700 1 --2- 172.21.15.117:0/4227058590 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fbdec0a2c20 0x7fbdec0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:19:04.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:04.060+0000 7fbe01600700 1 -- 172.21.15.117:0/4227058590 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fbdf80ce800 con 0x7fbe041046b0 2024-06-03T03:19:04.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:04.060+0000 7fbe03e00700 1 --2- 172.21.15.117:0/4227058590 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fbdec0a2c20 0x7fbdec0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:19:04.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:04.061+0000 7fbe03e00700 1 --2- 172.21.15.117:0/4227058590 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fbdec0a2c20 0x7fbdec0a50e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fbe04109c00 tx=0x7fbdf0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:19:04.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:04.065+0000 7fbe01600700 1 -- 172.21.15.117:0/4227058590 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fbdf809bb60 con 0x7fbe041046b0 2024-06-03T03:19:04.232 INFO:journalctl@ceph.mds.f.smithi183.stdout:Jun 03 03:19:03 smithi183 ceph-mds[67297]: -- 172.21.15.183:0/1548968543 <== mon.1 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e5c0041a40 con 0x55e5c0015000 2024-06-03T03:19:04.232 INFO:journalctl@ceph.mds.f.smithi183.stdout:Jun 03 03:19:03 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-f[67293]: starting mds.f at 2024-06-03T03:19:04.232 INFO:journalctl@ceph.mds.f.smithi183.stdout:Jun 03 03:19:03 smithi183 bash[67278]: 9477330b41945ed7fd5ed49deb4a6f497382a7dcfcea2a04570a233589b6b131 2024-06-03T03:19:04.232 INFO:journalctl@ceph.mds.f.smithi183.stdout:Jun 03 03:19:03 smithi183 systemd[1]: Started Ceph mds.f for a0e8473a-2156-11ef-bc9b-c7b262605968. 2024-06-03T03:19:04.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:03 smithi183 ceph-mon[37326]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:19:04.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:04.338+0000 7fbe0b600700 1 -- 172.21.15.117:0/4227058590 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbe0402dca0 con 0x7fbe041046b0 2024-06-03T03:19:04.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:04.339+0000 7fbe01600700 1 -- 172.21.15.117:0/4227058590 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5530 (secure 0 0 0) 0x7fbdf809b460 con 0x7fbe041046b0 2024-06-03T03:19:04.341 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T03:19:04.341 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3353001119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3353001119},{"type":"v1","addr":"172.21.15.117:6837","nonce":3353001119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3355662416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3355662416},{"type":"v1","addr":"172.21.15.151:6833","nonce":3355662416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/2429725085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2429725085},{"type":"v1","addr":"172.21.15.183:6833","nonce":2429725085}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/31153579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":31153579},{"type":"v1","addr":"172.21.15.151:6835","nonce":31153579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24466,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/2884460088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2884460088},{"type":"v1","addr":"172.21.15.117:6835","nonce":2884460088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24487,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/1009004320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1009004320},{"type":"v1","addr":"172.21.15.183:6835","nonce":1009004320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-06-03T03:19:04.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:04.343+0000 7fbe0b600700 1 -- 172.21.15.117:0/4227058590 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fbdec0a2c20 msgr2=0x7fbdec0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:19:04.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:04.343+0000 7fbe0b600700 1 --2- 172.21.15.117:0/4227058590 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fbdec0a2c20 0x7fbdec0a50e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fbe04109c00 tx=0x7fbdf0009260 comp rx=0 tx=0).stop 2024-06-03T03:19:04.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:04.343+0000 7fbe0b600700 1 -- 172.21.15.117:0/4227058590 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe041046b0 msgr2=0x7fbe0419ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:19:04.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:04.343+0000 7fbe0b600700 1 --2- 172.21.15.117:0/4227058590 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe041046b0 0x7fbe0419ae30 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fbdf800d750 tx=0x7fbdf800da60 comp rx=0 tx=0).stop 2024-06-03T03:19:04.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:04.343+0000 7fbe0b600700 1 -- 172.21.15.117:0/4227058590 shutdown_connections 2024-06-03T03:19:04.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:04.343+0000 7fbe0b600700 1 --2- 172.21.15.117:0/4227058590 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fbdec0a2c20 0x7fbdec0a50e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:04.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:04.343+0000 7fbe0b600700 1 --2- 172.21.15.117:0/4227058590 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe0410a6b0 0x7fbe0419f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:04.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:04.343+0000 7fbe0b600700 1 --2- 172.21.15.117:0/4227058590 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe041046b0 0x7fbe0419ae30 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:04.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:04.343+0000 7fbe0b600700 1 --2- 172.21.15.117:0/4227058590 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbe04105070 0x7fbe0419b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:04.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:04.343+0000 7fbe0b600700 1 -- 172.21.15.117:0/4227058590 >> 172.21.15.117:0/4227058590 conn(0x7fbe041001a0 msgr2=0x7fbe041006c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:19:04.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:04.343+0000 7fbe0b600700 1 -- 172.21.15.117:0/4227058590 shutdown_connections 2024-06-03T03:19:04.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:04.343+0000 7fbe0b600700 1 -- 172.21.15.117:0/4227058590 wait complete. 2024-06-03T03:19:04.345 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 7 2024-06-03T03:19:04.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:03 smithi151 ceph-mon[37043]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:19:05.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:04 smithi117 ceph-mon[31649]: mds.? [v2:172.21.15.183:6834/1009004320,v1:172.21.15.183:6835/1009004320] up:boot 2024-06-03T03:19:05.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:04 smithi117 ceph-mon[31649]: fsmap 6 up:standby 2024-06-03T03:19:05.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:04 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-03T03:19:05.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:04 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:05.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:04 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:05.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:04 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:05.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:04 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T03:19:05.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:04 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T03:19:05.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:04 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:19:05.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:04 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/4227058590' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T03:19:05.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:04 smithi183 ceph-mon[37326]: mds.? [v2:172.21.15.183:6834/1009004320,v1:172.21.15.183:6835/1009004320] up:boot 2024-06-03T03:19:05.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:04 smithi183 ceph-mon[37326]: fsmap 6 up:standby 2024-06-03T03:19:05.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:04 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-03T03:19:05.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:04 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:05.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:04 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:05.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:04 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:05.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:04 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T03:19:05.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:04 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T03:19:05.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:04 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:19:05.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:04 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/4227058590' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T03:19:05.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:04 smithi151 ceph-mon[37043]: mds.? [v2:172.21.15.183:6834/1009004320,v1:172.21.15.183:6835/1009004320] up:boot 2024-06-03T03:19:05.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:04 smithi151 ceph-mon[37043]: fsmap 6 up:standby 2024-06-03T03:19:05.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:04 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-03T03:19:05.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:04 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:05.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:04 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:05.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:04 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:05.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:04 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T03:19:05.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:04 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T03:19:05.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:04 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:19:05.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:04 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/4227058590' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T03:19:06.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:05 smithi117 ceph-mon[31649]: Deploying daemon mds.c on smithi183 2024-06-03T03:19:06.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:05 smithi117 ceph-mon[31649]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:19:06.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:05 smithi183 ceph-mon[37326]: Deploying daemon mds.c on smithi183 2024-06-03T03:19:06.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:05 smithi183 ceph-mon[37326]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:19:06.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:05 smithi151 ceph-mon[37043]: Deploying daemon mds.c on smithi183 2024-06-03T03:19:06.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:05 smithi151 ceph-mon[37043]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:19:06.982 INFO:journalctl@ceph.mds.c.smithi183.stdout:Jun 03 03:19:06 smithi183 systemd[1]: Started Ceph mds.c for a0e8473a-2156-11ef-bc9b-c7b262605968. 2024-06-03T03:19:07.259 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-03T03:19:07.714 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:19:08.104 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:07 smithi117 ceph-mon[31649]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:19:08.104 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:07 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:08.104 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:07 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:08.104 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:07 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:08.104 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:07 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T03:19:08.104 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:07 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T03:19:08.104 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:07 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:19:08.104 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:07 smithi117 ceph-mon[31649]: Deploying daemon mds.h on smithi151 2024-06-03T03:19:08.105 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:07 smithi117 ceph-mon[31649]: mds.? [v2:172.21.15.183:6836/2697518526,v1:172.21.15.183:6837/2697518526] up:boot 2024-06-03T03:19:08.105 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:07 smithi117 ceph-mon[31649]: fsmap 7 up:standby 2024-06-03T03:19:08.105 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:07 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-03T03:19:08.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:07 smithi151 ceph-mon[37043]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:19:08.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:07 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:08.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:07 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:08.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:07 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:08.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:07 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T03:19:08.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:07 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T03:19:08.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:07 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:19:08.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:07 smithi151 ceph-mon[37043]: Deploying daemon mds.h on smithi151 2024-06-03T03:19:08.158 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:07 smithi151 ceph-mon[37043]: mds.? [v2:172.21.15.183:6836/2697518526,v1:172.21.15.183:6837/2697518526] up:boot 2024-06-03T03:19:08.158 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:07 smithi151 ceph-mon[37043]: fsmap 7 up:standby 2024-06-03T03:19:08.158 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:07 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-03T03:19:08.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:07 smithi183 ceph-mon[37326]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:19:08.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:07 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:08.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:07 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:08.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:07 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:08.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:07 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T03:19:08.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:07 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T03:19:08.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:07 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:19:08.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:07 smithi183 ceph-mon[37326]: Deploying daemon mds.h on smithi151 2024-06-03T03:19:08.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:07 smithi183 ceph-mon[37326]: mds.? [v2:172.21.15.183:6836/2697518526,v1:172.21.15.183:6837/2697518526] up:boot 2024-06-03T03:19:08.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:07 smithi183 ceph-mon[37326]: fsmap 7 up:standby 2024-06-03T03:19:08.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:07 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-03T03:19:09.157 INFO:journalctl@ceph.mds.h.smithi151.stdout:Jun 03 03:19:09 smithi151 systemd[1]: Starting Ceph mds.h for a0e8473a-2156-11ef-bc9b-c7b262605968... 2024-06-03T03:19:09.493 INFO:journalctl@ceph.mds.h.smithi151.stdout:Jun 03 03:19:09 smithi151 podman[66927]: 2024-06-03 03:19:09.118497141 +0000 UTC m=+0.017228482 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T03:19:09.494 INFO:journalctl@ceph.mds.h.smithi151.stdout:Jun 03 03:19:09 smithi151 podman[66927]: 2024-06-03 03:19:09.240710486 +0000 UTC m=+0.139441857 container create ec4dc6e6e0197519548bd55228fd58c49aa5baff6d11cfa517061b80d7229a26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-h, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-06-03T03:19:09.494 INFO:journalctl@ceph.mds.h.smithi151.stdout:Jun 03 03:19:09 smithi151 podman[66927]: 2024-06-03 03:19:09.493409959 +0000 UTC m=+0.392141279 container init ec4dc6e6e0197519548bd55228fd58c49aa5baff6d11cfa517061b80d7229a26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-h, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, ceph=True, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-03T03:19:09.857 INFO:journalctl@ceph.mds.h.smithi151.stdout:Jun 03 03:19:09 smithi151 podman[66927]: 2024-06-03 03:19:09.496112196 +0000 UTC m=+0.394843518 container start ec4dc6e6e0197519548bd55228fd58c49aa5baff6d11cfa517061b80d7229a26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-h, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-06-03T03:19:09.857 INFO:journalctl@ceph.mds.h.smithi151.stdout:Jun 03 03:19:09 smithi151 ceph-mds[66946]: -- 172.21.15.151:0/4264269013 <== mon.2 v2:172.21.15.151:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e84e707a40 con 0x55e84e6dac00 2024-06-03T03:19:09.857 INFO:journalctl@ceph.mds.h.smithi151.stdout:Jun 03 03:19:09 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-h[66942]: starting mds.h at 2024-06-03T03:19:09.857 INFO:journalctl@ceph.mds.h.smithi151.stdout:Jun 03 03:19:09 smithi151 bash[66927]: ec4dc6e6e0197519548bd55228fd58c49aa5baff6d11cfa517061b80d7229a26 2024-06-03T03:19:09.857 INFO:journalctl@ceph.mds.h.smithi151.stdout:Jun 03 03:19:09 smithi151 systemd[1]: Started Ceph mds.h for a0e8473a-2156-11ef-bc9b-c7b262605968. 2024-06-03T03:19:10.120 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:09 smithi117 ceph-mon[31649]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:19:10.120 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:09 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:10.120 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:09 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:10.120 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:09 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:10.120 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:09 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T03:19:10.120 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:09 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T03:19:10.121 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:09 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:19:10.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:10.118+0000 7fcf6cc00700 1 -- 172.21.15.117:0/2109443662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf681046a0 msgr2=0x7fcf68104b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:19:10.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:10.118+0000 7fcf6cc00700 1 --2- 172.21.15.117:0/2109443662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf681046a0 0x7fcf68104b20 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fcf54009990 tx=0x7fcf54009ca0 comp rx=0 tx=0).stop 2024-06-03T03:19:10.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:10.118+0000 7fcf6cc00700 1 -- 172.21.15.117:0/2109443662 shutdown_connections 2024-06-03T03:19:10.122 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:10.118+0000 7fcf6cc00700 1 --2- 172.21.15.117:0/2109443662 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf68105060 0x7fcf68111570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:10.122 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:10.118+0000 7fcf6cc00700 1 --2- 172.21.15.117:0/2109443662 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf6810a6a0 0x7fcf6810aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:10.122 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:10.118+0000 7fcf6cc00700 1 --2- 172.21.15.117:0/2109443662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf681046a0 0x7fcf68104b20 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:10.122 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:10.118+0000 7fcf6cc00700 1 -- 172.21.15.117:0/2109443662 >> 172.21.15.117:0/2109443662 conn(0x7fcf68100190 msgr2=0x7fcf681025b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:19:10.122 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:10.118+0000 7fcf6cc00700 1 -- 172.21.15.117:0/2109443662 shutdown_connections 2024-06-03T03:19:10.122 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:10.118+0000 7fcf6cc00700 1 -- 172.21.15.117:0/2109443662 wait complete. 2024-06-03T03:19:10.122 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:10.120+0000 7fcf6cc00700 1 Processor -- start 2024-06-03T03:19:10.122 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:10.120+0000 7fcf6cc00700 1 -- start start 2024-06-03T03:19:10.122 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:10.120+0000 7fcf6cc00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf681046a0 0x7fcf6819aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:19:10.122 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:10.120+0000 7fcf6cc00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf68105060 0x7fcf6819b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:19:10.122 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:10.120+0000 7fcf6cc00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf6810a6a0 0x7fcf6819f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:19:10.122 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:10.120+0000 7fcf6cc00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcf68114a80 con 0x7fcf681046a0 2024-06-03T03:19:10.122 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:10.120+0000 7fcf6cc00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcf68114900 con 0x7fcf68105060 2024-06-03T03:19:10.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:10.120+0000 7fcf6cc00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcf68114c00 con 0x7fcf6810a6a0 2024-06-03T03:19:10.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:10.120+0000 7fcf65e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf681046a0 0x7fcf6819aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:19:10.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:10.121+0000 7fcf66800700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf6810a6a0 0x7fcf6819f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:19:10.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:10.121+0000 7fcf65e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf681046a0 0x7fcf6819aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41978/0 (socket says 172.21.15.117:41978) 2024-06-03T03:19:10.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:10.121+0000 7fcf66800700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf6810a6a0 0x7fcf6819f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:49396/0 (socket says 172.21.15.117:49396) 2024-06-03T03:19:10.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:10.121+0000 7fcf65e00700 1 -- 172.21.15.117:0/1263887574 learned_addr learned my addr 172.21.15.117:0/1263887574 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:19:10.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:10.121+0000 7fcf65400700 1 --2- 172.21.15.117:0/1263887574 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf68105060 0x7fcf6819b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:19:10.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:10.121+0000 7fcf65e00700 1 -- 172.21.15.117:0/1263887574 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf6810a6a0 msgr2=0x7fcf6819f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:19:10.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:10.121+0000 7fcf65e00700 1 --2- 172.21.15.117:0/1263887574 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf6810a6a0 0x7fcf6819f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:10.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:10.121+0000 7fcf65e00700 1 -- 172.21.15.117:0/1263887574 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf68105060 msgr2=0x7fcf6819b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:19:10.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:10.121+0000 7fcf65e00700 1 --2- 172.21.15.117:0/1263887574 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf68105060 0x7fcf6819b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:10.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:10.121+0000 7fcf65e00700 1 -- 172.21.15.117:0/1263887574 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf6819fe90 con 0x7fcf681046a0 2024-06-03T03:19:10.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:10.121+0000 7fcf65400700 1 --2- 172.21.15.117:0/1263887574 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf68105060 0x7fcf6819b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:19:10.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:10.121+0000 7fcf65e00700 1 --2- 172.21.15.117:0/1263887574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf681046a0 0x7fcf6819aed0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fcf5800e9a0 tx=0x7fcf5800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:19:10.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:10.122+0000 7fcf5ea00700 1 -- 172.21.15.117:0/1263887574 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5800ca10 con 0x7fcf681046a0 2024-06-03T03:19:10.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:10.122+0000 7fcf5ea00700 1 -- 172.21.15.117:0/1263887574 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf5800cbb0 con 0x7fcf681046a0 2024-06-03T03:19:10.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:10.122+0000 7fcf5ea00700 1 -- 172.21.15.117:0/1263887574 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf58018680 con 0x7fcf681046a0 2024-06-03T03:19:10.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:10.122+0000 7fcf6cc00700 1 -- 172.21.15.117:0/1263887574 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf681a0180 con 0x7fcf681046a0 2024-06-03T03:19:10.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:10.122+0000 7fcf6cc00700 1 -- 172.21.15.117:0/1263887574 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf681a7a00 con 0x7fcf681046a0 2024-06-03T03:19:10.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:10.123+0000 7fcf6cc00700 1 -- 172.21.15.117:0/1263887574 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf68069180 con 0x7fcf681046a0 2024-06-03T03:19:10.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:10.126+0000 7fcf5ea00700 1 -- 172.21.15.117:0/1263887574 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fcf58010430 con 0x7fcf681046a0 2024-06-03T03:19:10.128 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:10.127+0000 7fcf5ea00700 1 --2- 172.21.15.117:0/1263887574 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fcf4c0a2c20 0x7fcf4c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:19:10.128 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:10.127+0000 7fcf65400700 1 --2- 172.21.15.117:0/1263887574 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fcf4c0a2c20 0x7fcf4c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:19:10.128 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:10.127+0000 7fcf5ea00700 1 -- 172.21.15.117:0/1263887574 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fcf580cedf0 con 0x7fcf681046a0 2024-06-03T03:19:10.129 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:10.128+0000 7fcf65400700 1 --2- 172.21.15.117:0/1263887574 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fcf4c0a2c20 0x7fcf4c0a50e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fcf68102800 tx=0x7fcf5400b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:19:10.132 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:10.132+0000 7fcf5ea00700 1 -- 172.21.15.117:0/1263887574 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fcf5809c150 con 0x7fcf681046a0 2024-06-03T03:19:10.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:09 smithi151 ceph-mon[37043]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:19:10.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:09 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:10.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:09 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:10.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:09 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:10.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:09 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T03:19:10.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:09 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T03:19:10.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:09 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:19:10.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:09 smithi183 ceph-mon[37326]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:19:10.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:09 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:10.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:09 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:10.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:09 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:10.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:09 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T03:19:10.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:09 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T03:19:10.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:09 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:19:10.402 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:10.400+0000 7fcf6cc00700 1 -- 172.21.15.117:0/1263887574 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcf6802dca0 con 0x7fcf681046a0 2024-06-03T03:19:10.403 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:10.402+0000 7fcf5ea00700 1 -- 172.21.15.117:0/1263887574 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7186 (secure 0 0 0) 0x7fcf5809ba50 con 0x7fcf681046a0 2024-06-03T03:19:10.403 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T03:19:10.404 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3353001119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3353001119},{"type":"v1","addr":"172.21.15.117:6837","nonce":3353001119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3355662416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3355662416},{"type":"v1","addr":"172.21.15.151:6833","nonce":3355662416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/2429725085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2429725085},{"type":"v1","addr":"172.21.15.183:6833","nonce":2429725085}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/31153579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":31153579},{"type":"v1","addr":"172.21.15.151:6835","nonce":31153579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24466,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/2884460088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2884460088},{"type":"v1","addr":"172.21.15.117:6835","nonce":2884460088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24479,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/2392407848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":2392407848},{"type":"v1","addr":"172.21.15.151:6837","nonce":2392407848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24487,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/1009004320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1009004320},{"type":"v1","addr":"172.21.15.183:6835","nonce":1009004320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24499,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/2697518526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":2697518526},{"type":"v1","addr":"172.21.15.183:6837","nonce":2697518526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-06-03T03:19:10.406 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:10.405+0000 7fcf6cc00700 1 -- 172.21.15.117:0/1263887574 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fcf4c0a2c20 msgr2=0x7fcf4c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:19:10.406 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:10.405+0000 7fcf6cc00700 1 --2- 172.21.15.117:0/1263887574 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fcf4c0a2c20 0x7fcf4c0a50e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fcf68102800 tx=0x7fcf5400b260 comp rx=0 tx=0).stop 2024-06-03T03:19:10.406 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:10.405+0000 7fcf6cc00700 1 -- 172.21.15.117:0/1263887574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf681046a0 msgr2=0x7fcf6819aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:19:10.406 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:10.405+0000 7fcf6cc00700 1 --2- 172.21.15.117:0/1263887574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf681046a0 0x7fcf6819aed0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fcf5800e9a0 tx=0x7fcf5800ecb0 comp rx=0 tx=0).stop 2024-06-03T03:19:10.407 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:10.405+0000 7fcf6cc00700 1 -- 172.21.15.117:0/1263887574 shutdown_connections 2024-06-03T03:19:10.407 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:10.405+0000 7fcf6cc00700 1 --2- 172.21.15.117:0/1263887574 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fcf4c0a2c20 0x7fcf4c0a50e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:10.407 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:10.405+0000 7fcf6cc00700 1 --2- 172.21.15.117:0/1263887574 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf6810a6a0 0x7fcf6819f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:10.407 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:10.405+0000 7fcf6cc00700 1 --2- 172.21.15.117:0/1263887574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf681046a0 0x7fcf6819aed0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:10.407 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:10.405+0000 7fcf6cc00700 1 --2- 172.21.15.117:0/1263887574 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf68105060 0x7fcf6819b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:10.407 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:10.405+0000 7fcf6cc00700 1 -- 172.21.15.117:0/1263887574 >> 172.21.15.117:0/1263887574 conn(0x7fcf68100190 msgr2=0x7fcf68100980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:19:10.407 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:10.405+0000 7fcf6cc00700 1 -- 172.21.15.117:0/1263887574 shutdown_connections 2024-06-03T03:19:10.407 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:10.405+0000 7fcf6cc00700 1 -- 172.21.15.117:0/1263887574 wait complete. 2024-06-03T03:19:10.408 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 9 2024-06-03T03:19:11.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:10 smithi151 ceph-mon[37043]: Deploying daemon mds.i on smithi183 2024-06-03T03:19:11.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:10 smithi151 ceph-mon[37043]: mds.? [v2:172.21.15.151:6836/2392407848,v1:172.21.15.151:6837/2392407848] up:boot 2024-06-03T03:19:11.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:10 smithi151 ceph-mon[37043]: fsmap 8 up:standby 2024-06-03T03:19:11.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:10 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-03T03:19:11.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:10 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/1263887574' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T03:19:11.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:10 smithi117 ceph-mon[31649]: Deploying daemon mds.i on smithi183 2024-06-03T03:19:11.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:10 smithi117 ceph-mon[31649]: mds.? [v2:172.21.15.151:6836/2392407848,v1:172.21.15.151:6837/2392407848] up:boot 2024-06-03T03:19:11.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:10 smithi117 ceph-mon[31649]: fsmap 8 up:standby 2024-06-03T03:19:11.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:10 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-03T03:19:11.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:10 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/1263887574' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T03:19:11.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:10 smithi183 ceph-mon[37326]: Deploying daemon mds.i on smithi183 2024-06-03T03:19:11.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:10 smithi183 ceph-mon[37326]: mds.? [v2:172.21.15.151:6836/2392407848,v1:172.21.15.151:6837/2392407848] up:boot 2024-06-03T03:19:11.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:10 smithi183 ceph-mon[37326]: fsmap 8 up:standby 2024-06-03T03:19:11.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:10 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-03T03:19:11.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:10 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/1263887574' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T03:19:12.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:11 smithi151 ceph-mon[37043]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:19:12.179 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:11 smithi183 ceph-mon[37326]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:19:12.179 INFO:journalctl@ceph.mds.i.smithi183.stdout:Jun 03 03:19:11 smithi183 systemd[1]: Starting Ceph mds.i for a0e8473a-2156-11ef-bc9b-c7b262605968... 2024-06-03T03:19:12.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:11 smithi117 ceph-mon[31649]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:19:12.442 INFO:journalctl@ceph.mds.i.smithi183.stdout:Jun 03 03:19:12 smithi183 podman[67894]: 2024-06-03 03:19:12.081506251 +0000 UTC m=+0.017193945 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T03:19:12.442 INFO:journalctl@ceph.mds.i.smithi183.stdout:Jun 03 03:19:12 smithi183 podman[67894]: 2024-06-03 03:19:12.196950128 +0000 UTC m=+0.132637803 container create 71fa4d2bfe2ca211e5ad7fbe9d3510e2971251ce7223b690ad687f07daaf61ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-i, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-03T03:19:12.442 INFO:journalctl@ceph.mds.i.smithi183.stdout:Jun 03 03:19:12 smithi183 podman[67894]: 2024-06-03 03:19:12.416066747 +0000 UTC m=+0.351754423 container init 71fa4d2bfe2ca211e5ad7fbe9d3510e2971251ce7223b690ad687f07daaf61ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-06-03T03:19:12.442 INFO:journalctl@ceph.mds.i.smithi183.stdout:Jun 03 03:19:12 smithi183 podman[67894]: 2024-06-03 03:19:12.418620484 +0000 UTC m=+0.354308172 container start 71fa4d2bfe2ca211e5ad7fbe9d3510e2971251ce7223b690ad687f07daaf61ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc) 2024-06-03T03:19:12.732 INFO:journalctl@ceph.mds.i.smithi183.stdout:Jun 03 03:19:12 smithi183 ceph-mds[67913]: -- 172.21.15.183:0/369956614 <== mon.1 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5626c87c9a40 con 0x5626c879cc00 2024-06-03T03:19:12.732 INFO:journalctl@ceph.mds.i.smithi183.stdout:Jun 03 03:19:12 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-i[67909]: starting mds.i at 2024-06-03T03:19:12.732 INFO:journalctl@ceph.mds.i.smithi183.stdout:Jun 03 03:19:12 smithi183 bash[67894]: 71fa4d2bfe2ca211e5ad7fbe9d3510e2971251ce7223b690ad687f07daaf61ef 2024-06-03T03:19:12.733 INFO:journalctl@ceph.mds.i.smithi183.stdout:Jun 03 03:19:12 smithi183 systemd[1]: Started Ceph mds.i for a0e8473a-2156-11ef-bc9b-c7b262605968. 2024-06-03T03:19:13.399 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-03T03:19:13.837 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:13 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:13.838 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:13 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:13.838 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:13 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:13.838 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:13 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T03:19:13.838 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:13 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T03:19:13.838 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:13 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:19:13.838 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:13 smithi117 ceph-mon[31649]: Deploying daemon mds.g on smithi117 2024-06-03T03:19:13.838 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:13 smithi117 ceph-mon[31649]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:19:13.838 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:13 smithi117 ceph-mon[31649]: mds.? [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] up:boot 2024-06-03T03:19:13.838 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:13 smithi117 ceph-mon[31649]: fsmap 9 up:standby 2024-06-03T03:19:13.838 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:13 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-03T03:19:13.906 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:13 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:13.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:13 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:13.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:13 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:13.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:13 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T03:19:13.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:13 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T03:19:13.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:13 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:19:13.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:13 smithi151 ceph-mon[37043]: Deploying daemon mds.g on smithi117 2024-06-03T03:19:13.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:13 smithi151 ceph-mon[37043]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:19:13.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:13 smithi151 ceph-mon[37043]: mds.? [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] up:boot 2024-06-03T03:19:13.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:13 smithi151 ceph-mon[37043]: fsmap 9 up:standby 2024-06-03T03:19:13.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:13 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-03T03:19:13.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:13 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:13.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:13 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:13.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:13 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:13.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:13 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T03:19:13.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:13 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T03:19:13.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:13 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:19:13.983 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:13 smithi183 ceph-mon[37326]: Deploying daemon mds.g on smithi117 2024-06-03T03:19:13.983 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:13 smithi183 ceph-mon[37326]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:19:13.983 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:13 smithi183 ceph-mon[37326]: mds.? [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] up:boot 2024-06-03T03:19:13.983 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:13 smithi183 ceph-mon[37326]: fsmap 9 up:standby 2024-06-03T03:19:13.983 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:13 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-03T03:19:15.050 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:19:16.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:15 smithi151 ceph-mon[37043]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:19:16.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:15 smithi117 ceph-mon[31649]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:19:16.197 INFO:journalctl@ceph.mds.g.smithi117.stdout:Jun 03 03:19:15 smithi117 systemd[1]: Starting Ceph mds.g for a0e8473a-2156-11ef-bc9b-c7b262605968... 2024-06-03T03:19:16.231 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:15 smithi183 ceph-mon[37326]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:19:16.879 INFO:journalctl@ceph.mds.g.smithi117.stdout:Jun 03 03:19:16 smithi117 podman[70770]: 2024-06-03 03:19:16.690918652 +0000 UTC m=+0.032835839 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T03:19:16.879 INFO:journalctl@ceph.mds.g.smithi117.stdout:Jun 03 03:19:16 smithi117 podman[70770]: 2024-06-03 03:19:16.877296534 +0000 UTC m=+0.219213752 container create 7ea4d8f5f9746accef2fc49a6f4343d7231a9405df0e0f0077722066a5d8c1ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-g, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-03T03:19:17.441 INFO:journalctl@ceph.mds.g.smithi117.stdout:Jun 03 03:19:17 smithi117 podman[70770]: 2024-06-03 03:19:17.384022741 +0000 UTC m=+0.725939973 container init 7ea4d8f5f9746accef2fc49a6f4343d7231a9405df0e0f0077722066a5d8c1ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-g, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528) 2024-06-03T03:19:17.441 INFO:journalctl@ceph.mds.g.smithi117.stdout:Jun 03 03:19:17 smithi117 podman[70770]: 2024-06-03 03:19:17.392354357 +0000 UTC m=+0.734271565 container start 7ea4d8f5f9746accef2fc49a6f4343d7231a9405df0e0f0077722066a5d8c1ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-g, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.build-date=20240528) 2024-06-03T03:19:17.691 INFO:journalctl@ceph.mds.g.smithi117.stdout:Jun 03 03:19:17 smithi117 ceph-mds[70811]: -- 172.21.15.117:0/3910523017 <== mon.0 v2:172.21.15.117:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558df76c5a40 con 0x558df7698c00 2024-06-03T03:19:17.691 INFO:journalctl@ceph.mds.g.smithi117.stdout:Jun 03 03:19:17 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-g[70802]: starting mds.g at 2024-06-03T03:19:17.691 INFO:journalctl@ceph.mds.g.smithi117.stdout:Jun 03 03:19:17 smithi117 bash[70770]: 7ea4d8f5f9746accef2fc49a6f4343d7231a9405df0e0f0077722066a5d8c1ff 2024-06-03T03:19:17.691 INFO:journalctl@ceph.mds.g.smithi117.stdout:Jun 03 03:19:17 smithi117 systemd[1]: Started Ceph mds.g for a0e8473a-2156-11ef-bc9b-c7b262605968. 2024-06-03T03:19:17.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:17.685+0000 7fef85200700 1 -- 172.21.15.117:0/191467468 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef800746c0 msgr2=0x7fef80074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:19:17.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:17.685+0000 7fef85200700 1 --2- 172.21.15.117:0/191467468 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef800746c0 0x7fef80074aa0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fef70009990 tx=0x7fef70009ca0 comp rx=0 tx=0).stop 2024-06-03T03:19:17.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:17.685+0000 7fef85200700 1 -- 172.21.15.117:0/191467468 shutdown_connections 2024-06-03T03:19:17.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:17.685+0000 7fef85200700 1 --2- 172.21.15.117:0/191467468 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fef8010eef0 0x7fef801112e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:17.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:17.685+0000 7fef85200700 1 --2- 172.21.15.117:0/191467468 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef800746c0 0x7fef80074aa0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:17.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:17.685+0000 7fef85200700 1 --2- 172.21.15.117:0/191467468 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fef80074fe0 0x7fef8010e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:17.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:17.685+0000 7fef85200700 1 -- 172.21.15.117:0/191467468 >> 172.21.15.117:0/191467468 conn(0x7fef800795f0 msgr2=0x7fef80079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:19:17.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:17.685+0000 7fef85200700 1 -- 172.21.15.117:0/191467468 shutdown_connections 2024-06-03T03:19:17.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:17.685+0000 7fef85200700 1 -- 172.21.15.117:0/191467468 wait complete. 2024-06-03T03:19:17.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:17.686+0000 7fef85200700 1 Processor -- start 2024-06-03T03:19:17.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:17.686+0000 7fef85200700 1 -- start start 2024-06-03T03:19:17.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:17.686+0000 7fef85200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fef800746c0 0x7fef8019ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:19:17.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:17.686+0000 7fef85200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef80074fe0 0x7fef8019b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:19:17.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:17.686+0000 7fef85200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fef8010eef0 0x7fef8019f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:19:17.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:17.686+0000 7fef85200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fef801149e0 con 0x7fef80074fe0 2024-06-03T03:19:17.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:17.686+0000 7fef85200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fef80114860 con 0x7fef800746c0 2024-06-03T03:19:17.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:17.686+0000 7fef85200700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fef80114b60 con 0x7fef8010eef0 2024-06-03T03:19:17.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:17.687+0000 7fef7d400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef80074fe0 0x7fef8019b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:19:17.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:17.687+0000 7fef7de00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fef800746c0 0x7fef8019ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:19:17.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:17.687+0000 7fef7d400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef80074fe0 0x7fef8019b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:60698/0 (socket says 172.21.15.117:60698) 2024-06-03T03:19:17.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:17.687+0000 7fef7d400700 1 -- 172.21.15.117:0/2726300727 learned_addr learned my addr 172.21.15.117:0/2726300727 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:19:17.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:17.687+0000 7fef7e800700 1 --2- 172.21.15.117:0/2726300727 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fef8010eef0 0x7fef8019f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:19:17.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:17.687+0000 7fef7de00700 1 -- 172.21.15.117:0/2726300727 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fef8010eef0 msgr2=0x7fef8019f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:19:17.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:17.687+0000 7fef7de00700 1 --2- 172.21.15.117:0/2726300727 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fef8010eef0 0x7fef8019f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:17.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:17.687+0000 7fef7de00700 1 -- 172.21.15.117:0/2726300727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef80074fe0 msgr2=0x7fef8019b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:19:17.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:17.687+0000 7fef7de00700 1 --2- 172.21.15.117:0/2726300727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef80074fe0 0x7fef8019b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:17.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:17.687+0000 7fef7de00700 1 -- 172.21.15.117:0/2726300727 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef8019fe30 con 0x7fef800746c0 2024-06-03T03:19:17.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:17.688+0000 7fef7de00700 1 --2- 172.21.15.117:0/2726300727 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fef800746c0 0x7fef8019ae70 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fef70006010 tx=0x7fef7000bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:19:17.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:17.688+0000 7fef76a00700 1 -- 172.21.15.117:0/2726300727 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef70027070 con 0x7fef800746c0 2024-06-03T03:19:17.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:17.688+0000 7fef85200700 1 -- 172.21.15.117:0/2726300727 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef801a00c0 con 0x7fef800746c0 2024-06-03T03:19:17.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:17.688+0000 7fef76a00700 1 -- 172.21.15.117:0/2726300727 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef700056d0 con 0x7fef800746c0 2024-06-03T03:19:17.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:17.688+0000 7fef76a00700 1 -- 172.21.15.117:0/2726300727 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef70022d90 con 0x7fef800746c0 2024-06-03T03:19:17.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:17.688+0000 7fef85200700 1 -- 172.21.15.117:0/2726300727 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef801a79a0 con 0x7fef800746c0 2024-06-03T03:19:17.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:17.689+0000 7fef85200700 1 -- 172.21.15.117:0/2726300727 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef80069180 con 0x7fef800746c0 2024-06-03T03:19:17.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:17.692+0000 7fef76a00700 1 -- 172.21.15.117:0/2726300727 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fef7002a070 con 0x7fef800746c0 2024-06-03T03:19:17.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:17.694+0000 7fef76a00700 1 --2- 172.21.15.117:0/2726300727 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fef640a2c20 0x7fef640a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:19:17.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:17.694+0000 7fef76a00700 1 -- 172.21.15.117:0/2726300727 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fef700d0340 con 0x7fef800746c0 2024-06-03T03:19:17.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:17.695+0000 7fef7d400700 1 --2- 172.21.15.117:0/2726300727 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fef640a2c20 0x7fef640a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:19:17.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:17.695+0000 7fef7d400700 1 --2- 172.21.15.117:0/2726300727 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fef640a2c20 0x7fef640a50e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fef801109a0 tx=0x7fef6c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:19:17.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:17.695+0000 7fef76a00700 1 -- 172.21.15.117:0/2726300727 <== mon.2 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fef70031020 con 0x7fef800746c0 2024-06-03T03:19:17.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:17 smithi117 ceph-mon[31649]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:19:17.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:17 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:17.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:17 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:17.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:17 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:17.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:17 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T03:19:17.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:17 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T03:19:17.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:17 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:19:17.977 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:17.977+0000 7fef85200700 1 -- 172.21.15.117:0/2726300727 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fef8019c180 con 0x7fef800746c0 2024-06-03T03:19:17.978 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:17.978+0000 7fef76a00700 1 -- 172.21.15.117:0/2726300727 <== mon.2 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8842 (secure 0 0 0) 0x7fef70013070 con 0x7fef800746c0 2024-06-03T03:19:17.979 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T03:19:17.979 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3353001119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3353001119},{"type":"v1","addr":"172.21.15.117:6837","nonce":3353001119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14388,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6839/405782304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":405782304},{"type":"v1","addr":"172.21.15.117:6839","nonce":405782304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3355662416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3355662416},{"type":"v1","addr":"172.21.15.151:6833","nonce":3355662416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/2429725085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2429725085},{"type":"v1","addr":"172.21.15.183:6833","nonce":2429725085}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/31153579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":31153579},{"type":"v1","addr":"172.21.15.151:6835","nonce":31153579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24466,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/2884460088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2884460088},{"type":"v1","addr":"172.21.15.117:6835","nonce":2884460088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24479,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/2392407848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":2392407848},{"type":"v1","addr":"172.21.15.151:6837","nonce":2392407848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24487,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/1009004320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1009004320},{"type":"v1","addr":"172.21.15.183:6835","nonce":1009004320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24499,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/2697518526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":2697518526},{"type":"v1","addr":"172.21.15.183:6837","nonce":2697518526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24517,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/1439615083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1439615083},{"type":"v1","addr":"172.21.15.183:6839","nonce":1439615083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-06-03T03:19:17.982 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:17.980+0000 7fef85200700 1 -- 172.21.15.117:0/2726300727 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fef640a2c20 msgr2=0x7fef640a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:19:17.982 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:17.980+0000 7fef85200700 1 --2- 172.21.15.117:0/2726300727 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fef640a2c20 0x7fef640a50e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fef801109a0 tx=0x7fef6c009260 comp rx=0 tx=0).stop 2024-06-03T03:19:17.982 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:17.980+0000 7fef85200700 1 -- 172.21.15.117:0/2726300727 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fef800746c0 msgr2=0x7fef8019ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:19:17.982 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:17.980+0000 7fef85200700 1 --2- 172.21.15.117:0/2726300727 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fef800746c0 0x7fef8019ae70 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fef70006010 tx=0x7fef7000bab0 comp rx=0 tx=0).stop 2024-06-03T03:19:17.982 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:17.981+0000 7fef85200700 1 -- 172.21.15.117:0/2726300727 shutdown_connections 2024-06-03T03:19:17.982 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:17.981+0000 7fef85200700 1 --2- 172.21.15.117:0/2726300727 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fef640a2c20 0x7fef640a50e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:17.982 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:17.981+0000 7fef85200700 1 --2- 172.21.15.117:0/2726300727 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fef8010eef0 0x7fef8019f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:17.982 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:17.981+0000 7fef85200700 1 --2- 172.21.15.117:0/2726300727 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fef800746c0 0x7fef8019ae70 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:17.982 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:17.981+0000 7fef85200700 1 --2- 172.21.15.117:0/2726300727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef80074fe0 0x7fef8019b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:17.983 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:17.981+0000 7fef85200700 1 -- 172.21.15.117:0/2726300727 >> 172.21.15.117:0/2726300727 conn(0x7fef800795f0 msgr2=0x7fef800ff660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:19:17.983 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:17.981+0000 7fef85200700 1 -- 172.21.15.117:0/2726300727 shutdown_connections 2024-06-03T03:19:17.983 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:17.981+0000 7fef85200700 1 -- 172.21.15.117:0/2726300727 wait complete. 2024-06-03T03:19:17.983 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 11 2024-06-03T03:19:18.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:17 smithi151 ceph-mon[37043]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:19:18.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:17 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:18.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:17 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:18.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:17 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:18.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:17 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T03:19:18.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:17 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T03:19:18.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:17 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:19:18.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:17 smithi183 ceph-mon[37326]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:19:18.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:17 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:18.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:17 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:18.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:17 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:18.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:17 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T03:19:18.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:17 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T03:19:18.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:17 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:19:19.087 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:18 smithi117 ceph-mon[31649]: Deploying daemon mds.a on smithi117 2024-06-03T03:19:19.087 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:18 smithi117 ceph-mon[31649]: mds.? [v2:172.21.15.117:6838/405782304,v1:172.21.15.117:6839/405782304] up:boot 2024-06-03T03:19:19.087 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:18 smithi117 ceph-mon[31649]: fsmap 10 up:standby 2024-06-03T03:19:19.087 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:18 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-03T03:19:19.087 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:18 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/2726300727' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T03:19:19.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:18 smithi151 ceph-mon[37043]: Deploying daemon mds.a on smithi117 2024-06-03T03:19:19.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:18 smithi151 ceph-mon[37043]: mds.? [v2:172.21.15.117:6838/405782304,v1:172.21.15.117:6839/405782304] up:boot 2024-06-03T03:19:19.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:18 smithi151 ceph-mon[37043]: fsmap 10 up:standby 2024-06-03T03:19:19.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:18 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-03T03:19:19.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:18 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/2726300727' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T03:19:19.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:18 smithi183 ceph-mon[37326]: Deploying daemon mds.a on smithi117 2024-06-03T03:19:19.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:18 smithi183 ceph-mon[37326]: mds.? [v2:172.21.15.117:6838/405782304,v1:172.21.15.117:6839/405782304] up:boot 2024-06-03T03:19:19.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:18 smithi183 ceph-mon[37326]: fsmap 10 up:standby 2024-06-03T03:19:19.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:18 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-03T03:19:19.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:18 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/2726300727' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T03:19:20.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:19 smithi151 ceph-mon[37043]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:19:20.177 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:19 smithi117 ceph-mon[31649]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:19:20.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:19 smithi183 ceph-mon[37326]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:19:20.892 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-03T03:19:21.447 INFO:journalctl@ceph.mds.a.smithi117.stdout:Jun 03 03:19:21 smithi117 systemd[1]: Starting Ceph mds.a for a0e8473a-2156-11ef-bc9b-c7b262605968... 2024-06-03T03:19:21.879 INFO:journalctl@ceph.mds.a.smithi117.stdout:Jun 03 03:19:21 smithi117 podman[71164]: 2024-06-03 03:19:21.49631771 +0000 UTC m=+0.034141651 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T03:19:21.879 INFO:journalctl@ceph.mds.a.smithi117.stdout:Jun 03 03:19:21 smithi117 podman[71164]: 2024-06-03 03:19:21.643033576 +0000 UTC m=+0.180857554 container create 85a7fbdf4fdfcb6ce84706c3fa6c2af895304ca52171ea0574876d69742df3d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-a, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-03T03:19:22.100 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:19:22.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:21 smithi151 ceph-mon[37043]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:19:22.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:21 smithi117 ceph-mon[31649]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:19:22.197 INFO:journalctl@ceph.mds.a.smithi117.stdout:Jun 03 03:19:21 smithi117 podman[71164]: 2024-06-03 03:19:21.955888929 +0000 UTC m=+0.493712933 container init 85a7fbdf4fdfcb6ce84706c3fa6c2af895304ca52171ea0574876d69742df3d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-a, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-03T03:19:22.197 INFO:journalctl@ceph.mds.a.smithi117.stdout:Jun 03 03:19:21 smithi117 podman[71164]: 2024-06-03 03:19:21.966870434 +0000 UTC m=+0.504694442 container start 85a7fbdf4fdfcb6ce84706c3fa6c2af895304ca52171ea0574876d69742df3d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-03T03:19:22.197 INFO:journalctl@ceph.mds.a.smithi117.stdout:Jun 03 03:19:22 smithi117 ceph-mds[71202]: -- 172.21.15.117:0/1163043908 <== mon.0 v2:172.21.15.117:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a1edd8ba40 con 0x55a1edd5f000 2024-06-03T03:19:22.197 INFO:journalctl@ceph.mds.a.smithi117.stdout:Jun 03 03:19:22 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-a[71198]: starting mds.a at 2024-06-03T03:19:22.197 INFO:journalctl@ceph.mds.a.smithi117.stdout:Jun 03 03:19:22 smithi117 bash[71164]: 85a7fbdf4fdfcb6ce84706c3fa6c2af895304ca52171ea0574876d69742df3d2 2024-06-03T03:19:22.197 INFO:journalctl@ceph.mds.a.smithi117.stdout:Jun 03 03:19:22 smithi117 systemd[1]: Started Ceph mds.a for a0e8473a-2156-11ef-bc9b-c7b262605968. 2024-06-03T03:19:22.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:21 smithi183 ceph-mon[37326]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:19:23.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:23 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:23.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:23 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:23.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:23 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:23.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:23 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T03:19:23.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:23 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T03:19:23.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:23 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:19:23.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:23 smithi117 ceph-mon[31649]: Deploying daemon mds.b on smithi151 2024-06-03T03:19:23.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:23 smithi117 ceph-mon[31649]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:19:23.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:23 smithi117 ceph-mon[31649]: mds.? [v2:172.21.15.117:6840/1212242967,v1:172.21.15.117:6841/1212242967] up:boot 2024-06-03T03:19:23.448 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:23 smithi117 ceph-mon[31649]: fsmap 11 up:standby 2024-06-03T03:19:23.448 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:23 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-03T03:19:23.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:23 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:23.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:23 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:23.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:23 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:23.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:23 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T03:19:23.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:23 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T03:19:23.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:23 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:19:23.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:23 smithi183 ceph-mon[37326]: Deploying daemon mds.b on smithi151 2024-06-03T03:19:23.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:23 smithi183 ceph-mon[37326]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:19:23.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:23 smithi183 ceph-mon[37326]: mds.? [v2:172.21.15.117:6840/1212242967,v1:172.21.15.117:6841/1212242967] up:boot 2024-06-03T03:19:23.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:23 smithi183 ceph-mon[37326]: fsmap 11 up:standby 2024-06-03T03:19:23.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:23 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-03T03:19:23.656 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:23 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:23.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:23 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:23.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:23 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:23.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:23 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T03:19:23.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:23 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T03:19:23.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:23 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:19:23.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:23 smithi151 ceph-mon[37043]: Deploying daemon mds.b on smithi151 2024-06-03T03:19:23.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:23 smithi151 ceph-mon[37043]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:19:23.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:23 smithi151 ceph-mon[37043]: mds.? [v2:172.21.15.117:6840/1212242967,v1:172.21.15.117:6841/1212242967] up:boot 2024-06-03T03:19:23.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:23 smithi151 ceph-mon[37043]: fsmap 11 up:standby 2024-06-03T03:19:23.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:23 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-03T03:19:24.407 INFO:journalctl@ceph.mds.b.smithi151.stdout:Jun 03 03:19:24 smithi151 systemd[1]: Starting Ceph mds.b for a0e8473a-2156-11ef-bc9b-c7b262605968... 2024-06-03T03:19:24.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:24.694+0000 7fa0e5000700 1 -- 172.21.15.117:0/3385010534 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0e0074fe0 msgr2=0x7fa0e010e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:19:24.697 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:24.694+0000 7fa0e5000700 1 --2- 172.21.15.117:0/3385010534 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0e0074fe0 0x7fa0e010e950 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa0d0009990 tx=0x7fa0d0009ca0 comp rx=0 tx=0).stop 2024-06-03T03:19:24.697 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:24.695+0000 7fa0e5000700 1 -- 172.21.15.117:0/3385010534 shutdown_connections 2024-06-03T03:19:24.697 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:24.695+0000 7fa0e5000700 1 --2- 172.21.15.117:0/3385010534 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa0e010ee90 0x7fa0e0111280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:24.697 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:24.695+0000 7fa0e5000700 1 --2- 172.21.15.117:0/3385010534 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa0e00746c0 0x7fa0e0074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:24.697 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:24.695+0000 7fa0e5000700 1 --2- 172.21.15.117:0/3385010534 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0e0074fe0 0x7fa0e010e950 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:24.697 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:24.695+0000 7fa0e5000700 1 -- 172.21.15.117:0/3385010534 >> 172.21.15.117:0/3385010534 conn(0x7fa0e00795f0 msgr2=0x7fa0e0079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:19:24.697 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:24.695+0000 7fa0e5000700 1 -- 172.21.15.117:0/3385010534 shutdown_connections 2024-06-03T03:19:24.697 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:24.695+0000 7fa0e5000700 1 -- 172.21.15.117:0/3385010534 wait complete. 2024-06-03T03:19:24.697 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:24.696+0000 7fa0e5000700 1 Processor -- start 2024-06-03T03:19:24.698 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:24.696+0000 7fa0e5000700 1 -- start start 2024-06-03T03:19:24.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:24.696+0000 7fa0e5000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0e00746c0 0x7fa0e019af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:19:24.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:24.696+0000 7fa0e5000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa0e0074fe0 0x7fa0e019b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:19:24.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:24.696+0000 7fa0e5000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa0e010ee90 0x7fa0e019f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:19:24.700 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:24.696+0000 7fa0e5000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e0114aa0 con 0x7fa0e00746c0 2024-06-03T03:19:24.700 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:24.696+0000 7fa0e5000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e0114920 con 0x7fa0e0074fe0 2024-06-03T03:19:24.700 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:24.696+0000 7fa0e5000700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e0114c20 con 0x7fa0e010ee90 2024-06-03T03:19:24.700 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:24.697+0000 7fa0dde00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0e00746c0 0x7fa0e019af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:19:24.700 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:24.697+0000 7fa0dde00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0e00746c0 0x7fa0e019af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:60736/0 (socket says 172.21.15.117:60736) 2024-06-03T03:19:24.700 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:24.697+0000 7fa0d5400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa0e0074fe0 0x7fa0e019b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:19:24.700 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:24.697+0000 7fa0dde00700 1 -- 172.21.15.117:0/3576506967 learned_addr learned my addr 172.21.15.117:0/3576506967 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:19:24.700 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:24.697+0000 7fa0de800700 1 --2- 172.21.15.117:0/3576506967 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa0e010ee90 0x7fa0e019f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:19:24.700 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:24.697+0000 7fa0dde00700 1 -- 172.21.15.117:0/3576506967 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa0e010ee90 msgr2=0x7fa0e019f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:19:24.700 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:24.697+0000 7fa0dde00700 1 --2- 172.21.15.117:0/3576506967 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa0e010ee90 0x7fa0e019f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:24.700 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:24.697+0000 7fa0dde00700 1 -- 172.21.15.117:0/3576506967 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa0e0074fe0 msgr2=0x7fa0e019b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:19:24.700 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:24.697+0000 7fa0dde00700 1 --2- 172.21.15.117:0/3576506967 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa0e0074fe0 0x7fa0e019b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:24.700 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:24.697+0000 7fa0dde00700 1 -- 172.21.15.117:0/3576506967 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0e019ff20 con 0x7fa0e00746c0 2024-06-03T03:19:24.701 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:24.698+0000 7fa0dde00700 1 --2- 172.21.15.117:0/3576506967 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0e00746c0 0x7fa0e019af10 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa0c800b550 tx=0x7fa0c800b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:19:24.701 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:24.698+0000 7fa0d7400700 1 -- 172.21.15.117:0/3576506967 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c8015020 con 0x7fa0e00746c0 2024-06-03T03:19:24.701 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:24.698+0000 7fa0d7400700 1 -- 172.21.15.117:0/3576506967 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0c8011d10 con 0x7fa0e00746c0 2024-06-03T03:19:24.701 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:24.698+0000 7fa0e5000700 1 -- 172.21.15.117:0/3576506967 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0e01a0210 con 0x7fa0e00746c0 2024-06-03T03:19:24.701 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:24.698+0000 7fa0d7400700 1 -- 172.21.15.117:0/3576506967 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c8010db0 con 0x7fa0e00746c0 2024-06-03T03:19:24.701 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:24.698+0000 7fa0e5000700 1 -- 172.21.15.117:0/3576506967 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0e01a7a90 con 0x7fa0e00746c0 2024-06-03T03:19:24.701 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:24.699+0000 7fa0e5000700 1 -- 172.21.15.117:0/3576506967 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0e0069180 con 0x7fa0e00746c0 2024-06-03T03:19:24.702 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:24.701+0000 7fa0d7400700 1 -- 172.21.15.117:0/3576506967 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fa0c8018020 con 0x7fa0e00746c0 2024-06-03T03:19:24.703 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:24.702+0000 7fa0d7400700 1 --2- 172.21.15.117:0/3576506967 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fa0c40a2ca0 0x7fa0c40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:19:24.703 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:24.703+0000 7fa0d7400700 1 -- 172.21.15.117:0/3576506967 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fa0c80ce2a0 con 0x7fa0e00746c0 2024-06-03T03:19:24.703 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:24.703+0000 7fa0d5400700 1 --2- 172.21.15.117:0/3576506967 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fa0c40a2ca0 0x7fa0c40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:19:24.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:24.703+0000 7fa0d5400700 1 --2- 172.21.15.117:0/3576506967 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fa0c40a2ca0 0x7fa0c40a5160 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa0d0006010 tx=0x7fa0d0005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:19:24.708 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:24.707+0000 7fa0d7400700 1 -- 172.21.15.117:0/3576506967 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa0c801f020 con 0x7fa0e00746c0 2024-06-03T03:19:24.800 INFO:journalctl@ceph.mds.b.smithi151.stdout:Jun 03 03:19:24 smithi151 podman[67227]: 2024-06-03 03:19:24.451690916 +0000 UTC m=+0.017416756 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T03:19:24.800 INFO:journalctl@ceph.mds.b.smithi151.stdout:Jun 03 03:19:24 smithi151 podman[67227]: 2024-06-03 03:19:24.560875388 +0000 UTC m=+0.126601208 container create d52b90d82890f0419d36ec5576119bdbc9cae6385ec09eb82bd963456ecf7b8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-b, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, io.buildah.version=1.33.5) 2024-06-03T03:19:24.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:24.992+0000 7fa0e5000700 1 -- 172.21.15.117:0/3576506967 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa0e019c270 con 0x7fa0e00746c0 2024-06-03T03:19:24.994 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:24.993+0000 7fa0d7400700 1 -- 172.21.15.117:0/3576506967 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10500 (secure 0 0 0) 0x7fa0c809b600 con 0x7fa0e00746c0 2024-06-03T03:19:24.994 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T03:19:24.994 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3353001119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3353001119},{"type":"v1","addr":"172.21.15.117:6837","nonce":3353001119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14388,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6839/405782304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":405782304},{"type":"v1","addr":"172.21.15.117:6839","nonce":405782304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3355662416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3355662416},{"type":"v1","addr":"172.21.15.151:6833","nonce":3355662416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/2429725085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2429725085},{"type":"v1","addr":"172.21.15.183:6833","nonce":2429725085}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/31153579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":31153579},{"type":"v1","addr":"172.21.15.151:6835","nonce":31153579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24466,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/2884460088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2884460088},{"type":"v1","addr":"172.21.15.117:6835","nonce":2884460088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24479,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/2392407848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":2392407848},{"type":"v1","addr":"172.21.15.151:6837","nonce":2392407848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24487,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/1009004320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1009004320},{"type":"v1","addr":"172.21.15.183:6835","nonce":1009004320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24499,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/2697518526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":2697518526},{"type":"v1","addr":"172.21.15.183:6837","nonce":2697518526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24503,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6841/1212242967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6840","nonce":1212242967},{"type":"v1","addr":"172.21.15.117:6841","nonce":1212242967}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24512,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/3623444616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":3623444616},{"type":"v1","addr":"172.21.15.151:6839","nonce":3623444616}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24517,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/1439615083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1439615083},{"type":"v1","addr":"172.21.15.183:6839","nonce":1439615083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-06-03T03:19:24.997 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:24.996+0000 7fa0e5000700 1 -- 172.21.15.117:0/3576506967 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fa0c40a2ca0 msgr2=0x7fa0c40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:19:24.997 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:24.996+0000 7fa0e5000700 1 --2- 172.21.15.117:0/3576506967 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fa0c40a2ca0 0x7fa0c40a5160 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa0d0006010 tx=0x7fa0d0005a90 comp rx=0 tx=0).stop 2024-06-03T03:19:24.997 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:24.996+0000 7fa0e5000700 1 -- 172.21.15.117:0/3576506967 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0e00746c0 msgr2=0x7fa0e019af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:19:24.997 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:24.996+0000 7fa0e5000700 1 --2- 172.21.15.117:0/3576506967 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0e00746c0 0x7fa0e019af10 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa0c800b550 tx=0x7fa0c800b860 comp rx=0 tx=0).stop 2024-06-03T03:19:24.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:24.997+0000 7fa0e5000700 1 -- 172.21.15.117:0/3576506967 shutdown_connections 2024-06-03T03:19:24.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:24.997+0000 7fa0e5000700 1 --2- 172.21.15.117:0/3576506967 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fa0c40a2ca0 0x7fa0c40a5160 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:24.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:24.997+0000 7fa0e5000700 1 --2- 172.21.15.117:0/3576506967 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa0e010ee90 0x7fa0e019f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:24.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:24.997+0000 7fa0e5000700 1 --2- 172.21.15.117:0/3576506967 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0e00746c0 0x7fa0e019af10 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:24.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:24.997+0000 7fa0e5000700 1 --2- 172.21.15.117:0/3576506967 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa0e0074fe0 0x7fa0e019b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:24.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:24.997+0000 7fa0e5000700 1 -- 172.21.15.117:0/3576506967 >> 172.21.15.117:0/3576506967 conn(0x7fa0e00795f0 msgr2=0x7fa0e00ff600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:19:24.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:24.997+0000 7fa0e5000700 1 -- 172.21.15.117:0/3576506967 shutdown_connections 2024-06-03T03:19:24.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:24.997+0000 7fa0e5000700 1 -- 172.21.15.117:0/3576506967 wait complete. 2024-06-03T03:19:24.999 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 13 2024-06-03T03:19:25.157 INFO:journalctl@ceph.mds.b.smithi151.stdout:Jun 03 03:19:24 smithi151 podman[67227]: 2024-06-03 03:19:24.799215981 +0000 UTC m=+0.364941809 container init d52b90d82890f0419d36ec5576119bdbc9cae6385ec09eb82bd963456ecf7b8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-b, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-03T03:19:25.157 INFO:journalctl@ceph.mds.b.smithi151.stdout:Jun 03 03:19:24 smithi151 podman[67227]: 2024-06-03 03:19:24.803123804 +0000 UTC m=+0.368849639 container start d52b90d82890f0419d36ec5576119bdbc9cae6385ec09eb82bd963456ecf7b8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-b, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-03T03:19:25.157 INFO:journalctl@ceph.mds.b.smithi151.stdout:Jun 03 03:19:24 smithi151 ceph-mds[67246]: -- 172.21.15.151:0/1615696775 <== mon.2 v2:172.21.15.151:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563398d6ba40 con 0x563398d3e800 2024-06-03T03:19:25.157 INFO:journalctl@ceph.mds.b.smithi151.stdout:Jun 03 03:19:24 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-b[67242]: starting mds.b at 2024-06-03T03:19:25.157 INFO:journalctl@ceph.mds.b.smithi151.stdout:Jun 03 03:19:24 smithi151 bash[67227]: d52b90d82890f0419d36ec5576119bdbc9cae6385ec09eb82bd963456ecf7b8d 2024-06-03T03:19:25.157 INFO:journalctl@ceph.mds.b.smithi151.stdout:Jun 03 03:19:24 smithi151 systemd[1]: Started Ceph mds.b for a0e8473a-2156-11ef-bc9b-c7b262605968. 2024-06-03T03:19:26.008 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-06-03T03:19:26.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:25 smithi151 ceph-mon[37043]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:19:26.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:25 smithi151 ceph-mon[37043]: mds.? [v2:172.21.15.151:6838/3623444616,v1:172.21.15.151:6839/3623444616] up:boot 2024-06-03T03:19:26.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:25 smithi151 ceph-mon[37043]: fsmap 12 up:standby 2024-06-03T03:19:26.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:25 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-03T03:19:26.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:25 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:26.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:25 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:26.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:25 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/3576506967' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T03:19:26.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:25 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:26.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:25 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:26.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:25 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:26.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:25 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:19:26.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:25 smithi117 ceph-mon[31649]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:19:26.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:25 smithi117 ceph-mon[31649]: mds.? [v2:172.21.15.151:6838/3623444616,v1:172.21.15.151:6839/3623444616] up:boot 2024-06-03T03:19:26.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:25 smithi117 ceph-mon[31649]: fsmap 12 up:standby 2024-06-03T03:19:26.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:25 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-03T03:19:26.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:25 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:26.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:25 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:26.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:25 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/3576506967' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T03:19:26.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:25 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:26.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:25 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:26.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:25 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:26.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:25 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:19:26.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:25 smithi183 ceph-mon[37326]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:19:26.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:25 smithi183 ceph-mon[37326]: mds.? [v2:172.21.15.151:6838/3623444616,v1:172.21.15.151:6839/3623444616] up:boot 2024-06-03T03:19:26.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:25 smithi183 ceph-mon[37326]: fsmap 12 up:standby 2024-06-03T03:19:26.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:25 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-03T03:19:26.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:25 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:26.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:25 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:26.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:25 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/3576506967' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T03:19:26.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:25 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:26.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:25 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:26.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:25 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:26.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:25 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:19:27.183 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:19:28.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:27 smithi151 ceph-mon[37043]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:19:28.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:27 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:28.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:27 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:28.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:27 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:28.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:27 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:28.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:27 smithi117 ceph-mon[31649]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:19:28.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:27 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:28.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:27 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:28.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:27 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:28.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:27 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:28.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:27 smithi183 ceph-mon[37326]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:19:28.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:27 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:28.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:27 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:28.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:27 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:28.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:27 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:29.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:29.646+0000 7f3f45c00700 1 -- 172.21.15.117:0/2881107844 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f4006ab30 msgr2=0x7f3f4010f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:19:29.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:29.646+0000 7f3f45c00700 1 --2- 172.21.15.117:0/2881107844 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f4006ab30 0x7f3f4010f320 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3f3800b0a0 tx=0x7f3f3800b3b0 comp rx=0 tx=0).stop 2024-06-03T03:19:29.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:29.646+0000 7f3f45c00700 1 -- 172.21.15.117:0/2881107844 shutdown_connections 2024-06-03T03:19:29.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:29.646+0000 7f3f45c00700 1 --2- 172.21.15.117:0/2881107844 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f4006ab30 0x7f3f4010f320 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:29.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:29.646+0000 7f3f45c00700 1 --2- 172.21.15.117:0/2881107844 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3f40108500 0x7f3f401088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:29.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:29.646+0000 7f3f45c00700 1 --2- 172.21.15.117:0/2881107844 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f4006a170 0x7f3f4006a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:29.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:29.646+0000 7f3f45c00700 1 -- 172.21.15.117:0/2881107844 >> 172.21.15.117:0/2881107844 conn(0x7f3f40078340 msgr2=0x7f3f40078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:19:29.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:29.647+0000 7f3f45c00700 1 -- 172.21.15.117:0/2881107844 shutdown_connections 2024-06-03T03:19:29.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:29.647+0000 7f3f45c00700 1 -- 172.21.15.117:0/2881107844 wait complete. 2024-06-03T03:19:29.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:29.648+0000 7f3f45c00700 1 Processor -- start 2024-06-03T03:19:29.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:29.648+0000 7f3f45c00700 1 -- start start 2024-06-03T03:19:29.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:29.648+0000 7f3f45c00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f4006a170 0x7f3f40101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:19:29.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:29.648+0000 7f3f45c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3f4006ab30 0x7f3f401019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:19:29.650 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:29.648+0000 7f3f45c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f40108500 0x7f3f40101ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:19:29.650 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:29.648+0000 7f3f45c00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3f401127d0 con 0x7f3f40108500 2024-06-03T03:19:29.650 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:29.648+0000 7f3f45c00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3f40112650 con 0x7f3f4006ab30 2024-06-03T03:19:29.650 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:29.648+0000 7f3f45c00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3f40112950 con 0x7f3f4006a170 2024-06-03T03:19:29.650 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:29.649+0000 7f3f3ea00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f4006a170 0x7f3f40101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:19:29.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:29.649+0000 7f3f3ea00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f4006a170 0x7f3f40101460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:56970/0 (socket says 172.21.15.117:56970) 2024-06-03T03:19:29.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:29.649+0000 7f3f3ea00700 1 -- 172.21.15.117:0/500464384 learned_addr learned my addr 172.21.15.117:0/500464384 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:19:29.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:29.649+0000 7f3f3e000700 1 --2- 172.21.15.117:0/500464384 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3f4006ab30 0x7f3f401019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:19:29.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:29.649+0000 7f3f3f400700 1 --2- 172.21.15.117:0/500464384 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f40108500 0x7f3f40101ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:19:29.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:29.649+0000 7f3f3ea00700 1 -- 172.21.15.117:0/500464384 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3f4006ab30 msgr2=0x7f3f401019a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:19:29.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:29.649+0000 7f3f3ea00700 1 --2- 172.21.15.117:0/500464384 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3f4006ab30 0x7f3f401019a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:29.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:29.649+0000 7f3f3ea00700 1 -- 172.21.15.117:0/500464384 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f40108500 msgr2=0x7f3f40101ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:19:29.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:29.649+0000 7f3f3ea00700 1 --2- 172.21.15.117:0/500464384 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f40108500 0x7f3f40101ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:29.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:29.649+0000 7f3f3ea00700 1 -- 172.21.15.117:0/500464384 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f401a71b0 con 0x7f3f4006a170 2024-06-03T03:19:29.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:29.649+0000 7f3f3e000700 1 --2- 172.21.15.117:0/500464384 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3f4006ab30 0x7f3f401019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:19:29.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:29.650+0000 7f3f3ea00700 1 --2- 172.21.15.117:0/500464384 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f4006a170 0x7f3f40101460 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3f3000e9c0 tx=0x7f3f3000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:19:29.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:29.650+0000 7f3f37400700 1 -- 172.21.15.117:0/500464384 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f3000cbd0 con 0x7f3f4006a170 2024-06-03T03:19:29.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:29.650+0000 7f3f37400700 1 -- 172.21.15.117:0/500464384 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f3000ee60 con 0x7f3f4006a170 2024-06-03T03:19:29.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:29.650+0000 7f3f37400700 1 -- 172.21.15.117:0/500464384 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f30005630 con 0x7f3f4006a170 2024-06-03T03:19:29.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:29.650+0000 7f3f45c00700 1 -- 172.21.15.117:0/500464384 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f401a74a0 con 0x7f3f4006a170 2024-06-03T03:19:29.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:29.650+0000 7f3f45c00700 1 -- 172.21.15.117:0/500464384 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f401a79e0 con 0x7f3f4006a170 2024-06-03T03:19:29.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:29.651+0000 7f3f45c00700 1 -- 172.21.15.117:0/500464384 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f4010a9e0 con 0x7f3f4006a170 2024-06-03T03:19:29.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:29.653+0000 7f3f37400700 1 -- 172.21.15.117:0/500464384 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f3f30004750 con 0x7f3f4006a170 2024-06-03T03:19:29.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:29.655+0000 7f3f37400700 1 --2- 172.21.15.117:0/500464384 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f3f240a2ca0 0x7f3f240a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:19:29.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:29.655+0000 7f3f37400700 1 -- 172.21.15.117:0/500464384 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f3f300cedb0 con 0x7f3f4006a170 2024-06-03T03:19:29.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:29.655+0000 7f3f3e000700 1 --2- 172.21.15.117:0/500464384 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f3f240a2ca0 0x7f3f240a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:19:29.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:29.656+0000 7f3f3e000700 1 --2- 172.21.15.117:0/500464384 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f3f240a2ca0 0x7f3f240a5160 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3f40078980 tx=0x7f3f2c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:19:29.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:29.659+0000 7f3f37400700 1 -- 172.21.15.117:0/500464384 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3f300d3070 con 0x7f3f4006a170 2024-06-03T03:19:29.897 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:29.896+0000 7f3f45c00700 1 -- 172.21.15.117:0/500464384 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f3f40102990 con 0x7f3f4006a170 2024-06-03T03:19:29.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:29.903+0000 7f3f37400700 1 -- 172.21.15.117:0/500464384 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v81) ==== 84+0+65 (secure 0 0 0) 0x7f3f3009c110 con 0x7f3f4006a170 2024-06-03T03:19:29.905 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T03:19:29.905 INFO:teuthology.orchestra.run.smithi117.stdout:[ 2024-06-03T03:19:29.905 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:19:29.905 INFO:teuthology.orchestra.run.smithi117.stdout: "poolnum": 1, 2024-06-03T03:19:29.905 INFO:teuthology.orchestra.run.smithi117.stdout: "poolname": ".mgr" 2024-06-03T03:19:29.905 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:19:29.906 INFO:teuthology.orchestra.run.smithi117.stdout:] 2024-06-03T03:19:29.907 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:29.907+0000 7f3f45c00700 1 -- 172.21.15.117:0/500464384 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f3f240a2ca0 msgr2=0x7f3f240a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:19:29.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:29.907+0000 7f3f45c00700 1 --2- 172.21.15.117:0/500464384 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f3f240a2ca0 0x7f3f240a5160 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3f40078980 tx=0x7f3f2c009260 comp rx=0 tx=0).stop 2024-06-03T03:19:29.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:29.907+0000 7f3f45c00700 1 -- 172.21.15.117:0/500464384 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f4006a170 msgr2=0x7f3f40101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:19:29.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:29.907+0000 7f3f45c00700 1 --2- 172.21.15.117:0/500464384 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f4006a170 0x7f3f40101460 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3f3000e9c0 tx=0x7f3f3000ecd0 comp rx=0 tx=0).stop 2024-06-03T03:19:29.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:29.907+0000 7f3f45c00700 1 -- 172.21.15.117:0/500464384 shutdown_connections 2024-06-03T03:19:29.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:29.907+0000 7f3f45c00700 1 --2- 172.21.15.117:0/500464384 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f3f240a2ca0 0x7f3f240a5160 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:29.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:29.907+0000 7f3f45c00700 1 --2- 172.21.15.117:0/500464384 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f40108500 0x7f3f40101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:29.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:29.907+0000 7f3f45c00700 1 --2- 172.21.15.117:0/500464384 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f4006a170 0x7f3f40101460 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:29.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:29.907+0000 7f3f45c00700 1 --2- 172.21.15.117:0/500464384 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3f4006ab30 0x7f3f401019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:29.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:29.907+0000 7f3f45c00700 1 -- 172.21.15.117:0/500464384 >> 172.21.15.117:0/500464384 conn(0x7f3f40078340 msgr2=0x7f3f400fff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:19:29.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:29.908+0000 7f3f45c00700 1 -- 172.21.15.117:0/500464384 shutdown_connections 2024-06-03T03:19:29.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:29.908+0000 7f3f45c00700 1 -- 172.21.15.117:0/500464384 wait complete. 2024-06-03T03:19:30.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:29 smithi151 ceph-mon[37043]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:19:30.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:29 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:30.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:29 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:30.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:29 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:19:30.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:29 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:19:30.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:29 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:30.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:29 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:30.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:29 smithi117 ceph-mon[31649]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:19:30.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:29 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:30.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:29 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:30.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:29 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:19:30.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:29 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:19:30.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:29 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:30.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:29 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:30.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:29 smithi183 ceph-mon[37326]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:19:30.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:29 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:30.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:29 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:30.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:29 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:19:30.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:29 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:19:30.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:29 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:30.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:29 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:30.889 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-06-03T03:19:30.889 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-06-03T03:19:30.949 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:30 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/500464384' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-06-03T03:19:30.950 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:30 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:19:31.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:30 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/500464384' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-06-03T03:19:31.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:30 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:19:31.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:30 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/500464384' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-06-03T03:19:31.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:30 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:19:31.632 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:19:32.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:31 smithi117 ceph-mon[31649]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:19:32.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:31 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:32.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:31 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:32.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:31 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:32.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:31 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:32.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:31 smithi183 ceph-mon[37326]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:19:32.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:31 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:32.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:31 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:32.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:31 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:32.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:31 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:32.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:31 smithi151 ceph-mon[37043]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:19:32.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:31 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:32.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:31 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:32.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:31 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:32.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:31 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:34.077 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:33 smithi117 ceph-mon[31649]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:19:34.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:33 smithi183 ceph-mon[37326]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:19:34.238 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:34.236+0000 7f3770a00700 1 -- 172.21.15.117:0/3423847668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f376c106600 msgr2=0x7f376c10e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:19:34.238 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:34.236+0000 7f3770a00700 1 --2- 172.21.15.117:0/3423847668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f376c106600 0x7f376c10e1d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f376400b0d0 tx=0x7f376400b3e0 comp rx=0 tx=0).stop 2024-06-03T03:19:34.239 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:34.236+0000 7f3770a00700 1 -- 172.21.15.117:0/3423847668 shutdown_connections 2024-06-03T03:19:34.239 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:34.236+0000 7f3770a00700 1 --2- 172.21.15.117:0/3423847668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f376c106600 0x7f376c10e1d0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:34.239 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:34.236+0000 7f3770a00700 1 --2- 172.21.15.117:0/3423847668 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f376c101520 0x7f376c101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:34.239 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:34.236+0000 7f3770a00700 1 --2- 172.21.15.117:0/3423847668 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f376c101ed0 0x7f376c105f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:34.239 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:34.236+0000 7f3770a00700 1 -- 172.21.15.117:0/3423847668 >> 172.21.15.117:0/3423847668 conn(0x7f376c0fcdb0 msgr2=0x7f376c0ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:19:34.240 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:34.237+0000 7f3770a00700 1 -- 172.21.15.117:0/3423847668 shutdown_connections 2024-06-03T03:19:34.240 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:34.237+0000 7f3770a00700 1 -- 172.21.15.117:0/3423847668 wait complete. 2024-06-03T03:19:34.240 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:34.238+0000 7f3770a00700 1 Processor -- start 2024-06-03T03:19:34.240 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:34.238+0000 7f3770a00700 1 -- start start 2024-06-03T03:19:34.241 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:34.239+0000 7f3770a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f376c101520 0x7f376c1a0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:19:34.241 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:34.239+0000 7f3770a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f376c101ed0 0x7f376c1a0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:19:34.241 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:34.239+0000 7f3770a00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f376c106600 0x7f376c19ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:19:34.241 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:34.239+0000 7f3770a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f376c111640 con 0x7f376c101520 2024-06-03T03:19:34.241 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:34.239+0000 7f3770a00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f376c1114c0 con 0x7f376c101ed0 2024-06-03T03:19:34.241 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:34.239+0000 7f3770a00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f376c1117c0 con 0x7f376c106600 2024-06-03T03:19:34.242 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:34.239+0000 7f376a800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f376c101520 0x7f376c1a0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:19:34.242 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:34.239+0000 7f3769e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f376c101ed0 0x7f376c1a0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:19:34.242 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:34.239+0000 7f3769e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f376c101ed0 0x7f376c1a0f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:41514/0 (socket says 172.21.15.117:41514) 2024-06-03T03:19:34.242 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:34.239+0000 7f3769e00700 1 -- 172.21.15.117:0/2226115705 learned_addr learned my addr 172.21.15.117:0/2226115705 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:19:34.242 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:34.239+0000 7f376a800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f376c101520 0x7f376c1a0a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37178/0 (socket says 172.21.15.117:37178) 2024-06-03T03:19:34.242 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:34.239+0000 7f376b200700 1 --2- 172.21.15.117:0/2226115705 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f376c106600 0x7f376c19ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:19:34.244 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:34.240+0000 7f3769e00700 1 -- 172.21.15.117:0/2226115705 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f376c106600 msgr2=0x7f376c19ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:19:34.244 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:34.240+0000 7f3769e00700 1 --2- 172.21.15.117:0/2226115705 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f376c106600 0x7f376c19ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:34.244 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:34.240+0000 7f3769e00700 1 -- 172.21.15.117:0/2226115705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f376c101520 msgr2=0x7f376c1a0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:19:34.244 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:34.240+0000 7f3769e00700 1 --2- 172.21.15.117:0/2226115705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f376c101520 0x7f376c1a0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:34.244 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:34.240+0000 7f3769e00700 1 -- 172.21.15.117:0/2226115705 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f376c19b3f0 con 0x7f376c101ed0 2024-06-03T03:19:34.244 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:34.240+0000 7f376a800700 1 --2- 172.21.15.117:0/2226115705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f376c101520 0x7f376c1a0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:19:34.244 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:34.240+0000 7f376b200700 1 --2- 172.21.15.117:0/2226115705 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f376c106600 0x7f376c19ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T03:19:34.245 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:34.240+0000 7f3769e00700 1 --2- 172.21.15.117:0/2226115705 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f376c101ed0 0x7f376c1a0f80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f375800e970 tx=0x7f375800ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:19:34.245 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:34.240+0000 7f3763400700 1 -- 172.21.15.117:0/2226115705 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375800cbd0 con 0x7f376c101ed0 2024-06-03T03:19:34.245 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:34.240+0000 7f3763400700 1 -- 172.21.15.117:0/2226115705 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f375800cd70 con 0x7f376c101ed0 2024-06-03T03:19:34.245 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:34.240+0000 7f3770a00700 1 -- 172.21.15.117:0/2226115705 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f376c19b6e0 con 0x7f376c101ed0 2024-06-03T03:19:34.245 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:34.240+0000 7f3763400700 1 -- 172.21.15.117:0/2226115705 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37580056b0 con 0x7f376c101ed0 2024-06-03T03:19:34.245 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:34.240+0000 7f3770a00700 1 -- 172.21.15.117:0/2226115705 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f376c1a79c0 con 0x7f376c101ed0 2024-06-03T03:19:34.245 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:34.242+0000 7f3770a00700 1 -- 172.21.15.117:0/2226115705 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f376c102a00 con 0x7f376c101ed0 2024-06-03T03:19:34.246 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:34.244+0000 7f3763400700 1 -- 172.21.15.117:0/2226115705 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f3758004750 con 0x7f376c101ed0 2024-06-03T03:19:34.247 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:34.245+0000 7f3763400700 1 --2- 172.21.15.117:0/2226115705 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f37500a2c20 0x7f37500a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:19:34.247 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:34.246+0000 7f3763400700 1 -- 172.21.15.117:0/2226115705 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f37580cefe0 con 0x7f376c101ed0 2024-06-03T03:19:34.247 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:34.246+0000 7f376a800700 1 --2- 172.21.15.117:0/2226115705 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f37500a2c20 0x7f37500a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:19:34.248 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:34.246+0000 7f376a800700 1 --2- 172.21.15.117:0/2226115705 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f37500a2c20 0x7f37500a50e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f375c011490 tx=0x7f375c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:19:34.251 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:34.250+0000 7f3763400700 1 -- 172.21.15.117:0/2226115705 <== mon.2 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f375809c3a0 con 0x7f376c101ed0 2024-06-03T03:19:34.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:33 smithi151 ceph-mon[37043]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:19:34.503 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:34.501+0000 7f3770a00700 1 -- 172.21.15.117:0/2226115705 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f376c19bfd0 con 0x7f376c101ed0 2024-06-03T03:19:35.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:35.521+0000 7f3763400700 1 -- 172.21.15.117:0/2226115705 <== mon.2 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v82) ==== 138+0+0 (secure 0 0 0) 0x7f375809bca0 con 0x7f376c101ed0 2024-06-03T03:19:35.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:35.529+0000 7f3770a00700 1 -- 172.21.15.117:0/2226115705 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f37500a2c20 msgr2=0x7f37500a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:19:35.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:35.529+0000 7f3770a00700 1 --2- 172.21.15.117:0/2226115705 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f37500a2c20 0x7f37500a50e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f375c011490 tx=0x7f375c009260 comp rx=0 tx=0).stop 2024-06-03T03:19:35.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:35.529+0000 7f3770a00700 1 -- 172.21.15.117:0/2226115705 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f376c101ed0 msgr2=0x7f376c1a0f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:19:35.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:35.529+0000 7f3770a00700 1 --2- 172.21.15.117:0/2226115705 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f376c101ed0 0x7f376c1a0f80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f375800e970 tx=0x7f375800ec80 comp rx=0 tx=0).stop 2024-06-03T03:19:35.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:35.529+0000 7f3770a00700 1 -- 172.21.15.117:0/2226115705 shutdown_connections 2024-06-03T03:19:35.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:35.529+0000 7f3770a00700 1 --2- 172.21.15.117:0/2226115705 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f37500a2c20 0x7f37500a50e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:35.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:35.529+0000 7f3770a00700 1 --2- 172.21.15.117:0/2226115705 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f376c106600 0x7f376c19ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:35.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:35.529+0000 7f3770a00700 1 --2- 172.21.15.117:0/2226115705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f376c101520 0x7f376c1a0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:35.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:35.529+0000 7f3770a00700 1 --2- 172.21.15.117:0/2226115705 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f376c101ed0 0x7f376c1a0f80 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:35.533 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:35.529+0000 7f3770a00700 1 -- 172.21.15.117:0/2226115705 >> 172.21.15.117:0/2226115705 conn(0x7f376c0fcdb0 msgr2=0x7f376c10cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:19:35.533 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:35.530+0000 7f3770a00700 1 -- 172.21.15.117:0/2226115705 shutdown_connections 2024-06-03T03:19:35.533 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:35.530+0000 7f3770a00700 1 -- 172.21.15.117:0/2226115705 wait complete. 2024-06-03T03:19:35.535 INFO:teuthology.orchestra.run.smithi117.stderr:pool 'cephfs_metadata' created 2024-06-03T03:19:35.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:35 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/2226115705' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-06-03T03:19:35.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:35 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:35.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:35 smithi151 ceph-mon[37043]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-06-03T03:19:35.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:35 smithi151 ceph-mon[37043]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:19:35.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:35 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:35.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:35 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:19:35.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:35 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:19:35.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:35 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:35.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:35 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/2226115705' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-06-03T03:19:35.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:35 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:35.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:35 smithi117 ceph-mon[31649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-06-03T03:19:35.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:35 smithi117 ceph-mon[31649]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:19:35.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:35 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:35.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:35 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:19:35.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:35 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:19:35.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:35 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:35.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:35 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/2226115705' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-06-03T03:19:35.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:35 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:35.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:35 smithi183 ceph-mon[37326]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-06-03T03:19:35.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:35 smithi183 ceph-mon[37326]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:19:35.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:35 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:35.983 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:35 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:19:35.983 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:35 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:19:35.983 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:35 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:19:36.562 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-06-03T03:19:36.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:36 smithi151 ceph-mon[37043]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-06-03T03:19:36.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:36 smithi151 ceph-mon[37043]: osdmap e82: 12 total, 12 up, 12 in 2024-06-03T03:19:36.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:36 smithi117 ceph-mon[31649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-06-03T03:19:36.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:36 smithi117 ceph-mon[31649]: osdmap e82: 12 total, 12 up, 12 in 2024-06-03T03:19:36.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:36 smithi183 ceph-mon[37326]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-06-03T03:19:36.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:36 smithi183 ceph-mon[37326]: osdmap e82: 12 total, 12 up, 12 in 2024-06-03T03:19:36.985 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:19:37.858 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:37 smithi151 ceph-mon[37043]: osdmap e83: 12 total, 12 up, 12 in 2024-06-03T03:19:37.858 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:37 smithi151 ceph-mon[37043]: pgmap v319: 33 pgs: 1 creating+activating, 3 creating+peering, 2 active+clean, 27 unknown; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:19:37.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:37 smithi117 ceph-mon[31649]: osdmap e83: 12 total, 12 up, 12 in 2024-06-03T03:19:37.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:37 smithi117 ceph-mon[31649]: pgmap v319: 33 pgs: 1 creating+activating, 3 creating+peering, 2 active+clean, 27 unknown; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:19:37.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:37 smithi183 ceph-mon[37326]: osdmap e83: 12 total, 12 up, 12 in 2024-06-03T03:19:37.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:37 smithi183 ceph-mon[37326]: pgmap v319: 33 pgs: 1 creating+activating, 3 creating+peering, 2 active+clean, 27 unknown; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:19:38.696 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:38 smithi117 ceph-mon[31649]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-03T03:19:38.697 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:38 smithi117 ceph-mon[31649]: osdmap e84: 12 total, 12 up, 12 in 2024-06-03T03:19:38.906 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:38 smithi151 ceph-mon[37043]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-03T03:19:38.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:38 smithi151 ceph-mon[37043]: osdmap e84: 12 total, 12 up, 12 in 2024-06-03T03:19:38.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:38 smithi183 ceph-mon[37326]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-03T03:19:38.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:38 smithi183 ceph-mon[37326]: osdmap e84: 12 total, 12 up, 12 in 2024-06-03T03:19:39.331 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:39.328+0000 7fa27d000700 1 -- 172.21.15.117:0/554394686 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa27810eac0 msgr2=0x7fa27810eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:19:39.331 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:39.328+0000 7fa27d000700 1 --2- 172.21.15.117:0/554394686 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa27810eac0 0x7fa27810eea0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa268009990 tx=0x7fa268009ca0 comp rx=0 tx=0).stop 2024-06-03T03:19:39.331 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:39.329+0000 7fa27d000700 1 -- 172.21.15.117:0/554394686 shutdown_connections 2024-06-03T03:19:39.331 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:39.329+0000 7fa27d000700 1 --2- 172.21.15.117:0/554394686 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa278109480 0x7fa278115990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:39.331 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:39.329+0000 7fa27d000700 1 --2- 172.21.15.117:0/554394686 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa27810eac0 0x7fa27810eea0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:39.331 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:39.329+0000 7fa27d000700 1 --2- 172.21.15.117:0/554394686 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa278108ac0 0x7fa278108f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:39.332 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:39.329+0000 7fa27d000700 1 -- 172.21.15.117:0/554394686 >> 172.21.15.117:0/554394686 conn(0x7fa278075c60 msgr2=0x7fa2781072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:19:39.332 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:39.329+0000 7fa27d000700 1 -- 172.21.15.117:0/554394686 shutdown_connections 2024-06-03T03:19:39.332 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:39.329+0000 7fa27d000700 1 -- 172.21.15.117:0/554394686 wait complete. 2024-06-03T03:19:39.333 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:39.330+0000 7fa27d000700 1 Processor -- start 2024-06-03T03:19:39.333 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:39.330+0000 7fa27d000700 1 -- start start 2024-06-03T03:19:39.333 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:39.330+0000 7fa27d000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa278108ac0 0x7fa27819f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:19:39.333 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:39.330+0000 7fa27d000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa278109480 0x7fa27819f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:19:39.333 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:39.330+0000 7fa27d000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa27810eac0 0x7fa2781a3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:19:39.333 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:39.331+0000 7fa27d000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa278118e40 con 0x7fa27810eac0 2024-06-03T03:19:39.333 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:39.331+0000 7fa27d000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa278118cc0 con 0x7fa278108ac0 2024-06-03T03:19:39.333 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:39.331+0000 7fa27d000700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa278118fc0 con 0x7fa278109480 2024-06-03T03:19:39.333 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:39.331+0000 7fa276800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa27810eac0 0x7fa2781a3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:19:39.333 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:39.331+0000 7fa275400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa278109480 0x7fa27819f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:19:39.333 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:39.331+0000 7fa275e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa278108ac0 0x7fa27819f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:19:39.333 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:39.331+0000 7fa275e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa278108ac0 0x7fa27819f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:60684/0 (socket says 172.21.15.117:60684) 2024-06-03T03:19:39.333 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:39.331+0000 7fa275400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa278109480 0x7fa27819f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:37126/0 (socket says 172.21.15.117:37126) 2024-06-03T03:19:39.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:39.331+0000 7fa275e00700 1 -- 172.21.15.117:0/3350541926 learned_addr learned my addr 172.21.15.117:0/3350541926 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:19:39.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:39.331+0000 7fa275e00700 1 -- 172.21.15.117:0/3350541926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa278109480 msgr2=0x7fa27819f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:19:39.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:39.331+0000 7fa275e00700 1 --2- 172.21.15.117:0/3350541926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa278109480 0x7fa27819f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:39.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:39.331+0000 7fa275e00700 1 -- 172.21.15.117:0/3350541926 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa27810eac0 msgr2=0x7fa2781a3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:19:39.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:39.331+0000 7fa275e00700 1 --2- 172.21.15.117:0/3350541926 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa27810eac0 0x7fa2781a3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:39.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:39.331+0000 7fa275e00700 1 -- 172.21.15.117:0/3350541926 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2781a4260 con 0x7fa278108ac0 2024-06-03T03:19:39.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:39.332+0000 7fa275e00700 1 --2- 172.21.15.117:0/3350541926 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa278108ac0 0x7fa27819f250 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa268005910 tx=0x7fa26800bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:19:39.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:39.332+0000 7fa26ea00700 1 -- 172.21.15.117:0/3350541926 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa268027070 con 0x7fa278108ac0 2024-06-03T03:19:39.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:39.332+0000 7fa26ea00700 1 -- 172.21.15.117:0/3350541926 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa268004650 con 0x7fa278108ac0 2024-06-03T03:19:39.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:39.332+0000 7fa26ea00700 1 -- 172.21.15.117:0/3350541926 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa26800f460 con 0x7fa278108ac0 2024-06-03T03:19:39.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:39.332+0000 7fa27d000700 1 -- 172.21.15.117:0/3350541926 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2781a44f0 con 0x7fa278108ac0 2024-06-03T03:19:39.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:39.332+0000 7fa27d000700 1 -- 172.21.15.117:0/3350541926 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2781abdd0 con 0x7fa278108ac0 2024-06-03T03:19:39.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:39.333+0000 7fa27d000700 1 -- 172.21.15.117:0/3350541926 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa278111050 con 0x7fa278108ac0 2024-06-03T03:19:39.337 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:39.335+0000 7fa26ea00700 1 -- 172.21.15.117:0/3350541926 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fa26802a070 con 0x7fa278108ac0 2024-06-03T03:19:39.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:39.337+0000 7fa26ea00700 1 --2- 172.21.15.117:0/3350541926 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fa25c0a2ca0 0x7fa25c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:19:39.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:39.337+0000 7fa26ea00700 1 -- 172.21.15.117:0/3350541926 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7497+0+0 (secure 0 0 0) 0x7fa2680cffe0 con 0x7fa278108ac0 2024-06-03T03:19:39.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:39.337+0000 7fa275400700 1 --2- 172.21.15.117:0/3350541926 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fa25c0a2ca0 0x7fa25c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:19:39.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:39.338+0000 7fa275400700 1 --2- 172.21.15.117:0/3350541926 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fa25c0a2ca0 0x7fa25c0a5160 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa278077ba0 tx=0x7fa264009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:19:39.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:39.339+0000 7fa26ea00700 1 -- 172.21.15.117:0/3350541926 <== mon.2 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa268031070 con 0x7fa278108ac0 2024-06-03T03:19:39.564 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:39.562+0000 7fa27d000700 1 -- 172.21.15.117:0/3350541926 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7fa278069180 con 0x7fa278108ac0 2024-06-03T03:19:39.696 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:39 smithi117 ceph-mon[31649]: pgmap v321: 33 pgs: 1 creating+activating, 3 creating+peering, 16 active+clean, 13 unknown; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:19:39.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:39 smithi151 ceph-mon[37043]: pgmap v321: 33 pgs: 1 creating+activating, 3 creating+peering, 16 active+clean, 13 unknown; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:19:39.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:39 smithi183 ceph-mon[37326]: pgmap v321: 33 pgs: 1 creating+activating, 3 creating+peering, 16 active+clean, 13 unknown; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:19:40.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:40.571+0000 7fa26ea00700 1 -- 172.21.15.117:0/3350541926 <== mon.2 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v85) ==== 171+0+0 (secure 0 0 0) 0x7fa268013070 con 0x7fa278108ac0 2024-06-03T03:19:40.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:40.579+0000 7fa27d000700 1 -- 172.21.15.117:0/3350541926 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fa25c0a2ca0 msgr2=0x7fa25c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:19:40.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:40.579+0000 7fa27d000700 1 --2- 172.21.15.117:0/3350541926 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fa25c0a2ca0 0x7fa25c0a5160 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa278077ba0 tx=0x7fa264009260 comp rx=0 tx=0).stop 2024-06-03T03:19:40.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:40.579+0000 7fa27d000700 1 -- 172.21.15.117:0/3350541926 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa278108ac0 msgr2=0x7fa27819f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:19:40.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:40.579+0000 7fa27d000700 1 --2- 172.21.15.117:0/3350541926 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa278108ac0 0x7fa27819f250 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa268005910 tx=0x7fa26800bab0 comp rx=0 tx=0).stop 2024-06-03T03:19:40.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:40.580+0000 7fa27d000700 1 -- 172.21.15.117:0/3350541926 shutdown_connections 2024-06-03T03:19:40.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:40.580+0000 7fa27d000700 1 --2- 172.21.15.117:0/3350541926 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fa25c0a2ca0 0x7fa25c0a5160 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:40.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:40.580+0000 7fa27d000700 1 --2- 172.21.15.117:0/3350541926 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa27810eac0 0x7fa2781a3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:40.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:40.580+0000 7fa27d000700 1 --2- 172.21.15.117:0/3350541926 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa278108ac0 0x7fa27819f250 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:40.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:40.580+0000 7fa27d000700 1 --2- 172.21.15.117:0/3350541926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa278109480 0x7fa27819f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:40.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:40.580+0000 7fa27d000700 1 -- 172.21.15.117:0/3350541926 >> 172.21.15.117:0/3350541926 conn(0x7fa278075c60 msgr2=0x7fa278076280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:19:40.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:40.580+0000 7fa27d000700 1 -- 172.21.15.117:0/3350541926 shutdown_connections 2024-06-03T03:19:40.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:40.580+0000 7fa27d000700 1 -- 172.21.15.117:0/3350541926 wait complete. 2024-06-03T03:19:40.586 INFO:teuthology.orchestra.run.smithi117.stderr:pool 'cephfs_data' created 2024-06-03T03:19:40.883 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:40 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/3350541926' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-06-03T03:19:40.883 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:40 smithi183 ceph-mon[37326]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-06-03T03:19:40.906 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:40 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/3350541926' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-06-03T03:19:40.906 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:40 smithi151 ceph-mon[37043]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-06-03T03:19:40.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:40 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/3350541926' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-06-03T03:19:40.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:40 smithi117 ceph-mon[31649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-06-03T03:19:41.596 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-06-03T03:19:41.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:41 smithi151 ceph-mon[37043]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-06-03T03:19:41.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:41 smithi151 ceph-mon[37043]: osdmap e85: 12 total, 12 up, 12 in 2024-06-03T03:19:41.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:41 smithi151 ceph-mon[37043]: pgmap v323: 97 pgs: 64 unknown, 1 creating+activating, 3 creating+peering, 29 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:19:41.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:41 smithi117 ceph-mon[31649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-06-03T03:19:41.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:41 smithi117 ceph-mon[31649]: osdmap e85: 12 total, 12 up, 12 in 2024-06-03T03:19:41.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:41 smithi117 ceph-mon[31649]: pgmap v323: 97 pgs: 64 unknown, 1 creating+activating, 3 creating+peering, 29 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:19:41.983 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:41 smithi183 ceph-mon[37326]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-06-03T03:19:41.983 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:41 smithi183 ceph-mon[37326]: osdmap e85: 12 total, 12 up, 12 in 2024-06-03T03:19:41.983 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:41 smithi183 ceph-mon[37326]: pgmap v323: 97 pgs: 64 unknown, 1 creating+activating, 3 creating+peering, 29 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:19:42.050 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:19:42.906 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:42 smithi151 ceph-mon[37043]: osdmap e86: 12 total, 12 up, 12 in 2024-06-03T03:19:42.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:42 smithi117 ceph-mon[31649]: osdmap e86: 12 total, 12 up, 12 in 2024-06-03T03:19:42.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:42 smithi183 ceph-mon[37326]: osdmap e86: 12 total, 12 up, 12 in 2024-06-03T03:19:43.824 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:43 smithi117 ceph-mon[31649]: osdmap e87: 12 total, 12 up, 12 in 2024-06-03T03:19:43.824 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:43 smithi117 ceph-mon[31649]: pgmap v326: 97 pgs: 43 unknown, 2 creating+peering, 52 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:19:43.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:43 smithi151 ceph-mon[37043]: osdmap e87: 12 total, 12 up, 12 in 2024-06-03T03:19:43.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:43 smithi151 ceph-mon[37043]: pgmap v326: 97 pgs: 43 unknown, 2 creating+peering, 52 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:19:43.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:43 smithi183 ceph-mon[37326]: osdmap e87: 12 total, 12 up, 12 in 2024-06-03T03:19:43.983 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:43 smithi183 ceph-mon[37326]: pgmap v326: 97 pgs: 43 unknown, 2 creating+peering, 52 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:19:44.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:44.344+0000 7fd717e00700 1 -- 172.21.15.117:0/2029398504 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd718104f80 msgr2=0x7fd718111490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:19:44.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:44.344+0000 7fd717e00700 1 --2- 172.21.15.117:0/2029398504 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd718104f80 0x7fd718111490 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd71000b0a0 tx=0x7fd71000b3b0 comp rx=0 tx=0).stop 2024-06-03T03:19:44.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:44.344+0000 7fd717e00700 1 -- 172.21.15.117:0/2029398504 shutdown_connections 2024-06-03T03:19:44.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:44.344+0000 7fd717e00700 1 --2- 172.21.15.117:0/2029398504 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd718104f80 0x7fd718111490 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:44.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:44.344+0000 7fd717e00700 1 --2- 172.21.15.117:0/2029398504 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd71810a650 0x7fd71810aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:44.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:44.344+0000 7fd717e00700 1 --2- 172.21.15.117:0/2029398504 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd7181045c0 0x7fd718104a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:44.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:44.344+0000 7fd717e00700 1 -- 172.21.15.117:0/2029398504 >> 172.21.15.117:0/2029398504 conn(0x7fd7181000b0 msgr2=0x7fd7181024d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:19:44.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:44.345+0000 7fd717e00700 1 -- 172.21.15.117:0/2029398504 shutdown_connections 2024-06-03T03:19:44.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:44.345+0000 7fd717e00700 1 -- 172.21.15.117:0/2029398504 wait complete. 2024-06-03T03:19:44.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:44.346+0000 7fd717e00700 1 Processor -- start 2024-06-03T03:19:44.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:44.346+0000 7fd717e00700 1 -- start start 2024-06-03T03:19:44.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:44.347+0000 7fd717e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd7181045c0 0x7fd7181a0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:19:44.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:44.347+0000 7fd717e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd718104f80 0x7fd7181a0e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:19:44.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:44.347+0000 7fd717e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd71810a650 0x7fd71819ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:19:44.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:44.347+0000 7fd717e00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd718114960 con 0x7fd718104f80 2024-06-03T03:19:44.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:44.347+0000 7fd717e00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd7181147e0 con 0x7fd71810a650 2024-06-03T03:19:44.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:44.347+0000 7fd717e00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd718114ae0 con 0x7fd7181045c0 2024-06-03T03:19:44.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:44.347+0000 7fd716a00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd7181045c0 0x7fd7181a0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:19:44.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:44.347+0000 7fd716000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd718104f80 0x7fd7181a0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:19:44.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:44.347+0000 7fd716000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd718104f80 0x7fd7181a0e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40360/0 (socket says 172.21.15.117:40360) 2024-06-03T03:19:44.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:44.347+0000 7fd716a00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd7181045c0 0x7fd7181a0940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:37138/0 (socket says 172.21.15.117:37138) 2024-06-03T03:19:44.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:44.347+0000 7fd716000700 1 -- 172.21.15.117:0/1977800675 learned_addr learned my addr 172.21.15.117:0/1977800675 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:19:44.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:44.347+0000 7fd717400700 1 --2- 172.21.15.117:0/1977800675 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd71810a650 0x7fd71819ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:19:44.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:44.348+0000 7fd716000700 1 -- 172.21.15.117:0/1977800675 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd7181045c0 msgr2=0x7fd7181a0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:19:44.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:44.348+0000 7fd716000700 1 --2- 172.21.15.117:0/1977800675 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd7181045c0 0x7fd7181a0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:44.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:44.348+0000 7fd716000700 1 -- 172.21.15.117:0/1977800675 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd71810a650 msgr2=0x7fd71819ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:19:44.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:44.348+0000 7fd716000700 1 --2- 172.21.15.117:0/1977800675 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd71810a650 0x7fd71819ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:44.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:44.348+0000 7fd716000700 1 -- 172.21.15.117:0/1977800675 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd71819b260 con 0x7fd718104f80 2024-06-03T03:19:44.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:44.348+0000 7fd716000700 1 --2- 172.21.15.117:0/1977800675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd718104f80 0x7fd7181a0e80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd70400eb30 tx=0x7fd70400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:19:44.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:44.348+0000 7fd70f400700 1 -- 172.21.15.117:0/1977800675 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd70400cbd0 con 0x7fd718104f80 2024-06-03T03:19:44.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:44.348+0000 7fd70f400700 1 -- 172.21.15.117:0/1977800675 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd704004500 con 0x7fd718104f80 2024-06-03T03:19:44.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:44.348+0000 7fd717e00700 1 -- 172.21.15.117:0/1977800675 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd71819b550 con 0x7fd718104f80 2024-06-03T03:19:44.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:44.349+0000 7fd70f400700 1 -- 172.21.15.117:0/1977800675 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd704010430 con 0x7fd718104f80 2024-06-03T03:19:44.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:44.349+0000 7fd717e00700 1 -- 172.21.15.117:0/1977800675 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7181a7810 con 0x7fd718104f80 2024-06-03T03:19:44.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:44.350+0000 7fd717e00700 1 -- 172.21.15.117:0/1977800675 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd718069180 con 0x7fd718104f80 2024-06-03T03:19:44.354 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:44.353+0000 7fd70f400700 1 -- 172.21.15.117:0/1977800675 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fd704003f00 con 0x7fd718104f80 2024-06-03T03:19:44.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:44.354+0000 7fd70f400700 1 --2- 172.21.15.117:0/1977800675 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fd6fc0a2c20 0x7fd6fc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:19:44.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:44.354+0000 7fd70f400700 1 -- 172.21.15.117:0/1977800675 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7879+0+0 (secure 0 0 0) 0x7fd7040cf350 con 0x7fd718104f80 2024-06-03T03:19:44.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:44.354+0000 7fd716a00700 1 --2- 172.21.15.117:0/1977800675 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fd6fc0a2c20 0x7fd6fc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:19:44.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:44.355+0000 7fd716a00700 1 --2- 172.21.15.117:0/1977800675 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fd6fc0a2c20 0x7fd6fc0a50e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd718105700 tx=0x7fd708009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:19:44.358 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:44.357+0000 7fd70f400700 1 -- 172.21.15.117:0/1977800675 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd70409c450 con 0x7fd718104f80 2024-06-03T03:19:44.629 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:44.627+0000 7fd717e00700 1 -- 172.21.15.117:0/1977800675 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fd71819bff0 con 0x7fd718104f80 2024-06-03T03:19:44.906 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:44 smithi151 ceph-mon[37043]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-03T03:19:44.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:44 smithi117 ceph-mon[31649]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-03T03:19:44.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:44 smithi183 ceph-mon[37326]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-03T03:19:45.624 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:45.623+0000 7fd70f400700 1 -- 172.21.15.117:0/1977800675 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fd70409bd50 con 0x7fd718104f80 2024-06-03T03:19:45.627 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:45.627+0000 7fd717e00700 1 -- 172.21.15.117:0/1977800675 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fd6fc0a2c20 msgr2=0x7fd6fc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:19:45.628 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:45.627+0000 7fd717e00700 1 --2- 172.21.15.117:0/1977800675 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fd6fc0a2c20 0x7fd6fc0a50e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd718105700 tx=0x7fd708009260 comp rx=0 tx=0).stop 2024-06-03T03:19:45.628 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:45.627+0000 7fd717e00700 1 -- 172.21.15.117:0/1977800675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd718104f80 msgr2=0x7fd7181a0e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:19:45.628 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:45.627+0000 7fd717e00700 1 --2- 172.21.15.117:0/1977800675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd718104f80 0x7fd7181a0e80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd70400eb30 tx=0x7fd70400ee40 comp rx=0 tx=0).stop 2024-06-03T03:19:45.628 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:45.627+0000 7fd717e00700 1 -- 172.21.15.117:0/1977800675 shutdown_connections 2024-06-03T03:19:45.628 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:45.627+0000 7fd717e00700 1 --2- 172.21.15.117:0/1977800675 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fd6fc0a2c20 0x7fd6fc0a50e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:45.628 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:45.627+0000 7fd717e00700 1 --2- 172.21.15.117:0/1977800675 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd71810a650 0x7fd71819ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:45.628 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:45.627+0000 7fd717e00700 1 --2- 172.21.15.117:0/1977800675 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd7181045c0 0x7fd7181a0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:45.629 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:45.627+0000 7fd717e00700 1 --2- 172.21.15.117:0/1977800675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd718104f80 0x7fd7181a0e80 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:45.629 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:45.627+0000 7fd717e00700 1 -- 172.21.15.117:0/1977800675 >> 172.21.15.117:0/1977800675 conn(0x7fd7181000b0 msgr2=0x7fd718101650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:19:45.629 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:45.628+0000 7fd717e00700 1 -- 172.21.15.117:0/1977800675 shutdown_connections 2024-06-03T03:19:45.629 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:45.628+0000 7fd717e00700 1 -- 172.21.15.117:0/1977800675 wait complete. 2024-06-03T03:19:45.632 INFO:teuthology.orchestra.run.smithi117.stderr:new fs with metadata pool 2 and data pool 3 2024-06-03T03:19:45.906 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:45 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/1977800675' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-06-03T03:19:45.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:45 smithi151 ceph-mon[37043]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-06-03T03:19:45.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:45 smithi151 ceph-mon[37043]: pgmap v327: 97 pgs: 26 unknown, 2 creating+peering, 69 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:19:45.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:45 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/1977800675' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-06-03T03:19:45.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:45 smithi117 ceph-mon[31649]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-06-03T03:19:45.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:45 smithi117 ceph-mon[31649]: pgmap v327: 97 pgs: 26 unknown, 2 creating+peering, 69 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:19:45.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:45 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/1977800675' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-06-03T03:19:45.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:45 smithi183 ceph-mon[37326]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-06-03T03:19:45.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:45 smithi183 ceph-mon[37326]: pgmap v327: 97 pgs: 26 unknown, 2 creating+peering, 69 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:19:46.674 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-06-03T03:19:46.675 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-06-03T03:19:46.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:46 smithi151 ceph-mon[37043]: osdmap e88: 12 total, 12 up, 12 in 2024-06-03T03:19:46.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:46 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/1977800675' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-06-03T03:19:46.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:46 smithi151 ceph-mon[37043]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-06-03T03:19:46.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:46 smithi151 ceph-mon[37043]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-06-03T03:19:46.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:46 smithi117 ceph-mon[31649]: osdmap e88: 12 total, 12 up, 12 in 2024-06-03T03:19:46.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:46 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/1977800675' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-06-03T03:19:46.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:46 smithi117 ceph-mon[31649]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-06-03T03:19:46.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:46 smithi117 ceph-mon[31649]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-06-03T03:19:46.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:46 smithi183 ceph-mon[37326]: osdmap e88: 12 total, 12 up, 12 in 2024-06-03T03:19:46.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:46 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/1977800675' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-06-03T03:19:46.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:46 smithi183 ceph-mon[37326]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-06-03T03:19:46.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:46 smithi183 ceph-mon[37326]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-06-03T03:19:47.113 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:19:47.890 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:47 smithi183 ceph-mon[37326]: mds.? [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] up:active 2024-06-03T03:19:47.891 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:47 smithi183 ceph-mon[37326]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-06-03T03:19:47.891 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:47 smithi183 ceph-mon[37326]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-06-03T03:19:47.891 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:47 smithi183 ceph-mon[37326]: pgmap v329: 97 pgs: 97 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-06-03T03:19:47.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:47 smithi151 ceph-mon[37043]: mds.? [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] up:active 2024-06-03T03:19:47.908 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:47 smithi151 ceph-mon[37043]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-06-03T03:19:47.908 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:47 smithi151 ceph-mon[37043]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-06-03T03:19:47.908 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:47 smithi151 ceph-mon[37043]: pgmap v329: 97 pgs: 97 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-06-03T03:19:47.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:47 smithi117 ceph-mon[31649]: mds.? [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] up:active 2024-06-03T03:19:47.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:47 smithi117 ceph-mon[31649]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-06-03T03:19:47.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:47 smithi117 ceph-mon[31649]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-06-03T03:19:47.948 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:47 smithi117 ceph-mon[31649]: pgmap v329: 97 pgs: 97 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-06-03T03:19:48.232 INFO:journalctl@ceph.mds.i.smithi183.stdout:Jun 03 03:19:47 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-i[67909]: 2024-06-03T03:19:47.888+0000 7f655277c700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-06-03T03:19:48.868 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:48 smithi117 ceph-mon[31649]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-06-03T03:19:48.868 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:48 smithi117 ceph-mon[31649]: Cluster is now healthy 2024-06-03T03:19:48.906 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:48 smithi151 ceph-mon[37043]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-06-03T03:19:48.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:48 smithi151 ceph-mon[37043]: Cluster is now healthy 2024-06-03T03:19:48.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:48 smithi183 ceph-mon[37326]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-06-03T03:19:48.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:48 smithi183 ceph-mon[37326]: Cluster is now healthy 2024-06-03T03:19:49.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:49.407+0000 7fe31b800700 1 -- 172.21.15.117:0/138065683 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe31406ab10 msgr2=0x7fe31410f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:19:49.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:49.407+0000 7fe31b800700 1 --2- 172.21.15.117:0/138065683 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe31406ab10 0x7fe31410f320 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe30c00b0d0 tx=0x7fe30c00b3e0 comp rx=0 tx=0).stop 2024-06-03T03:19:49.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:49.408+0000 7fe31b800700 1 -- 172.21.15.117:0/138065683 shutdown_connections 2024-06-03T03:19:49.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:49.408+0000 7fe31b800700 1 --2- 172.21.15.117:0/138065683 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe31406ab10 0x7fe31410f320 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:49.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:49.408+0000 7fe31b800700 1 --2- 172.21.15.117:0/138065683 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe3141084e0 0x7fe3141088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:49.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:49.408+0000 7fe31b800700 1 --2- 172.21.15.117:0/138065683 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe31406a170 0x7fe31406a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:49.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:49.408+0000 7fe31b800700 1 -- 172.21.15.117:0/138065683 >> 172.21.15.117:0/138065683 conn(0x7fe3140782c0 msgr2=0x7fe3140786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:19:49.410 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:49.408+0000 7fe31b800700 1 -- 172.21.15.117:0/138065683 shutdown_connections 2024-06-03T03:19:49.410 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:49.408+0000 7fe31b800700 1 -- 172.21.15.117:0/138065683 wait complete. 2024-06-03T03:19:49.410 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:49.409+0000 7fe31b800700 1 Processor -- start 2024-06-03T03:19:49.410 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:49.409+0000 7fe31b800700 1 -- start start 2024-06-03T03:19:49.411 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:49.410+0000 7fe31b800700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe31406a170 0x7fe31419ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:19:49.411 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:49.410+0000 7fe31b800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe31406ab10 0x7fe31419b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:19:49.411 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:49.410+0000 7fe31b800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3141084e0 0x7fe31419f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:19:49.411 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:49.410+0000 7fe31b800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe314112780 con 0x7fe3141084e0 2024-06-03T03:19:49.411 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:49.410+0000 7fe31b800700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe314112600 con 0x7fe31406ab10 2024-06-03T03:19:49.411 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:49.410+0000 7fe31b800700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe314112900 con 0x7fe31406a170 2024-06-03T03:19:49.412 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:49.410+0000 7fe318e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe31406a170 0x7fe31419ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:19:49.412 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:49.410+0000 7fe319800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3141084e0 0x7fe31419f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:19:49.412 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:49.410+0000 7fe318e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe31406a170 0x7fe31419ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:60546/0 (socket says 172.21.15.117:60546) 2024-06-03T03:19:49.412 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:49.410+0000 7fe318e00700 1 -- 172.21.15.117:0/2194902849 learned_addr learned my addr 172.21.15.117:0/2194902849 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:19:49.413 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:49.410+0000 7fe313e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe31406ab10 0x7fe31419b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:19:49.413 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:49.410+0000 7fe318e00700 1 -- 172.21.15.117:0/2194902849 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe31406ab10 msgr2=0x7fe31419b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:19:49.413 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:49.410+0000 7fe318e00700 1 --2- 172.21.15.117:0/2194902849 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe31406ab10 0x7fe31419b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:49.413 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:49.410+0000 7fe318e00700 1 -- 172.21.15.117:0/2194902849 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3141084e0 msgr2=0x7fe31419f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:19:49.413 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:49.410+0000 7fe318e00700 1 --2- 172.21.15.117:0/2194902849 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3141084e0 0x7fe31419f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:49.413 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:49.411+0000 7fe318e00700 1 -- 172.21.15.117:0/2194902849 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe31419fdb0 con 0x7fe31406a170 2024-06-03T03:19:49.413 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:49.411+0000 7fe318e00700 1 --2- 172.21.15.117:0/2194902849 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe31406a170 0x7fe31419ada0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe30800e9a0 tx=0x7fe30800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:19:49.413 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:49.411+0000 7fe311600700 1 -- 172.21.15.117:0/2194902849 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe30800ca10 con 0x7fe31406a170 2024-06-03T03:19:49.413 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:49.411+0000 7fe31b800700 1 -- 172.21.15.117:0/2194902849 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3141a00a0 con 0x7fe31406a170 2024-06-03T03:19:49.413 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:49.411+0000 7fe311600700 1 -- 172.21.15.117:0/2194902849 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe30800cbb0 con 0x7fe31406a170 2024-06-03T03:19:49.413 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:49.411+0000 7fe311600700 1 -- 172.21.15.117:0/2194902849 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe308018680 con 0x7fe31406a170 2024-06-03T03:19:49.413 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:49.411+0000 7fe31b800700 1 -- 172.21.15.117:0/2194902849 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3141a7920 con 0x7fe31406a170 2024-06-03T03:19:49.414 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:49.412+0000 7fe31b800700 1 -- 172.21.15.117:0/2194902849 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe314069180 con 0x7fe31406a170 2024-06-03T03:19:49.415 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:49.415+0000 7fe311600700 1 -- 172.21.15.117:0/2194902849 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fe308010430 con 0x7fe31406a170 2024-06-03T03:19:49.416 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:49.416+0000 7fe311600700 1 --2- 172.21.15.117:0/2194902849 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fe2fc0a2c20 0x7fe2fc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:19:49.417 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:49.416+0000 7fe311600700 1 -- 172.21.15.117:0/2194902849 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fe3080cf4c0 con 0x7fe31406a170 2024-06-03T03:19:49.417 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:49.416+0000 7fe313e00700 1 --2- 172.21.15.117:0/2194902849 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fe2fc0a2c20 0x7fe2fc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:19:49.417 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:49.417+0000 7fe313e00700 1 --2- 172.21.15.117:0/2194902849 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fe2fc0a2c20 0x7fe2fc0a50e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe3140787e0 tx=0x7fe300009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:19:49.419 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:49.419+0000 7fe311600700 1 -- 172.21.15.117:0/2194902849 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe30809c4d0 con 0x7fe31406a170 2024-06-03T03:19:49.667 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:49.666+0000 7fe31b800700 1 -- 172.21.15.117:0/2194902849 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7fe31402dca0 con 0x7fe31406a170 2024-06-03T03:19:49.906 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:49 smithi151 ceph-mon[37043]: pgmap v330: 97 pgs: 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 851 B/s wr, 2 op/s 2024-06-03T03:19:49.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:49 smithi117 ceph-mon[31649]: pgmap v330: 97 pgs: 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 851 B/s wr, 2 op/s 2024-06-03T03:19:49.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:49 smithi183 ceph-mon[37326]: pgmap v330: 97 pgs: 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 851 B/s wr, 2 op/s 2024-06-03T03:19:50.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:50.656+0000 7fe311600700 1 -- 172.21.15.117:0/2194902849 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v89) ==== 157+0+0 (secure 0 0 0) 0x7fe30809bdd0 con 0x7fe31406a170 2024-06-03T03:19:50.662 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:50.661+0000 7fe31b800700 1 -- 172.21.15.117:0/2194902849 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fe2fc0a2c20 msgr2=0x7fe2fc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:19:50.662 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:50.661+0000 7fe31b800700 1 --2- 172.21.15.117:0/2194902849 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fe2fc0a2c20 0x7fe2fc0a50e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe3140787e0 tx=0x7fe300009260 comp rx=0 tx=0).stop 2024-06-03T03:19:50.662 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:50.661+0000 7fe31b800700 1 -- 172.21.15.117:0/2194902849 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe31406a170 msgr2=0x7fe31419ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:19:50.662 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:50.661+0000 7fe31b800700 1 --2- 172.21.15.117:0/2194902849 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe31406a170 0x7fe31419ada0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe30800e9a0 tx=0x7fe30800ecb0 comp rx=0 tx=0).stop 2024-06-03T03:19:50.662 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:50.661+0000 7fe31b800700 1 -- 172.21.15.117:0/2194902849 shutdown_connections 2024-06-03T03:19:50.662 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:50.661+0000 7fe31b800700 1 --2- 172.21.15.117:0/2194902849 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fe2fc0a2c20 0x7fe2fc0a50e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:50.662 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:50.661+0000 7fe31b800700 1 --2- 172.21.15.117:0/2194902849 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3141084e0 0x7fe31419f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:50.663 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:50.661+0000 7fe31b800700 1 --2- 172.21.15.117:0/2194902849 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe31406a170 0x7fe31419ada0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:50.663 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:50.661+0000 7fe31b800700 1 --2- 172.21.15.117:0/2194902849 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe31406ab10 0x7fe31419b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:50.663 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:50.661+0000 7fe31b800700 1 -- 172.21.15.117:0/2194902849 >> 172.21.15.117:0/2194902849 conn(0x7fe3140782c0 msgr2=0x7fe31410d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:19:50.663 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:50.661+0000 7fe31b800700 1 -- 172.21.15.117:0/2194902849 shutdown_connections 2024-06-03T03:19:50.663 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:50.661+0000 7fe31b800700 1 -- 172.21.15.117:0/2194902849 wait complete. 2024-06-03T03:19:50.906 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:50 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/2194902849' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-06-03T03:19:50.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:50 smithi151 ceph-mon[37043]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-06-03T03:19:50.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:50 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/2194902849' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-06-03T03:19:50.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:50 smithi117 ceph-mon[31649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-06-03T03:19:50.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:50 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/2194902849' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-06-03T03:19:50.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:50 smithi183 ceph-mon[37326]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-06-03T03:19:51.505 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-06-03T03:19:51.906 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:51 smithi151 ceph-mon[37043]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-06-03T03:19:51.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:51 smithi151 ceph-mon[37043]: osdmap e89: 12 total, 12 up, 12 in 2024-06-03T03:19:51.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:51 smithi151 ceph-mon[37043]: pgmap v332: 97 pgs: 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-06-03T03:19:51.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:51 smithi117 ceph-mon[31649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-06-03T03:19:51.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:51 smithi117 ceph-mon[31649]: osdmap e89: 12 total, 12 up, 12 in 2024-06-03T03:19:51.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:51 smithi117 ceph-mon[31649]: pgmap v332: 97 pgs: 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-06-03T03:19:51.951 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:19:51.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:51 smithi183 ceph-mon[37326]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-06-03T03:19:51.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:51 smithi183 ceph-mon[37326]: osdmap e89: 12 total, 12 up, 12 in 2024-06-03T03:19:51.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:51 smithi183 ceph-mon[37326]: pgmap v332: 97 pgs: 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-06-03T03:19:53.969 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:53 smithi117 ceph-mon[31649]: pgmap v333: 97 pgs: 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-06-03T03:19:54.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:53 smithi151 ceph-mon[37043]: pgmap v333: 97 pgs: 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-06-03T03:19:54.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:53 smithi183 ceph-mon[37326]: pgmap v333: 97 pgs: 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-06-03T03:19:54.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:54.262+0000 7efd75800700 1 -- 172.21.15.117:0/683301319 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd7010eef0 msgr2=0x7efd701112e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:19:54.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:54.262+0000 7efd75800700 1 --2- 172.21.15.117:0/683301319 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd7010eef0 0x7efd701112e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7efd68009990 tx=0x7efd68009ca0 comp rx=0 tx=0).stop 2024-06-03T03:19:54.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:54.262+0000 7efd75800700 1 -- 172.21.15.117:0/683301319 shutdown_connections 2024-06-03T03:19:54.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:54.262+0000 7efd75800700 1 --2- 172.21.15.117:0/683301319 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd7010eef0 0x7efd701112e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:54.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:54.262+0000 7efd75800700 1 --2- 172.21.15.117:0/683301319 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efd700746c0 0x7efd70074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:54.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:54.262+0000 7efd75800700 1 --2- 172.21.15.117:0/683301319 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efd70074fe0 0x7efd7010e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:54.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:54.262+0000 7efd75800700 1 -- 172.21.15.117:0/683301319 >> 172.21.15.117:0/683301319 conn(0x7efd70079560 msgr2=0x7efd70079970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:19:54.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:54.262+0000 7efd75800700 1 -- 172.21.15.117:0/683301319 shutdown_connections 2024-06-03T03:19:54.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:54.262+0000 7efd75800700 1 -- 172.21.15.117:0/683301319 wait complete. 2024-06-03T03:19:54.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:54.264+0000 7efd75800700 1 Processor -- start 2024-06-03T03:19:54.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:54.264+0000 7efd75800700 1 -- start start 2024-06-03T03:19:54.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:54.264+0000 7efd75800700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efd700746c0 0x7efd701a0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:19:54.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:54.264+0000 7efd75800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd70074fe0 0x7efd701a0e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:19:54.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:54.265+0000 7efd75800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efd7010eef0 0x7efd7019aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:19:54.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:54.265+0000 7efd75800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7efd701149a0 con 0x7efd70074fe0 2024-06-03T03:19:54.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:54.265+0000 7efd75800700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7efd70114820 con 0x7efd7010eef0 2024-06-03T03:19:54.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:54.265+0000 7efd75800700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7efd70114b20 con 0x7efd700746c0 2024-06-03T03:19:54.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:54.265+0000 7efd66000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd70074fe0 0x7efd701a0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:19:54.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:54.265+0000 7efd66000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd70074fe0 0x7efd701a0e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:48536/0 (socket says 172.21.15.117:48536) 2024-06-03T03:19:54.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:54.265+0000 7efd66000700 1 -- 172.21.15.117:0/1547931274 learned_addr learned my addr 172.21.15.117:0/1547931274 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:19:54.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:54.265+0000 7efd6ea00700 1 --2- 172.21.15.117:0/1547931274 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efd700746c0 0x7efd701a0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:19:54.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:54.265+0000 7efd6f400700 1 --2- 172.21.15.117:0/1547931274 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efd7010eef0 0x7efd7019aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:19:54.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:54.265+0000 7efd66000700 1 -- 172.21.15.117:0/1547931274 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efd700746c0 msgr2=0x7efd701a0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:19:54.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:54.265+0000 7efd66000700 1 --2- 172.21.15.117:0/1547931274 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efd700746c0 0x7efd701a0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:54.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:54.265+0000 7efd66000700 1 -- 172.21.15.117:0/1547931274 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efd7010eef0 msgr2=0x7efd7019aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:19:54.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:54.265+0000 7efd66000700 1 --2- 172.21.15.117:0/1547931274 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efd7010eef0 0x7efd7019aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:54.268 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:54.265+0000 7efd66000700 1 -- 172.21.15.117:0/1547931274 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd7019b300 con 0x7efd70074fe0 2024-06-03T03:19:54.268 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:54.266+0000 7efd66000700 1 --2- 172.21.15.117:0/1547931274 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd70074fe0 0x7efd701a0e80 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7efd5800d980 tx=0x7efd5800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:19:54.268 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:54.266+0000 7efd67e00700 1 -- 172.21.15.117:0/1547931274 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd58016070 con 0x7efd70074fe0 2024-06-03T03:19:54.268 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:54.266+0000 7efd67e00700 1 -- 172.21.15.117:0/1547931274 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd58004d10 con 0x7efd70074fe0 2024-06-03T03:19:54.268 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:54.266+0000 7efd75800700 1 -- 172.21.15.117:0/1547931274 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd7019b5f0 con 0x7efd70074fe0 2024-06-03T03:19:54.268 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:54.266+0000 7efd67e00700 1 -- 172.21.15.117:0/1547931274 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd58005670 con 0x7efd70074fe0 2024-06-03T03:19:54.268 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:54.266+0000 7efd75800700 1 -- 172.21.15.117:0/1547931274 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd701a7810 con 0x7efd70074fe0 2024-06-03T03:19:54.268 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:54.267+0000 7efd75800700 1 -- 172.21.15.117:0/1547931274 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd70050a00 con 0x7efd70074fe0 2024-06-03T03:19:54.270 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:54.269+0000 7efd67e00700 1 -- 172.21.15.117:0/1547931274 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7efd58004750 con 0x7efd70074fe0 2024-06-03T03:19:54.271 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:54.271+0000 7efd67e00700 1 --2- 172.21.15.117:0/1547931274 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7efd540a2cc0 0x7efd540a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:19:54.272 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:54.271+0000 7efd67e00700 1 -- 172.21.15.117:0/1547931274 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8277+0+0 (secure 0 0 0) 0x7efd580cf340 con 0x7efd70074fe0 2024-06-03T03:19:54.272 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:54.271+0000 7efd6ea00700 1 --2- 172.21.15.117:0/1547931274 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7efd540a2cc0 0x7efd540a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:19:54.272 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:54.272+0000 7efd6ea00700 1 --2- 172.21.15.117:0/1547931274 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7efd540a2cc0 0x7efd540a5180 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7efd60009550 tx=0x7efd600091f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:19:54.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:54.274+0000 7efd67e00700 1 -- 172.21.15.117:0/1547931274 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7efd5801f020 con 0x7efd70074fe0 2024-06-03T03:19:54.499 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:54.498+0000 7efd75800700 1 -- 172.21.15.117:0/1547931274 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7efd701a7a40 con 0x7efd70074fe0 2024-06-03T03:19:55.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:54 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/1547931274' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-06-03T03:19:55.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:54 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/1547931274' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-06-03T03:19:55.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:54 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/1547931274' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-06-03T03:19:55.889 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:55.888+0000 7efd67e00700 1 -- 172.21.15.117:0/1547931274 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v91) ==== 228+0+0 (secure 0 0 0) 0x7efd5809c210 con 0x7efd70074fe0 2024-06-03T03:19:55.896 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:55.894+0000 7efd75800700 1 -- 172.21.15.117:0/1547931274 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7efd540a2cc0 msgr2=0x7efd540a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:19:55.896 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:55.894+0000 7efd75800700 1 --2- 172.21.15.117:0/1547931274 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7efd540a2cc0 0x7efd540a5180 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7efd60009550 tx=0x7efd600091f0 comp rx=0 tx=0).stop 2024-06-03T03:19:55.896 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:55.894+0000 7efd75800700 1 -- 172.21.15.117:0/1547931274 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd70074fe0 msgr2=0x7efd701a0e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:19:55.897 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:55.894+0000 7efd75800700 1 --2- 172.21.15.117:0/1547931274 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd70074fe0 0x7efd701a0e80 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7efd5800d980 tx=0x7efd5800dc90 comp rx=0 tx=0).stop 2024-06-03T03:19:55.897 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:55.895+0000 7efd75800700 1 -- 172.21.15.117:0/1547931274 shutdown_connections 2024-06-03T03:19:55.897 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:55.895+0000 7efd75800700 1 --2- 172.21.15.117:0/1547931274 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7efd540a2cc0 0x7efd540a5180 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:55.897 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:55.895+0000 7efd75800700 1 --2- 172.21.15.117:0/1547931274 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efd7010eef0 0x7efd7019aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:55.897 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:55.895+0000 7efd75800700 1 --2- 172.21.15.117:0/1547931274 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efd700746c0 0x7efd701a0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:55.897 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:55.895+0000 7efd75800700 1 --2- 172.21.15.117:0/1547931274 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd70074fe0 0x7efd701a0e80 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:55.897 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:55.895+0000 7efd75800700 1 -- 172.21.15.117:0/1547931274 >> 172.21.15.117:0/1547931274 conn(0x7efd70079560 msgr2=0x7efd700ff5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:19:55.897 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:55.896+0000 7efd75800700 1 -- 172.21.15.117:0/1547931274 shutdown_connections 2024-06-03T03:19:55.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:55.896+0000 7efd75800700 1 -- 172.21.15.117:0/1547931274 wait complete. 2024-06-03T03:19:55.902 INFO:teuthology.orchestra.run.smithi117.stderr:pool 'cephfs_data_ec' created 2024-06-03T03:19:56.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:55 smithi151 ceph-mon[37043]: pgmap v334: 97 pgs: 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-06-03T03:19:56.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:55 smithi151 ceph-mon[37043]: osdmap e90: 12 total, 12 up, 12 in 2024-06-03T03:19:56.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:55 smithi117 ceph-mon[31649]: pgmap v334: 97 pgs: 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-06-03T03:19:56.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:55 smithi117 ceph-mon[31649]: osdmap e90: 12 total, 12 up, 12 in 2024-06-03T03:19:56.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:55 smithi183 ceph-mon[37326]: pgmap v334: 97 pgs: 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-06-03T03:19:56.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:55 smithi183 ceph-mon[37326]: osdmap e90: 12 total, 12 up, 12 in 2024-06-03T03:19:56.812 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-06-03T03:19:57.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:56 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/1547931274' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-06-03T03:19:57.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:56 smithi151 ceph-mon[37043]: osdmap e91: 12 total, 12 up, 12 in 2024-06-03T03:19:57.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:56 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/1547931274' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-06-03T03:19:57.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:56 smithi117 ceph-mon[31649]: osdmap e91: 12 total, 12 up, 12 in 2024-06-03T03:19:57.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:56 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/1547931274' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-06-03T03:19:57.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:56 smithi183 ceph-mon[37326]: osdmap e91: 12 total, 12 up, 12 in 2024-06-03T03:19:57.275 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:19:58.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:57 smithi117 ceph-mon[31649]: pgmap v337: 129 pgs: 2 creating+peering, 30 unknown, 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-06-03T03:19:58.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:57 smithi117 ceph-mon[31649]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-03T03:19:58.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:57 smithi117 ceph-mon[31649]: osdmap e92: 12 total, 12 up, 12 in 2024-06-03T03:19:58.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:57 smithi183 ceph-mon[37326]: pgmap v337: 129 pgs: 2 creating+peering, 30 unknown, 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-06-03T03:19:58.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:57 smithi183 ceph-mon[37326]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-03T03:19:58.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:57 smithi183 ceph-mon[37326]: osdmap e92: 12 total, 12 up, 12 in 2024-06-03T03:19:58.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:57 smithi151 ceph-mon[37043]: pgmap v337: 129 pgs: 2 creating+peering, 30 unknown, 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-06-03T03:19:58.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:57 smithi151 ceph-mon[37043]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-03T03:19:58.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:57 smithi151 ceph-mon[37043]: osdmap e92: 12 total, 12 up, 12 in 2024-06-03T03:19:59.049 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:58 smithi117 ceph-mon[31649]: osdmap e93: 12 total, 12 up, 12 in 2024-06-03T03:19:59.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:58 smithi183 ceph-mon[37326]: osdmap e93: 12 total, 12 up, 12 in 2024-06-03T03:19:59.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:58 smithi151 ceph-mon[37043]: osdmap e93: 12 total, 12 up, 12 in 2024-06-03T03:19:59.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:59.607+0000 7f6a1d200700 1 -- 172.21.15.117:0/223896615 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a18109390 msgr2=0x7f6a18115830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:19:59.614 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:59.607+0000 7f6a1d200700 1 --2- 172.21.15.117:0/223896615 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a18109390 0x7f6a18115830 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6a1000b0a0 tx=0x7f6a1000b3b0 comp rx=0 tx=0).stop 2024-06-03T03:19:59.614 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:59.607+0000 7f6a1d200700 1 -- 172.21.15.117:0/223896615 shutdown_connections 2024-06-03T03:19:59.614 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:59.607+0000 7f6a1d200700 1 --2- 172.21.15.117:0/223896615 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a18109390 0x7f6a18115830 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:59.615 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:59.607+0000 7f6a1d200700 1 --2- 172.21.15.117:0/223896615 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6a1810d8a0 0x7f6a1810dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:59.615 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:59.607+0000 7f6a1d200700 1 --2- 172.21.15.117:0/223896615 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a181089d0 0x7f6a18108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:59.615 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:59.607+0000 7f6a1d200700 1 -- 172.21.15.117:0/223896615 >> 172.21.15.117:0/223896615 conn(0x7f6a18078220 msgr2=0x7f6a181072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:19:59.615 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:59.607+0000 7f6a1d200700 1 -- 172.21.15.117:0/223896615 shutdown_connections 2024-06-03T03:19:59.615 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:59.607+0000 7f6a1d200700 1 -- 172.21.15.117:0/223896615 wait complete. 2024-06-03T03:19:59.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:59.608+0000 7f6a1d200700 1 Processor -- start 2024-06-03T03:19:59.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:59.608+0000 7f6a1d200700 1 -- start start 2024-06-03T03:19:59.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:59.609+0000 7f6a1d200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a181089d0 0x7f6a1819f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:19:59.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:59.609+0000 7f6a1d200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6a18109390 0x7f6a1819f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:19:59.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:59.609+0000 7f6a1d200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a1810d8a0 0x7f6a181a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:19:59.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:59.609+0000 7f6a1d200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6a18118d00 con 0x7f6a1810d8a0 2024-06-03T03:19:59.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:59.609+0000 7f6a1d200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6a18118b80 con 0x7f6a18109390 2024-06-03T03:19:59.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:59.609+0000 7f6a1d200700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6a18118e80 con 0x7f6a181089d0 2024-06-03T03:19:59.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:59.609+0000 7f6a17400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a181089d0 0x7f6a1819f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:19:59.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:59.609+0000 7f6a17400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a181089d0 0x7f6a1819f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:58546/0 (socket says 172.21.15.117:58546) 2024-06-03T03:19:59.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:59.609+0000 7f6a17400700 1 -- 172.21.15.117:0/1528608235 learned_addr learned my addr 172.21.15.117:0/1528608235 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:19:59.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:59.609+0000 7f6a16a00700 1 --2- 172.21.15.117:0/1528608235 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6a18109390 0x7f6a1819f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:19:59.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:59.610+0000 7f6a17400700 1 -- 172.21.15.117:0/1528608235 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6a18109390 msgr2=0x7f6a1819f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:19:59.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:59.610+0000 7f6a17400700 1 --2- 172.21.15.117:0/1528608235 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6a18109390 0x7f6a1819f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:59.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:59.610+0000 7f6a17400700 1 -- 172.21.15.117:0/1528608235 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a1810d8a0 msgr2=0x7f6a181a39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:19:59.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:59.610+0000 7f6a17e00700 1 --2- 172.21.15.117:0/1528608235 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a1810d8a0 0x7f6a181a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:19:59.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:59.610+0000 7f6a17400700 1 --2- 172.21.15.117:0/1528608235 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a1810d8a0 0x7f6a181a39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:59.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:59.610+0000 7f6a17400700 1 -- 172.21.15.117:0/1528608235 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a181a4110 con 0x7f6a181089d0 2024-06-03T03:19:59.620 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:59.610+0000 7f6a17e00700 1 --2- 172.21.15.117:0/1528608235 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a1810d8a0 0x7f6a181a39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T03:19:59.620 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:59.610+0000 7f6a17400700 1 --2- 172.21.15.117:0/1528608235 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a181089d0 0x7f6a1819f150 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6a0800eb30 tx=0x7f6a0800ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:19:59.620 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:59.610+0000 7f6a0fe00700 1 -- 172.21.15.117:0/1528608235 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a0800cc90 con 0x7f6a181089d0 2024-06-03T03:19:59.620 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:59.611+0000 7f6a0fe00700 1 -- 172.21.15.117:0/1528608235 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a08004500 con 0x7f6a181089d0 2024-06-03T03:19:59.620 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:59.611+0000 7f6a1d200700 1 -- 172.21.15.117:0/1528608235 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a181a4400 con 0x7f6a181089d0 2024-06-03T03:19:59.621 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:59.611+0000 7f6a0fe00700 1 -- 172.21.15.117:0/1528608235 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a08010430 con 0x7f6a181089d0 2024-06-03T03:19:59.621 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:59.611+0000 7f6a1d200700 1 -- 172.21.15.117:0/1528608235 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a181abc80 con 0x7f6a181089d0 2024-06-03T03:19:59.621 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:59.612+0000 7f6a1d200700 1 -- 172.21.15.117:0/1528608235 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a18069180 con 0x7f6a181089d0 2024-06-03T03:19:59.623 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:59.614+0000 7f6a0fe00700 1 -- 172.21.15.117:0/1528608235 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f6a08043ae0 con 0x7f6a181089d0 2024-06-03T03:19:59.623 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:59.615+0000 7f6a0fe00700 1 --2- 172.21.15.117:0/1528608235 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f69fc0a2c20 0x7f69fc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:19:59.623 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:59.615+0000 7f6a0fe00700 1 -- 172.21.15.117:0/1528608235 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8770+0+0 (secure 0 0 0) 0x7f6a080cf590 con 0x7f6a181089d0 2024-06-03T03:19:59.623 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:59.615+0000 7f6a16a00700 1 --2- 172.21.15.117:0/1528608235 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f69fc0a2c20 0x7f69fc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:19:59.623 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:59.616+0000 7f6a16a00700 1 --2- 172.21.15.117:0/1528608235 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f69fc0a2c20 0x7f69fc0a50e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6a181084b0 tx=0x7f6a04009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:19:59.624 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:59.618+0000 7f6a0fe00700 1 -- 172.21.15.117:0/1528608235 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6a0809c270 con 0x7f6a181089d0 2024-06-03T03:19:59.840 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:59.839+0000 7f6a1d200700 1 -- 172.21.15.117:0/1528608235 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f6a1802dca0 con 0x7f6a181089d0 2024-06-03T03:19:59.926 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:59.925+0000 7f6a0fe00700 1 -- 172.21.15.117:0/1528608235 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v94) ==== 169+0+0 (secure 0 0 0) 0x7f6a0809bb70 con 0x7f6a181089d0 2024-06-03T03:19:59.930 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:59.929+0000 7f6a1d200700 1 -- 172.21.15.117:0/1528608235 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f69fc0a2c20 msgr2=0x7f69fc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:19:59.930 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:59.929+0000 7f6a1d200700 1 --2- 172.21.15.117:0/1528608235 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f69fc0a2c20 0x7f69fc0a50e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6a181084b0 tx=0x7f6a04009260 comp rx=0 tx=0).stop 2024-06-03T03:19:59.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:59.929+0000 7f6a1d200700 1 -- 172.21.15.117:0/1528608235 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a181089d0 msgr2=0x7f6a1819f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:19:59.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:59.929+0000 7f6a1d200700 1 --2- 172.21.15.117:0/1528608235 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a181089d0 0x7f6a1819f150 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6a0800eb30 tx=0x7f6a0800ee40 comp rx=0 tx=0).stop 2024-06-03T03:19:59.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:59.930+0000 7f6a1d200700 1 -- 172.21.15.117:0/1528608235 shutdown_connections 2024-06-03T03:19:59.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:59.930+0000 7f6a1d200700 1 --2- 172.21.15.117:0/1528608235 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f69fc0a2c20 0x7f69fc0a50e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:59.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:59.930+0000 7f6a1d200700 1 --2- 172.21.15.117:0/1528608235 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a1810d8a0 0x7f6a181a39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:59.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:59.930+0000 7f6a1d200700 1 --2- 172.21.15.117:0/1528608235 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a181089d0 0x7f6a1819f150 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:59.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:59.930+0000 7f6a1d200700 1 --2- 172.21.15.117:0/1528608235 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6a18109390 0x7f6a1819f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:19:59.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:59.930+0000 7f6a1d200700 1 -- 172.21.15.117:0/1528608235 >> 172.21.15.117:0/1528608235 conn(0x7f6a18078220 msgr2=0x7f6a180787e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:19:59.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:59.930+0000 7f6a1d200700 1 -- 172.21.15.117:0/1528608235 shutdown_connections 2024-06-03T03:19:59.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:19:59.930+0000 7f6a1d200700 1 -- 172.21.15.117:0/1528608235 wait complete. 2024-06-03T03:19:59.936 INFO:teuthology.orchestra.run.smithi117.stderr:set pool 4 allow_ec_overwrites to true 2024-06-03T03:20:00.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:59 smithi117 ceph-mon[31649]: pgmap v340: 129 pgs: 2 creating+peering, 16 unknown, 111 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:20:00.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:59 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/1528608235' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-06-03T03:20:00.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:19:59 smithi117 ceph-mon[31649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-06-03T03:20:00.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:59 smithi183 ceph-mon[37326]: pgmap v340: 129 pgs: 2 creating+peering, 16 unknown, 111 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:20:00.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:59 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/1528608235' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-06-03T03:20:00.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:19:59 smithi183 ceph-mon[37326]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-06-03T03:20:00.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:59 smithi151 ceph-mon[37043]: pgmap v340: 129 pgs: 2 creating+peering, 16 unknown, 111 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:20:00.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:59 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/1528608235' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-06-03T03:20:00.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:19:59 smithi151 ceph-mon[37043]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-06-03T03:20:00.888 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph fs add_data_pool cephfs cephfs_data_ec 2024-06-03T03:20:00.960 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:00 smithi117 ceph-mon[31649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-06-03T03:20:00.960 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:00 smithi117 ceph-mon[31649]: osdmap e94: 12 total, 12 up, 12 in 2024-06-03T03:20:00.960 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:00 smithi117 ceph-mon[31649]: Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled 2024-06-03T03:20:00.960 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:00 smithi117 ceph-mon[31649]: [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-06-03T03:20:00.960 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:00 smithi117 ceph-mon[31649]: application not enabled on pool 'cephfs_data_ec' 2024-06-03T03:20:00.960 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:00 smithi117 ceph-mon[31649]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-06-03T03:20:01.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:00 smithi183 ceph-mon[37326]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-06-03T03:20:01.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:00 smithi183 ceph-mon[37326]: osdmap e94: 12 total, 12 up, 12 in 2024-06-03T03:20:01.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:00 smithi183 ceph-mon[37326]: Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled 2024-06-03T03:20:01.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:00 smithi183 ceph-mon[37326]: [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-06-03T03:20:01.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:00 smithi183 ceph-mon[37326]: application not enabled on pool 'cephfs_data_ec' 2024-06-03T03:20:01.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:00 smithi183 ceph-mon[37326]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-06-03T03:20:01.329 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:20:01.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:00 smithi151 ceph-mon[37043]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-06-03T03:20:01.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:00 smithi151 ceph-mon[37043]: osdmap e94: 12 total, 12 up, 12 in 2024-06-03T03:20:01.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:00 smithi151 ceph-mon[37043]: Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled 2024-06-03T03:20:01.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:00 smithi151 ceph-mon[37043]: [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-06-03T03:20:01.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:00 smithi151 ceph-mon[37043]: application not enabled on pool 'cephfs_data_ec' 2024-06-03T03:20:01.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:00 smithi151 ceph-mon[37043]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-06-03T03:20:02.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:01 smithi117 ceph-mon[31649]: pgmap v342: 129 pgs: 2 creating+peering, 4 unknown, 123 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:20:02.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:01 smithi183 ceph-mon[37326]: pgmap v342: 129 pgs: 2 creating+peering, 4 unknown, 123 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:20:02.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:01 smithi151 ceph-mon[37043]: pgmap v342: 129 pgs: 2 creating+peering, 4 unknown, 123 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:20:03.598 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:03.596+0000 7f3b49600700 1 -- 172.21.15.117:0/144656838 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b441033a0 msgr2=0x7f3b4406ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:20:03.598 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:03.596+0000 7f3b49600700 1 --2- 172.21.15.117:0/144656838 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b441033a0 0x7f3b4406ab60 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3b3400b0a0 tx=0x7f3b3400b3b0 comp rx=0 tx=0).stop 2024-06-03T03:20:03.598 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:03.596+0000 7f3b49600700 1 -- 172.21.15.117:0/144656838 shutdown_connections 2024-06-03T03:20:03.598 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:03.596+0000 7f3b49600700 1 --2- 172.21.15.117:0/144656838 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b4410ede0 0x7f3b441111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:03.598 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:03.596+0000 7f3b49600700 1 --2- 172.21.15.117:0/144656838 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b441033a0 0x7f3b4406ab60 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:03.598 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:03.596+0000 7f3b49600700 1 --2- 172.21.15.117:0/144656838 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3b4406b0a0 0x7f3b4406b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:03.598 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:03.596+0000 7f3b49600700 1 -- 172.21.15.117:0/144656838 >> 172.21.15.117:0/144656838 conn(0x7f3b44078340 msgr2=0x7f3b44078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:20:03.598 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:03.596+0000 7f3b49600700 1 -- 172.21.15.117:0/144656838 shutdown_connections 2024-06-03T03:20:03.599 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:03.596+0000 7f3b49600700 1 -- 172.21.15.117:0/144656838 wait complete. 2024-06-03T03:20:03.600 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:03.598+0000 7f3b49600700 1 Processor -- start 2024-06-03T03:20:03.600 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:03.598+0000 7f3b49600700 1 -- start start 2024-06-03T03:20:03.600 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:03.598+0000 7f3b49600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b4406b0a0 0x7f3b4419ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:20:03.600 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:03.598+0000 7f3b49600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b441033a0 0x7f3b4419b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:20:03.600 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:03.598+0000 7f3b49600700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3b4410ede0 0x7f3b4419f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:20:03.600 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:03.598+0000 7f3b49600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3b441149f0 con 0x7f3b441033a0 2024-06-03T03:20:03.600 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:03.598+0000 7f3b49600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3b44114870 con 0x7f3b4406b0a0 2024-06-03T03:20:03.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:03.598+0000 7f3b49600700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3b44114b70 con 0x7f3b4410ede0 2024-06-03T03:20:03.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:03.599+0000 7f3b3a000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b441033a0 0x7f3b4419b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:20:03.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:03.599+0000 7f3b3a000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b441033a0 0x7f3b4419b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:48506/0 (socket says 172.21.15.117:48506) 2024-06-03T03:20:03.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:03.599+0000 7f3b3a000700 1 -- 172.21.15.117:0/2471910002 learned_addr learned my addr 172.21.15.117:0/2471910002 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:20:03.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:03.599+0000 7f3b42a00700 1 --2- 172.21.15.117:0/2471910002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b4406b0a0 0x7f3b4419ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:20:03.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:03.599+0000 7f3b43400700 1 --2- 172.21.15.117:0/2471910002 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3b4410ede0 0x7f3b4419f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:20:03.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:03.599+0000 7f3b3a000700 1 -- 172.21.15.117:0/2471910002 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3b4410ede0 msgr2=0x7f3b4419f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:20:03.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:03.599+0000 7f3b3a000700 1 --2- 172.21.15.117:0/2471910002 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3b4410ede0 0x7f3b4419f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:03.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:03.599+0000 7f3b3a000700 1 -- 172.21.15.117:0/2471910002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b4406b0a0 msgr2=0x7f3b4419ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:20:03.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:03.599+0000 7f3b3a000700 1 --2- 172.21.15.117:0/2471910002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b4406b0a0 0x7f3b4419ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:03.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:03.599+0000 7f3b3a000700 1 -- 172.21.15.117:0/2471910002 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b4419fe30 con 0x7f3b441033a0 2024-06-03T03:20:03.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:03.599+0000 7f3b42a00700 1 --2- 172.21.15.117:0/2471910002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b4406b0a0 0x7f3b4419ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:20:03.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:03.599+0000 7f3b43400700 1 --2- 172.21.15.117:0/2471910002 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3b4410ede0 0x7f3b4419f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T03:20:03.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:03.600+0000 7f3b3a000700 1 --2- 172.21.15.117:0/2471910002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b441033a0 0x7f3b4419b360 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c00d950 tx=0x7f3b2c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:20:03.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:03.600+0000 7f3b3be00700 1 -- 172.21.15.117:0/2471910002 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c016070 con 0x7f3b441033a0 2024-06-03T03:20:03.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:03.600+0000 7f3b3be00700 1 -- 172.21.15.117:0/2471910002 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b2c009e20 con 0x7f3b441033a0 2024-06-03T03:20:03.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:03.600+0000 7f3b49600700 1 -- 172.21.15.117:0/2471910002 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b441a0120 con 0x7f3b441033a0 2024-06-03T03:20:03.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:03.600+0000 7f3b3be00700 1 -- 172.21.15.117:0/2471910002 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c00f620 con 0x7f3b441033a0 2024-06-03T03:20:03.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:03.600+0000 7f3b49600700 1 -- 172.21.15.117:0/2471910002 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b441a79a0 con 0x7f3b441033a0 2024-06-03T03:20:03.603 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:03.601+0000 7f3b49600700 1 -- 172.21.15.117:0/2471910002 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b441019d0 con 0x7f3b441033a0 2024-06-03T03:20:03.605 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:03.603+0000 7f3b3be00700 1 -- 172.21.15.117:0/2471910002 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f3b2c019070 con 0x7f3b441033a0 2024-06-03T03:20:03.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:03.604+0000 7f3b3be00700 1 --2- 172.21.15.117:0/2471910002 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f3b280a2c20 0x7f3b280a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:20:03.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:03.604+0000 7f3b3be00700 1 -- 172.21.15.117:0/2471910002 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8770+0+0 (secure 0 0 0) 0x7f3b2c0cee90 con 0x7f3b441033a0 2024-06-03T03:20:03.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:03.605+0000 7f3b42a00700 1 --2- 172.21.15.117:0/2471910002 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f3b280a2c20 0x7f3b280a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:20:03.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:03.605+0000 7f3b42a00700 1 --2- 172.21.15.117:0/2471910002 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f3b280a2c20 0x7f3b280a50e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3b34009fd0 tx=0x7f3b34006210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:20:03.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:03.610+0000 7f3b3be00700 1 -- 172.21.15.117:0/2471910002 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3b2c09bb70 con 0x7f3b441033a0 2024-06-03T03:20:03.882 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:03.881+0000 7f3b49600700 1 -- 172.21.15.117:0/2471910002 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f3b441a7ba0 con 0x7f3b441033a0 2024-06-03T03:20:03.959 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:03.957+0000 7f3b3be00700 1 -- 172.21.15.117:0/2471910002 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7f3b2c09b470 con 0x7f3b441033a0 2024-06-03T03:20:03.962 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:03.961+0000 7f3b49600700 1 -- 172.21.15.117:0/2471910002 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f3b280a2c20 msgr2=0x7f3b280a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:20:03.962 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:03.961+0000 7f3b49600700 1 --2- 172.21.15.117:0/2471910002 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f3b280a2c20 0x7f3b280a50e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3b34009fd0 tx=0x7f3b34006210 comp rx=0 tx=0).stop 2024-06-03T03:20:03.963 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:03.961+0000 7f3b49600700 1 -- 172.21.15.117:0/2471910002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b441033a0 msgr2=0x7f3b4419b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:20:03.963 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:03.961+0000 7f3b49600700 1 --2- 172.21.15.117:0/2471910002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b441033a0 0x7f3b4419b360 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c00d950 tx=0x7f3b2c00dc60 comp rx=0 tx=0).stop 2024-06-03T03:20:03.963 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:03.961+0000 7f3b49600700 1 -- 172.21.15.117:0/2471910002 shutdown_connections 2024-06-03T03:20:03.963 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:03.961+0000 7f3b49600700 1 --2- 172.21.15.117:0/2471910002 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f3b280a2c20 0x7f3b280a50e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:03.963 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:03.961+0000 7f3b49600700 1 --2- 172.21.15.117:0/2471910002 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3b4410ede0 0x7f3b4419f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:03.963 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:03.961+0000 7f3b49600700 1 --2- 172.21.15.117:0/2471910002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b4406b0a0 0x7f3b4419ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:03.963 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:03.961+0000 7f3b49600700 1 --2- 172.21.15.117:0/2471910002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b441033a0 0x7f3b4419b360 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:03.963 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:03.961+0000 7f3b49600700 1 -- 172.21.15.117:0/2471910002 >> 172.21.15.117:0/2471910002 conn(0x7f3b44078340 msgr2=0x7f3b44100760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:20:03.963 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:03.962+0000 7f3b49600700 1 -- 172.21.15.117:0/2471910002 shutdown_connections 2024-06-03T03:20:03.963 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:03.962+0000 7f3b49600700 1 -- 172.21.15.117:0/2471910002 wait complete. 2024-06-03T03:20:03.964 INFO:teuthology.orchestra.run.smithi117.stderr:added data pool 4 to fsmap 2024-06-03T03:20:04.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:03 smithi117 ceph-mon[31649]: pgmap v343: 129 pgs: 129 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:20:04.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:03 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/2471910002' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-06-03T03:20:04.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:03 smithi183 ceph-mon[37326]: pgmap v343: 129 pgs: 129 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:20:04.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:03 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/2471910002' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-06-03T03:20:04.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:03 smithi151 ceph-mon[37043]: pgmap v343: 129 pgs: 129 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:20:04.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:03 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/2471910002' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-06-03T03:20:04.886 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-03T03:20:05.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:04 smithi117 ceph-mon[31649]: osdmap e95: 12 total, 12 up, 12 in 2024-06-03T03:20:05.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:04 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/2471910002' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-06-03T03:20:05.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:04 smithi117 ceph-mon[31649]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-06-03T03:20:05.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:04 smithi183 ceph-mon[37326]: osdmap e95: 12 total, 12 up, 12 in 2024-06-03T03:20:05.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:04 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/2471910002' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-06-03T03:20:05.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:04 smithi183 ceph-mon[37326]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-06-03T03:20:05.340 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:20:05.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:04 smithi151 ceph-mon[37043]: osdmap e95: 12 total, 12 up, 12 in 2024-06-03T03:20:05.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:04 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/2471910002' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-06-03T03:20:05.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:04 smithi151 ceph-mon[37043]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-06-03T03:20:06.011 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:05 smithi117 ceph-mon[31649]: pgmap v345: 129 pgs: 129 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:20:06.011 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:05 smithi117 ceph-mon[31649]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-06-03T03:20:06.011 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:05 smithi117 ceph-mon[31649]: Cluster is now healthy 2024-06-03T03:20:06.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:05 smithi183 ceph-mon[37326]: pgmap v345: 129 pgs: 129 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:20:06.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:05 smithi183 ceph-mon[37326]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-06-03T03:20:06.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:05 smithi183 ceph-mon[37326]: Cluster is now healthy 2024-06-03T03:20:06.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:05 smithi151 ceph-mon[37043]: pgmap v345: 129 pgs: 129 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:20:06.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:05 smithi151 ceph-mon[37043]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-06-03T03:20:06.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:05 smithi151 ceph-mon[37043]: Cluster is now healthy 2024-06-03T03:20:07.797 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:07.795+0000 7fd835400700 1 -- 172.21.15.117:0/1659996848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd830074fe0 msgr2=0x7fd83010e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:20:07.797 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:07.795+0000 7fd835400700 1 --2- 172.21.15.117:0/1659996848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd830074fe0 0x7fd83010e950 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd818009990 tx=0x7fd818009ca0 comp rx=0 tx=0).stop 2024-06-03T03:20:07.797 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:07.795+0000 7fd835400700 1 -- 172.21.15.117:0/1659996848 shutdown_connections 2024-06-03T03:20:07.797 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:07.795+0000 7fd835400700 1 --2- 172.21.15.117:0/1659996848 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd83010ee90 0x7fd830111280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:07.797 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:07.795+0000 7fd835400700 1 --2- 172.21.15.117:0/1659996848 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd8300746c0 0x7fd830074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:07.797 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:07.795+0000 7fd835400700 1 --2- 172.21.15.117:0/1659996848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd830074fe0 0x7fd83010e950 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:07.797 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:07.795+0000 7fd835400700 1 -- 172.21.15.117:0/1659996848 >> 172.21.15.117:0/1659996848 conn(0x7fd8300795f0 msgr2=0x7fd830079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:20:07.797 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:07.796+0000 7fd835400700 1 -- 172.21.15.117:0/1659996848 shutdown_connections 2024-06-03T03:20:07.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:07.796+0000 7fd835400700 1 -- 172.21.15.117:0/1659996848 wait complete. 2024-06-03T03:20:07.799 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:07.797+0000 7fd835400700 1 Processor -- start 2024-06-03T03:20:07.799 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:07.797+0000 7fd835400700 1 -- start start 2024-06-03T03:20:07.799 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:07.797+0000 7fd835400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd8300746c0 0x7fd83019ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:20:07.799 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:07.797+0000 7fd835400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd830074fe0 0x7fd83019b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:20:07.799 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:07.797+0000 7fd835400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd83010ee90 0x7fd83019f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:20:07.799 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:07.797+0000 7fd835400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd8301149a0 con 0x7fd830074fe0 2024-06-03T03:20:07.799 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:07.797+0000 7fd835400700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd830114820 con 0x7fd8300746c0 2024-06-03T03:20:07.799 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:07.797+0000 7fd835400700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd830114b20 con 0x7fd83010ee90 2024-06-03T03:20:07.799 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:07.797+0000 7fd827e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd830074fe0 0x7fd83019b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:20:07.799 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:07.797+0000 7fd82e800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd8300746c0 0x7fd83019ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:20:07.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:07.798+0000 7fd82f200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd83010ee90 0x7fd83019f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:20:07.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:07.798+0000 7fd827e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd830074fe0 0x7fd83019b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:51508/0 (socket says 172.21.15.117:51508) 2024-06-03T03:20:07.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:07.798+0000 7fd82e800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd8300746c0 0x7fd83019ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:36128/0 (socket says 172.21.15.117:36128) 2024-06-03T03:20:07.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:07.798+0000 7fd827e00700 1 -- 172.21.15.117:0/1330852264 learned_addr learned my addr 172.21.15.117:0/1330852264 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:20:07.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:07.798+0000 7fd82f200700 1 -- 172.21.15.117:0/1330852264 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd8300746c0 msgr2=0x7fd83019ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:20:07.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:07.798+0000 7fd82f200700 1 --2- 172.21.15.117:0/1330852264 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd8300746c0 0x7fd83019ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:07.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:07.798+0000 7fd82f200700 1 -- 172.21.15.117:0/1330852264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd830074fe0 msgr2=0x7fd83019b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:20:07.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:07.798+0000 7fd82f200700 1 --2- 172.21.15.117:0/1330852264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd830074fe0 0x7fd83019b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:07.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:07.798+0000 7fd82f200700 1 -- 172.21.15.117:0/1330852264 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd83019fe40 con 0x7fd83010ee90 2024-06-03T03:20:07.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:07.798+0000 7fd82f200700 1 --2- 172.21.15.117:0/1330852264 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd83010ee90 0x7fd83019f700 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd82800ef90 tx=0x7fd82800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:20:07.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:07.799+0000 7fd827400700 1 -- 172.21.15.117:0/1330852264 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8280092e0 con 0x7fd83010ee90 2024-06-03T03:20:07.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:07.799+0000 7fd827400700 1 -- 172.21.15.117:0/1330852264 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd828004bb0 con 0x7fd83010ee90 2024-06-03T03:20:07.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:07.799+0000 7fd827400700 1 -- 172.21.15.117:0/1330852264 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd828004280 con 0x7fd83010ee90 2024-06-03T03:20:07.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:07.799+0000 7fd835400700 1 -- 172.21.15.117:0/1330852264 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8301a00d0 con 0x7fd83010ee90 2024-06-03T03:20:07.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:07.799+0000 7fd835400700 1 -- 172.21.15.117:0/1330852264 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8301a79b0 con 0x7fd83010ee90 2024-06-03T03:20:07.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:07.800+0000 7fd835400700 1 -- 172.21.15.117:0/1330852264 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd830069180 con 0x7fd83010ee90 2024-06-03T03:20:07.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:07.802+0000 7fd827400700 1 -- 172.21.15.117:0/1330852264 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fd82801b030 con 0x7fd83010ee90 2024-06-03T03:20:07.804 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:07.803+0000 7fd827400700 1 --2- 172.21.15.117:0/1330852264 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fd8140a2c20 0x7fd8140a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:20:07.804 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:07.803+0000 7fd827400700 1 -- 172.21.15.117:0/1330852264 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8802+0+0 (secure 0 0 0) 0x7fd8280cef10 con 0x7fd83010ee90 2024-06-03T03:20:07.804 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:07.803+0000 7fd82e800700 1 --2- 172.21.15.117:0/1330852264 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fd8140a2c20 0x7fd8140a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:20:07.805 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:07.804+0000 7fd82e800700 1 --2- 172.21.15.117:0/1330852264 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fd8140a2c20 0x7fd8140a50e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd820009c60 tx=0x7fd820009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:20:07.808 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:07.807+0000 7fd827400700 1 -- 172.21.15.117:0/1330852264 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd828022020 con 0x7fd83010ee90 2024-06-03T03:20:08.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:08.084+0000 7fd835400700 1 -- 172.21.15.117:0/1330852264 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd83002dca0 con 0x7fd83010ee90 2024-06-03T03:20:08.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:08.085+0000 7fd827400700 1 -- 172.21.15.117:0/1330852264 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11819 (secure 0 0 0) 0x7fd828010040 con 0x7fd83010ee90 2024-06-03T03:20:08.088 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T03:20:08.088 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3353001119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3353001119},{"type":"v1","addr":"172.21.15.117:6837","nonce":3353001119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14388,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6839/405782304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":405782304},{"type":"v1","addr":"172.21.15.117:6839","nonce":405782304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3355662416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3355662416},{"type":"v1","addr":"172.21.15.151:6833","nonce":3355662416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/2429725085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2429725085},{"type":"v1","addr":"172.21.15.183:6833","nonce":2429725085}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/31153579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":31153579},{"type":"v1","addr":"172.21.15.151:6835","nonce":31153579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24466,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/2884460088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2884460088},{"type":"v1","addr":"172.21.15.117:6835","nonce":2884460088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24479,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/2392407848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":2392407848},{"type":"v1","addr":"172.21.15.151:6837","nonce":2392407848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24487,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/1009004320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1009004320},{"type":"v1","addr":"172.21.15.183:6835","nonce":1009004320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24499,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/2697518526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":2697518526},{"type":"v1","addr":"172.21.15.183:6837","nonce":2697518526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24503,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6841/1212242967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6840","nonce":1212242967},{"type":"v1","addr":"172.21.15.117:6841","nonce":1212242967}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24512,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/3623444616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":3623444616},{"type":"v1","addr":"172.21.15.151:6839","nonce":3623444616}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-06-03T03:19:44.629822+0000","modified":"2024-06-03T03:20:03.883250+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24517},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24517":{"gid":24517,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":10,"addr":"172.21.15.183:6839/1439615083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1439615083},{"type":"v1","addr":"172.21.15.183:6839","nonce":1439615083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24517,"qdb_cluster":[24517]},"id":1}]} 2024-06-03T03:20:08.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:08.088+0000 7fd835400700 1 -- 172.21.15.117:0/1330852264 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fd8140a2c20 msgr2=0x7fd8140a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:20:08.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:08.088+0000 7fd835400700 1 --2- 172.21.15.117:0/1330852264 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fd8140a2c20 0x7fd8140a50e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd820009c60 tx=0x7fd820009260 comp rx=0 tx=0).stop 2024-06-03T03:20:08.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:08.088+0000 7fd835400700 1 -- 172.21.15.117:0/1330852264 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd83010ee90 msgr2=0x7fd83019f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:20:08.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:08.088+0000 7fd835400700 1 --2- 172.21.15.117:0/1330852264 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd83010ee90 0x7fd83019f700 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd82800ef90 tx=0x7fd82800c3a0 comp rx=0 tx=0).stop 2024-06-03T03:20:08.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:08.089+0000 7fd835400700 1 -- 172.21.15.117:0/1330852264 shutdown_connections 2024-06-03T03:20:08.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:08.089+0000 7fd835400700 1 --2- 172.21.15.117:0/1330852264 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fd8140a2c20 0x7fd8140a50e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:08.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:08.089+0000 7fd835400700 1 --2- 172.21.15.117:0/1330852264 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd83010ee90 0x7fd83019f700 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:08.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:08.089+0000 7fd835400700 1 --2- 172.21.15.117:0/1330852264 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd8300746c0 0x7fd83019ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:08.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:08.089+0000 7fd835400700 1 --2- 172.21.15.117:0/1330852264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd830074fe0 0x7fd83019b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:08.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:08.089+0000 7fd835400700 1 -- 172.21.15.117:0/1330852264 >> 172.21.15.117:0/1330852264 conn(0x7fd8300795f0 msgr2=0x7fd8300ff660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:20:08.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:08.089+0000 7fd835400700 1 -- 172.21.15.117:0/1330852264 shutdown_connections 2024-06-03T03:20:08.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:08.089+0000 7fd835400700 1 -- 172.21.15.117:0/1330852264 wait complete. 2024-06-03T03:20:08.093 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 17 2024-06-03T03:20:08.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:07 smithi183 ceph-mon[37326]: pgmap v346: 129 pgs: 129 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:20:08.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:07 smithi151 ceph-mon[37043]: pgmap v346: 129 pgs: 129 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:20:08.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:07 smithi117 ceph-mon[31649]: pgmap v346: 129 pgs: 129 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:20:08.945 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-03T03:20:09.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:08 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/1330852264' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T03:20:09.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:08 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/1330852264' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T03:20:09.387 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:20:09.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:08 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/1330852264' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T03:20:10.061 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:09 smithi117 ceph-mon[31649]: pgmap v347: 129 pgs: 129 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:20:10.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:09 smithi183 ceph-mon[37326]: pgmap v347: 129 pgs: 129 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:20:10.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:09 smithi151 ceph-mon[37043]: pgmap v347: 129 pgs: 129 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:20:11.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:11.735+0000 7f2cb1600700 1 -- 172.21.15.117:0/3946159529 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2cac1094c0 msgr2=0x7f2cac115980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:20:11.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:11.735+0000 7f2cb1600700 1 --2- 172.21.15.117:0/3946159529 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2cac1094c0 0x7f2cac115980 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2ca400b0a0 tx=0x7f2ca400b3b0 comp rx=0 tx=0).stop 2024-06-03T03:20:11.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:11.736+0000 7f2cb1600700 1 -- 172.21.15.117:0/3946159529 shutdown_connections 2024-06-03T03:20:11.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:11.736+0000 7f2cb1600700 1 --2- 172.21.15.117:0/3946159529 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2cac1094c0 0x7f2cac115980 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:11.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:11.736+0000 7f2cb1600700 1 --2- 172.21.15.117:0/3946159529 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2cac10eab0 0x7f2cac10ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:11.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:11.736+0000 7f2cb1600700 1 --2- 172.21.15.117:0/3946159529 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2cac108b00 0x7f2cac108f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:11.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:11.736+0000 7f2cb1600700 1 -- 172.21.15.117:0/3946159529 >> 172.21.15.117:0/3946159529 conn(0x7f2cac075c60 msgr2=0x7f2cac078080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:20:11.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:11.736+0000 7f2cb1600700 1 -- 172.21.15.117:0/3946159529 shutdown_connections 2024-06-03T03:20:11.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:11.736+0000 7f2cb1600700 1 -- 172.21.15.117:0/3946159529 wait complete. 2024-06-03T03:20:11.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:11.737+0000 7f2cb1600700 1 Processor -- start 2024-06-03T03:20:11.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:11.737+0000 7f2cb1600700 1 -- start start 2024-06-03T03:20:11.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:11.738+0000 7f2cb1600700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2cac108b00 0x7f2cac19f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:20:11.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:11.738+0000 7f2cb1600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2cac1094c0 0x7f2cac19f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:20:11.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:11.738+0000 7f2cb1600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2cac10eab0 0x7f2cac1a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:20:11.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:11.738+0000 7f2cb1600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2cac118e40 con 0x7f2cac10eab0 2024-06-03T03:20:11.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:11.738+0000 7f2cb1600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2cac118cc0 con 0x7f2cac1094c0 2024-06-03T03:20:11.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:11.738+0000 7f2cb1600700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2cac118fc0 con 0x7f2cac108b00 2024-06-03T03:20:11.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:11.738+0000 7f2cab400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2cac10eab0 0x7f2cac1a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:20:11.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:11.738+0000 7f2ca3e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2cac1094c0 0x7f2cac19f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:20:11.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:11.738+0000 7f2cab400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2cac10eab0 0x7f2cac1a3aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:51522/0 (socket says 172.21.15.117:51522) 2024-06-03T03:20:11.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:11.738+0000 7f2caaa00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2cac108b00 0x7f2cac19f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:20:11.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:11.738+0000 7f2cab400700 1 -- 172.21.15.117:0/67403125 learned_addr learned my addr 172.21.15.117:0/67403125 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:20:11.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:11.739+0000 7f2cab400700 1 -- 172.21.15.117:0/67403125 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2cac108b00 msgr2=0x7f2cac19f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:20:11.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:11.739+0000 7f2cab400700 1 --2- 172.21.15.117:0/67403125 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2cac108b00 0x7f2cac19f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:11.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:11.739+0000 7f2cab400700 1 -- 172.21.15.117:0/67403125 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2cac1094c0 msgr2=0x7f2cac19f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:20:11.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:11.739+0000 7f2cab400700 1 --2- 172.21.15.117:0/67403125 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2cac1094c0 0x7f2cac19f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:11.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:11.739+0000 7f2cab400700 1 -- 172.21.15.117:0/67403125 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cac1a41e0 con 0x7f2cac10eab0 2024-06-03T03:20:11.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:11.739+0000 7f2ca3e00700 1 --2- 172.21.15.117:0/67403125 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2cac1094c0 0x7f2cac19f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:20:11.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:11.739+0000 7f2cab400700 1 --2- 172.21.15.117:0/67403125 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2cac10eab0 0x7f2cac1a3aa0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2ca400b780 tx=0x7f2ca4004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:20:11.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:11.739+0000 7f2ca3400700 1 -- 172.21.15.117:0/67403125 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca4021070 con 0x7f2cac10eab0 2024-06-03T03:20:11.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:11.739+0000 7f2ca3400700 1 -- 172.21.15.117:0/67403125 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ca4007b80 con 0x7f2cac10eab0 2024-06-03T03:20:11.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:11.739+0000 7f2ca3400700 1 -- 172.21.15.117:0/67403125 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca40052a0 con 0x7f2cac10eab0 2024-06-03T03:20:11.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:11.739+0000 7f2cb1600700 1 -- 172.21.15.117:0/67403125 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cac1a4470 con 0x7f2cac10eab0 2024-06-03T03:20:11.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:11.740+0000 7f2cb1600700 1 -- 172.21.15.117:0/67403125 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cac1abd50 con 0x7f2cac10eab0 2024-06-03T03:20:11.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:11.741+0000 7f2cb1600700 1 -- 172.21.15.117:0/67403125 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cac069180 con 0x7f2cac10eab0 2024-06-03T03:20:11.743 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:11.743+0000 7f2ca3400700 1 -- 172.21.15.117:0/67403125 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f2ca4024070 con 0x7f2cac10eab0 2024-06-03T03:20:11.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:11.745+0000 7f2ca3400700 1 --2- 172.21.15.117:0/67403125 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f2c900a2ca0 0x7f2c900a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:20:11.746 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:11.745+0000 7f2ca3400700 1 -- 172.21.15.117:0/67403125 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8802+0+0 (secure 0 0 0) 0x7f2ca400f020 con 0x7f2cac10eab0 2024-06-03T03:20:11.746 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:11.745+0000 7f2caaa00700 1 --2- 172.21.15.117:0/67403125 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f2c900a2ca0 0x7f2c900a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:20:11.746 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:11.746+0000 7f2caaa00700 1 --2- 172.21.15.117:0/67403125 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f2c900a2ca0 0x7f2c900a5160 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c009680 tx=0x7f2c9c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:20:11.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:11.750+0000 7f2ca3400700 1 -- 172.21.15.117:0/67403125 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2ca402b020 con 0x7f2cac10eab0 2024-06-03T03:20:11.977 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:11.977+0000 7f2cb1600700 1 -- 172.21.15.117:0/67403125 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2cac02dca0 con 0x7f2cac10eab0 2024-06-03T03:20:11.983 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:11.982+0000 7f2ca3400700 1 -- 172.21.15.117:0/67403125 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+24187 (secure 0 0 0) 0x7f2ca4007d30 con 0x7f2cac10eab0 2024-06-03T03:20:11.984 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T03:20:11.984 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":95,"fsid":"a0e8473a-2156-11ef-bc9b-c7b262605968","created":"2024-06-03T03:10:08.446859+0000","modified":"2024-06-03T03:20:03.882941+0000","last_up_change":"2024-06-03T03:18:28.736904+0000","last_in_change":"2024-06-03T03:18:08.485711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-03T03:14:09.370986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-03T03:19:35.044683+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-03T03:19:40.097569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-03T03:19:55.394480+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8fefa089-9401-4a1c-b9fb-fe30c63e6a1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":4286644492},{"type":"v1","addr":"172.21.15.117:6803","nonce":4286644492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":4286644492},{"type":"v1","addr":"172.21.15.117:6805","nonce":4286644492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":4286644492},{"type":"v1","addr":"172.21.15.117:6809","nonce":4286644492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":4286644492},{"type":"v1","addr":"172.21.15.117:6807","nonce":4286644492}]},"public_addr":"172.21.15.117:6803/4286644492","cluster_addr":"172.21.15.117:6805/4286644492","heartbeat_back_addr":"172.21.15.117:6809/4286644492","heartbeat_front_addr":"172.21.15.117:6807/4286644492","state":["exists","up"]},{"osd":1,"uuid":"5b12b371-f155-4b67-a470-25cbe59a2999","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":262491158},{"type":"v1","addr":"172.21.15.151:6801","nonce":262491158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":262491158},{"type":"v1","addr":"172.21.15.151:6803","nonce":262491158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":262491158},{"type":"v1","addr":"172.21.15.151:6807","nonce":262491158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":262491158},{"type":"v1","addr":"172.21.15.151:6805","nonce":262491158}]},"public_addr":"172.21.15.151:6801/262491158","cluster_addr":"172.21.15.151:6803/262491158","heartbeat_back_addr":"172.21.15.151:6807/262491158","heartbeat_front_addr":"172.21.15.151:6805/262491158","state":["exists","up"]},{"osd":2,"uuid":"dbc17818-c54c-4ecf-b656-03578b95d8c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":3542453623},{"type":"v1","addr":"172.21.15.183:6801","nonce":3542453623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":3542453623},{"type":"v1","addr":"172.21.15.183:6803","nonce":3542453623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":3542453623},{"type":"v1","addr":"172.21.15.183:6807","nonce":3542453623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":3542453623},{"type":"v1","addr":"172.21.15.183:6805","nonce":3542453623}]},"public_addr":"172.21.15.183:6801/3542453623","cluster_addr":"172.21.15.183:6803/3542453623","heartbeat_back_addr":"172.21.15.183:6807/3542453623","heartbeat_front_addr":"172.21.15.183:6805/3542453623","state":["exists","up"]},{"osd":3,"uuid":"ca08d2ac-b1d6-4a49-9ae8-e3ec146899a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":589012315},{"type":"v1","addr":"172.21.15.117:6811","nonce":589012315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":589012315},{"type":"v1","addr":"172.21.15.117:6813","nonce":589012315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":589012315},{"type":"v1","addr":"172.21.15.117:6817","nonce":589012315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":589012315},{"type":"v1","addr":"172.21.15.117:6815","nonce":589012315}]},"public_addr":"172.21.15.117:6811/589012315","cluster_addr":"172.21.15.117:6813/589012315","heartbeat_back_addr":"172.21.15.117:6817/589012315","heartbeat_front_addr":"172.21.15.117:6815/589012315","state":["exists","up"]},{"osd":4,"uuid":"810d6c7a-023f-4930-82af-bc2bbfd647da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":1239048647},{"type":"v1","addr":"172.21.15.151:6809","nonce":1239048647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":1239048647},{"type":"v1","addr":"172.21.15.151:6811","nonce":1239048647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":1239048647},{"type":"v1","addr":"172.21.15.151:6815","nonce":1239048647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":1239048647},{"type":"v1","addr":"172.21.15.151:6813","nonce":1239048647}]},"public_addr":"172.21.15.151:6809/1239048647","cluster_addr":"172.21.15.151:6811/1239048647","heartbeat_back_addr":"172.21.15.151:6815/1239048647","heartbeat_front_addr":"172.21.15.151:6813/1239048647","state":["exists","up"]},{"osd":5,"uuid":"8ab6a5f0-34bb-48d8-9471-a7dc836e486e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":2985874686},{"type":"v1","addr":"172.21.15.183:6809","nonce":2985874686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":2985874686},{"type":"v1","addr":"172.21.15.183:6811","nonce":2985874686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":2985874686},{"type":"v1","addr":"172.21.15.183:6815","nonce":2985874686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":2985874686},{"type":"v1","addr":"172.21.15.183:6813","nonce":2985874686}]},"public_addr":"172.21.15.183:6809/2985874686","cluster_addr":"172.21.15.183:6811/2985874686","heartbeat_back_addr":"172.21.15.183:6815/2985874686","heartbeat_front_addr":"172.21.15.183:6813/2985874686","state":["exists","up"]},{"osd":6,"uuid":"78d15739-e9e0-4f19-ae5f-ee8527a5b527","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4250583193},{"type":"v1","addr":"172.21.15.117:6819","nonce":4250583193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":4250583193},{"type":"v1","addr":"172.21.15.117:6821","nonce":4250583193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":4250583193},{"type":"v1","addr":"172.21.15.117:6825","nonce":4250583193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4250583193},{"type":"v1","addr":"172.21.15.117:6823","nonce":4250583193}]},"public_addr":"172.21.15.117:6819/4250583193","cluster_addr":"172.21.15.117:6821/4250583193","heartbeat_back_addr":"172.21.15.117:6825/4250583193","heartbeat_front_addr":"172.21.15.117:6823/4250583193","state":["exists","up"]},{"osd":7,"uuid":"3724fe1f-f4d7-443b-aea8-ff3ab02bfdc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":386094645},{"type":"v1","addr":"172.21.15.151:6817","nonce":386094645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":386094645},{"type":"v1","addr":"172.21.15.151:6819","nonce":386094645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":386094645},{"type":"v1","addr":"172.21.15.151:6823","nonce":386094645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":386094645},{"type":"v1","addr":"172.21.15.151:6821","nonce":386094645}]},"public_addr":"172.21.15.151:6817/386094645","cluster_addr":"172.21.15.151:6819/386094645","heartbeat_back_addr":"172.21.15.151:6823/386094645","heartbeat_front_addr":"172.21.15.151:6821/386094645","state":["exists","up"]},{"osd":8,"uuid":"006ac092-f015-448e-a0c0-f7b0c4597a18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1829748222},{"type":"v1","addr":"172.21.15.183:6817","nonce":1829748222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1829748222},{"type":"v1","addr":"172.21.15.183:6819","nonce":1829748222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1829748222},{"type":"v1","addr":"172.21.15.183:6823","nonce":1829748222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1829748222},{"type":"v1","addr":"172.21.15.183:6821","nonce":1829748222}]},"public_addr":"172.21.15.183:6817/1829748222","cluster_addr":"172.21.15.183:6819/1829748222","heartbeat_back_addr":"172.21.15.183:6823/1829748222","heartbeat_front_addr":"172.21.15.183:6821/1829748222","state":["exists","up"]},{"osd":9,"uuid":"00a970e6-2502-459e-975c-badac93c9bd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2538315141},{"type":"v1","addr":"172.21.15.117:6827","nonce":2538315141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2538315141},{"type":"v1","addr":"172.21.15.117:6829","nonce":2538315141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2538315141},{"type":"v1","addr":"172.21.15.117:6833","nonce":2538315141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2538315141},{"type":"v1","addr":"172.21.15.117:6831","nonce":2538315141}]},"public_addr":"172.21.15.117:6827/2538315141","cluster_addr":"172.21.15.117:6829/2538315141","heartbeat_back_addr":"172.21.15.117:6833/2538315141","heartbeat_front_addr":"172.21.15.117:6831/2538315141","state":["exists","up"]},{"osd":10,"uuid":"ba2333cd-32a3-41ef-9ea5-287c707f512b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":3801045133},{"type":"v1","addr":"172.21.15.151:6825","nonce":3801045133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3801045133},{"type":"v1","addr":"172.21.15.151:6827","nonce":3801045133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3801045133},{"type":"v1","addr":"172.21.15.151:6831","nonce":3801045133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3801045133},{"type":"v1","addr":"172.21.15.151:6829","nonce":3801045133}]},"public_addr":"172.21.15.151:6825/3801045133","cluster_addr":"172.21.15.151:6827/3801045133","heartbeat_back_addr":"172.21.15.151:6831/3801045133","heartbeat_front_addr":"172.21.15.151:6829/3801045133","state":["exists","up"]},{"osd":11,"uuid":"1556d654-e6fa-42c1-b399-aa53f6837a1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3513899696},{"type":"v1","addr":"172.21.15.183:6825","nonce":3513899696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3513899696},{"type":"v1","addr":"172.21.15.183:6827","nonce":3513899696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3513899696},{"type":"v1","addr":"172.21.15.183:6831","nonce":3513899696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3513899696},{"type":"v1","addr":"172.21.15.183:6829","nonce":3513899696}]},"public_addr":"172.21.15.183:6825/3513899696","cluster_addr":"172.21.15.183:6827/3513899696","heartbeat_back_addr":"172.21.15.183:6831/3513899696","heartbeat_front_addr":"172.21.15.183:6829/3513899696","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:13:17.174041+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:13:42.237001+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:14:06.273315+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:14:39.912564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:15:06.063202+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:15:30.424818+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:16:05.623554+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:16:32.926393+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:16:58.165899+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:17:32.897477+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:18:00.720866+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:18:26.474441+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.117:0/186309008":"2024-06-04T03:11:24.655516+0000","172.21.15.117:6801/2142313590":"2024-06-04T03:11:24.655516+0000","172.21.15.117:6800/2142313590":"2024-06-04T03:11:24.655516+0000","172.21.15.117:0/2399725156":"2024-06-04T03:11:24.655516+0000","172.21.15.117:0/1453232061":"2024-06-04T03:10:50.552899+0000","172.21.15.117:0/4281674974":"2024-06-04T03:10:50.552899+0000","172.21.15.117:6800/2424554985":"2024-06-04T03:10:50.552899+0000","172.21.15.117:0/3946365225":"2024-06-04T03:11:24.655516+0000","172.21.15.117:0/1146162354":"2024-06-04T03:10:50.552899+0000","172.21.15.117:6801/2424554985":"2024-06-04T03:10:50.552899+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-03T03:20:11.986 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:11.986+0000 7f2cb1600700 1 -- 172.21.15.117:0/67403125 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f2c900a2ca0 msgr2=0x7f2c900a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:20:11.986 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:11.986+0000 7f2cb1600700 1 --2- 172.21.15.117:0/67403125 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f2c900a2ca0 0x7f2c900a5160 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c009680 tx=0x7f2c9c009260 comp rx=0 tx=0).stop 2024-06-03T03:20:11.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:11.986+0000 7f2cb1600700 1 -- 172.21.15.117:0/67403125 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2cac10eab0 msgr2=0x7f2cac1a3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:20:11.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:11.986+0000 7f2cb1600700 1 --2- 172.21.15.117:0/67403125 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2cac10eab0 0x7f2cac1a3aa0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2ca400b780 tx=0x7f2ca4004d00 comp rx=0 tx=0).stop 2024-06-03T03:20:11.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:11.986+0000 7f2cb1600700 1 -- 172.21.15.117:0/67403125 shutdown_connections 2024-06-03T03:20:11.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:11.986+0000 7f2cb1600700 1 --2- 172.21.15.117:0/67403125 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f2c900a2ca0 0x7f2c900a5160 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:11.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:11.986+0000 7f2cb1600700 1 --2- 172.21.15.117:0/67403125 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2cac10eab0 0x7f2cac1a3aa0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:11.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:11.986+0000 7f2cb1600700 1 --2- 172.21.15.117:0/67403125 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2cac108b00 0x7f2cac19f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:11.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:11.986+0000 7f2cb1600700 1 --2- 172.21.15.117:0/67403125 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2cac1094c0 0x7f2cac19f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:11.988 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:11.986+0000 7f2cb1600700 1 -- 172.21.15.117:0/67403125 >> 172.21.15.117:0/67403125 conn(0x7f2cac075c60 msgr2=0x7f2cac0761a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:20:11.988 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:11.986+0000 7f2cb1600700 1 -- 172.21.15.117:0/67403125 shutdown_connections 2024-06-03T03:20:11.988 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:11.986+0000 7f2cb1600700 1 -- 172.21.15.117:0/67403125 wait complete. 2024-06-03T03:20:12.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:11 smithi183 ceph-mon[37326]: pgmap v348: 129 pgs: 129 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:20:12.393 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:11 smithi117 ceph-mon[31649]: pgmap v348: 129 pgs: 129 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:20:12.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:11 smithi151 ceph-mon[37043]: pgmap v348: 129 pgs: 129 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:20:12.823 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-03T03:20:13.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:12 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/67403125' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T03:20:13.278 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:20:13.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:12 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/67403125' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T03:20:13.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:12 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/67403125' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T03:20:14.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:13 smithi117 ceph-mon[31649]: pgmap v349: 129 pgs: 129 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:20:14.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:13 smithi151 ceph-mon[37043]: pgmap v349: 129 pgs: 129 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:20:14.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:13 smithi183 ceph-mon[37326]: pgmap v349: 129 pgs: 129 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:20:15.615 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:15.613+0000 7fa64fa00700 1 -- 172.21.15.117:0/1001014977 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa648109480 msgr2=0x7fa648115990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:20:15.615 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:15.613+0000 7fa64fa00700 1 --2- 172.21.15.117:0/1001014977 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa648109480 0x7fa648115990 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa64000b0a0 tx=0x7fa64000b3b0 comp rx=0 tx=0).stop 2024-06-03T03:20:15.615 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:15.613+0000 7fa64fa00700 1 -- 172.21.15.117:0/1001014977 shutdown_connections 2024-06-03T03:20:15.615 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:15.613+0000 7fa64fa00700 1 --2- 172.21.15.117:0/1001014977 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa648109480 0x7fa648115990 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:15.615 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:15.613+0000 7fa64fa00700 1 --2- 172.21.15.117:0/1001014977 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa64810eac0 0x7fa64810eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:15.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:15.613+0000 7fa64fa00700 1 --2- 172.21.15.117:0/1001014977 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa648108ac0 0x7fa648108f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:15.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:15.613+0000 7fa64fa00700 1 -- 172.21.15.117:0/1001014977 >> 172.21.15.117:0/1001014977 conn(0x7fa648075c60 msgr2=0x7fa6481072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:20:15.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:15.613+0000 7fa64fa00700 1 -- 172.21.15.117:0/1001014977 shutdown_connections 2024-06-03T03:20:15.616 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:15 smithi117 ceph-mon[31649]: pgmap v350: 129 pgs: 129 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:20:15.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:15.613+0000 7fa64fa00700 1 -- 172.21.15.117:0/1001014977 wait complete. 2024-06-03T03:20:15.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:15.615+0000 7fa64fa00700 1 Processor -- start 2024-06-03T03:20:15.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:15.615+0000 7fa64fa00700 1 -- start start 2024-06-03T03:20:15.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:15.615+0000 7fa64fa00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa648108ac0 0x7fa64819f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:20:15.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:15.615+0000 7fa64fa00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa648109480 0x7fa64819f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:20:15.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:15.615+0000 7fa64fa00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa64810eac0 0x7fa6481a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:20:15.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:15.615+0000 7fa64fa00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa648118e50 con 0x7fa64810eac0 2024-06-03T03:20:15.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:15.615+0000 7fa64fa00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa648118cd0 con 0x7fa648109480 2024-06-03T03:20:15.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:15.615+0000 7fa64fa00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa648118fd0 con 0x7fa648108ac0 2024-06-03T03:20:15.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:15.616+0000 7fa64d000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa648108ac0 0x7fa64819f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:20:15.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:15.616+0000 7fa64d000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa648108ac0 0x7fa64819f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:40384/0 (socket says 172.21.15.117:40384) 2024-06-03T03:20:15.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:15.616+0000 7fa64d000700 1 -- 172.21.15.117:0/1373668555 learned_addr learned my addr 172.21.15.117:0/1373668555 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:20:15.620 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:15.616+0000 7fa64da00700 1 --2- 172.21.15.117:0/1373668555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa64810eac0 0x7fa6481a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:20:15.620 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:15.616+0000 7fa647e00700 1 --2- 172.21.15.117:0/1373668555 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa648109480 0x7fa64819f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:20:15.620 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:15.616+0000 7fa64da00700 1 -- 172.21.15.117:0/1373668555 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa648108ac0 msgr2=0x7fa64819f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:20:15.620 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:15.616+0000 7fa64da00700 1 --2- 172.21.15.117:0/1373668555 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa648108ac0 0x7fa64819f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:15.620 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:15.616+0000 7fa64da00700 1 -- 172.21.15.117:0/1373668555 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa648109480 msgr2=0x7fa64819f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:20:15.620 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:15.616+0000 7fa64da00700 1 --2- 172.21.15.117:0/1373668555 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa648109480 0x7fa64819f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:15.620 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:15.616+0000 7fa64da00700 1 -- 172.21.15.117:0/1373668555 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6481a41a0 con 0x7fa64810eac0 2024-06-03T03:20:15.621 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:15.617+0000 7fa64da00700 1 --2- 172.21.15.117:0/1373668555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa64810eac0 0x7fa6481a3a60 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa64000b0a0 tx=0x7fa640004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:20:15.621 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:15.617+0000 7fa645600700 1 -- 172.21.15.117:0/1373668555 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa640021070 con 0x7fa64810eac0 2024-06-03T03:20:15.621 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:15.617+0000 7fa645600700 1 -- 172.21.15.117:0/1373668555 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa640007b80 con 0x7fa64810eac0 2024-06-03T03:20:15.621 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:15.617+0000 7fa64fa00700 1 -- 172.21.15.117:0/1373668555 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6481a4430 con 0x7fa64810eac0 2024-06-03T03:20:15.621 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:15.617+0000 7fa645600700 1 -- 172.21.15.117:0/1373668555 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa640005040 con 0x7fa64810eac0 2024-06-03T03:20:15.621 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:15.617+0000 7fa64fa00700 1 -- 172.21.15.117:0/1373668555 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6481abd10 con 0x7fa64810eac0 2024-06-03T03:20:15.622 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:15.619+0000 7fa64fa00700 1 -- 172.21.15.117:0/1373668555 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa648069180 con 0x7fa64810eac0 2024-06-03T03:20:15.622 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:15.620+0000 7fa645600700 1 -- 172.21.15.117:0/1373668555 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fa640024070 con 0x7fa64810eac0 2024-06-03T03:20:15.623 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:15.622+0000 7fa645600700 1 --2- 172.21.15.117:0/1373668555 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fa6300a2c20 0x7fa6300a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:20:15.623 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:15.623+0000 7fa645600700 1 -- 172.21.15.117:0/1373668555 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8802+0+0 (secure 0 0 0) 0x7fa6400d0140 con 0x7fa64810eac0 2024-06-03T03:20:15.623 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:15.623+0000 7fa64d000700 1 --2- 172.21.15.117:0/1373668555 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fa6300a2c20 0x7fa6300a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:20:15.624 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:15.623+0000 7fa64d000700 1 --2- 172.21.15.117:0/1373668555 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fa6300a2c20 0x7fa6300a50e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa63c009ca0 tx=0x7fa63c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:20:15.628 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:15.627+0000 7fa645600700 1 -- 172.21.15.117:0/1373668555 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa640007d30 con 0x7fa64810eac0 2024-06-03T03:20:15.851 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:15.849+0000 7fa64fa00700 1 -- 172.21.15.117:0/1373668555 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa6481a04f0 con 0x7fa64810eac0 2024-06-03T03:20:15.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:15.851+0000 7fa645600700 1 -- 172.21.15.117:0/1373668555 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+24187 (secure 0 0 0) 0x7fa64009ce90 con 0x7fa64810eac0 2024-06-03T03:20:15.853 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T03:20:15.854 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":95,"fsid":"a0e8473a-2156-11ef-bc9b-c7b262605968","created":"2024-06-03T03:10:08.446859+0000","modified":"2024-06-03T03:20:03.882941+0000","last_up_change":"2024-06-03T03:18:28.736904+0000","last_in_change":"2024-06-03T03:18:08.485711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-03T03:14:09.370986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-03T03:19:35.044683+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-03T03:19:40.097569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-03T03:19:55.394480+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8fefa089-9401-4a1c-b9fb-fe30c63e6a1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":4286644492},{"type":"v1","addr":"172.21.15.117:6803","nonce":4286644492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":4286644492},{"type":"v1","addr":"172.21.15.117:6805","nonce":4286644492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":4286644492},{"type":"v1","addr":"172.21.15.117:6809","nonce":4286644492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":4286644492},{"type":"v1","addr":"172.21.15.117:6807","nonce":4286644492}]},"public_addr":"172.21.15.117:6803/4286644492","cluster_addr":"172.21.15.117:6805/4286644492","heartbeat_back_addr":"172.21.15.117:6809/4286644492","heartbeat_front_addr":"172.21.15.117:6807/4286644492","state":["exists","up"]},{"osd":1,"uuid":"5b12b371-f155-4b67-a470-25cbe59a2999","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":262491158},{"type":"v1","addr":"172.21.15.151:6801","nonce":262491158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":262491158},{"type":"v1","addr":"172.21.15.151:6803","nonce":262491158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":262491158},{"type":"v1","addr":"172.21.15.151:6807","nonce":262491158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":262491158},{"type":"v1","addr":"172.21.15.151:6805","nonce":262491158}]},"public_addr":"172.21.15.151:6801/262491158","cluster_addr":"172.21.15.151:6803/262491158","heartbeat_back_addr":"172.21.15.151:6807/262491158","heartbeat_front_addr":"172.21.15.151:6805/262491158","state":["exists","up"]},{"osd":2,"uuid":"dbc17818-c54c-4ecf-b656-03578b95d8c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":3542453623},{"type":"v1","addr":"172.21.15.183:6801","nonce":3542453623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":3542453623},{"type":"v1","addr":"172.21.15.183:6803","nonce":3542453623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":3542453623},{"type":"v1","addr":"172.21.15.183:6807","nonce":3542453623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":3542453623},{"type":"v1","addr":"172.21.15.183:6805","nonce":3542453623}]},"public_addr":"172.21.15.183:6801/3542453623","cluster_addr":"172.21.15.183:6803/3542453623","heartbeat_back_addr":"172.21.15.183:6807/3542453623","heartbeat_front_addr":"172.21.15.183:6805/3542453623","state":["exists","up"]},{"osd":3,"uuid":"ca08d2ac-b1d6-4a49-9ae8-e3ec146899a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":589012315},{"type":"v1","addr":"172.21.15.117:6811","nonce":589012315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":589012315},{"type":"v1","addr":"172.21.15.117:6813","nonce":589012315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":589012315},{"type":"v1","addr":"172.21.15.117:6817","nonce":589012315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":589012315},{"type":"v1","addr":"172.21.15.117:6815","nonce":589012315}]},"public_addr":"172.21.15.117:6811/589012315","cluster_addr":"172.21.15.117:6813/589012315","heartbeat_back_addr":"172.21.15.117:6817/589012315","heartbeat_front_addr":"172.21.15.117:6815/589012315","state":["exists","up"]},{"osd":4,"uuid":"810d6c7a-023f-4930-82af-bc2bbfd647da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":1239048647},{"type":"v1","addr":"172.21.15.151:6809","nonce":1239048647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":1239048647},{"type":"v1","addr":"172.21.15.151:6811","nonce":1239048647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":1239048647},{"type":"v1","addr":"172.21.15.151:6815","nonce":1239048647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":1239048647},{"type":"v1","addr":"172.21.15.151:6813","nonce":1239048647}]},"public_addr":"172.21.15.151:6809/1239048647","cluster_addr":"172.21.15.151:6811/1239048647","heartbeat_back_addr":"172.21.15.151:6815/1239048647","heartbeat_front_addr":"172.21.15.151:6813/1239048647","state":["exists","up"]},{"osd":5,"uuid":"8ab6a5f0-34bb-48d8-9471-a7dc836e486e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":2985874686},{"type":"v1","addr":"172.21.15.183:6809","nonce":2985874686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":2985874686},{"type":"v1","addr":"172.21.15.183:6811","nonce":2985874686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":2985874686},{"type":"v1","addr":"172.21.15.183:6815","nonce":2985874686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":2985874686},{"type":"v1","addr":"172.21.15.183:6813","nonce":2985874686}]},"public_addr":"172.21.15.183:6809/2985874686","cluster_addr":"172.21.15.183:6811/2985874686","heartbeat_back_addr":"172.21.15.183:6815/2985874686","heartbeat_front_addr":"172.21.15.183:6813/2985874686","state":["exists","up"]},{"osd":6,"uuid":"78d15739-e9e0-4f19-ae5f-ee8527a5b527","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4250583193},{"type":"v1","addr":"172.21.15.117:6819","nonce":4250583193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":4250583193},{"type":"v1","addr":"172.21.15.117:6821","nonce":4250583193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":4250583193},{"type":"v1","addr":"172.21.15.117:6825","nonce":4250583193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4250583193},{"type":"v1","addr":"172.21.15.117:6823","nonce":4250583193}]},"public_addr":"172.21.15.117:6819/4250583193","cluster_addr":"172.21.15.117:6821/4250583193","heartbeat_back_addr":"172.21.15.117:6825/4250583193","heartbeat_front_addr":"172.21.15.117:6823/4250583193","state":["exists","up"]},{"osd":7,"uuid":"3724fe1f-f4d7-443b-aea8-ff3ab02bfdc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":386094645},{"type":"v1","addr":"172.21.15.151:6817","nonce":386094645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":386094645},{"type":"v1","addr":"172.21.15.151:6819","nonce":386094645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":386094645},{"type":"v1","addr":"172.21.15.151:6823","nonce":386094645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":386094645},{"type":"v1","addr":"172.21.15.151:6821","nonce":386094645}]},"public_addr":"172.21.15.151:6817/386094645","cluster_addr":"172.21.15.151:6819/386094645","heartbeat_back_addr":"172.21.15.151:6823/386094645","heartbeat_front_addr":"172.21.15.151:6821/386094645","state":["exists","up"]},{"osd":8,"uuid":"006ac092-f015-448e-a0c0-f7b0c4597a18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1829748222},{"type":"v1","addr":"172.21.15.183:6817","nonce":1829748222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1829748222},{"type":"v1","addr":"172.21.15.183:6819","nonce":1829748222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1829748222},{"type":"v1","addr":"172.21.15.183:6823","nonce":1829748222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1829748222},{"type":"v1","addr":"172.21.15.183:6821","nonce":1829748222}]},"public_addr":"172.21.15.183:6817/1829748222","cluster_addr":"172.21.15.183:6819/1829748222","heartbeat_back_addr":"172.21.15.183:6823/1829748222","heartbeat_front_addr":"172.21.15.183:6821/1829748222","state":["exists","up"]},{"osd":9,"uuid":"00a970e6-2502-459e-975c-badac93c9bd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2538315141},{"type":"v1","addr":"172.21.15.117:6827","nonce":2538315141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2538315141},{"type":"v1","addr":"172.21.15.117:6829","nonce":2538315141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2538315141},{"type":"v1","addr":"172.21.15.117:6833","nonce":2538315141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2538315141},{"type":"v1","addr":"172.21.15.117:6831","nonce":2538315141}]},"public_addr":"172.21.15.117:6827/2538315141","cluster_addr":"172.21.15.117:6829/2538315141","heartbeat_back_addr":"172.21.15.117:6833/2538315141","heartbeat_front_addr":"172.21.15.117:6831/2538315141","state":["exists","up"]},{"osd":10,"uuid":"ba2333cd-32a3-41ef-9ea5-287c707f512b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":3801045133},{"type":"v1","addr":"172.21.15.151:6825","nonce":3801045133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3801045133},{"type":"v1","addr":"172.21.15.151:6827","nonce":3801045133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3801045133},{"type":"v1","addr":"172.21.15.151:6831","nonce":3801045133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3801045133},{"type":"v1","addr":"172.21.15.151:6829","nonce":3801045133}]},"public_addr":"172.21.15.151:6825/3801045133","cluster_addr":"172.21.15.151:6827/3801045133","heartbeat_back_addr":"172.21.15.151:6831/3801045133","heartbeat_front_addr":"172.21.15.151:6829/3801045133","state":["exists","up"]},{"osd":11,"uuid":"1556d654-e6fa-42c1-b399-aa53f6837a1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3513899696},{"type":"v1","addr":"172.21.15.183:6825","nonce":3513899696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3513899696},{"type":"v1","addr":"172.21.15.183:6827","nonce":3513899696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3513899696},{"type":"v1","addr":"172.21.15.183:6831","nonce":3513899696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3513899696},{"type":"v1","addr":"172.21.15.183:6829","nonce":3513899696}]},"public_addr":"172.21.15.183:6825/3513899696","cluster_addr":"172.21.15.183:6827/3513899696","heartbeat_back_addr":"172.21.15.183:6831/3513899696","heartbeat_front_addr":"172.21.15.183:6829/3513899696","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:13:17.174041+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:13:42.237001+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:14:06.273315+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:14:39.912564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:15:06.063202+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:15:30.424818+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:16:05.623554+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:16:32.926393+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:16:58.165899+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:17:32.897477+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:18:00.720866+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:18:26.474441+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.117:0/186309008":"2024-06-04T03:11:24.655516+0000","172.21.15.117:6801/2142313590":"2024-06-04T03:11:24.655516+0000","172.21.15.117:6800/2142313590":"2024-06-04T03:11:24.655516+0000","172.21.15.117:0/2399725156":"2024-06-04T03:11:24.655516+0000","172.21.15.117:0/1453232061":"2024-06-04T03:10:50.552899+0000","172.21.15.117:0/4281674974":"2024-06-04T03:10:50.552899+0000","172.21.15.117:6800/2424554985":"2024-06-04T03:10:50.552899+0000","172.21.15.117:0/3946365225":"2024-06-04T03:11:24.655516+0000","172.21.15.117:0/1146162354":"2024-06-04T03:10:50.552899+0000","172.21.15.117:6801/2424554985":"2024-06-04T03:10:50.552899+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-03T03:20:15.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:15.855+0000 7fa64fa00700 1 -- 172.21.15.117:0/1373668555 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fa6300a2c20 msgr2=0x7fa6300a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:20:15.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:15.855+0000 7fa64fa00700 1 --2- 172.21.15.117:0/1373668555 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fa6300a2c20 0x7fa6300a50e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa63c009ca0 tx=0x7fa63c009260 comp rx=0 tx=0).stop 2024-06-03T03:20:15.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:15.855+0000 7fa64fa00700 1 -- 172.21.15.117:0/1373668555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa64810eac0 msgr2=0x7fa6481a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:20:15.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:15.855+0000 7fa64fa00700 1 --2- 172.21.15.117:0/1373668555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa64810eac0 0x7fa6481a3a60 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa64000b0a0 tx=0x7fa640004d00 comp rx=0 tx=0).stop 2024-06-03T03:20:15.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:15.855+0000 7fa64fa00700 1 -- 172.21.15.117:0/1373668555 shutdown_connections 2024-06-03T03:20:15.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:15.855+0000 7fa64fa00700 1 --2- 172.21.15.117:0/1373668555 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fa6300a2c20 0x7fa6300a50e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:15.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:15.855+0000 7fa64fa00700 1 --2- 172.21.15.117:0/1373668555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa64810eac0 0x7fa6481a3a60 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:15.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:15.855+0000 7fa64fa00700 1 --2- 172.21.15.117:0/1373668555 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa648108ac0 0x7fa64819f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:15.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:15.855+0000 7fa64fa00700 1 --2- 172.21.15.117:0/1373668555 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa648109480 0x7fa64819f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:15.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:15.855+0000 7fa64fa00700 1 -- 172.21.15.117:0/1373668555 >> 172.21.15.117:0/1373668555 conn(0x7fa648075c60 msgr2=0x7fa648076220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:20:15.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:15.855+0000 7fa64fa00700 1 -- 172.21.15.117:0/1373668555 shutdown_connections 2024-06-03T03:20:15.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:15.855+0000 7fa64fa00700 1 -- 172.21.15.117:0/1373668555 wait complete. 2024-06-03T03:20:15.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:15 smithi151 ceph-mon[37043]: pgmap v350: 129 pgs: 129 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:20:15.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:15 smithi183 ceph-mon[37326]: pgmap v350: 129 pgs: 129 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:20:16.657 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:16 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/1373668555' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T03:20:16.748 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-03T03:20:16.906 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:16 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/1373668555' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T03:20:16.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:16 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/1373668555' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T03:20:17.192 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:20:17.695 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:17 smithi117 ceph-mon[31649]: pgmap v351: 129 pgs: 129 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:20:17.906 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:17 smithi151 ceph-mon[37043]: pgmap v351: 129 pgs: 129 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:20:17.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:17 smithi183 ceph-mon[37326]: pgmap v351: 129 pgs: 129 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:20:19.529 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:19.528+0000 7f3b61800700 1 -- 172.21.15.117:0/3005557650 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3b5c102740 msgr2=0x7f3b5c102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:20:19.530 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:19.528+0000 7f3b61800700 1 --2- 172.21.15.117:0/3005557650 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3b5c102740 0x7f3b5c102b20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c009f90 tx=0x7f3b4c00b330 comp rx=0 tx=0).stop 2024-06-03T03:20:19.530 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:19.528+0000 7f3b61800700 1 -- 172.21.15.117:0/3005557650 shutdown_connections 2024-06-03T03:20:19.530 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:19.528+0000 7f3b61800700 1 --2- 172.21.15.117:0/3005557650 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b5c107840 0x7f3b5c10f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:19.530 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:19.528+0000 7f3b61800700 1 --2- 172.21.15.117:0/3005557650 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3b5c102740 0x7f3b5c102b20 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:19.530 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:19.528+0000 7f3b61800700 1 --2- 172.21.15.117:0/3005557650 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b5c1030f0 0x7f3b5c107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:19.530 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:19.528+0000 7f3b61800700 1 -- 172.21.15.117:0/3005557650 >> 172.21.15.117:0/3005557650 conn(0x7f3b5c0795f0 msgr2=0x7f3b5c079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:20:19.530 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:19.528+0000 7f3b61800700 1 -- 172.21.15.117:0/3005557650 shutdown_connections 2024-06-03T03:20:19.530 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:19.528+0000 7f3b61800700 1 -- 172.21.15.117:0/3005557650 wait complete. 2024-06-03T03:20:19.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:19.529+0000 7f3b61800700 1 Processor -- start 2024-06-03T03:20:19.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:19.530+0000 7f3b61800700 1 -- start start 2024-06-03T03:20:19.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:19.530+0000 7f3b61800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b5c102740 0x7f3b5c19ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:20:19.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:19.530+0000 7f3b61800700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3b5c1030f0 0x7f3b5c19b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:20:19.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:19.530+0000 7f3b61800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b5c107840 0x7f3b5c19f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:20:19.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:19.530+0000 7f3b61800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3b5c1127d0 con 0x7f3b5c102740 2024-06-03T03:20:19.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:19.530+0000 7f3b61800700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3b5c112650 con 0x7f3b5c107840 2024-06-03T03:20:19.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:19.530+0000 7f3b61800700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3b5c112950 con 0x7f3b5c1030f0 2024-06-03T03:20:19.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:19.530+0000 7f3b5aa00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b5c102740 0x7f3b5c19ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:20:19.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:19.530+0000 7f3b5aa00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b5c102740 0x7f3b5c19ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:46300/0 (socket says 172.21.15.117:46300) 2024-06-03T03:20:19.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:19.530+0000 7f3b5aa00700 1 -- 172.21.15.117:0/364894155 learned_addr learned my addr 172.21.15.117:0/364894155 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:20:19.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:19.530+0000 7f3b5b400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b5c107840 0x7f3b5c19f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:20:19.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:19.530+0000 7f3b52000700 1 --2- 172.21.15.117:0/364894155 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3b5c1030f0 0x7f3b5c19b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:20:19.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:19.531+0000 7f3b5aa00700 1 -- 172.21.15.117:0/364894155 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3b5c1030f0 msgr2=0x7f3b5c19b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:20:19.533 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:19.531+0000 7f3b5aa00700 1 --2- 172.21.15.117:0/364894155 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3b5c1030f0 0x7f3b5c19b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:19.533 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:19.531+0000 7f3b5aa00700 1 -- 172.21.15.117:0/364894155 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b5c107840 msgr2=0x7f3b5c19f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:20:19.533 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:19.531+0000 7f3b5aa00700 1 --2- 172.21.15.117:0/364894155 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b5c107840 0x7f3b5c19f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:19.533 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:19.531+0000 7f3b5aa00700 1 -- 172.21.15.117:0/364894155 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b5c19fe20 con 0x7f3b5c102740 2024-06-03T03:20:19.533 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:19.531+0000 7f3b5b400700 1 --2- 172.21.15.117:0/364894155 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b5c107840 0x7f3b5c19f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:20:19.533 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:19.531+0000 7f3b5aa00700 1 --2- 172.21.15.117:0/364894155 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b5c102740 0x7f3b5c19ae10 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c00d010 tx=0x7f3b4c00bca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:20:19.534 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:19.531+0000 7f3b53e00700 1 -- 172.21.15.117:0/364894155 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b4c021070 con 0x7f3b5c102740 2024-06-03T03:20:19.534 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:19.531+0000 7f3b53e00700 1 -- 172.21.15.117:0/364894155 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b4c01d450 con 0x7f3b5c102740 2024-06-03T03:20:19.534 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:19.531+0000 7f3b61800700 1 -- 172.21.15.117:0/364894155 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b5c1a00b0 con 0x7f3b5c102740 2024-06-03T03:20:19.534 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:19.531+0000 7f3b53e00700 1 -- 172.21.15.117:0/364894155 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b4c0052a0 con 0x7f3b5c102740 2024-06-03T03:20:19.534 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:19.532+0000 7f3b61800700 1 -- 172.21.15.117:0/364894155 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b5c1a7990 con 0x7f3b5c102740 2024-06-03T03:20:19.534 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:19.533+0000 7f3b61800700 1 -- 172.21.15.117:0/364894155 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b5c069180 con 0x7f3b5c102740 2024-06-03T03:20:19.535 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:19.535+0000 7f3b53e00700 1 -- 172.21.15.117:0/364894155 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f3b4c024070 con 0x7f3b5c102740 2024-06-03T03:20:19.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:19.536+0000 7f3b53e00700 1 --2- 172.21.15.117:0/364894155 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f3b400a2c20 0x7f3b400a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:20:19.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:19.536+0000 7f3b53e00700 1 -- 172.21.15.117:0/364894155 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8802+0+0 (secure 0 0 0) 0x7f3b4c00f020 con 0x7f3b5c102740 2024-06-03T03:20:19.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:19.536+0000 7f3b52000700 1 --2- 172.21.15.117:0/364894155 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f3b400a2c20 0x7f3b400a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:20:19.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:19.537+0000 7f3b52000700 1 --2- 172.21.15.117:0/364894155 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f3b400a2c20 0x7f3b400a50e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3b44009750 tx=0x7f3b44009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:20:19.540 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:19.539+0000 7f3b53e00700 1 -- 172.21.15.117:0/364894155 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3b4c02b020 con 0x7f3b5c102740 2024-06-03T03:20:19.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:19.812+0000 7f3b61800700 1 -- 172.21.15.117:0/364894155 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3b5c19c170 con 0x7f3b5c102740 2024-06-03T03:20:19.816 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:19.814+0000 7f3b53e00700 1 -- 172.21.15.117:0/364894155 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11819 (secure 0 0 0) 0x7f3b5c19c170 con 0x7f3b5c102740 2024-06-03T03:20:19.817 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T03:20:19.817 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3353001119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3353001119},{"type":"v1","addr":"172.21.15.117:6837","nonce":3353001119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14388,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6839/405782304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":405782304},{"type":"v1","addr":"172.21.15.117:6839","nonce":405782304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3355662416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3355662416},{"type":"v1","addr":"172.21.15.151:6833","nonce":3355662416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/2429725085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2429725085},{"type":"v1","addr":"172.21.15.183:6833","nonce":2429725085}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/31153579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":31153579},{"type":"v1","addr":"172.21.15.151:6835","nonce":31153579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24466,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/2884460088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2884460088},{"type":"v1","addr":"172.21.15.117:6835","nonce":2884460088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24479,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/2392407848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":2392407848},{"type":"v1","addr":"172.21.15.151:6837","nonce":2392407848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24487,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/1009004320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1009004320},{"type":"v1","addr":"172.21.15.183:6835","nonce":1009004320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24499,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/2697518526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":2697518526},{"type":"v1","addr":"172.21.15.183:6837","nonce":2697518526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24503,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6841/1212242967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6840","nonce":1212242967},{"type":"v1","addr":"172.21.15.117:6841","nonce":1212242967}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24512,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/3623444616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":3623444616},{"type":"v1","addr":"172.21.15.151:6839","nonce":3623444616}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-06-03T03:19:44.629822+0000","modified":"2024-06-03T03:20:03.883250+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24517},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24517":{"gid":24517,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":10,"addr":"172.21.15.183:6839/1439615083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1439615083},{"type":"v1","addr":"172.21.15.183:6839","nonce":1439615083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24517,"qdb_cluster":[24517]},"id":1}]} 2024-06-03T03:20:19.818 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:19.816+0000 7f3b61800700 1 -- 172.21.15.117:0/364894155 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f3b400a2c20 msgr2=0x7f3b400a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:20:19.818 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:19.816+0000 7f3b61800700 1 --2- 172.21.15.117:0/364894155 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f3b400a2c20 0x7f3b400a50e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3b44009750 tx=0x7f3b44009310 comp rx=0 tx=0).stop 2024-06-03T03:20:19.819 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:19.816+0000 7f3b61800700 1 -- 172.21.15.117:0/364894155 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b5c102740 msgr2=0x7f3b5c19ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:20:19.819 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:19.817+0000 7f3b61800700 1 --2- 172.21.15.117:0/364894155 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b5c102740 0x7f3b5c19ae10 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c00d010 tx=0x7f3b4c00bca0 comp rx=0 tx=0).stop 2024-06-03T03:20:19.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:19.817+0000 7f3b61800700 1 -- 172.21.15.117:0/364894155 shutdown_connections 2024-06-03T03:20:19.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:19.817+0000 7f3b61800700 1 --2- 172.21.15.117:0/364894155 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f3b400a2c20 0x7f3b400a50e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:19.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:19.817+0000 7f3b61800700 1 --2- 172.21.15.117:0/364894155 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b5c107840 0x7f3b5c19f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:19.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:19.817+0000 7f3b61800700 1 --2- 172.21.15.117:0/364894155 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b5c102740 0x7f3b5c19ae10 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:19.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:19.817+0000 7f3b61800700 1 --2- 172.21.15.117:0/364894155 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3b5c1030f0 0x7f3b5c19b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:19.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:19.817+0000 7f3b61800700 1 -- 172.21.15.117:0/364894155 >> 172.21.15.117:0/364894155 conn(0x7f3b5c0795f0 msgr2=0x7f3b5c100760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:20:19.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:19.817+0000 7f3b61800700 1 -- 172.21.15.117:0/364894155 shutdown_connections 2024-06-03T03:20:19.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:19.817+0000 7f3b61800700 1 -- 172.21.15.117:0/364894155 wait complete. 2024-06-03T03:20:19.822 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 17 2024-06-03T03:20:20.158 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:19 smithi151 ceph-mon[37043]: pgmap v352: 129 pgs: 129 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:20:20.158 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:19 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/364894155' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T03:20:20.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:19 smithi117 ceph-mon[31649]: pgmap v352: 129 pgs: 129 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:20:20.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:19 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/364894155' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T03:20:20.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:19 smithi183 ceph-mon[37326]: pgmap v352: 129 pgs: 129 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:20:20.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:19 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/364894155' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T03:20:20.830 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-03T03:20:21.280 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:20:22.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:21 smithi151 ceph-mon[37043]: pgmap v353: 129 pgs: 129 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:20:22.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:21 smithi117 ceph-mon[31649]: pgmap v353: 129 pgs: 129 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:20:22.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:21 smithi183 ceph-mon[37326]: pgmap v353: 129 pgs: 129 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:20:23.614 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:23.613+0000 7f702f200700 1 -- 172.21.15.117:0/1201297925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f703010ebf0 msgr2=0x7f7030111060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:20:23.614 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:23.613+0000 7f702f200700 1 --2- 172.21.15.117:0/1201297925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f703010ebf0 0x7f7030111060 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7028009f90 tx=0x7f702800b330 comp rx=0 tx=0).stop 2024-06-03T03:20:23.614 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:23.613+0000 7f702f200700 1 -- 172.21.15.117:0/1201297925 shutdown_connections 2024-06-03T03:20:23.614 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:23.613+0000 7f702f200700 1 --2- 172.21.15.117:0/1201297925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f703010ebf0 0x7f7030111060 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:23.614 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:23.613+0000 7f702f200700 1 --2- 172.21.15.117:0/1201297925 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f70300748c0 0x7f7030074ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:23.614 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:23.613+0000 7f702f200700 1 --2- 172.21.15.117:0/1201297925 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70300751e0 0x7f703010e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:23.614 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:23.613+0000 7f702f200700 1 -- 172.21.15.117:0/1201297925 >> 172.21.15.117:0/1201297925 conn(0x7f70300fdea0 msgr2=0x7f70301002c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:20:23.615 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:23.613+0000 7f702f200700 1 -- 172.21.15.117:0/1201297925 shutdown_connections 2024-06-03T03:20:23.615 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:23.613+0000 7f702f200700 1 -- 172.21.15.117:0/1201297925 wait complete. 2024-06-03T03:20:23.615 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:23.615+0000 7f702f200700 1 Processor -- start 2024-06-03T03:20:23.615 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:23.615+0000 7f702f200700 1 -- start start 2024-06-03T03:20:23.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:23.615+0000 7f702f200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70300748c0 0x7f703019ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:20:23.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:23.615+0000 7f702f200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70300751e0 0x7f703019b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:20:23.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:23.615+0000 7f702f200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f703010ebf0 0x7f703019f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:20:23.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:23.615+0000 7f702f200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7030114920 con 0x7f70300748c0 2024-06-03T03:20:23.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:23.615+0000 7f702f200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f70301147a0 con 0x7f703010ebf0 2024-06-03T03:20:23.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:23.615+0000 7f702f200700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7030114aa0 con 0x7f70300751e0 2024-06-03T03:20:23.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:23.616+0000 7f702de00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70300748c0 0x7f703019ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:20:23.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:23.616+0000 7f702e800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f703010ebf0 0x7f703019f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:20:23.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:23.616+0000 7f702e800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f703010ebf0 0x7f703019f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:33384/0 (socket says 172.21.15.117:33384) 2024-06-03T03:20:23.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:23.616+0000 7f702e800700 1 -- 172.21.15.117:0/2492453749 learned_addr learned my addr 172.21.15.117:0/2492453749 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:20:23.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:23.616+0000 7f702de00700 1 --2- 172.21.15.117:0/2492453749 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70300748c0 0x7f703019ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:46324/0 (socket says 172.21.15.117:46324) 2024-06-03T03:20:23.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:23.616+0000 7f702d400700 1 --2- 172.21.15.117:0/2492453749 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70300751e0 0x7f703019b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:20:23.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:23.616+0000 7f702e800700 1 -- 172.21.15.117:0/2492453749 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70300751e0 msgr2=0x7f703019b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:20:23.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:23.616+0000 7f702e800700 1 --2- 172.21.15.117:0/2492453749 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70300751e0 0x7f703019b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:23.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:23.616+0000 7f702e800700 1 -- 172.21.15.117:0/2492453749 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70300748c0 msgr2=0x7f703019ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:20:23.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:23.616+0000 7f702e800700 1 --2- 172.21.15.117:0/2492453749 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70300748c0 0x7f703019ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:23.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:23.616+0000 7f702e800700 1 -- 172.21.15.117:0/2492453749 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f703019fd60 con 0x7f703010ebf0 2024-06-03T03:20:23.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:23.616+0000 7f702e800700 1 --2- 172.21.15.117:0/2492453749 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f703010ebf0 0x7f703019f620 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7028009550 tx=0x7f702800cfa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:20:23.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:23.617+0000 7f7026a00700 1 -- 172.21.15.117:0/2492453749 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7028021070 con 0x7f703010ebf0 2024-06-03T03:20:23.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:23.617+0000 7f7026a00700 1 -- 172.21.15.117:0/2492453749 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7028007ac0 con 0x7f703010ebf0 2024-06-03T03:20:23.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:23.617+0000 7f702f200700 1 -- 172.21.15.117:0/2492453749 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f703019fff0 con 0x7f703010ebf0 2024-06-03T03:20:23.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:23.617+0000 7f7026a00700 1 -- 172.21.15.117:0/2492453749 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70280052a0 con 0x7f703010ebf0 2024-06-03T03:20:23.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:23.617+0000 7f702f200700 1 -- 172.21.15.117:0/2492453749 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70301a78d0 con 0x7f703010ebf0 2024-06-03T03:20:23.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:23.618+0000 7f702f200700 1 -- 172.21.15.117:0/2492453749 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7030069180 con 0x7f703010ebf0 2024-06-03T03:20:23.620 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:23.620+0000 7f7026a00700 1 -- 172.21.15.117:0/2492453749 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f7028024070 con 0x7f703010ebf0 2024-06-03T03:20:23.622 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:23.621+0000 7f7026a00700 1 --2- 172.21.15.117:0/2492453749 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f70140a2c20 0x7f70140a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:20:23.622 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:23.621+0000 7f7026a00700 1 -- 172.21.15.117:0/2492453749 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8802+0+0 (secure 0 0 0) 0x7f70280d0290 con 0x7f703010ebf0 2024-06-03T03:20:23.622 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:23.621+0000 7f702de00700 1 --2- 172.21.15.117:0/2492453749 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f70140a2c20 0x7f70140a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:20:23.622 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:23.622+0000 7f702de00700 1 --2- 172.21.15.117:0/2492453749 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f70140a2c20 0x7f70140a50e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7020009680 tx=0x7f7020009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:20:23.624 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:23.624+0000 7f7026a00700 1 -- 172.21.15.117:0/2492453749 <== mon.2 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f702809d040 con 0x7f703010ebf0 2024-06-03T03:20:23.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:23.847+0000 7f702f200700 1 -- 172.21.15.117:0/2492453749 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f703002dca0 con 0x7f703010ebf0 2024-06-03T03:20:23.850 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:23.849+0000 7f7026a00700 1 -- 172.21.15.117:0/2492453749 <== mon.2 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+24187 (secure 0 0 0) 0x7f7028007c70 con 0x7f703010ebf0 2024-06-03T03:20:23.850 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T03:20:23.851 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":95,"fsid":"a0e8473a-2156-11ef-bc9b-c7b262605968","created":"2024-06-03T03:10:08.446859+0000","modified":"2024-06-03T03:20:03.882941+0000","last_up_change":"2024-06-03T03:18:28.736904+0000","last_in_change":"2024-06-03T03:18:08.485711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-03T03:14:09.370986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-03T03:19:35.044683+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-03T03:19:40.097569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-03T03:19:55.394480+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8fefa089-9401-4a1c-b9fb-fe30c63e6a1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":4286644492},{"type":"v1","addr":"172.21.15.117:6803","nonce":4286644492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":4286644492},{"type":"v1","addr":"172.21.15.117:6805","nonce":4286644492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":4286644492},{"type":"v1","addr":"172.21.15.117:6809","nonce":4286644492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":4286644492},{"type":"v1","addr":"172.21.15.117:6807","nonce":4286644492}]},"public_addr":"172.21.15.117:6803/4286644492","cluster_addr":"172.21.15.117:6805/4286644492","heartbeat_back_addr":"172.21.15.117:6809/4286644492","heartbeat_front_addr":"172.21.15.117:6807/4286644492","state":["exists","up"]},{"osd":1,"uuid":"5b12b371-f155-4b67-a470-25cbe59a2999","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":262491158},{"type":"v1","addr":"172.21.15.151:6801","nonce":262491158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":262491158},{"type":"v1","addr":"172.21.15.151:6803","nonce":262491158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":262491158},{"type":"v1","addr":"172.21.15.151:6807","nonce":262491158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":262491158},{"type":"v1","addr":"172.21.15.151:6805","nonce":262491158}]},"public_addr":"172.21.15.151:6801/262491158","cluster_addr":"172.21.15.151:6803/262491158","heartbeat_back_addr":"172.21.15.151:6807/262491158","heartbeat_front_addr":"172.21.15.151:6805/262491158","state":["exists","up"]},{"osd":2,"uuid":"dbc17818-c54c-4ecf-b656-03578b95d8c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":3542453623},{"type":"v1","addr":"172.21.15.183:6801","nonce":3542453623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":3542453623},{"type":"v1","addr":"172.21.15.183:6803","nonce":3542453623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":3542453623},{"type":"v1","addr":"172.21.15.183:6807","nonce":3542453623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":3542453623},{"type":"v1","addr":"172.21.15.183:6805","nonce":3542453623}]},"public_addr":"172.21.15.183:6801/3542453623","cluster_addr":"172.21.15.183:6803/3542453623","heartbeat_back_addr":"172.21.15.183:6807/3542453623","heartbeat_front_addr":"172.21.15.183:6805/3542453623","state":["exists","up"]},{"osd":3,"uuid":"ca08d2ac-b1d6-4a49-9ae8-e3ec146899a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":589012315},{"type":"v1","addr":"172.21.15.117:6811","nonce":589012315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":589012315},{"type":"v1","addr":"172.21.15.117:6813","nonce":589012315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":589012315},{"type":"v1","addr":"172.21.15.117:6817","nonce":589012315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":589012315},{"type":"v1","addr":"172.21.15.117:6815","nonce":589012315}]},"public_addr":"172.21.15.117:6811/589012315","cluster_addr":"172.21.15.117:6813/589012315","heartbeat_back_addr":"172.21.15.117:6817/589012315","heartbeat_front_addr":"172.21.15.117:6815/589012315","state":["exists","up"]},{"osd":4,"uuid":"810d6c7a-023f-4930-82af-bc2bbfd647da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":1239048647},{"type":"v1","addr":"172.21.15.151:6809","nonce":1239048647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":1239048647},{"type":"v1","addr":"172.21.15.151:6811","nonce":1239048647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":1239048647},{"type":"v1","addr":"172.21.15.151:6815","nonce":1239048647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":1239048647},{"type":"v1","addr":"172.21.15.151:6813","nonce":1239048647}]},"public_addr":"172.21.15.151:6809/1239048647","cluster_addr":"172.21.15.151:6811/1239048647","heartbeat_back_addr":"172.21.15.151:6815/1239048647","heartbeat_front_addr":"172.21.15.151:6813/1239048647","state":["exists","up"]},{"osd":5,"uuid":"8ab6a5f0-34bb-48d8-9471-a7dc836e486e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":2985874686},{"type":"v1","addr":"172.21.15.183:6809","nonce":2985874686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":2985874686},{"type":"v1","addr":"172.21.15.183:6811","nonce":2985874686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":2985874686},{"type":"v1","addr":"172.21.15.183:6815","nonce":2985874686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":2985874686},{"type":"v1","addr":"172.21.15.183:6813","nonce":2985874686}]},"public_addr":"172.21.15.183:6809/2985874686","cluster_addr":"172.21.15.183:6811/2985874686","heartbeat_back_addr":"172.21.15.183:6815/2985874686","heartbeat_front_addr":"172.21.15.183:6813/2985874686","state":["exists","up"]},{"osd":6,"uuid":"78d15739-e9e0-4f19-ae5f-ee8527a5b527","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4250583193},{"type":"v1","addr":"172.21.15.117:6819","nonce":4250583193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":4250583193},{"type":"v1","addr":"172.21.15.117:6821","nonce":4250583193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":4250583193},{"type":"v1","addr":"172.21.15.117:6825","nonce":4250583193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4250583193},{"type":"v1","addr":"172.21.15.117:6823","nonce":4250583193}]},"public_addr":"172.21.15.117:6819/4250583193","cluster_addr":"172.21.15.117:6821/4250583193","heartbeat_back_addr":"172.21.15.117:6825/4250583193","heartbeat_front_addr":"172.21.15.117:6823/4250583193","state":["exists","up"]},{"osd":7,"uuid":"3724fe1f-f4d7-443b-aea8-ff3ab02bfdc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":386094645},{"type":"v1","addr":"172.21.15.151:6817","nonce":386094645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":386094645},{"type":"v1","addr":"172.21.15.151:6819","nonce":386094645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":386094645},{"type":"v1","addr":"172.21.15.151:6823","nonce":386094645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":386094645},{"type":"v1","addr":"172.21.15.151:6821","nonce":386094645}]},"public_addr":"172.21.15.151:6817/386094645","cluster_addr":"172.21.15.151:6819/386094645","heartbeat_back_addr":"172.21.15.151:6823/386094645","heartbeat_front_addr":"172.21.15.151:6821/386094645","state":["exists","up"]},{"osd":8,"uuid":"006ac092-f015-448e-a0c0-f7b0c4597a18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1829748222},{"type":"v1","addr":"172.21.15.183:6817","nonce":1829748222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1829748222},{"type":"v1","addr":"172.21.15.183:6819","nonce":1829748222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1829748222},{"type":"v1","addr":"172.21.15.183:6823","nonce":1829748222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1829748222},{"type":"v1","addr":"172.21.15.183:6821","nonce":1829748222}]},"public_addr":"172.21.15.183:6817/1829748222","cluster_addr":"172.21.15.183:6819/1829748222","heartbeat_back_addr":"172.21.15.183:6823/1829748222","heartbeat_front_addr":"172.21.15.183:6821/1829748222","state":["exists","up"]},{"osd":9,"uuid":"00a970e6-2502-459e-975c-badac93c9bd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2538315141},{"type":"v1","addr":"172.21.15.117:6827","nonce":2538315141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2538315141},{"type":"v1","addr":"172.21.15.117:6829","nonce":2538315141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2538315141},{"type":"v1","addr":"172.21.15.117:6833","nonce":2538315141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2538315141},{"type":"v1","addr":"172.21.15.117:6831","nonce":2538315141}]},"public_addr":"172.21.15.117:6827/2538315141","cluster_addr":"172.21.15.117:6829/2538315141","heartbeat_back_addr":"172.21.15.117:6833/2538315141","heartbeat_front_addr":"172.21.15.117:6831/2538315141","state":["exists","up"]},{"osd":10,"uuid":"ba2333cd-32a3-41ef-9ea5-287c707f512b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":3801045133},{"type":"v1","addr":"172.21.15.151:6825","nonce":3801045133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3801045133},{"type":"v1","addr":"172.21.15.151:6827","nonce":3801045133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3801045133},{"type":"v1","addr":"172.21.15.151:6831","nonce":3801045133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3801045133},{"type":"v1","addr":"172.21.15.151:6829","nonce":3801045133}]},"public_addr":"172.21.15.151:6825/3801045133","cluster_addr":"172.21.15.151:6827/3801045133","heartbeat_back_addr":"172.21.15.151:6831/3801045133","heartbeat_front_addr":"172.21.15.151:6829/3801045133","state":["exists","up"]},{"osd":11,"uuid":"1556d654-e6fa-42c1-b399-aa53f6837a1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3513899696},{"type":"v1","addr":"172.21.15.183:6825","nonce":3513899696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3513899696},{"type":"v1","addr":"172.21.15.183:6827","nonce":3513899696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3513899696},{"type":"v1","addr":"172.21.15.183:6831","nonce":3513899696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3513899696},{"type":"v1","addr":"172.21.15.183:6829","nonce":3513899696}]},"public_addr":"172.21.15.183:6825/3513899696","cluster_addr":"172.21.15.183:6827/3513899696","heartbeat_back_addr":"172.21.15.183:6831/3513899696","heartbeat_front_addr":"172.21.15.183:6829/3513899696","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:13:17.174041+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:13:42.237001+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:14:06.273315+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:14:39.912564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:15:06.063202+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:15:30.424818+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:16:05.623554+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:16:32.926393+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:16:58.165899+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:17:32.897477+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:18:00.720866+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:18:26.474441+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.117:0/186309008":"2024-06-04T03:11:24.655516+0000","172.21.15.117:6801/2142313590":"2024-06-04T03:11:24.655516+0000","172.21.15.117:6800/2142313590":"2024-06-04T03:11:24.655516+0000","172.21.15.117:0/2399725156":"2024-06-04T03:11:24.655516+0000","172.21.15.117:0/1453232061":"2024-06-04T03:10:50.552899+0000","172.21.15.117:0/4281674974":"2024-06-04T03:10:50.552899+0000","172.21.15.117:6800/2424554985":"2024-06-04T03:10:50.552899+0000","172.21.15.117:0/3946365225":"2024-06-04T03:11:24.655516+0000","172.21.15.117:0/1146162354":"2024-06-04T03:10:50.552899+0000","172.21.15.117:6801/2424554985":"2024-06-04T03:10:50.552899+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-03T03:20:23.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:23.852+0000 7f702f200700 1 -- 172.21.15.117:0/2492453749 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f70140a2c20 msgr2=0x7f70140a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:20:23.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:23.852+0000 7f702f200700 1 --2- 172.21.15.117:0/2492453749 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f70140a2c20 0x7f70140a50e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7020009680 tx=0x7f7020009260 comp rx=0 tx=0).stop 2024-06-03T03:20:23.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:23.852+0000 7f702f200700 1 -- 172.21.15.117:0/2492453749 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f703010ebf0 msgr2=0x7f703019f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:20:23.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:23.852+0000 7f702f200700 1 --2- 172.21.15.117:0/2492453749 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f703010ebf0 0x7f703019f620 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7028009550 tx=0x7f702800cfa0 comp rx=0 tx=0).stop 2024-06-03T03:20:23.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:23.852+0000 7f702f200700 1 -- 172.21.15.117:0/2492453749 shutdown_connections 2024-06-03T03:20:23.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:23.852+0000 7f702f200700 1 --2- 172.21.15.117:0/2492453749 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f70140a2c20 0x7f70140a50e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:23.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:23.852+0000 7f702f200700 1 --2- 172.21.15.117:0/2492453749 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f703010ebf0 0x7f703019f620 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:23.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:23.852+0000 7f702f200700 1 --2- 172.21.15.117:0/2492453749 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70300748c0 0x7f703019ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:23.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:23.852+0000 7f702f200700 1 --2- 172.21.15.117:0/2492453749 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70300751e0 0x7f703019b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:23.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:23.852+0000 7f702f200700 1 -- 172.21.15.117:0/2492453749 >> 172.21.15.117:0/2492453749 conn(0x7f70300fdea0 msgr2=0x7f70300feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:20:23.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:23.853+0000 7f702f200700 1 -- 172.21.15.117:0/2492453749 shutdown_connections 2024-06-03T03:20:23.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:23.853+0000 7f702f200700 1 -- 172.21.15.117:0/2492453749 wait complete. 2024-06-03T03:20:24.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:23 smithi151 ceph-mon[37043]: pgmap v354: 129 pgs: 129 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:20:24.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:23 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/2492453749' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T03:20:24.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:23 smithi117 ceph-mon[31649]: pgmap v354: 129 pgs: 129 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:20:24.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:23 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/2492453749' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T03:20:24.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:23 smithi183 ceph-mon[37326]: pgmap v354: 129 pgs: 129 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:20:24.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:23 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/2492453749' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T03:20:24.672 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-06-03T03:19:44.629822+0000', 'modified': '2024-06-03T03:20:03.883250+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24517}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24517': {'gid': 24517, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.183:6839/1439615083', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6838', 'nonce': 1439615083}, {'type': 'v1', 'addr': '172.21.15.183:6839', 'nonce': 1439615083}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24517, 'qdb_cluster': [24517]} 2024-06-03T03:20:24.672 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-06-03T03:20:24.672 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph tell mds.i status 2024-06-03T03:20:25.122 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:20:25.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:24 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-06-03T03:20:25.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:24 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-06-03T03:20:25.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:24 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]: dispatch 2024-06-03T03:20:25.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:24 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]: dispatch 2024-06-03T03:20:25.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:24 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]: dispatch 2024-06-03T03:20:25.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:24 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]: dispatch 2024-06-03T03:20:25.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:24 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]: dispatch 2024-06-03T03:20:25.158 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:24 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-06-03T03:20:25.158 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:24 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]: dispatch 2024-06-03T03:20:25.158 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:24 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]: dispatch 2024-06-03T03:20:25.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:24 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-06-03T03:20:25.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:24 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-06-03T03:20:25.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:24 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]: dispatch 2024-06-03T03:20:25.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:24 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]: dispatch 2024-06-03T03:20:25.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:24 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]: dispatch 2024-06-03T03:20:25.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:24 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]: dispatch 2024-06-03T03:20:25.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:24 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]: dispatch 2024-06-03T03:20:25.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:24 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-06-03T03:20:25.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:24 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]: dispatch 2024-06-03T03:20:25.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:24 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]: dispatch 2024-06-03T03:20:25.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:24 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-06-03T03:20:25.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:24 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-06-03T03:20:25.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:24 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]: dispatch 2024-06-03T03:20:25.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:24 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]: dispatch 2024-06-03T03:20:25.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:24 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]: dispatch 2024-06-03T03:20:25.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:24 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]: dispatch 2024-06-03T03:20:25.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:24 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]: dispatch 2024-06-03T03:20:25.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:24 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-06-03T03:20:25.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:24 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]: dispatch 2024-06-03T03:20:25.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:24 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]: dispatch 2024-06-03T03:20:26.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:25 smithi151 ceph-mon[37043]: pgmap v355: 129 pgs: 129 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:20:26.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:25 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-06-03T03:20:26.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:25 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-06-03T03:20:26.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:25 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-06-03T03:20:26.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:25 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-06-03T03:20:26.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:25 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]': finished 2024-06-03T03:20:26.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:25 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]': finished 2024-06-03T03:20:26.158 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:25 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2024-06-03T03:20:26.158 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:25 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-06-03T03:20:26.158 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:25 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]': finished 2024-06-03T03:20:26.158 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:25 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2024-06-03T03:20:26.158 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:25 smithi151 ceph-mon[37043]: osdmap e96: 12 total, 12 up, 12 in 2024-06-03T03:20:26.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:25 smithi117 ceph-mon[31649]: pgmap v355: 129 pgs: 129 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:20:26.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:25 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-06-03T03:20:26.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:25 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-06-03T03:20:26.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:25 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-06-03T03:20:26.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:25 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-06-03T03:20:26.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:25 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]': finished 2024-06-03T03:20:26.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:25 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]': finished 2024-06-03T03:20:26.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:25 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2024-06-03T03:20:26.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:25 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-06-03T03:20:26.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:25 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]': finished 2024-06-03T03:20:26.199 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:25 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2024-06-03T03:20:26.199 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:25 smithi117 ceph-mon[31649]: osdmap e96: 12 total, 12 up, 12 in 2024-06-03T03:20:26.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:25 smithi183 ceph-mon[37326]: pgmap v355: 129 pgs: 129 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:20:26.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:25 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-06-03T03:20:26.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:25 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-06-03T03:20:26.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:25 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-06-03T03:20:26.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:25 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-06-03T03:20:26.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:25 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]': finished 2024-06-03T03:20:26.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:25 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]': finished 2024-06-03T03:20:26.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:25 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2024-06-03T03:20:26.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:25 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-06-03T03:20:26.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:25 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]': finished 2024-06-03T03:20:26.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:25 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2024-06-03T03:20:26.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:25 smithi183 ceph-mon[37326]: osdmap e96: 12 total, 12 up, 12 in 2024-06-03T03:20:26.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:26 smithi117 ceph-mon[31649]: osdmap e97: 12 total, 12 up, 12 in 2024-06-03T03:20:27.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:26 smithi151 ceph-mon[37043]: osdmap e97: 12 total, 12 up, 12 in 2024-06-03T03:20:27.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:26 smithi183 ceph-mon[37326]: osdmap e97: 12 total, 12 up, 12 in 2024-06-03T03:20:27.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.579+0000 7f55c9000700 1 -- 172.21.15.117:0/2723525380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55c4103ea0 msgr2=0x7f55c410c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:20:27.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.579+0000 7f55c9000700 1 --2- 172.21.15.117:0/2723525380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55c4103ea0 0x7f55c410c390 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f55ac009990 tx=0x7f55ac009ca0 comp rx=0 tx=0).stop 2024-06-03T03:20:27.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.580+0000 7f55c9000700 1 -- 172.21.15.117:0/2723525380 shutdown_connections 2024-06-03T03:20:27.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.580+0000 7f55c9000700 1 --2- 172.21.15.117:0/2723525380 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f55c410c9b0 0x7f55c410ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:27.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.580+0000 7f55c9000700 1 --2- 172.21.15.117:0/2723525380 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f55c4103580 0x7f55c4103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:27.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.580+0000 7f55c9000700 1 --2- 172.21.15.117:0/2723525380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55c4103ea0 0x7f55c410c390 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:27.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.580+0000 7f55c9000700 1 -- 172.21.15.117:0/2723525380 >> 172.21.15.117:0/2723525380 conn(0x7f55c40fcdb0 msgr2=0x7f55c40ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:20:27.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.580+0000 7f55c9000700 1 -- 172.21.15.117:0/2723525380 shutdown_connections 2024-06-03T03:20:27.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.580+0000 7f55c9000700 1 -- 172.21.15.117:0/2723525380 wait complete. 2024-06-03T03:20:27.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.581+0000 7f55c9000700 1 Processor -- start 2024-06-03T03:20:27.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.581+0000 7f55c9000700 1 -- start start 2024-06-03T03:20:27.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.581+0000 7f55c9000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55c4103580 0x7f55c419e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:20:27.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.581+0000 7f55c9000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f55c4103ea0 0x7f55c419ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:20:27.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.582+0000 7f55c9000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f55c410c9b0 0x7f55c4198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:20:27.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.582+0000 7f55c9000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f55c41126d0 con 0x7f55c4103580 2024-06-03T03:20:27.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.582+0000 7f55c9000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f55c4112550 con 0x7f55c4103ea0 2024-06-03T03:20:27.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.582+0000 7f55c9000700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f55c4112850 con 0x7f55c410c9b0 2024-06-03T03:20:27.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.582+0000 7f55c3400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55c4103580 0x7f55c419e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:20:27.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.582+0000 7f55c3400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55c4103580 0x7f55c419e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34144/0 (socket says 172.21.15.117:34144) 2024-06-03T03:20:27.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.582+0000 7f55c3400700 1 -- 172.21.15.117:0/1466249862 learned_addr learned my addr 172.21.15.117:0/1466249862 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:20:27.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.582+0000 7f55baa00700 1 --2- 172.21.15.117:0/1466249862 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f55c4103ea0 0x7f55c419ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:20:27.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.582+0000 7f55c3e00700 1 --2- 172.21.15.117:0/1466249862 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f55c410c9b0 0x7f55c4198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:20:27.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.582+0000 7f55c3400700 1 -- 172.21.15.117:0/1466249862 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f55c410c9b0 msgr2=0x7f55c4198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:20:27.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.582+0000 7f55c3400700 1 --2- 172.21.15.117:0/1466249862 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f55c410c9b0 0x7f55c4198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:27.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.582+0000 7f55c3400700 1 -- 172.21.15.117:0/1466249862 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f55c4103ea0 msgr2=0x7f55c419ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:20:27.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.582+0000 7f55c3400700 1 --2- 172.21.15.117:0/1466249862 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f55c4103ea0 0x7f55c419ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:27.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.582+0000 7f55c3400700 1 -- 172.21.15.117:0/1466249862 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55c4199170 con 0x7f55c4103580 2024-06-03T03:20:27.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.582+0000 7f55baa00700 1 --2- 172.21.15.117:0/1466249862 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f55c4103ea0 0x7f55c419ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:20:27.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.583+0000 7f55c3400700 1 --2- 172.21.15.117:0/1466249862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55c4103580 0x7f55c419e800 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f55b400c370 tx=0x7f55b400c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:20:27.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.583+0000 7f55c0c00700 1 -- 172.21.15.117:0/1466249862 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b40092e0 con 0x7f55c4103580 2024-06-03T03:20:27.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.583+0000 7f55c9000700 1 -- 172.21.15.117:0/1466249862 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55c4199400 con 0x7f55c4103580 2024-06-03T03:20:27.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.583+0000 7f55c0c00700 1 -- 172.21.15.117:0/1466249862 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55b400cbe0 con 0x7f55c4103580 2024-06-03T03:20:27.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.583+0000 7f55c0c00700 1 -- 172.21.15.117:0/1466249862 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b40047e0 con 0x7f55c4103580 2024-06-03T03:20:27.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.583+0000 7f55c9000700 1 -- 172.21.15.117:0/1466249862 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55c41a56d0 con 0x7f55c4103580 2024-06-03T03:20:27.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.586+0000 7f55c0c00700 1 -- 172.21.15.117:0/1466249862 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f55b401a020 con 0x7f55c4103580 2024-06-03T03:20:27.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.588+0000 7f55d4f8bb80 1 Processor -- start 2024-06-03T03:20:27.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.588+0000 7f55c0c00700 1 --2- 172.21.15.117:0/1466249862 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f55a80a2c20 0x7f55a80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:20:27.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.588+0000 7f55c0c00700 1 -- 172.21.15.117:0/1466249862 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7f55b40ced80 con 0x7f55c4103580 2024-06-03T03:20:27.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.588+0000 7f55d4f8bb80 1 -- start start 2024-06-03T03:20:27.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.588+0000 7f55baa00700 1 --2- 172.21.15.117:0/1466249862 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f55a80a2c20 0x7f55a80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:20:27.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.589+0000 7f55d4f8bb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650aed0adf0 0x5650aed903d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:20:27.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.589+0000 7f55d4f8bb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650aed90910 0x5650aed90d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:20:27.591 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.589+0000 7f55d4f8bb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650aed9d930 0x5650aed9fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:20:27.591 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.589+0000 7f55d4f8bb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5650aec9d9c0 con 0x5650aed90910 2024-06-03T03:20:27.591 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.589+0000 7f55d4f8bb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5650aece0ab0 con 0x5650aed0adf0 2024-06-03T03:20:27.591 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.589+0000 7f55d4f8bb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5650aec9b8a0 con 0x5650aed9d930 2024-06-03T03:20:27.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.589+0000 7f55baa00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650aed90910 0x5650aed90d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:20:27.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.589+0000 7f55c3400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650aed0adf0 0x5650aed903d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:20:27.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.589+0000 7f55baa00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650aed90910 0x5650aed90d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34158/0 (socket says 172.21.15.117:34158) 2024-06-03T03:20:27.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.589+0000 7f55baa00700 1 -- 172.21.15.117:0/2129404169 learned_addr learned my addr 172.21.15.117:0/2129404169 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:20:27.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.589+0000 7f55c3400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650aed0adf0 0x5650aed903d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:60174/0 (socket says 172.21.15.117:60174) 2024-06-03T03:20:27.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.589+0000 7f55c3e00700 1 --2- 172.21.15.117:0/2129404169 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650aed9d930 0x5650aed9fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:20:27.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.589+0000 7f55baa00700 1 --2- 172.21.15.117:0/1466249862 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f55a80a2c20 0x7f55a80a50e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f55ac0098e0 tx=0x7f55ac00b610 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:20:27.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.589+0000 7f55baa00700 1 -- 172.21.15.117:0/2129404169 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650aed9d930 msgr2=0x5650aed9fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:20:27.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.589+0000 7f55baa00700 1 --2- 172.21.15.117:0/2129404169 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650aed9d930 0x5650aed9fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:27.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.589+0000 7f55baa00700 1 -- 172.21.15.117:0/2129404169 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650aed0adf0 msgr2=0x5650aed903d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:20:27.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.589+0000 7f55baa00700 1 --2- 172.21.15.117:0/2129404169 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650aed0adf0 0x5650aed903d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:27.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.590+0000 7f55baa00700 1 -- 172.21.15.117:0/2129404169 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650aec88f40 con 0x5650aed90910 2024-06-03T03:20:27.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.590+0000 7f55c3400700 1 --2- 172.21.15.117:0/2129404169 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650aed0adf0 0x5650aed903d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:20:27.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.590+0000 7f55baa00700 1 --2- 172.21.15.117:0/2129404169 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650aed90910 0x5650aed90d90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f55ac009990 tx=0x7f55ac028000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bab1806ccc24e1f server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:20:27.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.590+0000 7f55b8c00700 1 -- 172.21.15.117:0/2129404169 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55ac034070 con 0x5650aed90910 2024-06-03T03:20:27.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.590+0000 7f55b8c00700 1 -- 172.21.15.117:0/2129404169 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55ac02b040 con 0x5650aed90910 2024-06-03T03:20:27.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.590+0000 7f55b8c00700 1 -- 172.21.15.117:0/2129404169 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55ac02fc50 con 0x5650aed90910 2024-06-03T03:20:27.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.591+0000 7f55d4f8bb80 1 -- 172.21.15.117:0/2129404169 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650aed90910 msgr2=0x5650aed90d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:20:27.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.591+0000 7f55d4f8bb80 1 --2- 172.21.15.117:0/2129404169 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650aed90910 0x5650aed90d90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f55ac009990 tx=0x7f55ac028000 comp rx=0 tx=0).stop 2024-06-03T03:20:27.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.591+0000 7f55d4f8bb80 1 -- 172.21.15.117:0/2129404169 shutdown_connections 2024-06-03T03:20:27.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.591+0000 7f55d4f8bb80 1 --2- 172.21.15.117:0/2129404169 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650aed9d930 0x5650aed9fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:27.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.591+0000 7f55d4f8bb80 1 --2- 172.21.15.117:0/2129404169 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650aed0adf0 0x5650aed903d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:27.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.591+0000 7f55d4f8bb80 1 --2- 172.21.15.117:0/2129404169 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650aed90910 0x5650aed90d90 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:27.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.591+0000 7f55d4f8bb80 1 -- 172.21.15.117:0/2129404169 >> 172.21.15.117:0/2129404169 conn(0x5650aecbb120 msgr2=0x5650aed8a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:20:27.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.591+0000 7f55d4f8bb80 1 -- 172.21.15.117:0/2129404169 shutdown_connections 2024-06-03T03:20:27.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.591+0000 7f55d4f8bb80 1 -- 172.21.15.117:0/2129404169 wait complete. 2024-06-03T03:20:27.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.592+0000 7f55d4f8bb80 1 -- start start 2024-06-03T03:20:27.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.592+0000 7f55d4f8bb80 1 Processor -- start 2024-06-03T03:20:27.596 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.593+0000 7f55d4f8bb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650aed0adf0 0x5650aed9d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:20:27.596 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.593+0000 7f55d4f8bb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650aed90910 0x5650aed9a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:20:27.596 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.593+0000 7f55d4f8bb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650aed9d930 0x5650aed9a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:20:27.596 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.593+0000 7f55d4f8bb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5650aec7cf10 con 0x5650aed90910 2024-06-03T03:20:27.596 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.593+0000 7f55d4f8bb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5650aecdca70 con 0x5650aed9d930 2024-06-03T03:20:27.596 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.593+0000 7f55d4f8bb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5650aeda0540 con 0x5650aed0adf0 2024-06-03T03:20:27.596 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.593+0000 7f55d4f8bb80 10 client.0 _ll_register_callbacks cb 0x5650aec7a6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:20:27.596 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.593+0000 7f55baa00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650aed90910 0x5650aed9a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:20:27.596 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.593+0000 7f55baa00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650aed90910 0x5650aed9a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34174/0 (socket says 172.21.15.117:34174) 2024-06-03T03:20:27.596 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.593+0000 7f55baa00700 1 -- 172.21.15.117:0/4104817790 learned_addr learned my addr 172.21.15.117:0/4104817790 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:20:27.596 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.593+0000 7f55c3e00700 1 --2- 172.21.15.117:0/4104817790 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650aed9d930 0x5650aed9a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:20:27.596 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.593+0000 7f55c3400700 1 --2- 172.21.15.117:0/4104817790 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650aed0adf0 0x5650aed9d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:20:27.598 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.593+0000 7f55baa00700 1 -- 172.21.15.117:0/4104817790 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650aed0adf0 msgr2=0x5650aed9d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:20:27.598 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.593+0000 7f55baa00700 1 --2- 172.21.15.117:0/4104817790 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650aed0adf0 0x5650aed9d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:27.598 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.593+0000 7f55baa00700 1 -- 172.21.15.117:0/4104817790 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650aed9d930 msgr2=0x5650aed9a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:20:27.598 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.593+0000 7f55baa00700 1 --2- 172.21.15.117:0/4104817790 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650aed9d930 0x5650aed9a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:27.598 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.593+0000 7f55baa00700 1 -- 172.21.15.117:0/4104817790 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650aec2f880 con 0x5650aed90910 2024-06-03T03:20:27.598 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.594+0000 7f55baa00700 1 --2- 172.21.15.117:0/4104817790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650aed90910 0x5650aed9a450 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f55ac032040 tx=0x7f55ac02f470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a303e88e9f24d918 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:20:27.598 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.594+0000 7f55b1600700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-06-03T03:20:27.599 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.594+0000 7f55b1600700 1 -- 172.21.15.117:0/4104817790 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55ac03d020 con 0x5650aed90910 2024-06-03T03:20:27.599 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.594+0000 7f55d4f8bb80 1 -- 172.21.15.117:0/4104817790 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55ac02b040 con 0x5650aed90910 2024-06-03T03:20:27.599 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.594+0000 7f55b1600700 1 -- 172.21.15.117:0/4104817790 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7f55ac003ce0 con 0x5650aed90910 2024-06-03T03:20:27.599 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.594+0000 7f55b1600700 1 -- 172.21.15.117:0/4104817790 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f55ac048050 con 0x5650aed90910 2024-06-03T03:20:27.599 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.594+0000 7f55d4f8bb80 10 client.14496 fetch_fsmap learned FSMap version 17 2024-06-03T03:20:27.599 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.594+0000 7f55d4f8bb80 1 -- 172.21.15.117:0/4104817790 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x5650aedaad10 con 0x5650aed90910 2024-06-03T03:20:27.599 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.595+0000 7f55b1600700 1 -- 172.21.15.117:0/4104817790 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55ac02b040 con 0x5650aed90910 2024-06-03T03:20:27.599 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.595+0000 7f55b1600700 1 -- 172.21.15.117:0/4104817790 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7f55ac025070 con 0x5650aed90910 2024-06-03T03:20:27.599 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.595+0000 7f55d4f8bb80 10 client.14496 fetch_fsmap finished waiting for FSMap version 17 2024-06-03T03:20:27.599 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.595+0000 7f55d4f8bb80 10 client.14496 resolve_mds: resolved name 'i' to daemon mds.i 2024-06-03T03:20:27.599 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.595+0000 7f55d4f8bb80 20 client.14496 populate_metadata read hostname 'smithi117' 2024-06-03T03:20:27.599 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.595+0000 7f55d4f8bb80 1 --2- 172.21.15.117:0/4104817790 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x5650aedace80 0x5650aedae2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:20:27.599 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.595+0000 7f55d4f8bb80 4 client.14496 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T03:20:27.599 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.595+0000 7f55d4f8bb80 1 -- 172.21.15.117:0/4104817790 --> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5650aecedfb0 con 0x5650aedace80 2024-06-03T03:20:27.600 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.596+0000 7f55c3400700 1 --2- 172.21.15.117:0/4104817790 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x5650aedace80 0x5650aedae2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:20:27.600 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.596+0000 7f55c3400700 1 --2- 172.21.15.117:0/4104817790 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x5650aedace80 0x5650aedae2f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a6e129e1759f116 server_cookie=cb72ca1f0c9fab23 in_seq=0 out_seq=0 2024-06-03T03:20:27.600 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.596+0000 7f55b1600700 10 client.14496 ms_handle_connect on v2:172.21.15.183:6838/1439615083 2024-06-03T03:20:27.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.597+0000 7f55b1600700 1 -- 172.21.15.117:0/4104817790 <== mds.0 v2:172.21.15.183:6838/1439615083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5650aecedfb0 con 0x5650aedace80 2024-06-03T03:20:27.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.597+0000 7f55b1600700 10 client.14496 handle_command_reply: tid=0 2024-06-03T03:20:27.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.597+0000 7f55d4f8bb80 1 client.14496 shutdown 2024-06-03T03:20:27.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.597+0000 7f55d4f8bb80 1 -- 172.21.15.117:0/4104817790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650aed90910 msgr2=0x5650aed9a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:20:27.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.597+0000 7f55d4f8bb80 1 --2- 172.21.15.117:0/4104817790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650aed90910 0x5650aed9a450 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f55ac032040 tx=0x7f55ac02f470 comp rx=0 tx=0).stop 2024-06-03T03:20:27.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.597+0000 7f55d4f8bb80 1 -- 172.21.15.117:0/4104817790 shutdown_connections 2024-06-03T03:20:27.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.597+0000 7f55d4f8bb80 1 --2- 172.21.15.117:0/4104817790 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x5650aedace80 0x5650aedae2f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:27.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.597+0000 7f55d4f8bb80 1 --2- 172.21.15.117:0/4104817790 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650aed9d930 0x5650aed9a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:27.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.597+0000 7f55d4f8bb80 1 --2- 172.21.15.117:0/4104817790 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650aed0adf0 0x5650aed9d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:27.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.597+0000 7f55d4f8bb80 1 --2- 172.21.15.117:0/4104817790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650aed90910 0x5650aed9a450 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:27.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.597+0000 7f55d4f8bb80 1 -- 172.21.15.117:0/4104817790 >> 172.21.15.117:0/4104817790 conn(0x5650aecbb120 msgr2=0x5650aed8a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:20:27.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.597+0000 7f55b1600700 0 client.14496 ms_handle_reset on v2:172.21.15.183:6838/1439615083 2024-06-03T03:20:27.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.598+0000 7f55d4f8bb80 1 -- 172.21.15.117:0/4104817790 shutdown_connections 2024-06-03T03:20:27.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.598+0000 7f55d4f8bb80 1 -- 172.21.15.117:0/4104817790 wait complete. 2024-06-03T03:20:27.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.598+0000 7f55d4f8bb80 20 client.14496 trim_cache size 0 max 16384 2024-06-03T03:20:27.623 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.621+0000 7f55d4f8bb80 1 Processor -- start 2024-06-03T03:20:27.623 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.621+0000 7f55d4f8bb80 1 -- start start 2024-06-03T03:20:27.623 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.621+0000 7f55d4f8bb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650aed0adf0 0x5650aed99320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:20:27.623 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.621+0000 7f55d4f8bb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650aed90910 0x5650aed976d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:20:27.623 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.621+0000 7f55d4f8bb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650aed9d930 0x5650aed97c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:20:27.623 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.621+0000 7f55d4f8bb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5650aedac630 con 0x5650aed0adf0 2024-06-03T03:20:27.623 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.621+0000 7f55d4f8bb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5650aedacd00 con 0x5650aed9d930 2024-06-03T03:20:27.623 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.621+0000 7f55d4f8bb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5650aedad3d0 con 0x5650aed90910 2024-06-03T03:20:27.624 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.622+0000 7f55baa00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650aed90910 0x5650aed976d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:20:27.624 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.622+0000 7f55c3e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650aed9d930 0x5650aed97c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:20:27.625 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.621+0000 7f55c3400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650aed0adf0 0x5650aed99320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:20:27.625 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.622+0000 7f55baa00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650aed90910 0x5650aed976d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:58840/0 (socket says 172.21.15.117:58840) 2024-06-03T03:20:27.625 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.622+0000 7f55baa00700 1 -- 172.21.15.117:0/1449914571 learned_addr learned my addr 172.21.15.117:0/1449914571 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:20:27.625 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.622+0000 7f55c3400700 1 -- 172.21.15.117:0/1449914571 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650aed90910 msgr2=0x5650aed976d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:20:27.625 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.622+0000 7f55c3400700 1 --2- 172.21.15.117:0/1449914571 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650aed90910 0x5650aed976d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:27.625 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.622+0000 7f55c3400700 1 -- 172.21.15.117:0/1449914571 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650aed9d930 msgr2=0x5650aed97c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:20:27.625 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.622+0000 7f55c3400700 1 --2- 172.21.15.117:0/1449914571 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650aed9d930 0x5650aed97c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:27.625 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.622+0000 7f55c3400700 1 -- 172.21.15.117:0/1449914571 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650aed8f520 con 0x5650aed0adf0 2024-06-03T03:20:27.625 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.622+0000 7f55baa00700 1 --2- 172.21.15.117:0/1449914571 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650aed90910 0x5650aed976d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:20:27.625 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.622+0000 7f55c3e00700 1 --2- 172.21.15.117:0/1449914571 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650aed9d930 0x5650aed97c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:20:27.625 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.622+0000 7f55c3400700 1 --2- 172.21.15.117:0/1449914571 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650aed0adf0 0x5650aed99320 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f55b40cc4c0 tx=0x7f55b40cc390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6ec0182250206a4 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:20:27.625 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.622+0000 7f55b3e00700 1 -- 172.21.15.117:0/1449914571 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b40b9e60 con 0x5650aed0adf0 2024-06-03T03:20:27.625 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.623+0000 7f55b3e00700 1 -- 172.21.15.117:0/1449914571 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55b409b920 con 0x5650aed0adf0 2024-06-03T03:20:27.626 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.623+0000 7f55b3e00700 1 -- 172.21.15.117:0/1449914571 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b400cd90 con 0x5650aed0adf0 2024-06-03T03:20:27.628 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.623+0000 7f55d4f8bb80 1 -- 172.21.15.117:0/1449914571 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650aed0adf0 msgr2=0x5650aed99320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:20:27.628 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.623+0000 7f55d4f8bb80 1 --2- 172.21.15.117:0/1449914571 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650aed0adf0 0x5650aed99320 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f55b40cc4c0 tx=0x7f55b40cc390 comp rx=0 tx=0).stop 2024-06-03T03:20:27.628 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.623+0000 7f55d4f8bb80 1 -- 172.21.15.117:0/1449914571 shutdown_connections 2024-06-03T03:20:27.629 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.623+0000 7f55d4f8bb80 1 --2- 172.21.15.117:0/1449914571 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650aed9d930 0x5650aed97c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:27.629 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.623+0000 7f55d4f8bb80 1 --2- 172.21.15.117:0/1449914571 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650aed0adf0 0x5650aed99320 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:27.629 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.623+0000 7f55d4f8bb80 1 --2- 172.21.15.117:0/1449914571 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650aed90910 0x5650aed976d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:27.629 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.623+0000 7f55d4f8bb80 1 -- 172.21.15.117:0/1449914571 >> 172.21.15.117:0/1449914571 conn(0x5650aecbb120 msgr2=0x5650aed98c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:20:27.629 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.624+0000 7f55d4f8bb80 1 -- 172.21.15.117:0/1449914571 shutdown_connections 2024-06-03T03:20:27.629 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.624+0000 7f55d4f8bb80 1 -- 172.21.15.117:0/1449914571 wait complete. 2024-06-03T03:20:27.629 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.624+0000 7f55d4f8bb80 1 -- start start 2024-06-03T03:20:27.629 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.624+0000 7f55d4f8bb80 1 Processor -- start 2024-06-03T03:20:27.629 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.625+0000 7f55d4f8bb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650aed0adf0 0x5650aedb1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:20:27.629 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.625+0000 7f55d4f8bb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650aed90910 0x5650aedb1d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:20:27.629 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.625+0000 7f55d4f8bb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650aed9d930 0x5650aedb5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:20:27.629 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.625+0000 7f55d4f8bb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5650aedae170 con 0x5650aed0adf0 2024-06-03T03:20:27.629 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.625+0000 7f55d4f8bb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5650aedadaa0 con 0x5650aed90910 2024-06-03T03:20:27.630 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.625+0000 7f55d4f8bb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5650aedae840 con 0x5650aed9d930 2024-06-03T03:20:27.630 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.625+0000 7f55d4f8bb80 10 client.0 _ll_register_callbacks cb 0x5650aec7a6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:20:27.630 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.625+0000 7f55c3400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650aed0adf0 0x5650aedb1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:20:27.630 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.625+0000 7f55c3400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650aed0adf0 0x5650aedb1840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34202/0 (socket says 172.21.15.117:34202) 2024-06-03T03:20:27.630 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.625+0000 7f55c3400700 1 -- 172.21.15.117:0/1761200861 learned_addr learned my addr 172.21.15.117:0/1761200861 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:20:27.630 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.625+0000 7f55baa00700 1 --2- 172.21.15.117:0/1761200861 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650aed90910 0x5650aedb1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:20:27.630 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.625+0000 7f55c3e00700 1 --2- 172.21.15.117:0/1761200861 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650aed9d930 0x5650aedb5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:20:27.630 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.626+0000 7f55c3e00700 1 -- 172.21.15.117:0/1761200861 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650aed90910 msgr2=0x5650aedb1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:20:27.630 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.626+0000 7f55c3e00700 1 --2- 172.21.15.117:0/1761200861 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650aed90910 0x5650aedb1d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:27.630 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.626+0000 7f55c3e00700 1 -- 172.21.15.117:0/1761200861 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650aed0adf0 msgr2=0x5650aedb1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:20:27.630 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.626+0000 7f55c3e00700 1 --2- 172.21.15.117:0/1761200861 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650aed0adf0 0x5650aedb1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:27.630 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.626+0000 7f55c3e00700 1 -- 172.21.15.117:0/1761200861 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650aedb2640 con 0x5650aed9d930 2024-06-03T03:20:27.630 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.626+0000 7f55baa00700 1 --2- 172.21.15.117:0/1761200861 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650aed90910 0x5650aedb1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:20:27.631 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.626+0000 7f55c3400700 1 --2- 172.21.15.117:0/1761200861 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650aed0adf0 0x5650aedb1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:20:27.631 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.626+0000 7f55c3e00700 1 --2- 172.21.15.117:0/1761200861 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650aed9d930 0x5650aedb5fb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f55bc009f90 tx=0x7f55bc003960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f211879f717b9d0b server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:20:27.631 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.626+0000 7f55b1600700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-06-03T03:20:27.631 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.626+0000 7f55b1600700 1 -- 172.21.15.117:0/1761200861 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55bc004830 con 0x5650aed9d930 2024-06-03T03:20:27.631 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.626+0000 7f55d4f8bb80 1 -- 172.21.15.117:0/1761200861 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55b409b920 con 0x5650aed9d930 2024-06-03T03:20:27.631 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.626+0000 7f55b1600700 1 -- 172.21.15.117:0/1761200861 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7f55bc0049d0 con 0x5650aed9d930 2024-06-03T03:20:27.631 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.627+0000 7f55b1600700 1 -- 172.21.15.117:0/1761200861 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f55bc02a050 con 0x5650aed9d930 2024-06-03T03:20:27.631 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.627+0000 7f55d4f8bb80 10 client.24640 fetch_fsmap learned FSMap version 17 2024-06-03T03:20:27.631 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.627+0000 7f55d4f8bb80 1 -- 172.21.15.117:0/1761200861 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x5650aee18df0 con 0x5650aed9d930 2024-06-03T03:20:27.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.627+0000 7f55b1600700 1 -- 172.21.15.117:0/1761200861 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55b409b920 con 0x5650aed9d930 2024-06-03T03:20:27.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.627+0000 7f55b1600700 1 -- 172.21.15.117:0/1761200861 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7f55bc010640 con 0x5650aed9d930 2024-06-03T03:20:27.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.627+0000 7f55d4f8bb80 10 client.24640 fetch_fsmap finished waiting for FSMap version 17 2024-06-03T03:20:27.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.627+0000 7f55d4f8bb80 10 client.24640 resolve_mds: resolved name 'i' to daemon mds.i 2024-06-03T03:20:27.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.627+0000 7f55d4f8bb80 20 client.24640 populate_metadata read hostname 'smithi117' 2024-06-03T03:20:27.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.627+0000 7f55d4f8bb80 1 --2- 172.21.15.117:0/1761200861 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x5650aee1da40 0x5650aee1c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:20:27.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.627+0000 7f55d4f8bb80 4 client.24640 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-03T03:20:27.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.627+0000 7f55d4f8bb80 1 -- 172.21.15.117:0/1761200861 --> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] -- command(tid 0: {"prefix": "status"}) -- 0x5650aece8530 con 0x5650aee1da40 2024-06-03T03:20:27.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.628+0000 7f55c3400700 1 --2- 172.21.15.117:0/1761200861 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x5650aee1da40 0x5650aee1c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:20:27.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.628+0000 7f55b1600700 10 client.24640 ms_handle_connect on v2:172.21.15.183:6838/1439615083 2024-06-03T03:20:27.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.628+0000 7f55c3400700 1 --2- 172.21.15.117:0/1761200861 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x5650aee1da40 0x5650aee1c750 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efdbb182c1986c server_cookie=ad9390f32fd6fe7a in_seq=0 out_seq=0 2024-06-03T03:20:27.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.628+0000 7f55b1600700 1 -- 172.21.15.117:0/1761200861 <== mds.0 v2:172.21.15.183:6838/1439615083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5650aece8530 con 0x5650aee1da40 2024-06-03T03:20:27.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.628+0000 7f55b1600700 10 client.24640 handle_command_reply: tid=0 2024-06-03T03:20:27.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.628+0000 7f55d4f8bb80 1 client.24640 shutdown 2024-06-03T03:20:27.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.629+0000 7f55d4f8bb80 1 -- 172.21.15.117:0/1761200861 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650aed9d930 msgr2=0x5650aedb5fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:20:27.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.629+0000 7f55d4f8bb80 1 --2- 172.21.15.117:0/1761200861 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650aed9d930 0x5650aedb5fb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f55bc009f90 tx=0x7f55bc003960 comp rx=0 tx=0).stop 2024-06-03T03:20:27.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.629+0000 7f55d4f8bb80 1 -- 172.21.15.117:0/1761200861 shutdown_connections 2024-06-03T03:20:27.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.629+0000 7f55d4f8bb80 1 --2- 172.21.15.117:0/1761200861 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x5650aee1da40 0x5650aee1c750 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:27.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.629+0000 7f55d4f8bb80 1 --2- 172.21.15.117:0/1761200861 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650aed9d930 0x5650aedb5fb0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:27.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.629+0000 7f55d4f8bb80 1 --2- 172.21.15.117:0/1761200861 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650aed0adf0 0x5650aedb1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:27.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.629+0000 7f55d4f8bb80 1 --2- 172.21.15.117:0/1761200861 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650aed90910 0x5650aedb1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:27.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.629+0000 7f55b1600700 0 client.24640 ms_handle_reset on v2:172.21.15.183:6838/1439615083 2024-06-03T03:20:27.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.629+0000 7f55d4f8bb80 1 -- 172.21.15.117:0/1761200861 >> 172.21.15.117:0/1761200861 conn(0x5650aecbb120 msgr2=0x5650aed994a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:20:27.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.629+0000 7f55d4f8bb80 1 -- 172.21.15.117:0/1761200861 shutdown_connections 2024-06-03T03:20:27.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.629+0000 7f55d4f8bb80 1 -- 172.21.15.117:0/1761200861 wait complete. 2024-06-03T03:20:27.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.629+0000 7f55d4f8bb80 20 client.24640 trim_cache size 0 max 16384 2024-06-03T03:20:27.635 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-06-03T03:20:27.635 INFO:teuthology.orchestra.run.smithi117.stdout: "cluster_fsid": "a0e8473a-2156-11ef-bc9b-c7b262605968", 2024-06-03T03:20:27.635 INFO:teuthology.orchestra.run.smithi117.stdout: "whoami": 0, 2024-06-03T03:20:27.635 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 24517, 2024-06-03T03:20:27.635 INFO:teuthology.orchestra.run.smithi117.stdout: "want_state": "up:active", 2024-06-03T03:20:27.635 INFO:teuthology.orchestra.run.smithi117.stdout: "state": "up:active", 2024-06-03T03:20:27.635 INFO:teuthology.orchestra.run.smithi117.stdout: "fs_name": "cephfs", 2024-06-03T03:20:27.636 INFO:teuthology.orchestra.run.smithi117.stdout: "rank_uptime": 42.005247838000003, 2024-06-03T03:20:27.636 INFO:teuthology.orchestra.run.smithi117.stdout: "mdsmap_epoch": 17, 2024-06-03T03:20:27.636 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_epoch": 97, 2024-06-03T03:20:27.636 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_epoch_barrier": 95, 2024-06-03T03:20:27.636 INFO:teuthology.orchestra.run.smithi117.stdout: "uptime": 75.186101170000001 2024-06-03T03:20:27.636 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-06-03T03:20:27.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.630+0000 7f55c9000700 1 -- 172.21.15.117:0/1466249862 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f55a80a2c20 msgr2=0x7f55a80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:20:27.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.630+0000 7f55c9000700 1 --2- 172.21.15.117:0/1466249862 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f55a80a2c20 0x7f55a80a50e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f55ac0098e0 tx=0x7f55ac00b610 comp rx=0 tx=0).stop 2024-06-03T03:20:27.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.630+0000 7f55c9000700 1 -- 172.21.15.117:0/1466249862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55c4103580 msgr2=0x7f55c419e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:20:27.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.630+0000 7f55c9000700 1 --2- 172.21.15.117:0/1466249862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55c4103580 0x7f55c419e800 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f55b400c370 tx=0x7f55b400c680 comp rx=0 tx=0).stop 2024-06-03T03:20:27.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.630+0000 7f55c9000700 1 -- 172.21.15.117:0/1466249862 shutdown_connections 2024-06-03T03:20:27.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.630+0000 7f55c9000700 1 --2- 172.21.15.117:0/1466249862 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f55a80a2c20 0x7f55a80a50e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:27.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.630+0000 7f55c9000700 1 --2- 172.21.15.117:0/1466249862 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f55c410c9b0 0x7f55c4198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:27.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.630+0000 7f55c9000700 1 --2- 172.21.15.117:0/1466249862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55c4103580 0x7f55c419e800 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:27.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.630+0000 7f55c9000700 1 --2- 172.21.15.117:0/1466249862 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f55c4103ea0 0x7f55c419ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:27.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.630+0000 7f55c9000700 1 -- 172.21.15.117:0/1466249862 >> 172.21.15.117:0/1466249862 conn(0x7f55c40fcdb0 msgr2=0x7f55c40fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:20:27.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.630+0000 7f55c9000700 1 -- 172.21.15.117:0/1466249862 shutdown_connections 2024-06-03T03:20:27.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:27.630+0000 7f55c9000700 1 -- 172.21.15.117:0/1466249862 wait complete. 2024-06-03T03:20:28.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:27 smithi117 ceph-mon[31649]: pgmap v358: 129 pgs: 3 peering, 126 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:20:28.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:27 smithi183 ceph-mon[37326]: pgmap v358: 129 pgs: 3 peering, 126 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:20:28.335 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:27 smithi151 ceph-mon[37043]: pgmap v358: 129 pgs: 3 peering, 126 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:20:28.556 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-06-03T03:20:28.556 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-06-03T03:20:28.556 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph fs ls 2024-06-03T03:20:28.998 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:20:30.188 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:29 smithi117 ceph-mon[31649]: pgmap v359: 129 pgs: 3 peering, 126 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:20:30.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:29 smithi183 ceph-mon[37326]: pgmap v359: 129 pgs: 3 peering, 126 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:20:30.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:29 smithi151 ceph-mon[37043]: pgmap v359: 129 pgs: 3 peering, 126 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:20:31.403 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:31.400+0000 7ff06c600700 1 -- 172.21.15.117:0/3834038398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff064108500 msgr2=0x7ff0641088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:20:31.404 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:31.400+0000 7ff06c600700 1 --2- 172.21.15.117:0/3834038398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff064108500 0x7ff0641088e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff058009990 tx=0x7ff058009ca0 comp rx=0 tx=0).stop 2024-06-03T03:20:31.404 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:31.400+0000 7ff06c600700 1 -- 172.21.15.117:0/3834038398 shutdown_connections 2024-06-03T03:20:31.404 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:31.400+0000 7ff06c600700 1 --2- 172.21.15.117:0/3834038398 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff06406ab30 0x7ff06410f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:31.404 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:31.400+0000 7ff06c600700 1 --2- 172.21.15.117:0/3834038398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff064108500 0x7ff0641088e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:31.404 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:31.400+0000 7ff06c600700 1 --2- 172.21.15.117:0/3834038398 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff06406a170 0x7ff06406a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:31.404 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:31.400+0000 7ff06c600700 1 -- 172.21.15.117:0/3834038398 >> 172.21.15.117:0/3834038398 conn(0x7ff064078340 msgr2=0x7ff064078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:20:31.404 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:31.400+0000 7ff06c600700 1 -- 172.21.15.117:0/3834038398 shutdown_connections 2024-06-03T03:20:31.404 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:31.400+0000 7ff06c600700 1 -- 172.21.15.117:0/3834038398 wait complete. 2024-06-03T03:20:31.404 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:31.402+0000 7ff06c600700 1 Processor -- start 2024-06-03T03:20:31.404 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:31.402+0000 7ff06c600700 1 -- start start 2024-06-03T03:20:31.406 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:31.402+0000 7ff06c600700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff06406a170 0x7ff06419adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:20:31.406 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:31.402+0000 7ff06c600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff06406ab30 0x7ff06419b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:20:31.406 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:31.402+0000 7ff06c600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff064108500 0x7ff06419f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:20:31.406 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:31.403+0000 7ff06c600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff064112760 con 0x7ff064108500 2024-06-03T03:20:31.406 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:31.403+0000 7ff06c600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff0641125e0 con 0x7ff06406ab30 2024-06-03T03:20:31.406 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:31.403+0000 7ff06c600700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff0641128e0 con 0x7ff06406a170 2024-06-03T03:20:31.406 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:31.403+0000 7ff069c00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff06406a170 0x7ff06419adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:20:31.406 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:31.403+0000 7ff06a600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff064108500 0x7ff06419f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:20:31.406 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:31.403+0000 7ff069c00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff06406a170 0x7ff06419adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:58870/0 (socket says 172.21.15.117:58870) 2024-06-03T03:20:31.406 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:31.403+0000 7ff069c00700 1 -- 172.21.15.117:0/1233083679 learned_addr learned my addr 172.21.15.117:0/1233083679 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:20:31.407 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:31.403+0000 7ff069200700 1 --2- 172.21.15.117:0/1233083679 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff06406ab30 0x7ff06419b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:20:31.407 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:31.403+0000 7ff069c00700 1 -- 172.21.15.117:0/1233083679 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff06406ab30 msgr2=0x7ff06419b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:20:31.407 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:31.403+0000 7ff069c00700 1 --2- 172.21.15.117:0/1233083679 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff06406ab30 0x7ff06419b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:31.407 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:31.403+0000 7ff069c00700 1 -- 172.21.15.117:0/1233083679 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff064108500 msgr2=0x7ff06419f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:20:31.407 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:31.403+0000 7ff069c00700 1 --2- 172.21.15.117:0/1233083679 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff064108500 0x7ff06419f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:31.407 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:31.403+0000 7ff069c00700 1 -- 172.21.15.117:0/1233083679 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff06419fe00 con 0x7ff06406a170 2024-06-03T03:20:31.407 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:31.403+0000 7ff06a600700 1 --2- 172.21.15.117:0/1233083679 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff064108500 0x7ff06419f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:20:31.407 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:31.404+0000 7ff069c00700 1 --2- 172.21.15.117:0/1233083679 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff06406a170 0x7ff06419adf0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff05800bae0 tx=0x7ff05800bb10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:20:31.407 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:31.404+0000 7ff062000700 1 -- 172.21.15.117:0/1233083679 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff058026070 con 0x7ff06406a170 2024-06-03T03:20:31.407 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:31.404+0000 7ff062000700 1 -- 172.21.15.117:0/1233083679 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff058003d70 con 0x7ff06406a170 2024-06-03T03:20:31.407 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:31.404+0000 7ff06c600700 1 -- 172.21.15.117:0/1233083679 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0641a0090 con 0x7ff06406a170 2024-06-03T03:20:31.407 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:31.404+0000 7ff062000700 1 -- 172.21.15.117:0/1233083679 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff05800fe50 con 0x7ff06406a170 2024-06-03T03:20:31.407 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:31.404+0000 7ff06c600700 1 -- 172.21.15.117:0/1233083679 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0641a7970 con 0x7ff06406a170 2024-06-03T03:20:31.408 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:31.405+0000 7ff06c600700 1 -- 172.21.15.117:0/1233083679 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff064069180 con 0x7ff06406a170 2024-06-03T03:20:31.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:31.407+0000 7ff062000700 1 -- 172.21.15.117:0/1233083679 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7ff058029070 con 0x7ff06406a170 2024-06-03T03:20:31.410 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:31.408+0000 7ff062000700 1 --2- 172.21.15.117:0/1233083679 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7ff04c0a2dc0 0x7ff04c0a5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:20:31.411 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:31.409+0000 7ff062000700 1 -- 172.21.15.117:0/1233083679 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7ff0580d03e0 con 0x7ff06406a170 2024-06-03T03:20:31.411 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:31.409+0000 7ff069200700 1 --2- 172.21.15.117:0/1233083679 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7ff04c0a2dc0 0x7ff04c0a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:20:31.411 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:31.410+0000 7ff069200700 1 --2- 172.21.15.117:0/1233083679 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7ff04c0a2dc0 0x7ff04c0a5280 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff050009680 tx=0x7ff050009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:20:31.414 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:31.413+0000 7ff062000700 1 -- 172.21.15.117:0/1233083679 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff058013070 con 0x7ff06406a170 2024-06-03T03:20:31.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:31.690+0000 7ff06c600700 1 -- 172.21.15.117:0/1233083679 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff0641a7b40 con 0x7ff06406a170 2024-06-03T03:20:31.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:31.690+0000 7ff062000700 1 -- 172.21.15.117:0/1233083679 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7ff05809d070 con 0x7ff06406a170 2024-06-03T03:20:31.692 INFO:teuthology.orchestra.run.smithi117.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-06-03T03:20:31.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:31.693+0000 7ff06c600700 1 -- 172.21.15.117:0/1233083679 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7ff04c0a2dc0 msgr2=0x7ff04c0a5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:20:31.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:31.693+0000 7ff06c600700 1 --2- 172.21.15.117:0/1233083679 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7ff04c0a2dc0 0x7ff04c0a5280 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff050009680 tx=0x7ff050009260 comp rx=0 tx=0).stop 2024-06-03T03:20:31.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:31.693+0000 7ff06c600700 1 -- 172.21.15.117:0/1233083679 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff06406a170 msgr2=0x7ff06419adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:20:31.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:31.693+0000 7ff06c600700 1 --2- 172.21.15.117:0/1233083679 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff06406a170 0x7ff06419adf0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff05800bae0 tx=0x7ff05800bb10 comp rx=0 tx=0).stop 2024-06-03T03:20:31.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:31.694+0000 7ff06c600700 1 -- 172.21.15.117:0/1233083679 shutdown_connections 2024-06-03T03:20:31.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:31.694+0000 7ff06c600700 1 --2- 172.21.15.117:0/1233083679 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7ff04c0a2dc0 0x7ff04c0a5280 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:31.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:31.694+0000 7ff06c600700 1 --2- 172.21.15.117:0/1233083679 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff064108500 0x7ff06419f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:31.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:31.694+0000 7ff06c600700 1 --2- 172.21.15.117:0/1233083679 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff06406a170 0x7ff06419adf0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:31.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:31.694+0000 7ff06c600700 1 --2- 172.21.15.117:0/1233083679 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff06406ab30 0x7ff06419b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:31.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:31.694+0000 7ff06c600700 1 -- 172.21.15.117:0/1233083679 >> 172.21.15.117:0/1233083679 conn(0x7ff064078340 msgr2=0x7ff0640ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:20:31.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:31.694+0000 7ff06c600700 1 -- 172.21.15.117:0/1233083679 shutdown_connections 2024-06-03T03:20:31.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:31.694+0000 7ff06c600700 1 -- 172.21.15.117:0/1233083679 wait complete. 2024-06-03T03:20:31.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:31 smithi117 ceph-mon[31649]: pgmap v360: 129 pgs: 2 peering, 127 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-06-03T03:20:31.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:31 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/1233083679' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-03T03:20:32.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:31 smithi183 ceph-mon[37326]: pgmap v360: 129 pgs: 2 peering, 127 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-06-03T03:20:32.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:31 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/1233083679' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-03T03:20:32.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:31 smithi151 ceph-mon[37043]: pgmap v360: 129 pgs: 2 peering, 127 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-06-03T03:20:32.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:31 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/1233083679' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-03T03:20:32.736 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-06-03T03:20:32.736 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-06-03T03:20:32.736 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi117.front.sepia.ceph.com 2024-06-03T03:20:32.736 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-06-03T03:20:32.736 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-06-03T03:20:32.737 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-06-03T03:20:32.737 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-06-03T03:20:32.737 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-06-03T03:20:32.737 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-06-03T03:20:32.737 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T03:20:32.737 DEBUG:teuthology.orchestra.run.smithi117:> ip addr 2024-06-03T03:20:32.792 INFO:teuthology.orchestra.run.smithi117.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-06-03T03:20:32.792 INFO:teuthology.orchestra.run.smithi117.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-06-03T03:20:32.793 INFO:teuthology.orchestra.run.smithi117.stdout: inet 127.0.0.1/8 scope host lo 2024-06-03T03:20:32.793 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft forever preferred_lft forever 2024-06-03T03:20:32.793 INFO:teuthology.orchestra.run.smithi117.stdout: inet6 ::1/128 scope host 2024-06-03T03:20:32.793 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft forever preferred_lft forever 2024-06-03T03:20:32.793 INFO:teuthology.orchestra.run.smithi117.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-03T03:20:32.793 INFO:teuthology.orchestra.run.smithi117.stdout: link/ether 0c:c4:7a:d8:a8:8a brd ff:ff:ff:ff:ff:ff 2024-06-03T03:20:32.793 INFO:teuthology.orchestra.run.smithi117.stdout: altname enp7s0f0 2024-06-03T03:20:32.793 INFO:teuthology.orchestra.run.smithi117.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-03T03:20:32.793 INFO:teuthology.orchestra.run.smithi117.stdout: link/ether 0c:c4:7a:d8:a8:8b brd ff:ff:ff:ff:ff:ff 2024-06-03T03:20:32.793 INFO:teuthology.orchestra.run.smithi117.stdout: altname enp7s0f1 2024-06-03T03:20:32.794 INFO:teuthology.orchestra.run.smithi117.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-03T03:20:32.794 INFO:teuthology.orchestra.run.smithi117.stdout: link/ether 0c:c4:7a:8f:44:62 brd ff:ff:ff:ff:ff:ff 2024-06-03T03:20:32.794 INFO:teuthology.orchestra.run.smithi117.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-06-03T03:20:32.794 INFO:teuthology.orchestra.run.smithi117.stdout: link/ether 0c:c4:7a:8f:44:63 brd ff:ff:ff:ff:ff:ff 2024-06-03T03:20:32.794 INFO:teuthology.orchestra.run.smithi117.stdout: inet 172.21.15.117/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-06-03T03:20:32.794 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft 41964sec preferred_lft 41964sec 2024-06-03T03:20:32.794 INFO:teuthology.orchestra.run.smithi117.stdout: inet6 fe80::ec4:7aff:fe8f:4463/64 scope link 2024-06-03T03:20:32.794 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft forever preferred_lft forever 2024-06-03T03:20:32.795 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-06-03T03:20:32.795 DEBUG:teuthology.orchestra.run.smithi117:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-03T03:20:32.795 DEBUG:teuthology.orchestra.run.smithi117:> set -e 2024-06-03T03:20:32.795 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip link add name ceph-brx type bridge 2024-06-03T03:20:32.795 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip addr flush dev ceph-brx 2024-06-03T03:20:32.795 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip link set ceph-brx up 2024-06-03T03:20:32.795 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-06-03T03:20:32.795 DEBUG:teuthology.orchestra.run.smithi117:> ') 2024-06-03T03:20:32.889 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-03T03:20:33.251 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-03T03:20:33.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T03:20:33.265 DEBUG:teuthology.orchestra.run.smithi117:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-06-03T03:20:33.328 INFO:teuthology.orchestra.run.smithi117.stdout:1 2024-06-03T03:20:33.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T03:20:33.332 DEBUG:teuthology.orchestra.run.smithi117:> route 2024-06-03T03:20:33.379 INFO:teuthology.orchestra.run.smithi117.stdout:Kernel IP routing table 2024-06-03T03:20:33.380 INFO:teuthology.orchestra.run.smithi117.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-06-03T03:20:33.380 INFO:teuthology.orchestra.run.smithi117.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-06-03T03:20:33.380 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-06-03T03:20:33.380 INFO:teuthology.orchestra.run.smithi117.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-06-03T03:20:33.381 DEBUG:teuthology.orchestra.run.smithi117:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-03T03:20:33.381 DEBUG:teuthology.orchestra.run.smithi117:> set -e 2024-06-03T03:20:33.381 DEBUG:teuthology.orchestra.run.smithi117:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-06-03T03:20:33.381 DEBUG:teuthology.orchestra.run.smithi117:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-06-03T03:20:33.381 DEBUG:teuthology.orchestra.run.smithi117:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-06-03T03:20:33.381 DEBUG:teuthology.orchestra.run.smithi117:> ') 2024-06-03T03:20:33.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-03T03:20:34.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:33 smithi117 ceph-mon[31649]: pgmap v361: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-06-03T03:20:34.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-03T03:20:34.220 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T03:20:34.220 DEBUG:teuthology.orchestra.run.smithi117:> ip netns list 2024-06-03T03:20:34.231 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:33 smithi183 ceph-mon[37326]: pgmap v361: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-06-03T03:20:34.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T03:20:34.276 DEBUG:teuthology.orchestra.run.smithi117:> ip netns list-id 2024-06-03T03:20:34.326 DEBUG:teuthology.orchestra.run.smithi117:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-03T03:20:34.326 DEBUG:teuthology.orchestra.run.smithi117:> set -e 2024-06-03T03:20:34.326 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-03T03:20:34.326 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-06-03T03:20:34.326 DEBUG:teuthology.orchestra.run.smithi117:> ') 2024-06-03T03:20:34.409 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:33 smithi151 ceph-mon[37043]: pgmap v361: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-06-03T03:20:34.498 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-03T03:20:34.499 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-03T03:20:34.507 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-06-03T03:20:34.507 DEBUG:teuthology.orchestra.run.smithi117:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-03T03:20:34.507 DEBUG:teuthology.orchestra.run.smithi117:> set -e 2024-06-03T03:20:34.507 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-06-03T03:20:34.507 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-06-03T03:20:34.507 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-06-03T03:20:34.507 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-06-03T03:20:34.508 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-06-03T03:20:34.508 DEBUG:teuthology.orchestra.run.smithi117:> ') 2024-06-03T03:20:34.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-03T03:20:34.799 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-03T03:20:34.809 DEBUG:teuthology.orchestra.run.smithi117:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-03T03:20:34.809 DEBUG:teuthology.orchestra.run.smithi117:> set -e 2024-06-03T03:20:34.809 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip link set brx.0 up 2024-06-03T03:20:34.809 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip link set dev brx.0 master ceph-brx 2024-06-03T03:20:34.809 DEBUG:teuthology.orchestra.run.smithi117:> ') 2024-06-03T03:20:34.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-03T03:20:35.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-03T03:20:35.012 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-06-03T03:20:35.012 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-06-03T03:20:35.012 DEBUG:teuthology.orchestra.run.smithi117:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-06-03T03:20:35.059 INFO:teuthology.orchestra.run.smithi117.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-06-03T03:20:35.059 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-06-03T03:20:35.059 DEBUG:teuthology.orchestra.run.smithi117:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-06-03T03:20:35.104 DEBUG:teuthology.orchestra.run.smithi117:> sudo modprobe fuse 2024-06-03T03:20:35.230 DEBUG:teuthology.orchestra.run.smithi117:> cat /proc/self/mounts | awk '{print $2}' 2024-06-03T03:20:35.286 INFO:teuthology.orchestra.run.smithi117.stdout:/proc 2024-06-03T03:20:35.286 INFO:teuthology.orchestra.run.smithi117.stdout:/sys 2024-06-03T03:20:35.286 INFO:teuthology.orchestra.run.smithi117.stdout:/dev 2024-06-03T03:20:35.286 INFO:teuthology.orchestra.run.smithi117.stdout:/sys/kernel/security 2024-06-03T03:20:35.286 INFO:teuthology.orchestra.run.smithi117.stdout:/dev/shm 2024-06-03T03:20:35.286 INFO:teuthology.orchestra.run.smithi117.stdout:/dev/pts 2024-06-03T03:20:35.286 INFO:teuthology.orchestra.run.smithi117.stdout:/run 2024-06-03T03:20:35.286 INFO:teuthology.orchestra.run.smithi117.stdout:/sys/fs/cgroup 2024-06-03T03:20:35.286 INFO:teuthology.orchestra.run.smithi117.stdout:/sys/fs/pstore 2024-06-03T03:20:35.287 INFO:teuthology.orchestra.run.smithi117.stdout:/sys/fs/bpf 2024-06-03T03:20:35.287 INFO:teuthology.orchestra.run.smithi117.stdout:/ 2024-06-03T03:20:35.287 INFO:teuthology.orchestra.run.smithi117.stdout:/sys/fs/selinux 2024-06-03T03:20:35.287 INFO:teuthology.orchestra.run.smithi117.stdout:/proc/sys/fs/binfmt_misc 2024-06-03T03:20:35.287 INFO:teuthology.orchestra.run.smithi117.stdout:/dev/hugepages 2024-06-03T03:20:35.287 INFO:teuthology.orchestra.run.smithi117.stdout:/dev/mqueue 2024-06-03T03:20:35.287 INFO:teuthology.orchestra.run.smithi117.stdout:/sys/kernel/debug 2024-06-03T03:20:35.287 INFO:teuthology.orchestra.run.smithi117.stdout:/sys/kernel/tracing 2024-06-03T03:20:35.287 INFO:teuthology.orchestra.run.smithi117.stdout:/sys/fs/fuse/connections 2024-06-03T03:20:35.287 INFO:teuthology.orchestra.run.smithi117.stdout:/sys/kernel/config 2024-06-03T03:20:35.287 INFO:teuthology.orchestra.run.smithi117.stdout:/run/credentials/systemd-sysctl.service 2024-06-03T03:20:35.287 INFO:teuthology.orchestra.run.smithi117.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-06-03T03:20:35.287 INFO:teuthology.orchestra.run.smithi117.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-06-03T03:20:35.288 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/nfs/rpc_pipefs 2024-06-03T03:20:35.288 INFO:teuthology.orchestra.run.smithi117.stdout:/run/user/1000 2024-06-03T03:20:35.288 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/ceph 2024-06-03T03:20:35.288 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay 2024-06-03T03:20:35.288 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/e04d43a5b19f3b780ca167f4bef91f12f650f7f9963bc85d2f34d8e864bc563f/merged 2024-06-03T03:20:35.288 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/49aecbe5b1a6a9bac4e30a1c37d239059c0a14b1e08f120c0d08b189b29d8540/merged 2024-06-03T03:20:35.288 INFO:teuthology.orchestra.run.smithi117.stdout:/run/user/0 2024-06-03T03:20:35.288 INFO:teuthology.orchestra.run.smithi117.stdout:/proc/sys/fs/binfmt_misc 2024-06-03T03:20:35.288 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/988becf3e01266ecda231e6268d8dcff387213d36bf0f70ac4171d813d1beb3e/merged 2024-06-03T03:20:35.288 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/b1b4ab210a1eb2a153ba97d3b9c52198533f348cced3914c75c1c3684edc5035/merged 2024-06-03T03:20:35.288 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/22da1ac732882fda7547c910fac99e88a57d1f105276353914a52c72167ff0d1/merged 2024-06-03T03:20:35.288 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/f267d15561608e24f453521b3855dba25a998885a8587e5c23a9b63ae482f1fb/merged 2024-06-03T03:20:35.288 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/06e0c3911621dba29f7c4059876e6ef1f2e14a1cf1e8d7b7ff4435e0d4110957/merged 2024-06-03T03:20:35.288 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/631353c7704a89cf52580563c2a1ed61ea6b7f8a6f75a8ffbb041209507baac8/merged 2024-06-03T03:20:35.289 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/919ad2c37db6bf9bb7a3ad8e4d8fb93bd047080e2b5b61f53d3cdc16db3dc898/merged 2024-06-03T03:20:35.289 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/2cca29e878b378cdb9d06281a8e1d9cd8ad948ec2b6d0b955f99495982a95579/merged 2024-06-03T03:20:35.289 INFO:teuthology.orchestra.run.smithi117.stdout:/run/netns 2024-06-03T03:20:35.289 INFO:teuthology.orchestra.run.smithi117.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-03T03:20:35.289 INFO:teuthology.orchestra.run.smithi117.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-03T03:20:35.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T03:20:35.290 DEBUG:teuthology.orchestra.run.smithi117:> ls /sys/fs/fuse/connections 2024-06-03T03:20:35.344 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-06-03T03:20:35.344 DEBUG:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-06-03T03:20:35.349 DEBUG:teuthology.orchestra.run.smithi117:> sudo modprobe fuse 2024-06-03T03:20:35.421 DEBUG:teuthology.orchestra.run.smithi117:> cat /proc/self/mounts | awk '{print $2}' 2024-06-03T03:20:35.463 INFO:teuthology.orchestra.run.smithi117.stdout:/proc 2024-06-03T03:20:35.463 INFO:teuthology.orchestra.run.smithi117.stdout:/sys 2024-06-03T03:20:35.463 INFO:teuthology.orchestra.run.smithi117.stdout:/dev 2024-06-03T03:20:35.463 INFO:teuthology.orchestra.run.smithi117.stdout:/sys/kernel/security 2024-06-03T03:20:35.463 INFO:teuthology.orchestra.run.smithi117.stdout:/dev/shm 2024-06-03T03:20:35.464 INFO:teuthology.orchestra.run.smithi117.stdout:/dev/pts 2024-06-03T03:20:35.464 INFO:teuthology.orchestra.run.smithi117.stdout:/run 2024-06-03T03:20:35.464 INFO:teuthology.orchestra.run.smithi117.stdout:/sys/fs/cgroup 2024-06-03T03:20:35.464 INFO:teuthology.orchestra.run.smithi117.stdout:/sys/fs/pstore 2024-06-03T03:20:35.464 INFO:teuthology.orchestra.run.smithi117.stdout:/sys/fs/bpf 2024-06-03T03:20:35.464 INFO:teuthology.orchestra.run.smithi117.stdout:/ 2024-06-03T03:20:35.464 INFO:teuthology.orchestra.run.smithi117.stdout:/sys/fs/selinux 2024-06-03T03:20:35.464 INFO:teuthology.orchestra.run.smithi117.stdout:/proc/sys/fs/binfmt_misc 2024-06-03T03:20:35.464 INFO:teuthology.orchestra.run.smithi117.stdout:/dev/hugepages 2024-06-03T03:20:35.464 INFO:teuthology.orchestra.run.smithi117.stdout:/dev/mqueue 2024-06-03T03:20:35.464 INFO:teuthology.orchestra.run.smithi117.stdout:/sys/kernel/debug 2024-06-03T03:20:35.464 INFO:teuthology.orchestra.run.smithi117.stdout:/sys/kernel/tracing 2024-06-03T03:20:35.464 INFO:teuthology.orchestra.run.smithi117.stdout:/sys/fs/fuse/connections 2024-06-03T03:20:35.464 INFO:teuthology.orchestra.run.smithi117.stdout:/sys/kernel/config 2024-06-03T03:20:35.465 INFO:teuthology.orchestra.run.smithi117.stdout:/run/credentials/systemd-sysctl.service 2024-06-03T03:20:35.465 INFO:teuthology.orchestra.run.smithi117.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-06-03T03:20:35.465 INFO:teuthology.orchestra.run.smithi117.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-06-03T03:20:35.465 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/nfs/rpc_pipefs 2024-06-03T03:20:35.465 INFO:teuthology.orchestra.run.smithi117.stdout:/run/user/1000 2024-06-03T03:20:35.465 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/ceph 2024-06-03T03:20:35.465 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay 2024-06-03T03:20:35.465 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/e04d43a5b19f3b780ca167f4bef91f12f650f7f9963bc85d2f34d8e864bc563f/merged 2024-06-03T03:20:35.465 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/49aecbe5b1a6a9bac4e30a1c37d239059c0a14b1e08f120c0d08b189b29d8540/merged 2024-06-03T03:20:35.465 INFO:teuthology.orchestra.run.smithi117.stdout:/run/user/0 2024-06-03T03:20:35.466 INFO:teuthology.orchestra.run.smithi117.stdout:/proc/sys/fs/binfmt_misc 2024-06-03T03:20:35.466 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/988becf3e01266ecda231e6268d8dcff387213d36bf0f70ac4171d813d1beb3e/merged 2024-06-03T03:20:35.466 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/b1b4ab210a1eb2a153ba97d3b9c52198533f348cced3914c75c1c3684edc5035/merged 2024-06-03T03:20:35.466 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/22da1ac732882fda7547c910fac99e88a57d1f105276353914a52c72167ff0d1/merged 2024-06-03T03:20:35.466 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/f267d15561608e24f453521b3855dba25a998885a8587e5c23a9b63ae482f1fb/merged 2024-06-03T03:20:35.466 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/06e0c3911621dba29f7c4059876e6ef1f2e14a1cf1e8d7b7ff4435e0d4110957/merged 2024-06-03T03:20:35.466 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/631353c7704a89cf52580563c2a1ed61ea6b7f8a6f75a8ffbb041209507baac8/merged 2024-06-03T03:20:35.466 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/919ad2c37db6bf9bb7a3ad8e4d8fb93bd047080e2b5b61f53d3cdc16db3dc898/merged 2024-06-03T03:20:35.466 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/2cca29e878b378cdb9d06281a8e1d9cd8ad948ec2b6d0b955f99495982a95579/merged 2024-06-03T03:20:35.466 INFO:teuthology.orchestra.run.smithi117.stdout:/run/netns 2024-06-03T03:20:35.466 INFO:teuthology.orchestra.run.smithi117.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-03T03:20:35.466 INFO:teuthology.orchestra.run.smithi117.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-03T03:20:35.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T03:20:35.467 DEBUG:teuthology.orchestra.run.smithi117:> ls /sys/fs/fuse/connections 2024-06-03T03:20:35.567 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi117.stderr:2024-06-03T03:20:35.565+0000 7f4403e5f480 -1 init, newargv = 0x55faf0d185e0 newargc=15 2024-06-03T03:20:35.567 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi117.stderr:ceph-fuse[74969]: starting ceph client 2024-06-03T03:20:35.577 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi117.stderr:ceph-fuse[74969]: starting fuse 2024-06-03T03:20:36.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:35 smithi117 ceph-mon[31649]: pgmap v362: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-06-03T03:20:36.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:35 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:20:36.197 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:35 smithi151 ceph-mon[37043]: pgmap v362: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-06-03T03:20:36.197 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:35 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:20:36.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:35 smithi183 ceph-mon[37326]: pgmap v362: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-06-03T03:20:36.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:35 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:20:36.648 DEBUG:teuthology.orchestra.run.smithi117:> sudo modprobe fuse 2024-06-03T03:20:36.734 DEBUG:teuthology.orchestra.run.smithi117:> cat /proc/self/mounts | awk '{print $2}' 2024-06-03T03:20:36.780 INFO:teuthology.orchestra.run.smithi117.stdout:/proc 2024-06-03T03:20:36.780 INFO:teuthology.orchestra.run.smithi117.stdout:/sys 2024-06-03T03:20:36.780 INFO:teuthology.orchestra.run.smithi117.stdout:/dev 2024-06-03T03:20:36.780 INFO:teuthology.orchestra.run.smithi117.stdout:/sys/kernel/security 2024-06-03T03:20:36.780 INFO:teuthology.orchestra.run.smithi117.stdout:/dev/shm 2024-06-03T03:20:36.781 INFO:teuthology.orchestra.run.smithi117.stdout:/dev/pts 2024-06-03T03:20:36.781 INFO:teuthology.orchestra.run.smithi117.stdout:/run 2024-06-03T03:20:36.781 INFO:teuthology.orchestra.run.smithi117.stdout:/sys/fs/cgroup 2024-06-03T03:20:36.781 INFO:teuthology.orchestra.run.smithi117.stdout:/sys/fs/pstore 2024-06-03T03:20:36.781 INFO:teuthology.orchestra.run.smithi117.stdout:/sys/fs/bpf 2024-06-03T03:20:36.781 INFO:teuthology.orchestra.run.smithi117.stdout:/ 2024-06-03T03:20:36.781 INFO:teuthology.orchestra.run.smithi117.stdout:/sys/fs/selinux 2024-06-03T03:20:36.781 INFO:teuthology.orchestra.run.smithi117.stdout:/proc/sys/fs/binfmt_misc 2024-06-03T03:20:36.781 INFO:teuthology.orchestra.run.smithi117.stdout:/dev/hugepages 2024-06-03T03:20:36.781 INFO:teuthology.orchestra.run.smithi117.stdout:/dev/mqueue 2024-06-03T03:20:36.781 INFO:teuthology.orchestra.run.smithi117.stdout:/sys/kernel/debug 2024-06-03T03:20:36.781 INFO:teuthology.orchestra.run.smithi117.stdout:/sys/kernel/tracing 2024-06-03T03:20:36.782 INFO:teuthology.orchestra.run.smithi117.stdout:/sys/fs/fuse/connections 2024-06-03T03:20:36.782 INFO:teuthology.orchestra.run.smithi117.stdout:/sys/kernel/config 2024-06-03T03:20:36.782 INFO:teuthology.orchestra.run.smithi117.stdout:/run/credentials/systemd-sysctl.service 2024-06-03T03:20:36.782 INFO:teuthology.orchestra.run.smithi117.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-06-03T03:20:36.782 INFO:teuthology.orchestra.run.smithi117.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-06-03T03:20:36.782 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/nfs/rpc_pipefs 2024-06-03T03:20:36.782 INFO:teuthology.orchestra.run.smithi117.stdout:/run/user/1000 2024-06-03T03:20:36.782 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/ceph 2024-06-03T03:20:36.782 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay 2024-06-03T03:20:36.782 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/e04d43a5b19f3b780ca167f4bef91f12f650f7f9963bc85d2f34d8e864bc563f/merged 2024-06-03T03:20:36.782 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/49aecbe5b1a6a9bac4e30a1c37d239059c0a14b1e08f120c0d08b189b29d8540/merged 2024-06-03T03:20:36.783 INFO:teuthology.orchestra.run.smithi117.stdout:/run/user/0 2024-06-03T03:20:36.783 INFO:teuthology.orchestra.run.smithi117.stdout:/proc/sys/fs/binfmt_misc 2024-06-03T03:20:36.783 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/988becf3e01266ecda231e6268d8dcff387213d36bf0f70ac4171d813d1beb3e/merged 2024-06-03T03:20:36.783 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/b1b4ab210a1eb2a153ba97d3b9c52198533f348cced3914c75c1c3684edc5035/merged 2024-06-03T03:20:36.783 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/22da1ac732882fda7547c910fac99e88a57d1f105276353914a52c72167ff0d1/merged 2024-06-03T03:20:36.783 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/f267d15561608e24f453521b3855dba25a998885a8587e5c23a9b63ae482f1fb/merged 2024-06-03T03:20:36.783 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/06e0c3911621dba29f7c4059876e6ef1f2e14a1cf1e8d7b7ff4435e0d4110957/merged 2024-06-03T03:20:36.783 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/631353c7704a89cf52580563c2a1ed61ea6b7f8a6f75a8ffbb041209507baac8/merged 2024-06-03T03:20:36.783 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/919ad2c37db6bf9bb7a3ad8e4d8fb93bd047080e2b5b61f53d3cdc16db3dc898/merged 2024-06-03T03:20:36.783 INFO:teuthology.orchestra.run.smithi117.stdout:/var/lib/containers/storage/overlay/2cca29e878b378cdb9d06281a8e1d9cd8ad948ec2b6d0b955f99495982a95579/merged 2024-06-03T03:20:36.783 INFO:teuthology.orchestra.run.smithi117.stdout:/run/netns 2024-06-03T03:20:36.783 INFO:teuthology.orchestra.run.smithi117.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-03T03:20:36.783 INFO:teuthology.orchestra.run.smithi117.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-03T03:20:36.783 INFO:teuthology.orchestra.run.smithi117.stdout:/home/ubuntu/cephtest/mnt.admin 2024-06-03T03:20:36.785 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T03:20:36.785 DEBUG:teuthology.orchestra.run.smithi117:> ls /sys/fs/fuse/connections 2024-06-03T03:20:36.877 INFO:teuthology.orchestra.run.smithi117.stdout:119 2024-06-03T03:20:36.877 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [119] 2024-06-03T03:20:36.877 DEBUG:teuthology.orchestra.run.smithi117:> sudo stdin-killer -- python3 -c ' 2024-06-03T03:20:36.878 DEBUG:teuthology.orchestra.run.smithi117:> import glob 2024-06-03T03:20:36.878 DEBUG:teuthology.orchestra.run.smithi117:> import re 2024-06-03T03:20:36.878 DEBUG:teuthology.orchestra.run.smithi117:> import os 2024-06-03T03:20:36.878 DEBUG:teuthology.orchestra.run.smithi117:> import subprocess 2024-06-03T03:20:36.878 DEBUG:teuthology.orchestra.run.smithi117:> 2024-06-03T03:20:36.878 DEBUG:teuthology.orchestra.run.smithi117:> def _find_admin_socket(client_name): 2024-06-03T03:20:36.878 DEBUG:teuthology.orchestra.run.smithi117:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-06-03T03:20:36.878 DEBUG:teuthology.orchestra.run.smithi117:> files = glob.glob(asok_path) 2024-06-03T03:20:36.878 DEBUG:teuthology.orchestra.run.smithi117:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-06-03T03:20:36.879 DEBUG:teuthology.orchestra.run.smithi117:> 2024-06-03T03:20:36.879 DEBUG:teuthology.orchestra.run.smithi117:> # Given a non-glob path, it better be there 2024-06-03T03:20:36.879 DEBUG:teuthology.orchestra.run.smithi117:> if "*" not in asok_path: 2024-06-03T03:20:36.879 DEBUG:teuthology.orchestra.run.smithi117:> assert(len(files) == 1) 2024-06-03T03:20:36.879 DEBUG:teuthology.orchestra.run.smithi117:> return files[0] 2024-06-03T03:20:36.879 DEBUG:teuthology.orchestra.run.smithi117:> 2024-06-03T03:20:36.879 DEBUG:teuthology.orchestra.run.smithi117:> for f in files: 2024-06-03T03:20:36.879 DEBUG:teuthology.orchestra.run.smithi117:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-06-03T03:20:36.879 DEBUG:teuthology.orchestra.run.smithi117:> if os.path.exists("/proc/{0}".format(pid)): 2024-06-03T03:20:36.879 DEBUG:teuthology.orchestra.run.smithi117:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-06-03T03:20:36.879 DEBUG:teuthology.orchestra.run.smithi117:> contents = proc_f.read() 2024-06-03T03:20:36.880 DEBUG:teuthology.orchestra.run.smithi117:> if mountpoint in contents: 2024-06-03T03:20:36.880 DEBUG:teuthology.orchestra.run.smithi117:> return f 2024-06-03T03:20:36.880 DEBUG:teuthology.orchestra.run.smithi117:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-06-03T03:20:36.880 DEBUG:teuthology.orchestra.run.smithi117:> 2024-06-03T03:20:36.880 DEBUG:teuthology.orchestra.run.smithi117:> print(_find_admin_socket("client.admin")) 2024-06-03T03:20:36.880 DEBUG:teuthology.orchestra.run.smithi117:> ' 2024-06-03T03:20:37.045 INFO:teuthology.orchestra.run.smithi117.stdout:/var/run/ceph/ceph-client.admin.74969.asok 2024-06-03T03:20:37.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-03T03:20:37.062 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.74969.asok 2024-06-03T03:20:37.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T03:20:37.062 DEBUG:teuthology.orchestra.run.smithi117:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.74969.asok status 2024-06-03T03:20:37.279 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-06-03T03:20:37.279 INFO:teuthology.orchestra.run.smithi117.stdout: "metadata": { 2024-06-03T03:20:37.279 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_sha1": "3d89accfe3201286f000e065a1faea2a88c7a6ba", 2024-06-03T03:20:37.279 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_version": "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)", 2024-06-03T03:20:37.279 INFO:teuthology.orchestra.run.smithi117.stdout: "entity_id": "admin", 2024-06-03T03:20:37.279 INFO:teuthology.orchestra.run.smithi117.stdout: "hostname": "smithi117", 2024-06-03T03:20:37.279 INFO:teuthology.orchestra.run.smithi117.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-06-03T03:20:37.279 INFO:teuthology.orchestra.run.smithi117.stdout: "pid": "74969", 2024-06-03T03:20:37.279 INFO:teuthology.orchestra.run.smithi117.stdout: "root": "/" 2024-06-03T03:20:37.279 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:20:37.280 INFO:teuthology.orchestra.run.smithi117.stdout: "dentry_count": 0, 2024-06-03T03:20:37.280 INFO:teuthology.orchestra.run.smithi117.stdout: "dentry_pinned_count": 0, 2024-06-03T03:20:37.280 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 24620, 2024-06-03T03:20:37.280 INFO:teuthology.orchestra.run.smithi117.stdout: "inst": { 2024-06-03T03:20:37.280 INFO:teuthology.orchestra.run.smithi117.stdout: "name": { 2024-06-03T03:20:37.280 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "client", 2024-06-03T03:20:37.280 INFO:teuthology.orchestra.run.smithi117.stdout: "num": 24620 2024-06-03T03:20:37.280 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:20:37.280 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": { 2024-06-03T03:20:37.280 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v1", 2024-06-03T03:20:37.280 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.117:0", 2024-06-03T03:20:37.280 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 2910494575 2024-06-03T03:20:37.280 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:20:37.281 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:20:37.281 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": { 2024-06-03T03:20:37.281 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v1", 2024-06-03T03:20:37.281 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.117:0", 2024-06-03T03:20:37.281 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 2910494575 2024-06-03T03:20:37.281 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:20:37.281 INFO:teuthology.orchestra.run.smithi117.stdout: "inst_str": "client.24620 172.21.15.117:0/2910494575", 2024-06-03T03:20:37.281 INFO:teuthology.orchestra.run.smithi117.stdout: "addr_str": "172.21.15.117:0/2910494575", 2024-06-03T03:20:37.281 INFO:teuthology.orchestra.run.smithi117.stdout: "inode_count": 1, 2024-06-03T03:20:37.281 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_epoch": 17, 2024-06-03T03:20:37.281 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_epoch": 97, 2024-06-03T03:20:37.281 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_epoch_barrier": 0, 2024-06-03T03:20:37.281 INFO:teuthology.orchestra.run.smithi117.stdout: "blocklisted": false, 2024-06-03T03:20:37.281 INFO:teuthology.orchestra.run.smithi117.stdout: "fs_name": "cephfs" 2024-06-03T03:20:37.281 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-06-03T03:20:37.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T03:20:37.293 DEBUG:teuthology.orchestra.run.smithi117:> stat --file-system '--printf=%T 2024-06-03T03:20:37.293 DEBUG:teuthology.orchestra.run.smithi117:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-06-03T03:20:37.336 INFO:teuthology.orchestra.run.smithi117.stdout:fuseblk 2024-06-03T03:20:37.336 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-06-03T03:20:37.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T03:20:37.337 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-06-03T03:20:37.442 DEBUG:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-06-03T03:20:37.533 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:37 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-06-03T03:20:37.559 INFO:teuthology.orchestra.run.smithi117.stdout:# file: . 2024-06-03T03:20:37.559 INFO:teuthology.orchestra.run.smithi117.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-06-03T03:20:37.559 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T03:20:37.560 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-03T03:20:37.568 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-06-03T03:20:37.569 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-03T03:20:37.614 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph osd blocklist ls 2024-06-03T03:20:38.087 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:20:38.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:37 smithi117 ceph-mon[31649]: pgmap v363: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-06-03T03:20:38.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:37 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:20:38.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:37 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:20:38.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:37 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:20:38.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:37 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:20:38.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:37 smithi183 ceph-mon[37326]: pgmap v363: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-06-03T03:20:38.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:37 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:20:38.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:37 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:20:38.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:37 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:20:38.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:37 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:20:38.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:37 smithi151 ceph-mon[37043]: pgmap v363: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-06-03T03:20:38.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:37 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:20:38.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:37 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:20:38.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:37 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:20:38.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:37 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:20:39.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:39 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:20:39.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:39 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:20:39.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:39 smithi117 ceph-mon[31649]: pgmap v364: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-06-03T03:20:39.336 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:39 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:20:39.336 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:39 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:20:39.336 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:39 smithi151 ceph-mon[37043]: pgmap v364: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-06-03T03:20:39.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:39 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:20:39.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:39 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:20:39.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:39 smithi183 ceph-mon[37326]: pgmap v364: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-06-03T03:20:40.125 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:40 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:20:40.125 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:40 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:20:40.125 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:40 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:20:40.383 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:40 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:20:40.383 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:40 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:20:40.383 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:40 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:20:40.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:40 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:20:40.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:40 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:20:40.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:40 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:20:40.679 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:40.677+0000 7f381c000700 1 -- 172.21.15.117:0/1918331632 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f38141073a0 msgr2=0x7f3814107780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:20:40.679 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:40.677+0000 7f381c000700 1 --2- 172.21.15.117:0/1918331632 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f38141073a0 0x7f3814107780 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3808009990 tx=0x7f3808009ca0 comp rx=0 tx=0).stop 2024-06-03T03:20:40.679 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:40.677+0000 7f381c000700 1 -- 172.21.15.117:0/1918331632 shutdown_connections 2024-06-03T03:20:40.680 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:40.677+0000 7f381c000700 1 --2- 172.21.15.117:0/1918331632 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3814102f10 0x7f381410f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:40.680 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:40.677+0000 7f381c000700 1 --2- 172.21.15.117:0/1918331632 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f38141073a0 0x7f3814107780 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:40.680 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:40.677+0000 7f381c000700 1 --2- 172.21.15.117:0/1918331632 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3814102550 0x7f38141029d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:40.680 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:40.677+0000 7f381c000700 1 -- 172.21.15.117:0/1918331632 >> 172.21.15.117:0/1918331632 conn(0x7f3814079650 msgr2=0x7f3814079a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:20:40.680 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:40.678+0000 7f381c000700 1 -- 172.21.15.117:0/1918331632 shutdown_connections 2024-06-03T03:20:40.680 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:40.678+0000 7f381c000700 1 -- 172.21.15.117:0/1918331632 wait complete. 2024-06-03T03:20:40.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:40.679+0000 7f381c000700 1 Processor -- start 2024-06-03T03:20:40.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:40.679+0000 7f381c000700 1 -- start start 2024-06-03T03:20:40.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:40.679+0000 7f381c000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3814102550 0x7f3814074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:20:40.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:40.679+0000 7f381c000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3814102f10 0x7f3814074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:20:40.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:40.679+0000 7f381c000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38141073a0 0x7f3814075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:20:40.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:40.679+0000 7f381c000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f38141127a0 con 0x7f38141073a0 2024-06-03T03:20:40.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:40.679+0000 7f381c000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3814112620 con 0x7f3814102f10 2024-06-03T03:20:40.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:40.679+0000 7f381c000700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3814112920 con 0x7f3814102550 2024-06-03T03:20:40.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:40.679+0000 7f3819600700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3814102550 0x7f3814074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:20:40.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:40.680+0000 7f3819600700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3814102550 0x7f3814074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:39892/0 (socket says 172.21.15.117:39892) 2024-06-03T03:20:40.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:40.680+0000 7f3819600700 1 -- 172.21.15.117:0/907776401 learned_addr learned my addr 172.21.15.117:0/907776401 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:20:40.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:40.680+0000 7f3818c00700 1 --2- 172.21.15.117:0/907776401 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3814102f10 0x7f3814074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:20:40.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:40.680+0000 7f381a000700 1 --2- 172.21.15.117:0/907776401 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38141073a0 0x7f3814075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:20:40.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:40.680+0000 7f3819600700 1 -- 172.21.15.117:0/907776401 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3814102f10 msgr2=0x7f3814074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:20:40.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:40.680+0000 7f3819600700 1 --2- 172.21.15.117:0/907776401 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3814102f10 0x7f3814074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:40.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:40.680+0000 7f3819600700 1 -- 172.21.15.117:0/907776401 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38141073a0 msgr2=0x7f3814075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:20:40.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:40.680+0000 7f3819600700 1 --2- 172.21.15.117:0/907776401 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38141073a0 0x7f3814075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:40.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:40.680+0000 7f3819600700 1 -- 172.21.15.117:0/907776401 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38141a7130 con 0x7f3814102550 2024-06-03T03:20:40.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:40.680+0000 7f3819600700 1 --2- 172.21.15.117:0/907776401 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3814102550 0x7f3814074790 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3808005910 tx=0x7f38080047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:20:40.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:40.680+0000 7f3812000700 1 -- 172.21.15.117:0/907776401 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3808027070 con 0x7f3814102550 2024-06-03T03:20:40.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:40.680+0000 7f381c000700 1 -- 172.21.15.117:0/907776401 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38141a73c0 con 0x7f3814102550 2024-06-03T03:20:40.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:40.680+0000 7f3812000700 1 -- 172.21.15.117:0/907776401 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f380800fde0 con 0x7f3814102550 2024-06-03T03:20:40.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:40.681+0000 7f3812000700 1 -- 172.21.15.117:0/907776401 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38080173f0 con 0x7f3814102550 2024-06-03T03:20:40.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:40.681+0000 7f381c000700 1 -- 172.21.15.117:0/907776401 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38141a7810 con 0x7f3814102550 2024-06-03T03:20:40.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:40.682+0000 7f381c000700 1 -- 172.21.15.117:0/907776401 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3814103b80 con 0x7f3814102550 2024-06-03T03:20:40.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:40.684+0000 7f3812000700 1 -- 172.21.15.117:0/907776401 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f380802a070 con 0x7f3814102550 2024-06-03T03:20:40.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:40.685+0000 7f3812000700 1 --2- 172.21.15.117:0/907776401 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f37fc0a2c20 0x7f37fc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:20:40.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:40.685+0000 7f3812000700 1 -- 172.21.15.117:0/907776401 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7f38080d03b0 con 0x7f3814102550 2024-06-03T03:20:40.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:40.685+0000 7f3818c00700 1 --2- 172.21.15.117:0/907776401 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f37fc0a2c20 0x7f37fc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:20:40.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:40.686+0000 7f3818c00700 1 --2- 172.21.15.117:0/907776401 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f37fc0a2c20 0x7f37fc0a50e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3800009680 tx=0x7f3800009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:20:40.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:40.688+0000 7f3812000700 1 -- 172.21.15.117:0/907776401 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3808013070 con 0x7f3814102550 2024-06-03T03:20:40.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:40.933+0000 7f381c000700 1 -- 172.21.15.117:0/907776401 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f381402dca0 con 0x7f3814102550 2024-06-03T03:20:40.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:40.933+0000 7f3812000700 1 -- 172.21.15.117:0/907776401 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v97) ==== 81+0+601 (secure 0 0 0) 0x7f380809d040 con 0x7f3814102550 2024-06-03T03:20:40.935 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:0/186309008 2024-06-04T03:11:24.655516+0000 2024-06-03T03:20:40.935 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:6801/2142313590 2024-06-04T03:11:24.655516+0000 2024-06-03T03:20:40.935 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:6800/2142313590 2024-06-04T03:11:24.655516+0000 2024-06-03T03:20:40.935 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:0/2399725156 2024-06-04T03:11:24.655516+0000 2024-06-03T03:20:40.935 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:0/1453232061 2024-06-04T03:10:50.552899+0000 2024-06-03T03:20:40.935 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:0/4281674974 2024-06-04T03:10:50.552899+0000 2024-06-03T03:20:40.935 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:6800/2424554985 2024-06-04T03:10:50.552899+0000 2024-06-03T03:20:40.936 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:0/3946365225 2024-06-04T03:11:24.655516+0000 2024-06-03T03:20:40.936 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:0/1146162354 2024-06-04T03:10:50.552899+0000 2024-06-03T03:20:40.936 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:6801/2424554985 2024-06-04T03:10:50.552899+0000 2024-06-03T03:20:40.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:40.937+0000 7f381c000700 1 -- 172.21.15.117:0/907776401 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f37fc0a2c20 msgr2=0x7f37fc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:20:40.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:40.937+0000 7f381c000700 1 --2- 172.21.15.117:0/907776401 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f37fc0a2c20 0x7f37fc0a50e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3800009680 tx=0x7f3800009260 comp rx=0 tx=0).stop 2024-06-03T03:20:40.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:40.937+0000 7f381c000700 1 -- 172.21.15.117:0/907776401 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3814102550 msgr2=0x7f3814074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:20:40.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:40.937+0000 7f381c000700 1 --2- 172.21.15.117:0/907776401 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3814102550 0x7f3814074790 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3808005910 tx=0x7f38080047b0 comp rx=0 tx=0).stop 2024-06-03T03:20:40.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:40.937+0000 7f381c000700 1 -- 172.21.15.117:0/907776401 shutdown_connections 2024-06-03T03:20:40.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:40.937+0000 7f381c000700 1 --2- 172.21.15.117:0/907776401 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f37fc0a2c20 0x7f37fc0a50e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:40.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:40.937+0000 7f381c000700 1 --2- 172.21.15.117:0/907776401 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38141073a0 0x7f3814075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:40.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:40.937+0000 7f381c000700 1 --2- 172.21.15.117:0/907776401 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3814102550 0x7f3814074790 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:40.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:40.937+0000 7f381c000700 1 --2- 172.21.15.117:0/907776401 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3814102f10 0x7f3814074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:40.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:40.937+0000 7f381c000700 1 -- 172.21.15.117:0/907776401 >> 172.21.15.117:0/907776401 conn(0x7f3814079650 msgr2=0x7f38141005e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:20:40.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:40.937+0000 7f381c000700 1 -- 172.21.15.117:0/907776401 shutdown_connections 2024-06-03T03:20:40.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:40.937+0000 7f381c000700 1 -- 172.21.15.117:0/907776401 wait complete. 2024-06-03T03:20:40.940 INFO:teuthology.orchestra.run.smithi117.stderr:listed 10 entries 2024-06-03T03:20:41.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:41 smithi117 ceph-mon[31649]: pgmap v365: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-06-03T03:20:41.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:41 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/907776401' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-03T03:20:41.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:41 smithi151 ceph-mon[37043]: pgmap v365: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-06-03T03:20:41.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:41 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/907776401' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-03T03:20:41.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:41 smithi183 ceph-mon[37326]: pgmap v365: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-06-03T03:20:41.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:41 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/907776401' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-03T03:20:41.727 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-06-03T03:20:41.727 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-03T03:20:41.782 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph osd blocklist ls 2024-06-03T03:20:42.242 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:20:44.133 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:43 smithi117 ceph-mon[31649]: pgmap v366: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-06-03T03:20:44.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:43 smithi151 ceph-mon[37043]: pgmap v366: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-06-03T03:20:44.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:43 smithi183 ceph-mon[37326]: pgmap v366: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-06-03T03:20:44.465 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:44.463+0000 7ff01c600700 1 -- 172.21.15.117:0/3080603520 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff01406ab10 msgr2=0x7ff01410f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:20:44.465 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:44.463+0000 7ff01c600700 1 --2- 172.21.15.117:0/3080603520 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff01406ab10 0x7ff01410f320 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff00c00b0d0 tx=0x7ff00c00b3e0 comp rx=0 tx=0).stop 2024-06-03T03:20:44.465 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:44.463+0000 7ff01c600700 1 -- 172.21.15.117:0/3080603520 shutdown_connections 2024-06-03T03:20:44.465 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:44.463+0000 7ff01c600700 1 --2- 172.21.15.117:0/3080603520 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff01406ab10 0x7ff01410f320 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:44.465 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:44.463+0000 7ff01c600700 1 --2- 172.21.15.117:0/3080603520 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff0141084e0 0x7ff0141088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:44.465 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:44.463+0000 7ff01c600700 1 --2- 172.21.15.117:0/3080603520 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff01406a170 0x7ff01406a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:44.466 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:44.463+0000 7ff01c600700 1 -- 172.21.15.117:0/3080603520 >> 172.21.15.117:0/3080603520 conn(0x7ff0140782c0 msgr2=0x7ff0140786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:20:44.466 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:44.464+0000 7ff01c600700 1 -- 172.21.15.117:0/3080603520 shutdown_connections 2024-06-03T03:20:44.466 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:44.464+0000 7ff01c600700 1 -- 172.21.15.117:0/3080603520 wait complete. 2024-06-03T03:20:44.468 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:44.465+0000 7ff01c600700 1 Processor -- start 2024-06-03T03:20:44.468 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:44.465+0000 7ff01c600700 1 -- start start 2024-06-03T03:20:44.468 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:44.465+0000 7ff01c600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff01406a170 0x7ff01419e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:20:44.468 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:44.465+0000 7ff01c600700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff01406ab10 0x7ff01419ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:20:44.468 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:44.465+0000 7ff01c600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0141084e0 0x7ff014198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:20:44.468 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:44.465+0000 7ff01c600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff0141127b0 con 0x7ff0141084e0 2024-06-03T03:20:44.468 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:44.465+0000 7ff01c600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff014112630 con 0x7ff01406a170 2024-06-03T03:20:44.468 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:44.465+0000 7ff01c600700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff014112930 con 0x7ff01406ab10 2024-06-03T03:20:44.468 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:44.466+0000 7ff01a600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0141084e0 0x7ff014198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:20:44.468 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:44.466+0000 7ff01a600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0141084e0 0x7ff014198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:38498/0 (socket says 172.21.15.117:38498) 2024-06-03T03:20:44.468 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:44.466+0000 7ff019c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff01406a170 0x7ff01419e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:20:44.469 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:44.466+0000 7ff01a600700 1 -- 172.21.15.117:0/2137854836 learned_addr learned my addr 172.21.15.117:0/2137854836 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:20:44.469 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:44.466+0000 7ff019200700 1 --2- 172.21.15.117:0/2137854836 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff01406ab10 0x7ff01419ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:20:44.469 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:44.466+0000 7ff01a600700 1 -- 172.21.15.117:0/2137854836 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff01406ab10 msgr2=0x7ff01419ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:20:44.469 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:44.466+0000 7ff01a600700 1 --2- 172.21.15.117:0/2137854836 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff01406ab10 0x7ff01419ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:44.469 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:44.466+0000 7ff01a600700 1 -- 172.21.15.117:0/2137854836 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff01406a170 msgr2=0x7ff01419e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:20:44.469 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:44.466+0000 7ff01a600700 1 --2- 172.21.15.117:0/2137854836 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff01406a170 0x7ff01419e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:44.469 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:44.466+0000 7ff01a600700 1 -- 172.21.15.117:0/2137854836 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff014199170 con 0x7ff0141084e0 2024-06-03T03:20:44.469 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:44.466+0000 7ff019c00700 1 --2- 172.21.15.117:0/2137854836 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff01406a170 0x7ff01419e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:20:44.469 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:44.466+0000 7ff01a600700 1 --2- 172.21.15.117:0/2137854836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0141084e0 0x7ff014198a10 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff00c00d010 tx=0x7ff00c0079a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:20:44.469 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:44.467+0000 7ff012000700 1 -- 172.21.15.117:0/2137854836 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff00c003c00 con 0x7ff0141084e0 2024-06-03T03:20:44.469 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:44.467+0000 7ff01c600700 1 -- 172.21.15.117:0/2137854836 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff014199400 con 0x7ff0141084e0 2024-06-03T03:20:44.469 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:44.467+0000 7ff012000700 1 -- 172.21.15.117:0/2137854836 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff00c022070 con 0x7ff0141084e0 2024-06-03T03:20:44.469 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:44.467+0000 7ff012000700 1 -- 172.21.15.117:0/2137854836 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff00c01b6c0 con 0x7ff0141084e0 2024-06-03T03:20:44.469 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:44.467+0000 7ff01c600700 1 -- 172.21.15.117:0/2137854836 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0141a56d0 con 0x7ff0141084e0 2024-06-03T03:20:44.470 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:44.468+0000 7ff01c600700 1 -- 172.21.15.117:0/2137854836 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff014069180 con 0x7ff0141084e0 2024-06-03T03:20:44.471 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:44.470+0000 7ff012000700 1 -- 172.21.15.117:0/2137854836 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7ff00c009ba0 con 0x7ff0141084e0 2024-06-03T03:20:44.472 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:44.471+0000 7ff012000700 1 --2- 172.21.15.117:0/2137854836 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7feffc0a2c20 0x7feffc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:20:44.472 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:44.472+0000 7ff012000700 1 -- 172.21.15.117:0/2137854836 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7ff00c00f020 con 0x7ff0141084e0 2024-06-03T03:20:44.472 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:44.472+0000 7ff019c00700 1 --2- 172.21.15.117:0/2137854836 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7feffc0a2c20 0x7feffc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:20:44.473 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:44.472+0000 7ff019c00700 1 --2- 172.21.15.117:0/2137854836 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7feffc0a2c20 0x7feffc0a50e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff0141012a0 tx=0x7ff008009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:20:44.477 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:44.476+0000 7ff012000700 1 -- 172.21.15.117:0/2137854836 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff00c09cda0 con 0x7ff0141084e0 2024-06-03T03:20:44.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:44.717+0000 7ff01c600700 1 -- 172.21.15.117:0/2137854836 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff01402dca0 con 0x7ff0141084e0 2024-06-03T03:20:44.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:44.718+0000 7ff012000700 1 -- 172.21.15.117:0/2137854836 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v97) ==== 81+0+601 (secure 0 0 0) 0x7ff00c09c6a0 con 0x7ff0141084e0 2024-06-03T03:20:44.719 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:0/186309008 2024-06-04T03:11:24.655516+0000 2024-06-03T03:20:44.719 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:6801/2142313590 2024-06-04T03:11:24.655516+0000 2024-06-03T03:20:44.719 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:6800/2142313590 2024-06-04T03:11:24.655516+0000 2024-06-03T03:20:44.719 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:0/2399725156 2024-06-04T03:11:24.655516+0000 2024-06-03T03:20:44.719 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:0/1453232061 2024-06-04T03:10:50.552899+0000 2024-06-03T03:20:44.719 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:0/4281674974 2024-06-04T03:10:50.552899+0000 2024-06-03T03:20:44.719 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:6800/2424554985 2024-06-04T03:10:50.552899+0000 2024-06-03T03:20:44.719 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:0/3946365225 2024-06-04T03:11:24.655516+0000 2024-06-03T03:20:44.719 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:0/1146162354 2024-06-04T03:10:50.552899+0000 2024-06-03T03:20:44.719 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:6801/2424554985 2024-06-04T03:10:50.552899+0000 2024-06-03T03:20:44.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:44.721+0000 7ff01c600700 1 -- 172.21.15.117:0/2137854836 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7feffc0a2c20 msgr2=0x7feffc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:20:44.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:44.721+0000 7ff01c600700 1 --2- 172.21.15.117:0/2137854836 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7feffc0a2c20 0x7feffc0a50e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff0141012a0 tx=0x7ff008009260 comp rx=0 tx=0).stop 2024-06-03T03:20:44.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:44.721+0000 7ff01c600700 1 -- 172.21.15.117:0/2137854836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0141084e0 msgr2=0x7ff014198a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:20:44.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:44.721+0000 7ff01c600700 1 --2- 172.21.15.117:0/2137854836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0141084e0 0x7ff014198a10 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff00c00d010 tx=0x7ff00c0079a0 comp rx=0 tx=0).stop 2024-06-03T03:20:44.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:44.721+0000 7ff01c600700 1 -- 172.21.15.117:0/2137854836 shutdown_connections 2024-06-03T03:20:44.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:44.721+0000 7ff01c600700 1 --2- 172.21.15.117:0/2137854836 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7feffc0a2c20 0x7feffc0a50e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:44.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:44.721+0000 7ff01c600700 1 --2- 172.21.15.117:0/2137854836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0141084e0 0x7ff014198a10 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:44.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:44.721+0000 7ff01c600700 1 --2- 172.21.15.117:0/2137854836 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff01406a170 0x7ff01419e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:44.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:44.721+0000 7ff01c600700 1 --2- 172.21.15.117:0/2137854836 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff01406ab10 0x7ff01419ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:44.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:44.721+0000 7ff01c600700 1 -- 172.21.15.117:0/2137854836 >> 172.21.15.117:0/2137854836 conn(0x7ff0140782c0 msgr2=0x7ff0140ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:20:44.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:44.721+0000 7ff01c600700 1 -- 172.21.15.117:0/2137854836 shutdown_connections 2024-06-03T03:20:44.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:44.721+0000 7ff01c600700 1 -- 172.21.15.117:0/2137854836 wait complete. 2024-06-03T03:20:44.723 INFO:teuthology.orchestra.run.smithi117.stderr:listed 10 entries 2024-06-03T03:20:45.126 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:44 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/2137854836' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-03T03:20:45.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:44 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/2137854836' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-03T03:20:45.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:44 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/2137854836' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-03T03:20:45.584 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi117.front.sepia.ceph.com... 2024-06-03T03:20:45.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T03:20:45.585 DEBUG:teuthology.orchestra.run.smithi117:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-06-03T03:20:45.668 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi117.stderr:ceph-fuse[74969]: fuse finished with error 0 and tester_r 0 2024-06-03T03:20:45.674 INFO:teuthology.orchestra.run:waiting for 300 2024-06-03T03:20:45.775 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi117.front.sepia.ceph.com 2024-06-03T03:20:45.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T03:20:45.775 DEBUG:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-06-03T03:20:45.847 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-03T03:20:46.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:45 smithi151 ceph-mon[37043]: pgmap v367: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-06-03T03:20:46.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:45 smithi117 ceph-mon[31649]: pgmap v367: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-06-03T03:20:46.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:45 smithi183 ceph-mon[37326]: pgmap v367: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-06-03T03:20:46.288 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:20:48.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:47 smithi151 ceph-mon[37043]: pgmap v368: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-06-03T03:20:48.161 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:47 smithi117 ceph-mon[31649]: pgmap v368: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-06-03T03:20:48.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:47 smithi183 ceph-mon[37326]: pgmap v368: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-06-03T03:20:48.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:48.717+0000 7f4687000700 1 -- 172.21.15.117:0/1459064188 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4680109480 msgr2=0x7f4680115990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:20:48.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:48.717+0000 7f4687000700 1 --2- 172.21.15.117:0/1459064188 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4680109480 0x7f4680115990 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f467800b0a0 tx=0x7f467800b3b0 comp rx=0 tx=0).stop 2024-06-03T03:20:48.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:48.717+0000 7f4687000700 1 -- 172.21.15.117:0/1459064188 shutdown_connections 2024-06-03T03:20:48.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:48.717+0000 7f4687000700 1 --2- 172.21.15.117:0/1459064188 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4680109480 0x7f4680115990 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:48.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:48.717+0000 7f4687000700 1 --2- 172.21.15.117:0/1459064188 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f468010eac0 0x7f468010eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:48.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:48.717+0000 7f4687000700 1 --2- 172.21.15.117:0/1459064188 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4680108ac0 0x7f4680108f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:48.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:48.717+0000 7f4687000700 1 -- 172.21.15.117:0/1459064188 >> 172.21.15.117:0/1459064188 conn(0x7f4680075c60 msgr2=0x7f46801072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:20:48.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:48.717+0000 7f4687000700 1 -- 172.21.15.117:0/1459064188 shutdown_connections 2024-06-03T03:20:48.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:48.717+0000 7f4687000700 1 -- 172.21.15.117:0/1459064188 wait complete. 2024-06-03T03:20:48.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:48.718+0000 7f4687000700 1 Processor -- start 2024-06-03T03:20:48.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:48.719+0000 7f4687000700 1 -- start start 2024-06-03T03:20:48.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:48.719+0000 7f4687000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4680108ac0 0x7f468019f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:20:48.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:48.719+0000 7f4687000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4680109480 0x7f468019f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:20:48.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:48.719+0000 7f4687000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f468010eac0 0x7f46801a3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:20:48.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:48.719+0000 7f4687000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4680118ea0 con 0x7f468010eac0 2024-06-03T03:20:48.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:48.719+0000 7f4687000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4680118d20 con 0x7f4680108ac0 2024-06-03T03:20:48.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:48.719+0000 7f4687000700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4680119020 con 0x7f4680109480 2024-06-03T03:20:48.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:48.720+0000 7f467f400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4680109480 0x7f468019f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:20:48.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:48.720+0000 7f467f400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4680109480 0x7f468019f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:41920/0 (socket says 172.21.15.117:41920) 2024-06-03T03:20:48.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:48.720+0000 7f467f400700 1 -- 172.21.15.117:0/785471983 learned_addr learned my addr 172.21.15.117:0/785471983 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:20:48.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:48.720+0000 7f467fe00700 1 --2- 172.21.15.117:0/785471983 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4680108ac0 0x7f468019f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:20:48.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:48.720+0000 7f467f400700 1 -- 172.21.15.117:0/785471983 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4680108ac0 msgr2=0x7f468019f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:20:48.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:48.720+0000 7f467f400700 1 --2- 172.21.15.117:0/785471983 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4680108ac0 0x7f468019f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:48.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:48.720+0000 7f467f400700 1 -- 172.21.15.117:0/785471983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f468010eac0 msgr2=0x7f46801a3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:20:48.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:48.720+0000 7f4685000700 1 --2- 172.21.15.117:0/785471983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f468010eac0 0x7f46801a3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:20:48.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:48.720+0000 7f467f400700 1 --2- 172.21.15.117:0/785471983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f468010eac0 0x7f46801a3ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:48.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:48.720+0000 7f467f400700 1 -- 172.21.15.117:0/785471983 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46801a4220 con 0x7f4680109480 2024-06-03T03:20:48.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:48.720+0000 7f4685000700 1 --2- 172.21.15.117:0/785471983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f468010eac0 0x7f46801a3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T03:20:48.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:48.721+0000 7f467f400700 1 --2- 172.21.15.117:0/785471983 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4680109480 0x7f468019f750 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f466c00eb30 tx=0x7f466c00ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:20:48.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:48.721+0000 7f467cc00700 1 -- 172.21.15.117:0/785471983 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f466c00cc90 con 0x7f4680109480 2024-06-03T03:20:48.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:48.721+0000 7f4687000700 1 -- 172.21.15.117:0/785471983 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46801a4510 con 0x7f4680109480 2024-06-03T03:20:48.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:48.721+0000 7f467cc00700 1 -- 172.21.15.117:0/785471983 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f466c004500 con 0x7f4680109480 2024-06-03T03:20:48.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:48.721+0000 7f467cc00700 1 -- 172.21.15.117:0/785471983 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f466c010430 con 0x7f4680109480 2024-06-03T03:20:48.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:48.721+0000 7f4687000700 1 -- 172.21.15.117:0/785471983 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46801abd90 con 0x7f4680109480 2024-06-03T03:20:48.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:48.722+0000 7f4687000700 1 -- 172.21.15.117:0/785471983 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4680069180 con 0x7f4680109480 2024-06-03T03:20:48.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:48.725+0000 7f467cc00700 1 -- 172.21.15.117:0/785471983 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f466c043ae0 con 0x7f4680109480 2024-06-03T03:20:48.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:48.726+0000 7f467cc00700 1 --2- 172.21.15.117:0/785471983 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f46640a2c20 0x7f46640a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:20:48.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:48.726+0000 7f467cc00700 1 -- 172.21.15.117:0/785471983 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7f466c0cf710 con 0x7f4680109480 2024-06-03T03:20:48.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:48.726+0000 7f467fe00700 1 --2- 172.21.15.117:0/785471983 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f46640a2c20 0x7f46640a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:20:48.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:48.727+0000 7f467fe00700 1 --2- 172.21.15.117:0/785471983 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f46640a2c20 0x7f46640a50e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4670009be0 tx=0x7f4670009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:20:48.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:48.728+0000 7f467cc00700 1 -- 172.21.15.117:0/785471983 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f466c09c2b0 con 0x7f4680109480 2024-06-03T03:20:48.951 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:48.950+0000 7f4687000700 1 -- 172.21.15.117:0/785471983 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f46801a0570 con 0x7f4680109480 2024-06-03T03:20:48.953 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:48.953+0000 7f467cc00700 1 -- 172.21.15.117:0/785471983 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+24713 (secure 0 0 0) 0x7f46801a0570 con 0x7f4680109480 2024-06-03T03:20:48.954 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T03:20:48.955 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":97,"fsid":"a0e8473a-2156-11ef-bc9b-c7b262605968","created":"2024-06-03T03:10:08.446859+0000","modified":"2024-06-03T03:20:25.885939+0000","last_up_change":"2024-06-03T03:18:28.736904+0000","last_in_change":"2024-06-03T03:18:08.485711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-03T03:14:09.370986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-03T03:19:35.044683+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-03T03:19:40.097569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-03T03:19:55.394480+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8fefa089-9401-4a1c-b9fb-fe30c63e6a1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":4286644492},{"type":"v1","addr":"172.21.15.117:6803","nonce":4286644492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":4286644492},{"type":"v1","addr":"172.21.15.117:6805","nonce":4286644492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":4286644492},{"type":"v1","addr":"172.21.15.117:6809","nonce":4286644492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":4286644492},{"type":"v1","addr":"172.21.15.117:6807","nonce":4286644492}]},"public_addr":"172.21.15.117:6803/4286644492","cluster_addr":"172.21.15.117:6805/4286644492","heartbeat_back_addr":"172.21.15.117:6809/4286644492","heartbeat_front_addr":"172.21.15.117:6807/4286644492","state":["exists","up"]},{"osd":1,"uuid":"5b12b371-f155-4b67-a470-25cbe59a2999","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":262491158},{"type":"v1","addr":"172.21.15.151:6801","nonce":262491158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":262491158},{"type":"v1","addr":"172.21.15.151:6803","nonce":262491158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":262491158},{"type":"v1","addr":"172.21.15.151:6807","nonce":262491158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":262491158},{"type":"v1","addr":"172.21.15.151:6805","nonce":262491158}]},"public_addr":"172.21.15.151:6801/262491158","cluster_addr":"172.21.15.151:6803/262491158","heartbeat_back_addr":"172.21.15.151:6807/262491158","heartbeat_front_addr":"172.21.15.151:6805/262491158","state":["exists","up"]},{"osd":2,"uuid":"dbc17818-c54c-4ecf-b656-03578b95d8c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":3542453623},{"type":"v1","addr":"172.21.15.183:6801","nonce":3542453623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":3542453623},{"type":"v1","addr":"172.21.15.183:6803","nonce":3542453623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":3542453623},{"type":"v1","addr":"172.21.15.183:6807","nonce":3542453623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":3542453623},{"type":"v1","addr":"172.21.15.183:6805","nonce":3542453623}]},"public_addr":"172.21.15.183:6801/3542453623","cluster_addr":"172.21.15.183:6803/3542453623","heartbeat_back_addr":"172.21.15.183:6807/3542453623","heartbeat_front_addr":"172.21.15.183:6805/3542453623","state":["exists","up"]},{"osd":3,"uuid":"ca08d2ac-b1d6-4a49-9ae8-e3ec146899a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":589012315},{"type":"v1","addr":"172.21.15.117:6811","nonce":589012315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":589012315},{"type":"v1","addr":"172.21.15.117:6813","nonce":589012315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":589012315},{"type":"v1","addr":"172.21.15.117:6817","nonce":589012315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":589012315},{"type":"v1","addr":"172.21.15.117:6815","nonce":589012315}]},"public_addr":"172.21.15.117:6811/589012315","cluster_addr":"172.21.15.117:6813/589012315","heartbeat_back_addr":"172.21.15.117:6817/589012315","heartbeat_front_addr":"172.21.15.117:6815/589012315","state":["exists","up"]},{"osd":4,"uuid":"810d6c7a-023f-4930-82af-bc2bbfd647da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":1239048647},{"type":"v1","addr":"172.21.15.151:6809","nonce":1239048647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":1239048647},{"type":"v1","addr":"172.21.15.151:6811","nonce":1239048647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":1239048647},{"type":"v1","addr":"172.21.15.151:6815","nonce":1239048647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":1239048647},{"type":"v1","addr":"172.21.15.151:6813","nonce":1239048647}]},"public_addr":"172.21.15.151:6809/1239048647","cluster_addr":"172.21.15.151:6811/1239048647","heartbeat_back_addr":"172.21.15.151:6815/1239048647","heartbeat_front_addr":"172.21.15.151:6813/1239048647","state":["exists","up"]},{"osd":5,"uuid":"8ab6a5f0-34bb-48d8-9471-a7dc836e486e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":2985874686},{"type":"v1","addr":"172.21.15.183:6809","nonce":2985874686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":2985874686},{"type":"v1","addr":"172.21.15.183:6811","nonce":2985874686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":2985874686},{"type":"v1","addr":"172.21.15.183:6815","nonce":2985874686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":2985874686},{"type":"v1","addr":"172.21.15.183:6813","nonce":2985874686}]},"public_addr":"172.21.15.183:6809/2985874686","cluster_addr":"172.21.15.183:6811/2985874686","heartbeat_back_addr":"172.21.15.183:6815/2985874686","heartbeat_front_addr":"172.21.15.183:6813/2985874686","state":["exists","up"]},{"osd":6,"uuid":"78d15739-e9e0-4f19-ae5f-ee8527a5b527","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4250583193},{"type":"v1","addr":"172.21.15.117:6819","nonce":4250583193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":4250583193},{"type":"v1","addr":"172.21.15.117:6821","nonce":4250583193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":4250583193},{"type":"v1","addr":"172.21.15.117:6825","nonce":4250583193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4250583193},{"type":"v1","addr":"172.21.15.117:6823","nonce":4250583193}]},"public_addr":"172.21.15.117:6819/4250583193","cluster_addr":"172.21.15.117:6821/4250583193","heartbeat_back_addr":"172.21.15.117:6825/4250583193","heartbeat_front_addr":"172.21.15.117:6823/4250583193","state":["exists","up"]},{"osd":7,"uuid":"3724fe1f-f4d7-443b-aea8-ff3ab02bfdc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":386094645},{"type":"v1","addr":"172.21.15.151:6817","nonce":386094645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":386094645},{"type":"v1","addr":"172.21.15.151:6819","nonce":386094645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":386094645},{"type":"v1","addr":"172.21.15.151:6823","nonce":386094645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":386094645},{"type":"v1","addr":"172.21.15.151:6821","nonce":386094645}]},"public_addr":"172.21.15.151:6817/386094645","cluster_addr":"172.21.15.151:6819/386094645","heartbeat_back_addr":"172.21.15.151:6823/386094645","heartbeat_front_addr":"172.21.15.151:6821/386094645","state":["exists","up"]},{"osd":8,"uuid":"006ac092-f015-448e-a0c0-f7b0c4597a18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1829748222},{"type":"v1","addr":"172.21.15.183:6817","nonce":1829748222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1829748222},{"type":"v1","addr":"172.21.15.183:6819","nonce":1829748222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1829748222},{"type":"v1","addr":"172.21.15.183:6823","nonce":1829748222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1829748222},{"type":"v1","addr":"172.21.15.183:6821","nonce":1829748222}]},"public_addr":"172.21.15.183:6817/1829748222","cluster_addr":"172.21.15.183:6819/1829748222","heartbeat_back_addr":"172.21.15.183:6823/1829748222","heartbeat_front_addr":"172.21.15.183:6821/1829748222","state":["exists","up"]},{"osd":9,"uuid":"00a970e6-2502-459e-975c-badac93c9bd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2538315141},{"type":"v1","addr":"172.21.15.117:6827","nonce":2538315141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2538315141},{"type":"v1","addr":"172.21.15.117:6829","nonce":2538315141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2538315141},{"type":"v1","addr":"172.21.15.117:6833","nonce":2538315141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2538315141},{"type":"v1","addr":"172.21.15.117:6831","nonce":2538315141}]},"public_addr":"172.21.15.117:6827/2538315141","cluster_addr":"172.21.15.117:6829/2538315141","heartbeat_back_addr":"172.21.15.117:6833/2538315141","heartbeat_front_addr":"172.21.15.117:6831/2538315141","state":["exists","up"]},{"osd":10,"uuid":"ba2333cd-32a3-41ef-9ea5-287c707f512b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":3801045133},{"type":"v1","addr":"172.21.15.151:6825","nonce":3801045133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3801045133},{"type":"v1","addr":"172.21.15.151:6827","nonce":3801045133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3801045133},{"type":"v1","addr":"172.21.15.151:6831","nonce":3801045133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3801045133},{"type":"v1","addr":"172.21.15.151:6829","nonce":3801045133}]},"public_addr":"172.21.15.151:6825/3801045133","cluster_addr":"172.21.15.151:6827/3801045133","heartbeat_back_addr":"172.21.15.151:6831/3801045133","heartbeat_front_addr":"172.21.15.151:6829/3801045133","state":["exists","up"]},{"osd":11,"uuid":"1556d654-e6fa-42c1-b399-aa53f6837a1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3513899696},{"type":"v1","addr":"172.21.15.183:6825","nonce":3513899696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3513899696},{"type":"v1","addr":"172.21.15.183:6827","nonce":3513899696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3513899696},{"type":"v1","addr":"172.21.15.183:6831","nonce":3513899696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3513899696},{"type":"v1","addr":"172.21.15.183:6829","nonce":3513899696}]},"public_addr":"172.21.15.183:6825/3513899696","cluster_addr":"172.21.15.183:6827/3513899696","heartbeat_back_addr":"172.21.15.183:6831/3513899696","heartbeat_front_addr":"172.21.15.183:6829/3513899696","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:13:17.174041+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:13:42.237001+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:14:06.273315+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:14:39.912564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:15:06.063202+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:15:30.424818+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:16:05.623554+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:16:32.926393+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:16:58.165899+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:17:32.897477+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:18:00.720866+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:18:26.474441+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.117:0/186309008":"2024-06-04T03:11:24.655516+0000","172.21.15.117:6801/2142313590":"2024-06-04T03:11:24.655516+0000","172.21.15.117:6800/2142313590":"2024-06-04T03:11:24.655516+0000","172.21.15.117:0/2399725156":"2024-06-04T03:11:24.655516+0000","172.21.15.117:0/1453232061":"2024-06-04T03:10:50.552899+0000","172.21.15.117:0/4281674974":"2024-06-04T03:10:50.552899+0000","172.21.15.117:6800/2424554985":"2024-06-04T03:10:50.552899+0000","172.21.15.117:0/3946365225":"2024-06-04T03:11:24.655516+0000","172.21.15.117:0/1146162354":"2024-06-04T03:10:50.552899+0000","172.21.15.117:6801/2424554985":"2024-06-04T03:10:50.552899+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-03T03:20:48.957 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:48.956+0000 7f4687000700 1 -- 172.21.15.117:0/785471983 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f46640a2c20 msgr2=0x7f46640a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:20:48.957 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:48.956+0000 7f4687000700 1 --2- 172.21.15.117:0/785471983 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f46640a2c20 0x7f46640a50e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4670009be0 tx=0x7f4670009260 comp rx=0 tx=0).stop 2024-06-03T03:20:48.957 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:48.956+0000 7f4687000700 1 -- 172.21.15.117:0/785471983 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4680109480 msgr2=0x7f468019f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:20:48.957 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:48.956+0000 7f4687000700 1 --2- 172.21.15.117:0/785471983 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4680109480 0x7f468019f750 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f466c00eb30 tx=0x7f466c00ee40 comp rx=0 tx=0).stop 2024-06-03T03:20:48.957 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:48.956+0000 7f4687000700 1 -- 172.21.15.117:0/785471983 shutdown_connections 2024-06-03T03:20:48.958 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:48.956+0000 7f4687000700 1 --2- 172.21.15.117:0/785471983 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f46640a2c20 0x7f46640a50e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:48.958 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:48.956+0000 7f4687000700 1 --2- 172.21.15.117:0/785471983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f468010eac0 0x7f46801a3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:48.958 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:48.956+0000 7f4687000700 1 --2- 172.21.15.117:0/785471983 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4680108ac0 0x7f468019f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:48.958 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:48.956+0000 7f4687000700 1 --2- 172.21.15.117:0/785471983 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4680109480 0x7f468019f750 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:48.958 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:48.956+0000 7f4687000700 1 -- 172.21.15.117:0/785471983 >> 172.21.15.117:0/785471983 conn(0x7f4680075c60 msgr2=0x7f4680076140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:20:48.958 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:48.956+0000 7f4687000700 1 -- 172.21.15.117:0/785471983 shutdown_connections 2024-06-03T03:20:48.958 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:48.957+0000 7f4687000700 1 -- 172.21.15.117:0/785471983 wait complete. 2024-06-03T03:20:49.981 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-03T03:20:50.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:49 smithi117 ceph-mon[31649]: pgmap v369: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-06-03T03:20:50.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:49 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/785471983' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T03:20:50.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:49 smithi183 ceph-mon[37326]: pgmap v369: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-06-03T03:20:50.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:49 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/785471983' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T03:20:50.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:49 smithi151 ceph-mon[37043]: pgmap v369: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-06-03T03:20:50.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:49 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/785471983' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T03:20:50.427 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:20:51.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:51 smithi117 ceph-mon[31649]: pgmap v370: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-06-03T03:20:52.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:51 smithi183 ceph-mon[37326]: pgmap v370: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-06-03T03:20:52.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:51 smithi151 ceph-mon[37043]: pgmap v370: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-06-03T03:20:53.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:53.091+0000 7f3d33200700 1 -- 172.21.15.117:0/654608714 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d2c10c9c0 msgr2=0x7f3d2c10ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:20:53.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:53.091+0000 7f3d33200700 1 --2- 172.21.15.117:0/654608714 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d2c10c9c0 0x7f3d2c10ee60 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3d2800b0a0 tx=0x7f3d2800b3b0 comp rx=0 tx=0).stop 2024-06-03T03:20:53.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:53.091+0000 7f3d33200700 1 -- 172.21.15.117:0/654608714 shutdown_connections 2024-06-03T03:20:53.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:53.091+0000 7f3d33200700 1 --2- 172.21.15.117:0/654608714 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d2c10c9c0 0x7f3d2c10ee60 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:53.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:53.091+0000 7f3d33200700 1 --2- 172.21.15.117:0/654608714 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3d2c1035e0 0x7f3d2c1039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:53.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:53.091+0000 7f3d33200700 1 --2- 172.21.15.117:0/654608714 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d2c103f00 0x7f3d2c10c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:53.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:53.092+0000 7f3d33200700 1 -- 172.21.15.117:0/654608714 >> 172.21.15.117:0/654608714 conn(0x7f3d2c0fcd70 msgr2=0x7f3d2c0ff190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:20:53.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:53.092+0000 7f3d33200700 1 -- 172.21.15.117:0/654608714 shutdown_connections 2024-06-03T03:20:53.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:53.092+0000 7f3d33200700 1 -- 172.21.15.117:0/654608714 wait complete. 2024-06-03T03:20:53.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:53.093+0000 7f3d33200700 1 Processor -- start 2024-06-03T03:20:53.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:53.093+0000 7f3d33200700 1 -- start start 2024-06-03T03:20:53.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:53.093+0000 7f3d33200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d2c1035e0 0x7f3d2c19ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:20:53.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:53.093+0000 7f3d33200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d2c103f00 0x7f3d2c19b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:20:53.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:53.093+0000 7f3d33200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3d2c10c9c0 0x7f3d2c19f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:20:53.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:53.093+0000 7f3d33200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3d2c112710 con 0x7f3d2c103f00 2024-06-03T03:20:53.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:53.093+0000 7f3d33200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3d2c112590 con 0x7f3d2c10c9c0 2024-06-03T03:20:53.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:53.093+0000 7f3d33200700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3d2c112890 con 0x7f3d2c1035e0 2024-06-03T03:20:53.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:53.094+0000 7f3d31400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d2c103f00 0x7f3d2c19b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:20:53.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:53.094+0000 7f3d31400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d2c103f00 0x7f3d2c19b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41646/0 (socket says 172.21.15.117:41646) 2024-06-03T03:20:53.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:53.094+0000 7f3d32800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3d2c10c9c0 0x7f3d2c19f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:20:53.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:53.094+0000 7f3d31400700 1 -- 172.21.15.117:0/2033901858 learned_addr learned my addr 172.21.15.117:0/2033901858 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:20:53.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:53.094+0000 7f3d31e00700 1 --2- 172.21.15.117:0/2033901858 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d2c1035e0 0x7f3d2c19ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:20:53.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:53.094+0000 7f3d32800700 1 -- 172.21.15.117:0/2033901858 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d2c1035e0 msgr2=0x7f3d2c19ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:20:53.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:53.094+0000 7f3d32800700 1 --2- 172.21.15.117:0/2033901858 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d2c1035e0 0x7f3d2c19ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:53.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:53.094+0000 7f3d32800700 1 -- 172.21.15.117:0/2033901858 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d2c103f00 msgr2=0x7f3d2c19b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:20:53.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:53.094+0000 7f3d32800700 1 --2- 172.21.15.117:0/2033901858 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d2c103f00 0x7f3d2c19b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:53.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:53.094+0000 7f3d32800700 1 -- 172.21.15.117:0/2033901858 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d2c19fd40 con 0x7f3d2c10c9c0 2024-06-03T03:20:53.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:53.094+0000 7f3d31e00700 1 --2- 172.21.15.117:0/2033901858 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d2c1035e0 0x7f3d2c19ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:20:53.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:53.095+0000 7f3d32800700 1 --2- 172.21.15.117:0/2033901858 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3d2c10c9c0 0x7f3d2c19f600 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3d2800b0a0 tx=0x7f3d28004d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:20:53.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:53.095+0000 7f3d26a00700 1 -- 172.21.15.117:0/2033901858 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d28021070 con 0x7f3d2c10c9c0 2024-06-03T03:20:53.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:53.095+0000 7f3d33200700 1 -- 172.21.15.117:0/2033901858 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d2c19ffd0 con 0x7f3d2c10c9c0 2024-06-03T03:20:53.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:53.095+0000 7f3d26a00700 1 -- 172.21.15.117:0/2033901858 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d28007b80 con 0x7f3d2c10c9c0 2024-06-03T03:20:53.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:53.095+0000 7f3d26a00700 1 -- 172.21.15.117:0/2033901858 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d28005040 con 0x7f3d2c10c9c0 2024-06-03T03:20:53.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:53.095+0000 7f3d33200700 1 -- 172.21.15.117:0/2033901858 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d2c1a78b0 con 0x7f3d2c10c9c0 2024-06-03T03:20:53.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:53.096+0000 7f3d33200700 1 -- 172.21.15.117:0/2033901858 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d2c069180 con 0x7f3d2c10c9c0 2024-06-03T03:20:53.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:53.098+0000 7f3d26a00700 1 -- 172.21.15.117:0/2033901858 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f3d28024070 con 0x7f3d2c10c9c0 2024-06-03T03:20:53.100 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:53.099+0000 7f3d26a00700 1 --2- 172.21.15.117:0/2033901858 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f3d140a2c20 0x7f3d140a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:20:53.100 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:53.100+0000 7f3d26a00700 1 -- 172.21.15.117:0/2033901858 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7f3d2800f020 con 0x7f3d2c10c9c0 2024-06-03T03:20:53.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:53.100+0000 7f3d31e00700 1 --2- 172.21.15.117:0/2033901858 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f3d140a2c20 0x7f3d140a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:20:53.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:53.100+0000 7f3d31e00700 1 --2- 172.21.15.117:0/2033901858 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f3d140a2c20 0x7f3d140a50e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3d20009680 tx=0x7f3d20009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:20:53.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:53.104+0000 7f3d26a00700 1 -- 172.21.15.117:0/2033901858 <== mon.2 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3d2802b020 con 0x7f3d2c10c9c0 2024-06-03T03:20:53.330 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:53.329+0000 7f3d33200700 1 -- 172.21.15.117:0/2033901858 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3d2c1a7a80 con 0x7f3d2c10c9c0 2024-06-03T03:20:53.332 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:53.331+0000 7f3d26a00700 1 -- 172.21.15.117:0/2033901858 <== mon.2 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+24713 (secure 0 0 0) 0x7f3d28007d30 con 0x7f3d2c10c9c0 2024-06-03T03:20:53.332 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T03:20:53.333 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":97,"fsid":"a0e8473a-2156-11ef-bc9b-c7b262605968","created":"2024-06-03T03:10:08.446859+0000","modified":"2024-06-03T03:20:25.885939+0000","last_up_change":"2024-06-03T03:18:28.736904+0000","last_in_change":"2024-06-03T03:18:08.485711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-03T03:14:09.370986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-03T03:19:35.044683+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-03T03:19:40.097569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-03T03:19:55.394480+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8fefa089-9401-4a1c-b9fb-fe30c63e6a1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":4286644492},{"type":"v1","addr":"172.21.15.117:6803","nonce":4286644492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":4286644492},{"type":"v1","addr":"172.21.15.117:6805","nonce":4286644492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":4286644492},{"type":"v1","addr":"172.21.15.117:6809","nonce":4286644492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":4286644492},{"type":"v1","addr":"172.21.15.117:6807","nonce":4286644492}]},"public_addr":"172.21.15.117:6803/4286644492","cluster_addr":"172.21.15.117:6805/4286644492","heartbeat_back_addr":"172.21.15.117:6809/4286644492","heartbeat_front_addr":"172.21.15.117:6807/4286644492","state":["exists","up"]},{"osd":1,"uuid":"5b12b371-f155-4b67-a470-25cbe59a2999","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":262491158},{"type":"v1","addr":"172.21.15.151:6801","nonce":262491158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":262491158},{"type":"v1","addr":"172.21.15.151:6803","nonce":262491158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":262491158},{"type":"v1","addr":"172.21.15.151:6807","nonce":262491158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":262491158},{"type":"v1","addr":"172.21.15.151:6805","nonce":262491158}]},"public_addr":"172.21.15.151:6801/262491158","cluster_addr":"172.21.15.151:6803/262491158","heartbeat_back_addr":"172.21.15.151:6807/262491158","heartbeat_front_addr":"172.21.15.151:6805/262491158","state":["exists","up"]},{"osd":2,"uuid":"dbc17818-c54c-4ecf-b656-03578b95d8c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":3542453623},{"type":"v1","addr":"172.21.15.183:6801","nonce":3542453623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":3542453623},{"type":"v1","addr":"172.21.15.183:6803","nonce":3542453623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":3542453623},{"type":"v1","addr":"172.21.15.183:6807","nonce":3542453623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":3542453623},{"type":"v1","addr":"172.21.15.183:6805","nonce":3542453623}]},"public_addr":"172.21.15.183:6801/3542453623","cluster_addr":"172.21.15.183:6803/3542453623","heartbeat_back_addr":"172.21.15.183:6807/3542453623","heartbeat_front_addr":"172.21.15.183:6805/3542453623","state":["exists","up"]},{"osd":3,"uuid":"ca08d2ac-b1d6-4a49-9ae8-e3ec146899a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":589012315},{"type":"v1","addr":"172.21.15.117:6811","nonce":589012315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":589012315},{"type":"v1","addr":"172.21.15.117:6813","nonce":589012315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":589012315},{"type":"v1","addr":"172.21.15.117:6817","nonce":589012315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":589012315},{"type":"v1","addr":"172.21.15.117:6815","nonce":589012315}]},"public_addr":"172.21.15.117:6811/589012315","cluster_addr":"172.21.15.117:6813/589012315","heartbeat_back_addr":"172.21.15.117:6817/589012315","heartbeat_front_addr":"172.21.15.117:6815/589012315","state":["exists","up"]},{"osd":4,"uuid":"810d6c7a-023f-4930-82af-bc2bbfd647da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":1239048647},{"type":"v1","addr":"172.21.15.151:6809","nonce":1239048647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":1239048647},{"type":"v1","addr":"172.21.15.151:6811","nonce":1239048647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":1239048647},{"type":"v1","addr":"172.21.15.151:6815","nonce":1239048647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":1239048647},{"type":"v1","addr":"172.21.15.151:6813","nonce":1239048647}]},"public_addr":"172.21.15.151:6809/1239048647","cluster_addr":"172.21.15.151:6811/1239048647","heartbeat_back_addr":"172.21.15.151:6815/1239048647","heartbeat_front_addr":"172.21.15.151:6813/1239048647","state":["exists","up"]},{"osd":5,"uuid":"8ab6a5f0-34bb-48d8-9471-a7dc836e486e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":2985874686},{"type":"v1","addr":"172.21.15.183:6809","nonce":2985874686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":2985874686},{"type":"v1","addr":"172.21.15.183:6811","nonce":2985874686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":2985874686},{"type":"v1","addr":"172.21.15.183:6815","nonce":2985874686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":2985874686},{"type":"v1","addr":"172.21.15.183:6813","nonce":2985874686}]},"public_addr":"172.21.15.183:6809/2985874686","cluster_addr":"172.21.15.183:6811/2985874686","heartbeat_back_addr":"172.21.15.183:6815/2985874686","heartbeat_front_addr":"172.21.15.183:6813/2985874686","state":["exists","up"]},{"osd":6,"uuid":"78d15739-e9e0-4f19-ae5f-ee8527a5b527","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4250583193},{"type":"v1","addr":"172.21.15.117:6819","nonce":4250583193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":4250583193},{"type":"v1","addr":"172.21.15.117:6821","nonce":4250583193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":4250583193},{"type":"v1","addr":"172.21.15.117:6825","nonce":4250583193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4250583193},{"type":"v1","addr":"172.21.15.117:6823","nonce":4250583193}]},"public_addr":"172.21.15.117:6819/4250583193","cluster_addr":"172.21.15.117:6821/4250583193","heartbeat_back_addr":"172.21.15.117:6825/4250583193","heartbeat_front_addr":"172.21.15.117:6823/4250583193","state":["exists","up"]},{"osd":7,"uuid":"3724fe1f-f4d7-443b-aea8-ff3ab02bfdc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":386094645},{"type":"v1","addr":"172.21.15.151:6817","nonce":386094645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":386094645},{"type":"v1","addr":"172.21.15.151:6819","nonce":386094645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":386094645},{"type":"v1","addr":"172.21.15.151:6823","nonce":386094645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":386094645},{"type":"v1","addr":"172.21.15.151:6821","nonce":386094645}]},"public_addr":"172.21.15.151:6817/386094645","cluster_addr":"172.21.15.151:6819/386094645","heartbeat_back_addr":"172.21.15.151:6823/386094645","heartbeat_front_addr":"172.21.15.151:6821/386094645","state":["exists","up"]},{"osd":8,"uuid":"006ac092-f015-448e-a0c0-f7b0c4597a18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1829748222},{"type":"v1","addr":"172.21.15.183:6817","nonce":1829748222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1829748222},{"type":"v1","addr":"172.21.15.183:6819","nonce":1829748222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1829748222},{"type":"v1","addr":"172.21.15.183:6823","nonce":1829748222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1829748222},{"type":"v1","addr":"172.21.15.183:6821","nonce":1829748222}]},"public_addr":"172.21.15.183:6817/1829748222","cluster_addr":"172.21.15.183:6819/1829748222","heartbeat_back_addr":"172.21.15.183:6823/1829748222","heartbeat_front_addr":"172.21.15.183:6821/1829748222","state":["exists","up"]},{"osd":9,"uuid":"00a970e6-2502-459e-975c-badac93c9bd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2538315141},{"type":"v1","addr":"172.21.15.117:6827","nonce":2538315141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2538315141},{"type":"v1","addr":"172.21.15.117:6829","nonce":2538315141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2538315141},{"type":"v1","addr":"172.21.15.117:6833","nonce":2538315141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2538315141},{"type":"v1","addr":"172.21.15.117:6831","nonce":2538315141}]},"public_addr":"172.21.15.117:6827/2538315141","cluster_addr":"172.21.15.117:6829/2538315141","heartbeat_back_addr":"172.21.15.117:6833/2538315141","heartbeat_front_addr":"172.21.15.117:6831/2538315141","state":["exists","up"]},{"osd":10,"uuid":"ba2333cd-32a3-41ef-9ea5-287c707f512b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":3801045133},{"type":"v1","addr":"172.21.15.151:6825","nonce":3801045133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3801045133},{"type":"v1","addr":"172.21.15.151:6827","nonce":3801045133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3801045133},{"type":"v1","addr":"172.21.15.151:6831","nonce":3801045133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3801045133},{"type":"v1","addr":"172.21.15.151:6829","nonce":3801045133}]},"public_addr":"172.21.15.151:6825/3801045133","cluster_addr":"172.21.15.151:6827/3801045133","heartbeat_back_addr":"172.21.15.151:6831/3801045133","heartbeat_front_addr":"172.21.15.151:6829/3801045133","state":["exists","up"]},{"osd":11,"uuid":"1556d654-e6fa-42c1-b399-aa53f6837a1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3513899696},{"type":"v1","addr":"172.21.15.183:6825","nonce":3513899696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3513899696},{"type":"v1","addr":"172.21.15.183:6827","nonce":3513899696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3513899696},{"type":"v1","addr":"172.21.15.183:6831","nonce":3513899696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3513899696},{"type":"v1","addr":"172.21.15.183:6829","nonce":3513899696}]},"public_addr":"172.21.15.183:6825/3513899696","cluster_addr":"172.21.15.183:6827/3513899696","heartbeat_back_addr":"172.21.15.183:6831/3513899696","heartbeat_front_addr":"172.21.15.183:6829/3513899696","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:13:17.174041+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:13:42.237001+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:14:06.273315+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:14:39.912564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:15:06.063202+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:15:30.424818+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:16:05.623554+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:16:32.926393+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:16:58.165899+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:17:32.897477+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:18:00.720866+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:18:26.474441+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.117:0/186309008":"2024-06-04T03:11:24.655516+0000","172.21.15.117:6801/2142313590":"2024-06-04T03:11:24.655516+0000","172.21.15.117:6800/2142313590":"2024-06-04T03:11:24.655516+0000","172.21.15.117:0/2399725156":"2024-06-04T03:11:24.655516+0000","172.21.15.117:0/1453232061":"2024-06-04T03:10:50.552899+0000","172.21.15.117:0/4281674974":"2024-06-04T03:10:50.552899+0000","172.21.15.117:6800/2424554985":"2024-06-04T03:10:50.552899+0000","172.21.15.117:0/3946365225":"2024-06-04T03:11:24.655516+0000","172.21.15.117:0/1146162354":"2024-06-04T03:10:50.552899+0000","172.21.15.117:6801/2424554985":"2024-06-04T03:10:50.552899+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-03T03:20:53.336 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:53.334+0000 7f3d33200700 1 -- 172.21.15.117:0/2033901858 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f3d140a2c20 msgr2=0x7f3d140a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:20:53.336 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:53.334+0000 7f3d33200700 1 --2- 172.21.15.117:0/2033901858 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f3d140a2c20 0x7f3d140a50e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3d20009680 tx=0x7f3d20009260 comp rx=0 tx=0).stop 2024-06-03T03:20:53.336 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:53.334+0000 7f3d33200700 1 -- 172.21.15.117:0/2033901858 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3d2c10c9c0 msgr2=0x7f3d2c19f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:20:53.336 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:53.334+0000 7f3d33200700 1 --2- 172.21.15.117:0/2033901858 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3d2c10c9c0 0x7f3d2c19f600 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3d2800b0a0 tx=0x7f3d28004d00 comp rx=0 tx=0).stop 2024-06-03T03:20:53.336 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:53.335+0000 7f3d33200700 1 -- 172.21.15.117:0/2033901858 shutdown_connections 2024-06-03T03:20:53.336 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:53.335+0000 7f3d33200700 1 --2- 172.21.15.117:0/2033901858 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f3d140a2c20 0x7f3d140a50e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:53.336 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:53.335+0000 7f3d33200700 1 --2- 172.21.15.117:0/2033901858 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3d2c10c9c0 0x7f3d2c19f600 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:53.336 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:53.335+0000 7f3d33200700 1 --2- 172.21.15.117:0/2033901858 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d2c1035e0 0x7f3d2c19ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:53.336 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:53.335+0000 7f3d33200700 1 --2- 172.21.15.117:0/2033901858 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d2c103f00 0x7f3d2c19b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:53.336 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:53.335+0000 7f3d33200700 1 -- 172.21.15.117:0/2033901858 >> 172.21.15.117:0/2033901858 conn(0x7f3d2c0fcd70 msgr2=0x7f3d2c10ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:20:53.337 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:53.335+0000 7f3d33200700 1 -- 172.21.15.117:0/2033901858 shutdown_connections 2024-06-03T03:20:53.337 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:53.335+0000 7f3d33200700 1 -- 172.21.15.117:0/2033901858 wait complete. 2024-06-03T03:20:54.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:53 smithi117 ceph-mon[31649]: pgmap v371: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-06-03T03:20:54.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:53 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/2033901858' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T03:20:54.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:53 smithi183 ceph-mon[37326]: pgmap v371: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-06-03T03:20:54.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:53 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/2033901858' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T03:20:54.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:53 smithi151 ceph-mon[37043]: pgmap v371: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-06-03T03:20:54.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:53 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/2033901858' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T03:20:54.411 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-06-03T03:20:54.855 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:20:56.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:55 smithi117 ceph-mon[31649]: pgmap v372: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-03T03:20:56.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:55 smithi183 ceph-mon[37326]: pgmap v372: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-03T03:20:56.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:55 smithi151 ceph-mon[37043]: pgmap v372: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-03T03:20:57.370 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:57.368+0000 7f2b2da00700 1 -- 172.21.15.117:0/3612028457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b28102ee0 msgr2=0x7f2b2810f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:20:57.370 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:57.368+0000 7f2b2da00700 1 --2- 172.21.15.117:0/3612028457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b28102ee0 0x7f2b2810f3c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2b20009990 tx=0x7f2b20009ca0 comp rx=0 tx=0).stop 2024-06-03T03:20:57.370 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:57.369+0000 7f2b2da00700 1 -- 172.21.15.117:0/3612028457 shutdown_connections 2024-06-03T03:20:57.370 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:57.369+0000 7f2b2da00700 1 --2- 172.21.15.117:0/3612028457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b28102ee0 0x7f2b2810f3c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:57.370 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:57.369+0000 7f2b2da00700 1 --2- 172.21.15.117:0/3612028457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b28107450 0x7f2b28107830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:57.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:57.369+0000 7f2b2da00700 1 --2- 172.21.15.117:0/3612028457 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2b28102540 0x7f2b281029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:57.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:57.369+0000 7f2b2da00700 1 -- 172.21.15.117:0/3612028457 >> 172.21.15.117:0/3612028457 conn(0x7f2b280795f0 msgr2=0x7f2b28079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:20:57.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:57.369+0000 7f2b2da00700 1 -- 172.21.15.117:0/3612028457 shutdown_connections 2024-06-03T03:20:57.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:57.369+0000 7f2b2da00700 1 -- 172.21.15.117:0/3612028457 wait complete. 2024-06-03T03:20:57.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:57.370+0000 7f2b2da00700 1 Processor -- start 2024-06-03T03:20:57.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:57.370+0000 7f2b2da00700 1 -- start start 2024-06-03T03:20:57.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:57.370+0000 7f2b2da00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b28102540 0x7f2b2819e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:20:57.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:57.370+0000 7f2b2da00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2b28102ee0 0x7f2b2819edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:20:57.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:57.370+0000 7f2b2da00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b28107450 0x7f2b28198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:20:57.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:57.370+0000 7f2b2da00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2b28112830 con 0x7f2b28107450 2024-06-03T03:20:57.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:57.370+0000 7f2b2da00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2b281126b0 con 0x7f2b28102ee0 2024-06-03T03:20:57.373 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:57.370+0000 7f2b2da00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2b281129b0 con 0x7f2b28102540 2024-06-03T03:20:57.373 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:57.371+0000 7f2b27400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b28107450 0x7f2b28198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:20:57.373 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:57.371+0000 7f2b26a00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b28102540 0x7f2b2819e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:20:57.373 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:57.371+0000 7f2b27400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b28107450 0x7f2b28198a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50046/0 (socket says 172.21.15.117:50046) 2024-06-03T03:20:57.373 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:57.371+0000 7f2b27400700 1 -- 172.21.15.117:0/1914972491 learned_addr learned my addr 172.21.15.117:0/1914972491 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:20:57.373 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:57.371+0000 7f2b1e000700 1 --2- 172.21.15.117:0/1914972491 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2b28102ee0 0x7f2b2819edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:20:57.373 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:57.371+0000 7f2b27400700 1 -- 172.21.15.117:0/1914972491 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b28102540 msgr2=0x7f2b2819e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:20:57.373 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:57.371+0000 7f2b27400700 1 --2- 172.21.15.117:0/1914972491 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b28102540 0x7f2b2819e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:57.373 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:57.371+0000 7f2b27400700 1 -- 172.21.15.117:0/1914972491 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2b28102ee0 msgr2=0x7f2b2819edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:20:57.373 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:57.371+0000 7f2b27400700 1 --2- 172.21.15.117:0/1914972491 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2b28102ee0 0x7f2b2819edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:57.373 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:57.371+0000 7f2b27400700 1 -- 172.21.15.117:0/1914972491 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b28199190 con 0x7f2b28107450 2024-06-03T03:20:57.373 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:57.371+0000 7f2b27400700 1 --2- 172.21.15.117:0/1914972491 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b28107450 0x7f2b28198a30 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2b200095b0 tx=0x7f2b2000b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:20:57.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:57.372+0000 7f2b1fe00700 1 -- 172.21.15.117:0/1914972491 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b20004830 con 0x7f2b28107450 2024-06-03T03:20:57.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:57.372+0000 7f2b1fe00700 1 -- 172.21.15.117:0/1914972491 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b20005020 con 0x7f2b28107450 2024-06-03T03:20:57.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:57.372+0000 7f2b1fe00700 1 -- 172.21.15.117:0/1914972491 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b20003a60 con 0x7f2b28107450 2024-06-03T03:20:57.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:57.372+0000 7f2b2da00700 1 -- 172.21.15.117:0/1914972491 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b28199420 con 0x7f2b28107450 2024-06-03T03:20:57.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:57.372+0000 7f2b2da00700 1 -- 172.21.15.117:0/1914972491 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b281a5740 con 0x7f2b28107450 2024-06-03T03:20:57.375 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:57.373+0000 7f2b2da00700 1 -- 172.21.15.117:0/1914972491 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b2810aa60 con 0x7f2b28107450 2024-06-03T03:20:57.375 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:57.375+0000 7f2b1fe00700 1 -- 172.21.15.117:0/1914972491 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f2b20009d10 con 0x7f2b28107450 2024-06-03T03:20:57.377 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:57.376+0000 7f2b1fe00700 1 --2- 172.21.15.117:0/1914972491 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f2b0c0a2ca0 0x7f2b0c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:20:57.377 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:57.377+0000 7f2b1fe00700 1 -- 172.21.15.117:0/1914972491 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7f2b200cfef0 con 0x7f2b28107450 2024-06-03T03:20:57.377 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:57.377+0000 7f2b26a00700 1 --2- 172.21.15.117:0/1914972491 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f2b0c0a2ca0 0x7f2b0c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:20:57.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:57.377+0000 7f2b26a00700 1 --2- 172.21.15.117:0/1914972491 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f2b0c0a2ca0 0x7f2b0c0a5160 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2b28199b70 tx=0x7f2b1800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:20:57.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:57.381+0000 7f2b1fe00700 1 -- 172.21.15.117:0/1914972491 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2b20013070 con 0x7f2b28107450 2024-06-03T03:20:57.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:57.653+0000 7f2b2da00700 1 -- 172.21.15.117:0/1914972491 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f2b2810a440 con 0x7f2b28107450 2024-06-03T03:20:57.926 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:57.925+0000 7f2b1fe00700 1 -- 172.21.15.117:0/1914972491 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7f2b2009cb20 con 0x7f2b28107450 2024-06-03T03:20:57.930 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:57.929+0000 7f2b2da00700 1 -- 172.21.15.117:0/1914972491 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f2b0c0a2ca0 msgr2=0x7f2b0c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:20:57.930 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:57.929+0000 7f2b2da00700 1 --2- 172.21.15.117:0/1914972491 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f2b0c0a2ca0 0x7f2b0c0a5160 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2b28199b70 tx=0x7f2b1800b040 comp rx=0 tx=0).stop 2024-06-03T03:20:57.930 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:57.929+0000 7f2b2da00700 1 -- 172.21.15.117:0/1914972491 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b28107450 msgr2=0x7f2b28198a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:20:57.930 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:57.929+0000 7f2b2da00700 1 --2- 172.21.15.117:0/1914972491 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b28107450 0x7f2b28198a30 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2b200095b0 tx=0x7f2b2000b880 comp rx=0 tx=0).stop 2024-06-03T03:20:57.930 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:57.929+0000 7f2b2da00700 1 -- 172.21.15.117:0/1914972491 shutdown_connections 2024-06-03T03:20:57.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:57.929+0000 7f2b2da00700 1 --2- 172.21.15.117:0/1914972491 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f2b0c0a2ca0 0x7f2b0c0a5160 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:57.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:57.929+0000 7f2b2da00700 1 --2- 172.21.15.117:0/1914972491 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b28107450 0x7f2b28198a30 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:57.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:57.929+0000 7f2b2da00700 1 --2- 172.21.15.117:0/1914972491 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b28102540 0x7f2b2819e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:57.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:57.929+0000 7f2b2da00700 1 --2- 172.21.15.117:0/1914972491 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2b28102ee0 0x7f2b2819edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:20:57.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:57.929+0000 7f2b2da00700 1 -- 172.21.15.117:0/1914972491 >> 172.21.15.117:0/1914972491 conn(0x7f2b280795f0 msgr2=0x7f2b281005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:20:57.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:57.930+0000 7f2b2da00700 1 -- 172.21.15.117:0/1914972491 shutdown_connections 2024-06-03T03:20:57.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:20:57.930+0000 7f2b2da00700 1 -- 172.21.15.117:0/1914972491 wait complete. 2024-06-03T03:20:58.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:57 smithi117 ceph-mon[31649]: pgmap v373: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-03T03:20:58.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:57 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/1914972491' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-06-03T03:20:58.209 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:57 smithi151 ceph-mon[37043]: pgmap v373: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-03T03:20:58.209 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:57 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/1914972491' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-06-03T03:20:58.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:57 smithi183 ceph-mon[37326]: pgmap v373: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-03T03:20:58.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:57 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/1914972491' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-06-03T03:20:59.164 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-06-03T03:20:59.165 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph fs set cephfs max_mds 5 2024-06-03T03:20:59.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:58 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/1914972491' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-06-03T03:20:59.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:58 smithi117 ceph-mon[31649]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-06-03T03:20:59.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:58 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/1914972491' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-06-03T03:20:59.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:58 smithi183 ceph-mon[37326]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-06-03T03:20:59.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:58 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/1914972491' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-06-03T03:20:59.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:58 smithi151 ceph-mon[37043]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-06-03T03:20:59.597 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:21:00.000 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:20:59 smithi117 ceph-mon[31649]: pgmap v374: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-03T03:21:00.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:20:59 smithi183 ceph-mon[37326]: pgmap v374: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-03T03:21:00.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:20:59 smithi151 ceph-mon[37043]: pgmap v374: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-03T03:21:01.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:01.985+0000 7fb4bfe00700 1 -- 172.21.15.117:0/377423366 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4b8109480 msgr2=0x7fb4b8115990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:21:01.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:01.985+0000 7fb4bfe00700 1 --2- 172.21.15.117:0/377423366 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4b8109480 0x7fb4b8115990 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb4b000b0a0 tx=0x7fb4b000b3b0 comp rx=0 tx=0).stop 2024-06-03T03:21:01.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:01.985+0000 7fb4bfe00700 1 -- 172.21.15.117:0/377423366 shutdown_connections 2024-06-03T03:21:01.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:01.985+0000 7fb4bfe00700 1 --2- 172.21.15.117:0/377423366 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4b8109480 0x7fb4b8115990 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:01.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:01.985+0000 7fb4bfe00700 1 --2- 172.21.15.117:0/377423366 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb4b810eac0 0x7fb4b810eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:01.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:01.985+0000 7fb4bfe00700 1 --2- 172.21.15.117:0/377423366 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4b8108ac0 0x7fb4b8108f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:01.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:01.985+0000 7fb4bfe00700 1 -- 172.21.15.117:0/377423366 >> 172.21.15.117:0/377423366 conn(0x7fb4b8075c60 msgr2=0x7fb4b81072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:21:01.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:01.985+0000 7fb4bfe00700 1 -- 172.21.15.117:0/377423366 shutdown_connections 2024-06-03T03:21:01.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:01.985+0000 7fb4bfe00700 1 -- 172.21.15.117:0/377423366 wait complete. 2024-06-03T03:21:01.988 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:01.987+0000 7fb4bfe00700 1 Processor -- start 2024-06-03T03:21:01.988 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:01.987+0000 7fb4bfe00700 1 -- start start 2024-06-03T03:21:01.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:01.987+0000 7fb4bfe00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb4b8108ac0 0x7fb4b819f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:21:01.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:01.987+0000 7fb4bfe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4b8109480 0x7fb4b819f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:21:01.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:01.987+0000 7fb4bfe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4b810eac0 0x7fb4b81a3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:21:01.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:01.987+0000 7fb4bfe00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b8118ea0 con 0x7fb4b8109480 2024-06-03T03:21:01.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:01.987+0000 7fb4bfe00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b8118d20 con 0x7fb4b810eac0 2024-06-03T03:21:01.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:01.987+0000 7fb4bfe00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b8119020 con 0x7fb4b8108ac0 2024-06-03T03:21:01.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:01.988+0000 7fb4bca00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4b8109480 0x7fb4b819f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:21:01.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:01.988+0000 7fb4bd400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb4b8108ac0 0x7fb4b819f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:21:01.990 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:01.988+0000 7fb4bd400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb4b8108ac0 0x7fb4b819f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:42362/0 (socket says 172.21.15.117:42362) 2024-06-03T03:21:01.990 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:01.988+0000 7fb4bd400700 1 -- 172.21.15.117:0/3203808179 learned_addr learned my addr 172.21.15.117:0/3203808179 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:21:01.990 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:01.988+0000 7fb4bde00700 1 --2- 172.21.15.117:0/3203808179 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4b810eac0 0x7fb4b81a3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:21:01.991 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:01.988+0000 7fb4bd400700 1 -- 172.21.15.117:0/3203808179 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4b810eac0 msgr2=0x7fb4b81a3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:21:01.991 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:01.988+0000 7fb4bd400700 1 --2- 172.21.15.117:0/3203808179 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4b810eac0 0x7fb4b81a3ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:01.991 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:01.988+0000 7fb4bd400700 1 -- 172.21.15.117:0/3203808179 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4b8109480 msgr2=0x7fb4b819f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:21:01.991 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:01.988+0000 7fb4bd400700 1 --2- 172.21.15.117:0/3203808179 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4b8109480 0x7fb4b819f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:01.991 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:01.988+0000 7fb4bd400700 1 -- 172.21.15.117:0/3203808179 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4b81a4220 con 0x7fb4b8108ac0 2024-06-03T03:21:01.991 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:01.988+0000 7fb4bca00700 1 --2- 172.21.15.117:0/3203808179 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4b8109480 0x7fb4b819f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:21:01.991 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:01.989+0000 7fb4bd400700 1 --2- 172.21.15.117:0/3203808179 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb4b8108ac0 0x7fb4b819f210 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac00eb30 tx=0x7fb4ac00ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:21:01.991 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:01.989+0000 7fb4b6000700 1 -- 172.21.15.117:0/3203808179 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4ac00cbd0 con 0x7fb4b8108ac0 2024-06-03T03:21:01.991 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:01.989+0000 7fb4b6000700 1 -- 172.21.15.117:0/3203808179 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4ac004500 con 0x7fb4b8108ac0 2024-06-03T03:21:01.991 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:01.989+0000 7fb4bfe00700 1 -- 172.21.15.117:0/3203808179 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4b81a4510 con 0x7fb4b8108ac0 2024-06-03T03:21:01.991 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:01.989+0000 7fb4b6000700 1 -- 172.21.15.117:0/3203808179 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4ac010430 con 0x7fb4b8108ac0 2024-06-03T03:21:01.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:01.989+0000 7fb4bfe00700 1 -- 172.21.15.117:0/3203808179 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4b81abd90 con 0x7fb4b8108ac0 2024-06-03T03:21:01.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:01.990+0000 7fb4bfe00700 1 -- 172.21.15.117:0/3203808179 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4b8069180 con 0x7fb4b8108ac0 2024-06-03T03:21:01.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:01.992+0000 7fb4b6000700 1 -- 172.21.15.117:0/3203808179 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fb4ac003f00 con 0x7fb4b8108ac0 2024-06-03T03:21:01.994 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:01.993+0000 7fb4b6000700 1 --2- 172.21.15.117:0/3203808179 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fb4a00a2c20 0x7fb4a00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:21:01.994 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:01.994+0000 7fb4b6000700 1 -- 172.21.15.117:0/3203808179 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7fb4ac0cf700 con 0x7fb4b8108ac0 2024-06-03T03:21:01.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:01.994+0000 7fb4bca00700 1 --2- 172.21.15.117:0/3203808179 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fb4a00a2c20 0x7fb4a00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:21:01.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:01.994+0000 7fb4bca00700 1 --2- 172.21.15.117:0/3203808179 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fb4a00a2c20 0x7fb4a00a50e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb4b8077ac0 tx=0x7fb4a4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:21:01.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:01.998+0000 7fb4b6000700 1 -- 172.21.15.117:0/3203808179 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb4ac09c2a0 con 0x7fb4b8108ac0 2024-06-03T03:21:02.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:01 smithi183 ceph-mon[37326]: pgmap v375: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-06-03T03:21:02.272 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:02.271+0000 7fb4bfe00700 1 -- 172.21.15.117:0/3203808179 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7fb4b81abfc0 con 0x7fb4b8108ac0 2024-06-03T03:21:02.273 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:01 smithi117 ceph-mon[31649]: pgmap v375: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-06-03T03:21:02.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:01 smithi151 ceph-mon[37043]: pgmap v375: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-06-03T03:21:02.970 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:02.969+0000 7fb4b6000700 1 -- 172.21.15.117:0/3203808179 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7fb4ac09bba0 con 0x7fb4b8108ac0 2024-06-03T03:21:02.970 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:02 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/3203808179' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-06-03T03:21:02.970 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:02 smithi117 ceph-mon[31649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-06-03T03:21:02.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:02.973+0000 7fb4bfe00700 1 -- 172.21.15.117:0/3203808179 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fb4a00a2c20 msgr2=0x7fb4a00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:21:02.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:02.973+0000 7fb4bfe00700 1 --2- 172.21.15.117:0/3203808179 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fb4a00a2c20 0x7fb4a00a50e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb4b8077ac0 tx=0x7fb4a4009260 comp rx=0 tx=0).stop 2024-06-03T03:21:02.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:02.973+0000 7fb4bfe00700 1 -- 172.21.15.117:0/3203808179 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb4b8108ac0 msgr2=0x7fb4b819f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:21:02.975 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:02.973+0000 7fb4bfe00700 1 --2- 172.21.15.117:0/3203808179 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb4b8108ac0 0x7fb4b819f210 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac00eb30 tx=0x7fb4ac00ee40 comp rx=0 tx=0).stop 2024-06-03T03:21:02.975 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:02.973+0000 7fb4bfe00700 1 -- 172.21.15.117:0/3203808179 shutdown_connections 2024-06-03T03:21:02.975 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:02.973+0000 7fb4bfe00700 1 --2- 172.21.15.117:0/3203808179 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fb4a00a2c20 0x7fb4a00a50e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:02.975 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:02.973+0000 7fb4bfe00700 1 --2- 172.21.15.117:0/3203808179 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4b810eac0 0x7fb4b81a3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:02.975 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:02.973+0000 7fb4bfe00700 1 --2- 172.21.15.117:0/3203808179 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb4b8108ac0 0x7fb4b819f210 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:02.975 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:02.973+0000 7fb4bfe00700 1 --2- 172.21.15.117:0/3203808179 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4b8109480 0x7fb4b819f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:02.975 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:02.973+0000 7fb4bfe00700 1 -- 172.21.15.117:0/3203808179 >> 172.21.15.117:0/3203808179 conn(0x7fb4b8075c60 msgr2=0x7fb4b8076140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:21:02.975 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:02.973+0000 7fb4bfe00700 1 -- 172.21.15.117:0/3203808179 shutdown_connections 2024-06-03T03:21:02.975 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:02.974+0000 7fb4bfe00700 1 -- 172.21.15.117:0/3203808179 wait complete. 2024-06-03T03:21:03.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:02 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/3203808179' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-06-03T03:21:03.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:02 smithi183 ceph-mon[37326]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-06-03T03:21:03.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:02 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/3203808179' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-06-03T03:21:03.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:02 smithi151 ceph-mon[37043]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-06-03T03:21:03.971 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-06-03T03:21:04.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:03 smithi117 ceph-mon[31649]: pgmap v376: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-06-03T03:21:04.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:03 smithi117 ceph-mon[31649]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-06-03T03:21:04.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:03 smithi117 ceph-mon[31649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-06-03T03:21:04.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:03 smithi117 ceph-mon[31649]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-06-03T03:21:04.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:03 smithi117 ceph-mon[31649]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-06-03T03:21:04.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:03 smithi117 ceph-mon[31649]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-06-03T03:21:04.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:03 smithi117 ceph-mon[31649]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-06-03T03:21:04.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:03 smithi183 ceph-mon[37326]: pgmap v376: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-06-03T03:21:04.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:03 smithi183 ceph-mon[37326]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-06-03T03:21:04.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:03 smithi183 ceph-mon[37326]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-06-03T03:21:04.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:03 smithi183 ceph-mon[37326]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-06-03T03:21:04.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:03 smithi183 ceph-mon[37326]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-06-03T03:21:04.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:03 smithi183 ceph-mon[37326]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-06-03T03:21:04.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:03 smithi183 ceph-mon[37326]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-06-03T03:21:04.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:03 smithi151 ceph-mon[37043]: pgmap v376: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-06-03T03:21:04.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:03 smithi151 ceph-mon[37043]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-06-03T03:21:04.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:03 smithi151 ceph-mon[37043]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-06-03T03:21:04.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:03 smithi151 ceph-mon[37043]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-06-03T03:21:04.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:03 smithi151 ceph-mon[37043]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-06-03T03:21:04.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:03 smithi151 ceph-mon[37043]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-06-03T03:21:04.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:03 smithi151 ceph-mon[37043]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-06-03T03:21:04.433 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:21:05.099 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:04 smithi117 ceph-mon[31649]: mds.? [v2:172.21.15.151:6838/3623444616,v1:172.21.15.151:6839/3623444616] up:active 2024-06-03T03:21:05.099 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:04 smithi117 ceph-mon[31649]: daemon mds.a assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-06-03T03:21:05.100 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:04 smithi117 ceph-mon[31649]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-06-03T03:21:05.100 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:04 smithi117 ceph-mon[31649]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=a=up:creating} 9 up:standby 2024-06-03T03:21:05.100 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:04 smithi117 ceph-mon[31649]: daemon mds.a is now active in filesystem cephfs as rank 2 2024-06-03T03:21:05.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:04 smithi183 ceph-mon[37326]: mds.? [v2:172.21.15.151:6838/3623444616,v1:172.21.15.151:6839/3623444616] up:active 2024-06-03T03:21:05.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:04 smithi183 ceph-mon[37326]: daemon mds.a assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-06-03T03:21:05.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:04 smithi183 ceph-mon[37326]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-06-03T03:21:05.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:04 smithi183 ceph-mon[37326]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=a=up:creating} 9 up:standby 2024-06-03T03:21:05.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:04 smithi183 ceph-mon[37326]: daemon mds.a is now active in filesystem cephfs as rank 2 2024-06-03T03:21:05.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:04 smithi151 ceph-mon[37043]: mds.? [v2:172.21.15.151:6838/3623444616,v1:172.21.15.151:6839/3623444616] up:active 2024-06-03T03:21:05.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:04 smithi151 ceph-mon[37043]: daemon mds.a assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-06-03T03:21:05.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:04 smithi151 ceph-mon[37043]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-06-03T03:21:05.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:04 smithi151 ceph-mon[37043]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=a=up:creating} 9 up:standby 2024-06-03T03:21:05.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:04 smithi151 ceph-mon[37043]: daemon mds.a is now active in filesystem cephfs as rank 2 2024-06-03T03:21:06.331 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:05 smithi117 ceph-mon[31649]: pgmap v377: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-06-03T03:21:06.331 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:05 smithi117 ceph-mon[31649]: mds.? [v2:172.21.15.117:6840/1212242967,v1:172.21.15.117:6841/1212242967] up:active 2024-06-03T03:21:06.331 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:05 smithi117 ceph-mon[31649]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-06-03T03:21:06.331 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:05 smithi117 ceph-mon[31649]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=a=up:active} 9 up:standby 2024-06-03T03:21:06.331 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:05 smithi117 ceph-mon[31649]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=a=up:active,3=c=up:creating} 8 up:standby 2024-06-03T03:21:06.331 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:05 smithi117 ceph-mon[31649]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-06-03T03:21:06.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:05 smithi151 ceph-mon[37043]: pgmap v377: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-06-03T03:21:06.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:05 smithi151 ceph-mon[37043]: mds.? [v2:172.21.15.117:6840/1212242967,v1:172.21.15.117:6841/1212242967] up:active 2024-06-03T03:21:06.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:05 smithi151 ceph-mon[37043]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-06-03T03:21:06.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:05 smithi151 ceph-mon[37043]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=a=up:active} 9 up:standby 2024-06-03T03:21:06.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:05 smithi151 ceph-mon[37043]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=a=up:active,3=c=up:creating} 8 up:standby 2024-06-03T03:21:06.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:05 smithi151 ceph-mon[37043]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-06-03T03:21:06.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:05 smithi183 ceph-mon[37326]: pgmap v377: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-06-03T03:21:06.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:05 smithi183 ceph-mon[37326]: mds.? [v2:172.21.15.117:6840/1212242967,v1:172.21.15.117:6841/1212242967] up:active 2024-06-03T03:21:06.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:05 smithi183 ceph-mon[37326]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-06-03T03:21:06.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:05 smithi183 ceph-mon[37326]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=a=up:active} 9 up:standby 2024-06-03T03:21:06.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:05 smithi183 ceph-mon[37326]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=a=up:active,3=c=up:creating} 8 up:standby 2024-06-03T03:21:06.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:05 smithi183 ceph-mon[37326]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-06-03T03:21:06.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:06.912+0000 7f8f8de00700 1 -- 172.21.15.117:0/2265353022 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f881046b0 msgr2=0x7f8f88104b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:21:06.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:06.912+0000 7f8f8de00700 1 --2- 172.21.15.117:0/2265353022 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f881046b0 0x7f8f88104b30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8f74009990 tx=0x7f8f74009ca0 comp rx=0 tx=0).stop 2024-06-03T03:21:06.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:06.912+0000 7f8f8de00700 1 -- 172.21.15.117:0/2265353022 shutdown_connections 2024-06-03T03:21:06.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:06.912+0000 7f8f8de00700 1 --2- 172.21.15.117:0/2265353022 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f88105070 0x7f8f88111580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:06.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:06.912+0000 7f8f8de00700 1 --2- 172.21.15.117:0/2265353022 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f8810a6b0 0x7f8f8810aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:06.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:06.912+0000 7f8f8de00700 1 --2- 172.21.15.117:0/2265353022 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f881046b0 0x7f8f88104b30 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:06.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:06.912+0000 7f8f8de00700 1 -- 172.21.15.117:0/2265353022 >> 172.21.15.117:0/2265353022 conn(0x7f8f881001a0 msgr2=0x7f8f881025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:21:06.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:06.912+0000 7f8f8de00700 1 -- 172.21.15.117:0/2265353022 shutdown_connections 2024-06-03T03:21:06.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:06.912+0000 7f8f8de00700 1 -- 172.21.15.117:0/2265353022 wait complete. 2024-06-03T03:21:06.916 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:06.913+0000 7f8f8de00700 1 Processor -- start 2024-06-03T03:21:06.916 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:06.914+0000 7f8f8de00700 1 -- start start 2024-06-03T03:21:06.916 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:06.914+0000 7f8f8de00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f881046b0 0x7f8f8819ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:21:06.916 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:06.914+0000 7f8f8de00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f88105070 0x7f8f8819b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:21:06.916 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:06.914+0000 7f8f8de00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f8810a6b0 0x7f8f8819f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:21:06.916 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:06.914+0000 7f8f8de00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8f881149e0 con 0x7f8f88105070 2024-06-03T03:21:06.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:06.914+0000 7f8f8de00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8f88114860 con 0x7f8f8810a6b0 2024-06-03T03:21:06.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:06.914+0000 7f8f8de00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8f88114b60 con 0x7f8f881046b0 2024-06-03T03:21:06.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:06.914+0000 7f8f86000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f88105070 0x7f8f8819b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:21:06.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:06.914+0000 7f8f86a00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f881046b0 0x7f8f8819ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:21:06.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:06.914+0000 7f8f86000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f88105070 0x7f8f8819b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41938/0 (socket says 172.21.15.117:41938) 2024-06-03T03:21:06.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:06.914+0000 7f8f86a00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f881046b0 0x7f8f8819ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:58448/0 (socket says 172.21.15.117:58448) 2024-06-03T03:21:06.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:06.915+0000 7f8f87400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f8810a6b0 0x7f8f8819f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:21:06.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:06.915+0000 7f8f86000700 1 -- 172.21.15.117:0/1588541689 learned_addr learned my addr 172.21.15.117:0/1588541689 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:21:06.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:06.915+0000 7f8f86a00700 1 -- 172.21.15.117:0/1588541689 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f8810a6b0 msgr2=0x7f8f8819f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:21:06.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:06.915+0000 7f8f86a00700 1 --2- 172.21.15.117:0/1588541689 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f8810a6b0 0x7f8f8819f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:06.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:06.915+0000 7f8f86a00700 1 -- 172.21.15.117:0/1588541689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f88105070 msgr2=0x7f8f8819b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:21:06.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:06.915+0000 7f8f86a00700 1 --2- 172.21.15.117:0/1588541689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f88105070 0x7f8f8819b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:06.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:06.915+0000 7f8f86a00700 1 -- 172.21.15.117:0/1588541689 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f8819fe40 con 0x7f8f881046b0 2024-06-03T03:21:06.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:06.915+0000 7f8f86a00700 1 --2- 172.21.15.117:0/1588541689 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f881046b0 0x7f8f8819ae30 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8f7800e9f0 tx=0x7f8f7800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:21:06.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:06.916+0000 7f8f8ca00700 1 -- 172.21.15.117:0/1588541689 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f7800c9c0 con 0x7f8f881046b0 2024-06-03T03:21:06.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:06.916+0000 7f8f8de00700 1 -- 172.21.15.117:0/1588541689 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f881a0130 con 0x7f8f881046b0 2024-06-03T03:21:06.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:06.916+0000 7f8f8de00700 1 -- 172.21.15.117:0/1588541689 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f881a79b0 con 0x7f8f881046b0 2024-06-03T03:21:06.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:06.916+0000 7f8f8ca00700 1 -- 172.21.15.117:0/1588541689 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f7800cb60 con 0x7f8f881046b0 2024-06-03T03:21:06.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:06.916+0000 7f8f8ca00700 1 -- 172.21.15.117:0/1588541689 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f78018680 con 0x7f8f881046b0 2024-06-03T03:21:06.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:06.917+0000 7f8f8de00700 1 -- 172.21.15.117:0/1588541689 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f68005330 con 0x7f8f881046b0 2024-06-03T03:21:06.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:06.919+0000 7f8f8ca00700 1 -- 172.21.15.117:0/1588541689 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f8f7800cd10 con 0x7f8f881046b0 2024-06-03T03:21:06.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:06.920+0000 7f8f8ca00700 1 --2- 172.21.15.117:0/1588541689 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f8f6c0a2ca0 0x7f8f6c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:21:06.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:06.921+0000 7f8f8ca00700 1 -- 172.21.15.117:0/1588541689 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7f8f780cf490 con 0x7f8f881046b0 2024-06-03T03:21:06.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:06.921+0000 7f8f86000700 1 --2- 172.21.15.117:0/1588541689 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f8f6c0a2ca0 0x7f8f6c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:21:06.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:06.922+0000 7f8f86000700 1 --2- 172.21.15.117:0/1588541689 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f8f6c0a2ca0 0x7f8f6c0a5160 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8f88109c00 tx=0x7f8f74009590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:21:06.923 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:06.923+0000 7f8f8ca00700 1 -- 172.21.15.117:0/1588541689 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8f7809c030 con 0x7f8f881046b0 2024-06-03T03:21:07.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:07 smithi117 ceph-mon[31649]: mds.? [v2:172.21.15.183:6836/2697518526,v1:172.21.15.183:6837/2697518526] up:active 2024-06-03T03:21:07.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:07 smithi117 ceph-mon[31649]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-06-03T03:21:07.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:07 smithi117 ceph-mon[31649]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-06-03T03:21:07.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:07 smithi117 ceph-mon[31649]: Cluster is now healthy 2024-06-03T03:21:07.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:07 smithi117 ceph-mon[31649]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=a=up:active,3=c=up:active} 8 up:standby 2024-06-03T03:21:07.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:07 smithi117 ceph-mon[31649]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-06-03T03:21:07.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:07 smithi117 ceph-mon[31649]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-06-03T03:21:07.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:07 smithi117 ceph-mon[31649]: pgmap v378: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s wr, 4 op/s 2024-06-03T03:21:07.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:07.195+0000 7f8f8de00700 1 -- 172.21.15.117:0/1588541689 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f8f68005620 con 0x7f8f881046b0 2024-06-03T03:21:07.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:07 smithi151 ceph-mon[37043]: mds.? [v2:172.21.15.183:6836/2697518526,v1:172.21.15.183:6837/2697518526] up:active 2024-06-03T03:21:07.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:07 smithi151 ceph-mon[37043]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-06-03T03:21:07.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:07 smithi151 ceph-mon[37043]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-06-03T03:21:07.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:07 smithi151 ceph-mon[37043]: Cluster is now healthy 2024-06-03T03:21:07.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:07 smithi151 ceph-mon[37043]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=a=up:active,3=c=up:active} 8 up:standby 2024-06-03T03:21:07.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:07 smithi151 ceph-mon[37043]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-06-03T03:21:07.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:07 smithi151 ceph-mon[37043]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-06-03T03:21:07.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:07 smithi151 ceph-mon[37043]: pgmap v378: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s wr, 4 op/s 2024-06-03T03:21:07.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:07 smithi183 ceph-mon[37326]: mds.? [v2:172.21.15.183:6836/2697518526,v1:172.21.15.183:6837/2697518526] up:active 2024-06-03T03:21:07.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:07 smithi183 ceph-mon[37326]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-06-03T03:21:07.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:07 smithi183 ceph-mon[37326]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-06-03T03:21:07.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:07 smithi183 ceph-mon[37326]: Cluster is now healthy 2024-06-03T03:21:07.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:07 smithi183 ceph-mon[37326]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=a=up:active,3=c=up:active} 8 up:standby 2024-06-03T03:21:07.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:07 smithi183 ceph-mon[37326]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-06-03T03:21:07.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:07 smithi183 ceph-mon[37326]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-06-03T03:21:07.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:07 smithi183 ceph-mon[37326]: pgmap v378: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s wr, 4 op/s 2024-06-03T03:21:08.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:08.044+0000 7f8f8ca00700 1 -- 172.21.15.117:0/1588541689 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v28) ==== 121+0+0 (secure 0 0 0) 0x7f8f7809b930 con 0x7f8f881046b0 2024-06-03T03:21:08.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:08.047+0000 7f8f8de00700 1 -- 172.21.15.117:0/1588541689 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f8f6c0a2ca0 msgr2=0x7f8f6c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:21:08.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:08.047+0000 7f8f8de00700 1 --2- 172.21.15.117:0/1588541689 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f8f6c0a2ca0 0x7f8f6c0a5160 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8f88109c00 tx=0x7f8f74009590 comp rx=0 tx=0).stop 2024-06-03T03:21:08.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:08.047+0000 7f8f8de00700 1 -- 172.21.15.117:0/1588541689 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f881046b0 msgr2=0x7f8f8819ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:21:08.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:08.047+0000 7f8f8de00700 1 --2- 172.21.15.117:0/1588541689 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f881046b0 0x7f8f8819ae30 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8f7800e9f0 tx=0x7f8f7800ed00 comp rx=0 tx=0).stop 2024-06-03T03:21:08.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:08.047+0000 7f8f8de00700 1 -- 172.21.15.117:0/1588541689 shutdown_connections 2024-06-03T03:21:08.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:08.047+0000 7f8f8de00700 1 --2- 172.21.15.117:0/1588541689 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f8f6c0a2ca0 0x7f8f6c0a5160 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:08.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:08.048+0000 7f8f8de00700 1 --2- 172.21.15.117:0/1588541689 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f8810a6b0 0x7f8f8819f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:08.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:08.048+0000 7f8f8de00700 1 --2- 172.21.15.117:0/1588541689 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f881046b0 0x7f8f8819ae30 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:08.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:08.048+0000 7f8f8de00700 1 --2- 172.21.15.117:0/1588541689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f88105070 0x7f8f8819b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:08.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:08.048+0000 7f8f8de00700 1 -- 172.21.15.117:0/1588541689 >> 172.21.15.117:0/1588541689 conn(0x7f8f881001a0 msgr2=0x7f8f881006c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:21:08.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:08.048+0000 7f8f8de00700 1 -- 172.21.15.117:0/1588541689 shutdown_connections 2024-06-03T03:21:08.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:08.048+0000 7f8f8de00700 1 -- 172.21.15.117:0/1588541689 wait complete. 2024-06-03T03:21:08.232 INFO:journalctl@ceph.mds.i.smithi183.stdout:Jun 03 03:21:07 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-i[67909]: 2024-06-03T03:21:07.894+0000 7f655277c700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-06-03T03:21:08.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:08 smithi183 ceph-mon[37326]: mds.? [v2:172.21.15.183:6834/1009004320,v1:172.21.15.183:6835/1009004320] up:active 2024-06-03T03:21:08.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:08 smithi183 ceph-mon[37326]: fsmap cephfs:5 7 up:standby 5 up:active 2024-06-03T03:21:08.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:08 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/1588541689' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-06-03T03:21:08.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:08 smithi183 ceph-mon[37326]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-06-03T03:21:08.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:08 smithi151 ceph-mon[37043]: mds.? [v2:172.21.15.183:6834/1009004320,v1:172.21.15.183:6835/1009004320] up:active 2024-06-03T03:21:08.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:08 smithi151 ceph-mon[37043]: fsmap cephfs:5 7 up:standby 5 up:active 2024-06-03T03:21:08.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:08 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/1588541689' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-06-03T03:21:08.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:08 smithi151 ceph-mon[37043]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-06-03T03:21:08.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:08 smithi117 ceph-mon[31649]: mds.? [v2:172.21.15.183:6834/1009004320,v1:172.21.15.183:6835/1009004320] up:active 2024-06-03T03:21:08.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:08 smithi117 ceph-mon[31649]: fsmap cephfs:5 7 up:standby 5 up:active 2024-06-03T03:21:08.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:08 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/1588541689' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-06-03T03:21:08.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:08 smithi117 ceph-mon[31649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-06-03T03:21:09.093 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-06-03T03:21:09.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:09 smithi151 ceph-mon[37043]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-06-03T03:21:09.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:09 smithi151 ceph-mon[37043]: fsmap cephfs:5 7 up:standby 5 up:active 2024-06-03T03:21:09.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:09 smithi151 ceph-mon[37043]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T03:21:09.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:09 smithi151 ceph-mon[37043]: pgmap v379: 129 pgs: 129 active+clean; 585 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s rd, 3.6 KiB/s wr, 10 op/s 2024-06-03T03:21:09.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:09 smithi117 ceph-mon[31649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-06-03T03:21:09.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:09 smithi117 ceph-mon[31649]: fsmap cephfs:5 7 up:standby 5 up:active 2024-06-03T03:21:09.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:09 smithi117 ceph-mon[31649]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T03:21:09.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:09 smithi117 ceph-mon[31649]: pgmap v379: 129 pgs: 129 active+clean; 585 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s rd, 3.6 KiB/s wr, 10 op/s 2024-06-03T03:21:09.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:09 smithi183 ceph-mon[37326]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-06-03T03:21:09.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:09 smithi183 ceph-mon[37326]: fsmap cephfs:5 7 up:standby 5 up:active 2024-06-03T03:21:09.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:09 smithi183 ceph-mon[37326]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T03:21:09.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:09 smithi183 ceph-mon[37326]: pgmap v379: 129 pgs: 129 active+clean; 585 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s rd, 3.6 KiB/s wr, 10 op/s 2024-06-03T03:21:09.530 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:21:11.797 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:11.795+0000 7fa501c00700 1 -- 172.21.15.117:0/2191713755 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa4fc101390 msgr2=0x7fa4fc101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:21:11.797 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:11.795+0000 7fa501c00700 1 --2- 172.21.15.117:0/2191713755 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa4fc101390 0x7fa4fc101810 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa4e8009990 tx=0x7fa4e8009ca0 comp rx=0 tx=0).stop 2024-06-03T03:21:11.797 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:11.796+0000 7fa501c00700 1 -- 172.21.15.117:0/2191713755 shutdown_connections 2024-06-03T03:21:11.797 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:11.796+0000 7fa501c00700 1 --2- 172.21.15.117:0/2191713755 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4fc101d50 0x7fa4fc10e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:11.797 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:11.796+0000 7fa501c00700 1 --2- 172.21.15.117:0/2191713755 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa4fc107400 0x7fa4fc1077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:11.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:11.796+0000 7fa501c00700 1 --2- 172.21.15.117:0/2191713755 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa4fc101390 0x7fa4fc101810 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:11.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:11.796+0000 7fa501c00700 1 -- 172.21.15.117:0/2191713755 >> 172.21.15.117:0/2191713755 conn(0x7fa4fc078340 msgr2=0x7fa4fc078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:21:11.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:11.796+0000 7fa501c00700 1 -- 172.21.15.117:0/2191713755 shutdown_connections 2024-06-03T03:21:11.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:11.796+0000 7fa501c00700 1 -- 172.21.15.117:0/2191713755 wait complete. 2024-06-03T03:21:11.799 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:11.797+0000 7fa501c00700 1 Processor -- start 2024-06-03T03:21:11.799 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:11.797+0000 7fa501c00700 1 -- start start 2024-06-03T03:21:11.799 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:11.797+0000 7fa501c00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa4fc101390 0x7fa4fc103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:21:11.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:11.797+0000 7fa501c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4fc101d50 0x7fa4fc1035b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:21:11.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:11.797+0000 7fa501c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa4fc107400 0x7fa4fc103af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:21:11.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:11.797+0000 7fa501c00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa4fc1116e0 con 0x7fa4fc101d50 2024-06-03T03:21:11.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:11.797+0000 7fa501c00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa4fc111560 con 0x7fa4fc107400 2024-06-03T03:21:11.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:11.797+0000 7fa501c00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa4fc111860 con 0x7fa4fc101390 2024-06-03T03:21:11.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:11.798+0000 7fa4faa00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa4fc101390 0x7fa4fc103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:21:11.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:11.798+0000 7fa4fa000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4fc101d50 0x7fa4fc1035b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:21:11.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:11.798+0000 7fa4faa00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa4fc101390 0x7fa4fc103070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:58480/0 (socket says 172.21.15.117:58480) 2024-06-03T03:21:11.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:11.798+0000 7fa4faa00700 1 -- 172.21.15.117:0/918235258 learned_addr learned my addr 172.21.15.117:0/918235258 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:21:11.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:11.798+0000 7fa4fb400700 1 --2- 172.21.15.117:0/918235258 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa4fc107400 0x7fa4fc103af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:21:11.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:11.798+0000 7fa4fa000700 1 -- 172.21.15.117:0/918235258 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa4fc101390 msgr2=0x7fa4fc103070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:21:11.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:11.798+0000 7fa4fa000700 1 --2- 172.21.15.117:0/918235258 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa4fc101390 0x7fa4fc103070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:11.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:11.798+0000 7fa4fa000700 1 -- 172.21.15.117:0/918235258 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa4fc107400 msgr2=0x7fa4fc103af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:21:11.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:11.798+0000 7fa4fa000700 1 --2- 172.21.15.117:0/918235258 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa4fc107400 0x7fa4fc103af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:11.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:11.798+0000 7fa4fa000700 1 -- 172.21.15.117:0/918235258 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4fc1a71c0 con 0x7fa4fc101d50 2024-06-03T03:21:11.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:11.798+0000 7fa4faa00700 1 --2- 172.21.15.117:0/918235258 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa4fc101390 0x7fa4fc103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:21:11.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:11.799+0000 7fa4fa000700 1 --2- 172.21.15.117:0/918235258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4fc101d50 0x7fa4fc1035b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa4e80098e0 tx=0x7fa4e800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:21:11.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:11.799+0000 7fa4f3400700 1 -- 172.21.15.117:0/918235258 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e8027070 con 0x7fa4fc101d50 2024-06-03T03:21:11.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:11.799+0000 7fa4f3400700 1 -- 172.21.15.117:0/918235258 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4e800be60 con 0x7fa4fc101d50 2024-06-03T03:21:11.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:11.800+0000 7fa4f3400700 1 -- 172.21.15.117:0/918235258 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e8022dd0 con 0x7fa4fc101d50 2024-06-03T03:21:11.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:11.800+0000 7fa501c00700 1 -- 172.21.15.117:0/918235258 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4fc1a7450 con 0x7fa4fc101d50 2024-06-03T03:21:11.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:11.800+0000 7fa501c00700 1 -- 172.21.15.117:0/918235258 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4fc1a7850 con 0x7fa4fc101d50 2024-06-03T03:21:11.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:11.801+0000 7fa501c00700 1 -- 172.21.15.117:0/918235258 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4fc069180 con 0x7fa4fc101d50 2024-06-03T03:21:11.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:11.803+0000 7fa4f3400700 1 -- 172.21.15.117:0/918235258 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fa4e8009d10 con 0x7fa4fc101d50 2024-06-03T03:21:11.805 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:11.804+0000 7fa4f3400700 1 --2- 172.21.15.117:0/918235258 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fa4e00a2c20 0x7fa4e00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:21:11.805 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:11.805+0000 7fa4f3400700 1 -- 172.21.15.117:0/918235258 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7fa4e80cfda0 con 0x7fa4fc101d50 2024-06-03T03:21:11.805 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:11.805+0000 7fa4faa00700 1 --2- 172.21.15.117:0/918235258 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fa4e00a2c20 0x7fa4e00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:21:11.806 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:11.805+0000 7fa4faa00700 1 --2- 172.21.15.117:0/918235258 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fa4e00a2c20 0x7fa4e00a50e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec011550 tx=0x7fa4ec009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:21:11.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:11.809+0000 7fa4f3400700 1 -- 172.21.15.117:0/918235258 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa4e805d850 con 0x7fa4fc101d50 2024-06-03T03:21:12.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:12.084+0000 7fa501c00700 1 -- 172.21.15.117:0/918235258 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fa4fc1a7b90 con 0x7fa4fc101d50 2024-06-03T03:21:12.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:11 smithi151 ceph-mon[37043]: pgmap v380: 129 pgs: 129 active+clean; 586 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 3.9 KiB/s wr, 14 op/s 2024-06-03T03:21:12.185 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:11 smithi183 ceph-mon[37326]: pgmap v380: 129 pgs: 129 active+clean; 586 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 3.9 KiB/s wr, 14 op/s 2024-06-03T03:21:12.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:11 smithi117 ceph-mon[31649]: pgmap v380: 129 pgs: 129 active+clean; 586 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 3.9 KiB/s wr, 14 op/s 2024-06-03T03:21:12.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:12.907+0000 7fa4f3400700 1 -- 172.21.15.117:0/918235258 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v30) ==== 115+0+0 (secure 0 0 0) 0x7fa4e8013070 con 0x7fa4fc101d50 2024-06-03T03:21:12.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:12.912+0000 7fa501c00700 1 -- 172.21.15.117:0/918235258 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fa4e00a2c20 msgr2=0x7fa4e00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:21:12.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:12.912+0000 7fa501c00700 1 --2- 172.21.15.117:0/918235258 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fa4e00a2c20 0x7fa4e00a50e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec011550 tx=0x7fa4ec009260 comp rx=0 tx=0).stop 2024-06-03T03:21:12.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:12.913+0000 7fa501c00700 1 -- 172.21.15.117:0/918235258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4fc101d50 msgr2=0x7fa4fc1035b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:21:12.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:12.913+0000 7fa501c00700 1 --2- 172.21.15.117:0/918235258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4fc101d50 0x7fa4fc1035b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa4e80098e0 tx=0x7fa4e800b8a0 comp rx=0 tx=0).stop 2024-06-03T03:21:12.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:12.913+0000 7fa501c00700 1 -- 172.21.15.117:0/918235258 shutdown_connections 2024-06-03T03:21:12.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:12.913+0000 7fa501c00700 1 --2- 172.21.15.117:0/918235258 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fa4e00a2c20 0x7fa4e00a50e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:12.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:12.913+0000 7fa501c00700 1 --2- 172.21.15.117:0/918235258 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa4fc107400 0x7fa4fc103af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:12.915 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:12.913+0000 7fa501c00700 1 --2- 172.21.15.117:0/918235258 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa4fc101390 0x7fa4fc103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:12.915 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:12.913+0000 7fa501c00700 1 --2- 172.21.15.117:0/918235258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4fc101d50 0x7fa4fc1035b0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:12.915 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:12.913+0000 7fa501c00700 1 -- 172.21.15.117:0/918235258 >> 172.21.15.117:0/918235258 conn(0x7fa4fc078340 msgr2=0x7fa4fc100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:21:12.915 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:12.913+0000 7fa501c00700 1 -- 172.21.15.117:0/918235258 shutdown_connections 2024-06-03T03:21:12.915 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:12.913+0000 7fa501c00700 1 -- 172.21.15.117:0/918235258 wait complete. 2024-06-03T03:21:13.158 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:12 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/918235258' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-06-03T03:21:13.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:12 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/918235258' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-06-03T03:21:13.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:12 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/918235258' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-06-03T03:21:13.909 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2024-06-03T03:21:13.909 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-06-03T03:21:14.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:13 smithi117 ceph-mon[31649]: pgmap v381: 129 pgs: 129 active+clean; 587 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 4.0 KiB/s wr, 19 op/s 2024-06-03T03:21:14.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:13 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/918235258' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-06-03T03:21:14.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:13 smithi117 ceph-mon[31649]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T03:21:14.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:13 smithi183 ceph-mon[37326]: pgmap v381: 129 pgs: 129 active+clean; 587 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 4.0 KiB/s wr, 19 op/s 2024-06-03T03:21:14.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:13 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/918235258' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-06-03T03:21:14.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:13 smithi183 ceph-mon[37326]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T03:21:14.353 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:21:14.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:13 smithi151 ceph-mon[37043]: pgmap v381: 129 pgs: 129 active+clean; 587 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 4.0 KiB/s wr, 19 op/s 2024-06-03T03:21:14.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:13 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/918235258' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-06-03T03:21:14.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:13 smithi151 ceph-mon[37043]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T03:21:16.171 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:15 smithi117 ceph-mon[31649]: pgmap v382: 129 pgs: 129 active+clean; 587 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 4.0 KiB/s wr, 19 op/s 2024-06-03T03:21:16.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:15 smithi183 ceph-mon[37326]: pgmap v382: 129 pgs: 129 active+clean; 587 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 4.0 KiB/s wr, 19 op/s 2024-06-03T03:21:16.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:15 smithi151 ceph-mon[37043]: pgmap v382: 129 pgs: 129 active+clean; 587 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 4.0 KiB/s wr, 19 op/s 2024-06-03T03:21:16.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:16.690+0000 7fbffe800700 1 -- 172.21.15.117:0/4090662354 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbff8107450 msgr2=0x7fbff8107830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:21:16.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:16.690+0000 7fbffe800700 1 --2- 172.21.15.117:0/4090662354 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbff8107450 0x7fbff8107830 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fbfe8009990 tx=0x7fbfe8009ca0 comp rx=0 tx=0).stop 2024-06-03T03:21:16.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:16.690+0000 7fbffe800700 1 -- 172.21.15.117:0/4090662354 shutdown_connections 2024-06-03T03:21:16.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:16.690+0000 7fbffe800700 1 --2- 172.21.15.117:0/4090662354 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbff8102ee0 0x7fbff810f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:16.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:16.690+0000 7fbffe800700 1 --2- 172.21.15.117:0/4090662354 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbff8107450 0x7fbff8107830 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:16.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:16.690+0000 7fbffe800700 1 --2- 172.21.15.117:0/4090662354 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbff8102540 0x7fbff81029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:16.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:16.690+0000 7fbffe800700 1 -- 172.21.15.117:0/4090662354 >> 172.21.15.117:0/4090662354 conn(0x7fbff80795f0 msgr2=0x7fbff8079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:21:16.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:16.690+0000 7fbffe800700 1 -- 172.21.15.117:0/4090662354 shutdown_connections 2024-06-03T03:21:16.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:16.690+0000 7fbffe800700 1 -- 172.21.15.117:0/4090662354 wait complete. 2024-06-03T03:21:16.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:16.691+0000 7fbffe800700 1 Processor -- start 2024-06-03T03:21:16.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:16.692+0000 7fbffe800700 1 -- start start 2024-06-03T03:21:16.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:16.692+0000 7fbffe800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbff8102540 0x7fbff819ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:21:16.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:16.692+0000 7fbffe800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbff8102ee0 0x7fbff819b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:21:16.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:16.692+0000 7fbffe800700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbff8107450 0x7fbff819f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:21:16.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:16.692+0000 7fbffe800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbff81127c0 con 0x7fbff8102540 2024-06-03T03:21:16.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:16.692+0000 7fbffe800700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbff8112640 con 0x7fbff8102ee0 2024-06-03T03:21:16.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:16.692+0000 7fbffe800700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbff8112940 con 0x7fbff8107450 2024-06-03T03:21:16.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:16.692+0000 7fbff7400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbff8102540 0x7fbff819ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:21:16.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:16.692+0000 7fbff7e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbff8107450 0x7fbff819f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:21:16.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:16.692+0000 7fbff6a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbff8102ee0 0x7fbff819b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:21:16.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:16.692+0000 7fbff7400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbff8102540 0x7fbff819ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41988/0 (socket says 172.21.15.117:41988) 2024-06-03T03:21:16.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:16.692+0000 7fbff7400700 1 -- 172.21.15.117:0/1398405363 learned_addr learned my addr 172.21.15.117:0/1398405363 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:21:16.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:16.693+0000 7fbff6a00700 1 -- 172.21.15.117:0/1398405363 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbff8107450 msgr2=0x7fbff819f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:21:16.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:16.693+0000 7fbff6a00700 1 --2- 172.21.15.117:0/1398405363 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbff8107450 0x7fbff819f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:16.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:16.693+0000 7fbff6a00700 1 -- 172.21.15.117:0/1398405363 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbff8102540 msgr2=0x7fbff819ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:21:16.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:16.693+0000 7fbff6a00700 1 --2- 172.21.15.117:0/1398405363 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbff8102540 0x7fbff819ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:16.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:16.693+0000 7fbff6a00700 1 -- 172.21.15.117:0/1398405363 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbff819fe60 con 0x7fbff8102ee0 2024-06-03T03:21:16.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:16.693+0000 7fbff6a00700 1 --2- 172.21.15.117:0/1398405363 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbff8102ee0 0x7fbff819b390 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fbfe400e9f0 tx=0x7fbfe400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:21:16.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:16.693+0000 7fbfefe00700 1 -- 172.21.15.117:0/1398405363 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe400c9c0 con 0x7fbff8102ee0 2024-06-03T03:21:16.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:16.693+0000 7fbfefe00700 1 -- 172.21.15.117:0/1398405363 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfe400cb60 con 0x7fbff8102ee0 2024-06-03T03:21:16.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:16.693+0000 7fbffe800700 1 -- 172.21.15.117:0/1398405363 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbff81a0150 con 0x7fbff8102ee0 2024-06-03T03:21:16.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:16.693+0000 7fbfefe00700 1 -- 172.21.15.117:0/1398405363 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe4018680 con 0x7fbff8102ee0 2024-06-03T03:21:16.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:16.694+0000 7fbffe800700 1 -- 172.21.15.117:0/1398405363 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbff81a79d0 con 0x7fbff8102ee0 2024-06-03T03:21:16.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:16.695+0000 7fbffe800700 1 -- 172.21.15.117:0/1398405363 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbff8069180 con 0x7fbff8102ee0 2024-06-03T03:21:16.698 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:16.697+0000 7fbfefe00700 1 -- 172.21.15.117:0/1398405363 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fbfe4010430 con 0x7fbff8102ee0 2024-06-03T03:21:16.700 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:16.699+0000 7fbfefe00700 1 --2- 172.21.15.117:0/1398405363 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fbfdc0a2ca0 0x7fbfdc0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:21:16.700 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:16.699+0000 7fbfefe00700 1 -- 172.21.15.117:0/1398405363 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7fbfe40cf790 con 0x7fbff8102ee0 2024-06-03T03:21:16.700 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:16.699+0000 7fbff7400700 1 --2- 172.21.15.117:0/1398405363 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fbfdc0a2ca0 0x7fbfdc0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:21:16.701 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:16.700+0000 7fbff7400700 1 --2- 172.21.15.117:0/1398405363 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fbfdc0a2ca0 0x7fbfdc0a5160 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fbfe8009750 tx=0x7fbfe8005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:21:16.701 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:16.701+0000 7fbfefe00700 1 -- 172.21.15.117:0/1398405363 <== mon.2 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fbfe409c330 con 0x7fbff8102ee0 2024-06-03T03:21:16.967 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:16.965+0000 7fbffe800700 1 -- 172.21.15.117:0/1398405363 --> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fbff8100ac0 con 0x7fbfdc0a2ca0 2024-06-03T03:21:17.002 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:17.000+0000 7fbfefe00700 1 -- 172.21.15.117:0/1398405363 <== mgr.14152 v2:172.21.15.117:6800/1398160887 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fbff8100ac0 con 0x7fbfdc0a2ca0 2024-06-03T03:21:17.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:17.002+0000 7fbffe800700 1 -- 172.21.15.117:0/1398405363 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fbfdc0a2ca0 msgr2=0x7fbfdc0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:21:17.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:17.003+0000 7fbffe800700 1 --2- 172.21.15.117:0/1398405363 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fbfdc0a2ca0 0x7fbfdc0a5160 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fbfe8009750 tx=0x7fbfe8005960 comp rx=0 tx=0).stop 2024-06-03T03:21:17.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:17.003+0000 7fbffe800700 1 -- 172.21.15.117:0/1398405363 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbff8102ee0 msgr2=0x7fbff819b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:21:17.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:17.003+0000 7fbffe800700 1 --2- 172.21.15.117:0/1398405363 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbff8102ee0 0x7fbff819b390 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fbfe400e9f0 tx=0x7fbfe400ed00 comp rx=0 tx=0).stop 2024-06-03T03:21:17.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:17.003+0000 7fbffe800700 1 -- 172.21.15.117:0/1398405363 shutdown_connections 2024-06-03T03:21:17.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:17.003+0000 7fbffe800700 1 --2- 172.21.15.117:0/1398405363 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fbfdc0a2ca0 0x7fbfdc0a5160 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:17.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:17.003+0000 7fbffe800700 1 --2- 172.21.15.117:0/1398405363 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbff8107450 0x7fbff819f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:17.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:17.003+0000 7fbffe800700 1 --2- 172.21.15.117:0/1398405363 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbff8102540 0x7fbff819ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:17.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:17.003+0000 7fbffe800700 1 --2- 172.21.15.117:0/1398405363 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbff8102ee0 0x7fbff819b390 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:17.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:17.003+0000 7fbffe800700 1 -- 172.21.15.117:0/1398405363 >> 172.21.15.117:0/1398405363 conn(0x7fbff80795f0 msgr2=0x7fbff81003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:21:17.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:17.003+0000 7fbffe800700 1 -- 172.21.15.117:0/1398405363 shutdown_connections 2024-06-03T03:21:17.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:17.003+0000 7fbffe800700 1 -- 172.21.15.117:0/1398405363 wait complete. 2024-06-03T03:21:17.429 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:21:16 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:21:16.974+0000 7fbb97600700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-03T03:21:17.430 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:21:16 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:21:16.974+0000 7fbb97600700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-03T03:21:17.430 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:21:16 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:21:16.974+0000 7fbb97600700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-03T03:21:17.431 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:21:16 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:21:16.974+0000 7fbb97600700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-03T03:21:17.431 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:21:16 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:21:16.974+0000 7fbb97600700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-03T03:21:17.875 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-06-03T03:21:18.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:17 smithi117 ceph-mon[31649]: pgmap v383: 129 pgs: 129 active+clean; 587 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 4.0 KiB/s wr, 19 op/s 2024-06-03T03:21:18.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:17 smithi117 ceph-mon[31649]: from='client.24674 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:21:18.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:17 smithi183 ceph-mon[37326]: pgmap v383: 129 pgs: 129 active+clean; 587 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 4.0 KiB/s wr, 19 op/s 2024-06-03T03:21:18.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:17 smithi183 ceph-mon[37326]: from='client.24674 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:21:18.324 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:21:18.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:17 smithi151 ceph-mon[37043]: pgmap v383: 129 pgs: 129 active+clean; 587 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 4.0 KiB/s wr, 19 op/s 2024-06-03T03:21:18.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:17 smithi151 ceph-mon[37043]: from='client.24674 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:21:19.009 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:18 smithi117 ceph-mon[31649]: mgrmap e17: x(active, since 9m), standbys: z, y 2024-06-03T03:21:19.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:18 smithi183 ceph-mon[37326]: mgrmap e17: x(active, since 9m), standbys: z, y 2024-06-03T03:21:19.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:18 smithi151 ceph-mon[37043]: mgrmap e17: x(active, since 9m), standbys: z, y 2024-06-03T03:21:20.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:19 smithi117 ceph-mon[31649]: pgmap v384: 129 pgs: 129 active+clean; 593 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.0 KiB/s wr, 19 op/s 2024-06-03T03:21:20.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:19 smithi183 ceph-mon[37326]: pgmap v384: 129 pgs: 129 active+clean; 593 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.0 KiB/s wr, 19 op/s 2024-06-03T03:21:20.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:19 smithi151 ceph-mon[37043]: pgmap v384: 129 pgs: 129 active+clean; 593 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.0 KiB/s wr, 19 op/s 2024-06-03T03:21:20.828 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:20.825+0000 7f8224200700 1 -- 172.21.15.117:0/2191703475 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f821c1046a0 msgr2=0x7f821c104b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:21:20.828 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:20.825+0000 7f8224200700 1 --2- 172.21.15.117:0/2191703475 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f821c1046a0 0x7f821c104b20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f82080098f0 tx=0x7f8208009c00 comp rx=0 tx=0).stop 2024-06-03T03:21:20.828 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:20.826+0000 7f8224200700 1 -- 172.21.15.117:0/2191703475 shutdown_connections 2024-06-03T03:21:20.828 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:20.826+0000 7f8224200700 1 --2- 172.21.15.117:0/2191703475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f821c105060 0x7f821c111570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:20.828 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:20.826+0000 7f8224200700 1 --2- 172.21.15.117:0/2191703475 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f821c10a6a0 0x7f821c10aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:20.828 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:20.826+0000 7f8224200700 1 --2- 172.21.15.117:0/2191703475 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f821c1046a0 0x7f821c104b20 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:20.828 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:20.826+0000 7f8224200700 1 -- 172.21.15.117:0/2191703475 >> 172.21.15.117:0/2191703475 conn(0x7f821c100190 msgr2=0x7f821c1025b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:21:20.828 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:20.826+0000 7f8224200700 1 -- 172.21.15.117:0/2191703475 shutdown_connections 2024-06-03T03:21:20.828 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:20.826+0000 7f8224200700 1 -- 172.21.15.117:0/2191703475 wait complete. 2024-06-03T03:21:20.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:20.827+0000 7f8224200700 1 Processor -- start 2024-06-03T03:21:20.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:20.827+0000 7f8224200700 1 -- start start 2024-06-03T03:21:20.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:20.828+0000 7f8224200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f821c1046a0 0x7f821c19aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:21:20.830 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:20.828+0000 7f8224200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f821c105060 0x7f821c19b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:21:20.830 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:20.828+0000 7f8224200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f821c10a6a0 0x7f821c19f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:21:20.830 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:20.828+0000 7f8224200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f821c114a80 con 0x7f821c105060 2024-06-03T03:21:20.830 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:20.828+0000 7f8224200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f821c114900 con 0x7f821c1046a0 2024-06-03T03:21:20.830 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:20.828+0000 7f8224200700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f821c114c00 con 0x7f821c10a6a0 2024-06-03T03:21:20.830 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:20.828+0000 7f8220e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f821c105060 0x7f821c19b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:21:20.830 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:20.828+0000 7f8221800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f821c1046a0 0x7f821c19aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:21:20.831 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:20.828+0000 7f8222200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f821c10a6a0 0x7f821c19f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:21:20.831 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:20.828+0000 7f8222200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f821c10a6a0 0x7f821c19f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:56780/0 (socket says 172.21.15.117:56780) 2024-06-03T03:21:20.831 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:20.828+0000 7f8222200700 1 -- 172.21.15.117:0/2349586987 learned_addr learned my addr 172.21.15.117:0/2349586987 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:21:20.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:20.829+0000 7f8222200700 1 -- 172.21.15.117:0/2349586987 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f821c1046a0 msgr2=0x7f821c19aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:21:20.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:20.829+0000 7f8222200700 1 --2- 172.21.15.117:0/2349586987 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f821c1046a0 0x7f821c19aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:20.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:20.829+0000 7f8222200700 1 -- 172.21.15.117:0/2349586987 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f821c105060 msgr2=0x7f821c19b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:21:20.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:20.829+0000 7f8222200700 1 --2- 172.21.15.117:0/2349586987 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f821c105060 0x7f821c19b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:20.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:20.829+0000 7f8222200700 1 -- 172.21.15.117:0/2349586987 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f821c19ff00 con 0x7f821c10a6a0 2024-06-03T03:21:20.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:20.829+0000 7f8221800700 1 --2- 172.21.15.117:0/2349586987 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f821c1046a0 0x7f821c19aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:21:20.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:20.829+0000 7f8220e00700 1 --2- 172.21.15.117:0/2349586987 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f821c105060 0x7f821c19b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:21:20.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:20.829+0000 7f8222200700 1 --2- 172.21.15.117:0/2349586987 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f821c10a6a0 0x7f821c19f7c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f821400ef30 tx=0x7f821400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:21:20.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:20.829+0000 7f821a000700 1 -- 172.21.15.117:0/2349586987 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82140092e0 con 0x7f821c10a6a0 2024-06-03T03:21:20.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:20.829+0000 7f821a000700 1 -- 172.21.15.117:0/2349586987 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8214004bb0 con 0x7f821c10a6a0 2024-06-03T03:21:20.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:20.829+0000 7f8224200700 1 -- 172.21.15.117:0/2349586987 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f821c1a0190 con 0x7f821c10a6a0 2024-06-03T03:21:20.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:20.830+0000 7f821a000700 1 -- 172.21.15.117:0/2349586987 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8214004280 con 0x7f821c10a6a0 2024-06-03T03:21:20.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:20.830+0000 7f8224200700 1 -- 172.21.15.117:0/2349586987 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f821c1a7a70 con 0x7f821c10a6a0 2024-06-03T03:21:20.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:20.831+0000 7f8224200700 1 -- 172.21.15.117:0/2349586987 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f821c069180 con 0x7f821c10a6a0 2024-06-03T03:21:20.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:20.833+0000 7f821a000700 1 -- 172.21.15.117:0/2349586987 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f821401b030 con 0x7f821c10a6a0 2024-06-03T03:21:20.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:20.834+0000 7f821a000700 1 --2- 172.21.15.117:0/2349586987 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f82040a2d30 0x7f82040a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:21:20.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:20.834+0000 7f821a000700 1 -- 172.21.15.117:0/2349586987 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7f82140cf3a0 con 0x7f821c10a6a0 2024-06-03T03:21:20.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:20.835+0000 7f8221800700 1 --2- 172.21.15.117:0/2349586987 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f82040a2d30 0x7f82040a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:21:20.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:20.835+0000 7f8221800700 1 --2- 172.21.15.117:0/2349586987 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f82040a2d30 0x7f82040a51f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8210005fd0 tx=0x7f8210009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:21:20.838 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:20.837+0000 7f821a000700 1 -- 172.21.15.117:0/2349586987 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8214022020 con 0x7f821c10a6a0 2024-06-03T03:21:21.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:21.100+0000 7f8224200700 1 -- 172.21.15.117:0/2349586987 --> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f821c0628d0 con 0x7f82040a2d30 2024-06-03T03:21:21.142 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:21.140+0000 7f821a000700 1 -- 172.21.15.117:0/2349586987 <== mgr.14152 v2:172.21.15.117:6800/1398160887 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f821c0628d0 con 0x7f82040a2d30 2024-06-03T03:21:21.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:21.143+0000 7f8224200700 1 -- 172.21.15.117:0/2349586987 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f82040a2d30 msgr2=0x7f82040a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:21:21.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:21.143+0000 7f8224200700 1 --2- 172.21.15.117:0/2349586987 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f82040a2d30 0x7f82040a51f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8210005fd0 tx=0x7f8210009310 comp rx=0 tx=0).stop 2024-06-03T03:21:21.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:21.143+0000 7f8224200700 1 -- 172.21.15.117:0/2349586987 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f821c10a6a0 msgr2=0x7f821c19f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:21:21.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:21.143+0000 7f8224200700 1 --2- 172.21.15.117:0/2349586987 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f821c10a6a0 0x7f821c19f7c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f821400ef30 tx=0x7f821400c3a0 comp rx=0 tx=0).stop 2024-06-03T03:21:21.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:21.143+0000 7f8224200700 1 -- 172.21.15.117:0/2349586987 shutdown_connections 2024-06-03T03:21:21.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:21.143+0000 7f8224200700 1 --2- 172.21.15.117:0/2349586987 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f82040a2d30 0x7f82040a51f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:21.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:21.143+0000 7f8224200700 1 --2- 172.21.15.117:0/2349586987 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f821c10a6a0 0x7f821c19f7c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:21.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:21.143+0000 7f8224200700 1 --2- 172.21.15.117:0/2349586987 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f821c1046a0 0x7f821c19aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:21.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:21.143+0000 7f8224200700 1 --2- 172.21.15.117:0/2349586987 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f821c105060 0x7f821c19b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:21.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:21.143+0000 7f8224200700 1 -- 172.21.15.117:0/2349586987 >> 172.21.15.117:0/2349586987 conn(0x7f821c100190 msgr2=0x7f821c100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:21:21.149 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:21.143+0000 7f8224200700 1 -- 172.21.15.117:0/2349586987 shutdown_connections 2024-06-03T03:21:21.149 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:21.143+0000 7f8224200700 1 -- 172.21.15.117:0/2349586987 wait complete. 2024-06-03T03:21:22.057 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-06-03T03:21:22.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:21 smithi117 ceph-mon[31649]: pgmap v385: 129 pgs: 129 active+clean; 593 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-06-03T03:21:22.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:21 smithi183 ceph-mon[37326]: pgmap v385: 129 pgs: 129 active+clean; 593 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-06-03T03:21:22.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:21 smithi151 ceph-mon[37043]: pgmap v385: 129 pgs: 129 active+clean; 593 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-06-03T03:21:22.491 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:21:23.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:22 smithi183 ceph-mon[37326]: from='client.24715 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:21:23.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:22 smithi151 ceph-mon[37043]: from='client.24715 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:21:23.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:22 smithi117 ceph-mon[31649]: from='client.24715 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:21:24.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:23 smithi117 ceph-mon[31649]: pgmap v386: 129 pgs: 129 active+clean; 607 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 2.2 KiB/s wr, 13 op/s 2024-06-03T03:21:24.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:23 smithi183 ceph-mon[37326]: pgmap v386: 129 pgs: 129 active+clean; 607 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 2.2 KiB/s wr, 13 op/s 2024-06-03T03:21:24.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:23 smithi151 ceph-mon[37043]: pgmap v386: 129 pgs: 129 active+clean; 607 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 2.2 KiB/s wr, 13 op/s 2024-06-03T03:21:24.894 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:24.891+0000 7fb5f4e00700 1 -- 172.21.15.117:0/2514921978 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb5f0109480 msgr2=0x7fb5f0115990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:21:24.894 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:24.891+0000 7fb5f4e00700 1 --2- 172.21.15.117:0/2514921978 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb5f0109480 0x7fb5f0115990 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb5e800b0a0 tx=0x7fb5e800b3b0 comp rx=0 tx=0).stop 2024-06-03T03:21:24.894 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:24.891+0000 7fb5f4e00700 1 -- 172.21.15.117:0/2514921978 shutdown_connections 2024-06-03T03:21:24.894 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:24.892+0000 7fb5f4e00700 1 --2- 172.21.15.117:0/2514921978 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb5f0109480 0x7fb5f0115990 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:24.894 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:24.892+0000 7fb5f4e00700 1 --2- 172.21.15.117:0/2514921978 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5f010eac0 0x7fb5f010eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:24.894 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:24.892+0000 7fb5f4e00700 1 --2- 172.21.15.117:0/2514921978 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb5f0108ac0 0x7fb5f0108f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:24.894 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:24.892+0000 7fb5f4e00700 1 -- 172.21.15.117:0/2514921978 >> 172.21.15.117:0/2514921978 conn(0x7fb5f0075c60 msgr2=0x7fb5f01072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:21:24.894 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:24.892+0000 7fb5f4e00700 1 -- 172.21.15.117:0/2514921978 shutdown_connections 2024-06-03T03:21:24.895 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:24.892+0000 7fb5f4e00700 1 -- 172.21.15.117:0/2514921978 wait complete. 2024-06-03T03:21:24.896 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:24.893+0000 7fb5f4e00700 1 Processor -- start 2024-06-03T03:21:24.896 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:24.893+0000 7fb5f4e00700 1 -- start start 2024-06-03T03:21:24.896 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:24.893+0000 7fb5f4e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb5f0108ac0 0x7fb5f019f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:21:24.896 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:24.893+0000 7fb5f4e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb5f0109480 0x7fb5f019f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:21:24.896 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:24.893+0000 7fb5f4e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5f010eac0 0x7fb5f01a3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:21:24.896 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:24.893+0000 7fb5f4e00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f0118e60 con 0x7fb5f010eac0 2024-06-03T03:21:24.896 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:24.893+0000 7fb5f4e00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f0118ce0 con 0x7fb5f0109480 2024-06-03T03:21:24.896 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:24.893+0000 7fb5f4e00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f0118fe0 con 0x7fb5f0108ac0 2024-06-03T03:21:24.896 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:24.894+0000 7fb5ede00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb5f0108ac0 0x7fb5f019f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:21:24.896 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:24.894+0000 7fb5ede00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb5f0108ac0 0x7fb5f019f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:56788/0 (socket says 172.21.15.117:56788) 2024-06-03T03:21:24.897 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:24.894+0000 7fb5ede00700 1 -- 172.21.15.117:0/2036734344 learned_addr learned my addr 172.21.15.117:0/2036734344 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:21:24.897 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:24.894+0000 7fb5ed400700 1 --2- 172.21.15.117:0/2036734344 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb5f0109480 0x7fb5f019f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:21:24.897 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:24.894+0000 7fb5ee800700 1 --2- 172.21.15.117:0/2036734344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5f010eac0 0x7fb5f01a3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:21:24.897 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:24.894+0000 7fb5ed400700 1 -- 172.21.15.117:0/2036734344 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb5f0108ac0 msgr2=0x7fb5f019f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:21:24.897 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:24.894+0000 7fb5ed400700 1 --2- 172.21.15.117:0/2036734344 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb5f0108ac0 0x7fb5f019f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:24.897 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:24.894+0000 7fb5ed400700 1 -- 172.21.15.117:0/2036734344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5f010eac0 msgr2=0x7fb5f01a3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:21:24.897 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:24.894+0000 7fb5ed400700 1 --2- 172.21.15.117:0/2036734344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5f010eac0 0x7fb5f01a3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:24.897 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:24.894+0000 7fb5ed400700 1 -- 172.21.15.117:0/2036734344 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5f01a4240 con 0x7fb5f0109480 2024-06-03T03:21:24.897 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:24.894+0000 7fb5ee800700 1 --2- 172.21.15.117:0/2036734344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5f010eac0 0x7fb5f01a3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:21:24.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:24.895+0000 7fb5ed400700 1 --2- 172.21.15.117:0/2036734344 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb5f0109480 0x7fb5f019f770 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc00e9f0 tx=0x7fb5dc00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:21:24.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:24.895+0000 7fb5e6a00700 1 -- 172.21.15.117:0/2036734344 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc004220 con 0x7fb5f0109480 2024-06-03T03:21:24.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:24.895+0000 7fb5f4e00700 1 -- 172.21.15.117:0/2036734344 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5f01a4530 con 0x7fb5f0109480 2024-06-03T03:21:24.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:24.895+0000 7fb5e6a00700 1 -- 172.21.15.117:0/2036734344 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5dc004d10 con 0x7fb5f0109480 2024-06-03T03:21:24.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:24.895+0000 7fb5e6a00700 1 -- 172.21.15.117:0/2036734344 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc005020 con 0x7fb5f0109480 2024-06-03T03:21:24.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:24.895+0000 7fb5f4e00700 1 -- 172.21.15.117:0/2036734344 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5f01abdb0 con 0x7fb5f0109480 2024-06-03T03:21:24.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:24.896+0000 7fb5f4e00700 1 -- 172.21.15.117:0/2036734344 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5f0069180 con 0x7fb5f0109480 2024-06-03T03:21:24.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:24.899+0000 7fb5e6a00700 1 -- 172.21.15.117:0/2036734344 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fb5dc004750 con 0x7fb5f0109480 2024-06-03T03:21:24.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:24.900+0000 7fb5e6a00700 1 --2- 172.21.15.117:0/2036734344 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fb5d40a2cb0 0x7fb5d40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:21:24.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:24.900+0000 7fb5e6a00700 1 -- 172.21.15.117:0/2036734344 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7fb5dc0cf3e0 con 0x7fb5f0109480 2024-06-03T03:21:24.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:24.900+0000 7fb5ede00700 1 --2- 172.21.15.117:0/2036734344 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fb5d40a2cb0 0x7fb5d40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:21:24.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:24.901+0000 7fb5ede00700 1 --2- 172.21.15.117:0/2036734344 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fb5d40a2cb0 0x7fb5d40a5170 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0009680 tx=0x7fb5e0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:21:24.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:24.905+0000 7fb5e6a00700 1 -- 172.21.15.117:0/2036734344 <== mon.2 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb5dc09be80 con 0x7fb5f0109480 2024-06-03T03:21:25.182 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:25.180+0000 7fb5f4e00700 1 -- 172.21.15.117:0/2036734344 --> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7fb5f0076850 con 0x7fb5d40a2cb0 2024-06-03T03:21:25.182 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:24 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]: dispatch 2024-06-03T03:21:25.183 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:24 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 10]}]: dispatch 2024-06-03T03:21:25.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:25.189+0000 7fb5e6a00700 1 -- 172.21.15.117:0/2036734344 <== mgr.14152 v2:172.21.15.117:6800/1398160887 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fb5f0076850 con 0x7fb5d40a2cb0 2024-06-03T03:21:25.191 INFO:teuthology.orchestra.run.smithi117.stdout:/volumes/qa/sv_0/b15fc72f-253f-4645-b8c2-044ca185d9a0 2024-06-03T03:21:25.194 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:25.192+0000 7fb5f4e00700 1 -- 172.21.15.117:0/2036734344 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fb5d40a2cb0 msgr2=0x7fb5d40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:21:25.194 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:25.192+0000 7fb5f4e00700 1 --2- 172.21.15.117:0/2036734344 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fb5d40a2cb0 0x7fb5d40a5170 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0009680 tx=0x7fb5e0009260 comp rx=0 tx=0).stop 2024-06-03T03:21:25.194 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:25.192+0000 7fb5f4e00700 1 -- 172.21.15.117:0/2036734344 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb5f0109480 msgr2=0x7fb5f019f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:21:25.194 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:25.192+0000 7fb5f4e00700 1 --2- 172.21.15.117:0/2036734344 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb5f0109480 0x7fb5f019f770 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc00e9f0 tx=0x7fb5dc00ed00 comp rx=0 tx=0).stop 2024-06-03T03:21:25.195 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:25.192+0000 7fb5f4e00700 1 -- 172.21.15.117:0/2036734344 shutdown_connections 2024-06-03T03:21:25.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:25.192+0000 7fb5f4e00700 1 --2- 172.21.15.117:0/2036734344 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fb5d40a2cb0 0x7fb5d40a5170 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:25.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:25.192+0000 7fb5f4e00700 1 --2- 172.21.15.117:0/2036734344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5f010eac0 0x7fb5f01a3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:25.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:25.192+0000 7fb5f4e00700 1 --2- 172.21.15.117:0/2036734344 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb5f0108ac0 0x7fb5f019f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:25.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:25.192+0000 7fb5f4e00700 1 --2- 172.21.15.117:0/2036734344 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb5f0109480 0x7fb5f019f770 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:25.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:25.192+0000 7fb5f4e00700 1 -- 172.21.15.117:0/2036734344 >> 172.21.15.117:0/2036734344 conn(0x7fb5f0075c60 msgr2=0x7fb5f0076140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:21:25.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:25.192+0000 7fb5f4e00700 1 -- 172.21.15.117:0/2036734344 shutdown_connections 2024-06-03T03:21:25.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:25.192+0000 7fb5f4e00700 1 -- 172.21.15.117:0/2036734344 wait complete. 2024-06-03T03:21:25.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:24 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]: dispatch 2024-06-03T03:21:25.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:24 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 10]}]: dispatch 2024-06-03T03:21:25.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:24 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]: dispatch 2024-06-03T03:21:25.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:24 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 10]}]: dispatch 2024-06-03T03:21:26.125 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-06-03T03:21:26.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:25 smithi151 ceph-mon[37043]: pgmap v387: 129 pgs: 129 active+clean; 607 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-06-03T03:21:26.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:25 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]': finished 2024-06-03T03:21:26.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:25 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 10]}]': finished 2024-06-03T03:21:26.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:25 smithi151 ceph-mon[37043]: osdmap e98: 12 total, 12 up, 12 in 2024-06-03T03:21:26.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:25 smithi117 ceph-mon[31649]: pgmap v387: 129 pgs: 129 active+clean; 607 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-06-03T03:21:26.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:25 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]': finished 2024-06-03T03:21:26.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:25 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 10]}]': finished 2024-06-03T03:21:26.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:25 smithi117 ceph-mon[31649]: osdmap e98: 12 total, 12 up, 12 in 2024-06-03T03:21:26.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:25 smithi183 ceph-mon[37326]: pgmap v387: 129 pgs: 129 active+clean; 607 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-06-03T03:21:26.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:25 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]': finished 2024-06-03T03:21:26.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:25 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 10]}]': finished 2024-06-03T03:21:26.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:25 smithi183 ceph-mon[37326]: osdmap e98: 12 total, 12 up, 12 in 2024-06-03T03:21:26.568 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:21:27.260 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:26 smithi117 ceph-mon[31649]: from='client.24692 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:21:27.260 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:26 smithi117 ceph-mon[31649]: osdmap e99: 12 total, 12 up, 12 in 2024-06-03T03:21:27.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:26 smithi151 ceph-mon[37043]: from='client.24692 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:21:27.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:26 smithi151 ceph-mon[37043]: osdmap e99: 12 total, 12 up, 12 in 2024-06-03T03:21:27.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:26 smithi183 ceph-mon[37326]: from='client.24692 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:21:27.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:26 smithi183 ceph-mon[37326]: osdmap e99: 12 total, 12 up, 12 in 2024-06-03T03:21:28.308 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:27 smithi151 ceph-mon[37043]: pgmap v390: 129 pgs: 129 active+clean; 607 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.4 KiB/s wr, 8 op/s 2024-06-03T03:21:28.341 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:27 smithi117 ceph-mon[31649]: pgmap v390: 129 pgs: 129 active+clean; 607 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.4 KiB/s wr, 8 op/s 2024-06-03T03:21:28.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:27 smithi183 ceph-mon[37326]: pgmap v390: 129 pgs: 129 active+clean; 607 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.4 KiB/s wr, 8 op/s 2024-06-03T03:21:28.966 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:28.965+0000 7f3d2cc00700 1 -- 172.21.15.117:0/3355824255 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d28109480 msgr2=0x7f3d28115990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:21:28.966 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:28.965+0000 7f3d2cc00700 1 --2- 172.21.15.117:0/3355824255 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d28109480 0x7f3d28115990 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3d2000b0a0 tx=0x7f3d2000b3b0 comp rx=0 tx=0).stop 2024-06-03T03:21:28.966 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:28.965+0000 7f3d2cc00700 1 -- 172.21.15.117:0/3355824255 shutdown_connections 2024-06-03T03:21:28.966 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:28.965+0000 7f3d2cc00700 1 --2- 172.21.15.117:0/3355824255 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d28109480 0x7f3d28115990 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:28.966 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:28.965+0000 7f3d2cc00700 1 --2- 172.21.15.117:0/3355824255 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d2810eac0 0x7f3d2810eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:28.966 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:28.965+0000 7f3d2cc00700 1 --2- 172.21.15.117:0/3355824255 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3d28108ac0 0x7f3d28108f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:28.966 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:28.965+0000 7f3d2cc00700 1 -- 172.21.15.117:0/3355824255 >> 172.21.15.117:0/3355824255 conn(0x7f3d28075c60 msgr2=0x7f3d281072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:21:28.967 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:28.965+0000 7f3d2cc00700 1 -- 172.21.15.117:0/3355824255 shutdown_connections 2024-06-03T03:21:28.967 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:28.965+0000 7f3d2cc00700 1 -- 172.21.15.117:0/3355824255 wait complete. 2024-06-03T03:21:28.968 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:28.966+0000 7f3d2cc00700 1 Processor -- start 2024-06-03T03:21:28.968 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:28.966+0000 7f3d2cc00700 1 -- start start 2024-06-03T03:21:28.968 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:28.966+0000 7f3d2cc00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3d28108ac0 0x7f3d2819f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:21:28.968 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:28.966+0000 7f3d2cc00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d28109480 0x7f3d2819f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:21:28.968 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:28.966+0000 7f3d2cc00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d2810eac0 0x7f3d281a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:21:28.968 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:28.966+0000 7f3d2cc00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3d28118e90 con 0x7f3d28109480 2024-06-03T03:21:28.968 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:28.966+0000 7f3d2cc00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3d28118d10 con 0x7f3d28108ac0 2024-06-03T03:21:28.968 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:28.966+0000 7f3d2cc00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3d28119010 con 0x7f3d2810eac0 2024-06-03T03:21:28.968 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:28.967+0000 7f3d25400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d28109480 0x7f3d2819f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:21:28.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:28.967+0000 7f3d25400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d28109480 0x7f3d2819f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:45732/0 (socket says 172.21.15.117:45732) 2024-06-03T03:21:28.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:28.967+0000 7f3d25400700 1 -- 172.21.15.117:0/3967970491 learned_addr learned my addr 172.21.15.117:0/3967970491 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:21:28.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:28.967+0000 7f3d25e00700 1 --2- 172.21.15.117:0/3967970491 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3d28108ac0 0x7f3d2819f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:21:28.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:28.967+0000 7f3d26800700 1 --2- 172.21.15.117:0/3967970491 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d2810eac0 0x7f3d281a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:21:28.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:28.967+0000 7f3d25400700 1 -- 172.21.15.117:0/3967970491 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d2810eac0 msgr2=0x7f3d281a3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:21:28.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:28.967+0000 7f3d25400700 1 --2- 172.21.15.117:0/3967970491 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d2810eac0 0x7f3d281a3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:28.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:28.967+0000 7f3d25400700 1 -- 172.21.15.117:0/3967970491 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3d28108ac0 msgr2=0x7f3d2819f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:21:28.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:28.967+0000 7f3d25400700 1 --2- 172.21.15.117:0/3967970491 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3d28108ac0 0x7f3d2819f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:28.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:28.967+0000 7f3d25400700 1 -- 172.21.15.117:0/3967970491 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d281a4230 con 0x7f3d28109480 2024-06-03T03:21:28.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:28.967+0000 7f3d26800700 1 --2- 172.21.15.117:0/3967970491 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d2810eac0 0x7f3d281a3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T03:21:28.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:28.967+0000 7f3d25e00700 1 --2- 172.21.15.117:0/3967970491 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3d28108ac0 0x7f3d2819f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:21:28.970 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:28.968+0000 7f3d25400700 1 --2- 172.21.15.117:0/3967970491 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d28109480 0x7f3d2819f760 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3d1400e9a0 tx=0x7f3d1400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:21:28.970 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:28.968+0000 7f3d1ea00700 1 -- 172.21.15.117:0/3967970491 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d1400cbd0 con 0x7f3d28109480 2024-06-03T03:21:28.970 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:28.968+0000 7f3d2cc00700 1 -- 172.21.15.117:0/3967970491 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d281a4520 con 0x7f3d28109480 2024-06-03T03:21:28.970 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:28.968+0000 7f3d1ea00700 1 -- 172.21.15.117:0/3967970491 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d1400cd70 con 0x7f3d28109480 2024-06-03T03:21:28.970 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:28.968+0000 7f3d2cc00700 1 -- 172.21.15.117:0/3967970491 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d281abda0 con 0x7f3d28109480 2024-06-03T03:21:28.970 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:28.968+0000 7f3d1ea00700 1 -- 172.21.15.117:0/3967970491 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d140186f0 con 0x7f3d28109480 2024-06-03T03:21:28.970 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:28.969+0000 7f3d2cc00700 1 -- 172.21.15.117:0/3967970491 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d28069180 con 0x7f3d28109480 2024-06-03T03:21:28.973 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:28.972+0000 7f3d1ea00700 1 -- 172.21.15.117:0/3967970491 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f3d14010430 con 0x7f3d28109480 2024-06-03T03:21:28.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:28.973+0000 7f3d1ea00700 1 --2- 172.21.15.117:0/3967970491 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f3d0c0a2cb0 0x7f3d0c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:21:28.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:28.973+0000 7f3d1ea00700 1 -- 172.21.15.117:0/3967970491 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f3d140cf850 con 0x7f3d28109480 2024-06-03T03:21:28.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:28.974+0000 7f3d25e00700 1 --2- 172.21.15.117:0/3967970491 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f3d0c0a2cb0 0x7f3d0c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:21:28.975 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:28.974+0000 7f3d25e00700 1 --2- 172.21.15.117:0/3967970491 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f3d0c0a2cb0 0x7f3d0c0a5170 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3d18009fd0 tx=0x7f3d18009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:21:28.979 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:28.978+0000 7f3d1ea00700 1 -- 172.21.15.117:0/3967970491 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3d1409c260 con 0x7f3d28109480 2024-06-03T03:21:29.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:29 smithi117 ceph-mon[31649]: pgmap v391: 129 pgs: 129 active+clean; 613 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 3.2 KiB/s wr, 13 op/s 2024-06-03T03:21:29.244 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:29.243+0000 7f3d2cc00700 1 -- 172.21.15.117:0/3967970491 --> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f3d280628d0 con 0x7f3d0c0a2cb0 2024-06-03T03:21:29.289 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:29.288+0000 7f3d1ea00700 1 -- 172.21.15.117:0/3967970491 <== mgr.14152 v2:172.21.15.117:6800/1398160887 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3d280628d0 con 0x7f3d0c0a2cb0 2024-06-03T03:21:29.293 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:29.291+0000 7f3d2cc00700 1 -- 172.21.15.117:0/3967970491 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f3d0c0a2cb0 msgr2=0x7f3d0c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:21:29.293 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:29.291+0000 7f3d2cc00700 1 --2- 172.21.15.117:0/3967970491 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f3d0c0a2cb0 0x7f3d0c0a5170 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3d18009fd0 tx=0x7f3d18009260 comp rx=0 tx=0).stop 2024-06-03T03:21:29.293 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:29.291+0000 7f3d2cc00700 1 -- 172.21.15.117:0/3967970491 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d28109480 msgr2=0x7f3d2819f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:21:29.293 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:29.291+0000 7f3d2cc00700 1 --2- 172.21.15.117:0/3967970491 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d28109480 0x7f3d2819f760 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3d1400e9a0 tx=0x7f3d1400ecb0 comp rx=0 tx=0).stop 2024-06-03T03:21:29.293 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:29.291+0000 7f3d2cc00700 1 -- 172.21.15.117:0/3967970491 shutdown_connections 2024-06-03T03:21:29.293 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:29.291+0000 7f3d2cc00700 1 --2- 172.21.15.117:0/3967970491 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f3d0c0a2cb0 0x7f3d0c0a5170 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:29.293 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:29.291+0000 7f3d2cc00700 1 --2- 172.21.15.117:0/3967970491 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d2810eac0 0x7f3d281a3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:29.293 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:29.291+0000 7f3d2cc00700 1 --2- 172.21.15.117:0/3967970491 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3d28108ac0 0x7f3d2819f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:29.293 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:29.291+0000 7f3d2cc00700 1 --2- 172.21.15.117:0/3967970491 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d28109480 0x7f3d2819f760 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:29.293 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:29.292+0000 7f3d2cc00700 1 -- 172.21.15.117:0/3967970491 >> 172.21.15.117:0/3967970491 conn(0x7f3d28075c60 msgr2=0x7f3d280761e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:21:29.293 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:29.292+0000 7f3d2cc00700 1 -- 172.21.15.117:0/3967970491 shutdown_connections 2024-06-03T03:21:29.293 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:29.292+0000 7f3d2cc00700 1 -- 172.21.15.117:0/3967970491 wait complete. 2024-06-03T03:21:29.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:29 smithi151 ceph-mon[37043]: pgmap v391: 129 pgs: 129 active+clean; 613 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 3.2 KiB/s wr, 13 op/s 2024-06-03T03:21:29.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:29 smithi183 ceph-mon[37326]: pgmap v391: 129 pgs: 129 active+clean; 613 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 3.2 KiB/s wr, 13 op/s 2024-06-03T03:21:30.207 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-06-03T03:21:30.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:30 smithi151 ceph-mon[37043]: from='client.14586 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:21:30.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:30 smithi117 ceph-mon[31649]: from='client.14586 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:21:30.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:30 smithi183 ceph-mon[37326]: from='client.14586 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:21:30.664 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:21:31.087 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:31 smithi117 ceph-mon[31649]: pgmap v392: 129 pgs: 129 active+clean; 613 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-06-03T03:21:31.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:31 smithi151 ceph-mon[37043]: pgmap v392: 129 pgs: 129 active+clean; 613 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-06-03T03:21:31.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:31 smithi183 ceph-mon[37326]: pgmap v392: 129 pgs: 129 active+clean; 613 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-06-03T03:21:32.967 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:32.964+0000 7f2f06200700 1 -- 172.21.15.117:0/1572744924 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f00105070 msgr2=0x7f2f00111580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:21:32.967 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:32.964+0000 7f2f06200700 1 --2- 172.21.15.117:0/1572744924 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f00105070 0x7f2f00111580 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2ef800b0a0 tx=0x7f2ef800b3b0 comp rx=0 tx=0).stop 2024-06-03T03:21:32.967 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:32.965+0000 7f2f06200700 1 -- 172.21.15.117:0/1572744924 shutdown_connections 2024-06-03T03:21:32.967 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:32.965+0000 7f2f06200700 1 --2- 172.21.15.117:0/1572744924 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f00105070 0x7f2f00111580 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:32.967 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:32.965+0000 7f2f06200700 1 --2- 172.21.15.117:0/1572744924 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2f0010a6b0 0x7f2f0010aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:32.967 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:32.965+0000 7f2f06200700 1 --2- 172.21.15.117:0/1572744924 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f001046b0 0x7f2f00104b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:32.967 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:32.965+0000 7f2f06200700 1 -- 172.21.15.117:0/1572744924 >> 172.21.15.117:0/1572744924 conn(0x7f2f001001a0 msgr2=0x7f2f001025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:21:32.967 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:32.965+0000 7f2f06200700 1 -- 172.21.15.117:0/1572744924 shutdown_connections 2024-06-03T03:21:32.967 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:32.965+0000 7f2f06200700 1 -- 172.21.15.117:0/1572744924 wait complete. 2024-06-03T03:21:32.967 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:32.966+0000 7f2f06200700 1 Processor -- start 2024-06-03T03:21:32.968 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:32.966+0000 7f2f06200700 1 -- start start 2024-06-03T03:21:32.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:32.966+0000 7f2f06200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f001046b0 0x7f2f0019ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:21:32.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:32.966+0000 7f2f06200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2f00105070 0x7f2f0019b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:21:32.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:32.966+0000 7f2f06200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f0010a6b0 0x7f2f0019f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:21:32.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:32.966+0000 7f2f06200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2f00114a50 con 0x7f2f001046b0 2024-06-03T03:21:32.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:32.966+0000 7f2f06200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2f001148d0 con 0x7f2f00105070 2024-06-03T03:21:32.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:32.966+0000 7f2f06200700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2f00114bd0 con 0x7f2f0010a6b0 2024-06-03T03:21:32.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:32.967+0000 7f2eff400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f001046b0 0x7f2f0019ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:21:32.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:32.967+0000 7f2eff400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f001046b0 0x7f2f0019ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:45742/0 (socket says 172.21.15.117:45742) 2024-06-03T03:21:32.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:32.967+0000 7f2eff400700 1 -- 172.21.15.117:0/1638115736 learned_addr learned my addr 172.21.15.117:0/1638115736 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:21:32.970 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:32.967+0000 7f2ef7e00700 1 --2- 172.21.15.117:0/1638115736 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2f00105070 0x7f2f0019b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:21:32.970 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:32.967+0000 7f2effe00700 1 --2- 172.21.15.117:0/1638115736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f0010a6b0 0x7f2f0019f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:21:32.970 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:32.967+0000 7f2eff400700 1 -- 172.21.15.117:0/1638115736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f0010a6b0 msgr2=0x7f2f0019f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:21:32.970 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:32.967+0000 7f2eff400700 1 --2- 172.21.15.117:0/1638115736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f0010a6b0 0x7f2f0019f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:32.970 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:32.967+0000 7f2eff400700 1 -- 172.21.15.117:0/1638115736 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2f00105070 msgr2=0x7f2f0019b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:21:32.970 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:32.967+0000 7f2eff400700 1 --2- 172.21.15.117:0/1638115736 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2f00105070 0x7f2f0019b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:32.970 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:32.967+0000 7f2eff400700 1 -- 172.21.15.117:0/1638115736 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f0019fe30 con 0x7f2f001046b0 2024-06-03T03:21:32.970 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:32.967+0000 7f2eff400700 1 --2- 172.21.15.117:0/1638115736 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f001046b0 0x7f2f0019ae20 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2ef000e9a0 tx=0x7f2ef000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:21:32.970 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:32.968+0000 7f2efcc00700 1 -- 172.21.15.117:0/1638115736 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef000ca10 con 0x7f2f001046b0 2024-06-03T03:21:32.970 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:32.968+0000 7f2efcc00700 1 -- 172.21.15.117:0/1638115736 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ef000cbb0 con 0x7f2f001046b0 2024-06-03T03:21:32.970 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:32.968+0000 7f2f06200700 1 -- 172.21.15.117:0/1638115736 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f001a0120 con 0x7f2f001046b0 2024-06-03T03:21:32.970 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:32.968+0000 7f2efcc00700 1 -- 172.21.15.117:0/1638115736 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef0018680 con 0x7f2f001046b0 2024-06-03T03:21:32.970 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:32.968+0000 7f2f06200700 1 -- 172.21.15.117:0/1638115736 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f001a79a0 con 0x7f2f001046b0 2024-06-03T03:21:32.971 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:32.969+0000 7f2f06200700 1 -- 172.21.15.117:0/1638115736 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f00069180 con 0x7f2f001046b0 2024-06-03T03:21:32.972 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:32.971+0000 7f2efcc00700 1 -- 172.21.15.117:0/1638115736 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f2ef0010430 con 0x7f2f001046b0 2024-06-03T03:21:32.973 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:32.972+0000 7f2efcc00700 1 --2- 172.21.15.117:0/1638115736 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f2ee40a2d30 0x7f2ee40a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:21:32.973 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:32.973+0000 7f2efcc00700 1 -- 172.21.15.117:0/1638115736 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f2ef00cf6f0 con 0x7f2f001046b0 2024-06-03T03:21:32.973 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:32.973+0000 7f2ef7e00700 1 --2- 172.21.15.117:0/1638115736 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f2ee40a2d30 0x7f2ee40a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:21:32.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:32.973+0000 7f2ef7e00700 1 --2- 172.21.15.117:0/1638115736 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f2ee40a2d30 0x7f2ee40a51f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2f00109c70 tx=0x7f2ee8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:21:32.978 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:32.977+0000 7f2efcc00700 1 -- 172.21.15.117:0/1638115736 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2ef009c100 con 0x7f2f001046b0 2024-06-03T03:21:33.237 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:33.236+0000 7f2f06200700 1 -- 172.21.15.117:0/1638115736 --> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f2f00100d90 con 0x7f2ee40a2d30 2024-06-03T03:21:33.248 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:33.247+0000 7f2efcc00700 1 -- 172.21.15.117:0/1638115736 <== mgr.14152 v2:172.21.15.117:6800/1398160887 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f2f00100d90 con 0x7f2ee40a2d30 2024-06-03T03:21:33.248 INFO:teuthology.orchestra.run.smithi117.stdout:/volumes/qa/sv_1/a202463a-d267-4cb0-ba31-bccad7941779 2024-06-03T03:21:33.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:33.249+0000 7f2f06200700 1 -- 172.21.15.117:0/1638115736 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f2ee40a2d30 msgr2=0x7f2ee40a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:21:33.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:33.249+0000 7f2f06200700 1 --2- 172.21.15.117:0/1638115736 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f2ee40a2d30 0x7f2ee40a51f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2f00109c70 tx=0x7f2ee8009260 comp rx=0 tx=0).stop 2024-06-03T03:21:33.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:33.249+0000 7f2f06200700 1 -- 172.21.15.117:0/1638115736 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f001046b0 msgr2=0x7f2f0019ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:21:33.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:33.249+0000 7f2f06200700 1 --2- 172.21.15.117:0/1638115736 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f001046b0 0x7f2f0019ae20 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2ef000e9a0 tx=0x7f2ef000ecb0 comp rx=0 tx=0).stop 2024-06-03T03:21:33.251 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:33.250+0000 7f2f06200700 1 -- 172.21.15.117:0/1638115736 shutdown_connections 2024-06-03T03:21:33.251 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:33.250+0000 7f2f06200700 1 --2- 172.21.15.117:0/1638115736 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f2ee40a2d30 0x7f2ee40a51f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:33.251 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:33.250+0000 7f2f06200700 1 --2- 172.21.15.117:0/1638115736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f0010a6b0 0x7f2f0019f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:33.251 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:33.250+0000 7f2f06200700 1 --2- 172.21.15.117:0/1638115736 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f001046b0 0x7f2f0019ae20 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:33.251 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:33.250+0000 7f2f06200700 1 --2- 172.21.15.117:0/1638115736 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2f00105070 0x7f2f0019b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:33.251 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:33.250+0000 7f2f06200700 1 -- 172.21.15.117:0/1638115736 >> 172.21.15.117:0/1638115736 conn(0x7f2f001001a0 msgr2=0x7f2f00100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:21:33.251 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:33.250+0000 7f2f06200700 1 -- 172.21.15.117:0/1638115736 shutdown_connections 2024-06-03T03:21:33.252 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:33.250+0000 7f2f06200700 1 -- 172.21.15.117:0/1638115736 wait complete. 2024-06-03T03:21:34.134 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:33 smithi117 ceph-mon[31649]: pgmap v393: 129 pgs: 129 active+clean; 632 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.0 KiB/s wr, 14 op/s 2024-06-03T03:21:34.201 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-03T03:21:34.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:33 smithi183 ceph-mon[37326]: pgmap v393: 129 pgs: 129 active+clean; 632 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.0 KiB/s wr, 14 op/s 2024-06-03T03:21:34.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:33 smithi151 ceph-mon[37043]: pgmap v393: 129 pgs: 129 active+clean; 632 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.0 KiB/s wr, 14 op/s 2024-06-03T03:21:34.645 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:21:35.049 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:34 smithi117 ceph-mon[31649]: from='client.14592 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:21:35.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:34 smithi183 ceph-mon[37326]: from='client.14592 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:21:35.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:34 smithi151 ceph-mon[37043]: from='client.14592 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:21:36.148 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:35 smithi117 ceph-mon[31649]: pgmap v394: 129 pgs: 129 active+clean; 632 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 3.1 KiB/s wr, 12 op/s 2024-06-03T03:21:36.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:35 smithi183 ceph-mon[37326]: pgmap v394: 129 pgs: 129 active+clean; 632 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 3.1 KiB/s wr, 12 op/s 2024-06-03T03:21:36.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:35 smithi151 ceph-mon[37043]: pgmap v394: 129 pgs: 129 active+clean; 632 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 3.1 KiB/s wr, 12 op/s 2024-06-03T03:21:37.032 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:37.031+0000 7fc720600700 1 -- 172.21.15.117:0/3364078957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc718074fe0 msgr2=0x7fc718113120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:21:37.032 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:37.031+0000 7fc720600700 1 --2- 172.21.15.117:0/3364078957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc718074fe0 0x7fc718113120 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc704009990 tx=0x7fc704009ca0 comp rx=0 tx=0).stop 2024-06-03T03:21:37.032 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:37.031+0000 7fc720600700 1 -- 172.21.15.117:0/3364078957 shutdown_connections 2024-06-03T03:21:37.032 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:37.031+0000 7fc720600700 1 --2- 172.21.15.117:0/3364078957 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc718113660 0x7fc718115a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:37.032 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:37.031+0000 7fc720600700 1 --2- 172.21.15.117:0/3364078957 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7180746c0 0x7fc718074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:37.033 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:37.031+0000 7fc720600700 1 --2- 172.21.15.117:0/3364078957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc718074fe0 0x7fc718113120 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:37.033 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:37.031+0000 7fc720600700 1 -- 172.21.15.117:0/3364078957 >> 172.21.15.117:0/3364078957 conn(0x7fc7180795f0 msgr2=0x7fc718079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:21:37.033 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:37.031+0000 7fc720600700 1 -- 172.21.15.117:0/3364078957 shutdown_connections 2024-06-03T03:21:37.033 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:37.031+0000 7fc720600700 1 -- 172.21.15.117:0/3364078957 wait complete. 2024-06-03T03:21:37.034 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:37.032+0000 7fc720600700 1 Processor -- start 2024-06-03T03:21:37.034 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:37.032+0000 7fc720600700 1 -- start start 2024-06-03T03:21:37.034 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:37.032+0000 7fc720600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7180746c0 0x7fc71819f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:21:37.034 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:37.032+0000 7fc720600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc718074fe0 0x7fc71819f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:21:37.034 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:37.032+0000 7fc720600700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc718113660 0x7fc7181a3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:21:37.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:37.032+0000 7fc720600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc718118e10 con 0x7fc718074fe0 2024-06-03T03:21:37.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:37.032+0000 7fc720600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc718118c90 con 0x7fc7180746c0 2024-06-03T03:21:37.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:37.032+0000 7fc720600700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc718118f90 con 0x7fc718113660 2024-06-03T03:21:37.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:37.033+0000 7fc71d200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc718074fe0 0x7fc71819f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:21:37.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:37.033+0000 7fc71d200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc718074fe0 0x7fc71819f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37506/0 (socket says 172.21.15.117:37506) 2024-06-03T03:21:37.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:37.033+0000 7fc71d200700 1 -- 172.21.15.117:0/3993109472 learned_addr learned my addr 172.21.15.117:0/3993109472 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:21:37.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:37.033+0000 7fc71dc00700 1 --2- 172.21.15.117:0/3993109472 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7180746c0 0x7fc71819f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:21:37.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:37.033+0000 7fc71e600700 1 --2- 172.21.15.117:0/3993109472 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc718113660 0x7fc7181a3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:21:37.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:37.033+0000 7fc71dc00700 1 -- 172.21.15.117:0/3993109472 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc718113660 msgr2=0x7fc7181a3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:21:37.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:37.033+0000 7fc71dc00700 1 --2- 172.21.15.117:0/3993109472 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc718113660 0x7fc7181a3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:37.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:37.033+0000 7fc71dc00700 1 -- 172.21.15.117:0/3993109472 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc718074fe0 msgr2=0x7fc71819f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:21:37.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:37.033+0000 7fc71dc00700 1 --2- 172.21.15.117:0/3993109472 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc718074fe0 0x7fc71819f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:37.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:37.033+0000 7fc71dc00700 1 -- 172.21.15.117:0/3993109472 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7181a4290 con 0x7fc7180746c0 2024-06-03T03:21:37.036 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:37.033+0000 7fc71e600700 1 --2- 172.21.15.117:0/3993109472 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc718113660 0x7fc7181a3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T03:21:37.036 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:37.033+0000 7fc71d200700 1 --2- 172.21.15.117:0/3993109472 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc718074fe0 0x7fc71819f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:21:37.036 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:37.033+0000 7fc71dc00700 1 --2- 172.21.15.117:0/3993109472 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7180746c0 0x7fc71819f280 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc70c00e9f0 tx=0x7fc70c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:21:37.036 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:37.034+0000 7fc716000700 1 -- 172.21.15.117:0/3993109472 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70c00cbd0 con 0x7fc7180746c0 2024-06-03T03:21:37.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:37.034+0000 7fc716000700 1 -- 172.21.15.117:0/3993109472 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc70c00cd70 con 0x7fc7180746c0 2024-06-03T03:21:37.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:37.034+0000 7fc720600700 1 -- 172.21.15.117:0/3993109472 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7181a4580 con 0x7fc7180746c0 2024-06-03T03:21:37.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:37.034+0000 7fc716000700 1 -- 172.21.15.117:0/3993109472 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70c018710 con 0x7fc7180746c0 2024-06-03T03:21:37.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:37.034+0000 7fc720600700 1 -- 172.21.15.117:0/3993109472 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7181abe00 con 0x7fc7180746c0 2024-06-03T03:21:37.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:37.035+0000 7fc720600700 1 -- 172.21.15.117:0/3993109472 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc718069180 con 0x7fc7180746c0 2024-06-03T03:21:37.038 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:37.037+0000 7fc716000700 1 -- 172.21.15.117:0/3993109472 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fc70c010430 con 0x7fc7180746c0 2024-06-03T03:21:37.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:37.038+0000 7fc716000700 1 --2- 172.21.15.117:0/3993109472 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fc7000a2d30 0x7fc7000a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:21:37.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:37.038+0000 7fc716000700 1 -- 172.21.15.117:0/3993109472 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7fc70c0cf770 con 0x7fc7180746c0 2024-06-03T03:21:37.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:37.039+0000 7fc71d200700 1 --2- 172.21.15.117:0/3993109472 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fc7000a2d30 0x7fc7000a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:21:37.040 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:37.039+0000 7fc71d200700 1 --2- 172.21.15.117:0/3993109472 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fc7000a2d30 0x7fc7000a51f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc704009750 tx=0x7fc70400b280 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:21:37.043 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:37.043+0000 7fc716000700 1 -- 172.21.15.117:0/3993109472 <== mon.2 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc70c09c180 con 0x7fc7180746c0 2024-06-03T03:21:37.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:37.315+0000 7fc720600700 1 -- 172.21.15.117:0/3993109472 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc7181abfd0 con 0x7fc7180746c0 2024-06-03T03:21:37.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:37.316+0000 7fc716000700 1 -- 172.21.15.117:0/3993109472 <== mon.2 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11955 (secure 0 0 0) 0x7fc70c09ba80 con 0x7fc7180746c0 2024-06-03T03:21:37.317 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T03:21:37.318 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3353001119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3353001119},{"type":"v1","addr":"172.21.15.117:6837","nonce":3353001119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14388,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6839/405782304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":405782304},{"type":"v1","addr":"172.21.15.117:6839","nonce":405782304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-03T03:19:44.629822+0000","modified":"2024-06-03T03:21:12.085445+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24517,"mds_1":24512,"mds_2":24503,"mds_3":24499,"mds_4":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24431":{"gid":24431,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6833/3355662416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3355662416},{"type":"v1","addr":"172.21.15.151:6833","nonce":3355662416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/2429725085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2429725085},{"type":"v1","addr":"172.21.15.183:6833","nonce":2429725085}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/31153579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":31153579},{"type":"v1","addr":"172.21.15.151:6835","nonce":31153579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"j","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/2884460088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2884460088},{"type":"v1","addr":"172.21.15.117:6835","nonce":2884460088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/2392407848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":2392407848},{"type":"v1","addr":"172.21.15.151:6837","nonce":2392407848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":32,"addr":"172.21.15.183:6835/1009004320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1009004320},{"type":"v1","addr":"172.21.15.183:6835","nonce":1009004320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.183:6837/2697518526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":2697518526},{"type":"v1","addr":"172.21.15.183:6837","nonce":2697518526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"a","rank":2,"incarnation":22,"state":"up:active","state_seq":27,"addr":"172.21.15.117:6841/1212242967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6840","nonce":1212242967},{"type":"v1","addr":"172.21.15.117:6841","nonce":1212242967}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.151:6839/3623444616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":3623444616},{"type":"v1","addr":"172.21.15.151:6839","nonce":3623444616}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":10,"addr":"172.21.15.183:6839/1439615083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1439615083},{"type":"v1","addr":"172.21.15.183:6839","nonce":1439615083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24517,"qdb_cluster":[24503,24517,24512,24487,24499]},"id":1}]} 2024-06-03T03:21:37.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:37.319+0000 7fc720600700 1 -- 172.21.15.117:0/3993109472 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fc7000a2d30 msgr2=0x7fc7000a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:21:37.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:37.319+0000 7fc720600700 1 --2- 172.21.15.117:0/3993109472 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fc7000a2d30 0x7fc7000a51f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc704009750 tx=0x7fc70400b280 comp rx=0 tx=0).stop 2024-06-03T03:21:37.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:37.319+0000 7fc720600700 1 -- 172.21.15.117:0/3993109472 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7180746c0 msgr2=0x7fc71819f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:21:37.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:37.319+0000 7fc720600700 1 --2- 172.21.15.117:0/3993109472 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7180746c0 0x7fc71819f280 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc70c00e9f0 tx=0x7fc70c00ed00 comp rx=0 tx=0).stop 2024-06-03T03:21:37.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:37.319+0000 7fc720600700 1 -- 172.21.15.117:0/3993109472 shutdown_connections 2024-06-03T03:21:37.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:37.319+0000 7fc720600700 1 --2- 172.21.15.117:0/3993109472 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fc7000a2d30 0x7fc7000a51f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:37.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:37.319+0000 7fc720600700 1 --2- 172.21.15.117:0/3993109472 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc718113660 0x7fc7181a3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:37.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:37.319+0000 7fc720600700 1 --2- 172.21.15.117:0/3993109472 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7180746c0 0x7fc71819f280 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:37.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:37.319+0000 7fc720600700 1 --2- 172.21.15.117:0/3993109472 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc718074fe0 0x7fc71819f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:37.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:37.319+0000 7fc720600700 1 -- 172.21.15.117:0/3993109472 >> 172.21.15.117:0/3993109472 conn(0x7fc7180795f0 msgr2=0x7fc718112420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:21:37.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:37.319+0000 7fc720600700 1 -- 172.21.15.117:0/3993109472 shutdown_connections 2024-06-03T03:21:37.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:37.320+0000 7fc720600700 1 -- 172.21.15.117:0/3993109472 wait complete. 2024-06-03T03:21:37.322 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 30 2024-06-03T03:21:38.115 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:37 smithi117 ceph-mon[31649]: pgmap v395: 129 pgs: 129 active+clean; 632 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 2.8 KiB/s wr, 11 op/s 2024-06-03T03:21:38.116 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:37 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/3993109472' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T03:21:38.193 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-03T03:21:38.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:37 smithi183 ceph-mon[37326]: pgmap v395: 129 pgs: 129 active+clean; 632 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 2.8 KiB/s wr, 11 op/s 2024-06-03T03:21:38.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:37 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/3993109472' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T03:21:38.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:37 smithi151 ceph-mon[37043]: pgmap v395: 129 pgs: 129 active+clean; 632 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 2.8 KiB/s wr, 11 op/s 2024-06-03T03:21:38.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:37 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/3993109472' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T03:21:38.643 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:21:40.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:39 smithi117 ceph-mon[31649]: pgmap v396: 129 pgs: 129 active+clean; 640 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 3.3 KiB/s wr, 13 op/s 2024-06-03T03:21:40.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:39 smithi183 ceph-mon[37326]: pgmap v396: 129 pgs: 129 active+clean; 640 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 3.3 KiB/s wr, 13 op/s 2024-06-03T03:21:40.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:39 smithi151 ceph-mon[37043]: pgmap v396: 129 pgs: 129 active+clean; 640 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 3.3 KiB/s wr, 13 op/s 2024-06-03T03:21:40.794 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:40.793+0000 7fe4c6200700 1 -- 172.21.15.117:0/1235908464 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe4c006b610 msgr2=0x7fe4c0107090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:21:40.794 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:40.793+0000 7fe4c6200700 1 --2- 172.21.15.117:0/1235908464 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe4c006b610 0x7fe4c0107090 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac009990 tx=0x7fe4ac009ca0 comp rx=0 tx=0).stop 2024-06-03T03:21:40.794 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:40.793+0000 7fe4c6200700 1 -- 172.21.15.117:0/1235908464 shutdown_connections 2024-06-03T03:21:40.795 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:40.793+0000 7fe4c6200700 1 --2- 172.21.15.117:0/1235908464 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe4c01077c0 0x7fe4c010f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:40.795 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:40.793+0000 7fe4c6200700 1 --2- 172.21.15.117:0/1235908464 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe4c006ac60 0x7fe4c006b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:40.795 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:40.793+0000 7fe4c6200700 1 --2- 172.21.15.117:0/1235908464 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe4c006b610 0x7fe4c0107090 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:40.795 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:40.793+0000 7fe4c6200700 1 -- 172.21.15.117:0/1235908464 >> 172.21.15.117:0/1235908464 conn(0x7fe4c00781d0 msgr2=0x7fe4c00785e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:21:40.795 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:40.793+0000 7fe4c6200700 1 -- 172.21.15.117:0/1235908464 shutdown_connections 2024-06-03T03:21:40.795 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:40.793+0000 7fe4c6200700 1 -- 172.21.15.117:0/1235908464 wait complete. 2024-06-03T03:21:40.795 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:40.794+0000 7fe4c6200700 1 Processor -- start 2024-06-03T03:21:40.796 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:40.795+0000 7fe4c6200700 1 -- start start 2024-06-03T03:21:40.796 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:40.795+0000 7fe4c6200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe4c006ac60 0x7fe4c0101620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:21:40.796 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:40.795+0000 7fe4c6200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe4c006b610 0x7fe4c0101b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:21:40.796 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:40.795+0000 7fe4c6200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe4c01077c0 0x7fe4c01020a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:21:40.796 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:40.795+0000 7fe4c6200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c0112700 con 0x7fe4c006b610 2024-06-03T03:21:40.797 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:40.795+0000 7fe4c6200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c0112580 con 0x7fe4c006ac60 2024-06-03T03:21:40.797 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:40.795+0000 7fe4c6200700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c0112880 con 0x7fe4c01077c0 2024-06-03T03:21:40.797 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:40.796+0000 7fe4c4e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe4c006ac60 0x7fe4c0101620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:21:40.797 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:40.796+0000 7fe4bfe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe4c006b610 0x7fe4c0101b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:21:40.797 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:40.796+0000 7fe4c4e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe4c006ac60 0x7fe4c0101620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:60066/0 (socket says 172.21.15.117:60066) 2024-06-03T03:21:40.797 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:40.796+0000 7fe4c4e00700 1 -- 172.21.15.117:0/2885875983 learned_addr learned my addr 172.21.15.117:0/2885875983 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:21:40.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:40.796+0000 7fe4c5800700 1 --2- 172.21.15.117:0/2885875983 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe4c01077c0 0x7fe4c01020a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:21:40.799 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:40.796+0000 7fe4c4e00700 1 -- 172.21.15.117:0/2885875983 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe4c01077c0 msgr2=0x7fe4c01020a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:21:40.799 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:40.796+0000 7fe4c4e00700 1 --2- 172.21.15.117:0/2885875983 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe4c01077c0 0x7fe4c01020a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:40.799 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:40.796+0000 7fe4c4e00700 1 -- 172.21.15.117:0/2885875983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe4c006b610 msgr2=0x7fe4c0101b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:21:40.799 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:40.796+0000 7fe4c4e00700 1 --2- 172.21.15.117:0/2885875983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe4c006b610 0x7fe4c0101b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:40.799 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:40.796+0000 7fe4c4e00700 1 -- 172.21.15.117:0/2885875983 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4c01a7120 con 0x7fe4c006ac60 2024-06-03T03:21:40.799 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:40.796+0000 7fe4c4e00700 1 --2- 172.21.15.117:0/2885875983 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe4c006ac60 0x7fe4c0101620 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe4b400e9a0 tx=0x7fe4b400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:21:40.799 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:40.797+0000 7fe4bd600700 1 -- 172.21.15.117:0/2885875983 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b400cbd0 con 0x7fe4c006ac60 2024-06-03T03:21:40.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:40.797+0000 7fe4bd600700 1 -- 172.21.15.117:0/2885875983 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4b400cd70 con 0x7fe4c006ac60 2024-06-03T03:21:40.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:40.797+0000 7fe4bd600700 1 -- 172.21.15.117:0/2885875983 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b4018750 con 0x7fe4c006ac60 2024-06-03T03:21:40.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:40.797+0000 7fe4c6200700 1 -- 172.21.15.117:0/2885875983 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4c01a7410 con 0x7fe4c006ac60 2024-06-03T03:21:40.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:40.797+0000 7fe4c6200700 1 -- 172.21.15.117:0/2885875983 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4c01a7970 con 0x7fe4c006ac60 2024-06-03T03:21:40.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:40.798+0000 7fe4c6200700 1 -- 172.21.15.117:0/2885875983 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4c0069180 con 0x7fe4c006ac60 2024-06-03T03:21:40.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:40.800+0000 7fe4bd600700 1 -- 172.21.15.117:0/2885875983 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fe4b4010430 con 0x7fe4c006ac60 2024-06-03T03:21:40.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:40.801+0000 7fe4bd600700 1 --2- 172.21.15.117:0/2885875983 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fe4a80a2d30 0x7fe4a80a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:21:40.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:40.801+0000 7fe4bd600700 1 -- 172.21.15.117:0/2885875983 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7fe4b40cf820 con 0x7fe4c006ac60 2024-06-03T03:21:40.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:40.802+0000 7fe4bfe00700 1 --2- 172.21.15.117:0/2885875983 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fe4a80a2d30 0x7fe4a80a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:21:40.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:40.802+0000 7fe4bfe00700 1 --2- 172.21.15.117:0/2885875983 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fe4a80a2d30 0x7fe4a80a51f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac00b2e0 tx=0x7fe4ac005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:21:40.806 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:40.805+0000 7fe4bd600700 1 -- 172.21.15.117:0/2885875983 <== mon.2 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe4b409c230 con 0x7fe4c006ac60 2024-06-03T03:21:41.043 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:41.042+0000 7fe4c6200700 1 -- 172.21.15.117:0/2885875983 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe4c002dca0 con 0x7fe4c006ac60 2024-06-03T03:21:41.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:41.045+0000 7fe4bd600700 1 -- 172.21.15.117:0/2885875983 <== mon.2 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+24807 (secure 0 0 0) 0x7fe4b409bb30 con 0x7fe4c006ac60 2024-06-03T03:21:41.047 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T03:21:41.047 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":99,"fsid":"a0e8473a-2156-11ef-bc9b-c7b262605968","created":"2024-06-03T03:10:08.446859+0000","modified":"2024-06-03T03:21:25.966405+0000","last_up_change":"2024-06-03T03:18:28.736904+0000","last_in_change":"2024-06-03T03:18:08.485711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-03T03:14:09.370986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-03T03:19:35.044683+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-03T03:19:40.097569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-03T03:19:55.394480+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8fefa089-9401-4a1c-b9fb-fe30c63e6a1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":4286644492},{"type":"v1","addr":"172.21.15.117:6803","nonce":4286644492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":4286644492},{"type":"v1","addr":"172.21.15.117:6805","nonce":4286644492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":4286644492},{"type":"v1","addr":"172.21.15.117:6809","nonce":4286644492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":4286644492},{"type":"v1","addr":"172.21.15.117:6807","nonce":4286644492}]},"public_addr":"172.21.15.117:6803/4286644492","cluster_addr":"172.21.15.117:6805/4286644492","heartbeat_back_addr":"172.21.15.117:6809/4286644492","heartbeat_front_addr":"172.21.15.117:6807/4286644492","state":["exists","up"]},{"osd":1,"uuid":"5b12b371-f155-4b67-a470-25cbe59a2999","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":262491158},{"type":"v1","addr":"172.21.15.151:6801","nonce":262491158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":262491158},{"type":"v1","addr":"172.21.15.151:6803","nonce":262491158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":262491158},{"type":"v1","addr":"172.21.15.151:6807","nonce":262491158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":262491158},{"type":"v1","addr":"172.21.15.151:6805","nonce":262491158}]},"public_addr":"172.21.15.151:6801/262491158","cluster_addr":"172.21.15.151:6803/262491158","heartbeat_back_addr":"172.21.15.151:6807/262491158","heartbeat_front_addr":"172.21.15.151:6805/262491158","state":["exists","up"]},{"osd":2,"uuid":"dbc17818-c54c-4ecf-b656-03578b95d8c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":3542453623},{"type":"v1","addr":"172.21.15.183:6801","nonce":3542453623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":3542453623},{"type":"v1","addr":"172.21.15.183:6803","nonce":3542453623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":3542453623},{"type":"v1","addr":"172.21.15.183:6807","nonce":3542453623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":3542453623},{"type":"v1","addr":"172.21.15.183:6805","nonce":3542453623}]},"public_addr":"172.21.15.183:6801/3542453623","cluster_addr":"172.21.15.183:6803/3542453623","heartbeat_back_addr":"172.21.15.183:6807/3542453623","heartbeat_front_addr":"172.21.15.183:6805/3542453623","state":["exists","up"]},{"osd":3,"uuid":"ca08d2ac-b1d6-4a49-9ae8-e3ec146899a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":589012315},{"type":"v1","addr":"172.21.15.117:6811","nonce":589012315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":589012315},{"type":"v1","addr":"172.21.15.117:6813","nonce":589012315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":589012315},{"type":"v1","addr":"172.21.15.117:6817","nonce":589012315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":589012315},{"type":"v1","addr":"172.21.15.117:6815","nonce":589012315}]},"public_addr":"172.21.15.117:6811/589012315","cluster_addr":"172.21.15.117:6813/589012315","heartbeat_back_addr":"172.21.15.117:6817/589012315","heartbeat_front_addr":"172.21.15.117:6815/589012315","state":["exists","up"]},{"osd":4,"uuid":"810d6c7a-023f-4930-82af-bc2bbfd647da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":1239048647},{"type":"v1","addr":"172.21.15.151:6809","nonce":1239048647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":1239048647},{"type":"v1","addr":"172.21.15.151:6811","nonce":1239048647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":1239048647},{"type":"v1","addr":"172.21.15.151:6815","nonce":1239048647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":1239048647},{"type":"v1","addr":"172.21.15.151:6813","nonce":1239048647}]},"public_addr":"172.21.15.151:6809/1239048647","cluster_addr":"172.21.15.151:6811/1239048647","heartbeat_back_addr":"172.21.15.151:6815/1239048647","heartbeat_front_addr":"172.21.15.151:6813/1239048647","state":["exists","up"]},{"osd":5,"uuid":"8ab6a5f0-34bb-48d8-9471-a7dc836e486e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":2985874686},{"type":"v1","addr":"172.21.15.183:6809","nonce":2985874686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":2985874686},{"type":"v1","addr":"172.21.15.183:6811","nonce":2985874686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":2985874686},{"type":"v1","addr":"172.21.15.183:6815","nonce":2985874686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":2985874686},{"type":"v1","addr":"172.21.15.183:6813","nonce":2985874686}]},"public_addr":"172.21.15.183:6809/2985874686","cluster_addr":"172.21.15.183:6811/2985874686","heartbeat_back_addr":"172.21.15.183:6815/2985874686","heartbeat_front_addr":"172.21.15.183:6813/2985874686","state":["exists","up"]},{"osd":6,"uuid":"78d15739-e9e0-4f19-ae5f-ee8527a5b527","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4250583193},{"type":"v1","addr":"172.21.15.117:6819","nonce":4250583193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":4250583193},{"type":"v1","addr":"172.21.15.117:6821","nonce":4250583193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":4250583193},{"type":"v1","addr":"172.21.15.117:6825","nonce":4250583193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4250583193},{"type":"v1","addr":"172.21.15.117:6823","nonce":4250583193}]},"public_addr":"172.21.15.117:6819/4250583193","cluster_addr":"172.21.15.117:6821/4250583193","heartbeat_back_addr":"172.21.15.117:6825/4250583193","heartbeat_front_addr":"172.21.15.117:6823/4250583193","state":["exists","up"]},{"osd":7,"uuid":"3724fe1f-f4d7-443b-aea8-ff3ab02bfdc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":386094645},{"type":"v1","addr":"172.21.15.151:6817","nonce":386094645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":386094645},{"type":"v1","addr":"172.21.15.151:6819","nonce":386094645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":386094645},{"type":"v1","addr":"172.21.15.151:6823","nonce":386094645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":386094645},{"type":"v1","addr":"172.21.15.151:6821","nonce":386094645}]},"public_addr":"172.21.15.151:6817/386094645","cluster_addr":"172.21.15.151:6819/386094645","heartbeat_back_addr":"172.21.15.151:6823/386094645","heartbeat_front_addr":"172.21.15.151:6821/386094645","state":["exists","up"]},{"osd":8,"uuid":"006ac092-f015-448e-a0c0-f7b0c4597a18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1829748222},{"type":"v1","addr":"172.21.15.183:6817","nonce":1829748222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1829748222},{"type":"v1","addr":"172.21.15.183:6819","nonce":1829748222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1829748222},{"type":"v1","addr":"172.21.15.183:6823","nonce":1829748222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1829748222},{"type":"v1","addr":"172.21.15.183:6821","nonce":1829748222}]},"public_addr":"172.21.15.183:6817/1829748222","cluster_addr":"172.21.15.183:6819/1829748222","heartbeat_back_addr":"172.21.15.183:6823/1829748222","heartbeat_front_addr":"172.21.15.183:6821/1829748222","state":["exists","up"]},{"osd":9,"uuid":"00a970e6-2502-459e-975c-badac93c9bd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2538315141},{"type":"v1","addr":"172.21.15.117:6827","nonce":2538315141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2538315141},{"type":"v1","addr":"172.21.15.117:6829","nonce":2538315141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2538315141},{"type":"v1","addr":"172.21.15.117:6833","nonce":2538315141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2538315141},{"type":"v1","addr":"172.21.15.117:6831","nonce":2538315141}]},"public_addr":"172.21.15.117:6827/2538315141","cluster_addr":"172.21.15.117:6829/2538315141","heartbeat_back_addr":"172.21.15.117:6833/2538315141","heartbeat_front_addr":"172.21.15.117:6831/2538315141","state":["exists","up"]},{"osd":10,"uuid":"ba2333cd-32a3-41ef-9ea5-287c707f512b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":3801045133},{"type":"v1","addr":"172.21.15.151:6825","nonce":3801045133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3801045133},{"type":"v1","addr":"172.21.15.151:6827","nonce":3801045133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3801045133},{"type":"v1","addr":"172.21.15.151:6831","nonce":3801045133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3801045133},{"type":"v1","addr":"172.21.15.151:6829","nonce":3801045133}]},"public_addr":"172.21.15.151:6825/3801045133","cluster_addr":"172.21.15.151:6827/3801045133","heartbeat_back_addr":"172.21.15.151:6831/3801045133","heartbeat_front_addr":"172.21.15.151:6829/3801045133","state":["exists","up"]},{"osd":11,"uuid":"1556d654-e6fa-42c1-b399-aa53f6837a1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3513899696},{"type":"v1","addr":"172.21.15.183:6825","nonce":3513899696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3513899696},{"type":"v1","addr":"172.21.15.183:6827","nonce":3513899696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3513899696},{"type":"v1","addr":"172.21.15.183:6831","nonce":3513899696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3513899696},{"type":"v1","addr":"172.21.15.183:6829","nonce":3513899696}]},"public_addr":"172.21.15.183:6825/3513899696","cluster_addr":"172.21.15.183:6827/3513899696","heartbeat_back_addr":"172.21.15.183:6831/3513899696","heartbeat_front_addr":"172.21.15.183:6829/3513899696","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:13:17.174041+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:13:42.237001+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:14:06.273315+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:14:39.912564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:15:06.063202+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:15:30.424818+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:16:05.623554+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:16:32.926393+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:16:58.165899+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:17:32.897477+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:18:00.720866+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:18:26.474441+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.117:0/186309008":"2024-06-04T03:11:24.655516+0000","172.21.15.117:6801/2142313590":"2024-06-04T03:11:24.655516+0000","172.21.15.117:6800/2142313590":"2024-06-04T03:11:24.655516+0000","172.21.15.117:0/2399725156":"2024-06-04T03:11:24.655516+0000","172.21.15.117:0/1453232061":"2024-06-04T03:10:50.552899+0000","172.21.15.117:0/4281674974":"2024-06-04T03:10:50.552899+0000","172.21.15.117:6800/2424554985":"2024-06-04T03:10:50.552899+0000","172.21.15.117:0/3946365225":"2024-06-04T03:11:24.655516+0000","172.21.15.117:0/1146162354":"2024-06-04T03:10:50.552899+0000","172.21.15.117:6801/2424554985":"2024-06-04T03:10:50.552899+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-03T03:21:41.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:41.047+0000 7fe4c6200700 1 -- 172.21.15.117:0/2885875983 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fe4a80a2d30 msgr2=0x7fe4a80a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:21:41.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:41.047+0000 7fe4c6200700 1 --2- 172.21.15.117:0/2885875983 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fe4a80a2d30 0x7fe4a80a51f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac00b2e0 tx=0x7fe4ac005fd0 comp rx=0 tx=0).stop 2024-06-03T03:21:41.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:41.047+0000 7fe4c6200700 1 -- 172.21.15.117:0/2885875983 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe4c006ac60 msgr2=0x7fe4c0101620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:21:41.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:41.047+0000 7fe4c6200700 1 --2- 172.21.15.117:0/2885875983 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe4c006ac60 0x7fe4c0101620 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe4b400e9a0 tx=0x7fe4b400ecb0 comp rx=0 tx=0).stop 2024-06-03T03:21:41.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:41.048+0000 7fe4c6200700 1 -- 172.21.15.117:0/2885875983 shutdown_connections 2024-06-03T03:21:41.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:41.048+0000 7fe4c6200700 1 --2- 172.21.15.117:0/2885875983 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fe4a80a2d30 0x7fe4a80a51f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:41.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:41.048+0000 7fe4c6200700 1 --2- 172.21.15.117:0/2885875983 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe4c01077c0 0x7fe4c01020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:41.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:41.048+0000 7fe4c6200700 1 --2- 172.21.15.117:0/2885875983 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe4c006ac60 0x7fe4c0101620 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:41.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:41.048+0000 7fe4c6200700 1 --2- 172.21.15.117:0/2885875983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe4c006b610 0x7fe4c0101b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:41.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:41.048+0000 7fe4c6200700 1 -- 172.21.15.117:0/2885875983 >> 172.21.15.117:0/2885875983 conn(0x7fe4c00781d0 msgr2=0x7fe4c00ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:21:41.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:41.048+0000 7fe4c6200700 1 -- 172.21.15.117:0/2885875983 shutdown_connections 2024-06-03T03:21:41.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:41.048+0000 7fe4c6200700 1 -- 172.21.15.117:0/2885875983 wait complete. 2024-06-03T03:21:41.158 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:40 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:21:41.158 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:40 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:21:41.158 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:40 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:21:41.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:40 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:21:41.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:40 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:21:41.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:40 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:21:41.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:40 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:21:41.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:40 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:21:41.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:40 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:21:41.912 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-06-03T03:21:41.913 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-03T03:21:42.172 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:41 smithi117 ceph-mon[31649]: pgmap v397: 129 pgs: 129 active+clean; 640 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-06-03T03:21:42.172 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:41 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/2885875983' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T03:21:42.172 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:41 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:21:42.172 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:41 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:21:42.172 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:41 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:21:42.172 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:41 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:21:42.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:41 smithi183 ceph-mon[37326]: pgmap v397: 129 pgs: 129 active+clean; 640 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-06-03T03:21:42.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:41 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/2885875983' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T03:21:42.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:41 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:21:42.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:41 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:21:42.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:41 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:21:42.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:41 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:21:42.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:41 smithi151 ceph-mon[37043]: pgmap v397: 129 pgs: 129 active+clean; 640 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-06-03T03:21:42.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:41 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/2885875983' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T03:21:42.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:41 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:21:42.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:41 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:21:42.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:41 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:21:42.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:41 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:21:43.062 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:21:44.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:43 smithi183 ceph-mon[37326]: pgmap v398: 129 pgs: 129 active+clean; 645 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.3 KiB/s wr, 13 op/s 2024-06-03T03:21:44.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:43 smithi151 ceph-mon[37043]: pgmap v398: 129 pgs: 129 active+clean; 645 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.3 KiB/s wr, 13 op/s 2024-06-03T03:21:44.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:43 smithi117 ceph-mon[31649]: pgmap v398: 129 pgs: 129 active+clean; 645 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.3 KiB/s wr, 13 op/s 2024-06-03T03:21:45.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:45.264+0000 7f722ec00700 1 -- 172.21.15.117:0/2889818325 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72281045c0 msgr2=0x7f7228104a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:21:45.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:45.264+0000 7f722ec00700 1 --2- 172.21.15.117:0/2889818325 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72281045c0 0x7f7228104a40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7214009990 tx=0x7f7214009ca0 comp rx=0 tx=0).stop 2024-06-03T03:21:45.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:45.264+0000 7f722ec00700 1 -- 172.21.15.117:0/2889818325 shutdown_connections 2024-06-03T03:21:45.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:45.264+0000 7f722ec00700 1 --2- 172.21.15.117:0/2889818325 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7228104f80 0x7f7228111490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:45.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:45.264+0000 7f722ec00700 1 --2- 172.21.15.117:0/2889818325 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f722810a650 0x7f722810aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:45.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:45.264+0000 7f722ec00700 1 --2- 172.21.15.117:0/2889818325 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72281045c0 0x7f7228104a40 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:45.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:45.264+0000 7f722ec00700 1 -- 172.21.15.117:0/2889818325 >> 172.21.15.117:0/2889818325 conn(0x7f72281000b0 msgr2=0x7f72281024d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:21:45.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:45.265+0000 7f722ec00700 1 -- 172.21.15.117:0/2889818325 shutdown_connections 2024-06-03T03:21:45.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:45.265+0000 7f722ec00700 1 -- 172.21.15.117:0/2889818325 wait complete. 2024-06-03T03:21:45.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:45.266+0000 7f722ec00700 1 Processor -- start 2024-06-03T03:21:45.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:45.266+0000 7f722ec00700 1 -- start start 2024-06-03T03:21:45.268 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:45.267+0000 7f722ec00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f72281045c0 0x7f72281a0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:21:45.268 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:45.267+0000 7f722ec00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7228104f80 0x7f72281a0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:21:45.268 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:45.267+0000 7f722ec00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f722810a650 0x7f722819ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:21:45.268 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:45.267+0000 7f722ec00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f72281148f0 con 0x7f7228104f80 2024-06-03T03:21:45.268 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:45.267+0000 7f722ec00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7228114770 con 0x7f72281045c0 2024-06-03T03:21:45.268 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:45.267+0000 7f722ec00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7228114a70 con 0x7f722810a650 2024-06-03T03:21:45.268 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:45.267+0000 7f722d800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f72281045c0 0x7f72281a0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:21:45.268 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:45.267+0000 7f722ce00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7228104f80 0x7f72281a0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:21:45.268 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:45.267+0000 7f722d800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f72281045c0 0x7f72281a0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:60084/0 (socket says 172.21.15.117:60084) 2024-06-03T03:21:45.268 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:45.267+0000 7f722d800700 1 -- 172.21.15.117:0/2175624954 learned_addr learned my addr 172.21.15.117:0/2175624954 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:21:45.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:45.267+0000 7f722e200700 1 --2- 172.21.15.117:0/2175624954 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f722810a650 0x7f722819ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:21:45.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:45.268+0000 7f722d800700 1 -- 172.21.15.117:0/2175624954 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f722810a650 msgr2=0x7f722819ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:21:45.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:45.268+0000 7f722d800700 1 --2- 172.21.15.117:0/2175624954 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f722810a650 0x7f722819ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:45.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:45.268+0000 7f722d800700 1 -- 172.21.15.117:0/2175624954 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7228104f80 msgr2=0x7f72281a0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:21:45.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:45.268+0000 7f722d800700 1 --2- 172.21.15.117:0/2175624954 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7228104f80 0x7f72281a0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:45.270 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:45.268+0000 7f722d800700 1 -- 172.21.15.117:0/2175624954 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f722819b270 con 0x7f72281045c0 2024-06-03T03:21:45.270 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:45.268+0000 7f722e200700 1 --2- 172.21.15.117:0/2175624954 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f722810a650 0x7f722819ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T03:21:45.270 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:45.268+0000 7f722d800700 1 --2- 172.21.15.117:0/2175624954 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f72281045c0 0x7f72281a0900 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f721c00d980 tx=0x7f721c00dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:21:45.270 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:45.268+0000 7f7226000700 1 -- 172.21.15.117:0/2175624954 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f721c017030 con 0x7f72281045c0 2024-06-03T03:21:45.270 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:45.268+0000 7f7226000700 1 -- 172.21.15.117:0/2175624954 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f721c004500 con 0x7f72281045c0 2024-06-03T03:21:45.270 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:45.268+0000 7f722ec00700 1 -- 172.21.15.117:0/2175624954 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f722819b560 con 0x7f72281045c0 2024-06-03T03:21:45.270 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:45.268+0000 7f7226000700 1 -- 172.21.15.117:0/2175624954 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f721c003aa0 con 0x7f72281045c0 2024-06-03T03:21:45.270 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:45.268+0000 7f722ec00700 1 -- 172.21.15.117:0/2175624954 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72281a77d0 con 0x7f72281045c0 2024-06-03T03:21:45.270 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:45.269+0000 7f722ec00700 1 -- 172.21.15.117:0/2175624954 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7228069180 con 0x7f72281045c0 2024-06-03T03:21:45.272 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:45.271+0000 7f7226000700 1 -- 172.21.15.117:0/2175624954 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f721c01a070 con 0x7f72281045c0 2024-06-03T03:21:45.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:45.273+0000 7f7226000700 1 --2- 172.21.15.117:0/2175624954 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f72100a2cb0 0x7f72100a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:21:45.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:45.273+0000 7f7226000700 1 -- 172.21.15.117:0/2175624954 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f721c0cf0c0 con 0x7f72281045c0 2024-06-03T03:21:45.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:45.273+0000 7f722ce00700 1 --2- 172.21.15.117:0/2175624954 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f72100a2cb0 0x7f72100a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:21:45.275 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:45.274+0000 7f722ce00700 1 --2- 172.21.15.117:0/2175624954 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f72100a2cb0 0x7f72100a5170 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7228109c20 tx=0x7f7214005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:21:45.276 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:45.275+0000 7f7226000700 1 -- 172.21.15.117:0/2175624954 <== mon.2 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f721c09bad0 con 0x7f72281045c0 2024-06-03T03:21:45.505 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:45.504+0000 7f722ec00700 1 -- 172.21.15.117:0/2175624954 --> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f72280628d0 con 0x7f72100a2cb0 2024-06-03T03:21:45.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:45.530+0000 7f7226000700 1 -- 172.21.15.117:0/2175624954 <== mgr.14152 v2:172.21.15.117:6800/1398160887 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340702 (secure 0 0 0) 0x7f72280628d0 con 0x7f72100a2cb0 2024-06-03T03:21:45.536 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T03:21:45.539 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:45.536+0000 7f722ec00700 1 -- 172.21.15.117:0/2175624954 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f72100a2cb0 msgr2=0x7f72100a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:21:45.539 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:45.536+0000 7f722ec00700 1 --2- 172.21.15.117:0/2175624954 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f72100a2cb0 0x7f72100a5170 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7228109c20 tx=0x7f7214005960 comp rx=0 tx=0).stop 2024-06-03T03:21:45.539 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:45.536+0000 7f722ec00700 1 -- 172.21.15.117:0/2175624954 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f72281045c0 msgr2=0x7f72281a0900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:21:45.539 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:45.536+0000 7f722ec00700 1 --2- 172.21.15.117:0/2175624954 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f72281045c0 0x7f72281a0900 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f721c00d980 tx=0x7f721c00dc90 comp rx=0 tx=0).stop 2024-06-03T03:21:45.540 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:45.536+0000 7f722ec00700 1 -- 172.21.15.117:0/2175624954 shutdown_connections 2024-06-03T03:21:45.540 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:45.536+0000 7f722ec00700 1 --2- 172.21.15.117:0/2175624954 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f72100a2cb0 0x7f72100a5170 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:45.540 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:45.536+0000 7f722ec00700 1 --2- 172.21.15.117:0/2175624954 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f722810a650 0x7f722819ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:45.540 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:45.536+0000 7f722ec00700 1 --2- 172.21.15.117:0/2175624954 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f72281045c0 0x7f72281a0900 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:45.540 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:45.536+0000 7f722ec00700 1 --2- 172.21.15.117:0/2175624954 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7228104f80 0x7f72281a0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:45.540 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:45.536+0000 7f722ec00700 1 -- 172.21.15.117:0/2175624954 >> 172.21.15.117:0/2175624954 conn(0x7f72281000b0 msgr2=0x7f7228101670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:21:45.540 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:45.536+0000 7f722ec00700 1 -- 172.21.15.117:0/2175624954 shutdown_connections 2024-06-03T03:21:45.540 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:45.536+0000 7f722ec00700 1 -- 172.21.15.117:0/2175624954 wait complete. 2024-06-03T03:21:45.540 INFO:teuthology.orchestra.run.smithi117.stderr:dumped all 2024-06-03T03:21:46.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:45 smithi183 ceph-mon[37326]: pgmap v399: 129 pgs: 129 active+clean; 645 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-06-03T03:21:46.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:45 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:21:46.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:45 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:21:46.371 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:45 smithi117 ceph-mon[31649]: pgmap v399: 129 pgs: 129 active+clean; 645 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-06-03T03:21:46.371 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:45 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:21:46.371 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:45 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:21:46.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:45 smithi151 ceph-mon[37043]: pgmap v399: 129 pgs: 129 active+clean; 645 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-06-03T03:21:46.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:45 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:21:46.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:45 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:21:46.445 INFO:teuthology.orchestra.run.smithi117.stdout:{"pg_ready":true,"pg_map":{"version":399,"stamp":"2024-06-03T03:21:44.855313+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660966,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":513,"num_read_kb":377,"num_write":437,"num_write_kb":3082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":356,"ondisk_log_size":356,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":370956,"kb_used_data":13452,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124570868,"statfs":{"total":1151940427776,"available":1151560568832,"internally_reserved":0,"allocated":13774848,"data_stored":8800273,"data_compressed":501066,"data_compressed_allocated":688128,"data_compressed_original":2875392,"omap_allocated":19142,"internal_metadata":365933882},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":14254,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":112,"num_read_kb":70,"num_write":13,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005904"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.054480+0000","last_change":"2024-06-03T03:19:41.626030+0000","last_active":"2024-06-03T03:21:26.054480+0000","last_peered":"2024-06-03T03:21:26.054480+0000","last_clean":"2024-06-03T03:21:26.054480+0000","last_became_active":"2024-06-03T03:19:41.625681+0000","last_became_peered":"2024-06-03T03:19:41.625681+0000","last_unstale":"2024-06-03T03:21:26.054480+0000","last_undegraded":"2024-06-03T03:21:26.054480+0000","last_fullsized":"2024-06-03T03:21:26.054480+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:35:46.396360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.976761+0000","last_change":"2024-06-03T03:19:41.619668+0000","last_active":"2024-06-03T03:21:25.976761+0000","last_peered":"2024-06-03T03:21:25.976761+0000","last_clean":"2024-06-03T03:21:25.976761+0000","last_became_active":"2024-06-03T03:19:41.619318+0000","last_became_peered":"2024-06-03T03:19:41.619318+0000","last_unstale":"2024-06-03T03:21:25.976761+0000","last_undegraded":"2024-06-03T03:21:25.976761+0000","last_fullsized":"2024-06-03T03:21:25.976761+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:07:21.656045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.975413+0000","last_change":"2024-06-03T03:19:41.623689+0000","last_active":"2024-06-03T03:21:25.975413+0000","last_peered":"2024-06-03T03:21:25.975413+0000","last_clean":"2024-06-03T03:21:25.975413+0000","last_became_active":"2024-06-03T03:19:41.623027+0000","last_became_peered":"2024-06-03T03:19:41.623027+0000","last_unstale":"2024-06-03T03:21:25.975413+0000","last_undegraded":"2024-06-03T03:21:25.975413+0000","last_fullsized":"2024-06-03T03:21:25.975413+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:18:05.584397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.981031+0000","last_change":"2024-06-03T03:19:41.603802+0000","last_active":"2024-06-03T03:21:25.981031+0000","last_peered":"2024-06-03T03:21:25.981031+0000","last_clean":"2024-06-03T03:21:25.981031+0000","last_became_active":"2024-06-03T03:19:41.603470+0000","last_became_peered":"2024-06-03T03:19:41.603470+0000","last_unstale":"2024-06-03T03:21:25.981031+0000","last_undegraded":"2024-06-03T03:21:25.981031+0000","last_fullsized":"2024-06-03T03:21:25.981031+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:56:10.636108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.973503+0000","last_change":"2024-06-03T03:19:41.636436+0000","last_active":"2024-06-03T03:21:25.973503+0000","last_peered":"2024-06-03T03:21:25.973503+0000","last_clean":"2024-06-03T03:21:25.973503+0000","last_became_active":"2024-06-03T03:19:41.636150+0000","last_became_peered":"2024-06-03T03:19:41.636150+0000","last_unstale":"2024-06-03T03:21:25.973503+0000","last_undegraded":"2024-06-03T03:21:25.973503+0000","last_fullsized":"2024-06-03T03:21:25.973503+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:02:10.928251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.053609+0000","last_change":"2024-06-03T03:19:41.622426+0000","last_active":"2024-06-03T03:21:26.053609+0000","last_peered":"2024-06-03T03:21:26.053609+0000","last_clean":"2024-06-03T03:21:26.053609+0000","last_became_active":"2024-06-03T03:19:41.622092+0000","last_became_peered":"2024-06-03T03:19:41.622092+0000","last_unstale":"2024-06-03T03:21:26.053609+0000","last_undegraded":"2024-06-03T03:21:26.053609+0000","last_fullsized":"2024-06-03T03:21:26.053609+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T14:08:52.666771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.052326+0000","last_change":"2024-06-03T03:19:41.608303+0000","last_active":"2024-06-03T03:21:26.052326+0000","last_peered":"2024-06-03T03:21:26.052326+0000","last_clean":"2024-06-03T03:21:26.052326+0000","last_became_active":"2024-06-03T03:19:41.607929+0000","last_became_peered":"2024-06-03T03:19:41.607929+0000","last_unstale":"2024-06-03T03:21:26.052326+0000","last_undegraded":"2024-06-03T03:21:26.052326+0000","last_fullsized":"2024-06-03T03:21:26.052326+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:40:05.643531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.056269+0000","last_change":"2024-06-03T03:19:41.606722+0000","last_active":"2024-06-03T03:21:26.056269+0000","last_peered":"2024-06-03T03:21:26.056269+0000","last_clean":"2024-06-03T03:21:26.056269+0000","last_became_active":"2024-06-03T03:19:41.606131+0000","last_became_peered":"2024-06-03T03:19:41.606131+0000","last_unstale":"2024-06-03T03:21:26.056269+0000","last_undegraded":"2024-06-03T03:21:26.056269+0000","last_fullsized":"2024-06-03T03:21:26.056269+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:09:17.306848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.053811+0000","last_change":"2024-06-03T03:19:41.609287+0000","last_active":"2024-06-03T03:21:26.053811+0000","last_peered":"2024-06-03T03:21:26.053811+0000","last_clean":"2024-06-03T03:21:26.053811+0000","last_became_active":"2024-06-03T03:19:41.608521+0000","last_became_peered":"2024-06-03T03:19:41.608521+0000","last_unstale":"2024-06-03T03:21:26.053811+0000","last_undegraded":"2024-06-03T03:21:26.053811+0000","last_fullsized":"2024-06-03T03:21:26.053811+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:35:37.128667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.054740+0000","last_change":"2024-06-03T03:19:41.605193+0000","last_active":"2024-06-03T03:21:26.054740+0000","last_peered":"2024-06-03T03:21:26.054740+0000","last_clean":"2024-06-03T03:21:26.054740+0000","last_became_active":"2024-06-03T03:19:41.604787+0000","last_became_peered":"2024-06-03T03:19:41.604787+0000","last_unstale":"2024-06-03T03:21:26.054740+0000","last_undegraded":"2024-06-03T03:21:26.054740+0000","last_fullsized":"2024-06-03T03:21:26.054740+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:02:48.334599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.052944+0000","last_change":"2024-06-03T03:19:41.611522+0000","last_active":"2024-06-03T03:21:26.052944+0000","last_peered":"2024-06-03T03:21:26.052944+0000","last_clean":"2024-06-03T03:21:26.052944+0000","last_became_active":"2024-06-03T03:19:41.610932+0000","last_became_peered":"2024-06-03T03:19:41.610932+0000","last_unstale":"2024-06-03T03:21:26.052944+0000","last_undegraded":"2024-06-03T03:21:26.052944+0000","last_fullsized":"2024-06-03T03:21:26.052944+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:35:44.702791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":15,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.055524+0000","last_change":"2024-06-03T03:20:25.950663+0000","last_active":"2024-06-03T03:21:26.055524+0000","last_peered":"2024-06-03T03:21:26.055524+0000","last_clean":"2024-06-03T03:21:26.055524+0000","last_became_active":"2024-06-03T03:20:25.949962+0000","last_became_peered":"2024-06-03T03:20:25.949962+0000","last_unstale":"2024-06-03T03:21:26.055524+0000","last_undegraded":"2024-06-03T03:21:26.055524+0000","last_fullsized":"2024-06-03T03:21:26.055524+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:55:33.972567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.977681+0000","last_change":"2024-06-03T03:19:41.633464+0000","last_active":"2024-06-03T03:21:25.977681+0000","last_peered":"2024-06-03T03:21:25.977681+0000","last_clean":"2024-06-03T03:21:25.977681+0000","last_became_active":"2024-06-03T03:19:41.630625+0000","last_became_peered":"2024-06-03T03:19:41.630625+0000","last_unstale":"2024-06-03T03:21:25.977681+0000","last_undegraded":"2024-06-03T03:21:25.977681+0000","last_fullsized":"2024-06-03T03:21:25.977681+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:23:28.799058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.975828+0000","last_change":"2024-06-03T03:19:41.627234+0000","last_active":"2024-06-03T03:21:25.975828+0000","last_peered":"2024-06-03T03:21:25.975828+0000","last_clean":"2024-06-03T03:21:25.975828+0000","last_became_active":"2024-06-03T03:19:41.626643+0000","last_became_peered":"2024-06-03T03:19:41.626643+0000","last_unstale":"2024-06-03T03:21:25.975828+0000","last_undegraded":"2024-06-03T03:21:25.975828+0000","last_fullsized":"2024-06-03T03:21:25.975828+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:37:44.126101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.976005+0000","last_change":"2024-06-03T03:19:41.621833+0000","last_active":"2024-06-03T03:21:25.976005+0000","last_peered":"2024-06-03T03:21:25.976005+0000","last_clean":"2024-06-03T03:21:25.976005+0000","last_became_active":"2024-06-03T03:19:41.620689+0000","last_became_peered":"2024-06-03T03:19:41.620689+0000","last_unstale":"2024-06-03T03:21:25.976005+0000","last_undegraded":"2024-06-03T03:21:25.976005+0000","last_fullsized":"2024-06-03T03:21:25.976005+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:39:31.188372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.052549+0000","last_change":"2024-06-03T03:19:41.624072+0000","last_active":"2024-06-03T03:21:26.052549+0000","last_peered":"2024-06-03T03:21:26.052549+0000","last_clean":"2024-06-03T03:21:26.052549+0000","last_became_active":"2024-06-03T03:19:41.623689+0000","last_became_peered":"2024-06-03T03:19:41.623689+0000","last_unstale":"2024-06-03T03:21:26.052549+0000","last_undegraded":"2024-06-03T03:21:26.052549+0000","last_fullsized":"2024-06-03T03:21:26.052549+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:38:24.075256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.054263+0000","last_change":"2024-06-03T03:19:41.626145+0000","last_active":"2024-06-03T03:21:26.054263+0000","last_peered":"2024-06-03T03:21:26.054263+0000","last_clean":"2024-06-03T03:21:26.054263+0000","last_became_active":"2024-06-03T03:19:41.625826+0000","last_became_peered":"2024-06-03T03:19:41.625826+0000","last_unstale":"2024-06-03T03:21:26.054263+0000","last_undegraded":"2024-06-03T03:21:26.054263+0000","last_fullsized":"2024-06-03T03:21:26.054263+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:04:10.996134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.056880+0000","last_change":"2024-06-03T03:19:41.615754+0000","last_active":"2024-06-03T03:21:26.056880+0000","last_peered":"2024-06-03T03:21:26.056880+0000","last_clean":"2024-06-03T03:21:26.056880+0000","last_became_active":"2024-06-03T03:19:41.614577+0000","last_became_peered":"2024-06-03T03:19:41.614577+0000","last_unstale":"2024-06-03T03:21:26.056880+0000","last_undegraded":"2024-06-03T03:21:26.056880+0000","last_fullsized":"2024-06-03T03:21:26.056880+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:45:50.084332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.053093+0000","last_change":"2024-06-03T03:19:41.633000+0000","last_active":"2024-06-03T03:21:26.053093+0000","last_peered":"2024-06-03T03:21:26.053093+0000","last_clean":"2024-06-03T03:21:26.053093+0000","last_became_active":"2024-06-03T03:19:41.632731+0000","last_became_peered":"2024-06-03T03:19:41.632731+0000","last_unstale":"2024-06-03T03:21:26.053093+0000","last_undegraded":"2024-06-03T03:21:26.053093+0000","last_fullsized":"2024-06-03T03:21:26.053093+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:10:29.990256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":35,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.052257+0000","last_change":"2024-06-03T03:20:26.133763+0000","last_active":"2024-06-03T03:21:26.052257+0000","last_peered":"2024-06-03T03:21:26.052257+0000","last_clean":"2024-06-03T03:21:26.052257+0000","last_became_active":"2024-06-03T03:20:26.133252+0000","last_became_peered":"2024-06-03T03:20:26.133252+0000","last_unstale":"2024-06-03T03:21:26.052257+0000","last_undegraded":"2024-06-03T03:21:26.052257+0000","last_fullsized":"2024-06-03T03:21:26.052257+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:33:21.461038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"97'13","reported_seq":92,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:40.117927+0000","last_change":"2024-06-03T03:19:36.542545+0000","last_active":"2024-06-03T03:21:40.117927+0000","last_peered":"2024-06-03T03:21:40.117927+0000","last_clean":"2024-06-03T03:21:40.117927+0000","last_became_active":"2024-06-03T03:19:36.541682+0000","last_became_peered":"2024-06-03T03:19:36.541682+0000","last_unstale":"2024-06-03T03:21:40.117927+0000","last_undegraded":"2024-06-03T03:21:40.117927+0000","last_fullsized":"2024-06-03T03:21:40.117927+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:06:23.909253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":34,"num_read_kb":34,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.058378+0000","last_change":"2024-06-03T03:19:41.605848+0000","last_active":"2024-06-03T03:21:26.058378+0000","last_peered":"2024-06-03T03:21:26.058378+0000","last_clean":"2024-06-03T03:21:26.058378+0000","last_became_active":"2024-06-03T03:19:41.603131+0000","last_became_peered":"2024-06-03T03:19:41.603131+0000","last_unstale":"2024-06-03T03:21:26.058378+0000","last_undegraded":"2024-06-03T03:21:26.058378+0000","last_fullsized":"2024-06-03T03:21:26.058378+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:07:38.931055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.980716+0000","last_change":"2024-06-03T03:19:56.923180+0000","last_active":"2024-06-03T03:21:25.980716+0000","last_peered":"2024-06-03T03:21:25.980716+0000","last_clean":"2024-06-03T03:21:25.980716+0000","last_became_active":"2024-06-03T03:19:56.922726+0000","last_became_peered":"2024-06-03T03:19:56.922726+0000","last_unstale":"2024-06-03T03:21:25.980716+0000","last_undegraded":"2024-06-03T03:21:25.980716+0000","last_fullsized":"2024-06-03T03:21:25.980716+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:18:45.727680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"97'2","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.058990+0000","last_change":"2024-06-03T03:19:36.746805+0000","last_active":"2024-06-03T03:21:26.058990+0000","last_peered":"2024-06-03T03:21:26.058990+0000","last_clean":"2024-06-03T03:21:26.058990+0000","last_became_active":"2024-06-03T03:19:36.745747+0000","last_became_peered":"2024-06-03T03:19:36.745747+0000","last_unstale":"2024-06-03T03:21:26.058990+0000","last_undegraded":"2024-06-03T03:21:26.058990+0000","last_fullsized":"2024-06-03T03:21:26.058990+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:54:28.427708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.976822+0000","last_change":"2024-06-03T03:19:41.630582+0000","last_active":"2024-06-03T03:21:25.976822+0000","last_peered":"2024-06-03T03:21:25.976822+0000","last_clean":"2024-06-03T03:21:25.976822+0000","last_became_active":"2024-06-03T03:19:41.630238+0000","last_became_peered":"2024-06-03T03:19:41.630238+0000","last_unstale":"2024-06-03T03:21:25.976822+0000","last_undegraded":"2024-06-03T03:21:25.976822+0000","last_fullsized":"2024-06-03T03:21:25.976822+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:49:58.039697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.980805+0000","last_change":"2024-06-03T03:19:56.925142+0000","last_active":"2024-06-03T03:21:25.980805+0000","last_peered":"2024-06-03T03:21:25.980805+0000","last_clean":"2024-06-03T03:21:25.980805+0000","last_became_active":"2024-06-03T03:19:56.924760+0000","last_became_peered":"2024-06-03T03:19:56.924760+0000","last_unstale":"2024-06-03T03:21:25.980805+0000","last_undegraded":"2024-06-03T03:21:25.980805+0000","last_fullsized":"2024-06-03T03:21:25.980805+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:11:27.537077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"97'12","reported_seq":56,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.974042+0000","last_change":"2024-06-03T03:19:36.543344+0000","last_active":"2024-06-03T03:21:25.974042+0000","last_peered":"2024-06-03T03:21:25.974042+0000","last_clean":"2024-06-03T03:21:25.974042+0000","last_became_active":"2024-06-03T03:19:36.542837+0000","last_became_peered":"2024-06-03T03:19:36.542837+0000","last_unstale":"2024-06-03T03:21:25.974042+0000","last_undegraded":"2024-06-03T03:21:25.974042+0000","last_fullsized":"2024-06-03T03:21:25.974042+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:55:05.451521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.053193+0000","last_change":"2024-06-03T03:19:41.629820+0000","last_active":"2024-06-03T03:21:26.053193+0000","last_peered":"2024-06-03T03:21:26.053193+0000","last_clean":"2024-06-03T03:21:26.053193+0000","last_became_active":"2024-06-03T03:19:41.629439+0000","last_became_peered":"2024-06-03T03:19:41.629439+0000","last_unstale":"2024-06-03T03:21:26.053193+0000","last_undegraded":"2024-06-03T03:21:26.053193+0000","last_fullsized":"2024-06-03T03:21:26.053193+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:05:27.438308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.980716+0000","last_change":"2024-06-03T03:19:56.923214+0000","last_active":"2024-06-03T03:21:25.980716+0000","last_peered":"2024-06-03T03:21:25.980716+0000","last_clean":"2024-06-03T03:21:25.980716+0000","last_became_active":"2024-06-03T03:19:56.922751+0000","last_became_peered":"2024-06-03T03:19:56.922751+0000","last_unstale":"2024-06-03T03:21:25.980716+0000","last_undegraded":"2024-06-03T03:21:25.980716+0000","last_fullsized":"2024-06-03T03:21:25.980716+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:05:59.249818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"97'5","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.973842+0000","last_change":"2024-06-03T03:19:36.538141+0000","last_active":"2024-06-03T03:21:25.973842+0000","last_peered":"2024-06-03T03:21:25.973842+0000","last_clean":"2024-06-03T03:21:25.973842+0000","last_became_active":"2024-06-03T03:19:36.537786+0000","last_became_peered":"2024-06-03T03:19:36.537786+0000","last_unstale":"2024-06-03T03:21:25.973842+0000","last_undegraded":"2024-06-03T03:21:25.973842+0000","last_fullsized":"2024-06-03T03:21:25.973842+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T15:03:32.859097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.054195+0000","last_change":"2024-06-03T03:19:41.608471+0000","last_active":"2024-06-03T03:21:26.054195+0000","last_peered":"2024-06-03T03:21:26.054195+0000","last_clean":"2024-06-03T03:21:26.054195+0000","last_became_active":"2024-06-03T03:19:41.608096+0000","last_became_peered":"2024-06-03T03:19:41.608096+0000","last_unstale":"2024-06-03T03:21:26.054195+0000","last_undegraded":"2024-06-03T03:21:26.054195+0000","last_fullsized":"2024-06-03T03:21:26.054195+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:31:10.541803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.052967+0000","last_change":"2024-06-03T03:19:56.915540+0000","last_active":"2024-06-03T03:21:26.052967+0000","last_peered":"2024-06-03T03:21:26.052967+0000","last_clean":"2024-06-03T03:21:26.052967+0000","last_became_active":"2024-06-03T03:19:56.915017+0000","last_became_peered":"2024-06-03T03:19:56.915017+0000","last_unstale":"2024-06-03T03:21:26.052967+0000","last_undegraded":"2024-06-03T03:21:26.052967+0000","last_fullsized":"2024-06-03T03:21:26.052967+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:56:23.158344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"97'6","reported_seq":88,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:41.128496+0000","last_change":"2024-06-03T03:19:36.542337+0000","last_active":"2024-06-03T03:21:41.128496+0000","last_peered":"2024-06-03T03:21:41.128496+0000","last_clean":"2024-06-03T03:21:41.128496+0000","last_became_active":"2024-06-03T03:19:36.541990+0000","last_became_peered":"2024-06-03T03:19:36.541990+0000","last_unstale":"2024-06-03T03:21:41.128496+0000","last_undegraded":"2024-06-03T03:21:41.128496+0000","last_fullsized":"2024-06-03T03:21:41.128496+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:08:15.656321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":38,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.973454+0000","last_change":"2024-06-03T03:19:41.616293+0000","last_active":"2024-06-03T03:21:25.973454+0000","last_peered":"2024-06-03T03:21:25.973454+0000","last_clean":"2024-06-03T03:21:25.973454+0000","last_became_active":"2024-06-03T03:19:41.615946+0000","last_became_peered":"2024-06-03T03:19:41.615946+0000","last_unstale":"2024-06-03T03:21:25.973454+0000","last_undegraded":"2024-06-03T03:21:25.973454+0000","last_fullsized":"2024-06-03T03:21:25.973454+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:18:43.835977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":35,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.053575+0000","last_change":"2024-06-03T03:20:26.146237+0000","last_active":"2024-06-03T03:21:26.053575+0000","last_peered":"2024-06-03T03:21:26.053575+0000","last_clean":"2024-06-03T03:21:26.053575+0000","last_became_active":"2024-06-03T03:20:26.145731+0000","last_became_peered":"2024-06-03T03:20:26.145731+0000","last_unstale":"2024-06-03T03:21:26.053575+0000","last_undegraded":"2024-06-03T03:21:26.053575+0000","last_fullsized":"2024-06-03T03:21:26.053575+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:15:29.723833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"97'7","reported_seq":51,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.053492+0000","last_change":"2024-06-03T03:19:36.546501+0000","last_active":"2024-06-03T03:21:26.053492+0000","last_peered":"2024-06-03T03:21:26.053492+0000","last_clean":"2024-06-03T03:21:26.053492+0000","last_became_active":"2024-06-03T03:19:36.546079+0000","last_became_peered":"2024-06-03T03:19:36.546079+0000","last_unstale":"2024-06-03T03:21:26.053492+0000","last_undegraded":"2024-06-03T03:21:26.053492+0000","last_fullsized":"2024-06-03T03:21:26.053492+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:16:39.473141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.054080+0000","last_change":"2024-06-03T03:20:26.149410+0000","last_active":"2024-06-03T03:21:26.054080+0000","last_peered":"2024-06-03T03:21:26.054080+0000","last_clean":"2024-06-03T03:21:26.054080+0000","last_became_active":"2024-06-03T03:20:26.149084+0000","last_became_peered":"2024-06-03T03:20:26.149084+0000","last_unstale":"2024-06-03T03:21:26.054080+0000","last_undegraded":"2024-06-03T03:21:26.054080+0000","last_fullsized":"2024-06-03T03:21:26.054080+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:06:22.316449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.054121+0000","last_change":"2024-06-03T03:19:56.926498+0000","last_active":"2024-06-03T03:21:26.054121+0000","last_peered":"2024-06-03T03:21:26.054121+0000","last_clean":"2024-06-03T03:21:26.054121+0000","last_became_active":"2024-06-03T03:19:56.926101+0000","last_became_peered":"2024-06-03T03:19:56.926101+0000","last_unstale":"2024-06-03T03:21:26.054121+0000","last_undegraded":"2024-06-03T03:21:26.054121+0000","last_fullsized":"2024-06-03T03:21:26.054121+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:00:12.024443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"97'3","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.977250+0000","last_change":"2024-06-03T03:19:36.553151+0000","last_active":"2024-06-03T03:21:25.977250+0000","last_peered":"2024-06-03T03:21:25.977250+0000","last_clean":"2024-06-03T03:21:25.977250+0000","last_became_active":"2024-06-03T03:19:36.551914+0000","last_became_peered":"2024-06-03T03:19:36.551914+0000","last_unstale":"2024-06-03T03:21:25.977250+0000","last_undegraded":"2024-06-03T03:21:25.977250+0000","last_fullsized":"2024-06-03T03:21:25.977250+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:00:42.053922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.054306+0000","last_change":"2024-06-03T03:20:26.147909+0000","last_active":"2024-06-03T03:21:26.054306+0000","last_peered":"2024-06-03T03:21:26.054306+0000","last_clean":"2024-06-03T03:21:26.054306+0000","last_became_active":"2024-06-03T03:20:26.146448+0000","last_became_peered":"2024-06-03T03:20:26.146448+0000","last_unstale":"2024-06-03T03:21:26.054306+0000","last_undegraded":"2024-06-03T03:21:26.054306+0000","last_fullsized":"2024-06-03T03:21:26.054306+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:43:30.878472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.976566+0000","last_change":"2024-06-03T03:19:56.941910+0000","last_active":"2024-06-03T03:21:25.976566+0000","last_peered":"2024-06-03T03:21:25.976566+0000","last_clean":"2024-06-03T03:21:25.976566+0000","last_became_active":"2024-06-03T03:19:56.940963+0000","last_became_peered":"2024-06-03T03:19:56.940963+0000","last_unstale":"2024-06-03T03:21:25.976566+0000","last_undegraded":"2024-06-03T03:21:25.976566+0000","last_fullsized":"2024-06-03T03:21:25.976566+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T14:18:32.109231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"97'3","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.052423+0000","last_change":"2024-06-03T03:19:36.538928+0000","last_active":"2024-06-03T03:21:26.052423+0000","last_peered":"2024-06-03T03:21:26.052423+0000","last_clean":"2024-06-03T03:21:26.052423+0000","last_became_active":"2024-06-03T03:19:36.538580+0000","last_became_peered":"2024-06-03T03:19:36.538580+0000","last_unstale":"2024-06-03T03:21:26.052423+0000","last_undegraded":"2024-06-03T03:21:26.052423+0000","last_fullsized":"2024-06-03T03:21:26.052423+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:08:05.906257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.054672+0000","last_change":"2024-06-03T03:19:41.623051+0000","last_active":"2024-06-03T03:21:26.054672+0000","last_peered":"2024-06-03T03:21:26.054672+0000","last_clean":"2024-06-03T03:21:26.054672+0000","last_became_active":"2024-06-03T03:19:41.622656+0000","last_became_peered":"2024-06-03T03:19:41.622656+0000","last_unstale":"2024-06-03T03:21:26.054672+0000","last_undegraded":"2024-06-03T03:21:26.054672+0000","last_fullsized":"2024-06-03T03:21:26.054672+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:11:34.587723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.056239+0000","last_change":"2024-06-03T03:19:56.921215+0000","last_active":"2024-06-03T03:21:26.056239+0000","last_peered":"2024-06-03T03:21:26.056239+0000","last_clean":"2024-06-03T03:21:26.056239+0000","last_became_active":"2024-06-03T03:19:56.920658+0000","last_became_peered":"2024-06-03T03:19:56.920658+0000","last_unstale":"2024-06-03T03:21:26.056239+0000","last_undegraded":"2024-06-03T03:21:26.056239+0000","last_fullsized":"2024-06-03T03:21:26.056239+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T14:04:54.163818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"97'4","reported_seq":116,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:41.112150+0000","last_change":"2024-06-03T03:19:36.545245+0000","last_active":"2024-06-03T03:21:41.112150+0000","last_peered":"2024-06-03T03:21:41.112150+0000","last_clean":"2024-06-03T03:21:41.112150+0000","last_became_active":"2024-06-03T03:19:36.544821+0000","last_became_peered":"2024-06-03T03:19:36.544821+0000","last_unstale":"2024-06-03T03:21:41.112150+0000","last_undegraded":"2024-06-03T03:21:41.112150+0000","last_fullsized":"2024-06-03T03:21:41.112150+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:55:24.011925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":68,"num_read_kb":68,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.053751+0000","last_change":"2024-06-03T03:19:41.616384+0000","last_active":"2024-06-03T03:21:26.053751+0000","last_peered":"2024-06-03T03:21:26.053751+0000","last_clean":"2024-06-03T03:21:26.053751+0000","last_became_active":"2024-06-03T03:19:41.616034+0000","last_became_peered":"2024-06-03T03:19:41.616034+0000","last_unstale":"2024-06-03T03:21:26.053751+0000","last_undegraded":"2024-06-03T03:21:26.053751+0000","last_fullsized":"2024-06-03T03:21:26.053751+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:10:06.823375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.054357+0000","last_change":"2024-06-03T03:19:56.933045+0000","last_active":"2024-06-03T03:21:26.054357+0000","last_peered":"2024-06-03T03:21:26.054357+0000","last_clean":"2024-06-03T03:21:26.054357+0000","last_became_active":"2024-06-03T03:19:56.928956+0000","last_became_peered":"2024-06-03T03:19:56.928956+0000","last_unstale":"2024-06-03T03:21:26.054357+0000","last_undegraded":"2024-06-03T03:21:26.054357+0000","last_fullsized":"2024-06-03T03:21:26.054357+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:37:44.270511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"97'4","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.053347+0000","last_change":"2024-06-03T03:19:36.541559+0000","last_active":"2024-06-03T03:21:26.053347+0000","last_peered":"2024-06-03T03:21:26.053347+0000","last_clean":"2024-06-03T03:21:26.053347+0000","last_became_active":"2024-06-03T03:19:36.541166+0000","last_became_peered":"2024-06-03T03:19:36.541166+0000","last_unstale":"2024-06-03T03:21:26.053347+0000","last_undegraded":"2024-06-03T03:21:26.053347+0000","last_fullsized":"2024-06-03T03:21:26.053347+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T14:49:26.847713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.977003+0000","last_change":"2024-06-03T03:19:41.629789+0000","last_active":"2024-06-03T03:21:25.977003+0000","last_peered":"2024-06-03T03:21:25.977003+0000","last_clean":"2024-06-03T03:21:25.977003+0000","last_became_active":"2024-06-03T03:19:41.628026+0000","last_became_peered":"2024-06-03T03:19:41.628026+0000","last_unstale":"2024-06-03T03:21:25.977003+0000","last_undegraded":"2024-06-03T03:21:25.977003+0000","last_fullsized":"2024-06-03T03:21:25.977003+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:56:26.411907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":35,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.056114+0000","last_change":"2024-06-03T03:21:26.056114+0000","last_active":"2024-06-03T03:21:26.056114+0000","last_peered":"2024-06-03T03:21:26.056114+0000","last_clean":"2024-06-03T03:21:26.056114+0000","last_became_active":"2024-06-03T03:21:26.055505+0000","last_became_peered":"2024-06-03T03:21:26.055505+0000","last_unstale":"2024-06-03T03:21:26.056114+0000","last_undegraded":"2024-06-03T03:21:26.056114+0000","last_fullsized":"2024-06-03T03:21:26.056114+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:22:55.722566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,10],"acting":[4,1,2,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"97'5","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.053488+0000","last_change":"2024-06-03T03:19:36.744930+0000","last_active":"2024-06-03T03:21:26.053488+0000","last_peered":"2024-06-03T03:21:26.053488+0000","last_clean":"2024-06-03T03:21:26.053488+0000","last_became_active":"2024-06-03T03:19:36.744567+0000","last_became_peered":"2024-06-03T03:19:36.744567+0000","last_unstale":"2024-06-03T03:21:26.053488+0000","last_undegraded":"2024-06-03T03:21:26.053488+0000","last_fullsized":"2024-06-03T03:21:26.053488+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:35:31.927211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.053330+0000","last_change":"2024-06-03T03:19:41.607604+0000","last_active":"2024-06-03T03:21:26.053330+0000","last_peered":"2024-06-03T03:21:26.053330+0000","last_clean":"2024-06-03T03:21:26.053330+0000","last_became_active":"2024-06-03T03:19:41.607148+0000","last_became_peered":"2024-06-03T03:19:41.607148+0000","last_unstale":"2024-06-03T03:21:26.053330+0000","last_undegraded":"2024-06-03T03:21:26.053330+0000","last_fullsized":"2024-06-03T03:21:26.053330+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:44:44.037836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.056102+0000","last_change":"2024-06-03T03:19:56.928689+0000","last_active":"2024-06-03T03:21:26.056102+0000","last_peered":"2024-06-03T03:21:26.056102+0000","last_clean":"2024-06-03T03:21:26.056102+0000","last_became_active":"2024-06-03T03:19:56.924955+0000","last_became_peered":"2024-06-03T03:19:56.924955+0000","last_unstale":"2024-06-03T03:21:26.056102+0000","last_undegraded":"2024-06-03T03:21:26.056102+0000","last_fullsized":"2024-06-03T03:21:26.056102+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:11:07.413430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"99'36","reported_seq":157,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:41.112189+0000","last_change":"2024-06-03T03:19:36.539688+0000","last_active":"2024-06-03T03:21:41.112189+0000","last_peered":"2024-06-03T03:21:41.112189+0000","last_clean":"2024-06-03T03:21:41.112189+0000","last_became_active":"2024-06-03T03:19:36.539302+0000","last_became_peered":"2024-06-03T03:19:36.539302+0000","last_unstale":"2024-06-03T03:21:41.112189+0000","last_undegraded":"2024-06-03T03:21:41.112189+0000","last_fullsized":"2024-06-03T03:21:41.112189+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:28:52.790227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63552,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":77,"num_read_kb":100,"num_write":38,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.973872+0000","last_change":"2024-06-03T03:19:41.608202+0000","last_active":"2024-06-03T03:21:25.973872+0000","last_peered":"2024-06-03T03:21:25.973872+0000","last_clean":"2024-06-03T03:21:25.973872+0000","last_became_active":"2024-06-03T03:19:41.607891+0000","last_became_peered":"2024-06-03T03:19:41.607891+0000","last_unstale":"2024-06-03T03:21:25.973872+0000","last_undegraded":"2024-06-03T03:21:25.973872+0000","last_fullsized":"2024-06-03T03:21:25.973872+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T14:10:44.042892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.054137+0000","last_change":"2024-06-03T03:19:56.923547+0000","last_active":"2024-06-03T03:21:26.054137+0000","last_peered":"2024-06-03T03:21:26.054137+0000","last_clean":"2024-06-03T03:21:26.054137+0000","last_became_active":"2024-06-03T03:19:56.923046+0000","last_became_peered":"2024-06-03T03:19:56.923046+0000","last_unstale":"2024-06-03T03:21:26.054137+0000","last_undegraded":"2024-06-03T03:21:26.054137+0000","last_fullsized":"2024-06-03T03:21:26.054137+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:32:19.438793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"97'9","reported_seq":54,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.053818+0000","last_change":"2024-06-03T03:19:36.547024+0000","last_active":"2024-06-03T03:21:26.053818+0000","last_peered":"2024-06-03T03:21:26.053818+0000","last_clean":"2024-06-03T03:21:26.053818+0000","last_became_active":"2024-06-03T03:19:36.546663+0000","last_became_peered":"2024-06-03T03:19:36.546663+0000","last_unstale":"2024-06-03T03:21:26.053818+0000","last_undegraded":"2024-06-03T03:21:26.053818+0000","last_fullsized":"2024-06-03T03:21:26.053818+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:58:41.446730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.977661+0000","last_change":"2024-06-03T03:19:41.630119+0000","last_active":"2024-06-03T03:21:25.977661+0000","last_peered":"2024-06-03T03:21:25.977661+0000","last_clean":"2024-06-03T03:21:25.977661+0000","last_became_active":"2024-06-03T03:19:41.629563+0000","last_became_peered":"2024-06-03T03:19:41.629563+0000","last_unstale":"2024-06-03T03:21:25.977661+0000","last_undegraded":"2024-06-03T03:21:25.977661+0000","last_fullsized":"2024-06-03T03:21:25.977661+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:49:33.983471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.053729+0000","last_change":"2024-06-03T03:19:56.941451+0000","last_active":"2024-06-03T03:21:26.053729+0000","last_peered":"2024-06-03T03:21:26.053729+0000","last_clean":"2024-06-03T03:21:26.053729+0000","last_became_active":"2024-06-03T03:19:56.941003+0000","last_became_peered":"2024-06-03T03:19:56.941003+0000","last_unstale":"2024-06-03T03:21:26.053729+0000","last_undegraded":"2024-06-03T03:21:26.053729+0000","last_fullsized":"2024-06-03T03:21:26.053729+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:28:58.505495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"97'5","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.053392+0000","last_change":"2024-06-03T03:19:36.547128+0000","last_active":"2024-06-03T03:21:26.053392+0000","last_peered":"2024-06-03T03:21:26.053392+0000","last_clean":"2024-06-03T03:21:26.053392+0000","last_became_active":"2024-06-03T03:19:36.546576+0000","last_became_peered":"2024-06-03T03:19:36.546576+0000","last_unstale":"2024-06-03T03:21:26.053392+0000","last_undegraded":"2024-06-03T03:21:26.053392+0000","last_fullsized":"2024-06-03T03:21:26.053392+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:23:30.196543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.052379+0000","last_change":"2024-06-03T03:19:41.634941+0000","last_active":"2024-06-03T03:21:26.052379+0000","last_peered":"2024-06-03T03:21:26.052379+0000","last_clean":"2024-06-03T03:21:26.052379+0000","last_became_active":"2024-06-03T03:19:41.634561+0000","last_became_peered":"2024-06-03T03:19:41.634561+0000","last_unstale":"2024-06-03T03:21:26.052379+0000","last_undegraded":"2024-06-03T03:21:26.052379+0000","last_fullsized":"2024-06-03T03:21:26.052379+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:00:27.258078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.056606+0000","last_change":"2024-06-03T03:19:56.934612+0000","last_active":"2024-06-03T03:21:26.056606+0000","last_peered":"2024-06-03T03:21:26.056606+0000","last_clean":"2024-06-03T03:21:26.056606+0000","last_became_active":"2024-06-03T03:19:56.930212+0000","last_became_peered":"2024-06-03T03:19:56.930212+0000","last_unstale":"2024-06-03T03:21:26.056606+0000","last_undegraded":"2024-06-03T03:21:26.056606+0000","last_fullsized":"2024-06-03T03:21:26.056606+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:42:38.490095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"97'5","reported_seq":85,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:41.119757+0000","last_change":"2024-06-03T03:19:36.542052+0000","last_active":"2024-06-03T03:21:41.119757+0000","last_peered":"2024-06-03T03:21:41.119757+0000","last_clean":"2024-06-03T03:21:41.119757+0000","last_became_active":"2024-06-03T03:19:36.540987+0000","last_became_peered":"2024-06-03T03:19:36.540987+0000","last_unstale":"2024-06-03T03:21:41.119757+0000","last_undegraded":"2024-06-03T03:21:41.119757+0000","last_fullsized":"2024-06-03T03:21:41.119757+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T15:18:48.821400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":38,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.052494+0000","last_change":"2024-06-03T03:19:41.607610+0000","last_active":"2024-06-03T03:21:26.052494+0000","last_peered":"2024-06-03T03:21:26.052494+0000","last_clean":"2024-06-03T03:21:26.052494+0000","last_became_active":"2024-06-03T03:19:41.607219+0000","last_became_peered":"2024-06-03T03:19:41.607219+0000","last_unstale":"2024-06-03T03:21:26.052494+0000","last_undegraded":"2024-06-03T03:21:26.052494+0000","last_fullsized":"2024-06-03T03:21:26.052494+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:06:20.707623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.979811+0000","last_change":"2024-06-03T03:19:56.936440+0000","last_active":"2024-06-03T03:21:25.979811+0000","last_peered":"2024-06-03T03:21:25.979811+0000","last_clean":"2024-06-03T03:21:25.979811+0000","last_became_active":"2024-06-03T03:19:56.936048+0000","last_became_peered":"2024-06-03T03:19:56.936048+0000","last_unstale":"2024-06-03T03:21:25.979811+0000","last_undegraded":"2024-06-03T03:21:25.979811+0000","last_fullsized":"2024-06-03T03:21:25.979811+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:22:15.758950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"97'17","reported_seq":61,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.973311+0000","last_change":"2024-06-03T03:19:36.736105+0000","last_active":"2024-06-03T03:21:25.973311+0000","last_peered":"2024-06-03T03:21:25.973311+0000","last_clean":"2024-06-03T03:21:25.973311+0000","last_became_active":"2024-06-03T03:19:36.735663+0000","last_became_peered":"2024-06-03T03:19:36.735663+0000","last_unstale":"2024-06-03T03:21:25.973311+0000","last_undegraded":"2024-06-03T03:21:25.973311+0000","last_fullsized":"2024-06-03T03:21:25.973311+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:41:46.216923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.052904+0000","last_change":"2024-06-03T03:19:41.609758+0000","last_active":"2024-06-03T03:21:26.052904+0000","last_peered":"2024-06-03T03:21:26.052904+0000","last_clean":"2024-06-03T03:21:26.052904+0000","last_became_active":"2024-06-03T03:19:41.609421+0000","last_became_peered":"2024-06-03T03:19:41.609421+0000","last_unstale":"2024-06-03T03:21:26.052904+0000","last_undegraded":"2024-06-03T03:21:26.052904+0000","last_fullsized":"2024-06-03T03:21:26.052904+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:45:54.919352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.052780+0000","last_change":"2024-06-03T03:19:56.925422+0000","last_active":"2024-06-03T03:21:26.052780+0000","last_peered":"2024-06-03T03:21:26.052780+0000","last_clean":"2024-06-03T03:21:26.052780+0000","last_became_active":"2024-06-03T03:19:56.924891+0000","last_became_peered":"2024-06-03T03:19:56.924891+0000","last_unstale":"2024-06-03T03:21:26.052780+0000","last_undegraded":"2024-06-03T03:21:26.052780+0000","last_fullsized":"2024-06-03T03:21:26.052780+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:34:59.986256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"97'2","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.973873+0000","last_change":"2024-06-03T03:19:36.540605+0000","last_active":"2024-06-03T03:21:25.973873+0000","last_peered":"2024-06-03T03:21:25.973873+0000","last_clean":"2024-06-03T03:21:25.973873+0000","last_became_active":"2024-06-03T03:19:36.540180+0000","last_became_peered":"2024-06-03T03:19:36.540180+0000","last_unstale":"2024-06-03T03:21:25.973873+0000","last_undegraded":"2024-06-03T03:21:25.973873+0000","last_fullsized":"2024-06-03T03:21:25.973873+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:01:26.556726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":43,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.053484+0000","last_change":"2024-06-03T03:20:25.904215+0000","last_active":"2024-06-03T03:21:26.053484+0000","last_peered":"2024-06-03T03:21:26.053484+0000","last_clean":"2024-06-03T03:21:26.053484+0000","last_became_active":"2024-06-03T03:20:25.903894+0000","last_became_peered":"2024-06-03T03:20:25.903894+0000","last_unstale":"2024-06-03T03:21:26.053484+0000","last_undegraded":"2024-06-03T03:21:26.053484+0000","last_fullsized":"2024-06-03T03:21:26.053484+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:02:07.587159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.973461+0000","last_change":"2024-06-03T03:19:56.910041+0000","last_active":"2024-06-03T03:21:25.973461+0000","last_peered":"2024-06-03T03:21:25.973461+0000","last_clean":"2024-06-03T03:21:25.973461+0000","last_became_active":"2024-06-03T03:19:56.909591+0000","last_became_peered":"2024-06-03T03:19:56.909591+0000","last_unstale":"2024-06-03T03:21:25.973461+0000","last_undegraded":"2024-06-03T03:21:25.973461+0000","last_fullsized":"2024-06-03T03:21:25.973461+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:42:20.016838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"97'1","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.055897+0000","last_change":"2024-06-03T03:19:36.741731+0000","last_active":"2024-06-03T03:21:26.055897+0000","last_peered":"2024-06-03T03:21:26.055897+0000","last_clean":"2024-06-03T03:21:26.055897+0000","last_became_active":"2024-06-03T03:19:36.741329+0000","last_became_peered":"2024-06-03T03:19:36.741329+0000","last_unstale":"2024-06-03T03:21:26.055897+0000","last_undegraded":"2024-06-03T03:21:26.055897+0000","last_fullsized":"2024-06-03T03:21:26.055897+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:02:45.778368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.053802+0000","last_change":"2024-06-03T03:19:41.632225+0000","last_active":"2024-06-03T03:21:26.053802+0000","last_peered":"2024-06-03T03:21:26.053802+0000","last_clean":"2024-06-03T03:21:26.053802+0000","last_became_active":"2024-06-03T03:19:41.631851+0000","last_became_peered":"2024-06-03T03:19:41.631851+0000","last_unstale":"2024-06-03T03:21:26.053802+0000","last_undegraded":"2024-06-03T03:21:26.053802+0000","last_fullsized":"2024-06-03T03:21:26.053802+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:52:23.364233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.977827+0000","last_change":"2024-06-03T03:19:56.934200+0000","last_active":"2024-06-03T03:21:25.977827+0000","last_peered":"2024-06-03T03:21:25.977827+0000","last_clean":"2024-06-03T03:21:25.977827+0000","last_became_active":"2024-06-03T03:19:56.932998+0000","last_became_peered":"2024-06-03T03:19:56.932998+0000","last_unstale":"2024-06-03T03:21:25.977827+0000","last_undegraded":"2024-06-03T03:21:25.977827+0000","last_fullsized":"2024-06-03T03:21:25.977827+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:45:28.916023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"97'5","reported_seq":88,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:44.124002+0000","last_change":"2024-06-03T03:19:36.546630+0000","last_active":"2024-06-03T03:21:44.124002+0000","last_peered":"2024-06-03T03:21:44.124002+0000","last_clean":"2024-06-03T03:21:44.124002+0000","last_became_active":"2024-06-03T03:19:36.546248+0000","last_became_peered":"2024-06-03T03:19:36.546248+0000","last_unstale":"2024-06-03T03:21:44.124002+0000","last_undegraded":"2024-06-03T03:21:44.124002+0000","last_fullsized":"2024-06-03T03:21:44.124002+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T15:09:50.323113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":38,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.054331+0000","last_change":"2024-06-03T03:19:41.624584+0000","last_active":"2024-06-03T03:21:26.054331+0000","last_peered":"2024-06-03T03:21:26.054331+0000","last_clean":"2024-06-03T03:21:26.054331+0000","last_became_active":"2024-06-03T03:19:41.624216+0000","last_became_peered":"2024-06-03T03:19:41.624216+0000","last_unstale":"2024-06-03T03:21:26.054331+0000","last_undegraded":"2024-06-03T03:21:26.054331+0000","last_fullsized":"2024-06-03T03:21:26.054331+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:22:09.645341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.979831+0000","last_change":"2024-06-03T03:19:56.937422+0000","last_active":"2024-06-03T03:21:25.979831+0000","last_peered":"2024-06-03T03:21:25.979831+0000","last_clean":"2024-06-03T03:21:25.979831+0000","last_became_active":"2024-06-03T03:19:56.934075+0000","last_became_peered":"2024-06-03T03:19:56.934075+0000","last_unstale":"2024-06-03T03:21:25.979831+0000","last_undegraded":"2024-06-03T03:21:25.979831+0000","last_fullsized":"2024-06-03T03:21:25.979831+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:25:45.273161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.052773+0000","last_change":"2024-06-03T03:19:41.589595+0000","last_active":"2024-06-03T03:21:26.052773+0000","last_peered":"2024-06-03T03:21:26.052773+0000","last_clean":"2024-06-03T03:21:26.052773+0000","last_became_active":"2024-06-03T03:19:41.589251+0000","last_became_peered":"2024-06-03T03:19:41.589251+0000","last_unstale":"2024-06-03T03:21:26.052773+0000","last_undegraded":"2024-06-03T03:21:26.052773+0000","last_fullsized":"2024-06-03T03:21:26.052773+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:51:40.417386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"97'5","reported_seq":86,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:43.121660+0000","last_change":"2024-06-03T03:19:36.737523+0000","last_active":"2024-06-03T03:21:43.121660+0000","last_peered":"2024-06-03T03:21:43.121660+0000","last_clean":"2024-06-03T03:21:43.121660+0000","last_became_active":"2024-06-03T03:19:36.736698+0000","last_became_peered":"2024-06-03T03:19:36.736698+0000","last_unstale":"2024-06-03T03:21:43.121660+0000","last_undegraded":"2024-06-03T03:21:43.121660+0000","last_fullsized":"2024-06-03T03:21:43.121660+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:14:46.330549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":37,"num_read_kb":37,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"97'1","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.052754+0000","last_change":"2024-06-03T03:19:56.924421+0000","last_active":"2024-06-03T03:21:26.052754+0000","last_peered":"2024-06-03T03:21:26.052754+0000","last_clean":"2024-06-03T03:21:26.052754+0000","last_became_active":"2024-06-03T03:19:56.924043+0000","last_became_peered":"2024-06-03T03:19:56.924043+0000","last_unstale":"2024-06-03T03:21:26.052754+0000","last_undegraded":"2024-06-03T03:21:26.052754+0000","last_fullsized":"2024-06-03T03:21:26.052754+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:00:47.674717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.055463+0000","last_change":"2024-06-03T03:19:41.613800+0000","last_active":"2024-06-03T03:21:26.055463+0000","last_peered":"2024-06-03T03:21:26.055463+0000","last_clean":"2024-06-03T03:21:26.055463+0000","last_became_active":"2024-06-03T03:19:41.613387+0000","last_became_peered":"2024-06-03T03:19:41.613387+0000","last_unstale":"2024-06-03T03:21:26.055463+0000","last_undegraded":"2024-06-03T03:21:26.055463+0000","last_fullsized":"2024-06-03T03:21:26.055463+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:04:04.785023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"97'6","reported_seq":53,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.980987+0000","last_change":"2024-06-03T03:19:36.544518+0000","last_active":"2024-06-03T03:21:25.980987+0000","last_peered":"2024-06-03T03:21:25.980987+0000","last_clean":"2024-06-03T03:21:25.980987+0000","last_became_active":"2024-06-03T03:19:36.544056+0000","last_became_peered":"2024-06-03T03:19:36.544056+0000","last_unstale":"2024-06-03T03:21:25.980987+0000","last_undegraded":"2024-06-03T03:21:25.980987+0000","last_fullsized":"2024-06-03T03:21:25.980987+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:33:25.012986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":366,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.052263+0000","last_change":"2024-06-03T03:18:05.481351+0000","last_active":"2024-06-03T03:21:26.052263+0000","last_peered":"2024-06-03T03:21:26.052263+0000","last_clean":"2024-06-03T03:21:26.052263+0000","last_became_active":"2024-06-03T03:18:05.480929+0000","last_became_peered":"2024-06-03T03:18:05.480929+0000","last_unstale":"2024-06-03T03:21:26.052263+0000","last_undegraded":"2024-06-03T03:21:26.052263+0000","last_fullsized":"2024-06-03T03:21:26.052263+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:14:09.558597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:14:09.558597+0000","last_clean_scrub_stamp":"2024-06-03T03:14:09.558597+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:54:45.563733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.054294+0000","last_change":"2024-06-03T03:19:56.931867+0000","last_active":"2024-06-03T03:21:26.054294+0000","last_peered":"2024-06-03T03:21:26.054294+0000","last_clean":"2024-06-03T03:21:26.054294+0000","last_became_active":"2024-06-03T03:19:56.931538+0000","last_became_peered":"2024-06-03T03:19:56.931538+0000","last_unstale":"2024-06-03T03:21:26.054294+0000","last_undegraded":"2024-06-03T03:21:26.054294+0000","last_fullsized":"2024-06-03T03:21:26.054294+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:55:15.144192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.976634+0000","last_change":"2024-06-03T03:19:41.622713+0000","last_active":"2024-06-03T03:21:25.976634+0000","last_peered":"2024-06-03T03:21:25.976634+0000","last_clean":"2024-06-03T03:21:25.976634+0000","last_became_active":"2024-06-03T03:19:41.622363+0000","last_became_peered":"2024-06-03T03:19:41.622363+0000","last_unstale":"2024-06-03T03:21:25.976634+0000","last_undegraded":"2024-06-03T03:21:25.976634+0000","last_fullsized":"2024-06-03T03:21:25.976634+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:43:57.671797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"97'4","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.052450+0000","last_change":"2024-06-03T03:19:36.539663+0000","last_active":"2024-06-03T03:21:26.052450+0000","last_peered":"2024-06-03T03:21:26.052450+0000","last_clean":"2024-06-03T03:21:26.052450+0000","last_became_active":"2024-06-03T03:19:36.539320+0000","last_became_peered":"2024-06-03T03:19:36.539320+0000","last_unstale":"2024-06-03T03:21:26.052450+0000","last_undegraded":"2024-06-03T03:21:26.052450+0000","last_fullsized":"2024-06-03T03:21:26.052450+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:36:50.699231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.053178+0000","last_change":"2024-06-03T03:19:56.938231+0000","last_active":"2024-06-03T03:21:26.053178+0000","last_peered":"2024-06-03T03:21:26.053178+0000","last_clean":"2024-06-03T03:21:26.053178+0000","last_became_active":"2024-06-03T03:19:56.937840+0000","last_became_peered":"2024-06-03T03:19:56.937840+0000","last_unstale":"2024-06-03T03:21:26.053178+0000","last_undegraded":"2024-06-03T03:21:26.053178+0000","last_fullsized":"2024-06-03T03:21:26.053178+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:02:33.916980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"97'4","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.973778+0000","last_change":"2024-06-03T03:19:36.737519+0000","last_active":"2024-06-03T03:21:25.973778+0000","last_peered":"2024-06-03T03:21:25.973778+0000","last_clean":"2024-06-03T03:21:25.973778+0000","last_became_active":"2024-06-03T03:19:36.737191+0000","last_became_peered":"2024-06-03T03:19:36.737191+0000","last_unstale":"2024-06-03T03:21:25.973778+0000","last_undegraded":"2024-06-03T03:21:25.973778+0000","last_fullsized":"2024-06-03T03:21:25.973778+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:50:21.389739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.978513+0000","last_change":"2024-06-03T03:19:41.633113+0000","last_active":"2024-06-03T03:21:25.978513+0000","last_peered":"2024-06-03T03:21:25.978513+0000","last_clean":"2024-06-03T03:21:25.978513+0000","last_became_active":"2024-06-03T03:19:41.632186+0000","last_became_peered":"2024-06-03T03:19:41.632186+0000","last_unstale":"2024-06-03T03:21:25.978513+0000","last_undegraded":"2024-06-03T03:21:25.978513+0000","last_fullsized":"2024-06-03T03:21:25.978513+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T15:12:26.835061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.056988+0000","last_change":"2024-06-03T03:19:56.928603+0000","last_active":"2024-06-03T03:21:26.056988+0000","last_peered":"2024-06-03T03:21:26.056988+0000","last_clean":"2024-06-03T03:21:26.056988+0000","last_became_active":"2024-06-03T03:19:56.928175+0000","last_became_peered":"2024-06-03T03:19:56.928175+0000","last_unstale":"2024-06-03T03:21:26.056988+0000","last_undegraded":"2024-06-03T03:21:26.056988+0000","last_fullsized":"2024-06-03T03:21:26.056988+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:26:03.531091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"99'18","reported_seq":62,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:35.647627+0000","last_change":"2024-06-03T03:19:36.550577+0000","last_active":"2024-06-03T03:21:35.647627+0000","last_peered":"2024-06-03T03:21:35.647627+0000","last_clean":"2024-06-03T03:21:35.647627+0000","last_became_active":"2024-06-03T03:19:36.549968+0000","last_became_peered":"2024-06-03T03:19:36.549968+0000","last_unstale":"2024-06-03T03:21:35.647627+0000","last_undegraded":"2024-06-03T03:21:35.647627+0000","last_fullsized":"2024-06-03T03:21:35.647627+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:22:43.892002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.058728+0000","last_change":"2024-06-03T03:19:41.613883+0000","last_active":"2024-06-03T03:21:26.058728+0000","last_peered":"2024-06-03T03:21:26.058728+0000","last_clean":"2024-06-03T03:21:26.058728+0000","last_became_active":"2024-06-03T03:19:41.613472+0000","last_became_peered":"2024-06-03T03:19:41.613472+0000","last_unstale":"2024-06-03T03:21:26.058728+0000","last_undegraded":"2024-06-03T03:21:26.058728+0000","last_fullsized":"2024-06-03T03:21:26.058728+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T14:38:42.044534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.052192+0000","last_change":"2024-06-03T03:19:56.914487+0000","last_active":"2024-06-03T03:21:26.052192+0000","last_peered":"2024-06-03T03:21:26.052192+0000","last_clean":"2024-06-03T03:21:26.052192+0000","last_became_active":"2024-06-03T03:19:56.913904+0000","last_became_peered":"2024-06-03T03:19:56.913904+0000","last_unstale":"2024-06-03T03:21:26.052192+0000","last_undegraded":"2024-06-03T03:21:26.052192+0000","last_fullsized":"2024-06-03T03:21:26.052192+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T15:13:24.084034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"97'7","reported_seq":51,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.981292+0000","last_change":"2024-06-03T03:19:36.671355+0000","last_active":"2024-06-03T03:21:25.981292+0000","last_peered":"2024-06-03T03:21:25.981292+0000","last_clean":"2024-06-03T03:21:25.981292+0000","last_became_active":"2024-06-03T03:19:36.670926+0000","last_became_peered":"2024-06-03T03:19:36.670926+0000","last_unstale":"2024-06-03T03:21:25.981292+0000","last_undegraded":"2024-06-03T03:21:25.981292+0000","last_fullsized":"2024-06-03T03:21:25.981292+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:38:50.914238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.976850+0000","last_change":"2024-06-03T03:19:41.629054+0000","last_active":"2024-06-03T03:21:25.976850+0000","last_peered":"2024-06-03T03:21:25.976850+0000","last_clean":"2024-06-03T03:21:25.976850+0000","last_became_active":"2024-06-03T03:19:41.627118+0000","last_became_peered":"2024-06-03T03:19:41.627118+0000","last_unstale":"2024-06-03T03:21:25.976850+0000","last_undegraded":"2024-06-03T03:21:25.976850+0000","last_fullsized":"2024-06-03T03:21:25.976850+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:59:40.645002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":35,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.058741+0000","last_change":"2024-06-03T03:21:26.058741+0000","last_active":"2024-06-03T03:21:26.058741+0000","last_peered":"2024-06-03T03:21:26.058741+0000","last_clean":"2024-06-03T03:21:26.058741+0000","last_became_active":"2024-06-03T03:21:26.058271+0000","last_became_peered":"2024-06-03T03:21:26.058271+0000","last_unstale":"2024-06-03T03:21:26.058741+0000","last_undegraded":"2024-06-03T03:21:26.058741+0000","last_fullsized":"2024-06-03T03:21:26.058741+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:31:32.359893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"97'2","reported_seq":52,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.052687+0000","last_change":"2024-06-03T03:20:26.132994+0000","last_active":"2024-06-03T03:21:26.052687+0000","last_peered":"2024-06-03T03:21:26.052687+0000","last_clean":"2024-06-03T03:21:26.052687+0000","last_became_active":"2024-06-03T03:20:26.132574+0000","last_became_peered":"2024-06-03T03:20:26.132574+0000","last_unstale":"2024-06-03T03:21:26.052687+0000","last_undegraded":"2024-06-03T03:21:26.052687+0000","last_fullsized":"2024-06-03T03:21:26.052687+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T15:12:33.599154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.973316+0000","last_change":"2024-06-03T03:19:41.603630+0000","last_active":"2024-06-03T03:21:25.973316+0000","last_peered":"2024-06-03T03:21:25.973316+0000","last_clean":"2024-06-03T03:21:25.973316+0000","last_became_active":"2024-06-03T03:19:41.603301+0000","last_became_peered":"2024-06-03T03:19:41.603301+0000","last_unstale":"2024-06-03T03:21:25.973316+0000","last_undegraded":"2024-06-03T03:21:25.973316+0000","last_fullsized":"2024-06-03T03:21:25.973316+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:20:33.076622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.057458+0000","last_change":"2024-06-03T03:19:56.929023+0000","last_active":"2024-06-03T03:21:26.057458+0000","last_peered":"2024-06-03T03:21:26.057458+0000","last_clean":"2024-06-03T03:21:26.057458+0000","last_became_active":"2024-06-03T03:19:56.928454+0000","last_became_peered":"2024-06-03T03:19:56.928454+0000","last_unstale":"2024-06-03T03:21:26.057458+0000","last_undegraded":"2024-06-03T03:21:26.057458+0000","last_fullsized":"2024-06-03T03:21:26.057458+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:17:31.755649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"97'7","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.056747+0000","last_change":"2024-06-03T03:19:36.551413+0000","last_active":"2024-06-03T03:21:26.056747+0000","last_peered":"2024-06-03T03:21:26.056747+0000","last_clean":"2024-06-03T03:21:26.056747+0000","last_became_active":"2024-06-03T03:19:36.550622+0000","last_became_peered":"2024-06-03T03:19:36.550622+0000","last_unstale":"2024-06-03T03:21:26.056747+0000","last_undegraded":"2024-06-03T03:21:26.056747+0000","last_fullsized":"2024-06-03T03:21:26.056747+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:50:09.472960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.053813+0000","last_change":"2024-06-03T03:19:41.624871+0000","last_active":"2024-06-03T03:21:26.053813+0000","last_peered":"2024-06-03T03:21:26.053813+0000","last_clean":"2024-06-03T03:21:26.053813+0000","last_became_active":"2024-06-03T03:19:41.624638+0000","last_became_peered":"2024-06-03T03:19:41.624638+0000","last_unstale":"2024-06-03T03:21:26.053813+0000","last_undegraded":"2024-06-03T03:21:26.053813+0000","last_fullsized":"2024-06-03T03:21:26.053813+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:21:16.193817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"99'1","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.981349+0000","last_change":"2024-06-03T03:19:56.924638+0000","last_active":"2024-06-03T03:21:25.981349+0000","last_peered":"2024-06-03T03:21:25.981349+0000","last_clean":"2024-06-03T03:21:25.981349+0000","last_became_active":"2024-06-03T03:19:56.924200+0000","last_became_peered":"2024-06-03T03:19:56.924200+0000","last_unstale":"2024-06-03T03:21:25.981349+0000","last_undegraded":"2024-06-03T03:21:25.981349+0000","last_fullsized":"2024-06-03T03:21:25.981349+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:38:01.138253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"97'3","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.052654+0000","last_change":"2024-06-03T03:19:36.546974+0000","last_active":"2024-06-03T03:21:26.052654+0000","last_peered":"2024-06-03T03:21:26.052654+0000","last_clean":"2024-06-03T03:21:26.052654+0000","last_became_active":"2024-06-03T03:19:36.546433+0000","last_became_peered":"2024-06-03T03:19:36.546433+0000","last_unstale":"2024-06-03T03:21:26.052654+0000","last_undegraded":"2024-06-03T03:21:26.052654+0000","last_fullsized":"2024-06-03T03:21:26.052654+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:41:46.980327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.055294+0000","last_change":"2024-06-03T03:19:41.622118+0000","last_active":"2024-06-03T03:21:26.055294+0000","last_peered":"2024-06-03T03:21:26.055294+0000","last_clean":"2024-06-03T03:21:26.055294+0000","last_became_active":"2024-06-03T03:19:41.620789+0000","last_became_peered":"2024-06-03T03:19:41.620789+0000","last_unstale":"2024-06-03T03:21:26.055294+0000","last_undegraded":"2024-06-03T03:21:26.055294+0000","last_fullsized":"2024-06-03T03:21:26.055294+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:00:36.232238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.981032+0000","last_change":"2024-06-03T03:19:56.921299+0000","last_active":"2024-06-03T03:21:25.981032+0000","last_peered":"2024-06-03T03:21:25.981032+0000","last_clean":"2024-06-03T03:21:25.981032+0000","last_became_active":"2024-06-03T03:19:56.920870+0000","last_became_peered":"2024-06-03T03:19:56.920870+0000","last_unstale":"2024-06-03T03:21:25.981032+0000","last_undegraded":"2024-06-03T03:21:25.981032+0000","last_fullsized":"2024-06-03T03:21:25.981032+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:10:15.062381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"97'4","reported_seq":22,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.053044+0000","last_change":"2024-06-03T03:20:25.903279+0000","last_active":"2024-06-03T03:21:26.053044+0000","last_peered":"2024-06-03T03:21:26.053044+0000","last_clean":"2024-06-03T03:21:26.053044+0000","last_became_active":"2024-06-03T03:20:25.900311+0000","last_became_peered":"2024-06-03T03:20:25.900311+0000","last_unstale":"2024-06-03T03:21:26.053044+0000","last_undegraded":"2024-06-03T03:21:26.053044+0000","last_fullsized":"2024-06-03T03:21:26.053044+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:25:49.708581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.052682+0000","last_change":"2024-06-03T03:19:41.628659+0000","last_active":"2024-06-03T03:21:26.052682+0000","last_peered":"2024-06-03T03:21:26.052682+0000","last_clean":"2024-06-03T03:21:26.052682+0000","last_became_active":"2024-06-03T03:19:41.628291+0000","last_became_peered":"2024-06-03T03:19:41.628291+0000","last_unstale":"2024-06-03T03:21:26.052682+0000","last_undegraded":"2024-06-03T03:21:26.052682+0000","last_fullsized":"2024-06-03T03:21:26.052682+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:26:49.579669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.052909+0000","last_change":"2024-06-03T03:19:56.922766+0000","last_active":"2024-06-03T03:21:26.052909+0000","last_peered":"2024-06-03T03:21:26.052909+0000","last_clean":"2024-06-03T03:21:26.052909+0000","last_became_active":"2024-06-03T03:19:56.922310+0000","last_became_peered":"2024-06-03T03:19:56.922310+0000","last_unstale":"2024-06-03T03:21:26.052909+0000","last_undegraded":"2024-06-03T03:21:26.052909+0000","last_fullsized":"2024-06-03T03:21:26.052909+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T14:35:53.133519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"97'5","reported_seq":57,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:41.112684+0000","last_change":"2024-06-03T03:20:26.133562+0000","last_active":"2024-06-03T03:21:41.112684+0000","last_peered":"2024-06-03T03:21:41.112684+0000","last_clean":"2024-06-03T03:21:41.112684+0000","last_became_active":"2024-06-03T03:20:26.133074+0000","last_became_peered":"2024-06-03T03:20:26.133074+0000","last_unstale":"2024-06-03T03:21:41.112684+0000","last_undegraded":"2024-06-03T03:21:41.112684+0000","last_fullsized":"2024-06-03T03:21:41.112684+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:51:33.717584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":37,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.053150+0000","last_change":"2024-06-03T03:19:41.611450+0000","last_active":"2024-06-03T03:21:26.053150+0000","last_peered":"2024-06-03T03:21:26.053150+0000","last_clean":"2024-06-03T03:21:26.053150+0000","last_became_active":"2024-06-03T03:19:41.610214+0000","last_became_peered":"2024-06-03T03:19:41.610214+0000","last_unstale":"2024-06-03T03:21:26.053150+0000","last_undegraded":"2024-06-03T03:21:26.053150+0000","last_fullsized":"2024-06-03T03:21:26.053150+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:01:01.521917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.975414+0000","last_change":"2024-06-03T03:19:56.935686+0000","last_active":"2024-06-03T03:21:25.975414+0000","last_peered":"2024-06-03T03:21:25.975414+0000","last_clean":"2024-06-03T03:21:25.975414+0000","last_became_active":"2024-06-03T03:19:56.935314+0000","last_became_peered":"2024-06-03T03:19:56.935314+0000","last_unstale":"2024-06-03T03:21:25.975414+0000","last_undegraded":"2024-06-03T03:21:25.975414+0000","last_fullsized":"2024-06-03T03:21:25.975414+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T15:10:16.966321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"97'3","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.053864+0000","last_change":"2024-06-03T03:19:36.547756+0000","last_active":"2024-06-03T03:21:26.053864+0000","last_peered":"2024-06-03T03:21:26.053864+0000","last_clean":"2024-06-03T03:21:26.053864+0000","last_became_active":"2024-06-03T03:19:36.547350+0000","last_became_peered":"2024-06-03T03:19:36.547350+0000","last_unstale":"2024-06-03T03:21:26.053864+0000","last_undegraded":"2024-06-03T03:21:26.053864+0000","last_fullsized":"2024-06-03T03:21:26.053864+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:57:39.214380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.053597+0000","last_change":"2024-06-03T03:19:41.621083+0000","last_active":"2024-06-03T03:21:26.053597+0000","last_peered":"2024-06-03T03:21:26.053597+0000","last_clean":"2024-06-03T03:21:26.053597+0000","last_became_active":"2024-06-03T03:19:41.620719+0000","last_became_peered":"2024-06-03T03:19:41.620719+0000","last_unstale":"2024-06-03T03:21:26.053597+0000","last_undegraded":"2024-06-03T03:21:26.053597+0000","last_fullsized":"2024-06-03T03:21:26.053597+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:50:35.007025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.052222+0000","last_change":"2024-06-03T03:19:56.915406+0000","last_active":"2024-06-03T03:21:26.052222+0000","last_peered":"2024-06-03T03:21:26.052222+0000","last_clean":"2024-06-03T03:21:26.052222+0000","last_became_active":"2024-06-03T03:19:56.914918+0000","last_became_peered":"2024-06-03T03:19:56.914918+0000","last_unstale":"2024-06-03T03:21:26.052222+0000","last_undegraded":"2024-06-03T03:21:26.052222+0000","last_fullsized":"2024-06-03T03:21:26.052222+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:42:59.324011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"97'5","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.979476+0000","last_change":"2024-06-03T03:19:36.549502+0000","last_active":"2024-06-03T03:21:25.979476+0000","last_peered":"2024-06-03T03:21:25.979476+0000","last_clean":"2024-06-03T03:21:25.979476+0000","last_became_active":"2024-06-03T03:19:36.547538+0000","last_became_peered":"2024-06-03T03:19:36.547538+0000","last_unstale":"2024-06-03T03:21:25.979476+0000","last_undegraded":"2024-06-03T03:21:25.979476+0000","last_fullsized":"2024-06-03T03:21:25.979476+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:28:50.558069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.054070+0000","last_change":"2024-06-03T03:19:41.607441+0000","last_active":"2024-06-03T03:21:26.054070+0000","last_peered":"2024-06-03T03:21:26.054070+0000","last_clean":"2024-06-03T03:21:26.054070+0000","last_became_active":"2024-06-03T03:19:41.607086+0000","last_became_peered":"2024-06-03T03:19:41.607086+0000","last_unstale":"2024-06-03T03:21:26.054070+0000","last_undegraded":"2024-06-03T03:21:26.054070+0000","last_fullsized":"2024-06-03T03:21:26.054070+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T14:53:50.091872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.053930+0000","last_change":"2024-06-03T03:19:41.607004+0000","last_active":"2024-06-03T03:21:26.053930+0000","last_peered":"2024-06-03T03:21:26.053930+0000","last_clean":"2024-06-03T03:21:26.053930+0000","last_became_active":"2024-06-03T03:19:41.606684+0000","last_became_peered":"2024-06-03T03:19:41.606684+0000","last_unstale":"2024-06-03T03:21:26.053930+0000","last_undegraded":"2024-06-03T03:21:26.053930+0000","last_fullsized":"2024-06-03T03:21:26.053930+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:48:55.320597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.052544+0000","last_change":"2024-06-03T03:19:41.589426+0000","last_active":"2024-06-03T03:21:26.052544+0000","last_peered":"2024-06-03T03:21:26.052544+0000","last_clean":"2024-06-03T03:21:26.052544+0000","last_became_active":"2024-06-03T03:19:41.589091+0000","last_became_peered":"2024-06-03T03:19:41.589091+0000","last_unstale":"2024-06-03T03:21:26.052544+0000","last_undegraded":"2024-06-03T03:21:26.052544+0000","last_fullsized":"2024-06-03T03:21:26.052544+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:40:34.313214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.052322+0000","last_change":"2024-06-03T03:19:41.626149+0000","last_active":"2024-06-03T03:21:26.052322+0000","last_peered":"2024-06-03T03:21:26.052322+0000","last_clean":"2024-06-03T03:21:26.052322+0000","last_became_active":"2024-06-03T03:19:41.625710+0000","last_became_peered":"2024-06-03T03:19:41.625710+0000","last_unstale":"2024-06-03T03:21:26.052322+0000","last_undegraded":"2024-06-03T03:21:26.052322+0000","last_fullsized":"2024-06-03T03:21:26.052322+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:15:15.220975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.976354+0000","last_change":"2024-06-03T03:19:41.636585+0000","last_active":"2024-06-03T03:21:25.976354+0000","last_peered":"2024-06-03T03:21:25.976354+0000","last_clean":"2024-06-03T03:21:25.976354+0000","last_became_active":"2024-06-03T03:19:41.636241+0000","last_became_peered":"2024-06-03T03:19:41.636241+0000","last_unstale":"2024-06-03T03:21:25.976354+0000","last_undegraded":"2024-06-03T03:21:25.976354+0000","last_fullsized":"2024-06-03T03:21:25.976354+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:19:28.548297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.978614+0000","last_change":"2024-06-03T03:19:41.629530+0000","last_active":"2024-06-03T03:21:25.978614+0000","last_peered":"2024-06-03T03:21:25.978614+0000","last_clean":"2024-06-03T03:21:25.978614+0000","last_became_active":"2024-06-03T03:19:41.629197+0000","last_became_peered":"2024-06-03T03:19:41.629197+0000","last_unstale":"2024-06-03T03:21:25.978614+0000","last_undegraded":"2024-06-03T03:21:25.978614+0000","last_fullsized":"2024-06-03T03:21:25.978614+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:12:02.807445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.053577+0000","last_change":"2024-06-03T03:19:41.606995+0000","last_active":"2024-06-03T03:21:26.053577+0000","last_peered":"2024-06-03T03:21:26.053577+0000","last_clean":"2024-06-03T03:21:26.053577+0000","last_became_active":"2024-06-03T03:19:41.606651+0000","last_became_peered":"2024-06-03T03:19:41.606651+0000","last_unstale":"2024-06-03T03:21:26.053577+0000","last_undegraded":"2024-06-03T03:21:26.053577+0000","last_fullsized":"2024-06-03T03:21:26.053577+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:07:11.751606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.053366+0000","last_change":"2024-06-03T03:19:41.609848+0000","last_active":"2024-06-03T03:21:26.053366+0000","last_peered":"2024-06-03T03:21:26.053366+0000","last_clean":"2024-06-03T03:21:26.053366+0000","last_became_active":"2024-06-03T03:19:41.609544+0000","last_became_peered":"2024-06-03T03:19:41.609544+0000","last_unstale":"2024-06-03T03:21:26.053366+0000","last_undegraded":"2024-06-03T03:21:26.053366+0000","last_fullsized":"2024-06-03T03:21:26.053366+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:12:00.804638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":43,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.059089+0000","last_change":"2024-06-03T03:20:25.912049+0000","last_active":"2024-06-03T03:21:26.059089+0000","last_peered":"2024-06-03T03:21:26.059089+0000","last_clean":"2024-06-03T03:21:26.059089+0000","last_became_active":"2024-06-03T03:20:25.910664+0000","last_became_peered":"2024-06-03T03:20:25.910664+0000","last_unstale":"2024-06-03T03:21:26.059089+0000","last_undegraded":"2024-06-03T03:21:26.059089+0000","last_fullsized":"2024-06-03T03:21:26.059089+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:47:19.282569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.052469+0000","last_change":"2024-06-03T03:19:41.610800+0000","last_active":"2024-06-03T03:21:26.052469+0000","last_peered":"2024-06-03T03:21:26.052469+0000","last_clean":"2024-06-03T03:21:26.052469+0000","last_became_active":"2024-06-03T03:19:41.610457+0000","last_became_peered":"2024-06-03T03:19:41.610457+0000","last_unstale":"2024-06-03T03:21:26.052469+0000","last_undegraded":"2024-06-03T03:21:26.052469+0000","last_fullsized":"2024-06-03T03:21:26.052469+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:31:06.415159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.056986+0000","last_change":"2024-06-03T03:19:41.617020+0000","last_active":"2024-06-03T03:21:26.056986+0000","last_peered":"2024-06-03T03:21:26.056986+0000","last_clean":"2024-06-03T03:21:26.056986+0000","last_became_active":"2024-06-03T03:19:41.616614+0000","last_became_peered":"2024-06-03T03:19:41.616614+0000","last_unstale":"2024-06-03T03:21:26.056986+0000","last_undegraded":"2024-06-03T03:21:26.056986+0000","last_fullsized":"2024-06-03T03:21:26.056986+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:35:32.657685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.053252+0000","last_change":"2024-06-03T03:19:41.622785+0000","last_active":"2024-06-03T03:21:26.053252+0000","last_peered":"2024-06-03T03:21:26.053252+0000","last_clean":"2024-06-03T03:21:26.053252+0000","last_became_active":"2024-06-03T03:19:41.622407+0000","last_became_peered":"2024-06-03T03:19:41.622407+0000","last_unstale":"2024-06-03T03:21:26.053252+0000","last_undegraded":"2024-06-03T03:21:26.053252+0000","last_fullsized":"2024-06-03T03:21:26.053252+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:53:50.804889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.978017+0000","last_change":"2024-06-03T03:19:41.633429+0000","last_active":"2024-06-03T03:21:25.978017+0000","last_peered":"2024-06-03T03:21:25.978017+0000","last_clean":"2024-06-03T03:21:25.978017+0000","last_became_active":"2024-06-03T03:19:41.631028+0000","last_became_peered":"2024-06-03T03:19:41.631028+0000","last_unstale":"2024-06-03T03:21:25.978017+0000","last_undegraded":"2024-06-03T03:21:25.978017+0000","last_fullsized":"2024-06-03T03:21:25.978017+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:37:10.666591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.053936+0000","last_change":"2024-06-03T03:19:41.606204+0000","last_active":"2024-06-03T03:21:26.053936+0000","last_peered":"2024-06-03T03:21:26.053936+0000","last_clean":"2024-06-03T03:21:26.053936+0000","last_became_active":"2024-06-03T03:19:41.605756+0000","last_became_peered":"2024-06-03T03:19:41.605756+0000","last_unstale":"2024-06-03T03:21:26.053936+0000","last_undegraded":"2024-06-03T03:21:26.053936+0000","last_fullsized":"2024-06-03T03:21:26.053936+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:24:32.507426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":245318,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70364,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":387,"num_read_kb":268,"num_write":254,"num_write_kb":142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":569344,"data_stored":215357,"data_compressed":5580,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":217,"ondisk_log_size":217,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383721,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32260,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712892,"statfs":{"total":95995035648,"available":95962001408,"internally_reserved":0,"allocated":1101824,"data_stored":570375,"data_compressed":32353,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645358,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32396,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712756,"statfs":{"total":95995035648,"available":95961862144,"internally_reserved":0,"allocated":1249280,"data_stored":1220743,"data_compressed":69963,"data_compressed_allocated":94208,"data_compressed_original":688128,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808883,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32280,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712872,"statfs":{"total":95995035648,"available":95961980928,"internally_reserved":0,"allocated":1130496,"data_stored":574401,"data_compressed":32353,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070522,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32272,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712880,"statfs":{"total":95995035648,"available":95961989120,"internally_reserved":0,"allocated":1122304,"data_stored":570683,"data_compressed":32353,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332160,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32324,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712828,"statfs":{"total":95995035648,"available":95961935872,"internally_reserved":0,"allocated":1175552,"data_stored":1167179,"data_compressed":68103,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495686,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1101824,"data_stored":565809,"data_compressed":32353,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574945,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28156,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716996,"statfs":{"total":95995035648,"available":95966203904,"internally_reserved":0,"allocated":1167360,"data_stored":630391,"data_compressed":34213,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738470,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28124,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717028,"statfs":{"total":95995035648,"available":95966236672,"internally_reserved":0,"allocated":1126400,"data_stored":567265,"data_compressed":32353,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411420,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36492,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708660,"statfs":{"total":95995035648,"available":95957667840,"internally_reserved":0,"allocated":1183744,"data_stored":634096,"data_compressed":34213,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182486,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1114112,"data_stored":565517,"data_compressed":32353,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920849,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28108,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717044,"statfs":{"total":95995035648,"available":95966253056,"internally_reserved":0,"allocated":1118208,"data_stored":571053,"data_compressed":32353,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757324,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32340,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712812,"statfs":{"total":95995035648,"available":95961919488,"internally_reserved":0,"allocated":1183744,"data_stored":1162761,"data_compressed":68103,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34106,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34106,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34106,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35750,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35750,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35750,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35750,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":65304,"data_compressed":1860,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":69009,"data_compressed":1860,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":65288,"data_compressed":1860,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-03T03:21:46.453 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-03T03:21:46.918 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:21:47.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:46 smithi183 ceph-mon[37326]: from='client.24722 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-03T03:21:47.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:46 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:21:47.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:46 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:21:47.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:46 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:21:47.280 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:46 smithi117 ceph-mon[31649]: from='client.24722 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-03T03:21:47.281 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:46 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:21:47.281 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:46 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:21:47.281 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:46 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:21:47.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:46 smithi151 ceph-mon[37043]: from='client.24722 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-03T03:21:47.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:46 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:21:47.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:46 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:21:47.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:46 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:21:48.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:47 smithi183 ceph-mon[37326]: pgmap v400: 129 pgs: 129 active+clean; 645 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-06-03T03:21:48.405 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:47 smithi117 ceph-mon[31649]: pgmap v400: 129 pgs: 129 active+clean; 645 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-06-03T03:21:48.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:47 smithi151 ceph-mon[37043]: pgmap v400: 129 pgs: 129 active+clean; 645 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-06-03T03:21:49.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:49.311+0000 7f26fce00700 1 -- 172.21.15.117:0/918596329 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f26f8113660 msgr2=0x7f26f8115a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:21:49.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:49.311+0000 7f26fce00700 1 --2- 172.21.15.117:0/918596329 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f26f8113660 0x7f26f8115a50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f26f000b0a0 tx=0x7f26f000b3b0 comp rx=0 tx=0).stop 2024-06-03T03:21:49.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:49.312+0000 7f26fce00700 1 -- 172.21.15.117:0/918596329 shutdown_connections 2024-06-03T03:21:49.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:49.312+0000 7f26fce00700 1 --2- 172.21.15.117:0/918596329 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f26f8113660 0x7f26f8115a50 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:49.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:49.312+0000 7f26fce00700 1 --2- 172.21.15.117:0/918596329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26f80746c0 0x7f26f8074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:49.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:49.312+0000 7f26fce00700 1 --2- 172.21.15.117:0/918596329 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26f8074fe0 0x7f26f8113120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:49.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:49.312+0000 7f26fce00700 1 -- 172.21.15.117:0/918596329 >> 172.21.15.117:0/918596329 conn(0x7f26f80795f0 msgr2=0x7f26f8079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:21:49.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:49.312+0000 7f26fce00700 1 -- 172.21.15.117:0/918596329 shutdown_connections 2024-06-03T03:21:49.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:49.312+0000 7f26fce00700 1 -- 172.21.15.117:0/918596329 wait complete. 2024-06-03T03:21:49.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:49.313+0000 7f26fce00700 1 Processor -- start 2024-06-03T03:21:49.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:49.313+0000 7f26fce00700 1 -- start start 2024-06-03T03:21:49.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:49.313+0000 7f26fce00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26f80746c0 0x7f26f819f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:21:49.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:49.314+0000 7f26fce00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f26f8074fe0 0x7f26f819f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:21:49.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:49.314+0000 7f26fce00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26f8113660 0x7f26f81a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:21:49.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:49.314+0000 7f26fce00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f26f8118df0 con 0x7f26f80746c0 2024-06-03T03:21:49.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:49.314+0000 7f26fce00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f26f8118c70 con 0x7f26f8074fe0 2024-06-03T03:21:49.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:49.314+0000 7f26fce00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f26f8118f70 con 0x7f26f8113660 2024-06-03T03:21:49.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:49.314+0000 7f26f5e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26f80746c0 0x7f26f819f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:21:49.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:49.314+0000 7f26f5e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26f80746c0 0x7f26f819f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53196/0 (socket says 172.21.15.117:53196) 2024-06-03T03:21:49.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:49.314+0000 7f26f5e00700 1 -- 172.21.15.117:0/3928952771 learned_addr learned my addr 172.21.15.117:0/3928952771 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:21:49.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:49.314+0000 7f26f5400700 1 --2- 172.21.15.117:0/3928952771 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f26f8074fe0 0x7f26f819f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:21:49.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:49.314+0000 7f26f6800700 1 --2- 172.21.15.117:0/3928952771 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26f8113660 0x7f26f81a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:21:49.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:49.314+0000 7f26f5400700 1 -- 172.21.15.117:0/3928952771 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26f8113660 msgr2=0x7f26f81a3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:21:49.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:49.314+0000 7f26f5400700 1 --2- 172.21.15.117:0/3928952771 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26f8113660 0x7f26f81a3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:49.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:49.314+0000 7f26f5400700 1 -- 172.21.15.117:0/3928952771 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26f80746c0 msgr2=0x7f26f819f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:21:49.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:49.314+0000 7f26f5400700 1 --2- 172.21.15.117:0/3928952771 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26f80746c0 0x7f26f819f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:49.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:49.314+0000 7f26f5400700 1 -- 172.21.15.117:0/3928952771 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26f81a4230 con 0x7f26f8074fe0 2024-06-03T03:21:49.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:49.314+0000 7f26f5e00700 1 --2- 172.21.15.117:0/3928952771 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26f80746c0 0x7f26f819f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:21:49.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:49.314+0000 7f26f6800700 1 --2- 172.21.15.117:0/3928952771 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26f8113660 0x7f26f81a3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:21:49.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:49.315+0000 7f26f5400700 1 --2- 172.21.15.117:0/3928952771 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f26f8074fe0 0x7f26f819f760 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f26e400e9f0 tx=0x7f26e400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:21:49.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:49.315+0000 7f26eea00700 1 -- 172.21.15.117:0/3928952771 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e4004220 con 0x7f26f8074fe0 2024-06-03T03:21:49.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:49.315+0000 7f26eea00700 1 -- 172.21.15.117:0/3928952771 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26e4004d10 con 0x7f26f8074fe0 2024-06-03T03:21:49.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:49.315+0000 7f26fce00700 1 -- 172.21.15.117:0/3928952771 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26f81a4520 con 0x7f26f8074fe0 2024-06-03T03:21:49.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:49.315+0000 7f26eea00700 1 -- 172.21.15.117:0/3928952771 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e4005020 con 0x7f26f8074fe0 2024-06-03T03:21:49.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:49.315+0000 7f26fce00700 1 -- 172.21.15.117:0/3928952771 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26f81abda0 con 0x7f26f8074fe0 2024-06-03T03:21:49.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:49.316+0000 7f26fce00700 1 -- 172.21.15.117:0/3928952771 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26f8069180 con 0x7f26f8074fe0 2024-06-03T03:21:49.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:49.319+0000 7f26eea00700 1 -- 172.21.15.117:0/3928952771 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f26e4004750 con 0x7f26f8074fe0 2024-06-03T03:21:49.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:49.320+0000 7f26eea00700 1 --2- 172.21.15.117:0/3928952771 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f26dc0a2cb0 0x7f26dc0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:21:49.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:49.320+0000 7f26eea00700 1 -- 172.21.15.117:0/3928952771 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f26e40cf400 con 0x7f26f8074fe0 2024-06-03T03:21:49.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:49.321+0000 7f26f5e00700 1 --2- 172.21.15.117:0/3928952771 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f26dc0a2cb0 0x7f26dc0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:21:49.322 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:49.321+0000 7f26f5e00700 1 --2- 172.21.15.117:0/3928952771 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f26dc0a2cb0 0x7f26dc0a5170 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f26e8011550 tx=0x7f26e8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:21:49.326 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:49.325+0000 7f26eea00700 1 -- 172.21.15.117:0/3928952771 <== mon.2 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f26e409be10 con 0x7f26f8074fe0 2024-06-03T03:21:49.546 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:49.545+0000 7f26fce00700 1 -- 172.21.15.117:0/3928952771 --> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f26f8112b30 con 0x7f26dc0a2cb0 2024-06-03T03:21:49.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:49.561+0000 7f26eea00700 1 -- 172.21.15.117:0/3928952771 <== mgr.14152 v2:172.21.15.117:6800/1398160887 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340701 (secure 0 0 0) 0x7f26f8112b30 con 0x7f26dc0a2cb0 2024-06-03T03:21:49.563 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T03:21:49.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:49.570+0000 7f26fce00700 1 -- 172.21.15.117:0/3928952771 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f26dc0a2cb0 msgr2=0x7f26dc0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:21:49.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:49.570+0000 7f26fce00700 1 --2- 172.21.15.117:0/3928952771 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f26dc0a2cb0 0x7f26dc0a5170 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f26e8011550 tx=0x7f26e8009260 comp rx=0 tx=0).stop 2024-06-03T03:21:49.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:49.570+0000 7f26fce00700 1 -- 172.21.15.117:0/3928952771 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f26f8074fe0 msgr2=0x7f26f819f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:21:49.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:49.570+0000 7f26fce00700 1 --2- 172.21.15.117:0/3928952771 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f26f8074fe0 0x7f26f819f760 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f26e400e9f0 tx=0x7f26e400ed00 comp rx=0 tx=0).stop 2024-06-03T03:21:49.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:49.570+0000 7f26fce00700 1 -- 172.21.15.117:0/3928952771 shutdown_connections 2024-06-03T03:21:49.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:49.570+0000 7f26fce00700 1 --2- 172.21.15.117:0/3928952771 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f26dc0a2cb0 0x7f26dc0a5170 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:49.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:49.570+0000 7f26fce00700 1 --2- 172.21.15.117:0/3928952771 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26f8113660 0x7f26f81a3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:49.572 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:49.570+0000 7f26fce00700 1 --2- 172.21.15.117:0/3928952771 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26f80746c0 0x7f26f819f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:49.572 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:49.570+0000 7f26fce00700 1 --2- 172.21.15.117:0/3928952771 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f26f8074fe0 0x7f26f819f760 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:49.572 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:49.570+0000 7f26fce00700 1 -- 172.21.15.117:0/3928952771 >> 172.21.15.117:0/3928952771 conn(0x7f26f80795f0 msgr2=0x7f26f8112420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:21:49.572 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:49.570+0000 7f26fce00700 1 -- 172.21.15.117:0/3928952771 shutdown_connections 2024-06-03T03:21:49.572 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:49.570+0000 7f26fce00700 1 -- 172.21.15.117:0/3928952771 wait complete. 2024-06-03T03:21:49.572 INFO:teuthology.orchestra.run.smithi117.stderr:dumped all 2024-06-03T03:21:50.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:49 smithi183 ceph-mon[37326]: pgmap v401: 129 pgs: 129 active+clean; 650 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-06-03T03:21:50.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:49 smithi151 ceph-mon[37043]: pgmap v401: 129 pgs: 129 active+clean; 650 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-06-03T03:21:50.422 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:49 smithi117 ceph-mon[31649]: pgmap v401: 129 pgs: 129 active+clean; 650 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-06-03T03:21:50.487 INFO:teuthology.orchestra.run.smithi117.stdout:{"pg_ready":true,"pg_map":{"version":401,"stamp":"2024-06-03T03:21:48.856935+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":665231,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":559,"num_read_kb":407,"num_write":438,"num_write_kb":3087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":357,"ondisk_log_size":357,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":370964,"kb_used_data":13460,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124570860,"statfs":{"total":1151940427776,"available":1151560560640,"internally_reserved":0,"allocated":13783040,"data_stored":8808803,"data_compressed":501066,"data_compressed_allocated":688128,"data_compressed_original":2875392,"omap_allocated":19142,"internal_metadata":365933882},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18519,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":143,"num_read_kb":90,"num_write":8,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005463"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.054480+0000","last_change":"2024-06-03T03:19:41.626030+0000","last_active":"2024-06-03T03:21:26.054480+0000","last_peered":"2024-06-03T03:21:26.054480+0000","last_clean":"2024-06-03T03:21:26.054480+0000","last_became_active":"2024-06-03T03:19:41.625681+0000","last_became_peered":"2024-06-03T03:19:41.625681+0000","last_unstale":"2024-06-03T03:21:26.054480+0000","last_undegraded":"2024-06-03T03:21:26.054480+0000","last_fullsized":"2024-06-03T03:21:26.054480+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:35:46.396360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.976761+0000","last_change":"2024-06-03T03:19:41.619668+0000","last_active":"2024-06-03T03:21:25.976761+0000","last_peered":"2024-06-03T03:21:25.976761+0000","last_clean":"2024-06-03T03:21:25.976761+0000","last_became_active":"2024-06-03T03:19:41.619318+0000","last_became_peered":"2024-06-03T03:19:41.619318+0000","last_unstale":"2024-06-03T03:21:25.976761+0000","last_undegraded":"2024-06-03T03:21:25.976761+0000","last_fullsized":"2024-06-03T03:21:25.976761+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:07:21.656045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.975413+0000","last_change":"2024-06-03T03:19:41.623689+0000","last_active":"2024-06-03T03:21:25.975413+0000","last_peered":"2024-06-03T03:21:25.975413+0000","last_clean":"2024-06-03T03:21:25.975413+0000","last_became_active":"2024-06-03T03:19:41.623027+0000","last_became_peered":"2024-06-03T03:19:41.623027+0000","last_unstale":"2024-06-03T03:21:25.975413+0000","last_undegraded":"2024-06-03T03:21:25.975413+0000","last_fullsized":"2024-06-03T03:21:25.975413+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:18:05.584397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.981031+0000","last_change":"2024-06-03T03:19:41.603802+0000","last_active":"2024-06-03T03:21:25.981031+0000","last_peered":"2024-06-03T03:21:25.981031+0000","last_clean":"2024-06-03T03:21:25.981031+0000","last_became_active":"2024-06-03T03:19:41.603470+0000","last_became_peered":"2024-06-03T03:19:41.603470+0000","last_unstale":"2024-06-03T03:21:25.981031+0000","last_undegraded":"2024-06-03T03:21:25.981031+0000","last_fullsized":"2024-06-03T03:21:25.981031+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:56:10.636108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.973503+0000","last_change":"2024-06-03T03:19:41.636436+0000","last_active":"2024-06-03T03:21:25.973503+0000","last_peered":"2024-06-03T03:21:25.973503+0000","last_clean":"2024-06-03T03:21:25.973503+0000","last_became_active":"2024-06-03T03:19:41.636150+0000","last_became_peered":"2024-06-03T03:19:41.636150+0000","last_unstale":"2024-06-03T03:21:25.973503+0000","last_undegraded":"2024-06-03T03:21:25.973503+0000","last_fullsized":"2024-06-03T03:21:25.973503+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:02:10.928251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.053609+0000","last_change":"2024-06-03T03:19:41.622426+0000","last_active":"2024-06-03T03:21:26.053609+0000","last_peered":"2024-06-03T03:21:26.053609+0000","last_clean":"2024-06-03T03:21:26.053609+0000","last_became_active":"2024-06-03T03:19:41.622092+0000","last_became_peered":"2024-06-03T03:19:41.622092+0000","last_unstale":"2024-06-03T03:21:26.053609+0000","last_undegraded":"2024-06-03T03:21:26.053609+0000","last_fullsized":"2024-06-03T03:21:26.053609+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T14:08:52.666771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.052326+0000","last_change":"2024-06-03T03:19:41.608303+0000","last_active":"2024-06-03T03:21:26.052326+0000","last_peered":"2024-06-03T03:21:26.052326+0000","last_clean":"2024-06-03T03:21:26.052326+0000","last_became_active":"2024-06-03T03:19:41.607929+0000","last_became_peered":"2024-06-03T03:19:41.607929+0000","last_unstale":"2024-06-03T03:21:26.052326+0000","last_undegraded":"2024-06-03T03:21:26.052326+0000","last_fullsized":"2024-06-03T03:21:26.052326+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:40:05.643531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.056269+0000","last_change":"2024-06-03T03:19:41.606722+0000","last_active":"2024-06-03T03:21:26.056269+0000","last_peered":"2024-06-03T03:21:26.056269+0000","last_clean":"2024-06-03T03:21:26.056269+0000","last_became_active":"2024-06-03T03:19:41.606131+0000","last_became_peered":"2024-06-03T03:19:41.606131+0000","last_unstale":"2024-06-03T03:21:26.056269+0000","last_undegraded":"2024-06-03T03:21:26.056269+0000","last_fullsized":"2024-06-03T03:21:26.056269+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:09:17.306848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.053811+0000","last_change":"2024-06-03T03:19:41.609287+0000","last_active":"2024-06-03T03:21:26.053811+0000","last_peered":"2024-06-03T03:21:26.053811+0000","last_clean":"2024-06-03T03:21:26.053811+0000","last_became_active":"2024-06-03T03:19:41.608521+0000","last_became_peered":"2024-06-03T03:19:41.608521+0000","last_unstale":"2024-06-03T03:21:26.053811+0000","last_undegraded":"2024-06-03T03:21:26.053811+0000","last_fullsized":"2024-06-03T03:21:26.053811+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:35:37.128667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.054740+0000","last_change":"2024-06-03T03:19:41.605193+0000","last_active":"2024-06-03T03:21:26.054740+0000","last_peered":"2024-06-03T03:21:26.054740+0000","last_clean":"2024-06-03T03:21:26.054740+0000","last_became_active":"2024-06-03T03:19:41.604787+0000","last_became_peered":"2024-06-03T03:19:41.604787+0000","last_unstale":"2024-06-03T03:21:26.054740+0000","last_undegraded":"2024-06-03T03:21:26.054740+0000","last_fullsized":"2024-06-03T03:21:26.054740+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:02:48.334599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.052944+0000","last_change":"2024-06-03T03:19:41.611522+0000","last_active":"2024-06-03T03:21:26.052944+0000","last_peered":"2024-06-03T03:21:26.052944+0000","last_clean":"2024-06-03T03:21:26.052944+0000","last_became_active":"2024-06-03T03:19:41.610932+0000","last_became_peered":"2024-06-03T03:19:41.610932+0000","last_unstale":"2024-06-03T03:21:26.052944+0000","last_undegraded":"2024-06-03T03:21:26.052944+0000","last_fullsized":"2024-06-03T03:21:26.052944+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:35:44.702791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":15,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.055524+0000","last_change":"2024-06-03T03:20:25.950663+0000","last_active":"2024-06-03T03:21:26.055524+0000","last_peered":"2024-06-03T03:21:26.055524+0000","last_clean":"2024-06-03T03:21:26.055524+0000","last_became_active":"2024-06-03T03:20:25.949962+0000","last_became_peered":"2024-06-03T03:20:25.949962+0000","last_unstale":"2024-06-03T03:21:26.055524+0000","last_undegraded":"2024-06-03T03:21:26.055524+0000","last_fullsized":"2024-06-03T03:21:26.055524+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:55:33.972567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.977681+0000","last_change":"2024-06-03T03:19:41.633464+0000","last_active":"2024-06-03T03:21:25.977681+0000","last_peered":"2024-06-03T03:21:25.977681+0000","last_clean":"2024-06-03T03:21:25.977681+0000","last_became_active":"2024-06-03T03:19:41.630625+0000","last_became_peered":"2024-06-03T03:19:41.630625+0000","last_unstale":"2024-06-03T03:21:25.977681+0000","last_undegraded":"2024-06-03T03:21:25.977681+0000","last_fullsized":"2024-06-03T03:21:25.977681+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:23:28.799058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.975828+0000","last_change":"2024-06-03T03:19:41.627234+0000","last_active":"2024-06-03T03:21:25.975828+0000","last_peered":"2024-06-03T03:21:25.975828+0000","last_clean":"2024-06-03T03:21:25.975828+0000","last_became_active":"2024-06-03T03:19:41.626643+0000","last_became_peered":"2024-06-03T03:19:41.626643+0000","last_unstale":"2024-06-03T03:21:25.975828+0000","last_undegraded":"2024-06-03T03:21:25.975828+0000","last_fullsized":"2024-06-03T03:21:25.975828+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:37:44.126101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.976005+0000","last_change":"2024-06-03T03:19:41.621833+0000","last_active":"2024-06-03T03:21:25.976005+0000","last_peered":"2024-06-03T03:21:25.976005+0000","last_clean":"2024-06-03T03:21:25.976005+0000","last_became_active":"2024-06-03T03:19:41.620689+0000","last_became_peered":"2024-06-03T03:19:41.620689+0000","last_unstale":"2024-06-03T03:21:25.976005+0000","last_undegraded":"2024-06-03T03:21:25.976005+0000","last_fullsized":"2024-06-03T03:21:25.976005+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:39:31.188372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.052549+0000","last_change":"2024-06-03T03:19:41.624072+0000","last_active":"2024-06-03T03:21:26.052549+0000","last_peered":"2024-06-03T03:21:26.052549+0000","last_clean":"2024-06-03T03:21:26.052549+0000","last_became_active":"2024-06-03T03:19:41.623689+0000","last_became_peered":"2024-06-03T03:19:41.623689+0000","last_unstale":"2024-06-03T03:21:26.052549+0000","last_undegraded":"2024-06-03T03:21:26.052549+0000","last_fullsized":"2024-06-03T03:21:26.052549+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:38:24.075256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.054263+0000","last_change":"2024-06-03T03:19:41.626145+0000","last_active":"2024-06-03T03:21:26.054263+0000","last_peered":"2024-06-03T03:21:26.054263+0000","last_clean":"2024-06-03T03:21:26.054263+0000","last_became_active":"2024-06-03T03:19:41.625826+0000","last_became_peered":"2024-06-03T03:19:41.625826+0000","last_unstale":"2024-06-03T03:21:26.054263+0000","last_undegraded":"2024-06-03T03:21:26.054263+0000","last_fullsized":"2024-06-03T03:21:26.054263+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:04:10.996134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.056880+0000","last_change":"2024-06-03T03:19:41.615754+0000","last_active":"2024-06-03T03:21:26.056880+0000","last_peered":"2024-06-03T03:21:26.056880+0000","last_clean":"2024-06-03T03:21:26.056880+0000","last_became_active":"2024-06-03T03:19:41.614577+0000","last_became_peered":"2024-06-03T03:19:41.614577+0000","last_unstale":"2024-06-03T03:21:26.056880+0000","last_undegraded":"2024-06-03T03:21:26.056880+0000","last_fullsized":"2024-06-03T03:21:26.056880+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:45:50.084332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.053093+0000","last_change":"2024-06-03T03:19:41.633000+0000","last_active":"2024-06-03T03:21:26.053093+0000","last_peered":"2024-06-03T03:21:26.053093+0000","last_clean":"2024-06-03T03:21:26.053093+0000","last_became_active":"2024-06-03T03:19:41.632731+0000","last_became_peered":"2024-06-03T03:19:41.632731+0000","last_unstale":"2024-06-03T03:21:26.053093+0000","last_undegraded":"2024-06-03T03:21:26.053093+0000","last_fullsized":"2024-06-03T03:21:26.053093+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:10:29.990256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":35,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.052257+0000","last_change":"2024-06-03T03:20:26.133763+0000","last_active":"2024-06-03T03:21:26.052257+0000","last_peered":"2024-06-03T03:21:26.052257+0000","last_clean":"2024-06-03T03:21:26.052257+0000","last_became_active":"2024-06-03T03:20:26.133252+0000","last_became_peered":"2024-06-03T03:20:26.133252+0000","last_unstale":"2024-06-03T03:21:26.052257+0000","last_undegraded":"2024-06-03T03:21:26.052257+0000","last_fullsized":"2024-06-03T03:21:26.052257+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:33:21.461038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"97'13","reported_seq":97,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:44.123360+0000","last_change":"2024-06-03T03:19:36.542545+0000","last_active":"2024-06-03T03:21:44.123360+0000","last_peered":"2024-06-03T03:21:44.123360+0000","last_clean":"2024-06-03T03:21:44.123360+0000","last_became_active":"2024-06-03T03:19:36.541682+0000","last_became_peered":"2024-06-03T03:19:36.541682+0000","last_unstale":"2024-06-03T03:21:44.123360+0000","last_undegraded":"2024-06-03T03:21:44.123360+0000","last_fullsized":"2024-06-03T03:21:44.123360+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:06:23.909253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":39,"num_read_kb":39,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.058378+0000","last_change":"2024-06-03T03:19:41.605848+0000","last_active":"2024-06-03T03:21:26.058378+0000","last_peered":"2024-06-03T03:21:26.058378+0000","last_clean":"2024-06-03T03:21:26.058378+0000","last_became_active":"2024-06-03T03:19:41.603131+0000","last_became_peered":"2024-06-03T03:19:41.603131+0000","last_unstale":"2024-06-03T03:21:26.058378+0000","last_undegraded":"2024-06-03T03:21:26.058378+0000","last_fullsized":"2024-06-03T03:21:26.058378+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:07:38.931055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.980716+0000","last_change":"2024-06-03T03:19:56.923180+0000","last_active":"2024-06-03T03:21:25.980716+0000","last_peered":"2024-06-03T03:21:25.980716+0000","last_clean":"2024-06-03T03:21:25.980716+0000","last_became_active":"2024-06-03T03:19:56.922726+0000","last_became_peered":"2024-06-03T03:19:56.922726+0000","last_unstale":"2024-06-03T03:21:25.980716+0000","last_undegraded":"2024-06-03T03:21:25.980716+0000","last_fullsized":"2024-06-03T03:21:25.980716+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:18:45.727680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"97'2","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.058990+0000","last_change":"2024-06-03T03:19:36.746805+0000","last_active":"2024-06-03T03:21:26.058990+0000","last_peered":"2024-06-03T03:21:26.058990+0000","last_clean":"2024-06-03T03:21:26.058990+0000","last_became_active":"2024-06-03T03:19:36.745747+0000","last_became_peered":"2024-06-03T03:19:36.745747+0000","last_unstale":"2024-06-03T03:21:26.058990+0000","last_undegraded":"2024-06-03T03:21:26.058990+0000","last_fullsized":"2024-06-03T03:21:26.058990+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:54:28.427708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.976822+0000","last_change":"2024-06-03T03:19:41.630582+0000","last_active":"2024-06-03T03:21:25.976822+0000","last_peered":"2024-06-03T03:21:25.976822+0000","last_clean":"2024-06-03T03:21:25.976822+0000","last_became_active":"2024-06-03T03:19:41.630238+0000","last_became_peered":"2024-06-03T03:19:41.630238+0000","last_unstale":"2024-06-03T03:21:25.976822+0000","last_undegraded":"2024-06-03T03:21:25.976822+0000","last_fullsized":"2024-06-03T03:21:25.976822+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:49:58.039697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.980805+0000","last_change":"2024-06-03T03:19:56.925142+0000","last_active":"2024-06-03T03:21:25.980805+0000","last_peered":"2024-06-03T03:21:25.980805+0000","last_clean":"2024-06-03T03:21:25.980805+0000","last_became_active":"2024-06-03T03:19:56.924760+0000","last_became_peered":"2024-06-03T03:19:56.924760+0000","last_unstale":"2024-06-03T03:21:25.980805+0000","last_undegraded":"2024-06-03T03:21:25.980805+0000","last_fullsized":"2024-06-03T03:21:25.980805+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:11:27.537077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"97'12","reported_seq":56,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.974042+0000","last_change":"2024-06-03T03:19:36.543344+0000","last_active":"2024-06-03T03:21:25.974042+0000","last_peered":"2024-06-03T03:21:25.974042+0000","last_clean":"2024-06-03T03:21:25.974042+0000","last_became_active":"2024-06-03T03:19:36.542837+0000","last_became_peered":"2024-06-03T03:19:36.542837+0000","last_unstale":"2024-06-03T03:21:25.974042+0000","last_undegraded":"2024-06-03T03:21:25.974042+0000","last_fullsized":"2024-06-03T03:21:25.974042+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:55:05.451521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.053193+0000","last_change":"2024-06-03T03:19:41.629820+0000","last_active":"2024-06-03T03:21:26.053193+0000","last_peered":"2024-06-03T03:21:26.053193+0000","last_clean":"2024-06-03T03:21:26.053193+0000","last_became_active":"2024-06-03T03:19:41.629439+0000","last_became_peered":"2024-06-03T03:19:41.629439+0000","last_unstale":"2024-06-03T03:21:26.053193+0000","last_undegraded":"2024-06-03T03:21:26.053193+0000","last_fullsized":"2024-06-03T03:21:26.053193+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:05:27.438308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.980716+0000","last_change":"2024-06-03T03:19:56.923214+0000","last_active":"2024-06-03T03:21:25.980716+0000","last_peered":"2024-06-03T03:21:25.980716+0000","last_clean":"2024-06-03T03:21:25.980716+0000","last_became_active":"2024-06-03T03:19:56.922751+0000","last_became_peered":"2024-06-03T03:19:56.922751+0000","last_unstale":"2024-06-03T03:21:25.980716+0000","last_undegraded":"2024-06-03T03:21:25.980716+0000","last_fullsized":"2024-06-03T03:21:25.980716+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:05:59.249818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"97'5","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.973842+0000","last_change":"2024-06-03T03:19:36.538141+0000","last_active":"2024-06-03T03:21:25.973842+0000","last_peered":"2024-06-03T03:21:25.973842+0000","last_clean":"2024-06-03T03:21:25.973842+0000","last_became_active":"2024-06-03T03:19:36.537786+0000","last_became_peered":"2024-06-03T03:19:36.537786+0000","last_unstale":"2024-06-03T03:21:25.973842+0000","last_undegraded":"2024-06-03T03:21:25.973842+0000","last_fullsized":"2024-06-03T03:21:25.973842+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T15:03:32.859097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.054195+0000","last_change":"2024-06-03T03:19:41.608471+0000","last_active":"2024-06-03T03:21:26.054195+0000","last_peered":"2024-06-03T03:21:26.054195+0000","last_clean":"2024-06-03T03:21:26.054195+0000","last_became_active":"2024-06-03T03:19:41.608096+0000","last_became_peered":"2024-06-03T03:19:41.608096+0000","last_unstale":"2024-06-03T03:21:26.054195+0000","last_undegraded":"2024-06-03T03:21:26.054195+0000","last_fullsized":"2024-06-03T03:21:26.054195+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:31:10.541803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.052967+0000","last_change":"2024-06-03T03:19:56.915540+0000","last_active":"2024-06-03T03:21:26.052967+0000","last_peered":"2024-06-03T03:21:26.052967+0000","last_clean":"2024-06-03T03:21:26.052967+0000","last_became_active":"2024-06-03T03:19:56.915017+0000","last_became_peered":"2024-06-03T03:19:56.915017+0000","last_unstale":"2024-06-03T03:21:26.052967+0000","last_undegraded":"2024-06-03T03:21:26.052967+0000","last_fullsized":"2024-06-03T03:21:26.052967+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:56:23.158344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"97'6","reported_seq":93,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:46.137652+0000","last_change":"2024-06-03T03:19:36.542337+0000","last_active":"2024-06-03T03:21:46.137652+0000","last_peered":"2024-06-03T03:21:46.137652+0000","last_clean":"2024-06-03T03:21:46.137652+0000","last_became_active":"2024-06-03T03:19:36.541990+0000","last_became_peered":"2024-06-03T03:19:36.541990+0000","last_unstale":"2024-06-03T03:21:46.137652+0000","last_undegraded":"2024-06-03T03:21:46.137652+0000","last_fullsized":"2024-06-03T03:21:46.137652+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:08:15.656321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":43,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.973454+0000","last_change":"2024-06-03T03:19:41.616293+0000","last_active":"2024-06-03T03:21:25.973454+0000","last_peered":"2024-06-03T03:21:25.973454+0000","last_clean":"2024-06-03T03:21:25.973454+0000","last_became_active":"2024-06-03T03:19:41.615946+0000","last_became_peered":"2024-06-03T03:19:41.615946+0000","last_unstale":"2024-06-03T03:21:25.973454+0000","last_undegraded":"2024-06-03T03:21:25.973454+0000","last_fullsized":"2024-06-03T03:21:25.973454+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:18:43.835977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":35,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.053575+0000","last_change":"2024-06-03T03:20:26.146237+0000","last_active":"2024-06-03T03:21:26.053575+0000","last_peered":"2024-06-03T03:21:26.053575+0000","last_clean":"2024-06-03T03:21:26.053575+0000","last_became_active":"2024-06-03T03:20:26.145731+0000","last_became_peered":"2024-06-03T03:20:26.145731+0000","last_unstale":"2024-06-03T03:21:26.053575+0000","last_undegraded":"2024-06-03T03:21:26.053575+0000","last_fullsized":"2024-06-03T03:21:26.053575+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:15:29.723833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"97'7","reported_seq":51,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.053492+0000","last_change":"2024-06-03T03:19:36.546501+0000","last_active":"2024-06-03T03:21:26.053492+0000","last_peered":"2024-06-03T03:21:26.053492+0000","last_clean":"2024-06-03T03:21:26.053492+0000","last_became_active":"2024-06-03T03:19:36.546079+0000","last_became_peered":"2024-06-03T03:19:36.546079+0000","last_unstale":"2024-06-03T03:21:26.053492+0000","last_undegraded":"2024-06-03T03:21:26.053492+0000","last_fullsized":"2024-06-03T03:21:26.053492+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:16:39.473141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.054080+0000","last_change":"2024-06-03T03:20:26.149410+0000","last_active":"2024-06-03T03:21:26.054080+0000","last_peered":"2024-06-03T03:21:26.054080+0000","last_clean":"2024-06-03T03:21:26.054080+0000","last_became_active":"2024-06-03T03:20:26.149084+0000","last_became_peered":"2024-06-03T03:20:26.149084+0000","last_unstale":"2024-06-03T03:21:26.054080+0000","last_undegraded":"2024-06-03T03:21:26.054080+0000","last_fullsized":"2024-06-03T03:21:26.054080+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:06:22.316449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.054121+0000","last_change":"2024-06-03T03:19:56.926498+0000","last_active":"2024-06-03T03:21:26.054121+0000","last_peered":"2024-06-03T03:21:26.054121+0000","last_clean":"2024-06-03T03:21:26.054121+0000","last_became_active":"2024-06-03T03:19:56.926101+0000","last_became_peered":"2024-06-03T03:19:56.926101+0000","last_unstale":"2024-06-03T03:21:26.054121+0000","last_undegraded":"2024-06-03T03:21:26.054121+0000","last_fullsized":"2024-06-03T03:21:26.054121+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:00:12.024443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"97'3","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.977250+0000","last_change":"2024-06-03T03:19:36.553151+0000","last_active":"2024-06-03T03:21:25.977250+0000","last_peered":"2024-06-03T03:21:25.977250+0000","last_clean":"2024-06-03T03:21:25.977250+0000","last_became_active":"2024-06-03T03:19:36.551914+0000","last_became_peered":"2024-06-03T03:19:36.551914+0000","last_unstale":"2024-06-03T03:21:25.977250+0000","last_undegraded":"2024-06-03T03:21:25.977250+0000","last_fullsized":"2024-06-03T03:21:25.977250+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:00:42.053922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.054306+0000","last_change":"2024-06-03T03:20:26.147909+0000","last_active":"2024-06-03T03:21:26.054306+0000","last_peered":"2024-06-03T03:21:26.054306+0000","last_clean":"2024-06-03T03:21:26.054306+0000","last_became_active":"2024-06-03T03:20:26.146448+0000","last_became_peered":"2024-06-03T03:20:26.146448+0000","last_unstale":"2024-06-03T03:21:26.054306+0000","last_undegraded":"2024-06-03T03:21:26.054306+0000","last_fullsized":"2024-06-03T03:21:26.054306+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:43:30.878472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.976566+0000","last_change":"2024-06-03T03:19:56.941910+0000","last_active":"2024-06-03T03:21:25.976566+0000","last_peered":"2024-06-03T03:21:25.976566+0000","last_clean":"2024-06-03T03:21:25.976566+0000","last_became_active":"2024-06-03T03:19:56.940963+0000","last_became_peered":"2024-06-03T03:19:56.940963+0000","last_unstale":"2024-06-03T03:21:25.976566+0000","last_undegraded":"2024-06-03T03:21:25.976566+0000","last_fullsized":"2024-06-03T03:21:25.976566+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T14:18:32.109231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"97'3","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.052423+0000","last_change":"2024-06-03T03:19:36.538928+0000","last_active":"2024-06-03T03:21:26.052423+0000","last_peered":"2024-06-03T03:21:26.052423+0000","last_clean":"2024-06-03T03:21:26.052423+0000","last_became_active":"2024-06-03T03:19:36.538580+0000","last_became_peered":"2024-06-03T03:19:36.538580+0000","last_unstale":"2024-06-03T03:21:26.052423+0000","last_undegraded":"2024-06-03T03:21:26.052423+0000","last_fullsized":"2024-06-03T03:21:26.052423+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:08:05.906257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.054672+0000","last_change":"2024-06-03T03:19:41.623051+0000","last_active":"2024-06-03T03:21:26.054672+0000","last_peered":"2024-06-03T03:21:26.054672+0000","last_clean":"2024-06-03T03:21:26.054672+0000","last_became_active":"2024-06-03T03:19:41.622656+0000","last_became_peered":"2024-06-03T03:19:41.622656+0000","last_unstale":"2024-06-03T03:21:26.054672+0000","last_undegraded":"2024-06-03T03:21:26.054672+0000","last_fullsized":"2024-06-03T03:21:26.054672+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:11:34.587723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.056239+0000","last_change":"2024-06-03T03:19:56.921215+0000","last_active":"2024-06-03T03:21:26.056239+0000","last_peered":"2024-06-03T03:21:26.056239+0000","last_clean":"2024-06-03T03:21:26.056239+0000","last_became_active":"2024-06-03T03:19:56.920658+0000","last_became_peered":"2024-06-03T03:19:56.920658+0000","last_unstale":"2024-06-03T03:21:26.056239+0000","last_undegraded":"2024-06-03T03:21:26.056239+0000","last_fullsized":"2024-06-03T03:21:26.056239+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T14:04:54.163818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"97'4","reported_seq":126,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:46.118622+0000","last_change":"2024-06-03T03:19:36.545245+0000","last_active":"2024-06-03T03:21:46.118622+0000","last_peered":"2024-06-03T03:21:46.118622+0000","last_clean":"2024-06-03T03:21:46.118622+0000","last_became_active":"2024-06-03T03:19:36.544821+0000","last_became_peered":"2024-06-03T03:19:36.544821+0000","last_unstale":"2024-06-03T03:21:46.118622+0000","last_undegraded":"2024-06-03T03:21:46.118622+0000","last_fullsized":"2024-06-03T03:21:46.118622+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:55:24.011925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":78,"num_read_kb":78,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.053751+0000","last_change":"2024-06-03T03:19:41.616384+0000","last_active":"2024-06-03T03:21:26.053751+0000","last_peered":"2024-06-03T03:21:26.053751+0000","last_clean":"2024-06-03T03:21:26.053751+0000","last_became_active":"2024-06-03T03:19:41.616034+0000","last_became_peered":"2024-06-03T03:19:41.616034+0000","last_unstale":"2024-06-03T03:21:26.053751+0000","last_undegraded":"2024-06-03T03:21:26.053751+0000","last_fullsized":"2024-06-03T03:21:26.053751+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:10:06.823375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.054357+0000","last_change":"2024-06-03T03:19:56.933045+0000","last_active":"2024-06-03T03:21:26.054357+0000","last_peered":"2024-06-03T03:21:26.054357+0000","last_clean":"2024-06-03T03:21:26.054357+0000","last_became_active":"2024-06-03T03:19:56.928956+0000","last_became_peered":"2024-06-03T03:19:56.928956+0000","last_unstale":"2024-06-03T03:21:26.054357+0000","last_undegraded":"2024-06-03T03:21:26.054357+0000","last_fullsized":"2024-06-03T03:21:26.054357+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:37:44.270511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"97'4","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.053347+0000","last_change":"2024-06-03T03:19:36.541559+0000","last_active":"2024-06-03T03:21:26.053347+0000","last_peered":"2024-06-03T03:21:26.053347+0000","last_clean":"2024-06-03T03:21:26.053347+0000","last_became_active":"2024-06-03T03:19:36.541166+0000","last_became_peered":"2024-06-03T03:19:36.541166+0000","last_unstale":"2024-06-03T03:21:26.053347+0000","last_undegraded":"2024-06-03T03:21:26.053347+0000","last_fullsized":"2024-06-03T03:21:26.053347+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T14:49:26.847713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.977003+0000","last_change":"2024-06-03T03:19:41.629789+0000","last_active":"2024-06-03T03:21:25.977003+0000","last_peered":"2024-06-03T03:21:25.977003+0000","last_clean":"2024-06-03T03:21:25.977003+0000","last_became_active":"2024-06-03T03:19:41.628026+0000","last_became_peered":"2024-06-03T03:19:41.628026+0000","last_unstale":"2024-06-03T03:21:25.977003+0000","last_undegraded":"2024-06-03T03:21:25.977003+0000","last_fullsized":"2024-06-03T03:21:25.977003+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:56:26.411907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":35,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.056114+0000","last_change":"2024-06-03T03:21:26.056114+0000","last_active":"2024-06-03T03:21:26.056114+0000","last_peered":"2024-06-03T03:21:26.056114+0000","last_clean":"2024-06-03T03:21:26.056114+0000","last_became_active":"2024-06-03T03:21:26.055505+0000","last_became_peered":"2024-06-03T03:21:26.055505+0000","last_unstale":"2024-06-03T03:21:26.056114+0000","last_undegraded":"2024-06-03T03:21:26.056114+0000","last_fullsized":"2024-06-03T03:21:26.056114+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:22:55.722566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,10],"acting":[4,1,2,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"97'5","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.053488+0000","last_change":"2024-06-03T03:19:36.744930+0000","last_active":"2024-06-03T03:21:26.053488+0000","last_peered":"2024-06-03T03:21:26.053488+0000","last_clean":"2024-06-03T03:21:26.053488+0000","last_became_active":"2024-06-03T03:19:36.744567+0000","last_became_peered":"2024-06-03T03:19:36.744567+0000","last_unstale":"2024-06-03T03:21:26.053488+0000","last_undegraded":"2024-06-03T03:21:26.053488+0000","last_fullsized":"2024-06-03T03:21:26.053488+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:35:31.927211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.053330+0000","last_change":"2024-06-03T03:19:41.607604+0000","last_active":"2024-06-03T03:21:26.053330+0000","last_peered":"2024-06-03T03:21:26.053330+0000","last_clean":"2024-06-03T03:21:26.053330+0000","last_became_active":"2024-06-03T03:19:41.607148+0000","last_became_peered":"2024-06-03T03:19:41.607148+0000","last_unstale":"2024-06-03T03:21:26.053330+0000","last_undegraded":"2024-06-03T03:21:26.053330+0000","last_fullsized":"2024-06-03T03:21:26.053330+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:44:44.037836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.056102+0000","last_change":"2024-06-03T03:19:56.928689+0000","last_active":"2024-06-03T03:21:26.056102+0000","last_peered":"2024-06-03T03:21:26.056102+0000","last_clean":"2024-06-03T03:21:26.056102+0000","last_became_active":"2024-06-03T03:19:56.924955+0000","last_became_peered":"2024-06-03T03:19:56.924955+0000","last_unstale":"2024-06-03T03:21:26.056102+0000","last_undegraded":"2024-06-03T03:21:26.056102+0000","last_fullsized":"2024-06-03T03:21:26.056102+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:11:07.413430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"99'37","reported_seq":169,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:46.118621+0000","last_change":"2024-06-03T03:19:36.539688+0000","last_active":"2024-06-03T03:21:46.118621+0000","last_peered":"2024-06-03T03:21:46.118621+0000","last_clean":"2024-06-03T03:21:46.118621+0000","last_became_active":"2024-06-03T03:19:36.539302+0000","last_became_peered":"2024-06-03T03:19:36.539302+0000","last_unstale":"2024-06-03T03:21:46.118621+0000","last_undegraded":"2024-06-03T03:21:46.118621+0000","last_fullsized":"2024-06-03T03:21:46.118621+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:28:52.790227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67817,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":88,"num_read_kb":110,"num_write":39,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.973872+0000","last_change":"2024-06-03T03:19:41.608202+0000","last_active":"2024-06-03T03:21:25.973872+0000","last_peered":"2024-06-03T03:21:25.973872+0000","last_clean":"2024-06-03T03:21:25.973872+0000","last_became_active":"2024-06-03T03:19:41.607891+0000","last_became_peered":"2024-06-03T03:19:41.607891+0000","last_unstale":"2024-06-03T03:21:25.973872+0000","last_undegraded":"2024-06-03T03:21:25.973872+0000","last_fullsized":"2024-06-03T03:21:25.973872+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T14:10:44.042892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.054137+0000","last_change":"2024-06-03T03:19:56.923547+0000","last_active":"2024-06-03T03:21:26.054137+0000","last_peered":"2024-06-03T03:21:26.054137+0000","last_clean":"2024-06-03T03:21:26.054137+0000","last_became_active":"2024-06-03T03:19:56.923046+0000","last_became_peered":"2024-06-03T03:19:56.923046+0000","last_unstale":"2024-06-03T03:21:26.054137+0000","last_undegraded":"2024-06-03T03:21:26.054137+0000","last_fullsized":"2024-06-03T03:21:26.054137+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:32:19.438793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"97'9","reported_seq":54,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.053818+0000","last_change":"2024-06-03T03:19:36.547024+0000","last_active":"2024-06-03T03:21:26.053818+0000","last_peered":"2024-06-03T03:21:26.053818+0000","last_clean":"2024-06-03T03:21:26.053818+0000","last_became_active":"2024-06-03T03:19:36.546663+0000","last_became_peered":"2024-06-03T03:19:36.546663+0000","last_unstale":"2024-06-03T03:21:26.053818+0000","last_undegraded":"2024-06-03T03:21:26.053818+0000","last_fullsized":"2024-06-03T03:21:26.053818+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:58:41.446730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.977661+0000","last_change":"2024-06-03T03:19:41.630119+0000","last_active":"2024-06-03T03:21:25.977661+0000","last_peered":"2024-06-03T03:21:25.977661+0000","last_clean":"2024-06-03T03:21:25.977661+0000","last_became_active":"2024-06-03T03:19:41.629563+0000","last_became_peered":"2024-06-03T03:19:41.629563+0000","last_unstale":"2024-06-03T03:21:25.977661+0000","last_undegraded":"2024-06-03T03:21:25.977661+0000","last_fullsized":"2024-06-03T03:21:25.977661+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:49:33.983471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.053729+0000","last_change":"2024-06-03T03:19:56.941451+0000","last_active":"2024-06-03T03:21:26.053729+0000","last_peered":"2024-06-03T03:21:26.053729+0000","last_clean":"2024-06-03T03:21:26.053729+0000","last_became_active":"2024-06-03T03:19:56.941003+0000","last_became_peered":"2024-06-03T03:19:56.941003+0000","last_unstale":"2024-06-03T03:21:26.053729+0000","last_undegraded":"2024-06-03T03:21:26.053729+0000","last_fullsized":"2024-06-03T03:21:26.053729+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:28:58.505495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"97'5","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.053392+0000","last_change":"2024-06-03T03:19:36.547128+0000","last_active":"2024-06-03T03:21:26.053392+0000","last_peered":"2024-06-03T03:21:26.053392+0000","last_clean":"2024-06-03T03:21:26.053392+0000","last_became_active":"2024-06-03T03:19:36.546576+0000","last_became_peered":"2024-06-03T03:19:36.546576+0000","last_unstale":"2024-06-03T03:21:26.053392+0000","last_undegraded":"2024-06-03T03:21:26.053392+0000","last_fullsized":"2024-06-03T03:21:26.053392+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:23:30.196543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.052379+0000","last_change":"2024-06-03T03:19:41.634941+0000","last_active":"2024-06-03T03:21:26.052379+0000","last_peered":"2024-06-03T03:21:26.052379+0000","last_clean":"2024-06-03T03:21:26.052379+0000","last_became_active":"2024-06-03T03:19:41.634561+0000","last_became_peered":"2024-06-03T03:19:41.634561+0000","last_unstale":"2024-06-03T03:21:26.052379+0000","last_undegraded":"2024-06-03T03:21:26.052379+0000","last_fullsized":"2024-06-03T03:21:26.052379+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:00:27.258078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.056606+0000","last_change":"2024-06-03T03:19:56.934612+0000","last_active":"2024-06-03T03:21:26.056606+0000","last_peered":"2024-06-03T03:21:26.056606+0000","last_clean":"2024-06-03T03:21:26.056606+0000","last_became_active":"2024-06-03T03:19:56.930212+0000","last_became_peered":"2024-06-03T03:19:56.930212+0000","last_unstale":"2024-06-03T03:21:26.056606+0000","last_undegraded":"2024-06-03T03:21:26.056606+0000","last_fullsized":"2024-06-03T03:21:26.056606+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:42:38.490095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"97'5","reported_seq":90,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:46.127798+0000","last_change":"2024-06-03T03:19:36.542052+0000","last_active":"2024-06-03T03:21:46.127798+0000","last_peered":"2024-06-03T03:21:46.127798+0000","last_clean":"2024-06-03T03:21:46.127798+0000","last_became_active":"2024-06-03T03:19:36.540987+0000","last_became_peered":"2024-06-03T03:19:36.540987+0000","last_unstale":"2024-06-03T03:21:46.127798+0000","last_undegraded":"2024-06-03T03:21:46.127798+0000","last_fullsized":"2024-06-03T03:21:46.127798+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T15:18:48.821400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":43,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.052494+0000","last_change":"2024-06-03T03:19:41.607610+0000","last_active":"2024-06-03T03:21:26.052494+0000","last_peered":"2024-06-03T03:21:26.052494+0000","last_clean":"2024-06-03T03:21:26.052494+0000","last_became_active":"2024-06-03T03:19:41.607219+0000","last_became_peered":"2024-06-03T03:19:41.607219+0000","last_unstale":"2024-06-03T03:21:26.052494+0000","last_undegraded":"2024-06-03T03:21:26.052494+0000","last_fullsized":"2024-06-03T03:21:26.052494+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:06:20.707623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.979811+0000","last_change":"2024-06-03T03:19:56.936440+0000","last_active":"2024-06-03T03:21:25.979811+0000","last_peered":"2024-06-03T03:21:25.979811+0000","last_clean":"2024-06-03T03:21:25.979811+0000","last_became_active":"2024-06-03T03:19:56.936048+0000","last_became_peered":"2024-06-03T03:19:56.936048+0000","last_unstale":"2024-06-03T03:21:25.979811+0000","last_undegraded":"2024-06-03T03:21:25.979811+0000","last_fullsized":"2024-06-03T03:21:25.979811+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:22:15.758950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"97'17","reported_seq":61,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.973311+0000","last_change":"2024-06-03T03:19:36.736105+0000","last_active":"2024-06-03T03:21:25.973311+0000","last_peered":"2024-06-03T03:21:25.973311+0000","last_clean":"2024-06-03T03:21:25.973311+0000","last_became_active":"2024-06-03T03:19:36.735663+0000","last_became_peered":"2024-06-03T03:19:36.735663+0000","last_unstale":"2024-06-03T03:21:25.973311+0000","last_undegraded":"2024-06-03T03:21:25.973311+0000","last_fullsized":"2024-06-03T03:21:25.973311+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:41:46.216923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.052904+0000","last_change":"2024-06-03T03:19:41.609758+0000","last_active":"2024-06-03T03:21:26.052904+0000","last_peered":"2024-06-03T03:21:26.052904+0000","last_clean":"2024-06-03T03:21:26.052904+0000","last_became_active":"2024-06-03T03:19:41.609421+0000","last_became_peered":"2024-06-03T03:19:41.609421+0000","last_unstale":"2024-06-03T03:21:26.052904+0000","last_undegraded":"2024-06-03T03:21:26.052904+0000","last_fullsized":"2024-06-03T03:21:26.052904+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:45:54.919352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.052780+0000","last_change":"2024-06-03T03:19:56.925422+0000","last_active":"2024-06-03T03:21:26.052780+0000","last_peered":"2024-06-03T03:21:26.052780+0000","last_clean":"2024-06-03T03:21:26.052780+0000","last_became_active":"2024-06-03T03:19:56.924891+0000","last_became_peered":"2024-06-03T03:19:56.924891+0000","last_unstale":"2024-06-03T03:21:26.052780+0000","last_undegraded":"2024-06-03T03:21:26.052780+0000","last_fullsized":"2024-06-03T03:21:26.052780+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:34:59.986256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"97'2","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.973873+0000","last_change":"2024-06-03T03:19:36.540605+0000","last_active":"2024-06-03T03:21:25.973873+0000","last_peered":"2024-06-03T03:21:25.973873+0000","last_clean":"2024-06-03T03:21:25.973873+0000","last_became_active":"2024-06-03T03:19:36.540180+0000","last_became_peered":"2024-06-03T03:19:36.540180+0000","last_unstale":"2024-06-03T03:21:25.973873+0000","last_undegraded":"2024-06-03T03:21:25.973873+0000","last_fullsized":"2024-06-03T03:21:25.973873+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:01:26.556726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":43,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.053484+0000","last_change":"2024-06-03T03:20:25.904215+0000","last_active":"2024-06-03T03:21:26.053484+0000","last_peered":"2024-06-03T03:21:26.053484+0000","last_clean":"2024-06-03T03:21:26.053484+0000","last_became_active":"2024-06-03T03:20:25.903894+0000","last_became_peered":"2024-06-03T03:20:25.903894+0000","last_unstale":"2024-06-03T03:21:26.053484+0000","last_undegraded":"2024-06-03T03:21:26.053484+0000","last_fullsized":"2024-06-03T03:21:26.053484+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:02:07.587159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.973461+0000","last_change":"2024-06-03T03:19:56.910041+0000","last_active":"2024-06-03T03:21:25.973461+0000","last_peered":"2024-06-03T03:21:25.973461+0000","last_clean":"2024-06-03T03:21:25.973461+0000","last_became_active":"2024-06-03T03:19:56.909591+0000","last_became_peered":"2024-06-03T03:19:56.909591+0000","last_unstale":"2024-06-03T03:21:25.973461+0000","last_undegraded":"2024-06-03T03:21:25.973461+0000","last_fullsized":"2024-06-03T03:21:25.973461+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:42:20.016838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"97'1","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.055897+0000","last_change":"2024-06-03T03:19:36.741731+0000","last_active":"2024-06-03T03:21:26.055897+0000","last_peered":"2024-06-03T03:21:26.055897+0000","last_clean":"2024-06-03T03:21:26.055897+0000","last_became_active":"2024-06-03T03:19:36.741329+0000","last_became_peered":"2024-06-03T03:19:36.741329+0000","last_unstale":"2024-06-03T03:21:26.055897+0000","last_undegraded":"2024-06-03T03:21:26.055897+0000","last_fullsized":"2024-06-03T03:21:26.055897+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:02:45.778368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.053848+0000","last_change":"2024-06-03T03:19:41.632225+0000","last_active":"2024-06-03T03:21:26.053848+0000","last_peered":"2024-06-03T03:21:26.053848+0000","last_clean":"2024-06-03T03:21:26.053848+0000","last_became_active":"2024-06-03T03:19:41.631851+0000","last_became_peered":"2024-06-03T03:19:41.631851+0000","last_unstale":"2024-06-03T03:21:26.053848+0000","last_undegraded":"2024-06-03T03:21:26.053848+0000","last_fullsized":"2024-06-03T03:21:26.053848+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:52:23.364233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.977827+0000","last_change":"2024-06-03T03:19:56.934200+0000","last_active":"2024-06-03T03:21:25.977827+0000","last_peered":"2024-06-03T03:21:25.977827+0000","last_clean":"2024-06-03T03:21:25.977827+0000","last_became_active":"2024-06-03T03:19:56.932998+0000","last_became_peered":"2024-06-03T03:19:56.932998+0000","last_unstale":"2024-06-03T03:21:25.977827+0000","last_undegraded":"2024-06-03T03:21:25.977827+0000","last_fullsized":"2024-06-03T03:21:25.977827+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:45:28.916023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"97'5","reported_seq":88,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:44.124002+0000","last_change":"2024-06-03T03:19:36.546630+0000","last_active":"2024-06-03T03:21:44.124002+0000","last_peered":"2024-06-03T03:21:44.124002+0000","last_clean":"2024-06-03T03:21:44.124002+0000","last_became_active":"2024-06-03T03:19:36.546248+0000","last_became_peered":"2024-06-03T03:19:36.546248+0000","last_unstale":"2024-06-03T03:21:44.124002+0000","last_undegraded":"2024-06-03T03:21:44.124002+0000","last_fullsized":"2024-06-03T03:21:44.124002+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T15:09:50.323113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":38,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.054380+0000","last_change":"2024-06-03T03:19:41.624584+0000","last_active":"2024-06-03T03:21:26.054380+0000","last_peered":"2024-06-03T03:21:26.054380+0000","last_clean":"2024-06-03T03:21:26.054380+0000","last_became_active":"2024-06-03T03:19:41.624216+0000","last_became_peered":"2024-06-03T03:19:41.624216+0000","last_unstale":"2024-06-03T03:21:26.054380+0000","last_undegraded":"2024-06-03T03:21:26.054380+0000","last_fullsized":"2024-06-03T03:21:26.054380+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:22:09.645341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.979831+0000","last_change":"2024-06-03T03:19:56.937422+0000","last_active":"2024-06-03T03:21:25.979831+0000","last_peered":"2024-06-03T03:21:25.979831+0000","last_clean":"2024-06-03T03:21:25.979831+0000","last_became_active":"2024-06-03T03:19:56.934075+0000","last_became_peered":"2024-06-03T03:19:56.934075+0000","last_unstale":"2024-06-03T03:21:25.979831+0000","last_undegraded":"2024-06-03T03:21:25.979831+0000","last_fullsized":"2024-06-03T03:21:25.979831+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:25:45.273161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.052773+0000","last_change":"2024-06-03T03:19:41.589595+0000","last_active":"2024-06-03T03:21:26.052773+0000","last_peered":"2024-06-03T03:21:26.052773+0000","last_clean":"2024-06-03T03:21:26.052773+0000","last_became_active":"2024-06-03T03:19:41.589251+0000","last_became_peered":"2024-06-03T03:19:41.589251+0000","last_unstale":"2024-06-03T03:21:26.052773+0000","last_undegraded":"2024-06-03T03:21:26.052773+0000","last_fullsized":"2024-06-03T03:21:26.052773+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:51:40.417386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"97'5","reported_seq":91,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:47.129297+0000","last_change":"2024-06-03T03:19:36.737523+0000","last_active":"2024-06-03T03:21:47.129297+0000","last_peered":"2024-06-03T03:21:47.129297+0000","last_clean":"2024-06-03T03:21:47.129297+0000","last_became_active":"2024-06-03T03:19:36.736698+0000","last_became_peered":"2024-06-03T03:19:36.736698+0000","last_unstale":"2024-06-03T03:21:47.129297+0000","last_undegraded":"2024-06-03T03:21:47.129297+0000","last_fullsized":"2024-06-03T03:21:47.129297+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:14:46.330549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":42,"num_read_kb":42,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"97'1","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.052754+0000","last_change":"2024-06-03T03:19:56.924421+0000","last_active":"2024-06-03T03:21:26.052754+0000","last_peered":"2024-06-03T03:21:26.052754+0000","last_clean":"2024-06-03T03:21:26.052754+0000","last_became_active":"2024-06-03T03:19:56.924043+0000","last_became_peered":"2024-06-03T03:19:56.924043+0000","last_unstale":"2024-06-03T03:21:26.052754+0000","last_undegraded":"2024-06-03T03:21:26.052754+0000","last_fullsized":"2024-06-03T03:21:26.052754+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:00:47.674717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.055629+0000","last_change":"2024-06-03T03:19:41.613800+0000","last_active":"2024-06-03T03:21:26.055629+0000","last_peered":"2024-06-03T03:21:26.055629+0000","last_clean":"2024-06-03T03:21:26.055629+0000","last_became_active":"2024-06-03T03:19:41.613387+0000","last_became_peered":"2024-06-03T03:19:41.613387+0000","last_unstale":"2024-06-03T03:21:26.055629+0000","last_undegraded":"2024-06-03T03:21:26.055629+0000","last_fullsized":"2024-06-03T03:21:26.055629+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:04:04.785023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"97'6","reported_seq":53,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.980987+0000","last_change":"2024-06-03T03:19:36.544518+0000","last_active":"2024-06-03T03:21:25.980987+0000","last_peered":"2024-06-03T03:21:25.980987+0000","last_clean":"2024-06-03T03:21:25.980987+0000","last_became_active":"2024-06-03T03:19:36.544056+0000","last_became_peered":"2024-06-03T03:19:36.544056+0000","last_unstale":"2024-06-03T03:21:25.980987+0000","last_undegraded":"2024-06-03T03:21:25.980987+0000","last_fullsized":"2024-06-03T03:21:25.980987+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:33:25.012986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":366,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.052263+0000","last_change":"2024-06-03T03:18:05.481351+0000","last_active":"2024-06-03T03:21:26.052263+0000","last_peered":"2024-06-03T03:21:26.052263+0000","last_clean":"2024-06-03T03:21:26.052263+0000","last_became_active":"2024-06-03T03:18:05.480929+0000","last_became_peered":"2024-06-03T03:18:05.480929+0000","last_unstale":"2024-06-03T03:21:26.052263+0000","last_undegraded":"2024-06-03T03:21:26.052263+0000","last_fullsized":"2024-06-03T03:21:26.052263+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:14:09.558597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:14:09.558597+0000","last_clean_scrub_stamp":"2024-06-03T03:14:09.558597+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:54:45.563733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.054294+0000","last_change":"2024-06-03T03:19:56.931867+0000","last_active":"2024-06-03T03:21:26.054294+0000","last_peered":"2024-06-03T03:21:26.054294+0000","last_clean":"2024-06-03T03:21:26.054294+0000","last_became_active":"2024-06-03T03:19:56.931538+0000","last_became_peered":"2024-06-03T03:19:56.931538+0000","last_unstale":"2024-06-03T03:21:26.054294+0000","last_undegraded":"2024-06-03T03:21:26.054294+0000","last_fullsized":"2024-06-03T03:21:26.054294+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:55:15.144192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.976793+0000","last_change":"2024-06-03T03:19:41.622713+0000","last_active":"2024-06-03T03:21:25.976793+0000","last_peered":"2024-06-03T03:21:25.976793+0000","last_clean":"2024-06-03T03:21:25.976793+0000","last_became_active":"2024-06-03T03:19:41.622363+0000","last_became_peered":"2024-06-03T03:19:41.622363+0000","last_unstale":"2024-06-03T03:21:25.976793+0000","last_undegraded":"2024-06-03T03:21:25.976793+0000","last_fullsized":"2024-06-03T03:21:25.976793+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:43:57.671797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"97'4","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.052450+0000","last_change":"2024-06-03T03:19:36.539663+0000","last_active":"2024-06-03T03:21:26.052450+0000","last_peered":"2024-06-03T03:21:26.052450+0000","last_clean":"2024-06-03T03:21:26.052450+0000","last_became_active":"2024-06-03T03:19:36.539320+0000","last_became_peered":"2024-06-03T03:19:36.539320+0000","last_unstale":"2024-06-03T03:21:26.052450+0000","last_undegraded":"2024-06-03T03:21:26.052450+0000","last_fullsized":"2024-06-03T03:21:26.052450+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:36:50.699231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.053178+0000","last_change":"2024-06-03T03:19:56.938231+0000","last_active":"2024-06-03T03:21:26.053178+0000","last_peered":"2024-06-03T03:21:26.053178+0000","last_clean":"2024-06-03T03:21:26.053178+0000","last_became_active":"2024-06-03T03:19:56.937840+0000","last_became_peered":"2024-06-03T03:19:56.937840+0000","last_unstale":"2024-06-03T03:21:26.053178+0000","last_undegraded":"2024-06-03T03:21:26.053178+0000","last_fullsized":"2024-06-03T03:21:26.053178+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:02:33.916980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"97'4","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.973969+0000","last_change":"2024-06-03T03:19:36.737519+0000","last_active":"2024-06-03T03:21:25.973969+0000","last_peered":"2024-06-03T03:21:25.973969+0000","last_clean":"2024-06-03T03:21:25.973969+0000","last_became_active":"2024-06-03T03:19:36.737191+0000","last_became_peered":"2024-06-03T03:19:36.737191+0000","last_unstale":"2024-06-03T03:21:25.973969+0000","last_undegraded":"2024-06-03T03:21:25.973969+0000","last_fullsized":"2024-06-03T03:21:25.973969+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:50:21.389739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.978646+0000","last_change":"2024-06-03T03:19:41.633113+0000","last_active":"2024-06-03T03:21:25.978646+0000","last_peered":"2024-06-03T03:21:25.978646+0000","last_clean":"2024-06-03T03:21:25.978646+0000","last_became_active":"2024-06-03T03:19:41.632186+0000","last_became_peered":"2024-06-03T03:19:41.632186+0000","last_unstale":"2024-06-03T03:21:25.978646+0000","last_undegraded":"2024-06-03T03:21:25.978646+0000","last_fullsized":"2024-06-03T03:21:25.978646+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T15:12:26.835061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.056988+0000","last_change":"2024-06-03T03:19:56.928603+0000","last_active":"2024-06-03T03:21:26.056988+0000","last_peered":"2024-06-03T03:21:26.056988+0000","last_clean":"2024-06-03T03:21:26.056988+0000","last_became_active":"2024-06-03T03:19:56.928175+0000","last_became_peered":"2024-06-03T03:19:56.928175+0000","last_unstale":"2024-06-03T03:21:26.056988+0000","last_undegraded":"2024-06-03T03:21:26.056988+0000","last_fullsized":"2024-06-03T03:21:26.056988+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:26:03.531091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"99'18","reported_seq":63,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:39.651924+0000","last_change":"2024-06-03T03:19:36.550577+0000","last_active":"2024-06-03T03:21:39.651924+0000","last_peered":"2024-06-03T03:21:39.651924+0000","last_clean":"2024-06-03T03:21:39.651924+0000","last_became_active":"2024-06-03T03:19:36.549968+0000","last_became_peered":"2024-06-03T03:19:36.549968+0000","last_unstale":"2024-06-03T03:21:39.651924+0000","last_undegraded":"2024-06-03T03:21:39.651924+0000","last_fullsized":"2024-06-03T03:21:39.651924+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:22:43.892002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.058728+0000","last_change":"2024-06-03T03:19:41.613883+0000","last_active":"2024-06-03T03:21:26.058728+0000","last_peered":"2024-06-03T03:21:26.058728+0000","last_clean":"2024-06-03T03:21:26.058728+0000","last_became_active":"2024-06-03T03:19:41.613472+0000","last_became_peered":"2024-06-03T03:19:41.613472+0000","last_unstale":"2024-06-03T03:21:26.058728+0000","last_undegraded":"2024-06-03T03:21:26.058728+0000","last_fullsized":"2024-06-03T03:21:26.058728+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T14:38:42.044534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.052192+0000","last_change":"2024-06-03T03:19:56.914487+0000","last_active":"2024-06-03T03:21:26.052192+0000","last_peered":"2024-06-03T03:21:26.052192+0000","last_clean":"2024-06-03T03:21:26.052192+0000","last_became_active":"2024-06-03T03:19:56.913904+0000","last_became_peered":"2024-06-03T03:19:56.913904+0000","last_unstale":"2024-06-03T03:21:26.052192+0000","last_undegraded":"2024-06-03T03:21:26.052192+0000","last_fullsized":"2024-06-03T03:21:26.052192+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T15:13:24.084034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"97'7","reported_seq":52,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.981407+0000","last_change":"2024-06-03T03:19:36.671355+0000","last_active":"2024-06-03T03:21:25.981407+0000","last_peered":"2024-06-03T03:21:25.981407+0000","last_clean":"2024-06-03T03:21:25.981407+0000","last_became_active":"2024-06-03T03:19:36.670926+0000","last_became_peered":"2024-06-03T03:19:36.670926+0000","last_unstale":"2024-06-03T03:21:25.981407+0000","last_undegraded":"2024-06-03T03:21:25.981407+0000","last_fullsized":"2024-06-03T03:21:25.981407+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:38:50.914238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.976850+0000","last_change":"2024-06-03T03:19:41.629054+0000","last_active":"2024-06-03T03:21:25.976850+0000","last_peered":"2024-06-03T03:21:25.976850+0000","last_clean":"2024-06-03T03:21:25.976850+0000","last_became_active":"2024-06-03T03:19:41.627118+0000","last_became_peered":"2024-06-03T03:19:41.627118+0000","last_unstale":"2024-06-03T03:21:25.976850+0000","last_undegraded":"2024-06-03T03:21:25.976850+0000","last_fullsized":"2024-06-03T03:21:25.976850+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:59:40.645002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":35,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.058741+0000","last_change":"2024-06-03T03:21:26.058741+0000","last_active":"2024-06-03T03:21:26.058741+0000","last_peered":"2024-06-03T03:21:26.058741+0000","last_clean":"2024-06-03T03:21:26.058741+0000","last_became_active":"2024-06-03T03:21:26.058271+0000","last_became_peered":"2024-06-03T03:21:26.058271+0000","last_unstale":"2024-06-03T03:21:26.058741+0000","last_undegraded":"2024-06-03T03:21:26.058741+0000","last_fullsized":"2024-06-03T03:21:26.058741+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:31:32.359893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"97'2","reported_seq":52,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.052687+0000","last_change":"2024-06-03T03:20:26.132994+0000","last_active":"2024-06-03T03:21:26.052687+0000","last_peered":"2024-06-03T03:21:26.052687+0000","last_clean":"2024-06-03T03:21:26.052687+0000","last_became_active":"2024-06-03T03:20:26.132574+0000","last_became_peered":"2024-06-03T03:20:26.132574+0000","last_unstale":"2024-06-03T03:21:26.052687+0000","last_undegraded":"2024-06-03T03:21:26.052687+0000","last_fullsized":"2024-06-03T03:21:26.052687+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T15:12:33.599154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.973316+0000","last_change":"2024-06-03T03:19:41.603630+0000","last_active":"2024-06-03T03:21:25.973316+0000","last_peered":"2024-06-03T03:21:25.973316+0000","last_clean":"2024-06-03T03:21:25.973316+0000","last_became_active":"2024-06-03T03:19:41.603301+0000","last_became_peered":"2024-06-03T03:19:41.603301+0000","last_unstale":"2024-06-03T03:21:25.973316+0000","last_undegraded":"2024-06-03T03:21:25.973316+0000","last_fullsized":"2024-06-03T03:21:25.973316+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:20:33.076622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.057458+0000","last_change":"2024-06-03T03:19:56.929023+0000","last_active":"2024-06-03T03:21:26.057458+0000","last_peered":"2024-06-03T03:21:26.057458+0000","last_clean":"2024-06-03T03:21:26.057458+0000","last_became_active":"2024-06-03T03:19:56.928454+0000","last_became_peered":"2024-06-03T03:19:56.928454+0000","last_unstale":"2024-06-03T03:21:26.057458+0000","last_undegraded":"2024-06-03T03:21:26.057458+0000","last_fullsized":"2024-06-03T03:21:26.057458+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:17:31.755649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"97'7","reported_seq":51,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.056909+0000","last_change":"2024-06-03T03:19:36.551413+0000","last_active":"2024-06-03T03:21:26.056909+0000","last_peered":"2024-06-03T03:21:26.056909+0000","last_clean":"2024-06-03T03:21:26.056909+0000","last_became_active":"2024-06-03T03:19:36.550622+0000","last_became_peered":"2024-06-03T03:19:36.550622+0000","last_unstale":"2024-06-03T03:21:26.056909+0000","last_undegraded":"2024-06-03T03:21:26.056909+0000","last_fullsized":"2024-06-03T03:21:26.056909+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:50:09.472960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.053813+0000","last_change":"2024-06-03T03:19:41.624871+0000","last_active":"2024-06-03T03:21:26.053813+0000","last_peered":"2024-06-03T03:21:26.053813+0000","last_clean":"2024-06-03T03:21:26.053813+0000","last_became_active":"2024-06-03T03:19:41.624638+0000","last_became_peered":"2024-06-03T03:19:41.624638+0000","last_unstale":"2024-06-03T03:21:26.053813+0000","last_undegraded":"2024-06-03T03:21:26.053813+0000","last_fullsized":"2024-06-03T03:21:26.053813+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:21:16.193817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"99'1","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.981349+0000","last_change":"2024-06-03T03:19:56.924638+0000","last_active":"2024-06-03T03:21:25.981349+0000","last_peered":"2024-06-03T03:21:25.981349+0000","last_clean":"2024-06-03T03:21:25.981349+0000","last_became_active":"2024-06-03T03:19:56.924200+0000","last_became_peered":"2024-06-03T03:19:56.924200+0000","last_unstale":"2024-06-03T03:21:25.981349+0000","last_undegraded":"2024-06-03T03:21:25.981349+0000","last_fullsized":"2024-06-03T03:21:25.981349+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:38:01.138253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"97'3","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.052654+0000","last_change":"2024-06-03T03:19:36.546974+0000","last_active":"2024-06-03T03:21:26.052654+0000","last_peered":"2024-06-03T03:21:26.052654+0000","last_clean":"2024-06-03T03:21:26.052654+0000","last_became_active":"2024-06-03T03:19:36.546433+0000","last_became_peered":"2024-06-03T03:19:36.546433+0000","last_unstale":"2024-06-03T03:21:26.052654+0000","last_undegraded":"2024-06-03T03:21:26.052654+0000","last_fullsized":"2024-06-03T03:21:26.052654+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:41:46.980327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.055294+0000","last_change":"2024-06-03T03:19:41.622118+0000","last_active":"2024-06-03T03:21:26.055294+0000","last_peered":"2024-06-03T03:21:26.055294+0000","last_clean":"2024-06-03T03:21:26.055294+0000","last_became_active":"2024-06-03T03:19:41.620789+0000","last_became_peered":"2024-06-03T03:19:41.620789+0000","last_unstale":"2024-06-03T03:21:26.055294+0000","last_undegraded":"2024-06-03T03:21:26.055294+0000","last_fullsized":"2024-06-03T03:21:26.055294+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:00:36.232238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.981032+0000","last_change":"2024-06-03T03:19:56.921299+0000","last_active":"2024-06-03T03:21:25.981032+0000","last_peered":"2024-06-03T03:21:25.981032+0000","last_clean":"2024-06-03T03:21:25.981032+0000","last_became_active":"2024-06-03T03:19:56.920870+0000","last_became_peered":"2024-06-03T03:19:56.920870+0000","last_unstale":"2024-06-03T03:21:25.981032+0000","last_undegraded":"2024-06-03T03:21:25.981032+0000","last_fullsized":"2024-06-03T03:21:25.981032+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:10:15.062381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"97'4","reported_seq":22,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.053044+0000","last_change":"2024-06-03T03:20:25.903279+0000","last_active":"2024-06-03T03:21:26.053044+0000","last_peered":"2024-06-03T03:21:26.053044+0000","last_clean":"2024-06-03T03:21:26.053044+0000","last_became_active":"2024-06-03T03:20:25.900311+0000","last_became_peered":"2024-06-03T03:20:25.900311+0000","last_unstale":"2024-06-03T03:21:26.053044+0000","last_undegraded":"2024-06-03T03:21:26.053044+0000","last_fullsized":"2024-06-03T03:21:26.053044+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:25:49.708581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.052682+0000","last_change":"2024-06-03T03:19:41.628659+0000","last_active":"2024-06-03T03:21:26.052682+0000","last_peered":"2024-06-03T03:21:26.052682+0000","last_clean":"2024-06-03T03:21:26.052682+0000","last_became_active":"2024-06-03T03:19:41.628291+0000","last_became_peered":"2024-06-03T03:19:41.628291+0000","last_unstale":"2024-06-03T03:21:26.052682+0000","last_undegraded":"2024-06-03T03:21:26.052682+0000","last_fullsized":"2024-06-03T03:21:26.052682+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:26:49.579669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.052909+0000","last_change":"2024-06-03T03:19:56.922766+0000","last_active":"2024-06-03T03:21:26.052909+0000","last_peered":"2024-06-03T03:21:26.052909+0000","last_clean":"2024-06-03T03:21:26.052909+0000","last_became_active":"2024-06-03T03:19:56.922310+0000","last_became_peered":"2024-06-03T03:19:56.922310+0000","last_unstale":"2024-06-03T03:21:26.052909+0000","last_undegraded":"2024-06-03T03:21:26.052909+0000","last_fullsized":"2024-06-03T03:21:26.052909+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T14:35:53.133519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"97'5","reported_seq":62,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:46.119362+0000","last_change":"2024-06-03T03:20:26.133562+0000","last_active":"2024-06-03T03:21:46.119362+0000","last_peered":"2024-06-03T03:21:46.119362+0000","last_clean":"2024-06-03T03:21:46.119362+0000","last_became_active":"2024-06-03T03:20:26.133074+0000","last_became_peered":"2024-06-03T03:20:26.133074+0000","last_unstale":"2024-06-03T03:21:46.119362+0000","last_undegraded":"2024-06-03T03:21:46.119362+0000","last_fullsized":"2024-06-03T03:21:46.119362+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:51:33.717584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":42,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.053150+0000","last_change":"2024-06-03T03:19:41.611450+0000","last_active":"2024-06-03T03:21:26.053150+0000","last_peered":"2024-06-03T03:21:26.053150+0000","last_clean":"2024-06-03T03:21:26.053150+0000","last_became_active":"2024-06-03T03:19:41.610214+0000","last_became_peered":"2024-06-03T03:19:41.610214+0000","last_unstale":"2024-06-03T03:21:26.053150+0000","last_undegraded":"2024-06-03T03:21:26.053150+0000","last_fullsized":"2024-06-03T03:21:26.053150+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:01:01.521917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.975414+0000","last_change":"2024-06-03T03:19:56.935686+0000","last_active":"2024-06-03T03:21:25.975414+0000","last_peered":"2024-06-03T03:21:25.975414+0000","last_clean":"2024-06-03T03:21:25.975414+0000","last_became_active":"2024-06-03T03:19:56.935314+0000","last_became_peered":"2024-06-03T03:19:56.935314+0000","last_unstale":"2024-06-03T03:21:25.975414+0000","last_undegraded":"2024-06-03T03:21:25.975414+0000","last_fullsized":"2024-06-03T03:21:25.975414+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T15:10:16.966321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"97'3","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.053864+0000","last_change":"2024-06-03T03:19:36.547756+0000","last_active":"2024-06-03T03:21:26.053864+0000","last_peered":"2024-06-03T03:21:26.053864+0000","last_clean":"2024-06-03T03:21:26.053864+0000","last_became_active":"2024-06-03T03:19:36.547350+0000","last_became_peered":"2024-06-03T03:19:36.547350+0000","last_unstale":"2024-06-03T03:21:26.053864+0000","last_undegraded":"2024-06-03T03:21:26.053864+0000","last_fullsized":"2024-06-03T03:21:26.053864+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:57:39.214380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.053597+0000","last_change":"2024-06-03T03:19:41.621083+0000","last_active":"2024-06-03T03:21:26.053597+0000","last_peered":"2024-06-03T03:21:26.053597+0000","last_clean":"2024-06-03T03:21:26.053597+0000","last_became_active":"2024-06-03T03:19:41.620719+0000","last_became_peered":"2024-06-03T03:19:41.620719+0000","last_unstale":"2024-06-03T03:21:26.053597+0000","last_undegraded":"2024-06-03T03:21:26.053597+0000","last_fullsized":"2024-06-03T03:21:26.053597+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:50:35.007025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":28,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.052222+0000","last_change":"2024-06-03T03:19:56.915406+0000","last_active":"2024-06-03T03:21:26.052222+0000","last_peered":"2024-06-03T03:21:26.052222+0000","last_clean":"2024-06-03T03:21:26.052222+0000","last_became_active":"2024-06-03T03:19:56.914918+0000","last_became_peered":"2024-06-03T03:19:56.914918+0000","last_unstale":"2024-06-03T03:21:26.052222+0000","last_undegraded":"2024-06-03T03:21:26.052222+0000","last_fullsized":"2024-06-03T03:21:26.052222+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:42:59.324011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"97'5","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.979476+0000","last_change":"2024-06-03T03:19:36.549502+0000","last_active":"2024-06-03T03:21:25.979476+0000","last_peered":"2024-06-03T03:21:25.979476+0000","last_clean":"2024-06-03T03:21:25.979476+0000","last_became_active":"2024-06-03T03:19:36.547538+0000","last_became_peered":"2024-06-03T03:19:36.547538+0000","last_unstale":"2024-06-03T03:21:25.979476+0000","last_undegraded":"2024-06-03T03:21:25.979476+0000","last_fullsized":"2024-06-03T03:21:25.979476+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:28:50.558069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.054070+0000","last_change":"2024-06-03T03:19:41.607441+0000","last_active":"2024-06-03T03:21:26.054070+0000","last_peered":"2024-06-03T03:21:26.054070+0000","last_clean":"2024-06-03T03:21:26.054070+0000","last_became_active":"2024-06-03T03:19:41.607086+0000","last_became_peered":"2024-06-03T03:19:41.607086+0000","last_unstale":"2024-06-03T03:21:26.054070+0000","last_undegraded":"2024-06-03T03:21:26.054070+0000","last_fullsized":"2024-06-03T03:21:26.054070+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T14:53:50.091872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.053930+0000","last_change":"2024-06-03T03:19:41.607004+0000","last_active":"2024-06-03T03:21:26.053930+0000","last_peered":"2024-06-03T03:21:26.053930+0000","last_clean":"2024-06-03T03:21:26.053930+0000","last_became_active":"2024-06-03T03:19:41.606684+0000","last_became_peered":"2024-06-03T03:19:41.606684+0000","last_unstale":"2024-06-03T03:21:26.053930+0000","last_undegraded":"2024-06-03T03:21:26.053930+0000","last_fullsized":"2024-06-03T03:21:26.053930+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:48:55.320597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.052544+0000","last_change":"2024-06-03T03:19:41.589426+0000","last_active":"2024-06-03T03:21:26.052544+0000","last_peered":"2024-06-03T03:21:26.052544+0000","last_clean":"2024-06-03T03:21:26.052544+0000","last_became_active":"2024-06-03T03:19:41.589091+0000","last_became_peered":"2024-06-03T03:19:41.589091+0000","last_unstale":"2024-06-03T03:21:26.052544+0000","last_undegraded":"2024-06-03T03:21:26.052544+0000","last_fullsized":"2024-06-03T03:21:26.052544+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:40:34.313214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.052322+0000","last_change":"2024-06-03T03:19:41.626149+0000","last_active":"2024-06-03T03:21:26.052322+0000","last_peered":"2024-06-03T03:21:26.052322+0000","last_clean":"2024-06-03T03:21:26.052322+0000","last_became_active":"2024-06-03T03:19:41.625710+0000","last_became_peered":"2024-06-03T03:19:41.625710+0000","last_unstale":"2024-06-03T03:21:26.052322+0000","last_undegraded":"2024-06-03T03:21:26.052322+0000","last_fullsized":"2024-06-03T03:21:26.052322+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:15:15.220975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.976354+0000","last_change":"2024-06-03T03:19:41.636585+0000","last_active":"2024-06-03T03:21:25.976354+0000","last_peered":"2024-06-03T03:21:25.976354+0000","last_clean":"2024-06-03T03:21:25.976354+0000","last_became_active":"2024-06-03T03:19:41.636241+0000","last_became_peered":"2024-06-03T03:19:41.636241+0000","last_unstale":"2024-06-03T03:21:25.976354+0000","last_undegraded":"2024-06-03T03:21:25.976354+0000","last_fullsized":"2024-06-03T03:21:25.976354+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:19:28.548297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.978614+0000","last_change":"2024-06-03T03:19:41.629530+0000","last_active":"2024-06-03T03:21:25.978614+0000","last_peered":"2024-06-03T03:21:25.978614+0000","last_clean":"2024-06-03T03:21:25.978614+0000","last_became_active":"2024-06-03T03:19:41.629197+0000","last_became_peered":"2024-06-03T03:19:41.629197+0000","last_unstale":"2024-06-03T03:21:25.978614+0000","last_undegraded":"2024-06-03T03:21:25.978614+0000","last_fullsized":"2024-06-03T03:21:25.978614+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:12:02.807445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.053577+0000","last_change":"2024-06-03T03:19:41.606995+0000","last_active":"2024-06-03T03:21:26.053577+0000","last_peered":"2024-06-03T03:21:26.053577+0000","last_clean":"2024-06-03T03:21:26.053577+0000","last_became_active":"2024-06-03T03:19:41.606651+0000","last_became_peered":"2024-06-03T03:19:41.606651+0000","last_unstale":"2024-06-03T03:21:26.053577+0000","last_undegraded":"2024-06-03T03:21:26.053577+0000","last_fullsized":"2024-06-03T03:21:26.053577+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:07:11.751606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.053366+0000","last_change":"2024-06-03T03:19:41.609848+0000","last_active":"2024-06-03T03:21:26.053366+0000","last_peered":"2024-06-03T03:21:26.053366+0000","last_clean":"2024-06-03T03:21:26.053366+0000","last_became_active":"2024-06-03T03:19:41.609544+0000","last_became_peered":"2024-06-03T03:19:41.609544+0000","last_unstale":"2024-06-03T03:21:26.053366+0000","last_undegraded":"2024-06-03T03:21:26.053366+0000","last_fullsized":"2024-06-03T03:21:26.053366+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:12:00.804638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":43,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.059089+0000","last_change":"2024-06-03T03:20:25.912049+0000","last_active":"2024-06-03T03:21:26.059089+0000","last_peered":"2024-06-03T03:21:26.059089+0000","last_clean":"2024-06-03T03:21:26.059089+0000","last_became_active":"2024-06-03T03:20:25.910664+0000","last_became_peered":"2024-06-03T03:20:25.910664+0000","last_unstale":"2024-06-03T03:21:26.059089+0000","last_undegraded":"2024-06-03T03:21:26.059089+0000","last_fullsized":"2024-06-03T03:21:26.059089+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:47:19.282569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.052469+0000","last_change":"2024-06-03T03:19:41.610800+0000","last_active":"2024-06-03T03:21:26.052469+0000","last_peered":"2024-06-03T03:21:26.052469+0000","last_clean":"2024-06-03T03:21:26.052469+0000","last_became_active":"2024-06-03T03:19:41.610457+0000","last_became_peered":"2024-06-03T03:19:41.610457+0000","last_unstale":"2024-06-03T03:21:26.052469+0000","last_undegraded":"2024-06-03T03:21:26.052469+0000","last_fullsized":"2024-06-03T03:21:26.052469+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:31:06.415159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.056986+0000","last_change":"2024-06-03T03:19:41.617020+0000","last_active":"2024-06-03T03:21:26.056986+0000","last_peered":"2024-06-03T03:21:26.056986+0000","last_clean":"2024-06-03T03:21:26.056986+0000","last_became_active":"2024-06-03T03:19:41.616614+0000","last_became_peered":"2024-06-03T03:19:41.616614+0000","last_unstale":"2024-06-03T03:21:26.056986+0000","last_undegraded":"2024-06-03T03:21:26.056986+0000","last_fullsized":"2024-06-03T03:21:26.056986+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:35:32.657685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":38,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.053252+0000","last_change":"2024-06-03T03:19:41.622785+0000","last_active":"2024-06-03T03:21:26.053252+0000","last_peered":"2024-06-03T03:21:26.053252+0000","last_clean":"2024-06-03T03:21:26.053252+0000","last_became_active":"2024-06-03T03:19:41.622407+0000","last_became_peered":"2024-06-03T03:19:41.622407+0000","last_unstale":"2024-06-03T03:21:26.053252+0000","last_undegraded":"2024-06-03T03:21:26.053252+0000","last_fullsized":"2024-06-03T03:21:26.053252+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:53:50.804889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.978017+0000","last_change":"2024-06-03T03:19:41.633429+0000","last_active":"2024-06-03T03:21:25.978017+0000","last_peered":"2024-06-03T03:21:25.978017+0000","last_clean":"2024-06-03T03:21:25.978017+0000","last_became_active":"2024-06-03T03:19:41.631028+0000","last_became_peered":"2024-06-03T03:19:41.631028+0000","last_unstale":"2024-06-03T03:21:25.978017+0000","last_undegraded":"2024-06-03T03:21:25.978017+0000","last_fullsized":"2024-06-03T03:21:25.978017+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:37:10.666591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.053936+0000","last_change":"2024-06-03T03:19:41.606204+0000","last_active":"2024-06-03T03:21:26.053936+0000","last_peered":"2024-06-03T03:21:26.053936+0000","last_clean":"2024-06-03T03:21:26.053936+0000","last_became_active":"2024-06-03T03:19:41.605756+0000","last_became_peered":"2024-06-03T03:19:41.605756+0000","last_unstale":"2024-06-03T03:21:26.053936+0000","last_undegraded":"2024-06-03T03:21:26.053936+0000","last_fullsized":"2024-06-03T03:21:26.053936+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:24:32.507426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":245318,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":74629,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":433,"num_read_kb":298,"num_write":255,"num_write_kb":147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":223887,"data_compressed":5580,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":218,"ondisk_log_size":218,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383721,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32260,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712892,"statfs":{"total":95995035648,"available":95962001408,"internally_reserved":0,"allocated":1101824,"data_stored":570375,"data_compressed":32353,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645359,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32400,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712752,"statfs":{"total":95995035648,"available":95961858048,"internally_reserved":0,"allocated":1253376,"data_stored":1225008,"data_compressed":69963,"data_compressed_allocated":94208,"data_compressed_original":688128,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808884,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32280,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712872,"statfs":{"total":95995035648,"available":95961980928,"internally_reserved":0,"allocated":1130496,"data_stored":574401,"data_compressed":32353,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070523,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32272,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712880,"statfs":{"total":95995035648,"available":95961989120,"internally_reserved":0,"allocated":1122304,"data_stored":570683,"data_compressed":32353,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332160,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32324,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712828,"statfs":{"total":95995035648,"available":95961935872,"internally_reserved":0,"allocated":1175552,"data_stored":1167179,"data_compressed":68103,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495687,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1101824,"data_stored":565809,"data_compressed":32353,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574946,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28160,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716992,"statfs":{"total":95995035648,"available":95966199808,"internally_reserved":0,"allocated":1171456,"data_stored":634656,"data_compressed":34213,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738470,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28124,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717028,"statfs":{"total":95995035648,"available":95966236672,"internally_reserved":0,"allocated":1126400,"data_stored":567265,"data_compressed":32353,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411420,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36492,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708660,"statfs":{"total":95995035648,"available":95957667840,"internally_reserved":0,"allocated":1183744,"data_stored":634096,"data_compressed":34213,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182487,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1114112,"data_stored":565517,"data_compressed":32353,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920850,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28108,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717044,"statfs":{"total":95995035648,"available":95966253056,"internally_reserved":0,"allocated":1118208,"data_stored":571053,"data_compressed":32353,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757325,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32340,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712812,"statfs":{"total":95995035648,"available":95961919488,"internally_reserved":0,"allocated":1183744,"data_stored":1162761,"data_compressed":68103,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34106,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34106,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34106,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35750,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35750,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35750,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35750,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":69569,"data_compressed":1860,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":69009,"data_compressed":1860,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":69553,"data_compressed":1860,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-03T03:21:50.493 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-06-03T03:21:50.493 INFO:tasks.cephadm:Setting up client nodes... 2024-06-03T03:21:50.494 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-06-03T03:21:50.970 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:21:51.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:50 smithi183 ceph-mon[37326]: from='client.24728 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-03T03:21:51.396 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:50 smithi117 ceph-mon[31649]: from='client.24728 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-03T03:21:51.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:50 smithi151 ceph-mon[37043]: from='client.24728 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-03T03:21:52.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:51 smithi183 ceph-mon[37326]: pgmap v402: 129 pgs: 129 active+clean; 650 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1023 B/s wr, 9 op/s 2024-06-03T03:21:52.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:51 smithi151 ceph-mon[37043]: pgmap v402: 129 pgs: 129 active+clean; 650 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1023 B/s wr, 9 op/s 2024-06-03T03:21:52.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:51 smithi117 ceph-mon[31649]: pgmap v402: 129 pgs: 129 active+clean; 650 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1023 B/s wr, 9 op/s 2024-06-03T03:21:53.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:53.482+0000 7f0288600700 1 -- 172.21.15.117:0/2004330414 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0280103710 msgr2=0x7f0280103b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:21:53.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:53.482+0000 7f0288600700 1 --2- 172.21.15.117:0/2004330414 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0280103710 0x7f0280103b70 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f026c0099c0 tx=0x7f026c009cd0 comp rx=0 tx=0).stop 2024-06-03T03:21:53.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:53.482+0000 7f0288600700 1 -- 172.21.15.117:0/2004330414 shutdown_connections 2024-06-03T03:21:53.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:53.482+0000 7f0288600700 1 --2- 172.21.15.117:0/2004330414 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02801040b0 0x7f028010a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:53.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:53.482+0000 7f0288600700 1 --2- 172.21.15.117:0/2004330414 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0280102510 0x7f0280102930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:53.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:53.482+0000 7f0288600700 1 --2- 172.21.15.117:0/2004330414 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0280103710 0x7f0280103b70 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:53.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:53.482+0000 7f0288600700 1 -- 172.21.15.117:0/2004330414 >> 172.21.15.117:0/2004330414 conn(0x7f02800fdab0 msgr2=0x7f02800ffef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:21:53.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:53.483+0000 7f0288600700 1 -- 172.21.15.117:0/2004330414 shutdown_connections 2024-06-03T03:21:53.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:53.483+0000 7f0288600700 1 -- 172.21.15.117:0/2004330414 wait complete. 2024-06-03T03:21:53.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:53.484+0000 7f0288600700 1 Processor -- start 2024-06-03T03:21:53.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:53.484+0000 7f0288600700 1 -- start start 2024-06-03T03:21:53.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:53.484+0000 7f0288600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0280102510 0x7f028019a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:21:53.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:53.484+0000 7f0288600700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0280103710 0x7f028019ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:21:53.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:53.484+0000 7f0288600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02801040b0 0x7f02801a1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:21:53.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:53.484+0000 7f0288600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f028010e140 con 0x7f02801040b0 2024-06-03T03:21:53.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:53.484+0000 7f0288600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f028010dfc0 con 0x7f0280102510 2024-06-03T03:21:53.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:53.484+0000 7f0288600700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f028010e2c0 con 0x7f0280103710 2024-06-03T03:21:53.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:53.484+0000 7f0285200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0280103710 0x7f028019ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:21:53.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:53.484+0000 7f0285200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0280103710 0x7f028019ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:37502/0 (socket says 172.21.15.117:37502) 2024-06-03T03:21:53.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:53.484+0000 7f0285200700 1 -- 172.21.15.117:0/2027473240 learned_addr learned my addr 172.21.15.117:0/2027473240 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:21:53.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:53.484+0000 7f0286600700 1 --2- 172.21.15.117:0/2027473240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02801040b0 0x7f02801a1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:21:53.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:53.485+0000 7f0285c00700 1 --2- 172.21.15.117:0/2027473240 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0280102510 0x7f028019a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:21:53.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:53.485+0000 7f0285200700 1 -- 172.21.15.117:0/2027473240 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0280102510 msgr2=0x7f028019a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:21:53.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:53.485+0000 7f0285200700 1 --2- 172.21.15.117:0/2027473240 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0280102510 0x7f028019a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:53.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:53.485+0000 7f0285200700 1 -- 172.21.15.117:0/2027473240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02801040b0 msgr2=0x7f02801a1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:21:53.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:53.485+0000 7f0285200700 1 --2- 172.21.15.117:0/2027473240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02801040b0 0x7f02801a1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:53.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:53.485+0000 7f0285200700 1 -- 172.21.15.117:0/2027473240 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02801a2550 con 0x7f0280103710 2024-06-03T03:21:53.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:53.485+0000 7f0285200700 1 --2- 172.21.15.117:0/2027473240 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0280103710 0x7f028019ae30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f026c00f690 tx=0x7f026c009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:21:53.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:53.486+0000 7f027e000700 1 -- 172.21.15.117:0/2027473240 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f026c027070 con 0x7f0280103710 2024-06-03T03:21:53.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:53.486+0000 7f0288600700 1 -- 172.21.15.117:0/2027473240 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02801a27e0 con 0x7f0280103710 2024-06-03T03:21:53.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:53.486+0000 7f0288600700 1 -- 172.21.15.117:0/2027473240 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02801a2c90 con 0x7f0280103710 2024-06-03T03:21:53.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:53.486+0000 7f027e000700 1 -- 172.21.15.117:0/2027473240 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f026c022470 con 0x7f0280103710 2024-06-03T03:21:53.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:53.486+0000 7f027e000700 1 -- 172.21.15.117:0/2027473240 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f026c009e60 con 0x7f0280103710 2024-06-03T03:21:53.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:53.487+0000 7f0288600700 1 -- 172.21.15.117:0/2027473240 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0280050a00 con 0x7f0280103710 2024-06-03T03:21:53.489 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:53.489+0000 7f027e000700 1 -- 172.21.15.117:0/2027473240 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f026c02a070 con 0x7f0280103710 2024-06-03T03:21:53.491 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:53.490+0000 7f027e000700 1 --2- 172.21.15.117:0/2027473240 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f02680a2de0 0x7f02680a52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:21:53.491 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:53.490+0000 7f027e000700 1 -- 172.21.15.117:0/2027473240 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f026c0d03e0 con 0x7f0280103710 2024-06-03T03:21:53.491 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:53.491+0000 7f0285c00700 1 --2- 172.21.15.117:0/2027473240 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f02680a2de0 0x7f02680a52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:21:53.492 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:53.491+0000 7f0285c00700 1 --2- 172.21.15.117:0/2027473240 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f02680a2de0 0x7f02680a52a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0280103570 tx=0x7f0274009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:21:53.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:53.493+0000 7f027e000700 1 -- 172.21.15.117:0/2027473240 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f026c013070 con 0x7f0280103710 2024-06-03T03:21:53.806 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:53.805+0000 7f0288600700 1 -- 172.21.15.117:0/2027473240 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f02801a30b0 con 0x7f0280103710 2024-06-03T03:21:53.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:53.812+0000 7f027e000700 1 -- 172.21.15.117:0/2027473240 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f026c09ce80 con 0x7f0280103710 2024-06-03T03:21:53.813 INFO:teuthology.orchestra.run.smithi117.stdout:[client.0] 2024-06-03T03:21:53.814 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQDRNl1mDXwiMBAA0S6k3oi28wH9GR6xpTL1Pg== 2024-06-03T03:21:53.816 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:53.815+0000 7f0288600700 1 -- 172.21.15.117:0/2027473240 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f02680a2de0 msgr2=0x7f02680a52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:21:53.816 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:53.815+0000 7f0288600700 1 --2- 172.21.15.117:0/2027473240 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f02680a2de0 0x7f02680a52a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0280103570 tx=0x7f0274009260 comp rx=0 tx=0).stop 2024-06-03T03:21:53.816 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:53.815+0000 7f0288600700 1 -- 172.21.15.117:0/2027473240 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0280103710 msgr2=0x7f028019ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:21:53.816 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:53.815+0000 7f0288600700 1 --2- 172.21.15.117:0/2027473240 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0280103710 0x7f028019ae30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f026c00f690 tx=0x7f026c009580 comp rx=0 tx=0).stop 2024-06-03T03:21:53.817 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:53.816+0000 7f0288600700 1 -- 172.21.15.117:0/2027473240 shutdown_connections 2024-06-03T03:21:53.817 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:53.816+0000 7f0288600700 1 --2- 172.21.15.117:0/2027473240 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f02680a2de0 0x7f02680a52a0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:53.817 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:53.816+0000 7f0288600700 1 --2- 172.21.15.117:0/2027473240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02801040b0 0x7f02801a1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:53.817 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:53.816+0000 7f0288600700 1 --2- 172.21.15.117:0/2027473240 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0280102510 0x7f028019a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:53.817 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:53.816+0000 7f0288600700 1 --2- 172.21.15.117:0/2027473240 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0280103710 0x7f028019ae30 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:53.817 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:53.816+0000 7f0288600700 1 -- 172.21.15.117:0/2027473240 >> 172.21.15.117:0/2027473240 conn(0x7f02800fdab0 msgr2=0x7f02800746c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:21:53.818 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:53.816+0000 7f0288600700 1 -- 172.21.15.117:0/2027473240 shutdown_connections 2024-06-03T03:21:53.818 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:53.816+0000 7f0288600700 1 -- 172.21.15.117:0/2027473240 wait complete. 2024-06-03T03:21:54.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:53 smithi117 ceph-mon[31649]: pgmap v403: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-06-03T03:21:54.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:53 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/2027473240' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-03T03:21:54.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:53 smithi117 ceph-mon[31649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-03T03:21:54.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:53 smithi117 ceph-mon[31649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-03T03:21:54.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:53 smithi151 ceph-mon[37043]: pgmap v403: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-06-03T03:21:54.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:53 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/2027473240' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-03T03:21:54.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:53 smithi151 ceph-mon[37043]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-03T03:21:54.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:53 smithi151 ceph-mon[37043]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-03T03:21:54.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:53 smithi183 ceph-mon[37326]: pgmap v403: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-06-03T03:21:54.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:53 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/2027473240' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-03T03:21:54.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:53 smithi183 ceph-mon[37326]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-03T03:21:54.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:53 smithi183 ceph-mon[37326]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-03T03:21:54.705 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-06-03T03:21:54.705 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-06-03T03:21:54.705 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-06-03T03:21:54.833 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-06-03T03:21:54.833 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-06-03T03:21:54.834 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph mgr dump --format=json 2024-06-03T03:21:55.329 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:21:56.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:55 smithi151 ceph-mon[37043]: pgmap v404: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 9 op/s 2024-06-03T03:21:56.428 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:55 smithi117 ceph-mon[31649]: pgmap v404: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 9 op/s 2024-06-03T03:21:56.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:55 smithi183 ceph-mon[37326]: pgmap v404: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 9 op/s 2024-06-03T03:21:57.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:57.721+0000 7ff0fb400700 1 -- 172.21.15.117:0/2659504583 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff0f4102550 msgr2=0x7ff0f41029d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:21:57.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:57.721+0000 7ff0fb400700 1 --2- 172.21.15.117:0/2659504583 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff0f4102550 0x7ff0f41029d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff0e00099c0 tx=0x7ff0e0009cd0 comp rx=0 tx=0).stop 2024-06-03T03:21:57.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:57.721+0000 7ff0fb400700 1 -- 172.21.15.117:0/2659504583 shutdown_connections 2024-06-03T03:21:57.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:57.721+0000 7ff0fb400700 1 --2- 172.21.15.117:0/2659504583 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0f4102f10 0x7ff0f410f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:57.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:57.721+0000 7ff0fb400700 1 --2- 172.21.15.117:0/2659504583 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0f41073a0 0x7ff0f4107780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:57.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:57.721+0000 7ff0fb400700 1 --2- 172.21.15.117:0/2659504583 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff0f4102550 0x7ff0f41029d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:57.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:57.721+0000 7ff0fb400700 1 -- 172.21.15.117:0/2659504583 >> 172.21.15.117:0/2659504583 conn(0x7ff0f4079650 msgr2=0x7ff0f4079a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:21:57.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:57.721+0000 7ff0fb400700 1 -- 172.21.15.117:0/2659504583 shutdown_connections 2024-06-03T03:21:57.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:57.722+0000 7ff0fb400700 1 -- 172.21.15.117:0/2659504583 wait complete. 2024-06-03T03:21:57.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:57.723+0000 7ff0fb400700 1 Processor -- start 2024-06-03T03:21:57.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:57.723+0000 7ff0fb400700 1 -- start start 2024-06-03T03:21:57.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:57.723+0000 7ff0fb400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff0f4102550 0x7ff0f419e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:21:57.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:57.723+0000 7ff0fb400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0f4102f10 0x7ff0f419ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:21:57.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:57.723+0000 7ff0fb400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0f41073a0 0x7ff0f4198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:21:57.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:57.723+0000 7ff0fb400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f41127d0 con 0x7ff0f4102f10 2024-06-03T03:21:57.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:57.723+0000 7ff0fb400700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f4112650 con 0x7ff0f4102550 2024-06-03T03:21:57.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:57.723+0000 7ff0fb400700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f4112950 con 0x7ff0f41073a0 2024-06-03T03:21:57.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:57.724+0000 7ff0f3e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0f4102f10 0x7ff0f419ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:21:57.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:57.724+0000 7ff0f9400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0f41073a0 0x7ff0f4198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:21:57.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:57.724+0000 7ff0f8a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff0f4102550 0x7ff0f419e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:21:57.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:57.724+0000 7ff0f9400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0f41073a0 0x7ff0f4198af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:55480/0 (socket says 172.21.15.117:55480) 2024-06-03T03:21:57.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:57.724+0000 7ff0f3e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0f4102f10 0x7ff0f419ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:43404/0 (socket says 172.21.15.117:43404) 2024-06-03T03:21:57.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:57.724+0000 7ff0f9400700 1 -- 172.21.15.117:0/1682017762 learned_addr learned my addr 172.21.15.117:0/1682017762 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:21:57.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:57.724+0000 7ff0f8a00700 1 -- 172.21.15.117:0/1682017762 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0f41073a0 msgr2=0x7ff0f4198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:21:57.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:57.724+0000 7ff0f8a00700 1 --2- 172.21.15.117:0/1682017762 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0f41073a0 0x7ff0f4198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:57.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:57.724+0000 7ff0f8a00700 1 -- 172.21.15.117:0/1682017762 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0f4102f10 msgr2=0x7ff0f419ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:21:57.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:57.724+0000 7ff0f8a00700 1 --2- 172.21.15.117:0/1682017762 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0f4102f10 0x7ff0f419ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:57.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:57.724+0000 7ff0f8a00700 1 -- 172.21.15.117:0/1682017762 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0f4199250 con 0x7ff0f4102550 2024-06-03T03:21:57.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:57.724+0000 7ff0f8a00700 1 --2- 172.21.15.117:0/1682017762 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff0f4102550 0x7ff0f419e930 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff0e800e9f0 tx=0x7ff0e800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:21:57.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:57.725+0000 7ff0f1600700 1 -- 172.21.15.117:0/1682017762 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e8004220 con 0x7ff0f4102550 2024-06-03T03:21:57.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:57.725+0000 7ff0f1600700 1 -- 172.21.15.117:0/1682017762 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0e8004d10 con 0x7ff0f4102550 2024-06-03T03:21:57.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:57.725+0000 7ff0fb400700 1 -- 172.21.15.117:0/1682017762 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0f4199540 con 0x7ff0f4102550 2024-06-03T03:21:57.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:57.725+0000 7ff0f1600700 1 -- 172.21.15.117:0/1682017762 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e8005020 con 0x7ff0f4102550 2024-06-03T03:21:57.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:57.725+0000 7ff0fb400700 1 -- 172.21.15.117:0/1682017762 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0f41a5800 con 0x7ff0f4102550 2024-06-03T03:21:57.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:57.726+0000 7ff0fb400700 1 -- 172.21.15.117:0/1682017762 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0f4103b80 con 0x7ff0f4102550 2024-06-03T03:21:57.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:57.728+0000 7ff0f1600700 1 -- 172.21.15.117:0/1682017762 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7ff0e8004750 con 0x7ff0f4102550 2024-06-03T03:21:57.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:57.729+0000 7ff0f1600700 1 --2- 172.21.15.117:0/1682017762 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7ff0dc0a2d30 0x7ff0dc0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:21:57.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:57.730+0000 7ff0f1600700 1 -- 172.21.15.117:0/1682017762 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7ff0e80cf440 con 0x7ff0f4102550 2024-06-03T03:21:57.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:57.730+0000 7ff0f3e00700 1 --2- 172.21.15.117:0/1682017762 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7ff0dc0a2d30 0x7ff0dc0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:21:57.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:57.730+0000 7ff0f3e00700 1 --2- 172.21.15.117:0/1682017762 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7ff0dc0a2d30 0x7ff0dc0a51f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff0f4079c30 tx=0x7ff0e0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:21:57.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:57.732+0000 7ff0f1600700 1 -- 172.21.15.117:0/1682017762 <== mon.2 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff0e809bee0 con 0x7ff0f4102550 2024-06-03T03:21:58.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:58.021+0000 7ff0fb400700 1 -- 172.21.15.117:0/1682017762 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7ff0f4199fe0 con 0x7ff0f4102550 2024-06-03T03:21:58.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:58.027+0000 7ff0f1600700 1 -- 172.21.15.117:0/1682017762 <== mon.2 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+263210 (secure 0 0 0) 0x7ff0e809b7e0 con 0x7ff0f4102550 2024-06-03T03:21:58.033 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T03:21:58.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:58.032+0000 7ff0fb400700 1 -- 172.21.15.117:0/1682017762 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7ff0dc0a2d30 msgr2=0x7ff0dc0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:21:58.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:58.032+0000 7ff0fb400700 1 --2- 172.21.15.117:0/1682017762 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7ff0dc0a2d30 0x7ff0dc0a51f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff0f4079c30 tx=0x7ff0e0005fb0 comp rx=0 tx=0).stop 2024-06-03T03:21:58.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:58.032+0000 7ff0fb400700 1 -- 172.21.15.117:0/1682017762 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff0f4102550 msgr2=0x7ff0f419e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:21:58.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:58.032+0000 7ff0fb400700 1 --2- 172.21.15.117:0/1682017762 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff0f4102550 0x7ff0f419e930 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff0e800e9f0 tx=0x7ff0e800ed00 comp rx=0 tx=0).stop 2024-06-03T03:21:58.036 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:58.032+0000 7ff0fb400700 1 -- 172.21.15.117:0/1682017762 shutdown_connections 2024-06-03T03:21:58.036 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:58.032+0000 7ff0fb400700 1 --2- 172.21.15.117:0/1682017762 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7ff0dc0a2d30 0x7ff0dc0a51f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:58.036 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:58.032+0000 7ff0fb400700 1 --2- 172.21.15.117:0/1682017762 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0f41073a0 0x7ff0f4198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:58.036 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:58.032+0000 7ff0fb400700 1 --2- 172.21.15.117:0/1682017762 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff0f4102550 0x7ff0f419e930 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:58.036 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:58.032+0000 7ff0fb400700 1 --2- 172.21.15.117:0/1682017762 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0f4102f10 0x7ff0f419ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:21:58.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:58.032+0000 7ff0fb400700 1 -- 172.21.15.117:0/1682017762 >> 172.21.15.117:0/1682017762 conn(0x7ff0f4079650 msgr2=0x7ff0f41005c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:21:58.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:58.032+0000 7ff0fb400700 1 -- 172.21.15.117:0/1682017762 shutdown_connections 2024-06-03T03:21:58.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:21:58.032+0000 7ff0fb400700 1 -- 172.21.15.117:0/1682017762 wait complete. 2024-06-03T03:21:58.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:57 smithi151 ceph-mon[37043]: pgmap v405: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 9 op/s 2024-06-03T03:21:58.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:57 smithi117 ceph-mon[31649]: pgmap v405: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 9 op/s 2024-06-03T03:21:58.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:57 smithi183 ceph-mon[37326]: pgmap v405: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 9 op/s 2024-06-03T03:21:59.029 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":1398160887},{"type":"v1","addr":"172.21.15.117:6801","nonce":1398160887}]},"active_addr":"172.21.15.117:6801/1398160887","active_change":"2024-06-03T03:11:24.655863+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.117:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.117:0","nonce":1130602469}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.117:0","nonce":1908883785}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.117:0","nonce":2581180598}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.117:0","nonce":2352316279}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.117:0","nonce":2476075325}]}]} 2024-06-03T03:21:59.034 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-06-03T03:21:59.034 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-06-03T03:21:59.035 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-03T03:21:59.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:21:59 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/1682017762' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-06-03T03:21:59.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:21:59 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/1682017762' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-06-03T03:21:59.466 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:21:59.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:21:59 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/1682017762' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-06-03T03:22:00.132 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:22:00 smithi117 ceph-mon[31649]: pgmap v406: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 682 B/s wr, 12 op/s 2024-06-03T03:22:00.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:22:00 smithi151 ceph-mon[37043]: pgmap v406: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 682 B/s wr, 12 op/s 2024-06-03T03:22:00.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:22:00 smithi183 ceph-mon[37326]: pgmap v406: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 682 B/s wr, 12 op/s 2024-06-03T03:22:01.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:22:01 smithi117 ceph-mon[31649]: pgmap v407: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 8 op/s 2024-06-03T03:22:01.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:22:01 smithi151 ceph-mon[37043]: pgmap v407: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 8 op/s 2024-06-03T03:22:01.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:22:01 smithi183 ceph-mon[37326]: pgmap v407: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 8 op/s 2024-06-03T03:22:01.787 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:01.782+0000 7f54ade00700 1 -- 172.21.15.117:0/3139464859 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54a8101860 msgr2=0x7f54a810d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:01.788 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:01.782+0000 7f54ade00700 1 --2- 172.21.15.117:0/3139464859 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54a8101860 0x7f54a810d990 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f54a000b0d0 tx=0x7f54a000b3e0 comp rx=0 tx=0).stop 2024-06-03T03:22:01.788 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:01.782+0000 7f54ade00700 1 -- 172.21.15.117:0/3139464859 shutdown_connections 2024-06-03T03:22:01.788 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:01.782+0000 7f54ade00700 1 --2- 172.21.15.117:0/3139464859 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54a8101860 0x7f54a810d990 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:01.788 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:01.782+0000 7f54ade00700 1 --2- 172.21.15.117:0/3139464859 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f54a8106b30 0x7f54a8106f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:01.789 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:01.782+0000 7f54ade00700 1 --2- 172.21.15.117:0/3139464859 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54a8100ea0 0x7f54a8101320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:01.789 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:01.782+0000 7f54ade00700 1 -- 172.21.15.117:0/3139464859 >> 172.21.15.117:0/3139464859 conn(0x7f54a8077590 msgr2=0x7f54a80fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:22:01.789 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:01.782+0000 7f54ade00700 1 -- 172.21.15.117:0/3139464859 shutdown_connections 2024-06-03T03:22:01.789 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:01.782+0000 7f54ade00700 1 -- 172.21.15.117:0/3139464859 wait complete. 2024-06-03T03:22:01.789 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:01.783+0000 7f54ade00700 1 Processor -- start 2024-06-03T03:22:01.789 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:01.783+0000 7f54ade00700 1 -- start start 2024-06-03T03:22:01.789 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:01.784+0000 7f54ade00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54a8100ea0 0x7f54a80fca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:01.790 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:01.784+0000 7f54ade00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54a8101860 0x7f54a80fcf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:01.790 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:01.784+0000 7f54ade00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f54a8106b30 0x7f54a80fd480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:01.790 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:01.784+0000 7f54ade00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f54a81105e0 con 0x7f54a8101860 2024-06-03T03:22:01.790 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:01.784+0000 7f54ade00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f54a8110460 con 0x7f54a8106b30 2024-06-03T03:22:01.790 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:01.784+0000 7f54ade00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f54a8110760 con 0x7f54a8100ea0 2024-06-03T03:22:01.790 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:01.784+0000 7f54a6000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54a8101860 0x7f54a80fcf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:01.791 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:01.784+0000 7f54a6000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54a8101860 0x7f54a80fcf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:43420/0 (socket says 172.21.15.117:43420) 2024-06-03T03:22:01.791 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:01.784+0000 7f54a6000700 1 -- 172.21.15.117:0/3803295834 learned_addr learned my addr 172.21.15.117:0/3803295834 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:22:01.792 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:01.784+0000 7f54a6a00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54a8100ea0 0x7f54a80fca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:01.792 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:01.784+0000 7f54a7400700 1 --2- 172.21.15.117:0/3803295834 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f54a8106b30 0x7f54a80fd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:01.792 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:01.784+0000 7f54a6000700 1 -- 172.21.15.117:0/3803295834 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54a8100ea0 msgr2=0x7f54a80fca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:01.792 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:01.784+0000 7f54a6000700 1 --2- 172.21.15.117:0/3803295834 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54a8100ea0 0x7f54a80fca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:01.792 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:01.784+0000 7f54a6000700 1 -- 172.21.15.117:0/3803295834 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f54a8106b30 msgr2=0x7f54a80fd480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:01.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:01.784+0000 7f54a6000700 1 --2- 172.21.15.117:0/3803295834 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f54a8106b30 0x7f54a80fd480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:01.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:01.784+0000 7f54a6000700 1 -- 172.21.15.117:0/3803295834 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54a81a7100 con 0x7f54a8101860 2024-06-03T03:22:01.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:01.784+0000 7f54a7400700 1 --2- 172.21.15.117:0/3803295834 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f54a8106b30 0x7f54a80fd480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:22:01.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:01.785+0000 7f54a6000700 1 --2- 172.21.15.117:0/3803295834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54a8101860 0x7f54a80fcf40 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f549400ead0 tx=0x7f549400ede0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:22:01.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:01.785+0000 7f54aca00700 1 -- 172.21.15.117:0/3803295834 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f549400ccd0 con 0x7f54a8101860 2024-06-03T03:22:01.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:01.785+0000 7f54aca00700 1 -- 172.21.15.117:0/3803295834 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5494004d10 con 0x7f54a8101860 2024-06-03T03:22:01.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:01.785+0000 7f54ade00700 1 -- 172.21.15.117:0/3803295834 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54a81a73f0 con 0x7f54a8101860 2024-06-03T03:22:01.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:01.785+0000 7f54aca00700 1 -- 172.21.15.117:0/3803295834 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54940155f0 con 0x7f54a8101860 2024-06-03T03:22:01.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:01.785+0000 7f54ade00700 1 -- 172.21.15.117:0/3803295834 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54a81a7930 con 0x7f54a8101860 2024-06-03T03:22:01.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:01.787+0000 7f54ade00700 1 -- 172.21.15.117:0/3803295834 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54a8102200 con 0x7f54a8101860 2024-06-03T03:22:01.794 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:01.788+0000 7f54aca00700 1 -- 172.21.15.117:0/3803295834 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f5494004750 con 0x7f54a8101860 2024-06-03T03:22:01.794 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:01.790+0000 7f54aca00700 1 --2- 172.21.15.117:0/3803295834 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f548c0a2d30 0x7f548c0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:01.794 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:01.790+0000 7f54aca00700 1 -- 172.21.15.117:0/3803295834 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f54940cf3b0 con 0x7f54a8101860 2024-06-03T03:22:01.795 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:01.790+0000 7f54a6a00700 1 --2- 172.21.15.117:0/3803295834 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f548c0a2d30 0x7f548c0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:01.795 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:01.791+0000 7f54a6a00700 1 --2- 172.21.15.117:0/3803295834 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f548c0a2d30 0x7f548c0a51f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5498005950 tx=0x7f549800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:22:01.795 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:01.795+0000 7f54aca00700 1 -- 172.21.15.117:0/3803295834 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f549409be50 con 0x7f54a8101860 2024-06-03T03:22:02.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:02.021+0000 7f54ade00700 1 -- 172.21.15.117:0/3803295834 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f54a802dca0 con 0x7f54a8101860 2024-06-03T03:22:02.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:02.025+0000 7f54aca00700 1 -- 172.21.15.117:0/3803295834 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+24807 (secure 0 0 0) 0x7f549409b750 con 0x7f54a8101860 2024-06-03T03:22:02.027 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T03:22:02.027 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":99,"fsid":"a0e8473a-2156-11ef-bc9b-c7b262605968","created":"2024-06-03T03:10:08.446859+0000","modified":"2024-06-03T03:21:25.966405+0000","last_up_change":"2024-06-03T03:18:28.736904+0000","last_in_change":"2024-06-03T03:18:08.485711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-03T03:14:09.370986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-03T03:19:35.044683+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-03T03:19:40.097569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-03T03:19:55.394480+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8fefa089-9401-4a1c-b9fb-fe30c63e6a1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":4286644492},{"type":"v1","addr":"172.21.15.117:6803","nonce":4286644492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":4286644492},{"type":"v1","addr":"172.21.15.117:6805","nonce":4286644492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":4286644492},{"type":"v1","addr":"172.21.15.117:6809","nonce":4286644492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":4286644492},{"type":"v1","addr":"172.21.15.117:6807","nonce":4286644492}]},"public_addr":"172.21.15.117:6803/4286644492","cluster_addr":"172.21.15.117:6805/4286644492","heartbeat_back_addr":"172.21.15.117:6809/4286644492","heartbeat_front_addr":"172.21.15.117:6807/4286644492","state":["exists","up"]},{"osd":1,"uuid":"5b12b371-f155-4b67-a470-25cbe59a2999","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":262491158},{"type":"v1","addr":"172.21.15.151:6801","nonce":262491158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":262491158},{"type":"v1","addr":"172.21.15.151:6803","nonce":262491158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":262491158},{"type":"v1","addr":"172.21.15.151:6807","nonce":262491158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":262491158},{"type":"v1","addr":"172.21.15.151:6805","nonce":262491158}]},"public_addr":"172.21.15.151:6801/262491158","cluster_addr":"172.21.15.151:6803/262491158","heartbeat_back_addr":"172.21.15.151:6807/262491158","heartbeat_front_addr":"172.21.15.151:6805/262491158","state":["exists","up"]},{"osd":2,"uuid":"dbc17818-c54c-4ecf-b656-03578b95d8c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":3542453623},{"type":"v1","addr":"172.21.15.183:6801","nonce":3542453623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":3542453623},{"type":"v1","addr":"172.21.15.183:6803","nonce":3542453623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":3542453623},{"type":"v1","addr":"172.21.15.183:6807","nonce":3542453623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":3542453623},{"type":"v1","addr":"172.21.15.183:6805","nonce":3542453623}]},"public_addr":"172.21.15.183:6801/3542453623","cluster_addr":"172.21.15.183:6803/3542453623","heartbeat_back_addr":"172.21.15.183:6807/3542453623","heartbeat_front_addr":"172.21.15.183:6805/3542453623","state":["exists","up"]},{"osd":3,"uuid":"ca08d2ac-b1d6-4a49-9ae8-e3ec146899a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":589012315},{"type":"v1","addr":"172.21.15.117:6811","nonce":589012315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":589012315},{"type":"v1","addr":"172.21.15.117:6813","nonce":589012315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":589012315},{"type":"v1","addr":"172.21.15.117:6817","nonce":589012315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":589012315},{"type":"v1","addr":"172.21.15.117:6815","nonce":589012315}]},"public_addr":"172.21.15.117:6811/589012315","cluster_addr":"172.21.15.117:6813/589012315","heartbeat_back_addr":"172.21.15.117:6817/589012315","heartbeat_front_addr":"172.21.15.117:6815/589012315","state":["exists","up"]},{"osd":4,"uuid":"810d6c7a-023f-4930-82af-bc2bbfd647da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":1239048647},{"type":"v1","addr":"172.21.15.151:6809","nonce":1239048647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":1239048647},{"type":"v1","addr":"172.21.15.151:6811","nonce":1239048647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":1239048647},{"type":"v1","addr":"172.21.15.151:6815","nonce":1239048647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":1239048647},{"type":"v1","addr":"172.21.15.151:6813","nonce":1239048647}]},"public_addr":"172.21.15.151:6809/1239048647","cluster_addr":"172.21.15.151:6811/1239048647","heartbeat_back_addr":"172.21.15.151:6815/1239048647","heartbeat_front_addr":"172.21.15.151:6813/1239048647","state":["exists","up"]},{"osd":5,"uuid":"8ab6a5f0-34bb-48d8-9471-a7dc836e486e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":2985874686},{"type":"v1","addr":"172.21.15.183:6809","nonce":2985874686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":2985874686},{"type":"v1","addr":"172.21.15.183:6811","nonce":2985874686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":2985874686},{"type":"v1","addr":"172.21.15.183:6815","nonce":2985874686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":2985874686},{"type":"v1","addr":"172.21.15.183:6813","nonce":2985874686}]},"public_addr":"172.21.15.183:6809/2985874686","cluster_addr":"172.21.15.183:6811/2985874686","heartbeat_back_addr":"172.21.15.183:6815/2985874686","heartbeat_front_addr":"172.21.15.183:6813/2985874686","state":["exists","up"]},{"osd":6,"uuid":"78d15739-e9e0-4f19-ae5f-ee8527a5b527","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4250583193},{"type":"v1","addr":"172.21.15.117:6819","nonce":4250583193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":4250583193},{"type":"v1","addr":"172.21.15.117:6821","nonce":4250583193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":4250583193},{"type":"v1","addr":"172.21.15.117:6825","nonce":4250583193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4250583193},{"type":"v1","addr":"172.21.15.117:6823","nonce":4250583193}]},"public_addr":"172.21.15.117:6819/4250583193","cluster_addr":"172.21.15.117:6821/4250583193","heartbeat_back_addr":"172.21.15.117:6825/4250583193","heartbeat_front_addr":"172.21.15.117:6823/4250583193","state":["exists","up"]},{"osd":7,"uuid":"3724fe1f-f4d7-443b-aea8-ff3ab02bfdc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":386094645},{"type":"v1","addr":"172.21.15.151:6817","nonce":386094645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":386094645},{"type":"v1","addr":"172.21.15.151:6819","nonce":386094645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":386094645},{"type":"v1","addr":"172.21.15.151:6823","nonce":386094645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":386094645},{"type":"v1","addr":"172.21.15.151:6821","nonce":386094645}]},"public_addr":"172.21.15.151:6817/386094645","cluster_addr":"172.21.15.151:6819/386094645","heartbeat_back_addr":"172.21.15.151:6823/386094645","heartbeat_front_addr":"172.21.15.151:6821/386094645","state":["exists","up"]},{"osd":8,"uuid":"006ac092-f015-448e-a0c0-f7b0c4597a18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1829748222},{"type":"v1","addr":"172.21.15.183:6817","nonce":1829748222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1829748222},{"type":"v1","addr":"172.21.15.183:6819","nonce":1829748222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1829748222},{"type":"v1","addr":"172.21.15.183:6823","nonce":1829748222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1829748222},{"type":"v1","addr":"172.21.15.183:6821","nonce":1829748222}]},"public_addr":"172.21.15.183:6817/1829748222","cluster_addr":"172.21.15.183:6819/1829748222","heartbeat_back_addr":"172.21.15.183:6823/1829748222","heartbeat_front_addr":"172.21.15.183:6821/1829748222","state":["exists","up"]},{"osd":9,"uuid":"00a970e6-2502-459e-975c-badac93c9bd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2538315141},{"type":"v1","addr":"172.21.15.117:6827","nonce":2538315141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2538315141},{"type":"v1","addr":"172.21.15.117:6829","nonce":2538315141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2538315141},{"type":"v1","addr":"172.21.15.117:6833","nonce":2538315141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2538315141},{"type":"v1","addr":"172.21.15.117:6831","nonce":2538315141}]},"public_addr":"172.21.15.117:6827/2538315141","cluster_addr":"172.21.15.117:6829/2538315141","heartbeat_back_addr":"172.21.15.117:6833/2538315141","heartbeat_front_addr":"172.21.15.117:6831/2538315141","state":["exists","up"]},{"osd":10,"uuid":"ba2333cd-32a3-41ef-9ea5-287c707f512b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":3801045133},{"type":"v1","addr":"172.21.15.151:6825","nonce":3801045133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3801045133},{"type":"v1","addr":"172.21.15.151:6827","nonce":3801045133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3801045133},{"type":"v1","addr":"172.21.15.151:6831","nonce":3801045133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3801045133},{"type":"v1","addr":"172.21.15.151:6829","nonce":3801045133}]},"public_addr":"172.21.15.151:6825/3801045133","cluster_addr":"172.21.15.151:6827/3801045133","heartbeat_back_addr":"172.21.15.151:6831/3801045133","heartbeat_front_addr":"172.21.15.151:6829/3801045133","state":["exists","up"]},{"osd":11,"uuid":"1556d654-e6fa-42c1-b399-aa53f6837a1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3513899696},{"type":"v1","addr":"172.21.15.183:6825","nonce":3513899696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3513899696},{"type":"v1","addr":"172.21.15.183:6827","nonce":3513899696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3513899696},{"type":"v1","addr":"172.21.15.183:6831","nonce":3513899696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3513899696},{"type":"v1","addr":"172.21.15.183:6829","nonce":3513899696}]},"public_addr":"172.21.15.183:6825/3513899696","cluster_addr":"172.21.15.183:6827/3513899696","heartbeat_back_addr":"172.21.15.183:6831/3513899696","heartbeat_front_addr":"172.21.15.183:6829/3513899696","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:13:17.174041+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:13:42.237001+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:14:06.273315+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:14:39.912564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:15:06.063202+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:15:30.424818+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:16:05.623554+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:16:32.926393+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:16:58.165899+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:17:32.897477+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:18:00.720866+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:18:26.474441+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.117:0/186309008":"2024-06-04T03:11:24.655516+0000","172.21.15.117:6801/2142313590":"2024-06-04T03:11:24.655516+0000","172.21.15.117:6800/2142313590":"2024-06-04T03:11:24.655516+0000","172.21.15.117:0/2399725156":"2024-06-04T03:11:24.655516+0000","172.21.15.117:0/1453232061":"2024-06-04T03:10:50.552899+0000","172.21.15.117:0/4281674974":"2024-06-04T03:10:50.552899+0000","172.21.15.117:6800/2424554985":"2024-06-04T03:10:50.552899+0000","172.21.15.117:0/3946365225":"2024-06-04T03:11:24.655516+0000","172.21.15.117:0/1146162354":"2024-06-04T03:10:50.552899+0000","172.21.15.117:6801/2424554985":"2024-06-04T03:10:50.552899+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-03T03:22:02.030 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:02.029+0000 7f54ade00700 1 -- 172.21.15.117:0/3803295834 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f548c0a2d30 msgr2=0x7f548c0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:02.030 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:02.029+0000 7f54ade00700 1 --2- 172.21.15.117:0/3803295834 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f548c0a2d30 0x7f548c0a51f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5498005950 tx=0x7f549800b410 comp rx=0 tx=0).stop 2024-06-03T03:22:02.030 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:02.029+0000 7f54ade00700 1 -- 172.21.15.117:0/3803295834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54a8101860 msgr2=0x7f54a80fcf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:02.030 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:02.029+0000 7f54ade00700 1 --2- 172.21.15.117:0/3803295834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54a8101860 0x7f54a80fcf40 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f549400ead0 tx=0x7f549400ede0 comp rx=0 tx=0).stop 2024-06-03T03:22:02.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:02.029+0000 7f54ade00700 1 -- 172.21.15.117:0/3803295834 shutdown_connections 2024-06-03T03:22:02.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:02.029+0000 7f54ade00700 1 --2- 172.21.15.117:0/3803295834 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f548c0a2d30 0x7f548c0a51f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:02.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:02.029+0000 7f54ade00700 1 --2- 172.21.15.117:0/3803295834 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f54a8106b30 0x7f54a80fd480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:02.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:02.029+0000 7f54ade00700 1 --2- 172.21.15.117:0/3803295834 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54a8100ea0 0x7f54a80fca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:02.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:02.029+0000 7f54ade00700 1 --2- 172.21.15.117:0/3803295834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54a8101860 0x7f54a80fcf40 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:02.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:02.029+0000 7f54ade00700 1 -- 172.21.15.117:0/3803295834 >> 172.21.15.117:0/3803295834 conn(0x7f54a8077590 msgr2=0x7f54a80feee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:22:02.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:02.029+0000 7f54ade00700 1 -- 172.21.15.117:0/3803295834 shutdown_connections 2024-06-03T03:22:02.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:02.030+0000 7f54ade00700 1 -- 172.21.15.117:0/3803295834 wait complete. 2024-06-03T03:22:02.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:22:02 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/3803295834' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T03:22:02.433 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:22:02 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/3803295834' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T03:22:02.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:22:02 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/3803295834' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T03:22:03.065 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-06-03T03:22:03.066 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-03T03:22:03.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:22:03 smithi151 ceph-mon[37043]: pgmap v408: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 11 op/s 2024-06-03T03:22:03.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:22:03 smithi117 ceph-mon[31649]: pgmap v408: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 11 op/s 2024-06-03T03:22:03.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:22:03 smithi183 ceph-mon[37326]: pgmap v408: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 11 op/s 2024-06-03T03:22:03.487 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:22:05.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:05.940+0000 7f8be6600700 1 -- 172.21.15.117:0/3805455145 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8be01012e0 msgr2=0x7f8be0101760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:05.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:05.940+0000 7f8be6600700 1 --2- 172.21.15.117:0/3805455145 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8be01012e0 0x7f8be0101760 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8bd8009f90 tx=0x7f8bd800b330 comp rx=0 tx=0).stop 2024-06-03T03:22:05.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:05.940+0000 7f8be6600700 1 -- 172.21.15.117:0/3805455145 shutdown_connections 2024-06-03T03:22:05.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:05.940+0000 7f8be6600700 1 --2- 172.21.15.117:0/3805455145 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8be01012e0 0x7f8be0101760 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:05.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:05.940+0000 7f8be6600700 1 --2- 172.21.15.117:0/3805455145 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8be0106140 0x7f8be0106520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:05.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:05.940+0000 7f8be6600700 1 --2- 172.21.15.117:0/3805455145 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8be0106af0 0x7f8be0100cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:05.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:05.940+0000 7f8be6600700 1 -- 172.21.15.117:0/3805455145 >> 172.21.15.117:0/3805455145 conn(0x7f8be0076640 msgr2=0x7f8be0076a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:22:05.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:05.940+0000 7f8be6600700 1 -- 172.21.15.117:0/3805455145 shutdown_connections 2024-06-03T03:22:05.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:05.940+0000 7f8be6600700 1 -- 172.21.15.117:0/3805455145 wait complete. 2024-06-03T03:22:05.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:05.941+0000 7f8be6600700 1 Processor -- start 2024-06-03T03:22:05.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:05.941+0000 7f8be6600700 1 -- start start 2024-06-03T03:22:05.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:05.942+0000 7f8be6600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8be01012e0 0x7f8be019f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:05.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:05.942+0000 7f8be6600700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8be0106140 0x7f8be019f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:05.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:05.942+0000 7f8be6600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8be0106af0 0x7f8be01a3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:05.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:05.942+0000 7f8be6600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8be0118ec0 con 0x7f8be01012e0 2024-06-03T03:22:05.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:05.942+0000 7f8be6600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8be0118d40 con 0x7f8be0106af0 2024-06-03T03:22:05.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:05.942+0000 7f8be6600700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8be0119040 con 0x7f8be0106140 2024-06-03T03:22:05.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:05.942+0000 7f8be5200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8be01012e0 0x7f8be019f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:05.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:05.942+0000 7f8be5c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8be0106af0 0x7f8be01a3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:05.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:05.942+0000 7f8be5200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8be01012e0 0x7f8be019f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:43448/0 (socket says 172.21.15.117:43448) 2024-06-03T03:22:05.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:05.942+0000 7f8bdfe00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8be0106140 0x7f8be019f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:05.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:05.942+0000 7f8be5200700 1 -- 172.21.15.117:0/1866610018 learned_addr learned my addr 172.21.15.117:0/1866610018 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:22:05.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:05.943+0000 7f8be5c00700 1 -- 172.21.15.117:0/1866610018 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8be0106140 msgr2=0x7f8be019f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:05.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:05.943+0000 7f8be5c00700 1 --2- 172.21.15.117:0/1866610018 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8be0106140 0x7f8be019f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:05.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:05.943+0000 7f8be5c00700 1 -- 172.21.15.117:0/1866610018 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8be01012e0 msgr2=0x7f8be019f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:05.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:05.943+0000 7f8be5c00700 1 --2- 172.21.15.117:0/1866610018 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8be01012e0 0x7f8be019f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:05.946 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:05.943+0000 7f8be5c00700 1 -- 172.21.15.117:0/1866610018 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8be01a4280 con 0x7f8be0106af0 2024-06-03T03:22:05.946 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:05.943+0000 7f8be5c00700 1 --2- 172.21.15.117:0/1866610018 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8be0106af0 0x7f8be01a3b40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8bd8009f90 tx=0x7f8bd800cfa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:22:05.946 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:05.943+0000 7f8bdd600700 1 -- 172.21.15.117:0/1866610018 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd8021070 con 0x7f8be0106af0 2024-06-03T03:22:05.946 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:05.943+0000 7f8bdd600700 1 -- 172.21.15.117:0/1866610018 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bd8007ac0 con 0x7f8be0106af0 2024-06-03T03:22:05.946 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:05.943+0000 7f8bdd600700 1 -- 172.21.15.117:0/1866610018 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd80052a0 con 0x7f8be0106af0 2024-06-03T03:22:05.946 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:05.943+0000 7f8be6600700 1 -- 172.21.15.117:0/1866610018 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8be01a4510 con 0x7f8be0106af0 2024-06-03T03:22:05.947 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:05.944+0000 7f8be6600700 1 -- 172.21.15.117:0/1866610018 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8be01abdf0 con 0x7f8be0106af0 2024-06-03T03:22:05.947 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:05.945+0000 7f8be6600700 1 -- 172.21.15.117:0/1866610018 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8be0069180 con 0x7f8be0106af0 2024-06-03T03:22:05.947 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:05.947+0000 7f8bdd600700 1 -- 172.21.15.117:0/1866610018 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f8bd8024070 con 0x7f8be0106af0 2024-06-03T03:22:05.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:05.948+0000 7f8bdd600700 1 --2- 172.21.15.117:0/1866610018 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f8bc80a2cb0 0x7f8bc80a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:05.949 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:05.948+0000 7f8bdd600700 1 -- 172.21.15.117:0/1866610018 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f8bd800f020 con 0x7f8be0106af0 2024-06-03T03:22:05.949 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:05.948+0000 7f8be5200700 1 --2- 172.21.15.117:0/1866610018 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f8bc80a2cb0 0x7f8bc80a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:05.949 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:05.949+0000 7f8be5200700 1 --2- 172.21.15.117:0/1866610018 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f8bc80a2cb0 0x7f8bc80a5170 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8bd4009cf0 tx=0x7f8bd4009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:22:05.953 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:05.952+0000 7f8bdd600700 1 -- 172.21.15.117:0/1866610018 <== mon.2 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8bd802b020 con 0x7f8be0106af0 2024-06-03T03:22:06.180 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:06.179+0000 7f8be6600700 1 -- 172.21.15.117:0/1866610018 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8be01abfc0 con 0x7f8be0106af0 2024-06-03T03:22:06.180 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:22:05 smithi117 ceph-mon[31649]: pgmap v409: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:22:06.182 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:06.181+0000 7f8bdd600700 1 -- 172.21.15.117:0/1866610018 <== mon.2 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+24807 (secure 0 0 0) 0x7f8bd8007c70 con 0x7f8be0106af0 2024-06-03T03:22:06.183 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T03:22:06.183 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":99,"fsid":"a0e8473a-2156-11ef-bc9b-c7b262605968","created":"2024-06-03T03:10:08.446859+0000","modified":"2024-06-03T03:21:25.966405+0000","last_up_change":"2024-06-03T03:18:28.736904+0000","last_in_change":"2024-06-03T03:18:08.485711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-03T03:14:09.370986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-03T03:19:35.044683+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-03T03:19:40.097569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-03T03:19:55.394480+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8fefa089-9401-4a1c-b9fb-fe30c63e6a1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":4286644492},{"type":"v1","addr":"172.21.15.117:6803","nonce":4286644492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":4286644492},{"type":"v1","addr":"172.21.15.117:6805","nonce":4286644492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":4286644492},{"type":"v1","addr":"172.21.15.117:6809","nonce":4286644492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":4286644492},{"type":"v1","addr":"172.21.15.117:6807","nonce":4286644492}]},"public_addr":"172.21.15.117:6803/4286644492","cluster_addr":"172.21.15.117:6805/4286644492","heartbeat_back_addr":"172.21.15.117:6809/4286644492","heartbeat_front_addr":"172.21.15.117:6807/4286644492","state":["exists","up"]},{"osd":1,"uuid":"5b12b371-f155-4b67-a470-25cbe59a2999","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":262491158},{"type":"v1","addr":"172.21.15.151:6801","nonce":262491158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":262491158},{"type":"v1","addr":"172.21.15.151:6803","nonce":262491158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":262491158},{"type":"v1","addr":"172.21.15.151:6807","nonce":262491158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":262491158},{"type":"v1","addr":"172.21.15.151:6805","nonce":262491158}]},"public_addr":"172.21.15.151:6801/262491158","cluster_addr":"172.21.15.151:6803/262491158","heartbeat_back_addr":"172.21.15.151:6807/262491158","heartbeat_front_addr":"172.21.15.151:6805/262491158","state":["exists","up"]},{"osd":2,"uuid":"dbc17818-c54c-4ecf-b656-03578b95d8c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":3542453623},{"type":"v1","addr":"172.21.15.183:6801","nonce":3542453623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":3542453623},{"type":"v1","addr":"172.21.15.183:6803","nonce":3542453623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":3542453623},{"type":"v1","addr":"172.21.15.183:6807","nonce":3542453623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":3542453623},{"type":"v1","addr":"172.21.15.183:6805","nonce":3542453623}]},"public_addr":"172.21.15.183:6801/3542453623","cluster_addr":"172.21.15.183:6803/3542453623","heartbeat_back_addr":"172.21.15.183:6807/3542453623","heartbeat_front_addr":"172.21.15.183:6805/3542453623","state":["exists","up"]},{"osd":3,"uuid":"ca08d2ac-b1d6-4a49-9ae8-e3ec146899a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":589012315},{"type":"v1","addr":"172.21.15.117:6811","nonce":589012315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":589012315},{"type":"v1","addr":"172.21.15.117:6813","nonce":589012315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":589012315},{"type":"v1","addr":"172.21.15.117:6817","nonce":589012315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":589012315},{"type":"v1","addr":"172.21.15.117:6815","nonce":589012315}]},"public_addr":"172.21.15.117:6811/589012315","cluster_addr":"172.21.15.117:6813/589012315","heartbeat_back_addr":"172.21.15.117:6817/589012315","heartbeat_front_addr":"172.21.15.117:6815/589012315","state":["exists","up"]},{"osd":4,"uuid":"810d6c7a-023f-4930-82af-bc2bbfd647da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":1239048647},{"type":"v1","addr":"172.21.15.151:6809","nonce":1239048647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":1239048647},{"type":"v1","addr":"172.21.15.151:6811","nonce":1239048647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":1239048647},{"type":"v1","addr":"172.21.15.151:6815","nonce":1239048647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":1239048647},{"type":"v1","addr":"172.21.15.151:6813","nonce":1239048647}]},"public_addr":"172.21.15.151:6809/1239048647","cluster_addr":"172.21.15.151:6811/1239048647","heartbeat_back_addr":"172.21.15.151:6815/1239048647","heartbeat_front_addr":"172.21.15.151:6813/1239048647","state":["exists","up"]},{"osd":5,"uuid":"8ab6a5f0-34bb-48d8-9471-a7dc836e486e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":2985874686},{"type":"v1","addr":"172.21.15.183:6809","nonce":2985874686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":2985874686},{"type":"v1","addr":"172.21.15.183:6811","nonce":2985874686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":2985874686},{"type":"v1","addr":"172.21.15.183:6815","nonce":2985874686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":2985874686},{"type":"v1","addr":"172.21.15.183:6813","nonce":2985874686}]},"public_addr":"172.21.15.183:6809/2985874686","cluster_addr":"172.21.15.183:6811/2985874686","heartbeat_back_addr":"172.21.15.183:6815/2985874686","heartbeat_front_addr":"172.21.15.183:6813/2985874686","state":["exists","up"]},{"osd":6,"uuid":"78d15739-e9e0-4f19-ae5f-ee8527a5b527","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4250583193},{"type":"v1","addr":"172.21.15.117:6819","nonce":4250583193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":4250583193},{"type":"v1","addr":"172.21.15.117:6821","nonce":4250583193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":4250583193},{"type":"v1","addr":"172.21.15.117:6825","nonce":4250583193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4250583193},{"type":"v1","addr":"172.21.15.117:6823","nonce":4250583193}]},"public_addr":"172.21.15.117:6819/4250583193","cluster_addr":"172.21.15.117:6821/4250583193","heartbeat_back_addr":"172.21.15.117:6825/4250583193","heartbeat_front_addr":"172.21.15.117:6823/4250583193","state":["exists","up"]},{"osd":7,"uuid":"3724fe1f-f4d7-443b-aea8-ff3ab02bfdc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":386094645},{"type":"v1","addr":"172.21.15.151:6817","nonce":386094645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":386094645},{"type":"v1","addr":"172.21.15.151:6819","nonce":386094645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":386094645},{"type":"v1","addr":"172.21.15.151:6823","nonce":386094645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":386094645},{"type":"v1","addr":"172.21.15.151:6821","nonce":386094645}]},"public_addr":"172.21.15.151:6817/386094645","cluster_addr":"172.21.15.151:6819/386094645","heartbeat_back_addr":"172.21.15.151:6823/386094645","heartbeat_front_addr":"172.21.15.151:6821/386094645","state":["exists","up"]},{"osd":8,"uuid":"006ac092-f015-448e-a0c0-f7b0c4597a18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1829748222},{"type":"v1","addr":"172.21.15.183:6817","nonce":1829748222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1829748222},{"type":"v1","addr":"172.21.15.183:6819","nonce":1829748222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1829748222},{"type":"v1","addr":"172.21.15.183:6823","nonce":1829748222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1829748222},{"type":"v1","addr":"172.21.15.183:6821","nonce":1829748222}]},"public_addr":"172.21.15.183:6817/1829748222","cluster_addr":"172.21.15.183:6819/1829748222","heartbeat_back_addr":"172.21.15.183:6823/1829748222","heartbeat_front_addr":"172.21.15.183:6821/1829748222","state":["exists","up"]},{"osd":9,"uuid":"00a970e6-2502-459e-975c-badac93c9bd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2538315141},{"type":"v1","addr":"172.21.15.117:6827","nonce":2538315141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2538315141},{"type":"v1","addr":"172.21.15.117:6829","nonce":2538315141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2538315141},{"type":"v1","addr":"172.21.15.117:6833","nonce":2538315141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2538315141},{"type":"v1","addr":"172.21.15.117:6831","nonce":2538315141}]},"public_addr":"172.21.15.117:6827/2538315141","cluster_addr":"172.21.15.117:6829/2538315141","heartbeat_back_addr":"172.21.15.117:6833/2538315141","heartbeat_front_addr":"172.21.15.117:6831/2538315141","state":["exists","up"]},{"osd":10,"uuid":"ba2333cd-32a3-41ef-9ea5-287c707f512b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":3801045133},{"type":"v1","addr":"172.21.15.151:6825","nonce":3801045133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3801045133},{"type":"v1","addr":"172.21.15.151:6827","nonce":3801045133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3801045133},{"type":"v1","addr":"172.21.15.151:6831","nonce":3801045133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3801045133},{"type":"v1","addr":"172.21.15.151:6829","nonce":3801045133}]},"public_addr":"172.21.15.151:6825/3801045133","cluster_addr":"172.21.15.151:6827/3801045133","heartbeat_back_addr":"172.21.15.151:6831/3801045133","heartbeat_front_addr":"172.21.15.151:6829/3801045133","state":["exists","up"]},{"osd":11,"uuid":"1556d654-e6fa-42c1-b399-aa53f6837a1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3513899696},{"type":"v1","addr":"172.21.15.183:6825","nonce":3513899696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3513899696},{"type":"v1","addr":"172.21.15.183:6827","nonce":3513899696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3513899696},{"type":"v1","addr":"172.21.15.183:6831","nonce":3513899696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3513899696},{"type":"v1","addr":"172.21.15.183:6829","nonce":3513899696}]},"public_addr":"172.21.15.183:6825/3513899696","cluster_addr":"172.21.15.183:6827/3513899696","heartbeat_back_addr":"172.21.15.183:6831/3513899696","heartbeat_front_addr":"172.21.15.183:6829/3513899696","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:13:17.174041+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:13:42.237001+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:14:06.273315+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:14:39.912564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:15:06.063202+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:15:30.424818+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:16:05.623554+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:16:32.926393+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:16:58.165899+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:17:32.897477+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:18:00.720866+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:18:26.474441+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.117:0/186309008":"2024-06-04T03:11:24.655516+0000","172.21.15.117:6801/2142313590":"2024-06-04T03:11:24.655516+0000","172.21.15.117:6800/2142313590":"2024-06-04T03:11:24.655516+0000","172.21.15.117:0/2399725156":"2024-06-04T03:11:24.655516+0000","172.21.15.117:0/1453232061":"2024-06-04T03:10:50.552899+0000","172.21.15.117:0/4281674974":"2024-06-04T03:10:50.552899+0000","172.21.15.117:6800/2424554985":"2024-06-04T03:10:50.552899+0000","172.21.15.117:0/3946365225":"2024-06-04T03:11:24.655516+0000","172.21.15.117:0/1146162354":"2024-06-04T03:10:50.552899+0000","172.21.15.117:6801/2424554985":"2024-06-04T03:10:50.552899+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-03T03:22:06.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:06.185+0000 7f8be6600700 1 -- 172.21.15.117:0/1866610018 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f8bc80a2cb0 msgr2=0x7f8bc80a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:06.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:06.185+0000 7f8be6600700 1 --2- 172.21.15.117:0/1866610018 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f8bc80a2cb0 0x7f8bc80a5170 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8bd4009cf0 tx=0x7f8bd4009310 comp rx=0 tx=0).stop 2024-06-03T03:22:06.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:06.185+0000 7f8be6600700 1 -- 172.21.15.117:0/1866610018 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8be0106af0 msgr2=0x7f8be01a3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:06.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:06.185+0000 7f8be6600700 1 --2- 172.21.15.117:0/1866610018 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8be0106af0 0x7f8be01a3b40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8bd8009f90 tx=0x7f8bd800cfa0 comp rx=0 tx=0).stop 2024-06-03T03:22:06.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:06.185+0000 7f8be6600700 1 -- 172.21.15.117:0/1866610018 shutdown_connections 2024-06-03T03:22:06.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:06.185+0000 7f8be6600700 1 --2- 172.21.15.117:0/1866610018 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f8bc80a2cb0 0x7f8bc80a5170 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:06.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:06.185+0000 7f8be6600700 1 --2- 172.21.15.117:0/1866610018 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8be0106af0 0x7f8be01a3b40 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:06.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:06.185+0000 7f8be6600700 1 --2- 172.21.15.117:0/1866610018 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8be01012e0 0x7f8be019f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:06.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:06.185+0000 7f8be6600700 1 --2- 172.21.15.117:0/1866610018 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8be0106140 0x7f8be019f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:06.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:06.185+0000 7f8be6600700 1 -- 172.21.15.117:0/1866610018 >> 172.21.15.117:0/1866610018 conn(0x7f8be0076640 msgr2=0x7f8be0101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:22:06.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:06.185+0000 7f8be6600700 1 -- 172.21.15.117:0/1866610018 shutdown_connections 2024-06-03T03:22:06.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:06.185+0000 7f8be6600700 1 -- 172.21.15.117:0/1866610018 wait complete. 2024-06-03T03:22:06.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:22:05 smithi183 ceph-mon[37326]: pgmap v409: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:22:06.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:22:05 smithi151 ceph-mon[37043]: pgmap v409: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:22:07.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:22:06 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/1866610018' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T03:22:07.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:22:06 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/1866610018' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T03:22:07.263 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-06-03T03:22:07.264 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-06-03T03:22:07.264 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-06-03T03:22:07.265 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-06-03T03:22:07.265 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-06-03T03:22:07.265 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-06-03T03:22:07.266 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-06-03T03:22:07.266 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-06-03T03:22:07.267 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-06-03T03:22:07.267 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-06-03T03:22:07.267 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-06-03T03:22:07.268 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-06-03T03:22:07.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:22:06 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/1866610018' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T03:22:07.939 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:22:07 smithi117 ceph-mon[31649]: pgmap v410: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-03T03:22:08.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:22:07 smithi183 ceph-mon[37326]: pgmap v410: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-03T03:22:08.244 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:22:08.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:22:07 smithi151 ceph-mon[37043]: pgmap v410: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-03T03:22:08.455 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:22:08.458 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:22:08.458 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:22:08.465 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:22:08.470 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:22:08.476 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:22:08.494 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:22:08.494 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:22:08.495 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:22:08.496 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:22:08.536 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:22:10.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:22:09 smithi117 ceph-mon[31649]: pgmap v411: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-03T03:22:10.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:22:09 smithi183 ceph-mon[37326]: pgmap v411: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-03T03:22:10.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:22:09 smithi151 ceph-mon[37043]: pgmap v411: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-03T03:22:12.030 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:22:11 smithi117 ceph-mon[31649]: pgmap v412: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-03T03:22:12.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:22:11 smithi183 ceph-mon[37326]: pgmap v412: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-03T03:22:12.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:22:11 smithi151 ceph-mon[37043]: pgmap v412: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-03T03:22:14.146 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:22:13 smithi117 ceph-mon[31649]: pgmap v413: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-03T03:22:14.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:22:13 smithi183 ceph-mon[37326]: pgmap v413: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-03T03:22:14.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:22:13 smithi151 ceph-mon[37043]: pgmap v413: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-03T03:22:16.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:22:15 smithi117 ceph-mon[31649]: pgmap v414: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:22:16.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:22:15 smithi183 ceph-mon[37326]: pgmap v414: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:22:16.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:22:15 smithi151 ceph-mon[37043]: pgmap v414: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:22:18.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:22:17 smithi183 ceph-mon[37326]: pgmap v415: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-03T03:22:18.248 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:22:17 smithi117 ceph-mon[31649]: pgmap v415: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-03T03:22:18.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:22:17 smithi151 ceph-mon[37043]: pgmap v415: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-03T03:22:19.260 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:18.683+0000 7f0604e00700 1 -- 172.21.15.117:0/1205008188 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f060006a170 msgr2=0x7f060006a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:19.260 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:18.683+0000 7f0604e00700 1 --2- 172.21.15.117:0/1205008188 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f060006a170 0x7f060006a5f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f05ec009990 tx=0x7f05ec009ca0 comp rx=0 tx=0).stop 2024-06-03T03:22:19.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:18.683+0000 7f0604e00700 1 -- 172.21.15.117:0/1205008188 shutdown_connections 2024-06-03T03:22:19.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:18.683+0000 7f0604e00700 1 --2- 172.21.15.117:0/1205008188 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f060006ab30 0x7f060010f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:19.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:18.683+0000 7f0604e00700 1 --2- 172.21.15.117:0/1205008188 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0600108500 0x7f06001088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:19.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:18.683+0000 7f0604e00700 1 --2- 172.21.15.117:0/1205008188 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f060006a170 0x7f060006a5f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:19.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:18.683+0000 7f0604e00700 1 -- 172.21.15.117:0/1205008188 >> 172.21.15.117:0/1205008188 conn(0x7f0600078340 msgr2=0x7f0600078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:22:19.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:18.683+0000 7f0604e00700 1 -- 172.21.15.117:0/1205008188 shutdown_connections 2024-06-03T03:22:19.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:18.683+0000 7f0604e00700 1 -- 172.21.15.117:0/1205008188 wait complete. 2024-06-03T03:22:19.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:18.684+0000 7f0604e00700 1 Processor -- start 2024-06-03T03:22:19.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:18.685+0000 7f0604e00700 1 -- start start 2024-06-03T03:22:19.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:18.685+0000 7f0604e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f060006a170 0x7f060019ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:19.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:18.685+0000 7f0604e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f060006ab30 0x7f060019b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:19.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:18.685+0000 7f0604e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0600108500 0x7f060019f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:19.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:18.685+0000 7f0604e00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0600112780 con 0x7f060006a170 2024-06-03T03:22:19.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:18.685+0000 7f0604e00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0600112600 con 0x7f060006ab30 2024-06-03T03:22:19.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:18.685+0000 7f0604e00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0600112900 con 0x7f0600108500 2024-06-03T03:22:19.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:18.685+0000 7f05fde00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f060006a170 0x7f060019ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:19.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:18.685+0000 7f05fde00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f060006a170 0x7f060019ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:58720/0 (socket says 172.21.15.117:58720) 2024-06-03T03:22:19.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:18.685+0000 7f05fde00700 1 -- 172.21.15.117:0/1305157332 learned_addr learned my addr 172.21.15.117:0/1305157332 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:22:19.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:18.686+0000 7f05fe800700 1 --2- 172.21.15.117:0/1305157332 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0600108500 0x7f060019f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:19.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:18.686+0000 7f05fd400700 1 --2- 172.21.15.117:0/1305157332 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f060006ab30 0x7f060019b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:19.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:18.686+0000 7f05fde00700 1 -- 172.21.15.117:0/1305157332 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0600108500 msgr2=0x7f060019f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:19.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:18.686+0000 7f05fde00700 1 --2- 172.21.15.117:0/1305157332 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0600108500 0x7f060019f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:19.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:18.686+0000 7f05fde00700 1 -- 172.21.15.117:0/1305157332 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f060006ab30 msgr2=0x7f060019b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:19.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:18.686+0000 7f05fde00700 1 --2- 172.21.15.117:0/1305157332 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f060006ab30 0x7f060019b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:19.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:18.686+0000 7f05fde00700 1 -- 172.21.15.117:0/1305157332 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f060019fdf0 con 0x7f060006a170 2024-06-03T03:22:19.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:18.686+0000 7f05fde00700 1 --2- 172.21.15.117:0/1305157332 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f060006a170 0x7f060019ade0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f05f000d980 tx=0x7f05f000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:22:19.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:18.686+0000 7f05f6a00700 1 -- 172.21.15.117:0/1305157332 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f0016070 con 0x7f060006a170 2024-06-03T03:22:19.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:18.686+0000 7f0604e00700 1 -- 172.21.15.117:0/1305157332 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06001a00e0 con 0x7f060006a170 2024-06-03T03:22:19.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:18.686+0000 7f05f6a00700 1 -- 172.21.15.117:0/1305157332 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05f0004d10 con 0x7f060006a170 2024-06-03T03:22:19.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:18.686+0000 7f05f6a00700 1 -- 172.21.15.117:0/1305157332 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f00055f0 con 0x7f060006a170 2024-06-03T03:22:19.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:18.687+0000 7f0604e00700 1 -- 172.21.15.117:0/1305157332 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06001a7960 con 0x7f060006a170 2024-06-03T03:22:19.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:18.687+0000 7f0604e00700 1 -- 172.21.15.117:0/1305157332 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f060010e0b0 con 0x7f060006a170 2024-06-03T03:22:19.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:18.690+0000 7f05f6a00700 1 -- 172.21.15.117:0/1305157332 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f05f0004750 con 0x7f060006a170 2024-06-03T03:22:19.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:18.691+0000 7f05f6a00700 1 --2- 172.21.15.117:0/1305157332 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f05e40a2cb0 0x7f05e40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:19.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:18.691+0000 7f05f6a00700 1 -- 172.21.15.117:0/1305157332 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f05f00cfa10 con 0x7f060006a170 2024-06-03T03:22:19.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:18.691+0000 7f05fd400700 1 --2- 172.21.15.117:0/1305157332 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f05e40a2cb0 0x7f05e40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:19.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:18.692+0000 7f05f6a00700 1 --2- 172.21.15.117:0/1305157332 >> [v2:172.21.15.183:6824/3513899696,v1:172.21.15.183:6825/3513899696] conn(0x7f05e40adf10 0x7f05e40b03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:19.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:18.692+0000 7f05f6a00700 1 -- 172.21.15.117:0/1305157332 --> [v2:172.21.15.183:6824/3513899696,v1:172.21.15.183:6825/3513899696] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f05f0015ec0 con 0x7f05e40adf10 2024-06-03T03:22:19.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:18.692+0000 7f05f6a00700 1 -- 172.21.15.117:0/1305157332 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7f05f00cfdc0 con 0x7f060006a170 2024-06-03T03:22:19.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:18.692+0000 7f05fe800700 1 --2- 172.21.15.117:0/1305157332 >> [v2:172.21.15.183:6824/3513899696,v1:172.21.15.183:6825/3513899696] conn(0x7f05e40adf10 0x7f05e40b03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:19.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:18.692+0000 7f05fd400700 1 --2- 172.21.15.117:0/1305157332 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f05e40a2cb0 0x7f05e40a5170 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f05ec006010 tx=0x7f05ec00b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:22:19.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:18.693+0000 7f05fe800700 1 --2- 172.21.15.117:0/1305157332 >> [v2:172.21.15.183:6824/3513899696,v1:172.21.15.183:6825/3513899696] conn(0x7f05e40adf10 0x7f05e40b03b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:22:19.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:18.694+0000 7f05f6a00700 1 -- 172.21.15.117:0/1305157332 <== osd.11 v2:172.21.15.183:6824/3513899696 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f05f0015ec0 con 0x7f05e40adf10 2024-06-03T03:22:19.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:18.712+0000 7f0604e00700 1 -- 172.21.15.117:0/1305157332 --> [v2:172.21.15.183:6824/3513899696,v1:172.21.15.183:6825/3513899696] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0600103bc0 con 0x7f05e40adf10 2024-06-03T03:22:19.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:18.713+0000 7f05f6a00700 1 -- 172.21.15.117:0/1305157332 <== osd.11 v2:172.21.15.183:6824/3513899696 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0600103bc0 con 0x7f05e40adf10 2024-06-03T03:22:19.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:18.713+0000 7f0604e00700 1 -- 172.21.15.117:0/1305157332 >> [v2:172.21.15.183:6824/3513899696,v1:172.21.15.183:6825/3513899696] conn(0x7f05e40adf10 msgr2=0x7f05e40b03b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:19.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:18.713+0000 7f0604e00700 1 --2- 172.21.15.117:0/1305157332 >> [v2:172.21.15.183:6824/3513899696,v1:172.21.15.183:6825/3513899696] conn(0x7f05e40adf10 0x7f05e40b03b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:19.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:18.713+0000 7f0604e00700 1 -- 172.21.15.117:0/1305157332 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f05e40a2cb0 msgr2=0x7f05e40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:19.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:18.713+0000 7f0604e00700 1 --2- 172.21.15.117:0/1305157332 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f05e40a2cb0 0x7f05e40a5170 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f05ec006010 tx=0x7f05ec00b540 comp rx=0 tx=0).stop 2024-06-03T03:22:19.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:18.713+0000 7f0604e00700 1 -- 172.21.15.117:0/1305157332 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f060006a170 msgr2=0x7f060019ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:19.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:18.713+0000 7f0604e00700 1 --2- 172.21.15.117:0/1305157332 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f060006a170 0x7f060019ade0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f05f000d980 tx=0x7f05f000dc90 comp rx=0 tx=0).stop 2024-06-03T03:22:19.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:18.713+0000 7f05fe800700 1 -- 172.21.15.117:0/1305157332 reap_dead start 2024-06-03T03:22:19.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:18.713+0000 7f0604e00700 1 -- 172.21.15.117:0/1305157332 shutdown_connections 2024-06-03T03:22:19.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:18.713+0000 7f0604e00700 1 -- 172.21.15.117:0/1305157332 >> 172.21.15.117:0/1305157332 conn(0x7f0600078340 msgr2=0x7f060010d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:22:19.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:18.714+0000 7f0604e00700 1 -- 172.21.15.117:0/1305157332 shutdown_connections 2024-06-03T03:22:19.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:18.714+0000 7f0604e00700 1 -- 172.21.15.117:0/1305157332 wait complete. 2024-06-03T03:22:20.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:22:19 smithi183 ceph-mon[37326]: pgmap v416: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-03T03:22:20.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:22:19 smithi151 ceph-mon[37043]: pgmap v416: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-03T03:22:20.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:22:19 smithi117 ceph-mon[31649]: pgmap v416: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-03T03:22:22.228 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:22:21 smithi117 ceph-mon[31649]: pgmap v417: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-03T03:22:22.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:22:21 smithi183 ceph-mon[37326]: pgmap v417: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-03T03:22:22.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:22:21 smithi151 ceph-mon[37043]: pgmap v417: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-03T03:22:23.486 INFO:teuthology.orchestra.run.smithi117.stdout:343597383728 2024-06-03T03:22:23.486 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-06-03T03:22:24.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:22:23 smithi183 ceph-mon[37326]: pgmap v418: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-03T03:22:24.335 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:22:23 smithi117 ceph-mon[31649]: pgmap v418: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-03T03:22:24.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:22:23 smithi151 ceph-mon[37043]: pgmap v418: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-03T03:22:26.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:22:25 smithi117 ceph-mon[31649]: pgmap v419: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:22:26.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:22:25 smithi151 ceph-mon[37043]: pgmap v419: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:22:26.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:22:25 smithi183 ceph-mon[37326]: pgmap v419: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:22:27.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:27.045+0000 7fbb0a800700 1 -- 172.21.15.117:0/1747200965 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb041046b0 msgr2=0x7fbb04104b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:27.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:27.045+0000 7fbb0a800700 1 --2- 172.21.15.117:0/1747200965 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb041046b0 0x7fbb04104b30 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbaec009990 tx=0x7fbaec009ca0 comp rx=0 tx=0).stop 2024-06-03T03:22:27.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:27.045+0000 7fbb0a800700 1 -- 172.21.15.117:0/1747200965 shutdown_connections 2024-06-03T03:22:27.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:27.045+0000 7fbb0a800700 1 --2- 172.21.15.117:0/1747200965 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb04105070 0x7fbb04111580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:27.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:27.045+0000 7fbb0a800700 1 --2- 172.21.15.117:0/1747200965 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbb0410a6b0 0x7fbb0410aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:27.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:27.045+0000 7fbb0a800700 1 --2- 172.21.15.117:0/1747200965 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb041046b0 0x7fbb04104b30 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:27.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:27.045+0000 7fbb0a800700 1 -- 172.21.15.117:0/1747200965 >> 172.21.15.117:0/1747200965 conn(0x7fbb041001a0 msgr2=0x7fbb041025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:22:27.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:27.045+0000 7fbb0a800700 1 -- 172.21.15.117:0/1747200965 shutdown_connections 2024-06-03T03:22:27.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:27.045+0000 7fbb0a800700 1 -- 172.21.15.117:0/1747200965 wait complete. 2024-06-03T03:22:27.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:27.046+0000 7fbb0a800700 1 Processor -- start 2024-06-03T03:22:27.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:27.046+0000 7fbb0a800700 1 -- start start 2024-06-03T03:22:27.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:27.047+0000 7fbb0a800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb041046b0 0x7fbb0419ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:27.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:27.047+0000 7fbb0a800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb04105070 0x7fbb0419b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:27.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:27.047+0000 7fbb0a800700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbb0410a6b0 0x7fbb0419f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:27.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:27.047+0000 7fbb0a800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbb04114a40 con 0x7fbb04105070 2024-06-03T03:22:27.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:27.047+0000 7fbb0a800700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbb041148c0 con 0x7fbb041046b0 2024-06-03T03:22:27.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:27.047+0000 7fbb0a800700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbb04114bc0 con 0x7fbb0410a6b0 2024-06-03T03:22:27.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:27.047+0000 7fbb03400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb041046b0 0x7fbb0419ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:27.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:27.047+0000 7fbafaa00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb04105070 0x7fbb0419b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:27.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:27.047+0000 7fbb03e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbb0410a6b0 0x7fbb0419f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:27.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:27.047+0000 7fbafaa00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb04105070 0x7fbb0419b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56368/0 (socket says 172.21.15.117:56368) 2024-06-03T03:22:27.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:27.047+0000 7fbb03400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb041046b0 0x7fbb0419ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:47122/0 (socket says 172.21.15.117:47122) 2024-06-03T03:22:27.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:27.047+0000 7fbafaa00700 1 -- 172.21.15.117:0/3311945397 learned_addr learned my addr 172.21.15.117:0/3311945397 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:22:27.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:27.048+0000 7fbb03400700 1 -- 172.21.15.117:0/3311945397 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbb0410a6b0 msgr2=0x7fbb0419f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:27.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:27.048+0000 7fbb03400700 1 --2- 172.21.15.117:0/3311945397 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbb0410a6b0 0x7fbb0419f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:27.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:27.048+0000 7fbb03400700 1 -- 172.21.15.117:0/3311945397 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb04105070 msgr2=0x7fbb0419b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:27.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:27.048+0000 7fbb03400700 1 --2- 172.21.15.117:0/3311945397 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb04105070 0x7fbb0419b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:27.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:27.048+0000 7fbb03400700 1 -- 172.21.15.117:0/3311945397 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb0419fe50 con 0x7fbb041046b0 2024-06-03T03:22:27.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:27.048+0000 7fbb03e00700 1 --2- 172.21.15.117:0/3311945397 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbb0410a6b0 0x7fbb0419f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:22:27.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:27.048+0000 7fbb03400700 1 --2- 172.21.15.117:0/3311945397 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb041046b0 0x7fbb0419ae90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fbaf400efc0 tx=0x7fbaf400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:22:27.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:27.048+0000 7fbb00c00700 1 -- 172.21.15.117:0/3311945397 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf40092e0 con 0x7fbb041046b0 2024-06-03T03:22:27.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:27.049+0000 7fbb00c00700 1 -- 172.21.15.117:0/3311945397 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbaf4004bb0 con 0x7fbb041046b0 2024-06-03T03:22:27.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:27.049+0000 7fbb0a800700 1 -- 172.21.15.117:0/3311945397 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb041a00e0 con 0x7fbb041046b0 2024-06-03T03:22:27.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:27.049+0000 7fbb00c00700 1 -- 172.21.15.117:0/3311945397 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf4004280 con 0x7fbb041046b0 2024-06-03T03:22:27.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:27.049+0000 7fbb0a800700 1 -- 172.21.15.117:0/3311945397 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb041a79c0 con 0x7fbb041046b0 2024-06-03T03:22:27.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:27.050+0000 7fbb0a800700 1 -- 172.21.15.117:0/3311945397 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbb04062c20 con 0x7fbb041046b0 2024-06-03T03:22:27.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:27.052+0000 7fbb00c00700 1 -- 172.21.15.117:0/3311945397 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fbaf401b030 con 0x7fbb041046b0 2024-06-03T03:22:27.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:27.053+0000 7fbb00c00700 1 --2- 172.21.15.117:0/3311945397 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fbae80a2cb0 0x7fbae80a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:27.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:27.053+0000 7fbb00c00700 1 -- 172.21.15.117:0/3311945397 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7fbaf40cf4f0 con 0x7fbb041046b0 2024-06-03T03:22:27.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:27.054+0000 7fbafaa00700 1 --2- 172.21.15.117:0/3311945397 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fbae80a2cb0 0x7fbae80a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:27.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:27.054+0000 7fbb00c00700 1 --2- 172.21.15.117:0/3311945397 >> [v2:172.21.15.183:6816/1829748222,v1:172.21.15.183:6817/1829748222] conn(0x7fbae80adf10 0x7fbae80b03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:27.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:27.054+0000 7fbb00c00700 1 -- 172.21.15.117:0/3311945397 --> [v2:172.21.15.183:6816/1829748222,v1:172.21.15.183:6817/1829748222] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbae80b0a80 con 0x7fbae80adf10 2024-06-03T03:22:27.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:27.054+0000 7fbb00c00700 1 -- 172.21.15.117:0/3311945397 <== mon.2 v2:172.21.15.151:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7fbaf40cf8a0 con 0x7fbb041046b0 2024-06-03T03:22:27.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:27.054+0000 7fbb03e00700 1 --2- 172.21.15.117:0/3311945397 >> [v2:172.21.15.183:6816/1829748222,v1:172.21.15.183:6817/1829748222] conn(0x7fbae80adf10 0x7fbae80b03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:27.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:27.054+0000 7fbafaa00700 1 --2- 172.21.15.117:0/3311945397 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fbae80a2cb0 0x7fbae80a5170 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fbb041027d0 tx=0x7fbaec009590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:22:27.056 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:27.055+0000 7fbb03e00700 1 --2- 172.21.15.117:0/3311945397 >> [v2:172.21.15.183:6816/1829748222,v1:172.21.15.183:6817/1829748222] conn(0x7fbae80adf10 0x7fbae80b03b0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:22:27.056 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:27.056+0000 7fbb00c00700 1 -- 172.21.15.117:0/3311945397 <== osd.8 v2:172.21.15.183:6816/1829748222 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbae80b0a80 con 0x7fbae80adf10 2024-06-03T03:22:27.074 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:27.073+0000 7fbb0a800700 1 -- 172.21.15.117:0/3311945397 --> [v2:172.21.15.183:6816/1829748222,v1:172.21.15.183:6817/1829748222] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbb040630a0 con 0x7fbae80adf10 2024-06-03T03:22:27.075 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:27.074+0000 7fbb00c00700 1 -- 172.21.15.117:0/3311945397 <== osd.8 v2:172.21.15.183:6816/1829748222 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fbb040630a0 con 0x7fbae80adf10 2024-06-03T03:22:27.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:27.075+0000 7fbb0a800700 1 -- 172.21.15.117:0/3311945397 >> [v2:172.21.15.183:6816/1829748222,v1:172.21.15.183:6817/1829748222] conn(0x7fbae80adf10 msgr2=0x7fbae80b03b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:27.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:27.075+0000 7fbb0a800700 1 --2- 172.21.15.117:0/3311945397 >> [v2:172.21.15.183:6816/1829748222,v1:172.21.15.183:6817/1829748222] conn(0x7fbae80adf10 0x7fbae80b03b0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:27.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:27.075+0000 7fbb0a800700 1 -- 172.21.15.117:0/3311945397 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fbae80a2cb0 msgr2=0x7fbae80a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:27.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:27.075+0000 7fbb0a800700 1 --2- 172.21.15.117:0/3311945397 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fbae80a2cb0 0x7fbae80a5170 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fbb041027d0 tx=0x7fbaec009590 comp rx=0 tx=0).stop 2024-06-03T03:22:27.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:27.075+0000 7fbb0a800700 1 -- 172.21.15.117:0/3311945397 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb041046b0 msgr2=0x7fbb0419ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:27.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:27.075+0000 7fbb0a800700 1 --2- 172.21.15.117:0/3311945397 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb041046b0 0x7fbb0419ae90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fbaf400efc0 tx=0x7fbaf400c3a0 comp rx=0 tx=0).stop 2024-06-03T03:22:27.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:27.075+0000 7fbb03e00700 1 -- 172.21.15.117:0/3311945397 reap_dead start 2024-06-03T03:22:27.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:27.075+0000 7fbb0a800700 1 -- 172.21.15.117:0/3311945397 shutdown_connections 2024-06-03T03:22:27.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:27.075+0000 7fbb0a800700 1 -- 172.21.15.117:0/3311945397 >> 172.21.15.117:0/3311945397 conn(0x7fbb041001a0 msgr2=0x7fbb04100850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:22:27.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:27.075+0000 7fbb0a800700 1 -- 172.21.15.117:0/3311945397 shutdown_connections 2024-06-03T03:22:27.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:27.075+0000 7fbb0a800700 1 -- 172.21.15.117:0/3311945397 wait complete. 2024-06-03T03:22:27.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.719+0000 7f4037a00700 1 -- 172.21.15.117:0/3441569487 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f403010a690 msgr2=0x7f403010aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:27.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.719+0000 7f4037a00700 1 --2- 172.21.15.117:0/3441569487 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f403010a690 0x7f403010aa70 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4024009990 tx=0x7f4024009ca0 comp rx=0 tx=0).stop 2024-06-03T03:22:27.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.719+0000 7f4037a00700 1 -- 172.21.15.117:0/3441569487 shutdown_connections 2024-06-03T03:22:27.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.719+0000 7f4037a00700 1 --2- 172.21.15.117:0/3441569487 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4030105050 0x7f4030111560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:27.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.719+0000 7f4037a00700 1 --2- 172.21.15.117:0/3441569487 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f403010a690 0x7f403010aa70 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:27.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.719+0000 7f4037a00700 1 --2- 172.21.15.117:0/3441569487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4030104690 0x7f4030104b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:27.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.719+0000 7f4037a00700 1 -- 172.21.15.117:0/3441569487 >> 172.21.15.117:0/3441569487 conn(0x7f40301001a0 msgr2=0x7f40301025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:22:27.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.719+0000 7f4037a00700 1 -- 172.21.15.117:0/3441569487 shutdown_connections 2024-06-03T03:22:27.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.719+0000 7f4037a00700 1 -- 172.21.15.117:0/3441569487 wait complete. 2024-06-03T03:22:27.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.720+0000 7f4037a00700 1 Processor -- start 2024-06-03T03:22:27.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.720+0000 7f4037a00700 1 -- start start 2024-06-03T03:22:27.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.721+0000 7f4037a00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4030104690 0x7f403019ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:27.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.721+0000 7f4037a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4030105050 0x7f403019b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:27.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.721+0000 7f4037a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f403010a690 0x7f403019f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:27.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.721+0000 7f4037a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f40301149c0 con 0x7f4030105050 2024-06-03T03:22:27.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.721+0000 7f4037a00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4030114840 con 0x7f403010a690 2024-06-03T03:22:27.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.721+0000 7f4037a00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4030114b40 con 0x7f4030104690 2024-06-03T03:22:27.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.721+0000 7f402fe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4030105050 0x7f403019b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:27.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.721+0000 7f402fe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4030105050 0x7f403019b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:58732/0 (socket says 172.21.15.117:58732) 2024-06-03T03:22:27.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.721+0000 7f402fe00700 1 -- 172.21.15.117:0/2623442854 learned_addr learned my addr 172.21.15.117:0/2623442854 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:22:27.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.721+0000 7f4035000700 1 --2- 172.21.15.117:0/2623442854 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4030104690 0x7f403019ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:27.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.721+0000 7f4035a00700 1 --2- 172.21.15.117:0/2623442854 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f403010a690 0x7f403019f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:27.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.721+0000 7f402fe00700 1 -- 172.21.15.117:0/2623442854 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4030104690 msgr2=0x7f403019ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:27.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.721+0000 7f402fe00700 1 --2- 172.21.15.117:0/2623442854 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4030104690 0x7f403019ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:27.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.721+0000 7f402fe00700 1 -- 172.21.15.117:0/2623442854 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f403010a690 msgr2=0x7f403019f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:27.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.721+0000 7f402fe00700 1 --2- 172.21.15.117:0/2623442854 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f403010a690 0x7f403019f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:27.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.721+0000 7f402fe00700 1 -- 172.21.15.117:0/2623442854 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f403019fe40 con 0x7f4030105050 2024-06-03T03:22:27.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.722+0000 7f4035a00700 1 --2- 172.21.15.117:0/2623442854 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f403010a690 0x7f403019f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:22:27.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.722+0000 7f402fe00700 1 --2- 172.21.15.117:0/2623442854 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4030105050 0x7f403019b370 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f401c00e9a0 tx=0x7f401c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:22:27.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.722+0000 7f402d600700 1 -- 172.21.15.117:0/2623442854 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f401c00ca10 con 0x7f4030105050 2024-06-03T03:22:27.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.722+0000 7f402d600700 1 -- 172.21.15.117:0/2623442854 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f401c00cbb0 con 0x7f4030105050 2024-06-03T03:22:27.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.722+0000 7f4037a00700 1 -- 172.21.15.117:0/2623442854 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40301a0130 con 0x7f4030105050 2024-06-03T03:22:27.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.722+0000 7f402d600700 1 -- 172.21.15.117:0/2623442854 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f401c018680 con 0x7f4030105050 2024-06-03T03:22:27.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.722+0000 7f4037a00700 1 -- 172.21.15.117:0/2623442854 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40301a79b0 con 0x7f4030105050 2024-06-03T03:22:27.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.723+0000 7f4037a00700 1 -- 172.21.15.117:0/2623442854 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4030062c20 con 0x7f4030105050 2024-06-03T03:22:27.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.725+0000 7f402d600700 1 -- 172.21.15.117:0/2623442854 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f401c010430 con 0x7f4030105050 2024-06-03T03:22:27.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.727+0000 7f402d600700 1 --2- 172.21.15.117:0/2623442854 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f40180a2cb0 0x7f40180a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:27.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.727+0000 7f402d600700 1 -- 172.21.15.117:0/2623442854 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f401c0cffe0 con 0x7f4030105050 2024-06-03T03:22:27.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.727+0000 7f4035000700 1 --2- 172.21.15.117:0/2623442854 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f40180a2cb0 0x7f40180a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:27.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.727+0000 7f402d600700 1 --2- 172.21.15.117:0/2623442854 >> [v2:172.21.15.151:6800/262491158,v1:172.21.15.151:6801/262491158] conn(0x7f40180adf10 0x7f40180b03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:27.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.727+0000 7f402d600700 1 -- 172.21.15.117:0/2623442854 --> [v2:172.21.15.151:6800/262491158,v1:172.21.15.151:6801/262491158] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f401c01bec0 con 0x7f40180adf10 2024-06-03T03:22:27.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.727+0000 7f402d600700 1 -- 172.21.15.117:0/2623442854 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7f401c0d0390 con 0x7f4030105050 2024-06-03T03:22:27.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.728+0000 7f4035a00700 1 --2- 172.21.15.117:0/2623442854 >> [v2:172.21.15.151:6800/262491158,v1:172.21.15.151:6801/262491158] conn(0x7f40180adf10 0x7f40180b03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:27.489 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.728+0000 7f4035000700 1 --2- 172.21.15.117:0/2623442854 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f40180a2cb0 0x7f40180a5170 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f40240098e0 tx=0x7f4024005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:22:27.489 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.728+0000 7f4035a00700 1 --2- 172.21.15.117:0/2623442854 >> [v2:172.21.15.151:6800/262491158,v1:172.21.15.151:6801/262491158] conn(0x7f40180adf10 0x7f40180b03b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:22:27.489 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.729+0000 7f402d600700 1 -- 172.21.15.117:0/2623442854 <== osd.1 v2:172.21.15.151:6800/262491158 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f401c01bec0 con 0x7f40180adf10 2024-06-03T03:22:27.489 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.748+0000 7f4037a00700 1 -- 172.21.15.117:0/2623442854 --> [v2:172.21.15.151:6800/262491158,v1:172.21.15.151:6801/262491158] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f40300630a0 con 0x7f40180adf10 2024-06-03T03:22:27.489 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.749+0000 7f402d600700 1 -- 172.21.15.117:0/2623442854 <== osd.1 v2:172.21.15.151:6800/262491158 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f40300630a0 con 0x7f40180adf10 2024-06-03T03:22:27.489 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.749+0000 7f4037a00700 1 -- 172.21.15.117:0/2623442854 >> [v2:172.21.15.151:6800/262491158,v1:172.21.15.151:6801/262491158] conn(0x7f40180adf10 msgr2=0x7f40180b03b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:27.489 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.749+0000 7f4037a00700 1 --2- 172.21.15.117:0/2623442854 >> [v2:172.21.15.151:6800/262491158,v1:172.21.15.151:6801/262491158] conn(0x7f40180adf10 0x7f40180b03b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:27.490 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.749+0000 7f4037a00700 1 -- 172.21.15.117:0/2623442854 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f40180a2cb0 msgr2=0x7f40180a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:27.490 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.749+0000 7f4037a00700 1 --2- 172.21.15.117:0/2623442854 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f40180a2cb0 0x7f40180a5170 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f40240098e0 tx=0x7f4024005960 comp rx=0 tx=0).stop 2024-06-03T03:22:27.490 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.749+0000 7f4037a00700 1 -- 172.21.15.117:0/2623442854 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4030105050 msgr2=0x7f403019b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:27.490 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.749+0000 7f4037a00700 1 --2- 172.21.15.117:0/2623442854 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4030105050 0x7f403019b370 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f401c00e9a0 tx=0x7f401c00ecb0 comp rx=0 tx=0).stop 2024-06-03T03:22:27.490 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.749+0000 7f4035a00700 1 -- 172.21.15.117:0/2623442854 reap_dead start 2024-06-03T03:22:27.490 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.750+0000 7f4037a00700 1 -- 172.21.15.117:0/2623442854 shutdown_connections 2024-06-03T03:22:27.490 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.750+0000 7f4037a00700 1 -- 172.21.15.117:0/2623442854 >> 172.21.15.117:0/2623442854 conn(0x7f40301001a0 msgr2=0x7f40301008c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:22:27.490 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.750+0000 7f4037a00700 1 -- 172.21.15.117:0/2623442854 shutdown_connections 2024-06-03T03:22:27.490 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.750+0000 7f4037a00700 1 -- 172.21.15.117:0/2623442854 wait complete. 2024-06-03T03:22:28.131 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:22:27 smithi117 ceph-mon[31649]: pgmap v420: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-03T03:22:28.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:22:27 smithi151 ceph-mon[37043]: pgmap v420: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-03T03:22:28.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:22:27 smithi183 ceph-mon[37326]: pgmap v420: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-03T03:22:29.306 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:29.011+0000 7ff04f200700 1 -- 172.21.15.117:0/1774088284 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0501089d0 msgr2=0x7ff050108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:29.306 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:29.011+0000 7ff04f200700 1 --2- 172.21.15.117:0/1774088284 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0501089d0 0x7ff050108e50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff03c009990 tx=0x7ff03c009ca0 comp rx=0 tx=0).stop 2024-06-03T03:22:29.307 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:29.011+0000 7ff04f200700 1 -- 172.21.15.117:0/1774088284 shutdown_connections 2024-06-03T03:22:29.307 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:29.011+0000 7ff04f200700 1 --2- 172.21.15.117:0/1774088284 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff050109390 0x7ff050115830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:29.307 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:29.011+0000 7ff04f200700 1 --2- 172.21.15.117:0/1774088284 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff05010d8a0 0x7ff05010dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:29.307 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:29.011+0000 7ff04f200700 1 --2- 172.21.15.117:0/1774088284 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0501089d0 0x7ff050108e50 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:29.307 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:29.011+0000 7ff04f200700 1 -- 172.21.15.117:0/1774088284 >> 172.21.15.117:0/1774088284 conn(0x7ff050078220 msgr2=0x7ff0501072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:22:29.307 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:29.011+0000 7ff04f200700 1 -- 172.21.15.117:0/1774088284 shutdown_connections 2024-06-03T03:22:29.307 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:29.011+0000 7ff04f200700 1 -- 172.21.15.117:0/1774088284 wait complete. 2024-06-03T03:22:29.307 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:29.012+0000 7ff04f200700 1 Processor -- start 2024-06-03T03:22:29.307 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:29.012+0000 7ff04f200700 1 -- start start 2024-06-03T03:22:29.307 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:29.013+0000 7ff04f200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0501089d0 0x7ff05019f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:29.307 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:29.013+0000 7ff04f200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff050109390 0x7ff05019f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:29.307 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:29.013+0000 7ff04f200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff05010d8a0 0x7ff0501a3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:29.307 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:29.013+0000 7ff04f200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff050118d40 con 0x7ff0501089d0 2024-06-03T03:22:29.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:29.013+0000 7ff04f200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff050118bc0 con 0x7ff05010d8a0 2024-06-03T03:22:29.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:29.013+0000 7ff04f200700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff050118ec0 con 0x7ff050109390 2024-06-03T03:22:29.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:29.013+0000 7ff04de00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0501089d0 0x7ff05019f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:29.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:29.013+0000 7ff04de00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0501089d0 0x7ff05019f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56374/0 (socket says 172.21.15.117:56374) 2024-06-03T03:22:29.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:29.013+0000 7ff04d400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff050109390 0x7ff05019f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:29.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:29.013+0000 7ff04de00700 1 -- 172.21.15.117:0/1885920978 learned_addr learned my addr 172.21.15.117:0/1885920978 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:22:29.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:29.013+0000 7ff04e800700 1 --2- 172.21.15.117:0/1885920978 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff05010d8a0 0x7ff0501a3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:29.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:29.014+0000 7ff04d400700 1 -- 172.21.15.117:0/1885920978 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff05010d8a0 msgr2=0x7ff0501a3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:29.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:29.014+0000 7ff04d400700 1 --2- 172.21.15.117:0/1885920978 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff05010d8a0 0x7ff0501a3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:29.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:29.014+0000 7ff04d400700 1 -- 172.21.15.117:0/1885920978 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0501089d0 msgr2=0x7ff05019f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:29.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:29.014+0000 7ff04d400700 1 --2- 172.21.15.117:0/1885920978 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0501089d0 0x7ff05019f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:29.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:29.014+0000 7ff04d400700 1 -- 172.21.15.117:0/1885920978 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0501a4170 con 0x7ff050109390 2024-06-03T03:22:29.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:29.014+0000 7ff04de00700 1 --2- 172.21.15.117:0/1885920978 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0501089d0 0x7ff05019f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:22:29.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:29.014+0000 7ff04d400700 1 --2- 172.21.15.117:0/1885920978 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff050109390 0x7ff05019f6f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff03c0098e0 tx=0x7ff03c00b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:22:29.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:29.014+0000 7ff046a00700 1 -- 172.21.15.117:0/1885920978 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff03c027070 con 0x7ff050109390 2024-06-03T03:22:29.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:29.014+0000 7ff046a00700 1 -- 172.21.15.117:0/1885920978 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff03c003b60 con 0x7ff050109390 2024-06-03T03:22:29.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:29.014+0000 7ff04f200700 1 -- 172.21.15.117:0/1885920978 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0501a4400 con 0x7ff050109390 2024-06-03T03:22:29.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:29.014+0000 7ff046a00700 1 -- 172.21.15.117:0/1885920978 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff03c022d50 con 0x7ff050109390 2024-06-03T03:22:29.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:29.014+0000 7ff04f200700 1 -- 172.21.15.117:0/1885920978 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0501abce0 con 0x7ff050109390 2024-06-03T03:22:29.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:29.015+0000 7ff04f200700 1 -- 172.21.15.117:0/1885920978 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff050062ba0 con 0x7ff050109390 2024-06-03T03:22:29.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:29.017+0000 7ff046a00700 1 -- 172.21.15.117:0/1885920978 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7ff03c02a070 con 0x7ff050109390 2024-06-03T03:22:29.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:29.018+0000 7ff046a00700 1 --2- 172.21.15.117:0/1885920978 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7ff0340a2d30 0x7ff0340a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:29.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:29.019+0000 7ff046a00700 1 -- 172.21.15.117:0/1885920978 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7ff03c0d0f60 con 0x7ff050109390 2024-06-03T03:22:29.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:29.019+0000 7ff04de00700 1 --2- 172.21.15.117:0/1885920978 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7ff0340a2d30 0x7ff0340a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:29.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:29.019+0000 7ff046a00700 1 --2- 172.21.15.117:0/1885920978 >> [v2:172.21.15.151:6808/1239048647,v1:172.21.15.151:6809/1239048647] conn(0x7ff0340adf90 0x7ff0340b0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:29.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:29.019+0000 7ff046a00700 1 -- 172.21.15.117:0/1885920978 --> [v2:172.21.15.151:6808/1239048647,v1:172.21.15.151:6809/1239048647] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff0340b0b00 con 0x7ff0340adf90 2024-06-03T03:22:29.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:29.019+0000 7ff046a00700 1 -- 172.21.15.117:0/1885920978 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7ff03c0d1310 con 0x7ff050109390 2024-06-03T03:22:29.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:29.019+0000 7ff04de00700 1 --2- 172.21.15.117:0/1885920978 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7ff0340a2d30 0x7ff0340a51f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff040011550 tx=0x7ff040009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:22:29.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:29.019+0000 7ff04e800700 1 --2- 172.21.15.117:0/1885920978 >> [v2:172.21.15.151:6808/1239048647,v1:172.21.15.151:6809/1239048647] conn(0x7ff0340adf90 0x7ff0340b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:29.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:29.020+0000 7ff04e800700 1 --2- 172.21.15.117:0/1885920978 >> [v2:172.21.15.151:6808/1239048647,v1:172.21.15.151:6809/1239048647] conn(0x7ff0340adf90 0x7ff0340b0430 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:22:29.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:29.021+0000 7ff046a00700 1 -- 172.21.15.117:0/1885920978 <== osd.4 v2:172.21.15.151:6808/1239048647 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff0340b0b00 con 0x7ff0340adf90 2024-06-03T03:22:29.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:29.039+0000 7ff04f200700 1 -- 172.21.15.117:0/1885920978 --> [v2:172.21.15.151:6808/1239048647,v1:172.21.15.151:6809/1239048647] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff050062fa0 con 0x7ff0340adf90 2024-06-03T03:22:29.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:29.040+0000 7ff046a00700 1 -- 172.21.15.117:0/1885920978 <== osd.4 v2:172.21.15.151:6808/1239048647 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff050062fa0 con 0x7ff0340adf90 2024-06-03T03:22:29.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:29.040+0000 7ff04f200700 1 -- 172.21.15.117:0/1885920978 >> [v2:172.21.15.151:6808/1239048647,v1:172.21.15.151:6809/1239048647] conn(0x7ff0340adf90 msgr2=0x7ff0340b0430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:29.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:29.040+0000 7ff04f200700 1 --2- 172.21.15.117:0/1885920978 >> [v2:172.21.15.151:6808/1239048647,v1:172.21.15.151:6809/1239048647] conn(0x7ff0340adf90 0x7ff0340b0430 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:29.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:29.040+0000 7ff04f200700 1 -- 172.21.15.117:0/1885920978 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7ff0340a2d30 msgr2=0x7ff0340a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:29.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:29.040+0000 7ff04f200700 1 --2- 172.21.15.117:0/1885920978 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7ff0340a2d30 0x7ff0340a51f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff040011550 tx=0x7ff040009260 comp rx=0 tx=0).stop 2024-06-03T03:22:29.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:29.040+0000 7ff04f200700 1 -- 172.21.15.117:0/1885920978 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff050109390 msgr2=0x7ff05019f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:29.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:29.040+0000 7ff04f200700 1 --2- 172.21.15.117:0/1885920978 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff050109390 0x7ff05019f6f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff03c0098e0 tx=0x7ff03c00b8a0 comp rx=0 tx=0).stop 2024-06-03T03:22:29.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:29.040+0000 7ff04e800700 1 -- 172.21.15.117:0/1885920978 reap_dead start 2024-06-03T03:22:29.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:29.040+0000 7ff04f200700 1 -- 172.21.15.117:0/1885920978 shutdown_connections 2024-06-03T03:22:29.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:29.040+0000 7ff04f200700 1 -- 172.21.15.117:0/1885920978 >> 172.21.15.117:0/1885920978 conn(0x7ff050078220 msgr2=0x7ff0500787e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:22:29.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:29.041+0000 7ff04f200700 1 -- 172.21.15.117:0/1885920978 shutdown_connections 2024-06-03T03:22:29.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:29.041+0000 7ff04f200700 1 -- 172.21.15.117:0/1885920978 wait complete. 2024-06-03T03:22:30.129 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:22:29 smithi117 ceph-mon[31649]: pgmap v421: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-03T03:22:30.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:30.195+0000 7f4754a00700 1 -- 172.21.15.117:0/372842137 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f475010a6b0 msgr2=0x7f475010aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:30.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:30.195+0000 7f4754a00700 1 --2- 172.21.15.117:0/372842137 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f475010a6b0 0x7f475010aa90 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4740009990 tx=0x7f4740009ca0 comp rx=0 tx=0).stop 2024-06-03T03:22:30.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:30.195+0000 7f4754a00700 1 -- 172.21.15.117:0/372842137 shutdown_connections 2024-06-03T03:22:30.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:30.195+0000 7f4754a00700 1 --2- 172.21.15.117:0/372842137 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4750105070 0x7f4750111580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:30.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:30.195+0000 7f4754a00700 1 --2- 172.21.15.117:0/372842137 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f475010a6b0 0x7f475010aa90 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:30.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:30.195+0000 7f4754a00700 1 --2- 172.21.15.117:0/372842137 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f47501046b0 0x7f4750104b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:30.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:30.195+0000 7f4754a00700 1 -- 172.21.15.117:0/372842137 >> 172.21.15.117:0/372842137 conn(0x7f47501001a0 msgr2=0x7f47501025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:22:30.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:30.195+0000 7f4754a00700 1 -- 172.21.15.117:0/372842137 shutdown_connections 2024-06-03T03:22:30.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:30.196+0000 7f4754a00700 1 -- 172.21.15.117:0/372842137 wait complete. 2024-06-03T03:22:30.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:30.197+0000 7f4754a00700 1 Processor -- start 2024-06-03T03:22:30.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:30.197+0000 7f4754a00700 1 -- start start 2024-06-03T03:22:30.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:30.197+0000 7f4754a00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47501046b0 0x7f475019ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:30.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:30.197+0000 7f4754a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4750105070 0x7f475019b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:30.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:30.197+0000 7f4754a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f475010a6b0 0x7f475019f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:30.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:30.197+0000 7f4754a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4750114a40 con 0x7f475010a6b0 2024-06-03T03:22:30.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:30.197+0000 7f4754a00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f47501148c0 con 0x7f4750105070 2024-06-03T03:22:30.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:30.197+0000 7f4754a00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4750114bc0 con 0x7f47501046b0 2024-06-03T03:22:30.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:30.197+0000 7f474de00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47501046b0 0x7f475019ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:30.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:30.198+0000 7f474de00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47501046b0 0x7f475019ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:51926/0 (socket says 172.21.15.117:51926) 2024-06-03T03:22:30.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:30.198+0000 7f474e800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f475010a6b0 0x7f475019f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:30.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:30.198+0000 7f474de00700 1 -- 172.21.15.117:0/3268766012 learned_addr learned my addr 172.21.15.117:0/3268766012 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:22:30.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:30.198+0000 7f474d400700 1 --2- 172.21.15.117:0/3268766012 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4750105070 0x7f475019b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:30.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:30.198+0000 7f474de00700 1 -- 172.21.15.117:0/3268766012 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4750105070 msgr2=0x7f475019b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:30.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:30.198+0000 7f474de00700 1 --2- 172.21.15.117:0/3268766012 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4750105070 0x7f475019b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:30.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:30.198+0000 7f474de00700 1 -- 172.21.15.117:0/3268766012 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f475010a6b0 msgr2=0x7f475019f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:30.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:30.198+0000 7f474de00700 1 --2- 172.21.15.117:0/3268766012 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f475010a6b0 0x7f475019f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:30.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:30.198+0000 7f474de00700 1 -- 172.21.15.117:0/3268766012 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f475019fe60 con 0x7f47501046b0 2024-06-03T03:22:30.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:30.198+0000 7f474d400700 1 --2- 172.21.15.117:0/3268766012 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4750105070 0x7f475019b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:22:30.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:30.198+0000 7f474de00700 1 --2- 172.21.15.117:0/3268766012 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47501046b0 0x7f475019ae50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4740000c00 tx=0x7f474000ba90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:22:30.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:30.199+0000 7f4746a00700 1 -- 172.21.15.117:0/3268766012 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4740027070 con 0x7f47501046b0 2024-06-03T03:22:30.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:30.199+0000 7f4754a00700 1 -- 172.21.15.117:0/3268766012 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47501a00f0 con 0x7f47501046b0 2024-06-03T03:22:30.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:30.199+0000 7f4746a00700 1 -- 172.21.15.117:0/3268766012 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4740003d70 con 0x7f47501046b0 2024-06-03T03:22:30.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:30.199+0000 7f4746a00700 1 -- 172.21.15.117:0/3268766012 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f474000f460 con 0x7f47501046b0 2024-06-03T03:22:30.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:30.199+0000 7f4754a00700 1 -- 172.21.15.117:0/3268766012 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47501a79d0 con 0x7f47501046b0 2024-06-03T03:22:30.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:30.200+0000 7f4754a00700 1 -- 172.21.15.117:0/3268766012 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4750062c20 con 0x7f47501046b0 2024-06-03T03:22:30.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:30.202+0000 7f4746a00700 1 -- 172.21.15.117:0/3268766012 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f474002a070 con 0x7f47501046b0 2024-06-03T03:22:30.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:30.203+0000 7f4746a00700 1 --2- 172.21.15.117:0/3268766012 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f47340a2cb0 0x7f47340a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:30.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:30.203+0000 7f4746a00700 1 -- 172.21.15.117:0/3268766012 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f47400d0fe0 con 0x7f47501046b0 2024-06-03T03:22:30.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:30.203+0000 7f474d400700 1 --2- 172.21.15.117:0/3268766012 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f47340a2cb0 0x7f47340a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:30.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:30.204+0000 7f4746a00700 1 --2- 172.21.15.117:0/3268766012 >> [v2:172.21.15.117:6802/4286644492,v1:172.21.15.117:6803/4286644492] conn(0x7f47340adf10 0x7f47340b03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:30.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:30.204+0000 7f4746a00700 1 -- 172.21.15.117:0/3268766012 --> [v2:172.21.15.117:6802/4286644492,v1:172.21.15.117:6803/4286644492] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f47340b0a80 con 0x7f47340adf10 2024-06-03T03:22:30.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:30.204+0000 7f4746a00700 1 -- 172.21.15.117:0/3268766012 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7f47400d1390 con 0x7f47501046b0 2024-06-03T03:22:30.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:30.204+0000 7f474e800700 1 --2- 172.21.15.117:0/3268766012 >> [v2:172.21.15.117:6802/4286644492,v1:172.21.15.117:6803/4286644492] conn(0x7f47340adf10 0x7f47340b03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:30.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:30.204+0000 7f474d400700 1 --2- 172.21.15.117:0/3268766012 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f47340a2cb0 0x7f47340a5170 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4750109be0 tx=0x7f473c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:22:30.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:30.205+0000 7f474e800700 1 --2- 172.21.15.117:0/3268766012 >> [v2:172.21.15.117:6802/4286644492,v1:172.21.15.117:6803/4286644492] conn(0x7f47340adf10 0x7f47340b03b0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:22:30.206 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:30.206+0000 7f4746a00700 1 -- 172.21.15.117:0/3268766012 <== osd.0 v2:172.21.15.117:6802/4286644492 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f47340b0a80 con 0x7f47340adf10 2024-06-03T03:22:30.224 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:30.224+0000 7f4754a00700 1 -- 172.21.15.117:0/3268766012 --> [v2:172.21.15.117:6802/4286644492,v1:172.21.15.117:6803/4286644492] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f47500630a0 con 0x7f47340adf10 2024-06-03T03:22:30.225 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:30.224+0000 7f4746a00700 1 -- 172.21.15.117:0/3268766012 <== osd.0 v2:172.21.15.117:6802/4286644492 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f47500630a0 con 0x7f47340adf10 2024-06-03T03:22:30.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:30.225+0000 7f4754a00700 1 -- 172.21.15.117:0/3268766012 >> [v2:172.21.15.117:6802/4286644492,v1:172.21.15.117:6803/4286644492] conn(0x7f47340adf10 msgr2=0x7f47340b03b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:30.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:30.225+0000 7f4754a00700 1 --2- 172.21.15.117:0/3268766012 >> [v2:172.21.15.117:6802/4286644492,v1:172.21.15.117:6803/4286644492] conn(0x7f47340adf10 0x7f47340b03b0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:30.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:30.225+0000 7f4754a00700 1 -- 172.21.15.117:0/3268766012 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f47340a2cb0 msgr2=0x7f47340a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:30.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:30.225+0000 7f4754a00700 1 --2- 172.21.15.117:0/3268766012 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f47340a2cb0 0x7f47340a5170 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4750109be0 tx=0x7f473c009260 comp rx=0 tx=0).stop 2024-06-03T03:22:30.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:30.225+0000 7f4754a00700 1 -- 172.21.15.117:0/3268766012 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47501046b0 msgr2=0x7f475019ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:30.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:30.225+0000 7f4754a00700 1 --2- 172.21.15.117:0/3268766012 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47501046b0 0x7f475019ae50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4740000c00 tx=0x7f474000ba90 comp rx=0 tx=0).stop 2024-06-03T03:22:30.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:30.225+0000 7f474e800700 1 -- 172.21.15.117:0/3268766012 reap_dead start 2024-06-03T03:22:30.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:30.225+0000 7f4754a00700 1 -- 172.21.15.117:0/3268766012 shutdown_connections 2024-06-03T03:22:30.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:30.225+0000 7f4754a00700 1 -- 172.21.15.117:0/3268766012 >> 172.21.15.117:0/3268766012 conn(0x7f47501001a0 msgr2=0x7f4750100720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:22:30.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:30.225+0000 7f4754a00700 1 -- 172.21.15.117:0/3268766012 shutdown_connections 2024-06-03T03:22:30.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:30.225+0000 7f4754a00700 1 -- 172.21.15.117:0/3268766012 wait complete. 2024-06-03T03:22:30.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:22:29 smithi151 ceph-mon[37043]: pgmap v421: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-03T03:22:30.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:22:29 smithi183 ceph-mon[37326]: pgmap v421: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-03T03:22:31.078 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.031+0000 7fc30b200700 1 -- 172.21.15.117:0/1780517259 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc30c10ecc0 msgr2=0x7fc30c1110b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:31.079 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.031+0000 7fc30b200700 1 --2- 172.21.15.117:0/1780517259 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc30c10ecc0 0x7fc30c1110b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc304009f90 tx=0x7fc30400b330 comp rx=0 tx=0).stop 2024-06-03T03:22:31.079 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.031+0000 7fc30b200700 1 -- 172.21.15.117:0/1780517259 shutdown_connections 2024-06-03T03:22:31.079 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.031+0000 7fc30b200700 1 --2- 172.21.15.117:0/1780517259 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc30c10ecc0 0x7fc30c1110b0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:31.079 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.031+0000 7fc30b200700 1 --2- 172.21.15.117:0/1780517259 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc30c101110 0x7fc30c1014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:31.079 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.031+0000 7fc30b200700 1 --2- 172.21.15.117:0/1780517259 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc30c101ac0 0x7fc30c10e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:31.079 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.031+0000 7fc30b200700 1 -- 172.21.15.117:0/1780517259 >> 172.21.15.117:0/1780517259 conn(0x7fc30c0fcd90 msgr2=0x7fc30c0ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:22:31.079 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.031+0000 7fc30b200700 1 -- 172.21.15.117:0/1780517259 shutdown_connections 2024-06-03T03:22:31.079 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.031+0000 7fc30b200700 1 -- 172.21.15.117:0/1780517259 wait complete. 2024-06-03T03:22:31.079 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.032+0000 7fc30b200700 1 Processor -- start 2024-06-03T03:22:31.079 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.033+0000 7fc30b200700 1 -- start start 2024-06-03T03:22:31.079 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.033+0000 7fc30b200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc30c101110 0x7fc30c19ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:31.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.033+0000 7fc30b200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc30c101ac0 0x7fc30c19b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:31.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.033+0000 7fc30b200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc30c10ecc0 0x7fc30c19f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:31.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.033+0000 7fc30b200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc30c114930 con 0x7fc30c10ecc0 2024-06-03T03:22:31.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.033+0000 7fc30b200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc30c1147b0 con 0x7fc30c101110 2024-06-03T03:22:31.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.033+0000 7fc30b200700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc30c114ab0 con 0x7fc30c101ac0 2024-06-03T03:22:31.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.033+0000 7fc30a800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc30c10ecc0 0x7fc30c19f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:31.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.034+0000 7fc30a800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc30c10ecc0 0x7fc30c19f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56412/0 (socket says 172.21.15.117:56412) 2024-06-03T03:22:31.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.034+0000 7fc30a800700 1 -- 172.21.15.117:0/1346215614 learned_addr learned my addr 172.21.15.117:0/1346215614 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:22:31.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.034+0000 7fc309400700 1 --2- 172.21.15.117:0/1346215614 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc30c101ac0 0x7fc30c19b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:31.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.034+0000 7fc309e00700 1 --2- 172.21.15.117:0/1346215614 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc30c101110 0x7fc30c19ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:31.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.034+0000 7fc30a800700 1 -- 172.21.15.117:0/1346215614 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc30c101ac0 msgr2=0x7fc30c19b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:31.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.034+0000 7fc30a800700 1 --2- 172.21.15.117:0/1346215614 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc30c101ac0 0x7fc30c19b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:31.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.034+0000 7fc30a800700 1 -- 172.21.15.117:0/1346215614 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc30c101110 msgr2=0x7fc30c19ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:31.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.034+0000 7fc30a800700 1 --2- 172.21.15.117:0/1346215614 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc30c101110 0x7fc30c19ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:31.081 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.034+0000 7fc30a800700 1 -- 172.21.15.117:0/1346215614 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc30c19fd60 con 0x7fc30c10ecc0 2024-06-03T03:22:31.081 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.034+0000 7fc30a800700 1 --2- 172.21.15.117:0/1346215614 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc30c10ecc0 0x7fc30c19f620 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc30400cfa0 tx=0x7fc30400cfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:22:31.081 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.035+0000 7fc302a00700 1 -- 172.21.15.117:0/1346215614 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc304021070 con 0x7fc30c10ecc0 2024-06-03T03:22:31.081 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.035+0000 7fc302a00700 1 -- 172.21.15.117:0/1346215614 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc304007ac0 con 0x7fc30c10ecc0 2024-06-03T03:22:31.081 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.035+0000 7fc30b200700 1 -- 172.21.15.117:0/1346215614 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc30c19fff0 con 0x7fc30c10ecc0 2024-06-03T03:22:31.082 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.035+0000 7fc302a00700 1 -- 172.21.15.117:0/1346215614 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3040052a0 con 0x7fc30c10ecc0 2024-06-03T03:22:31.082 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.035+0000 7fc30b200700 1 -- 172.21.15.117:0/1346215614 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc30c1a78d0 con 0x7fc30c10ecc0 2024-06-03T03:22:31.082 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.036+0000 7fc30b200700 1 -- 172.21.15.117:0/1346215614 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc30c062c20 con 0x7fc30c10ecc0 2024-06-03T03:22:31.082 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.038+0000 7fc302a00700 1 -- 172.21.15.117:0/1346215614 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fc304024070 con 0x7fc30c10ecc0 2024-06-03T03:22:31.082 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.040+0000 7fc302a00700 1 --2- 172.21.15.117:0/1346215614 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fc2f00a2cb0 0x7fc2f00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:31.082 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.040+0000 7fc302a00700 1 -- 172.21.15.117:0/1346215614 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7fc30400f020 con 0x7fc30c10ecc0 2024-06-03T03:22:31.082 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.040+0000 7fc309e00700 1 --2- 172.21.15.117:0/1346215614 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fc2f00a2cb0 0x7fc2f00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:31.082 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.041+0000 7fc302a00700 1 --2- 172.21.15.117:0/1346215614 >> [v2:172.21.15.151:6816/386094645,v1:172.21.15.151:6817/386094645] conn(0x7fc2f00adf10 0x7fc2f00b03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:31.082 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.041+0000 7fc302a00700 1 -- 172.21.15.117:0/1346215614 --> [v2:172.21.15.151:6816/386094645,v1:172.21.15.151:6817/386094645] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc2f00b0a80 con 0x7fc2f00adf10 2024-06-03T03:22:31.082 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.041+0000 7fc302a00700 1 -- 172.21.15.117:0/1346215614 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7fc3040d1190 con 0x7fc30c10ecc0 2024-06-03T03:22:31.082 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.041+0000 7fc309e00700 1 --2- 172.21.15.117:0/1346215614 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fc2f00a2cb0 0x7fc2f00a5170 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc009750 tx=0x7fc2fc009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:22:31.082 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.042+0000 7fc309400700 1 --2- 172.21.15.117:0/1346215614 >> [v2:172.21.15.151:6816/386094645,v1:172.21.15.151:6817/386094645] conn(0x7fc2f00adf10 0x7fc2f00b03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:31.082 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.042+0000 7fc309400700 1 --2- 172.21.15.117:0/1346215614 >> [v2:172.21.15.151:6816/386094645,v1:172.21.15.151:6817/386094645] conn(0x7fc2f00adf10 0x7fc2f00b03b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:22:31.083 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.043+0000 7fc302a00700 1 -- 172.21.15.117:0/1346215614 <== osd.7 v2:172.21.15.151:6816/386094645 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc2f00b0a80 con 0x7fc2f00adf10 2024-06-03T03:22:31.083 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.062+0000 7fc30b200700 1 -- 172.21.15.117:0/1346215614 --> [v2:172.21.15.151:6816/386094645,v1:172.21.15.151:6817/386094645] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc30c0630a0 con 0x7fc2f00adf10 2024-06-03T03:22:31.083 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.062+0000 7fc302a00700 1 -- 172.21.15.117:0/1346215614 <== osd.7 v2:172.21.15.151:6816/386094645 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fc30c0630a0 con 0x7fc2f00adf10 2024-06-03T03:22:31.083 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.063+0000 7fc30b200700 1 -- 172.21.15.117:0/1346215614 >> [v2:172.21.15.151:6816/386094645,v1:172.21.15.151:6817/386094645] conn(0x7fc2f00adf10 msgr2=0x7fc2f00b03b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:31.083 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.063+0000 7fc30b200700 1 --2- 172.21.15.117:0/1346215614 >> [v2:172.21.15.151:6816/386094645,v1:172.21.15.151:6817/386094645] conn(0x7fc2f00adf10 0x7fc2f00b03b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:31.083 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.063+0000 7fc30b200700 1 -- 172.21.15.117:0/1346215614 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fc2f00a2cb0 msgr2=0x7fc2f00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:31.083 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.063+0000 7fc30b200700 1 --2- 172.21.15.117:0/1346215614 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fc2f00a2cb0 0x7fc2f00a5170 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc009750 tx=0x7fc2fc009310 comp rx=0 tx=0).stop 2024-06-03T03:22:31.083 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.063+0000 7fc30b200700 1 -- 172.21.15.117:0/1346215614 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc30c10ecc0 msgr2=0x7fc30c19f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:31.083 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.063+0000 7fc30b200700 1 --2- 172.21.15.117:0/1346215614 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc30c10ecc0 0x7fc30c19f620 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc30400cfa0 tx=0x7fc30400cfd0 comp rx=0 tx=0).stop 2024-06-03T03:22:31.083 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.063+0000 7fc30a800700 1 -- 172.21.15.117:0/1346215614 reap_dead start 2024-06-03T03:22:31.083 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.064+0000 7fc30b200700 1 -- 172.21.15.117:0/1346215614 shutdown_connections 2024-06-03T03:22:31.083 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.064+0000 7fc30b200700 1 -- 172.21.15.117:0/1346215614 >> 172.21.15.117:0/1346215614 conn(0x7fc30c0fcd90 msgr2=0x7fc30c10fe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:22:31.083 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.064+0000 7fc30b200700 1 -- 172.21.15.117:0/1346215614 shutdown_connections 2024-06-03T03:22:31.083 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.064+0000 7fc30b200700 1 -- 172.21.15.117:0/1346215614 wait complete. 2024-06-03T03:22:31.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.752+0000 7f13c3a00700 1 -- 172.21.15.117:0/1655520725 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f13bc102740 msgr2=0x7f13bc102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:31.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.752+0000 7f13c3a00700 1 --2- 172.21.15.117:0/1655520725 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f13bc102740 0x7f13bc102b20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f13b00098f0 tx=0x7f13b0009c00 comp rx=0 tx=0).stop 2024-06-03T03:22:31.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.752+0000 7f13c3a00700 1 -- 172.21.15.117:0/1655520725 shutdown_connections 2024-06-03T03:22:31.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.752+0000 7f13c3a00700 1 --2- 172.21.15.117:0/1655520725 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13bc107840 0x7f13bc10f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:31.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.752+0000 7f13c3a00700 1 --2- 172.21.15.117:0/1655520725 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f13bc102740 0x7f13bc102b20 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:31.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.752+0000 7f13c3a00700 1 --2- 172.21.15.117:0/1655520725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13bc1030f0 0x7f13bc107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:31.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.752+0000 7f13c3a00700 1 -- 172.21.15.117:0/1655520725 >> 172.21.15.117:0/1655520725 conn(0x7f13bc0795f0 msgr2=0x7f13bc079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:22:31.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.753+0000 7f13c3a00700 1 -- 172.21.15.117:0/1655520725 shutdown_connections 2024-06-03T03:22:31.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.753+0000 7f13c3a00700 1 -- 172.21.15.117:0/1655520725 wait complete. 2024-06-03T03:22:31.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.754+0000 7f13c3a00700 1 Processor -- start 2024-06-03T03:22:31.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.754+0000 7f13c3a00700 1 -- start start 2024-06-03T03:22:31.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.754+0000 7f13c3a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13bc102740 0x7f13bc19e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:31.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.754+0000 7f13c3a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13bc1030f0 0x7f13bc19ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:31.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.754+0000 7f13c3a00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f13bc107840 0x7f13bc198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:31.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.754+0000 7f13c3a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f13bc1127d0 con 0x7f13bc102740 2024-06-03T03:22:31.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.754+0000 7f13c3a00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f13bc112650 con 0x7f13bc1030f0 2024-06-03T03:22:31.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.754+0000 7f13c3a00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f13bc112950 con 0x7f13bc107840 2024-06-03T03:22:31.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.755+0000 7f13c1000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13bc102740 0x7f13bc19e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:31.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.755+0000 7f13c1000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13bc102740 0x7f13bc19e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56454/0 (socket says 172.21.15.117:56454) 2024-06-03T03:22:31.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.755+0000 7f13c1000700 1 -- 172.21.15.117:0/3187141041 learned_addr learned my addr 172.21.15.117:0/3187141041 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:22:31.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.755+0000 7f13bbe00700 1 --2- 172.21.15.117:0/3187141041 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13bc1030f0 0x7f13bc19ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:31.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.755+0000 7f13c1a00700 1 --2- 172.21.15.117:0/3187141041 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f13bc107840 0x7f13bc198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:31.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.755+0000 7f13c1000700 1 -- 172.21.15.117:0/3187141041 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f13bc107840 msgr2=0x7f13bc198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:31.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.755+0000 7f13c1000700 1 --2- 172.21.15.117:0/3187141041 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f13bc107840 0x7f13bc198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:31.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.755+0000 7f13c1000700 1 -- 172.21.15.117:0/3187141041 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13bc1030f0 msgr2=0x7f13bc19ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:31.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.755+0000 7f13c1000700 1 --2- 172.21.15.117:0/3187141041 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13bc1030f0 0x7f13bc19ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:31.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.755+0000 7f13c1000700 1 -- 172.21.15.117:0/3187141041 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13bc199230 con 0x7f13bc102740 2024-06-03T03:22:31.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.755+0000 7f13c1000700 1 --2- 172.21.15.117:0/3187141041 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13bc102740 0x7f13bc19e8c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f13b0009fd0 tx=0x7f13b00094b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:22:31.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.756+0000 7f13b9600700 1 -- 172.21.15.117:0/3187141041 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b0025070 con 0x7f13bc102740 2024-06-03T03:22:31.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.756+0000 7f13b9600700 1 -- 172.21.15.117:0/3187141041 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13b000ba40 con 0x7f13bc102740 2024-06-03T03:22:31.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.756+0000 7f13b9600700 1 -- 172.21.15.117:0/3187141041 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b0004c50 con 0x7f13bc102740 2024-06-03T03:22:31.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.756+0000 7f13c3a00700 1 -- 172.21.15.117:0/3187141041 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13bc1994c0 con 0x7f13bc102740 2024-06-03T03:22:31.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.756+0000 7f13c3a00700 1 -- 172.21.15.117:0/3187141041 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13bc0746c0 con 0x7f13bc102740 2024-06-03T03:22:31.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.757+0000 7f13c3a00700 1 -- 172.21.15.117:0/3187141041 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f13bc062be0 con 0x7f13bc102740 2024-06-03T03:22:31.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.759+0000 7f13b9600700 1 -- 172.21.15.117:0/3187141041 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f13b0028070 con 0x7f13bc102740 2024-06-03T03:22:31.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.761+0000 7f13b9600700 1 --2- 172.21.15.117:0/3187141041 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f13a40a2cb0 0x7f13a40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:31.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.761+0000 7f13b9600700 1 -- 172.21.15.117:0/3187141041 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f13b00d04d0 con 0x7f13bc102740 2024-06-03T03:22:31.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.761+0000 7f13bbe00700 1 --2- 172.21.15.117:0/3187141041 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f13a40a2cb0 0x7f13a40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:31.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.761+0000 7f13b9600700 1 --2- 172.21.15.117:0/3187141041 >> [v2:172.21.15.117:6818/4250583193,v1:172.21.15.117:6819/4250583193] conn(0x7f13a40adf10 0x7f13a40b03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:31.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.761+0000 7f13b9600700 1 -- 172.21.15.117:0/3187141041 --> [v2:172.21.15.117:6818/4250583193,v1:172.21.15.117:6819/4250583193] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f13a40b0a80 con 0x7f13a40adf10 2024-06-03T03:22:31.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.761+0000 7f13b9600700 1 -- 172.21.15.117:0/3187141041 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7f13b00d0880 con 0x7f13bc102740 2024-06-03T03:22:31.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.761+0000 7f13c1a00700 1 --2- 172.21.15.117:0/3187141041 >> [v2:172.21.15.117:6818/4250583193,v1:172.21.15.117:6819/4250583193] conn(0x7f13a40adf10 0x7f13a40b03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:31.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.762+0000 7f13bbe00700 1 --2- 172.21.15.117:0/3187141041 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f13a40a2cb0 0x7f13a40a5170 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f13bc102070 tx=0x7f13a8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:22:31.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.762+0000 7f13c1a00700 1 --2- 172.21.15.117:0/3187141041 >> [v2:172.21.15.117:6818/4250583193,v1:172.21.15.117:6819/4250583193] conn(0x7f13a40adf10 0x7f13a40b03b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:22:31.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.763+0000 7f13b9600700 1 -- 172.21.15.117:0/3187141041 <== osd.6 v2:172.21.15.117:6818/4250583193 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f13a40b0a80 con 0x7f13a40adf10 2024-06-03T03:22:31.782 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.781+0000 7f13c3a00700 1 -- 172.21.15.117:0/3187141041 --> [v2:172.21.15.117:6818/4250583193,v1:172.21.15.117:6819/4250583193] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f13bc063060 con 0x7f13a40adf10 2024-06-03T03:22:31.782 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.782+0000 7f13b9600700 1 -- 172.21.15.117:0/3187141041 <== osd.6 v2:172.21.15.117:6818/4250583193 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f13bc063060 con 0x7f13a40adf10 2024-06-03T03:22:31.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.782+0000 7f13c3a00700 1 -- 172.21.15.117:0/3187141041 >> [v2:172.21.15.117:6818/4250583193,v1:172.21.15.117:6819/4250583193] conn(0x7f13a40adf10 msgr2=0x7f13a40b03b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:31.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.782+0000 7f13c3a00700 1 --2- 172.21.15.117:0/3187141041 >> [v2:172.21.15.117:6818/4250583193,v1:172.21.15.117:6819/4250583193] conn(0x7f13a40adf10 0x7f13a40b03b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:31.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.782+0000 7f13c3a00700 1 -- 172.21.15.117:0/3187141041 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f13a40a2cb0 msgr2=0x7f13a40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:31.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.782+0000 7f13c3a00700 1 --2- 172.21.15.117:0/3187141041 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f13a40a2cb0 0x7f13a40a5170 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f13bc102070 tx=0x7f13a8009260 comp rx=0 tx=0).stop 2024-06-03T03:22:31.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.782+0000 7f13c3a00700 1 -- 172.21.15.117:0/3187141041 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13bc102740 msgr2=0x7f13bc19e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:31.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.782+0000 7f13c3a00700 1 --2- 172.21.15.117:0/3187141041 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13bc102740 0x7f13bc19e8c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f13b0009fd0 tx=0x7f13b00094b0 comp rx=0 tx=0).stop 2024-06-03T03:22:31.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.782+0000 7f13c1a00700 1 -- 172.21.15.117:0/3187141041 reap_dead start 2024-06-03T03:22:31.784 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.782+0000 7f13c3a00700 1 -- 172.21.15.117:0/3187141041 shutdown_connections 2024-06-03T03:22:31.784 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.782+0000 7f13c3a00700 1 -- 172.21.15.117:0/3187141041 >> 172.21.15.117:0/3187141041 conn(0x7f13bc0795f0 msgr2=0x7f13bc100760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:22:31.784 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.783+0000 7f13c3a00700 1 -- 172.21.15.117:0/3187141041 shutdown_connections 2024-06-03T03:22:31.784 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.783+0000 7f13c3a00700 1 -- 172.21.15.117:0/3187141041 wait complete. 2024-06-03T03:22:32.305 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:22:31 smithi117 ceph-mon[31649]: pgmap v422: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-03T03:22:32.371 INFO:teuthology.orchestra.run.smithi117.stdout:141733920859 2024-06-03T03:22:32.371 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-06-03T03:22:32.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:22:31 smithi151 ceph-mon[37043]: pgmap v422: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-03T03:22:32.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:22:31 smithi183 ceph-mon[37326]: pgmap v422: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-03T03:22:32.867 INFO:teuthology.orchestra.run.smithi117.stdout:55834574954 2024-06-03T03:22:32.867 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-06-03T03:22:32.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.833+0000 7f4df9e00700 1 -- 172.21.15.117:0/1032131639 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4df40746c0 msgr2=0x7f4df4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:32.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.833+0000 7f4df9e00700 1 --2- 172.21.15.117:0/1032131639 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4df40746c0 0x7f4df4074aa0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4de4009990 tx=0x7f4de4009ca0 comp rx=0 tx=0).stop 2024-06-03T03:22:32.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.833+0000 7f4df9e00700 1 -- 172.21.15.117:0/1032131639 shutdown_connections 2024-06-03T03:22:32.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.833+0000 7f4df9e00700 1 --2- 172.21.15.117:0/1032131639 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4df410ee90 0x7f4df4111280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:32.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.833+0000 7f4df9e00700 1 --2- 172.21.15.117:0/1032131639 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4df40746c0 0x7f4df4074aa0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:32.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.833+0000 7f4df9e00700 1 --2- 172.21.15.117:0/1032131639 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4df4074fe0 0x7f4df410e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:32.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.833+0000 7f4df9e00700 1 -- 172.21.15.117:0/1032131639 >> 172.21.15.117:0/1032131639 conn(0x7f4df40795f0 msgr2=0x7f4df4079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:22:32.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.833+0000 7f4df9e00700 1 -- 172.21.15.117:0/1032131639 shutdown_connections 2024-06-03T03:22:32.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.833+0000 7f4df9e00700 1 -- 172.21.15.117:0/1032131639 wait complete. 2024-06-03T03:22:32.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.834+0000 7f4df9e00700 1 Processor -- start 2024-06-03T03:22:32.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.834+0000 7f4df9e00700 1 -- start start 2024-06-03T03:22:32.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.835+0000 7f4df9e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4df40746c0 0x7f4df419aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:32.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.835+0000 7f4df9e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4df4074fe0 0x7f4df419b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:32.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.835+0000 7f4df9e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4df410ee90 0x7f4df419f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:32.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.835+0000 7f4df9e00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4df4114a70 con 0x7f4df4074fe0 2024-06-03T03:22:32.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.835+0000 7f4df9e00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4df41148f0 con 0x7f4df40746c0 2024-06-03T03:22:32.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.835+0000 7f4df9e00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4df4114bf0 con 0x7f4df410ee90 2024-06-03T03:22:32.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.835+0000 7f4df2a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4df40746c0 0x7f4df419aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:32.910 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.835+0000 7f4df2a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4df40746c0 0x7f4df419aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:48724/0 (socket says 172.21.15.117:48724) 2024-06-03T03:22:32.910 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.835+0000 7f4df2a00700 1 -- 172.21.15.117:0/2411778928 learned_addr learned my addr 172.21.15.117:0/2411778928 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:22:32.910 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.835+0000 7f4df2000700 1 --2- 172.21.15.117:0/2411778928 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4df4074fe0 0x7f4df419b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:32.910 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.835+0000 7f4df3400700 1 --2- 172.21.15.117:0/2411778928 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4df410ee90 0x7f4df419f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:32.910 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.836+0000 7f4df2a00700 1 -- 172.21.15.117:0/2411778928 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4df410ee90 msgr2=0x7f4df419f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:32.910 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.836+0000 7f4df2a00700 1 --2- 172.21.15.117:0/2411778928 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4df410ee90 0x7f4df419f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:32.910 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.836+0000 7f4df2a00700 1 -- 172.21.15.117:0/2411778928 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4df4074fe0 msgr2=0x7f4df419b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:32.910 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.836+0000 7f4df2a00700 1 --2- 172.21.15.117:0/2411778928 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4df4074fe0 0x7f4df419b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:32.910 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.836+0000 7f4df2a00700 1 -- 172.21.15.117:0/2411778928 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4df419fe60 con 0x7f4df40746c0 2024-06-03T03:22:32.910 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.836+0000 7f4df2a00700 1 --2- 172.21.15.117:0/2411778928 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4df40746c0 0x7f4df419aea0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4de4005910 tx=0x7f4de400bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:22:32.910 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.836+0000 7f4df8a00700 1 -- 172.21.15.117:0/2411778928 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de4027070 con 0x7f4df40746c0 2024-06-03T03:22:32.910 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.836+0000 7f4df9e00700 1 -- 172.21.15.117:0/2411778928 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4df41a00f0 con 0x7f4df40746c0 2024-06-03T03:22:32.911 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.836+0000 7f4df8a00700 1 -- 172.21.15.117:0/2411778928 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4de4004650 con 0x7f4df40746c0 2024-06-03T03:22:32.911 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.836+0000 7f4df8a00700 1 -- 172.21.15.117:0/2411778928 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de400f460 con 0x7f4df40746c0 2024-06-03T03:22:32.911 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.837+0000 7f4df9e00700 1 -- 172.21.15.117:0/2411778928 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4df41a79d0 con 0x7f4df40746c0 2024-06-03T03:22:32.911 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.837+0000 7f4df9e00700 1 -- 172.21.15.117:0/2411778928 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4df4062ba0 con 0x7f4df40746c0 2024-06-03T03:22:32.911 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.840+0000 7f4df8a00700 1 -- 172.21.15.117:0/2411778928 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f4de402a070 con 0x7f4df40746c0 2024-06-03T03:22:32.911 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.841+0000 7f4df8a00700 1 --2- 172.21.15.117:0/2411778928 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f4dd80a2d30 0x7f4dd80a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:32.911 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.841+0000 7f4df8a00700 1 -- 172.21.15.117:0/2411778928 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f4de40d0b70 con 0x7f4df40746c0 2024-06-03T03:22:32.911 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.841+0000 7f4df2000700 1 --2- 172.21.15.117:0/2411778928 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f4dd80a2d30 0x7f4dd80a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:32.911 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.841+0000 7f4df8a00700 1 --2- 172.21.15.117:0/2411778928 >> [v2:172.21.15.183:6800/3542453623,v1:172.21.15.183:6801/3542453623] conn(0x7f4dd80adf90 0x7f4dd80b0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:32.911 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.841+0000 7f4df8a00700 1 -- 172.21.15.117:0/2411778928 --> [v2:172.21.15.183:6800/3542453623,v1:172.21.15.183:6801/3542453623] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4dd80b0b00 con 0x7f4dd80adf90 2024-06-03T03:22:32.911 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.841+0000 7f4df8a00700 1 -- 172.21.15.117:0/2411778928 <== mon.2 v2:172.21.15.151:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7f4de40d0f20 con 0x7f4df40746c0 2024-06-03T03:22:32.911 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.842+0000 7f4df3400700 1 --2- 172.21.15.117:0/2411778928 >> [v2:172.21.15.183:6800/3542453623,v1:172.21.15.183:6801/3542453623] conn(0x7f4dd80adf90 0x7f4dd80b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:32.911 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.843+0000 7f4df3400700 1 --2- 172.21.15.117:0/2411778928 >> [v2:172.21.15.183:6800/3542453623,v1:172.21.15.183:6801/3542453623] conn(0x7f4dd80adf90 0x7f4dd80b0430 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:22:32.911 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.843+0000 7f4df2000700 1 --2- 172.21.15.117:0/2411778928 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f4dd80a2d30 0x7f4dd80a51f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4df4110b50 tx=0x7f4de0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:22:32.912 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.843+0000 7f4df8a00700 1 -- 172.21.15.117:0/2411778928 <== osd.2 v2:172.21.15.183:6800/3542453623 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4dd80b0b00 con 0x7f4dd80adf90 2024-06-03T03:22:32.912 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.865+0000 7f4df9e00700 1 -- 172.21.15.117:0/2411778928 --> [v2:172.21.15.183:6800/3542453623,v1:172.21.15.183:6801/3542453623] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4df4062fa0 con 0x7f4dd80adf90 2024-06-03T03:22:32.912 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.866+0000 7f4df8a00700 1 -- 172.21.15.117:0/2411778928 <== osd.2 v2:172.21.15.183:6800/3542453623 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4df4062fa0 con 0x7f4dd80adf90 2024-06-03T03:22:32.912 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.866+0000 7f4df9e00700 1 -- 172.21.15.117:0/2411778928 >> [v2:172.21.15.183:6800/3542453623,v1:172.21.15.183:6801/3542453623] conn(0x7f4dd80adf90 msgr2=0x7f4dd80b0430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:32.912 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.866+0000 7f4df9e00700 1 --2- 172.21.15.117:0/2411778928 >> [v2:172.21.15.183:6800/3542453623,v1:172.21.15.183:6801/3542453623] conn(0x7f4dd80adf90 0x7f4dd80b0430 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:32.912 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.866+0000 7f4df9e00700 1 -- 172.21.15.117:0/2411778928 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f4dd80a2d30 msgr2=0x7f4dd80a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:32.912 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.866+0000 7f4df9e00700 1 --2- 172.21.15.117:0/2411778928 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f4dd80a2d30 0x7f4dd80a51f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4df4110b50 tx=0x7f4de0009260 comp rx=0 tx=0).stop 2024-06-03T03:22:32.913 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.866+0000 7f4df9e00700 1 -- 172.21.15.117:0/2411778928 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4df40746c0 msgr2=0x7f4df419aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:32.913 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.866+0000 7f4df9e00700 1 --2- 172.21.15.117:0/2411778928 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4df40746c0 0x7f4df419aea0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4de4005910 tx=0x7f4de400bab0 comp rx=0 tx=0).stop 2024-06-03T03:22:32.913 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.867+0000 7f4df3400700 1 -- 172.21.15.117:0/2411778928 reap_dead start 2024-06-03T03:22:32.913 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.867+0000 7f4df9e00700 1 -- 172.21.15.117:0/2411778928 shutdown_connections 2024-06-03T03:22:32.913 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.867+0000 7f4df9e00700 1 -- 172.21.15.117:0/2411778928 >> 172.21.15.117:0/2411778928 conn(0x7f4df40795f0 msgr2=0x7f4df40ff600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:22:32.913 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.867+0000 7f4df9e00700 1 -- 172.21.15.117:0/2411778928 shutdown_connections 2024-06-03T03:22:32.913 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:24.867+0000 7f4df9e00700 1 -- 172.21.15.117:0/2411778928 wait complete. 2024-06-03T03:22:33.806 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.580+0000 7fc3d0600700 1 -- 172.21.15.117:0/3917830338 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3c8107840 msgr2=0x7fc3c810f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:33.806 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.580+0000 7fc3d0600700 1 --2- 172.21.15.117:0/3917830338 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3c8107840 0x7fc3c810f3c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc3c0009fc0 tx=0x7fc3c000b330 comp rx=0 tx=0).stop 2024-06-03T03:22:33.806 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.580+0000 7fc3d0600700 1 -- 172.21.15.117:0/3917830338 shutdown_connections 2024-06-03T03:22:33.806 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.580+0000 7fc3d0600700 1 --2- 172.21.15.117:0/3917830338 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3c8107840 0x7fc3c810f3c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:33.806 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.580+0000 7fc3d0600700 1 --2- 172.21.15.117:0/3917830338 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3c8102740 0x7fc3c8102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:33.806 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.580+0000 7fc3d0600700 1 --2- 172.21.15.117:0/3917830338 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc3c81030f0 0x7fc3c8107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:33.807 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.580+0000 7fc3d0600700 1 -- 172.21.15.117:0/3917830338 >> 172.21.15.117:0/3917830338 conn(0x7fc3c80795f0 msgr2=0x7fc3c8079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:22:33.807 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.580+0000 7fc3d0600700 1 -- 172.21.15.117:0/3917830338 shutdown_connections 2024-06-03T03:22:33.807 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.580+0000 7fc3d0600700 1 -- 172.21.15.117:0/3917830338 wait complete. 2024-06-03T03:22:33.807 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.581+0000 7fc3d0600700 1 Processor -- start 2024-06-03T03:22:33.807 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.581+0000 7fc3d0600700 1 -- start start 2024-06-03T03:22:33.807 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.582+0000 7fc3d0600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc3c8102740 0x7fc3c819e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:33.807 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.582+0000 7fc3d0600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3c81030f0 0x7fc3c819ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:33.807 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.582+0000 7fc3d0600700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3c8107840 0x7fc3c81989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:33.807 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.582+0000 7fc3d0600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c81127b0 con 0x7fc3c81030f0 2024-06-03T03:22:33.807 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.582+0000 7fc3d0600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c8112630 con 0x7fc3c8102740 2024-06-03T03:22:33.807 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.582+0000 7fc3d0600700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c8112930 con 0x7fc3c8107840 2024-06-03T03:22:33.807 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.582+0000 7fc3cdc00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc3c8102740 0x7fc3c819e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:33.807 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.582+0000 7fc3cd200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3c81030f0 0x7fc3c819ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:33.808 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.582+0000 7fc3cdc00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc3c8102740 0x7fc3c819e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:47200/0 (socket says 172.21.15.117:47200) 2024-06-03T03:22:33.808 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.582+0000 7fc3cd200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3c81030f0 0x7fc3c819ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56438/0 (socket says 172.21.15.117:56438) 2024-06-03T03:22:33.808 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.582+0000 7fc3cdc00700 1 -- 172.21.15.117:0/1777138590 learned_addr learned my addr 172.21.15.117:0/1777138590 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:22:33.808 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.582+0000 7fc3ce600700 1 --2- 172.21.15.117:0/1777138590 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3c8107840 0x7fc3c81989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:33.808 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.582+0000 7fc3cdc00700 1 -- 172.21.15.117:0/1777138590 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3c8107840 msgr2=0x7fc3c81989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:33.808 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.582+0000 7fc3cdc00700 1 --2- 172.21.15.117:0/1777138590 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3c8107840 0x7fc3c81989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:33.808 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.582+0000 7fc3cdc00700 1 -- 172.21.15.117:0/1777138590 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3c81030f0 msgr2=0x7fc3c819ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:33.808 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.582+0000 7fc3cdc00700 1 --2- 172.21.15.117:0/1777138590 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3c81030f0 0x7fc3c819ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:33.808 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.582+0000 7fc3cdc00700 1 -- 172.21.15.117:0/1777138590 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3c8199110 con 0x7fc3c8102740 2024-06-03T03:22:33.808 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.582+0000 7fc3ce600700 1 --2- 172.21.15.117:0/1777138590 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3c8107840 0x7fc3c81989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T03:22:33.808 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.583+0000 7fc3cdc00700 1 --2- 172.21.15.117:0/1777138590 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc3c8102740 0x7fc3c819e7f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc00e9a0 tx=0x7fc3bc00ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:22:33.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.583+0000 7fc3c6000700 1 -- 172.21.15.117:0/1777138590 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3bc00cbd0 con 0x7fc3c8102740 2024-06-03T03:22:33.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.583+0000 7fc3d0600700 1 -- 172.21.15.117:0/1777138590 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3c8199400 con 0x7fc3c8102740 2024-06-03T03:22:33.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.583+0000 7fc3c6000700 1 -- 172.21.15.117:0/1777138590 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3bc00cd70 con 0x7fc3c8102740 2024-06-03T03:22:33.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.583+0000 7fc3c6000700 1 -- 172.21.15.117:0/1777138590 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3bc018750 con 0x7fc3c8102740 2024-06-03T03:22:33.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.583+0000 7fc3d0600700 1 -- 172.21.15.117:0/1777138590 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3c80746c0 con 0x7fc3c8102740 2024-06-03T03:22:33.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.584+0000 7fc3d0600700 1 -- 172.21.15.117:0/1777138590 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc3c8062c20 con 0x7fc3c8102740 2024-06-03T03:22:33.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.586+0000 7fc3c6000700 1 -- 172.21.15.117:0/1777138590 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fc3bc010430 con 0x7fc3c8102740 2024-06-03T03:22:33.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.588+0000 7fc3c6000700 1 --2- 172.21.15.117:0/1777138590 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fc3b00a2cb0 0x7fc3b00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:33.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.589+0000 7fc3c6000700 1 -- 172.21.15.117:0/1777138590 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7fc3bc0d0080 con 0x7fc3c8102740 2024-06-03T03:22:33.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.589+0000 7fc3cd200700 1 --2- 172.21.15.117:0/1777138590 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fc3b00a2cb0 0x7fc3b00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:33.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.589+0000 7fc3c6000700 1 --2- 172.21.15.117:0/1777138590 >> [v2:172.21.15.151:6824/3801045133,v1:172.21.15.151:6825/3801045133] conn(0x7fc3b00adf10 0x7fc3b00b03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:33.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.589+0000 7fc3c6000700 1 -- 172.21.15.117:0/1777138590 --> [v2:172.21.15.151:6824/3801045133,v1:172.21.15.151:6825/3801045133] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc3b00b0a80 con 0x7fc3b00adf10 2024-06-03T03:22:33.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.589+0000 7fc3c6000700 1 -- 172.21.15.117:0/1777138590 <== mon.2 v2:172.21.15.151:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7fc3bc0d0430 con 0x7fc3c8102740 2024-06-03T03:22:33.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.590+0000 7fc3ce600700 1 --2- 172.21.15.117:0/1777138590 >> [v2:172.21.15.151:6824/3801045133,v1:172.21.15.151:6825/3801045133] conn(0x7fc3b00adf10 0x7fc3b00b03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:33.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.590+0000 7fc3cd200700 1 --2- 172.21.15.117:0/1777138590 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fc3b00a2cb0 0x7fc3b00a5170 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8108d00 tx=0x7fc3b4009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:22:33.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.590+0000 7fc3ce600700 1 --2- 172.21.15.117:0/1777138590 >> [v2:172.21.15.151:6824/3801045133,v1:172.21.15.151:6825/3801045133] conn(0x7fc3b00adf10 0x7fc3b00b03b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:22:33.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.592+0000 7fc3c6000700 1 -- 172.21.15.117:0/1777138590 <== osd.10 v2:172.21.15.151:6824/3801045133 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc3b00b0a80 con 0x7fc3b00adf10 2024-06-03T03:22:33.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.627+0000 7fc3d0600700 1 -- 172.21.15.117:0/1777138590 --> [v2:172.21.15.151:6824/3801045133,v1:172.21.15.151:6825/3801045133] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc3c8063020 con 0x7fc3b00adf10 2024-06-03T03:22:33.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.627+0000 7fc3c6000700 1 -- 172.21.15.117:0/1777138590 <== osd.10 v2:172.21.15.151:6824/3801045133 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fc3c8063020 con 0x7fc3b00adf10 2024-06-03T03:22:33.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.628+0000 7fc3d0600700 1 -- 172.21.15.117:0/1777138590 >> [v2:172.21.15.151:6824/3801045133,v1:172.21.15.151:6825/3801045133] conn(0x7fc3b00adf10 msgr2=0x7fc3b00b03b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:33.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.628+0000 7fc3d0600700 1 --2- 172.21.15.117:0/1777138590 >> [v2:172.21.15.151:6824/3801045133,v1:172.21.15.151:6825/3801045133] conn(0x7fc3b00adf10 0x7fc3b00b03b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:33.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.628+0000 7fc3d0600700 1 -- 172.21.15.117:0/1777138590 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fc3b00a2cb0 msgr2=0x7fc3b00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:33.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.628+0000 7fc3d0600700 1 --2- 172.21.15.117:0/1777138590 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fc3b00a2cb0 0x7fc3b00a5170 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8108d00 tx=0x7fc3b4009310 comp rx=0 tx=0).stop 2024-06-03T03:22:33.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.628+0000 7fc3d0600700 1 -- 172.21.15.117:0/1777138590 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc3c8102740 msgr2=0x7fc3c819e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:33.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.628+0000 7fc3d0600700 1 --2- 172.21.15.117:0/1777138590 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc3c8102740 0x7fc3c819e7f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc00e9a0 tx=0x7fc3bc00ed60 comp rx=0 tx=0).stop 2024-06-03T03:22:33.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.628+0000 7fc3ce600700 1 -- 172.21.15.117:0/1777138590 reap_dead start 2024-06-03T03:22:33.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.628+0000 7fc3d0600700 1 -- 172.21.15.117:0/1777138590 shutdown_connections 2024-06-03T03:22:33.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.628+0000 7fc3d0600700 1 -- 172.21.15.117:0/1777138590 >> 172.21.15.117:0/1777138590 conn(0x7fc3c80795f0 msgr2=0x7fc3c8100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:22:33.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.629+0000 7fc3d0600700 1 -- 172.21.15.117:0/1777138590 shutdown_connections 2024-06-03T03:22:33.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:31.629+0000 7fc3d0600700 1 -- 172.21.15.117:0/1777138590 wait complete. 2024-06-03T03:22:34.174 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:22:34 smithi117 ceph-mon[31649]: pgmap v423: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s 2024-06-03T03:22:34.364 INFO:teuthology.orchestra.run.smithi117.stdout:77309411429 2024-06-03T03:22:34.364 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-06-03T03:22:34.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:22:34 smithi151 ceph-mon[37043]: pgmap v423: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s 2024-06-03T03:22:34.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:22:34 smithi183 ceph-mon[37326]: pgmap v423: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s 2024-06-03T03:22:34.732 INFO:teuthology.orchestra.run.smithi117.stdout:309237645368 2024-06-03T03:22:34.733 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-06-03T03:22:35.459 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:35.088+0000 7f5ae2400700 1 -- 172.21.15.117:0/99911222 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5adc101370 msgr2=0x7f5adc101750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:35.459 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:35.088+0000 7f5ae2400700 1 --2- 172.21.15.117:0/99911222 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5adc101370 0x7f5adc101750 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5ad0009990 tx=0x7f5ad0009ca0 comp rx=0 tx=0).stop 2024-06-03T03:22:35.459 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:35.088+0000 7f5ae2400700 1 -- 172.21.15.117:0/99911222 shutdown_connections 2024-06-03T03:22:35.459 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:35.088+0000 7f5ae2400700 1 --2- 172.21.15.117:0/99911222 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5adc106780 0x7f5adc106c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:35.459 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:35.088+0000 7f5ae2400700 1 --2- 172.21.15.117:0/99911222 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5adc101370 0x7f5adc101750 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:35.459 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:35.088+0000 7f5ae2400700 1 --2- 172.21.15.117:0/99911222 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5adc101c90 0x7f5adc106160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:35.459 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:35.088+0000 7f5ae2400700 1 -- 172.21.15.117:0/99911222 >> 172.21.15.117:0/99911222 conn(0x7f5adc0fcdb0 msgr2=0x7f5adc0ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:22:35.459 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:35.088+0000 7f5ae2400700 1 -- 172.21.15.117:0/99911222 shutdown_connections 2024-06-03T03:22:35.459 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:35.088+0000 7f5ae2400700 1 -- 172.21.15.117:0/99911222 wait complete. 2024-06-03T03:22:35.459 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:35.089+0000 7f5ae2400700 1 Processor -- start 2024-06-03T03:22:35.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:35.089+0000 7f5ae2400700 1 -- start start 2024-06-03T03:22:35.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:35.090+0000 7f5ae2400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5adc101370 0x7f5adc19f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:35.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:35.090+0000 7f5ae2400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5adc101c90 0x7f5adc19f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:35.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:35.090+0000 7f5ae2400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5adc106780 0x7f5adc1a3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:35.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:35.090+0000 7f5ae2400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5adc118d50 con 0x7f5adc101370 2024-06-03T03:22:35.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:35.090+0000 7f5ae2400700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5adc118bd0 con 0x7f5adc106780 2024-06-03T03:22:35.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:35.090+0000 7f5ae2400700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5adc118ed0 con 0x7f5adc101c90 2024-06-03T03:22:35.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:35.090+0000 7f5ae1000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5adc101370 0x7f5adc19f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:35.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:35.090+0000 7f5ae1000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5adc101370 0x7f5adc19f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56476/0 (socket says 172.21.15.117:56476) 2024-06-03T03:22:35.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:35.090+0000 7f5ae1000700 1 -- 172.21.15.117:0/574108935 learned_addr learned my addr 172.21.15.117:0/574108935 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:22:35.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:35.090+0000 7f5adbe00700 1 --2- 172.21.15.117:0/574108935 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5adc101c90 0x7f5adc19f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:35.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:35.090+0000 7f5ae1a00700 1 --2- 172.21.15.117:0/574108935 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5adc106780 0x7f5adc1a3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:35.461 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:35.091+0000 7f5adbe00700 1 -- 172.21.15.117:0/574108935 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5adc106780 msgr2=0x7f5adc1a3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:35.461 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:35.091+0000 7f5adbe00700 1 --2- 172.21.15.117:0/574108935 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5adc106780 0x7f5adc1a3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:35.461 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:35.091+0000 7f5adbe00700 1 -- 172.21.15.117:0/574108935 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5adc101370 msgr2=0x7f5adc19f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:35.461 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:35.091+0000 7f5adbe00700 1 --2- 172.21.15.117:0/574108935 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5adc101370 0x7f5adc19f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:35.461 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:35.091+0000 7f5adbe00700 1 -- 172.21.15.117:0/574108935 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5adc1a4200 con 0x7f5adc101c90 2024-06-03T03:22:35.461 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:35.091+0000 7f5ae1000700 1 --2- 172.21.15.117:0/574108935 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5adc101370 0x7f5adc19f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:22:35.461 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:35.091+0000 7f5adbe00700 1 --2- 172.21.15.117:0/574108935 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5adc101c90 0x7f5adc19f780 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5ac800e9c0 tx=0x7f5ac800ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:22:35.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:35.091+0000 7f5ad9600700 1 -- 172.21.15.117:0/574108935 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac8004220 con 0x7f5adc101c90 2024-06-03T03:22:35.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:35.091+0000 7f5ad9600700 1 -- 172.21.15.117:0/574108935 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ac80043c0 con 0x7f5adc101c90 2024-06-03T03:22:35.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:35.091+0000 7f5ae2400700 1 -- 172.21.15.117:0/574108935 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5adc1a44f0 con 0x7f5adc101c90 2024-06-03T03:22:35.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:35.091+0000 7f5ad9600700 1 -- 172.21.15.117:0/574108935 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac8003cd0 con 0x7f5adc101c90 2024-06-03T03:22:35.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:35.091+0000 7f5ae2400700 1 -- 172.21.15.117:0/574108935 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5adc1abd70 con 0x7f5adc101c90 2024-06-03T03:22:35.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:35.092+0000 7f5ae2400700 1 -- 172.21.15.117:0/574108935 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5adc105b60 con 0x7f5adc101c90 2024-06-03T03:22:35.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:35.094+0000 7f5ad9600700 1 -- 172.21.15.117:0/574108935 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f5ac800c7c0 con 0x7f5adc101c90 2024-06-03T03:22:35.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:35.096+0000 7f5ad9600700 1 --2- 172.21.15.117:0/574108935 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f5ac40a2cb0 0x7f5ac40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:35.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:35.096+0000 7f5ad9600700 1 -- 172.21.15.117:0/574108935 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f5ac80cf900 con 0x7f5adc101c90 2024-06-03T03:22:35.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:35.096+0000 7f5ae1000700 1 --2- 172.21.15.117:0/574108935 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f5ac40a2cb0 0x7f5ac40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:35.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:35.096+0000 7f5ad9600700 1 --2- 172.21.15.117:0/574108935 >> [v2:172.21.15.183:6808/2985874686,v1:172.21.15.183:6809/2985874686] conn(0x7f5ac40adf10 0x7f5ac40b03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:35.463 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:35.096+0000 7f5ad9600700 1 -- 172.21.15.117:0/574108935 --> [v2:172.21.15.183:6808/2985874686,v1:172.21.15.183:6809/2985874686] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5ac800be50 con 0x7f5ac40adf10 2024-06-03T03:22:35.463 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:35.096+0000 7f5ad9600700 1 -- 172.21.15.117:0/574108935 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7f5ac80cfcb0 con 0x7f5adc101c90 2024-06-03T03:22:35.463 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:35.097+0000 7f5ae1a00700 1 --2- 172.21.15.117:0/574108935 >> [v2:172.21.15.183:6808/2985874686,v1:172.21.15.183:6809/2985874686] conn(0x7f5ac40adf10 0x7f5ac40b03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:35.463 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:35.097+0000 7f5ae1000700 1 --2- 172.21.15.117:0/574108935 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f5ac40a2cb0 0x7f5ac40a5170 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5ad0009750 tx=0x7f5ad000b280 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:22:35.463 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:35.097+0000 7f5ae1a00700 1 --2- 172.21.15.117:0/574108935 >> [v2:172.21.15.183:6808/2985874686,v1:172.21.15.183:6809/2985874686] conn(0x7f5ac40adf10 0x7f5ac40b03b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:22:35.463 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:35.098+0000 7f5ad9600700 1 -- 172.21.15.117:0/574108935 <== osd.5 v2:172.21.15.183:6808/2985874686 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5ac800be50 con 0x7f5ac40adf10 2024-06-03T03:22:35.463 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:35.117+0000 7f5ae2400700 1 -- 172.21.15.117:0/574108935 --> [v2:172.21.15.183:6808/2985874686,v1:172.21.15.183:6809/2985874686] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5adc06aa50 con 0x7f5ac40adf10 2024-06-03T03:22:35.463 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:35.117+0000 7f5ad9600700 1 -- 172.21.15.117:0/574108935 <== osd.5 v2:172.21.15.183:6808/2985874686 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5adc06aa50 con 0x7f5ac40adf10 2024-06-03T03:22:35.463 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:35.118+0000 7f5ae2400700 1 -- 172.21.15.117:0/574108935 >> [v2:172.21.15.183:6808/2985874686,v1:172.21.15.183:6809/2985874686] conn(0x7f5ac40adf10 msgr2=0x7f5ac40b03b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:35.463 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:35.118+0000 7f5ae2400700 1 --2- 172.21.15.117:0/574108935 >> [v2:172.21.15.183:6808/2985874686,v1:172.21.15.183:6809/2985874686] conn(0x7f5ac40adf10 0x7f5ac40b03b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:35.463 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:35.118+0000 7f5ae2400700 1 -- 172.21.15.117:0/574108935 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f5ac40a2cb0 msgr2=0x7f5ac40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:35.463 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:35.118+0000 7f5ae2400700 1 --2- 172.21.15.117:0/574108935 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f5ac40a2cb0 0x7f5ac40a5170 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5ad0009750 tx=0x7f5ad000b280 comp rx=0 tx=0).stop 2024-06-03T03:22:35.463 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:35.118+0000 7f5ae2400700 1 -- 172.21.15.117:0/574108935 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5adc101c90 msgr2=0x7f5adc19f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:35.464 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:35.118+0000 7f5ae2400700 1 --2- 172.21.15.117:0/574108935 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5adc101c90 0x7f5adc19f780 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5ac800e9c0 tx=0x7f5ac800ecd0 comp rx=0 tx=0).stop 2024-06-03T03:22:35.464 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:35.118+0000 7f5ae1a00700 1 -- 172.21.15.117:0/574108935 reap_dead start 2024-06-03T03:22:35.464 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:35.118+0000 7f5ae2400700 1 -- 172.21.15.117:0/574108935 shutdown_connections 2024-06-03T03:22:35.464 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:35.118+0000 7f5ae2400700 1 -- 172.21.15.117:0/574108935 >> 172.21.15.117:0/574108935 conn(0x7f5adc0fcdb0 msgr2=0x7f5adc104130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:22:35.464 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:35.118+0000 7f5ae2400700 1 -- 172.21.15.117:0/574108935 shutdown_connections 2024-06-03T03:22:35.464 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:35.118+0000 7f5ae2400700 1 -- 172.21.15.117:0/574108935 wait complete. 2024-06-03T03:22:36.374 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:22:36 smithi117 ceph-mon[31649]: pgmap v424: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:22:36.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:22:36 smithi151 ceph-mon[37043]: pgmap v424: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:22:36.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:22:36 smithi183 ceph-mon[37326]: pgmap v424: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:22:36.583 INFO:teuthology.orchestra.run.smithi117.stdout:34359738480 2024-06-03T03:22:36.583 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-06-03T03:22:36.897 INFO:teuthology.orchestra.run.smithi117.stdout:253403070532 2024-06-03T03:22:36.898 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-06-03T03:22:37.313 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:22:37 smithi117 ceph-mon[31649]: pgmap v425: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-03T03:22:37.371 INFO:teuthology.orchestra.run.smithi117.stdout:163208757335 2024-06-03T03:22:37.371 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-06-03T03:22:37.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:22:37 smithi151 ceph-mon[37043]: pgmap v425: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-03T03:22:37.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:22:37 smithi183 ceph-mon[37326]: pgmap v425: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-03T03:22:38.045 INFO:teuthology.orchestra.run.smithi117.stdout:197568495696 2024-06-03T03:22:38.045 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-06-03T03:22:39.046 INFO:teuthology.orchestra.run.smithi117.stdout:219043332170 2024-06-03T03:22:39.046 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-06-03T03:22:39.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.272+0000 7f6377800700 1 -- 172.21.15.117:0/753082377 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6370102740 msgr2=0x7f6370102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:39.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.272+0000 7f6377800700 1 --2- 172.21.15.117:0/753082377 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6370102740 0x7f6370102b20 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f63640099c0 tx=0x7f6364009cd0 comp rx=0 tx=0).stop 2024-06-03T03:22:39.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.272+0000 7f6377800700 1 -- 172.21.15.117:0/753082377 shutdown_connections 2024-06-03T03:22:39.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.272+0000 7f6377800700 1 --2- 172.21.15.117:0/753082377 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6370107840 0x7f637010f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:39.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.272+0000 7f6377800700 1 --2- 172.21.15.117:0/753082377 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6370102740 0x7f6370102b20 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:39.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.272+0000 7f6377800700 1 --2- 172.21.15.117:0/753082377 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f63701030f0 0x7f6370107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:39.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.272+0000 7f6377800700 1 -- 172.21.15.117:0/753082377 >> 172.21.15.117:0/753082377 conn(0x7f63700795f0 msgr2=0x7f6370079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:22:39.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.273+0000 7f6377800700 1 -- 172.21.15.117:0/753082377 shutdown_connections 2024-06-03T03:22:39.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.273+0000 7f6377800700 1 -- 172.21.15.117:0/753082377 wait complete. 2024-06-03T03:22:39.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.274+0000 7f6377800700 1 Processor -- start 2024-06-03T03:22:39.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.274+0000 7f6377800700 1 -- start start 2024-06-03T03:22:39.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.274+0000 7f6377800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6370102740 0x7f637019ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:39.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.274+0000 7f6377800700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f63701030f0 0x7f637019b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:39.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.274+0000 7f6377800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6370107840 0x7f637019f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:39.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.274+0000 7f6377800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f63701127b0 con 0x7f6370107840 2024-06-03T03:22:39.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.274+0000 7f6377800700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6370112630 con 0x7f6370102740 2024-06-03T03:22:39.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.274+0000 7f6377800700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6370112930 con 0x7f63701030f0 2024-06-03T03:22:39.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.275+0000 7f6375800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6370107840 0x7f637019f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:39.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.275+0000 7f6375800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6370107840 0x7f637019f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:42196/0 (socket says 172.21.15.117:42196) 2024-06-03T03:22:39.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.275+0000 7f636fe00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f63701030f0 0x7f637019b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:39.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.275+0000 7f6375800700 1 -- 172.21.15.117:0/11408482 learned_addr learned my addr 172.21.15.117:0/11408482 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:22:39.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.275+0000 7f6374e00700 1 --2- 172.21.15.117:0/11408482 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6370102740 0x7f637019ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:39.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.275+0000 7f636fe00700 1 -- 172.21.15.117:0/11408482 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6370102740 msgr2=0x7f637019ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:39.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.275+0000 7f636fe00700 1 --2- 172.21.15.117:0/11408482 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6370102740 0x7f637019ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:39.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.275+0000 7f636fe00700 1 -- 172.21.15.117:0/11408482 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6370107840 msgr2=0x7f637019f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:39.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.275+0000 7f636fe00700 1 --2- 172.21.15.117:0/11408482 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6370107840 0x7f637019f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:39.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.275+0000 7f636fe00700 1 -- 172.21.15.117:0/11408482 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f637019fde0 con 0x7f63701030f0 2024-06-03T03:22:39.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.276+0000 7f636fe00700 1 --2- 172.21.15.117:0/11408482 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f63701030f0 0x7f637019b360 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f635c00e9f0 tx=0x7f635c00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:22:39.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.276+0000 7f636d600700 1 -- 172.21.15.117:0/11408482 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f635c00cb70 con 0x7f63701030f0 2024-06-03T03:22:39.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.276+0000 7f6377800700 1 -- 172.21.15.117:0/11408482 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63701a00d0 con 0x7f63701030f0 2024-06-03T03:22:39.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.276+0000 7f6377800700 1 -- 172.21.15.117:0/11408482 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63701a7950 con 0x7f63701030f0 2024-06-03T03:22:39.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.276+0000 7f636d600700 1 -- 172.21.15.117:0/11408482 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f635c00ee90 con 0x7f63701030f0 2024-06-03T03:22:39.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.276+0000 7f636d600700 1 -- 172.21.15.117:0/11408482 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f635c003e00 con 0x7f63701030f0 2024-06-03T03:22:39.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.277+0000 7f6377800700 1 -- 172.21.15.117:0/11408482 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6370075a20 con 0x7f63701030f0 2024-06-03T03:22:39.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.279+0000 7f636d600700 1 -- 172.21.15.117:0/11408482 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f635c004000 con 0x7f63701030f0 2024-06-03T03:22:39.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.282+0000 7f636d600700 1 --2- 172.21.15.117:0/11408482 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f63580a2d30 0x7f63580a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:39.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.282+0000 7f636d600700 1 -- 172.21.15.117:0/11408482 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f635c0cf610 con 0x7f63701030f0 2024-06-03T03:22:39.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.282+0000 7f6374e00700 1 --2- 172.21.15.117:0/11408482 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f63580a2d30 0x7f63580a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:39.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.282+0000 7f636d600700 1 --2- 172.21.15.117:0/11408482 >> [v2:172.21.15.117:6810/589012315,v1:172.21.15.117:6811/589012315] conn(0x7f63580adf90 0x7f63580b0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:39.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.282+0000 7f636d600700 1 -- 172.21.15.117:0/11408482 --> [v2:172.21.15.117:6810/589012315,v1:172.21.15.117:6811/589012315] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f635c00be50 con 0x7f63580adf90 2024-06-03T03:22:39.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.282+0000 7f636d600700 1 -- 172.21.15.117:0/11408482 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7f635c0d6050 con 0x7f63701030f0 2024-06-03T03:22:39.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.283+0000 7f6375800700 1 --2- 172.21.15.117:0/11408482 >> [v2:172.21.15.117:6810/589012315,v1:172.21.15.117:6811/589012315] conn(0x7f63580adf90 0x7f63580b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:39.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.283+0000 7f6374e00700 1 --2- 172.21.15.117:0/11408482 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f63580a2d30 0x7f63580a51f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6364000c00 tx=0x7f6364005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:22:39.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.283+0000 7f6375800700 1 --2- 172.21.15.117:0/11408482 >> [v2:172.21.15.117:6810/589012315,v1:172.21.15.117:6811/589012315] conn(0x7f63580adf90 0x7f63580b0430 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:22:39.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.284+0000 7f636d600700 1 -- 172.21.15.117:0/11408482 <== osd.3 v2:172.21.15.117:6810/589012315 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f635c00be50 con 0x7f63580adf90 2024-06-03T03:22:39.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.303+0000 7f6377800700 1 -- 172.21.15.117:0/11408482 --> [v2:172.21.15.117:6810/589012315,v1:172.21.15.117:6811/589012315] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6370103bd0 con 0x7f63580adf90 2024-06-03T03:22:39.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.303+0000 7f636d600700 1 -- 172.21.15.117:0/11408482 <== osd.3 v2:172.21.15.117:6810/589012315 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6370103bd0 con 0x7f63580adf90 2024-06-03T03:22:39.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.304+0000 7f6377800700 1 -- 172.21.15.117:0/11408482 >> [v2:172.21.15.117:6810/589012315,v1:172.21.15.117:6811/589012315] conn(0x7f63580adf90 msgr2=0x7f63580b0430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:39.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.304+0000 7f6377800700 1 --2- 172.21.15.117:0/11408482 >> [v2:172.21.15.117:6810/589012315,v1:172.21.15.117:6811/589012315] conn(0x7f63580adf90 0x7f63580b0430 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:39.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.304+0000 7f6377800700 1 -- 172.21.15.117:0/11408482 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f63580a2d30 msgr2=0x7f63580a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:39.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.304+0000 7f6377800700 1 --2- 172.21.15.117:0/11408482 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f63580a2d30 0x7f63580a51f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6364000c00 tx=0x7f6364005fb0 comp rx=0 tx=0).stop 2024-06-03T03:22:39.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.304+0000 7f6377800700 1 -- 172.21.15.117:0/11408482 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f63701030f0 msgr2=0x7f637019b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:39.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.304+0000 7f6377800700 1 --2- 172.21.15.117:0/11408482 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f63701030f0 0x7f637019b360 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f635c00e9f0 tx=0x7f635c00ed00 comp rx=0 tx=0).stop 2024-06-03T03:22:39.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.304+0000 7f6375800700 1 -- 172.21.15.117:0/11408482 reap_dead start 2024-06-03T03:22:39.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.304+0000 7f6377800700 1 -- 172.21.15.117:0/11408482 shutdown_connections 2024-06-03T03:22:39.206 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.304+0000 7f6377800700 1 -- 172.21.15.117:0/11408482 >> 172.21.15.117:0/11408482 conn(0x7f63700795f0 msgr2=0x7f6370108990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:22:39.206 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.304+0000 7f6377800700 1 -- 172.21.15.117:0/11408482 shutdown_connections 2024-06-03T03:22:39.206 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.304+0000 7f6377800700 1 -- 172.21.15.117:0/11408482 wait complete. 2024-06-03T03:22:39.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.624+0000 7f540e800700 1 -- 172.21.15.117:0/1831162573 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54081030f0 msgr2=0x7f5408107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:39.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.624+0000 7f540e800700 1 --2- 172.21.15.117:0/1831162573 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54081030f0 0x7f5408107110 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f53f40099c0 tx=0x7f53f4009cd0 comp rx=0 tx=0).stop 2024-06-03T03:22:39.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.624+0000 7f540e800700 1 -- 172.21.15.117:0/1831162573 shutdown_connections 2024-06-03T03:22:39.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.624+0000 7f540e800700 1 --2- 172.21.15.117:0/1831162573 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5408107840 0x7f540810f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:39.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.624+0000 7f540e800700 1 --2- 172.21.15.117:0/1831162573 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5408102740 0x7f5408102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:39.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.624+0000 7f540e800700 1 --2- 172.21.15.117:0/1831162573 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54081030f0 0x7f5408107110 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:39.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.624+0000 7f540e800700 1 -- 172.21.15.117:0/1831162573 >> 172.21.15.117:0/1831162573 conn(0x7f54080795f0 msgr2=0x7f5408079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:22:39.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.624+0000 7f540e800700 1 -- 172.21.15.117:0/1831162573 shutdown_connections 2024-06-03T03:22:39.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.624+0000 7f540e800700 1 -- 172.21.15.117:0/1831162573 wait complete. 2024-06-03T03:22:39.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.625+0000 7f540e800700 1 Processor -- start 2024-06-03T03:22:39.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.625+0000 7f540e800700 1 -- start start 2024-06-03T03:22:39.538 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.626+0000 7f540e800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5408102740 0x7f540819adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:39.538 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.626+0000 7f540e800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f54081030f0 0x7f540819b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:39.538 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.626+0000 7f540e800700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5408107840 0x7f540819f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:39.538 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.626+0000 7f540e800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f54081127a0 con 0x7f5408102740 2024-06-03T03:22:39.538 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.626+0000 7f540e800700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5408112620 con 0x7f54081030f0 2024-06-03T03:22:39.538 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.626+0000 7f540e800700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5408112920 con 0x7f5408107840 2024-06-03T03:22:39.538 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.626+0000 7f5407400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5408102740 0x7f540819adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:39.538 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.626+0000 7f5407400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5408102740 0x7f540819adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:42202/0 (socket says 172.21.15.117:42202) 2024-06-03T03:22:39.538 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.626+0000 7f5407400700 1 -- 172.21.15.117:0/1495368211 learned_addr learned my addr 172.21.15.117:0/1495368211 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:22:39.538 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.626+0000 7f5407e00700 1 --2- 172.21.15.117:0/1495368211 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5408107840 0x7f540819f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:39.538 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.626+0000 7f5406a00700 1 --2- 172.21.15.117:0/1495368211 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f54081030f0 0x7f540819b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:39.538 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.626+0000 7f5407e00700 1 -- 172.21.15.117:0/1495368211 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f54081030f0 msgr2=0x7f540819b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:39.538 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.626+0000 7f5407e00700 1 --2- 172.21.15.117:0/1495368211 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f54081030f0 0x7f540819b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:39.539 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.626+0000 7f5407e00700 1 -- 172.21.15.117:0/1495368211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5408102740 msgr2=0x7f540819adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:39.539 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.626+0000 7f5407e00700 1 --2- 172.21.15.117:0/1495368211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5408102740 0x7f540819adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:39.539 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.626+0000 7f5407e00700 1 -- 172.21.15.117:0/1495368211 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f540819fe00 con 0x7f5408107840 2024-06-03T03:22:39.539 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.626+0000 7f5407400700 1 --2- 172.21.15.117:0/1495368211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5408102740 0x7f540819adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:22:39.539 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.627+0000 7f5407e00700 1 --2- 172.21.15.117:0/1495368211 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5408107840 0x7f540819f6c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f540000efc0 tx=0x7f540000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:22:39.539 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.627+0000 7f53ffe00700 1 -- 172.21.15.117:0/1495368211 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54000092e0 con 0x7f5408107840 2024-06-03T03:22:39.539 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.627+0000 7f540e800700 1 -- 172.21.15.117:0/1495368211 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54081a0090 con 0x7f5408107840 2024-06-03T03:22:39.539 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.627+0000 7f53ffe00700 1 -- 172.21.15.117:0/1495368211 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5400004bb0 con 0x7f5408107840 2024-06-03T03:22:39.539 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.627+0000 7f53ffe00700 1 -- 172.21.15.117:0/1495368211 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5400004280 con 0x7f5408107840 2024-06-03T03:22:39.539 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.627+0000 7f540e800700 1 -- 172.21.15.117:0/1495368211 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54081a7970 con 0x7f5408107840 2024-06-03T03:22:39.539 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.628+0000 7f540e800700 1 -- 172.21.15.117:0/1495368211 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f540810ea70 con 0x7f5408107840 2024-06-03T03:22:39.539 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.630+0000 7f53ffe00700 1 -- 172.21.15.117:0/1495368211 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f540001b030 con 0x7f5408107840 2024-06-03T03:22:39.539 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.632+0000 7f53ffe00700 1 --2- 172.21.15.117:0/1495368211 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f53ec0a2cb0 0x7f53ec0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:39.539 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.632+0000 7f53ffe00700 1 -- 172.21.15.117:0/1495368211 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f54000cfa90 con 0x7f5408107840 2024-06-03T03:22:39.540 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.632+0000 7f5407400700 1 --2- 172.21.15.117:0/1495368211 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f53ec0a2cb0 0x7f53ec0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:39.540 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.632+0000 7f53ffe00700 1 --2- 172.21.15.117:0/1495368211 >> [v2:172.21.15.117:6826/2538315141,v1:172.21.15.117:6827/2538315141] conn(0x7f53ec0adf10 0x7f53ec0b03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:39.540 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.632+0000 7f53ffe00700 1 -- 172.21.15.117:0/1495368211 --> [v2:172.21.15.117:6826/2538315141,v1:172.21.15.117:6827/2538315141] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f540001fe50 con 0x7f53ec0adf10 2024-06-03T03:22:39.540 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.632+0000 7f53ffe00700 1 -- 172.21.15.117:0/1495368211 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7f54000cfe40 con 0x7f5408107840 2024-06-03T03:22:39.540 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.633+0000 7f5406a00700 1 --2- 172.21.15.117:0/1495368211 >> [v2:172.21.15.117:6826/2538315141,v1:172.21.15.117:6827/2538315141] conn(0x7f53ec0adf10 0x7f53ec0b03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:39.540 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.633+0000 7f5407400700 1 --2- 172.21.15.117:0/1495368211 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f53ec0a2cb0 0x7f53ec0a5170 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f53f8011550 tx=0x7f53f8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:22:39.540 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.633+0000 7f5406a00700 1 --2- 172.21.15.117:0/1495368211 >> [v2:172.21.15.117:6826/2538315141,v1:172.21.15.117:6827/2538315141] conn(0x7f53ec0adf10 0x7f53ec0b03b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:22:39.540 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.634+0000 7f53ffe00700 1 -- 172.21.15.117:0/1495368211 <== osd.9 v2:172.21.15.117:6826/2538315141 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f540001fe50 con 0x7f53ec0adf10 2024-06-03T03:22:39.541 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.652+0000 7f540e800700 1 -- 172.21.15.117:0/1495368211 --> [v2:172.21.15.117:6826/2538315141,v1:172.21.15.117:6827/2538315141] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5408103bd0 con 0x7f53ec0adf10 2024-06-03T03:22:39.541 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.653+0000 7f53ffe00700 1 -- 172.21.15.117:0/1495368211 <== osd.9 v2:172.21.15.117:6826/2538315141 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5408103bd0 con 0x7f53ec0adf10 2024-06-03T03:22:39.541 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.653+0000 7f540e800700 1 -- 172.21.15.117:0/1495368211 >> [v2:172.21.15.117:6826/2538315141,v1:172.21.15.117:6827/2538315141] conn(0x7f53ec0adf10 msgr2=0x7f53ec0b03b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:39.541 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.653+0000 7f540e800700 1 --2- 172.21.15.117:0/1495368211 >> [v2:172.21.15.117:6826/2538315141,v1:172.21.15.117:6827/2538315141] conn(0x7f53ec0adf10 0x7f53ec0b03b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:39.541 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.653+0000 7f540e800700 1 -- 172.21.15.117:0/1495368211 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f53ec0a2cb0 msgr2=0x7f53ec0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:39.541 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.653+0000 7f540e800700 1 --2- 172.21.15.117:0/1495368211 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f53ec0a2cb0 0x7f53ec0a5170 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f53f8011550 tx=0x7f53f8009260 comp rx=0 tx=0).stop 2024-06-03T03:22:39.541 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.653+0000 7f540e800700 1 -- 172.21.15.117:0/1495368211 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5408107840 msgr2=0x7f540819f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:39.542 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.653+0000 7f540e800700 1 --2- 172.21.15.117:0/1495368211 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5408107840 0x7f540819f6c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f540000efc0 tx=0x7f540000c3a0 comp rx=0 tx=0).stop 2024-06-03T03:22:39.542 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.653+0000 7f5407e00700 1 -- 172.21.15.117:0/1495368211 reap_dead start 2024-06-03T03:22:39.542 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.654+0000 7f540e800700 1 -- 172.21.15.117:0/1495368211 shutdown_connections 2024-06-03T03:22:39.542 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.654+0000 7f540e800700 1 -- 172.21.15.117:0/1495368211 >> 172.21.15.117:0/1495368211 conn(0x7f54080795f0 msgr2=0x7f54081007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:22:39.542 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.654+0000 7f540e800700 1 -- 172.21.15.117:0/1495368211 shutdown_connections 2024-06-03T03:22:39.542 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:38.654+0000 7f540e800700 1 -- 172.21.15.117:0/1495368211 wait complete. 2024-06-03T03:22:40.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:22:39 smithi117 ceph-mon[31649]: pgmap v426: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s 2024-06-03T03:22:40.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:22:39 smithi183 ceph-mon[37326]: pgmap v426: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s 2024-06-03T03:22:40.387 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:22:40.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:22:39 smithi151 ceph-mon[37043]: pgmap v426: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s 2024-06-03T03:22:40.417 INFO:teuthology.orchestra.run.smithi117.stdout:107374182498 2024-06-03T03:22:40.418 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-06-03T03:22:40.974 INFO:teuthology.orchestra.run.smithi117.stdout:287762808895 2024-06-03T03:22:40.974 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-06-03T03:22:41.306 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:22:41.851 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:22:41.860 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:22:41.861 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:22:41.865 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:22:41.871 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:22:41.872 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:22:41.873 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:22:42.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:22:41 smithi117 ceph-mon[31649]: pgmap v427: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-06-03T03:22:42.231 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:22:41 smithi183 ceph-mon[37326]: pgmap v427: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-06-03T03:22:42.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:22:41 smithi151 ceph-mon[37043]: pgmap v427: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-06-03T03:22:42.451 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:22:44.166 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:22:43 smithi117 ceph-mon[31649]: pgmap v428: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-03T03:22:44.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:22:43 smithi183 ceph-mon[37326]: pgmap v428: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-03T03:22:44.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:22:43 smithi151 ceph-mon[37043]: pgmap v428: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-03T03:22:46.146 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:22:45 smithi117 ceph-mon[31649]: pgmap v429: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-06-03T03:22:46.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:22:45 smithi183 ceph-mon[37326]: pgmap v429: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-06-03T03:22:46.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:22:45 smithi151 ceph-mon[37043]: pgmap v429: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-06-03T03:22:48.235 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:22:47 smithi183 ceph-mon[37326]: pgmap v430: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-03T03:22:48.236 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:22:47 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:22:48.236 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:22:47 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:22:48.236 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:22:47 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:22:48.236 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:22:47 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:22:48.236 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:22:47 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:22:48.261 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:22:47 smithi117 ceph-mon[31649]: pgmap v430: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-03T03:22:48.261 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:22:47 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:22:48.261 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:22:47 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:22:48.261 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:22:47 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:22:48.262 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:22:47 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:22:48.262 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:22:47 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:22:48.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:22:47 smithi151 ceph-mon[37043]: pgmap v430: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-03T03:22:48.408 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:22:47 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:22:48.408 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:22:47 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:22:48.408 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:22:47 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:22:48.408 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:22:47 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:22:48.408 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:22:47 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:22:50.215 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:22:49 smithi117 ceph-mon[31649]: pgmap v431: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-03T03:22:50.215 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:22:49 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:22:50.215 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:22:49 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:22:50.215 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:22:49 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:22:50.231 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:22:49 smithi183 ceph-mon[37326]: pgmap v431: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-03T03:22:50.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:22:49 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:22:50.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:22:49 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:22:50.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:22:49 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:22:50.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:22:49 smithi151 ceph-mon[37043]: pgmap v431: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-03T03:22:50.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:22:49 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:22:50.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:22:49 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:22:50.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:22:49 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:22:52.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:22:51 smithi117 ceph-mon[31649]: pgmap v432: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-06-03T03:22:52.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:22:51 smithi183 ceph-mon[37326]: pgmap v432: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-06-03T03:22:52.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:22:51 smithi151 ceph-mon[37043]: pgmap v432: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-06-03T03:22:54.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:22:53 smithi183 ceph-mon[37326]: pgmap v433: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-06-03T03:22:54.260 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:22:53 smithi117 ceph-mon[31649]: pgmap v433: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-06-03T03:22:54.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:22:53 smithi151 ceph-mon[37043]: pgmap v433: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-06-03T03:22:55.932 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.355+0000 7fd54f200700 1 -- 172.21.15.117:0/3711993123 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd5500748c0 msgr2=0x7fd550074ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:55.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.355+0000 7fd54f200700 1 --2- 172.21.15.117:0/3711993123 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd5500748c0 0x7fd550074ca0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd540009990 tx=0x7fd540009ca0 comp rx=0 tx=0).stop 2024-06-03T03:22:55.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.355+0000 7fd54f200700 1 -- 172.21.15.117:0/3711993123 shutdown_connections 2024-06-03T03:22:55.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.355+0000 7fd54f200700 1 --2- 172.21.15.117:0/3711993123 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd55010ec00 0x7fd550111070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:55.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.355+0000 7fd54f200700 1 --2- 172.21.15.117:0/3711993123 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd5500748c0 0x7fd550074ca0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:55.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.355+0000 7fd54f200700 1 --2- 172.21.15.117:0/3711993123 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5500751e0 0x7fd55010e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:55.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.355+0000 7fd54f200700 1 -- 172.21.15.117:0/3711993123 >> 172.21.15.117:0/3711993123 conn(0x7fd5500fdf10 msgr2=0x7fd550100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:22:55.934 INFO:teuthology.orchestra.run.smithi117.stdout:343597383736 2024-06-03T03:22:55.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.356+0000 7fd54f200700 1 -- 172.21.15.117:0/3711993123 shutdown_connections 2024-06-03T03:22:55.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.356+0000 7fd54f200700 1 -- 172.21.15.117:0/3711993123 wait complete. 2024-06-03T03:22:55.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.357+0000 7fd54f200700 1 Processor -- start 2024-06-03T03:22:55.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.357+0000 7fd54f200700 1 -- start start 2024-06-03T03:22:55.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.357+0000 7fd54f200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5500748c0 0x7fd55019ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:55.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.357+0000 7fd54f200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5500751e0 0x7fd55019b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:55.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.357+0000 7fd54f200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd55010ec00 0x7fd55019f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:55.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.357+0000 7fd54f200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd5501148d0 con 0x7fd5500751e0 2024-06-03T03:22:55.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.357+0000 7fd54f200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd550114750 con 0x7fd55010ec00 2024-06-03T03:22:55.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.357+0000 7fd54f200700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd550114a50 con 0x7fd5500748c0 2024-06-03T03:22:55.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.357+0000 7fd54de00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5500748c0 0x7fd55019ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:55.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.357+0000 7fd54de00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5500748c0 0x7fd55019ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:59492/0 (socket says 172.21.15.117:59492) 2024-06-03T03:22:55.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.357+0000 7fd54d400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5500751e0 0x7fd55019b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:55.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.357+0000 7fd54de00700 1 -- 172.21.15.117:0/2912165585 learned_addr learned my addr 172.21.15.117:0/2912165585 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:22:55.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.358+0000 7fd54e800700 1 --2- 172.21.15.117:0/2912165585 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd55010ec00 0x7fd55019f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:55.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.358+0000 7fd54de00700 1 -- 172.21.15.117:0/2912165585 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd55010ec00 msgr2=0x7fd55019f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:55.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.358+0000 7fd54de00700 1 --2- 172.21.15.117:0/2912165585 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd55010ec00 0x7fd55019f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:55.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.358+0000 7fd54de00700 1 -- 172.21.15.117:0/2912165585 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5500751e0 msgr2=0x7fd55019b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:55.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.358+0000 7fd54de00700 1 --2- 172.21.15.117:0/2912165585 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5500751e0 0x7fd55019b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:55.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.358+0000 7fd54de00700 1 -- 172.21.15.117:0/2912165585 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd55019fd60 con 0x7fd5500748c0 2024-06-03T03:22:55.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.358+0000 7fd54de00700 1 --2- 172.21.15.117:0/2912165585 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5500748c0 0x7fd55019ada0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd540000c00 tx=0x7fd54000bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:22:55.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.359+0000 7fd546a00700 1 -- 172.21.15.117:0/2912165585 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd540027070 con 0x7fd5500748c0 2024-06-03T03:22:55.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.359+0000 7fd54f200700 1 -- 172.21.15.117:0/2912165585 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd55019fff0 con 0x7fd5500748c0 2024-06-03T03:22:55.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.359+0000 7fd54f200700 1 -- 172.21.15.117:0/2912165585 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5501a78d0 con 0x7fd5500748c0 2024-06-03T03:22:55.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.359+0000 7fd546a00700 1 -- 172.21.15.117:0/2912165585 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd540003d70 con 0x7fd5500748c0 2024-06-03T03:22:55.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.359+0000 7fd546a00700 1 -- 172.21.15.117:0/2912165585 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5400043d0 con 0x7fd5500748c0 2024-06-03T03:22:55.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.360+0000 7fd54f200700 1 -- 172.21.15.117:0/2912165585 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd530005330 con 0x7fd5500748c0 2024-06-03T03:22:55.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.362+0000 7fd546a00700 1 -- 172.21.15.117:0/2912165585 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fd54002a070 con 0x7fd5500748c0 2024-06-03T03:22:55.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.363+0000 7fd546a00700 1 --2- 172.21.15.117:0/2912165585 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fd5340a2ed0 0x7fd5340a5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:55.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.363+0000 7fd546a00700 1 -- 172.21.15.117:0/2912165585 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7fd5400d0400 con 0x7fd5500748c0 2024-06-03T03:22:55.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.363+0000 7fd54d400700 1 --2- 172.21.15.117:0/2912165585 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fd5340a2ed0 0x7fd5340a5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:55.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.364+0000 7fd54d400700 1 --2- 172.21.15.117:0/2912165585 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fd5340a2ed0 0x7fd5340a5390 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd550101aa0 tx=0x7fd53c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:22:55.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.366+0000 7fd546a00700 1 -- 172.21.15.117:0/2912165585 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd540013070 con 0x7fd5500748c0 2024-06-03T03:22:55.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.608+0000 7fd54f200700 1 -- 172.21.15.117:0/2912165585 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fd5300058f0 con 0x7fd5500748c0 2024-06-03T03:22:55.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.609+0000 7fd546a00700 1 -- 172.21.15.117:0/2912165585 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fd54009cea0 con 0x7fd5500748c0 2024-06-03T03:22:55.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.612+0000 7fd54f200700 1 -- 172.21.15.117:0/2912165585 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fd5340a2ed0 msgr2=0x7fd5340a5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:55.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.612+0000 7fd54f200700 1 --2- 172.21.15.117:0/2912165585 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fd5340a2ed0 0x7fd5340a5390 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd550101aa0 tx=0x7fd53c009260 comp rx=0 tx=0).stop 2024-06-03T03:22:55.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.612+0000 7fd54f200700 1 -- 172.21.15.117:0/2912165585 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5500748c0 msgr2=0x7fd55019ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:55.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.612+0000 7fd54f200700 1 --2- 172.21.15.117:0/2912165585 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5500748c0 0x7fd55019ada0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd540000c00 tx=0x7fd54000bab0 comp rx=0 tx=0).stop 2024-06-03T03:22:55.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.612+0000 7fd54f200700 1 -- 172.21.15.117:0/2912165585 shutdown_connections 2024-06-03T03:22:55.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.612+0000 7fd54f200700 1 --2- 172.21.15.117:0/2912165585 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fd5340a2ed0 0x7fd5340a5390 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:55.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.612+0000 7fd54f200700 1 --2- 172.21.15.117:0/2912165585 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd55010ec00 0x7fd55019f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:55.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.612+0000 7fd54f200700 1 --2- 172.21.15.117:0/2912165585 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5500748c0 0x7fd55019ada0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:55.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.612+0000 7fd54f200700 1 --2- 172.21.15.117:0/2912165585 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5500751e0 0x7fd55019b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:55.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.612+0000 7fd54f200700 1 -- 172.21.15.117:0/2912165585 >> 172.21.15.117:0/2912165585 conn(0x7fd5500fdf10 msgr2=0x7fd5500febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:22:55.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.612+0000 7fd54f200700 1 -- 172.21.15.117:0/2912165585 shutdown_connections 2024-06-03T03:22:55.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.612+0000 7fd54f200700 1 -- 172.21.15.117:0/2912165585 wait complete. 2024-06-03T03:22:56.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:22:55 smithi183 ceph-mon[37326]: pgmap v434: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:22:56.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:22:55 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/2912165585' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-06-03T03:22:56.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:22:55 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/3392540119' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-06-03T03:22:56.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:22:55 smithi117 ceph-mon[31649]: pgmap v434: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:22:56.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:22:55 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/2912165585' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-06-03T03:22:56.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:22:55 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/3392540119' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-06-03T03:22:56.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:22:55 smithi151 ceph-mon[37043]: pgmap v434: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:22:56.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:22:55 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/2912165585' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-06-03T03:22:56.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:22:55 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/3392540119' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-06-03T03:22:56.729 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:22:57.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:22:57 smithi117 ceph-mon[31649]: mgrmap e18: x(active, since 11m), standbys: z, y 2024-06-03T03:22:57.294 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.105+0000 7fb1bb200700 1 -- 172.21.15.117:0/3441853185 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb1bc107700 msgr2=0x7fb1bc10f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:57.294 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.105+0000 7fb1bb200700 1 --2- 172.21.15.117:0/3441853185 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb1bc107700 0x7fb1bc10f2d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb1b4009fc0 tx=0x7fb1b400b330 comp rx=0 tx=0).stop 2024-06-03T03:22:57.294 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.105+0000 7fb1bb200700 1 -- 172.21.15.117:0/3441853185 shutdown_connections 2024-06-03T03:22:57.294 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.105+0000 7fb1bb200700 1 --2- 172.21.15.117:0/3441853185 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb1bc107700 0x7fb1bc10f2d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:57.294 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.105+0000 7fb1bb200700 1 --2- 172.21.15.117:0/3441853185 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb1bc102620 0x7fb1bc102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:57.295 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.105+0000 7fb1bb200700 1 --2- 172.21.15.117:0/3441853185 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1bc102fd0 0x7fb1bc107020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:57.295 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.105+0000 7fb1bb200700 1 -- 172.21.15.117:0/3441853185 >> 172.21.15.117:0/3441853185 conn(0x7fb1bc0fdf10 msgr2=0x7fb1bc100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:22:57.295 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.106+0000 7fb1bb200700 1 -- 172.21.15.117:0/3441853185 shutdown_connections 2024-06-03T03:22:57.295 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.106+0000 7fb1bb200700 1 -- 172.21.15.117:0/3441853185 wait complete. 2024-06-03T03:22:57.295 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.107+0000 7fb1bb200700 1 Processor -- start 2024-06-03T03:22:57.295 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.107+0000 7fb1bb200700 1 -- start start 2024-06-03T03:22:57.295 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.108+0000 7fb1bb200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1bc102620 0x7fb1bc074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:57.295 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.108+0000 7fb1bb200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb1bc102fd0 0x7fb1bc074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:57.295 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.108+0000 7fb1bb200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb1bc107700 0x7fb1bc075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:57.295 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.108+0000 7fb1bb200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb1bc112740 con 0x7fb1bc102620 2024-06-03T03:22:57.295 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.108+0000 7fb1bb200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb1bc1125c0 con 0x7fb1bc107700 2024-06-03T03:22:57.295 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.108+0000 7fb1bb200700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb1bc1128c0 con 0x7fb1bc102fd0 2024-06-03T03:22:57.296 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.108+0000 7fb1b9e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1bc102620 0x7fb1bc074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:57.296 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.108+0000 7fb1b9400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb1bc102fd0 0x7fb1bc074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:57.296 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.108+0000 7fb1b9e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1bc102620 0x7fb1bc074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:36666/0 (socket says 172.21.15.117:36666) 2024-06-03T03:22:57.296 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.108+0000 7fb1b9e00700 1 -- 172.21.15.117:0/3204012249 learned_addr learned my addr 172.21.15.117:0/3204012249 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:22:57.296 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.109+0000 7fb1ba800700 1 --2- 172.21.15.117:0/3204012249 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb1bc107700 0x7fb1bc075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:57.296 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.109+0000 7fb1b9400700 1 -- 172.21.15.117:0/3204012249 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb1bc107700 msgr2=0x7fb1bc075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:57.296 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.109+0000 7fb1b9400700 1 --2- 172.21.15.117:0/3204012249 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb1bc107700 0x7fb1bc075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:57.296 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.109+0000 7fb1b9400700 1 -- 172.21.15.117:0/3204012249 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1bc102620 msgr2=0x7fb1bc074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:57.296 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.109+0000 7fb1b9400700 1 --2- 172.21.15.117:0/3204012249 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1bc102620 0x7fb1bc074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:57.296 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.109+0000 7fb1b9400700 1 -- 172.21.15.117:0/3204012249 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1bc1a7060 con 0x7fb1bc102fd0 2024-06-03T03:22:57.296 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.109+0000 7fb1ba800700 1 --2- 172.21.15.117:0/3204012249 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb1bc107700 0x7fb1bc075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T03:22:57.296 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.109+0000 7fb1b9400700 1 --2- 172.21.15.117:0/3204012249 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb1bc102fd0 0x7fb1bc074cd0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb1a8005fd0 tx=0x7fb1a800ee70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:22:57.297 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.109+0000 7fb1b2a00700 1 -- 172.21.15.117:0/3204012249 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a800cc90 con 0x7fb1bc102fd0 2024-06-03T03:22:57.297 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.110+0000 7fb1bb200700 1 -- 172.21.15.117:0/3204012249 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1bc1a7350 con 0x7fb1bc102fd0 2024-06-03T03:22:57.297 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.110+0000 7fb1b2a00700 1 -- 172.21.15.117:0/3204012249 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1a800ce30 con 0x7fb1bc102fd0 2024-06-03T03:22:57.297 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.110+0000 7fb1b2a00700 1 -- 172.21.15.117:0/3204012249 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a8010690 con 0x7fb1bc102fd0 2024-06-03T03:22:57.297 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.110+0000 7fb1bb200700 1 -- 172.21.15.117:0/3204012249 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1bc1a78e0 con 0x7fb1bc102fd0 2024-06-03T03:22:57.297 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.111+0000 7fb1bb200700 1 -- 172.21.15.117:0/3204012249 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1bc069180 con 0x7fb1bc102fd0 2024-06-03T03:22:57.297 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.114+0000 7fb1b2a00700 1 -- 172.21.15.117:0/3204012249 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fb1a8043ae0 con 0x7fb1bc102fd0 2024-06-03T03:22:57.297 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.115+0000 7fb1b2a00700 1 --2- 172.21.15.117:0/3204012249 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fb1a00a2ca0 0x7fb1a00a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:57.297 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.115+0000 7fb1b2a00700 1 -- 172.21.15.117:0/3204012249 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7fb1a80cf680 con 0x7fb1bc102fd0 2024-06-03T03:22:57.297 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.115+0000 7fb1b9e00700 1 --2- 172.21.15.117:0/3204012249 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fb1a00a2ca0 0x7fb1a00a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:57.297 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.116+0000 7fb1b9e00700 1 --2- 172.21.15.117:0/3204012249 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fb1a00a2ca0 0x7fb1a00a5160 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac009680 tx=0x7fb1ac009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:22:57.298 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.117+0000 7fb1b2a00700 1 -- 172.21.15.117:0/3204012249 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb1a809c1f0 con 0x7fb1bc102fd0 2024-06-03T03:22:57.357 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.356+0000 7fb1bb200700 1 -- 172.21.15.117:0/3204012249 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fb1bc075c20 con 0x7fb1bc102fd0 2024-06-03T03:22:57.358 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.357+0000 7fb1b2a00700 1 -- 172.21.15.117:0/3204012249 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb1a809baf0 con 0x7fb1bc102fd0 2024-06-03T03:22:57.358 INFO:teuthology.orchestra.run.smithi117.stdout:197568495701 2024-06-03T03:22:57.361 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.360+0000 7fb1bb200700 1 -- 172.21.15.117:0/3204012249 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fb1a00a2ca0 msgr2=0x7fb1a00a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:57.361 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.360+0000 7fb1bb200700 1 --2- 172.21.15.117:0/3204012249 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fb1a00a2ca0 0x7fb1a00a5160 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac009680 tx=0x7fb1ac009260 comp rx=0 tx=0).stop 2024-06-03T03:22:57.361 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.360+0000 7fb1bb200700 1 -- 172.21.15.117:0/3204012249 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb1bc102fd0 msgr2=0x7fb1bc074cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:57.361 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.360+0000 7fb1bb200700 1 --2- 172.21.15.117:0/3204012249 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb1bc102fd0 0x7fb1bc074cd0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb1a8005fd0 tx=0x7fb1a800ee70 comp rx=0 tx=0).stop 2024-06-03T03:22:57.361 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.360+0000 7fb1bb200700 1 -- 172.21.15.117:0/3204012249 shutdown_connections 2024-06-03T03:22:57.362 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.360+0000 7fb1bb200700 1 --2- 172.21.15.117:0/3204012249 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fb1a00a2ca0 0x7fb1a00a5160 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:57.362 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.360+0000 7fb1bb200700 1 --2- 172.21.15.117:0/3204012249 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb1bc107700 0x7fb1bc075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:57.362 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.360+0000 7fb1bb200700 1 --2- 172.21.15.117:0/3204012249 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1bc102620 0x7fb1bc074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:57.362 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.360+0000 7fb1bb200700 1 --2- 172.21.15.117:0/3204012249 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb1bc102fd0 0x7fb1bc074cd0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:57.362 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.361+0000 7fb1bb200700 1 -- 172.21.15.117:0/3204012249 >> 172.21.15.117:0/3204012249 conn(0x7fb1bc0fdf10 msgr2=0x7fb1bc0fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:22:57.362 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.361+0000 7fb1bb200700 1 -- 172.21.15.117:0/3204012249 shutdown_connections 2024-06-03T03:22:57.363 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.361+0000 7fb1bb200700 1 -- 172.21.15.117:0/3204012249 wait complete. 2024-06-03T03:22:57.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:22:57 smithi151 ceph-mon[37043]: mgrmap e18: x(active, since 11m), standbys: z, y 2024-06-03T03:22:57.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:22:57 smithi183 ceph-mon[37326]: mgrmap e18: x(active, since 11m), standbys: z, y 2024-06-03T03:22:57.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.614+0000 7fc6e1a00700 1 -- 172.21.15.117:0/2392180908 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6dc108500 msgr2=0x7fc6dc1088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:57.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.614+0000 7fc6e1a00700 1 --2- 172.21.15.117:0/2392180908 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6dc108500 0x7fc6dc1088e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc009990 tx=0x7fc6cc009ca0 comp rx=0 tx=0).stop 2024-06-03T03:22:57.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.614+0000 7fc6e1a00700 1 -- 172.21.15.117:0/2392180908 shutdown_connections 2024-06-03T03:22:57.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.614+0000 7fc6e1a00700 1 --2- 172.21.15.117:0/2392180908 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc6dc06ab30 0x7fc6dc10f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:57.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.614+0000 7fc6e1a00700 1 --2- 172.21.15.117:0/2392180908 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6dc108500 0x7fc6dc1088e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:57.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.614+0000 7fc6e1a00700 1 --2- 172.21.15.117:0/2392180908 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc6dc06a170 0x7fc6dc06a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:57.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.614+0000 7fc6e1a00700 1 -- 172.21.15.117:0/2392180908 >> 172.21.15.117:0/2392180908 conn(0x7fc6dc078340 msgr2=0x7fc6dc078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:22:57.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.614+0000 7fc6e1a00700 1 -- 172.21.15.117:0/2392180908 shutdown_connections 2024-06-03T03:22:57.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.614+0000 7fc6e1a00700 1 -- 172.21.15.117:0/2392180908 wait complete. 2024-06-03T03:22:57.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.616+0000 7fc6e1a00700 1 Processor -- start 2024-06-03T03:22:57.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.616+0000 7fc6e1a00700 1 -- start start 2024-06-03T03:22:57.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.616+0000 7fc6e1a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc6dc06a170 0x7fc6dc19e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:57.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.616+0000 7fc6e1a00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc6dc06ab30 0x7fc6dc19ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:57.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.616+0000 7fc6e1a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6dc108500 0x7fc6dc198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:57.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.616+0000 7fc6e1a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc6dc112760 con 0x7fc6dc108500 2024-06-03T03:22:57.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.616+0000 7fc6e1a00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc6dc1125e0 con 0x7fc6dc06a170 2024-06-03T03:22:57.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.616+0000 7fc6e1a00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc6dc1128e0 con 0x7fc6dc06ab30 2024-06-03T03:22:57.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.617+0000 7fc6db400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6dc108500 0x7fc6dc198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:57.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.617+0000 7fc6daa00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc6dc06a170 0x7fc6dc19e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:57.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.617+0000 7fc6db400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6dc108500 0x7fc6dc198aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:36688/0 (socket says 172.21.15.117:36688) 2024-06-03T03:22:57.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.617+0000 7fc6daa00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc6dc06a170 0x7fc6dc19e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:57714/0 (socket says 172.21.15.117:57714) 2024-06-03T03:22:57.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.617+0000 7fc6db400700 1 -- 172.21.15.117:0/915877296 learned_addr learned my addr 172.21.15.117:0/915877296 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:22:57.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.617+0000 7fc6da000700 1 --2- 172.21.15.117:0/915877296 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc6dc06ab30 0x7fc6dc19ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:57.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.617+0000 7fc6db400700 1 -- 172.21.15.117:0/915877296 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc6dc06ab30 msgr2=0x7fc6dc19ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:57.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.617+0000 7fc6db400700 1 --2- 172.21.15.117:0/915877296 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc6dc06ab30 0x7fc6dc19ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:57.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.617+0000 7fc6db400700 1 -- 172.21.15.117:0/915877296 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc6dc06a170 msgr2=0x7fc6dc19e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:57.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.617+0000 7fc6db400700 1 --2- 172.21.15.117:0/915877296 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc6dc06a170 0x7fc6dc19e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:57.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.618+0000 7fc6db400700 1 -- 172.21.15.117:0/915877296 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6dc199200 con 0x7fc6dc108500 2024-06-03T03:22:57.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.618+0000 7fc6daa00700 1 --2- 172.21.15.117:0/915877296 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc6dc06a170 0x7fc6dc19e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:22:57.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.618+0000 7fc6db400700 1 --2- 172.21.15.117:0/915877296 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6dc108500 0x7fc6dc198aa0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc6d400ef30 tx=0x7fc6d400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:22:57.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.618+0000 7fc6d3400700 1 -- 172.21.15.117:0/915877296 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6d40092e0 con 0x7fc6dc108500 2024-06-03T03:22:57.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.618+0000 7fc6d3400700 1 -- 172.21.15.117:0/915877296 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6d4004bb0 con 0x7fc6dc108500 2024-06-03T03:22:57.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.618+0000 7fc6e1a00700 1 -- 172.21.15.117:0/915877296 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6dc199490 con 0x7fc6dc108500 2024-06-03T03:22:57.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.618+0000 7fc6d3400700 1 -- 172.21.15.117:0/915877296 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6d4004280 con 0x7fc6dc108500 2024-06-03T03:22:57.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.618+0000 7fc6e1a00700 1 -- 172.21.15.117:0/915877296 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6dc1a57b0 con 0x7fc6dc108500 2024-06-03T03:22:57.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.620+0000 7fc6e1a00700 1 -- 172.21.15.117:0/915877296 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6dc10a9e0 con 0x7fc6dc108500 2024-06-03T03:22:57.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.622+0000 7fc6d3400700 1 -- 172.21.15.117:0/915877296 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fc6d401b030 con 0x7fc6dc108500 2024-06-03T03:22:57.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.623+0000 7fc6d3400700 1 --2- 172.21.15.117:0/915877296 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fc6c00a2ca0 0x7fc6c00a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:57.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.624+0000 7fc6d3400700 1 -- 172.21.15.117:0/915877296 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7fc6d40cedf0 con 0x7fc6dc108500 2024-06-03T03:22:57.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.624+0000 7fc6daa00700 1 --2- 172.21.15.117:0/915877296 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fc6c00a2ca0 0x7fc6c00a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:57.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.624+0000 7fc6daa00700 1 --2- 172.21.15.117:0/915877296 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fc6c00a2ca0 0x7fc6c00a5160 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc199be0 tx=0x7fc6cc005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:22:57.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.628+0000 7fc6d3400700 1 -- 172.21.15.117:0/915877296 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc6d4022020 con 0x7fc6dc108500 2024-06-03T03:22:57.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.864+0000 7fc6e1a00700 1 -- 172.21.15.117:0/915877296 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fc6dc199fd0 con 0x7fc6dc108500 2024-06-03T03:22:57.866 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.865+0000 7fc6d3400700 1 -- 172.21.15.117:0/915877296 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fc6d4010040 con 0x7fc6dc108500 2024-06-03T03:22:57.866 INFO:teuthology.orchestra.run.smithi117.stdout:141733920864 2024-06-03T03:22:57.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.868+0000 7fc6e1a00700 1 -- 172.21.15.117:0/915877296 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fc6c00a2ca0 msgr2=0x7fc6c00a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:57.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.868+0000 7fc6e1a00700 1 --2- 172.21.15.117:0/915877296 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fc6c00a2ca0 0x7fc6c00a5160 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc199be0 tx=0x7fc6cc005960 comp rx=0 tx=0).stop 2024-06-03T03:22:57.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.868+0000 7fc6e1a00700 1 -- 172.21.15.117:0/915877296 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6dc108500 msgr2=0x7fc6dc198aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:57.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.868+0000 7fc6e1a00700 1 --2- 172.21.15.117:0/915877296 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6dc108500 0x7fc6dc198aa0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc6d400ef30 tx=0x7fc6d400c3a0 comp rx=0 tx=0).stop 2024-06-03T03:22:57.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.869+0000 7fc6e1a00700 1 -- 172.21.15.117:0/915877296 shutdown_connections 2024-06-03T03:22:57.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.869+0000 7fc6e1a00700 1 --2- 172.21.15.117:0/915877296 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fc6c00a2ca0 0x7fc6c00a5160 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:57.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.869+0000 7fc6e1a00700 1 --2- 172.21.15.117:0/915877296 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6dc108500 0x7fc6dc198aa0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:57.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.869+0000 7fc6e1a00700 1 --2- 172.21.15.117:0/915877296 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc6dc06a170 0x7fc6dc19e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:57.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.869+0000 7fc6e1a00700 1 --2- 172.21.15.117:0/915877296 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc6dc06ab30 0x7fc6dc19ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:57.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.869+0000 7fc6e1a00700 1 -- 172.21.15.117:0/915877296 >> 172.21.15.117:0/915877296 conn(0x7fc6dc078340 msgr2=0x7fc6dc0fff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:22:57.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.869+0000 7fc6e1a00700 1 -- 172.21.15.117:0/915877296 shutdown_connections 2024-06-03T03:22:57.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.869+0000 7fc6e1a00700 1 -- 172.21.15.117:0/915877296 wait complete. 2024-06-03T03:22:58.177 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.238+0000 7f921de00700 1 -- 172.21.15.117:0/3152836430 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9218074830 msgr2=0x7f9218074c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:58.177 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.238+0000 7f921de00700 1 --2- 172.21.15.117:0/3152836430 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9218074830 0x7f9218074c10 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f920c009990 tx=0x7f920c009ca0 comp rx=0 tx=0).stop 2024-06-03T03:22:58.178 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.239+0000 7f921de00700 1 -- 172.21.15.117:0/3152836430 shutdown_connections 2024-06-03T03:22:58.178 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.239+0000 7f921de00700 1 --2- 172.21.15.117:0/3152836430 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f92181133d0 0x7f92181157c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:58.178 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.239+0000 7f921de00700 1 --2- 172.21.15.117:0/3152836430 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9218074830 0x7f9218074c10 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:58.178 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.239+0000 7f921de00700 1 --2- 172.21.15.117:0/3152836430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9218075150 0x7f9218112e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:58.178 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.239+0000 7f921de00700 1 -- 172.21.15.117:0/3152836430 >> 172.21.15.117:0/3152836430 conn(0x7f92180fdf10 msgr2=0x7f9218100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:22:58.178 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.239+0000 7f921de00700 1 -- 172.21.15.117:0/3152836430 shutdown_connections 2024-06-03T03:22:58.178 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.239+0000 7f921de00700 1 -- 172.21.15.117:0/3152836430 wait complete. 2024-06-03T03:22:58.178 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.240+0000 7f921de00700 1 Processor -- start 2024-06-03T03:22:58.178 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.240+0000 7f921de00700 1 -- start start 2024-06-03T03:22:58.178 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.241+0000 7f921de00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9218074830 0x7f921819f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:58.178 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.241+0000 7f921de00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9218075150 0x7f921819f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:58.178 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.241+0000 7f921de00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92181133d0 0x7f92181a3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:58.179 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.241+0000 7f921de00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9218118d40 con 0x7f92181133d0 2024-06-03T03:22:58.179 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.241+0000 7f921de00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9218118bc0 con 0x7f9218074830 2024-06-03T03:22:58.179 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.241+0000 7f921de00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9218118ec0 con 0x7f9218075150 2024-06-03T03:22:58.179 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.241+0000 7f921d400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92181133d0 0x7f92181a3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:58.179 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.241+0000 7f921ca00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9218074830 0x7f921819f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:58.179 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.241+0000 7f921ca00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9218074830 0x7f921819f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:57692/0 (socket says 172.21.15.117:57692) 2024-06-03T03:22:58.179 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.241+0000 7f921ca00700 1 -- 172.21.15.117:0/2114748252 learned_addr learned my addr 172.21.15.117:0/2114748252 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:22:58.179 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.241+0000 7f9217e00700 1 --2- 172.21.15.117:0/2114748252 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9218075150 0x7f921819f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:58.179 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.242+0000 7f921ca00700 1 -- 172.21.15.117:0/2114748252 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9218075150 msgr2=0x7f921819f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:58.179 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.242+0000 7f921ca00700 1 --2- 172.21.15.117:0/2114748252 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9218075150 0x7f921819f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:58.179 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.242+0000 7f921ca00700 1 -- 172.21.15.117:0/2114748252 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92181133d0 msgr2=0x7f92181a3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:58.179 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.242+0000 7f921ca00700 1 --2- 172.21.15.117:0/2114748252 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92181133d0 0x7f92181a3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:58.179 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.242+0000 7f921ca00700 1 -- 172.21.15.117:0/2114748252 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92181a41c0 con 0x7f9218074830 2024-06-03T03:22:58.179 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.242+0000 7f921ca00700 1 --2- 172.21.15.117:0/2114748252 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9218074830 0x7f921819f1b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f920c005910 tx=0x7f920c00bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:22:58.180 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.242+0000 7f9215600700 1 -- 172.21.15.117:0/2114748252 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f920c027070 con 0x7f9218074830 2024-06-03T03:22:58.180 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.242+0000 7f9215600700 1 -- 172.21.15.117:0/2114748252 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f920c003d70 con 0x7f9218074830 2024-06-03T03:22:58.180 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.243+0000 7f9215600700 1 -- 172.21.15.117:0/2114748252 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f920c00f460 con 0x7f9218074830 2024-06-03T03:22:58.180 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.243+0000 7f921de00700 1 -- 172.21.15.117:0/2114748252 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92181a4450 con 0x7f9218074830 2024-06-03T03:22:58.180 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.243+0000 7f921de00700 1 -- 172.21.15.117:0/2114748252 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92181abd30 con 0x7f9218074830 2024-06-03T03:22:58.180 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.244+0000 7f921de00700 1 -- 172.21.15.117:0/2114748252 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9218075f70 con 0x7f9218074830 2024-06-03T03:22:58.180 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.246+0000 7f9215600700 1 -- 172.21.15.117:0/2114748252 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f920c02a070 con 0x7f9218074830 2024-06-03T03:22:58.180 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.247+0000 7f9215600700 1 --2- 172.21.15.117:0/2114748252 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f92000a2ca0 0x7f92000a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:58.180 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.248+0000 7f9215600700 1 -- 172.21.15.117:0/2114748252 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f920c0d08f0 con 0x7f9218074830 2024-06-03T03:22:58.180 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.248+0000 7f9217e00700 1 --2- 172.21.15.117:0/2114748252 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f92000a2ca0 0x7f92000a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:58.180 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.249+0000 7f9217e00700 1 --2- 172.21.15.117:0/2114748252 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f92000a2ca0 0x7f92000a5160 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f92181061c0 tx=0x7f920400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:22:58.180 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.249+0000 7f9215600700 1 -- 172.21.15.117:0/2114748252 <== mon.2 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f920c031070 con 0x7f9218074830 2024-06-03T03:22:58.180 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.487+0000 7f921de00700 1 -- 172.21.15.117:0/2114748252 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f921802dca0 con 0x7f9218074830 2024-06-03T03:22:58.180 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.488+0000 7f9215600700 1 -- 172.21.15.117:0/2114748252 <== mon.2 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f920c013070 con 0x7f9218074830 2024-06-03T03:22:58.181 INFO:teuthology.orchestra.run.smithi117.stdout:253403070538 2024-06-03T03:22:58.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.492+0000 7f921de00700 1 -- 172.21.15.117:0/2114748252 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f92000a2ca0 msgr2=0x7f92000a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:58.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.492+0000 7f921de00700 1 --2- 172.21.15.117:0/2114748252 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f92000a2ca0 0x7f92000a5160 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f92181061c0 tx=0x7f920400b410 comp rx=0 tx=0).stop 2024-06-03T03:22:58.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.492+0000 7f921de00700 1 -- 172.21.15.117:0/2114748252 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9218074830 msgr2=0x7f921819f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:58.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.492+0000 7f921de00700 1 --2- 172.21.15.117:0/2114748252 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9218074830 0x7f921819f1b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f920c005910 tx=0x7f920c00bab0 comp rx=0 tx=0).stop 2024-06-03T03:22:58.182 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.492+0000 7f921de00700 1 -- 172.21.15.117:0/2114748252 shutdown_connections 2024-06-03T03:22:58.182 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.492+0000 7f921de00700 1 --2- 172.21.15.117:0/2114748252 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f92000a2ca0 0x7f92000a5160 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:58.182 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.492+0000 7f921de00700 1 --2- 172.21.15.117:0/2114748252 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92181133d0 0x7f92181a3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:58.182 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.492+0000 7f921de00700 1 --2- 172.21.15.117:0/2114748252 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9218074830 0x7f921819f1b0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:58.182 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.492+0000 7f921de00700 1 --2- 172.21.15.117:0/2114748252 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9218075150 0x7f921819f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:58.182 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.492+0000 7f921de00700 1 -- 172.21.15.117:0/2114748252 >> 172.21.15.117:0/2114748252 conn(0x7f92180fdf10 msgr2=0x7f92180febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:22:58.182 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.492+0000 7f921de00700 1 -- 172.21.15.117:0/2114748252 shutdown_connections 2024-06-03T03:22:58.182 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:57.492+0000 7f921de00700 1 -- 172.21.15.117:0/2114748252 wait complete. 2024-06-03T03:22:58.256 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:22:58 smithi117 ceph-mon[31649]: pgmap v435: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 8 op/s 2024-06-03T03:22:58.256 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:22:58 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/3204012249' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-06-03T03:22:58.256 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:22:58 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/2114748252' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-06-03T03:22:58.256 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:22:58 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/915877296' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-06-03T03:22:58.308 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:22:58 smithi151 ceph-mon[37043]: pgmap v435: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 8 op/s 2024-06-03T03:22:58.308 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:22:58 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/3204012249' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-06-03T03:22:58.308 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:22:58 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/2114748252' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-06-03T03:22:58.308 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:22:58 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/915877296' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-06-03T03:22:58.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:22:58 smithi183 ceph-mon[37326]: pgmap v435: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 8 op/s 2024-06-03T03:22:58.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:22:58 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/3204012249' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-06-03T03:22:58.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:22:58 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/2114748252' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-06-03T03:22:58.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:22:58 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/915877296' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-06-03T03:22:58.529 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.449+0000 7fb4f7e00700 1 -- 172.21.15.117:0/1435560296 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4f01030f0 msgr2=0x7fb4f0107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:58.529 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.449+0000 7fb4f7e00700 1 --2- 172.21.15.117:0/1435560296 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4f01030f0 0x7fb4f0107110 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc009920 tx=0x7fb4dc009c30 comp rx=0 tx=0).stop 2024-06-03T03:22:58.529 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.450+0000 7fb4f7e00700 1 -- 172.21.15.117:0/1435560296 shutdown_connections 2024-06-03T03:22:58.529 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.450+0000 7fb4f7e00700 1 --2- 172.21.15.117:0/1435560296 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb4f0107840 0x7fb4f010f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:58.529 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.450+0000 7fb4f7e00700 1 --2- 172.21.15.117:0/1435560296 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4f0102740 0x7fb4f0102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:58.529 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.450+0000 7fb4f7e00700 1 --2- 172.21.15.117:0/1435560296 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4f01030f0 0x7fb4f0107110 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:58.529 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.450+0000 7fb4f7e00700 1 -- 172.21.15.117:0/1435560296 >> 172.21.15.117:0/1435560296 conn(0x7fb4f00795f0 msgr2=0x7fb4f0079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:22:58.530 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.450+0000 7fb4f7e00700 1 -- 172.21.15.117:0/1435560296 shutdown_connections 2024-06-03T03:22:58.530 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.450+0000 7fb4f7e00700 1 -- 172.21.15.117:0/1435560296 wait complete. 2024-06-03T03:22:58.530 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.451+0000 7fb4f7e00700 1 Processor -- start 2024-06-03T03:22:58.530 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.451+0000 7fb4f7e00700 1 -- start start 2024-06-03T03:22:58.530 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.452+0000 7fb4f7e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4f0102740 0x7fb4f019e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:58.530 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.452+0000 7fb4f7e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb4f01030f0 0x7fb4f019eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:58.530 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.452+0000 7fb4f7e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4f0107840 0x7fb4f0198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:58.530 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.452+0000 7fb4f7e00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f0112750 con 0x7fb4f0107840 2024-06-03T03:22:58.530 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.452+0000 7fb4f7e00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f01125d0 con 0x7fb4f0102740 2024-06-03T03:22:58.530 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.452+0000 7fb4f7e00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f01128d0 con 0x7fb4f01030f0 2024-06-03T03:22:58.530 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.452+0000 7fb4f5400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4f0102740 0x7fb4f019e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:58.530 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.452+0000 7fb4f5e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4f0107840 0x7fb4f0198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:58.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.452+0000 7fb4f5400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4f0102740 0x7fb4f019e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:54820/0 (socket says 172.21.15.117:54820) 2024-06-03T03:22:58.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.452+0000 7fb4f5400700 1 -- 172.21.15.117:0/3392540119 learned_addr learned my addr 172.21.15.117:0/3392540119 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:22:58.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.452+0000 7fb4f4a00700 1 --2- 172.21.15.117:0/3392540119 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb4f01030f0 0x7fb4f019eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:58.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.452+0000 7fb4f5400700 1 -- 172.21.15.117:0/3392540119 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb4f01030f0 msgr2=0x7fb4f019eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:58.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.452+0000 7fb4f5400700 1 --2- 172.21.15.117:0/3392540119 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb4f01030f0 0x7fb4f019eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:58.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.452+0000 7fb4f5400700 1 -- 172.21.15.117:0/3392540119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4f0107840 msgr2=0x7fb4f0198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:58.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.452+0000 7fb4f5400700 1 --2- 172.21.15.117:0/3392540119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4f0107840 0x7fb4f0198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:58.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.453+0000 7fb4f5400700 1 -- 172.21.15.117:0/3392540119 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4f01991d0 con 0x7fb4f0102740 2024-06-03T03:22:58.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.453+0000 7fb4f5400700 1 --2- 172.21.15.117:0/3392540119 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4f0102740 0x7fb4f019e860 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb4e400e9c0 tx=0x7fb4e400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:22:58.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.453+0000 7fb4ee000700 1 -- 172.21.15.117:0/3392540119 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e4004a30 con 0x7fb4f0102740 2024-06-03T03:22:58.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.453+0000 7fb4ee000700 1 -- 172.21.15.117:0/3392540119 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4e400cca0 con 0x7fb4f0102740 2024-06-03T03:22:58.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.453+0000 7fb4f7e00700 1 -- 172.21.15.117:0/3392540119 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4f01994c0 con 0x7fb4f0102740 2024-06-03T03:22:58.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.453+0000 7fb4ee000700 1 -- 172.21.15.117:0/3392540119 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e4005680 con 0x7fb4f0102740 2024-06-03T03:22:58.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.453+0000 7fb4f7e00700 1 -- 172.21.15.117:0/3392540119 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4f01a5730 con 0x7fb4f0102740 2024-06-03T03:22:58.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.454+0000 7fb4f7e00700 1 -- 172.21.15.117:0/3392540119 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4f0069180 con 0x7fb4f0102740 2024-06-03T03:22:58.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.457+0000 7fb4ee000700 1 -- 172.21.15.117:0/3392540119 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fb4e400c7c0 con 0x7fb4f0102740 2024-06-03T03:22:58.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.458+0000 7fb4ee000700 1 --2- 172.21.15.117:0/3392540119 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fb4d80a2cb0 0x7fb4d80a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:58.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.458+0000 7fb4ee000700 1 -- 172.21.15.117:0/3392540119 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7fb4e40cf8b0 con 0x7fb4f0102740 2024-06-03T03:22:58.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.458+0000 7fb4f4a00700 1 --2- 172.21.15.117:0/3392540119 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fb4d80a2cb0 0x7fb4d80a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:58.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.459+0000 7fb4f4a00700 1 --2- 172.21.15.117:0/3392540119 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fb4d80a2cb0 0x7fb4d80a5170 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc00c010 tx=0x7fb4dc019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:22:58.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.461+0000 7fb4ee000700 1 -- 172.21.15.117:0/3392540119 <== mon.2 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb4e4053850 con 0x7fb4f0102740 2024-06-03T03:22:58.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.707+0000 7fb4f7e00700 1 -- 172.21.15.117:0/3392540119 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fb4f0199f80 con 0x7fb4f0102740 2024-06-03T03:22:58.533 INFO:teuthology.orchestra.run.smithi117.stdout:219043332175 2024-06-03T03:22:58.533 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.708+0000 7fb4ee000700 1 -- 172.21.15.117:0/3392540119 <== mon.2 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb4f0199f80 con 0x7fb4f0102740 2024-06-03T03:22:58.533 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.711+0000 7fb4f7e00700 1 -- 172.21.15.117:0/3392540119 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fb4d80a2cb0 msgr2=0x7fb4d80a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:58.533 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.711+0000 7fb4f7e00700 1 --2- 172.21.15.117:0/3392540119 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fb4d80a2cb0 0x7fb4d80a5170 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc00c010 tx=0x7fb4dc019040 comp rx=0 tx=0).stop 2024-06-03T03:22:58.533 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.711+0000 7fb4f7e00700 1 -- 172.21.15.117:0/3392540119 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4f0102740 msgr2=0x7fb4f019e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:58.533 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.711+0000 7fb4f7e00700 1 --2- 172.21.15.117:0/3392540119 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4f0102740 0x7fb4f019e860 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb4e400e9c0 tx=0x7fb4e400ecd0 comp rx=0 tx=0).stop 2024-06-03T03:22:58.533 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.711+0000 7fb4f7e00700 1 -- 172.21.15.117:0/3392540119 shutdown_connections 2024-06-03T03:22:58.533 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.711+0000 7fb4f7e00700 1 --2- 172.21.15.117:0/3392540119 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fb4d80a2cb0 0x7fb4d80a5170 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:58.534 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.711+0000 7fb4f7e00700 1 --2- 172.21.15.117:0/3392540119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4f0107840 0x7fb4f0198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:58.534 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.711+0000 7fb4f7e00700 1 --2- 172.21.15.117:0/3392540119 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4f0102740 0x7fb4f019e860 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:58.534 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.711+0000 7fb4f7e00700 1 --2- 172.21.15.117:0/3392540119 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb4f01030f0 0x7fb4f019eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:58.534 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.711+0000 7fb4f7e00700 1 -- 172.21.15.117:0/3392540119 >> 172.21.15.117:0/3392540119 conn(0x7fb4f00795f0 msgr2=0x7fb4f0100860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:22:58.534 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.711+0000 7fb4f7e00700 1 -- 172.21.15.117:0/3392540119 shutdown_connections 2024-06-03T03:22:58.534 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:55.711+0000 7fb4f7e00700 1 -- 172.21.15.117:0/3392540119 wait complete. 2024-06-03T03:22:59.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:22:59 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/34847527' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-06-03T03:22:59.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:22:59 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/1074697990' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-06-03T03:22:59.326 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.014+0000 7fc8a9a00700 1 -- 172.21.15.117:0/494800780 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8a40751e0 msgr2=0x7fc8a410e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:59.326 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.014+0000 7fc8a9a00700 1 --2- 172.21.15.117:0/494800780 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8a40751e0 0x7fc8a410e6c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc890009990 tx=0x7fc890009ca0 comp rx=0 tx=0).stop 2024-06-03T03:22:59.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.014+0000 7fc8a9a00700 1 -- 172.21.15.117:0/494800780 shutdown_connections 2024-06-03T03:22:59.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.014+0000 7fc8a9a00700 1 --2- 172.21.15.117:0/494800780 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8a410ec00 0x7fc8a4111070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:59.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.014+0000 7fc8a9a00700 1 --2- 172.21.15.117:0/494800780 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc8a40748c0 0x7fc8a4074ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:59.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.014+0000 7fc8a9a00700 1 --2- 172.21.15.117:0/494800780 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8a40751e0 0x7fc8a410e6c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:59.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.014+0000 7fc8a9a00700 1 -- 172.21.15.117:0/494800780 >> 172.21.15.117:0/494800780 conn(0x7fc8a40fdf10 msgr2=0x7fc8a4100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:22:59.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.014+0000 7fc8a9a00700 1 -- 172.21.15.117:0/494800780 shutdown_connections 2024-06-03T03:22:59.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.014+0000 7fc8a9a00700 1 -- 172.21.15.117:0/494800780 wait complete. 2024-06-03T03:22:59.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.015+0000 7fc8a9a00700 1 Processor -- start 2024-06-03T03:22:59.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.015+0000 7fc8a9a00700 1 -- start start 2024-06-03T03:22:59.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.015+0000 7fc8a9a00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc8a40748c0 0x7fc8a419ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:59.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.015+0000 7fc8a9a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8a40751e0 0x7fc8a419b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:59.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.015+0000 7fc8a9a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8a410ec00 0x7fc8a419f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:59.328 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.015+0000 7fc8a9a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a4114920 con 0x7fc8a40751e0 2024-06-03T03:22:59.328 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.015+0000 7fc8a9a00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a41147a0 con 0x7fc8a410ec00 2024-06-03T03:22:59.328 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.015+0000 7fc8a9a00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a4114aa0 con 0x7fc8a40748c0 2024-06-03T03:22:59.328 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.016+0000 7fc8a3e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc8a40748c0 0x7fc8a419ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:59.328 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.016+0000 7fc8a3400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8a40751e0 0x7fc8a419b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:59.328 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.016+0000 7fc8a3e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc8a40748c0 0x7fc8a419ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:51174/0 (socket says 172.21.15.117:51174) 2024-06-03T03:22:59.328 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.016+0000 7fc8a3e00700 1 -- 172.21.15.117:0/34847527 learned_addr learned my addr 172.21.15.117:0/34847527 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:22:59.328 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.016+0000 7fc8a9000700 1 --2- 172.21.15.117:0/34847527 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8a410ec00 0x7fc8a419f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:59.328 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.016+0000 7fc8a3400700 1 -- 172.21.15.117:0/34847527 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc8a40748c0 msgr2=0x7fc8a419ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:59.328 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.016+0000 7fc8a3400700 1 --2- 172.21.15.117:0/34847527 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc8a40748c0 0x7fc8a419ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:59.328 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.016+0000 7fc8a3400700 1 -- 172.21.15.117:0/34847527 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8a410ec00 msgr2=0x7fc8a419f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:59.328 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.016+0000 7fc8a3400700 1 --2- 172.21.15.117:0/34847527 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8a410ec00 0x7fc8a419f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:59.328 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.016+0000 7fc8a3400700 1 -- 172.21.15.117:0/34847527 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8a419fda0 con 0x7fc8a40751e0 2024-06-03T03:22:59.329 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.016+0000 7fc8a3e00700 1 --2- 172.21.15.117:0/34847527 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc8a40748c0 0x7fc8a419ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:22:59.329 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.016+0000 7fc8a9000700 1 --2- 172.21.15.117:0/34847527 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8a410ec00 0x7fc8a419f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T03:22:59.329 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.016+0000 7fc8a3400700 1 --2- 172.21.15.117:0/34847527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8a40751e0 0x7fc8a419b2d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc890006010 tx=0x7fc89000bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:22:59.329 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.017+0000 7fc8a0c00700 1 -- 172.21.15.117:0/34847527 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc890027070 con 0x7fc8a40751e0 2024-06-03T03:22:59.329 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.017+0000 7fc8a0c00700 1 -- 172.21.15.117:0/34847527 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc890003d70 con 0x7fc8a40751e0 2024-06-03T03:22:59.329 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.017+0000 7fc8a0c00700 1 -- 172.21.15.117:0/34847527 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc89000f460 con 0x7fc8a40751e0 2024-06-03T03:22:59.329 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.017+0000 7fc8a9a00700 1 -- 172.21.15.117:0/34847527 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8a41a0030 con 0x7fc8a40751e0 2024-06-03T03:22:59.329 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.017+0000 7fc8a9a00700 1 -- 172.21.15.117:0/34847527 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8a41a7910 con 0x7fc8a40751e0 2024-06-03T03:22:59.329 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.018+0000 7fc8a9a00700 1 -- 172.21.15.117:0/34847527 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8a4069180 con 0x7fc8a40751e0 2024-06-03T03:22:59.329 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.021+0000 7fc8a0c00700 1 -- 172.21.15.117:0/34847527 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fc89002a070 con 0x7fc8a40751e0 2024-06-03T03:22:59.329 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.022+0000 7fc8a0c00700 1 --2- 172.21.15.117:0/34847527 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fc8880a2ca0 0x7fc8880a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:22:59.329 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.022+0000 7fc8a0c00700 1 -- 172.21.15.117:0/34847527 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7fc8900d02c0 con 0x7fc8a40751e0 2024-06-03T03:22:59.329 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.023+0000 7fc8a3e00700 1 --2- 172.21.15.117:0/34847527 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fc8880a2ca0 0x7fc8880a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:22:59.330 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.023+0000 7fc8a3e00700 1 --2- 172.21.15.117:0/34847527 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fc8880a2ca0 0x7fc8880a5160 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc894011550 tx=0x7fc894009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:22:59.330 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.027+0000 7fc8a0c00700 1 -- 172.21.15.117:0/34847527 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc89005e850 con 0x7fc8a40751e0 2024-06-03T03:22:59.330 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.255+0000 7fc8a9a00700 1 -- 172.21.15.117:0/34847527 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fc8a402dca0 con 0x7fc8a40751e0 2024-06-03T03:22:59.330 INFO:teuthology.orchestra.run.smithi117.stdout:55834574961 2024-06-03T03:22:59.330 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.256+0000 7fc8a0c00700 1 -- 172.21.15.117:0/34847527 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc890013070 con 0x7fc8a40751e0 2024-06-03T03:22:59.330 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.259+0000 7fc8a9a00700 1 -- 172.21.15.117:0/34847527 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fc8880a2ca0 msgr2=0x7fc8880a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:59.330 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.259+0000 7fc8a9a00700 1 --2- 172.21.15.117:0/34847527 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fc8880a2ca0 0x7fc8880a5160 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc894011550 tx=0x7fc894009260 comp rx=0 tx=0).stop 2024-06-03T03:22:59.331 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.259+0000 7fc8a9a00700 1 -- 172.21.15.117:0/34847527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8a40751e0 msgr2=0x7fc8a419b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:22:59.331 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.259+0000 7fc8a9a00700 1 --2- 172.21.15.117:0/34847527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8a40751e0 0x7fc8a419b2d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc890006010 tx=0x7fc89000bab0 comp rx=0 tx=0).stop 2024-06-03T03:22:59.331 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.259+0000 7fc8a9a00700 1 -- 172.21.15.117:0/34847527 shutdown_connections 2024-06-03T03:22:59.331 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.259+0000 7fc8a9a00700 1 --2- 172.21.15.117:0/34847527 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fc8880a2ca0 0x7fc8880a5160 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:59.331 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.259+0000 7fc8a9a00700 1 --2- 172.21.15.117:0/34847527 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8a410ec00 0x7fc8a419f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:59.331 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.259+0000 7fc8a9a00700 1 --2- 172.21.15.117:0/34847527 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc8a40748c0 0x7fc8a419ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:59.331 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.259+0000 7fc8a9a00700 1 --2- 172.21.15.117:0/34847527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8a40751e0 0x7fc8a419b2d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:22:59.331 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.259+0000 7fc8a9a00700 1 -- 172.21.15.117:0/34847527 >> 172.21.15.117:0/34847527 conn(0x7fc8a40fdf10 msgr2=0x7fc8a40febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:22:59.331 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.259+0000 7fc8a9a00700 1 -- 172.21.15.117:0/34847527 shutdown_connections 2024-06-03T03:22:59.331 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.259+0000 7fc8a9a00700 1 -- 172.21.15.117:0/34847527 wait complete. 2024-06-03T03:22:59.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:22:59 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/34847527' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-06-03T03:22:59.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:22:59 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/1074697990' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-06-03T03:22:59.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:22:59 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/34847527' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-06-03T03:22:59.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:22:59 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/1074697990' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-06-03T03:23:00.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:23:00 smithi117 ceph-mon[31649]: pgmap v436: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 12 op/s 2024-06-03T03:23:00.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:23:00 smithi151 ceph-mon[37043]: pgmap v436: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 12 op/s 2024-06-03T03:23:00.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:23:00 smithi183 ceph-mon[37326]: pgmap v436: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 12 op/s 2024-06-03T03:23:00.767 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332170 got 219043332175 for osd.7 2024-06-03T03:23:00.767 DEBUG:teuthology.parallel:result is None 2024-06-03T03:23:00.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:00.877+0000 7fce73a00700 1 -- 172.21.15.117:0/3775924918 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce6c074fe0 msgr2=0x7fce6c10e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:00.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:00.877+0000 7fce73a00700 1 --2- 172.21.15.117:0/3775924918 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce6c074fe0 0x7fce6c10e950 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fce580098f0 tx=0x7fce58009c00 comp rx=0 tx=0).stop 2024-06-03T03:23:00.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:00.877+0000 7fce73a00700 1 -- 172.21.15.117:0/3775924918 shutdown_connections 2024-06-03T03:23:00.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:00.878+0000 7fce73a00700 1 --2- 172.21.15.117:0/3775924918 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce6c10ee90 0x7fce6c111280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:00.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:00.878+0000 7fce73a00700 1 --2- 172.21.15.117:0/3775924918 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce6c0746c0 0x7fce6c074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:00.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:00.878+0000 7fce73a00700 1 --2- 172.21.15.117:0/3775924918 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce6c074fe0 0x7fce6c10e950 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:00.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:00.878+0000 7fce73a00700 1 -- 172.21.15.117:0/3775924918 >> 172.21.15.117:0/3775924918 conn(0x7fce6c079640 msgr2=0x7fce6c079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:23:00.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:00.878+0000 7fce73a00700 1 -- 172.21.15.117:0/3775924918 shutdown_connections 2024-06-03T03:23:00.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:00.878+0000 7fce73a00700 1 -- 172.21.15.117:0/3775924918 wait complete. 2024-06-03T03:23:00.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:00.879+0000 7fce73a00700 1 Processor -- start 2024-06-03T03:23:00.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:00.879+0000 7fce73a00700 1 -- start start 2024-06-03T03:23:00.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:00.880+0000 7fce73a00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce6c0746c0 0x7fce6c19af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:00.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:00.880+0000 7fce73a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce6c074fe0 0x7fce6c19b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:00.882 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:00.880+0000 7fce73a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce6c10ee90 0x7fce6c19f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:00.882 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:00.880+0000 7fce73a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fce6c114a70 con 0x7fce6c10ee90 2024-06-03T03:23:00.882 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:00.880+0000 7fce73a00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fce6c1148f0 con 0x7fce6c074fe0 2024-06-03T03:23:00.882 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:00.880+0000 7fce73a00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fce6c114bf0 con 0x7fce6c0746c0 2024-06-03T03:23:00.882 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:00.880+0000 7fce71000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce6c0746c0 0x7fce6c19af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:00.883 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:00.880+0000 7fce6be00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce6c074fe0 0x7fce6c19b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:00.883 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:00.880+0000 7fce71a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce6c10ee90 0x7fce6c19f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:00.883 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:00.880+0000 7fce6be00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce6c074fe0 0x7fce6c19b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:57764/0 (socket says 172.21.15.117:57764) 2024-06-03T03:23:00.883 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:00.880+0000 7fce71000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce6c0746c0 0x7fce6c19af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:51210/0 (socket says 172.21.15.117:51210) 2024-06-03T03:23:00.883 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:00.880+0000 7fce6be00700 1 -- 172.21.15.117:0/4258091682 learned_addr learned my addr 172.21.15.117:0/4258091682 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:23:00.883 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:00.881+0000 7fce6be00700 1 -- 172.21.15.117:0/4258091682 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce6c0746c0 msgr2=0x7fce6c19af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:00.883 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:00.881+0000 7fce6be00700 1 --2- 172.21.15.117:0/4258091682 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce6c0746c0 0x7fce6c19af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:00.883 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:00.881+0000 7fce6be00700 1 -- 172.21.15.117:0/4258091682 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce6c10ee90 msgr2=0x7fce6c19f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:00.883 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:00.881+0000 7fce6be00700 1 --2- 172.21.15.117:0/4258091682 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce6c10ee90 0x7fce6c19f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:00.883 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:00.881+0000 7fce6be00700 1 -- 172.21.15.117:0/4258091682 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce6c19ff40 con 0x7fce6c074fe0 2024-06-03T03:23:00.883 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:00.881+0000 7fce6be00700 1 --2- 172.21.15.117:0/4258091682 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce6c074fe0 0x7fce6c19b4c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fce58005950 tx=0x7fce5800fb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:23:00.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:00.881+0000 7fce69600700 1 -- 172.21.15.117:0/4258091682 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce58005140 con 0x7fce6c074fe0 2024-06-03T03:23:00.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:00.882+0000 7fce73a00700 1 -- 172.21.15.117:0/4258091682 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce6c1a01d0 con 0x7fce6c074fe0 2024-06-03T03:23:00.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:00.882+0000 7fce69600700 1 -- 172.21.15.117:0/4258091682 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce5800b560 con 0x7fce6c074fe0 2024-06-03T03:23:00.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:00.882+0000 7fce69600700 1 -- 172.21.15.117:0/4258091682 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce58003ae0 con 0x7fce6c074fe0 2024-06-03T03:23:00.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:00.882+0000 7fce73a00700 1 -- 172.21.15.117:0/4258091682 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce6c1a7ab0 con 0x7fce6c074fe0 2024-06-03T03:23:00.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:00.883+0000 7fce73a00700 1 -- 172.21.15.117:0/4258091682 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce6c069180 con 0x7fce6c074fe0 2024-06-03T03:23:00.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:00.885+0000 7fce69600700 1 -- 172.21.15.117:0/4258091682 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fce5800ba80 con 0x7fce6c074fe0 2024-06-03T03:23:00.888 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:00.886+0000 7fce69600700 1 --2- 172.21.15.117:0/4258091682 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fce540a2c20 0x7fce540a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:00.888 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:00.886+0000 7fce69600700 1 -- 172.21.15.117:0/4258091682 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7fce580d0370 con 0x7fce6c074fe0 2024-06-03T03:23:00.888 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:00.886+0000 7fce71000700 1 --2- 172.21.15.117:0/4258091682 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fce540a2c20 0x7fce540a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:00.888 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:00.887+0000 7fce71000700 1 --2- 172.21.15.117:0/4258091682 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fce540a2c20 0x7fce540a50e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fce60009c60 tx=0x7fce60009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:23:00.892 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:00.890+0000 7fce69600700 1 -- 172.21.15.117:0/4258091682 <== mon.2 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fce58013030 con 0x7fce6c074fe0 2024-06-03T03:23:01.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:01.137+0000 7fce73a00700 1 -- 172.21.15.117:0/4258091682 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fce6c02dca0 con 0x7fce6c074fe0 2024-06-03T03:23:01.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:01.138+0000 7fce69600700 1 -- 172.21.15.117:0/4258091682 <== mon.2 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fce58021380 con 0x7fce6c074fe0 2024-06-03T03:23:01.140 INFO:teuthology.orchestra.run.smithi117.stdout:309237645374 2024-06-03T03:23:01.142 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:01.141+0000 7fce73a00700 1 -- 172.21.15.117:0/4258091682 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fce540a2c20 msgr2=0x7fce540a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:01.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:01.141+0000 7fce73a00700 1 --2- 172.21.15.117:0/4258091682 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fce540a2c20 0x7fce540a50e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fce60009c60 tx=0x7fce60009260 comp rx=0 tx=0).stop 2024-06-03T03:23:01.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:01.141+0000 7fce73a00700 1 -- 172.21.15.117:0/4258091682 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce6c074fe0 msgr2=0x7fce6c19b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:01.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:01.141+0000 7fce73a00700 1 --2- 172.21.15.117:0/4258091682 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce6c074fe0 0x7fce6c19b4c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fce58005950 tx=0x7fce5800fb20 comp rx=0 tx=0).stop 2024-06-03T03:23:01.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:01.141+0000 7fce73a00700 1 -- 172.21.15.117:0/4258091682 shutdown_connections 2024-06-03T03:23:01.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:01.141+0000 7fce73a00700 1 --2- 172.21.15.117:0/4258091682 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fce540a2c20 0x7fce540a50e0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:01.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:01.141+0000 7fce73a00700 1 --2- 172.21.15.117:0/4258091682 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce6c10ee90 0x7fce6c19f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:01.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:01.141+0000 7fce73a00700 1 --2- 172.21.15.117:0/4258091682 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce6c0746c0 0x7fce6c19af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:01.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:01.141+0000 7fce73a00700 1 --2- 172.21.15.117:0/4258091682 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce6c074fe0 0x7fce6c19b4c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:01.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:01.141+0000 7fce73a00700 1 -- 172.21.15.117:0/4258091682 >> 172.21.15.117:0/4258091682 conn(0x7fce6c079640 msgr2=0x7fce6c0ff740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:23:01.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:01.141+0000 7fce73a00700 1 -- 172.21.15.117:0/4258091682 shutdown_connections 2024-06-03T03:23:01.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:01.141+0000 7fce73a00700 1 -- 172.21.15.117:0/4258091682 wait complete. 2024-06-03T03:23:01.225 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.587+0000 7f669b400700 1 -- 172.21.15.117:0/872698311 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f669410a680 msgr2=0x7f669410aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:01.225 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.587+0000 7f669b400700 1 --2- 172.21.15.117:0/872698311 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f669410a680 0x7f669410aa60 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f66880098f0 tx=0x7f6688009c00 comp rx=0 tx=0).stop 2024-06-03T03:23:01.225 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.587+0000 7f669b400700 1 -- 172.21.15.117:0/872698311 shutdown_connections 2024-06-03T03:23:01.225 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.587+0000 7f669b400700 1 --2- 172.21.15.117:0/872698311 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6694105040 0x7f6694111550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:01.225 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.587+0000 7f669b400700 1 --2- 172.21.15.117:0/872698311 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f669410a680 0x7f669410aa60 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:01.225 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.587+0000 7f669b400700 1 --2- 172.21.15.117:0/872698311 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6694104680 0x7f6694104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:01.225 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.587+0000 7f669b400700 1 -- 172.21.15.117:0/872698311 >> 172.21.15.117:0/872698311 conn(0x7f6694100150 msgr2=0x7f6694102570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:23:01.225 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.587+0000 7f669b400700 1 -- 172.21.15.117:0/872698311 shutdown_connections 2024-06-03T03:23:01.225 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.587+0000 7f669b400700 1 -- 172.21.15.117:0/872698311 wait complete. 2024-06-03T03:23:01.225 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.588+0000 7f669b400700 1 Processor -- start 2024-06-03T03:23:01.225 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.588+0000 7f669b400700 1 -- start start 2024-06-03T03:23:01.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.589+0000 7f669b400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6694104680 0x7f669419ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:01.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.589+0000 7f669b400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6694105040 0x7f669419b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:01.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.589+0000 7f669b400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f669410a680 0x7f669419f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:01.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.589+0000 7f669b400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6694114a00 con 0x7f6694104680 2024-06-03T03:23:01.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.589+0000 7f669b400700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6694114880 con 0x7f6694105040 2024-06-03T03:23:01.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.589+0000 7f669b400700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6694114b80 con 0x7f669410a680 2024-06-03T03:23:01.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.589+0000 7f6693e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6694105040 0x7f669419b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:01.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.589+0000 7f6698a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6694104680 0x7f669419ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:01.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.589+0000 7f6693e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6694105040 0x7f669419b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:57744/0 (socket says 172.21.15.117:57744) 2024-06-03T03:23:01.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.589+0000 7f6693e00700 1 -- 172.21.15.117:0/1074697990 learned_addr learned my addr 172.21.15.117:0/1074697990 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:23:01.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.589+0000 7f6699400700 1 --2- 172.21.15.117:0/1074697990 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f669410a680 0x7f669419f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:01.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.590+0000 7f6698a00700 1 -- 172.21.15.117:0/1074697990 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f669410a680 msgr2=0x7f669419f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:01.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.590+0000 7f6698a00700 1 --2- 172.21.15.117:0/1074697990 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f669410a680 0x7f669419f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:01.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.590+0000 7f6698a00700 1 -- 172.21.15.117:0/1074697990 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6694105040 msgr2=0x7f669419b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:01.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.590+0000 7f6698a00700 1 --2- 172.21.15.117:0/1074697990 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6694105040 0x7f669419b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:01.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.590+0000 7f6698a00700 1 -- 172.21.15.117:0/1074697990 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f669419fe60 con 0x7f6694104680 2024-06-03T03:23:01.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.590+0000 7f6698a00700 1 --2- 172.21.15.117:0/1074697990 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6694104680 0x7f669419ae50 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6688000c00 tx=0x7f668800fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:23:01.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.590+0000 7f6691600700 1 -- 172.21.15.117:0/1074697990 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6688005040 con 0x7f6694104680 2024-06-03T03:23:01.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.590+0000 7f669b400700 1 -- 172.21.15.117:0/1074697990 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66941a00f0 con 0x7f6694104680 2024-06-03T03:23:01.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.590+0000 7f6691600700 1 -- 172.21.15.117:0/1074697990 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66880056c0 con 0x7f6694104680 2024-06-03T03:23:01.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.590+0000 7f6691600700 1 -- 172.21.15.117:0/1074697990 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6688003a00 con 0x7f6694104680 2024-06-03T03:23:01.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.591+0000 7f669b400700 1 -- 172.21.15.117:0/1074697990 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66941a79d0 con 0x7f6694104680 2024-06-03T03:23:01.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.592+0000 7f669b400700 1 -- 172.21.15.117:0/1074697990 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6694105dc0 con 0x7f6694104680 2024-06-03T03:23:01.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.594+0000 7f6691600700 1 -- 172.21.15.117:0/1074697990 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f668800b560 con 0x7f6694104680 2024-06-03T03:23:01.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.595+0000 7f6691600700 1 --2- 172.21.15.117:0/1074697990 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f667c0a2c20 0x7f667c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:01.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.595+0000 7f6691600700 1 -- 172.21.15.117:0/1074697990 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f66880d02a0 con 0x7f6694104680 2024-06-03T03:23:01.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.595+0000 7f6693e00700 1 --2- 172.21.15.117:0/1074697990 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f667c0a2c20 0x7f667c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:01.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.596+0000 7f6693e00700 1 --2- 172.21.15.117:0/1074697990 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f667c0a2c20 0x7f667c0a50e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6694102750 tx=0x7f6680009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:23:01.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.600+0000 7f6691600700 1 -- 172.21.15.117:0/1074697990 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6688013030 con 0x7f6694104680 2024-06-03T03:23:01.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.829+0000 7f669b400700 1 -- 172.21.15.117:0/1074697990 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f669402dca0 con 0x7f6694104680 2024-06-03T03:23:01.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.830+0000 7f6691600700 1 -- 172.21.15.117:0/1074697990 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f6688021380 con 0x7f6694104680 2024-06-03T03:23:01.228 INFO:teuthology.orchestra.run.smithi117.stdout:163208757339 2024-06-03T03:23:01.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.833+0000 7f669b400700 1 -- 172.21.15.117:0/1074697990 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f667c0a2c20 msgr2=0x7f667c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:01.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.833+0000 7f669b400700 1 --2- 172.21.15.117:0/1074697990 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f667c0a2c20 0x7f667c0a50e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6694102750 tx=0x7f6680009260 comp rx=0 tx=0).stop 2024-06-03T03:23:01.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.833+0000 7f669b400700 1 -- 172.21.15.117:0/1074697990 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6694104680 msgr2=0x7f669419ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:01.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.833+0000 7f669b400700 1 --2- 172.21.15.117:0/1074697990 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6694104680 0x7f669419ae50 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6688000c00 tx=0x7f668800fa00 comp rx=0 tx=0).stop 2024-06-03T03:23:01.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.834+0000 7f669b400700 1 -- 172.21.15.117:0/1074697990 shutdown_connections 2024-06-03T03:23:01.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.834+0000 7f669b400700 1 --2- 172.21.15.117:0/1074697990 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f667c0a2c20 0x7f667c0a50e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:01.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.834+0000 7f669b400700 1 --2- 172.21.15.117:0/1074697990 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f669410a680 0x7f669419f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:01.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.834+0000 7f669b400700 1 --2- 172.21.15.117:0/1074697990 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6694104680 0x7f669419ae50 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:01.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.834+0000 7f669b400700 1 --2- 172.21.15.117:0/1074697990 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6694105040 0x7f669419b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:01.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.834+0000 7f669b400700 1 -- 172.21.15.117:0/1074697990 >> 172.21.15.117:0/1074697990 conn(0x7f6694100150 msgr2=0x7f6694100860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:23:01.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.834+0000 7f669b400700 1 -- 172.21.15.117:0/1074697990 shutdown_connections 2024-06-03T03:23:01.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:22:58.834+0000 7f669b400700 1 -- 172.21.15.117:0/1074697990 wait complete. 2024-06-03T03:23:01.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:23:01 smithi151 ceph-mon[37043]: pgmap v437: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-06-03T03:23:01.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:23:01 smithi117 ceph-mon[31649]: pgmap v437: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-06-03T03:23:01.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:23:01 smithi183 ceph-mon[37326]: pgmap v437: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-06-03T03:23:02.285 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:23:02 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/4258091682' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-06-03T03:23:02.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:23:02 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/4258091682' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-06-03T03:23:02.440 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757335 got 163208757339 for osd.5 2024-06-03T03:23:02.441 DEBUG:teuthology.parallel:result is None 2024-06-03T03:23:02.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:23:02 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/4258091682' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-06-03T03:23:02.534 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.454+0000 7fa5bca00700 1 -- 172.21.15.117:0/407655291 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5b8104f80 msgr2=0x7fa5b8111490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:02.534 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.454+0000 7fa5bca00700 1 --2- 172.21.15.117:0/407655291 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5b8104f80 0x7fa5b8111490 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa5b000b0a0 tx=0x7fa5b000b3b0 comp rx=0 tx=0).stop 2024-06-03T03:23:02.535 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.454+0000 7fa5bca00700 1 -- 172.21.15.117:0/407655291 shutdown_connections 2024-06-03T03:23:02.535 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.454+0000 7fa5bca00700 1 --2- 172.21.15.117:0/407655291 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5b8104f80 0x7fa5b8111490 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:02.535 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.454+0000 7fa5bca00700 1 --2- 172.21.15.117:0/407655291 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5b810a650 0x7fa5b810aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:02.535 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.454+0000 7fa5bca00700 1 --2- 172.21.15.117:0/407655291 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa5b81045c0 0x7fa5b8104a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:02.535 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.454+0000 7fa5bca00700 1 -- 172.21.15.117:0/407655291 >> 172.21.15.117:0/407655291 conn(0x7fa5b81000b0 msgr2=0x7fa5b81024d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:23:02.535 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.455+0000 7fa5bca00700 1 -- 172.21.15.117:0/407655291 shutdown_connections 2024-06-03T03:23:02.535 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.455+0000 7fa5bca00700 1 -- 172.21.15.117:0/407655291 wait complete. 2024-06-03T03:23:02.535 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.456+0000 7fa5bca00700 1 Processor -- start 2024-06-03T03:23:02.535 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.456+0000 7fa5bca00700 1 -- start start 2024-06-03T03:23:02.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.456+0000 7fa5bca00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa5b81045c0 0x7fa5b81a08f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:02.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.456+0000 7fa5bca00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5b8104f80 0x7fa5b81a0e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:02.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.456+0000 7fa5bca00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5b810a650 0x7fa5b819ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:02.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.456+0000 7fa5bca00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b8114960 con 0x7fa5b8104f80 2024-06-03T03:23:02.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.456+0000 7fa5bca00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b81147e0 con 0x7fa5b81045c0 2024-06-03T03:23:02.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.456+0000 7fa5bca00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b8114ae0 con 0x7fa5b810a650 2024-06-03T03:23:02.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.456+0000 7fa5b6a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5b8104f80 0x7fa5b81a0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:02.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.456+0000 7fa5b7400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa5b81045c0 0x7fa5b81a08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:02.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.456+0000 7fa5b6a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5b8104f80 0x7fa5b81a0e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:36774/0 (socket says 172.21.15.117:36774) 2024-06-03T03:23:02.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.456+0000 7fa5b6a00700 1 -- 172.21.15.117:0/3699123091 learned_addr learned my addr 172.21.15.117:0/3699123091 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:23:02.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.457+0000 7fa5b7e00700 1 --2- 172.21.15.117:0/3699123091 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5b810a650 0x7fa5b819ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:02.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.457+0000 7fa5b7400700 1 -- 172.21.15.117:0/3699123091 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5b810a650 msgr2=0x7fa5b819ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:02.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.457+0000 7fa5b7400700 1 --2- 172.21.15.117:0/3699123091 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5b810a650 0x7fa5b819ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:02.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.457+0000 7fa5b7400700 1 -- 172.21.15.117:0/3699123091 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5b8104f80 msgr2=0x7fa5b81a0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:02.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.457+0000 7fa5b7400700 1 --2- 172.21.15.117:0/3699123091 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5b8104f80 0x7fa5b81a0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:02.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.457+0000 7fa5b7400700 1 -- 172.21.15.117:0/3699123091 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5b819b260 con 0x7fa5b81045c0 2024-06-03T03:23:02.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.457+0000 7fa5b6a00700 1 --2- 172.21.15.117:0/3699123091 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5b8104f80 0x7fa5b81a0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:23:02.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.457+0000 7fa5b7400700 1 --2- 172.21.15.117:0/3699123091 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa5b81045c0 0x7fa5b81a08f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa5a800e9c0 tx=0x7fa5a800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:23:02.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.457+0000 7fa5afe00700 1 -- 172.21.15.117:0/3699123091 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a8004230 con 0x7fa5b81045c0 2024-06-03T03:23:02.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.457+0000 7fa5bca00700 1 -- 172.21.15.117:0/3699123091 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5b819b550 con 0x7fa5b81045c0 2024-06-03T03:23:02.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.458+0000 7fa5afe00700 1 -- 172.21.15.117:0/3699123091 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5a80043d0 con 0x7fa5b81045c0 2024-06-03T03:23:02.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.458+0000 7fa5afe00700 1 -- 172.21.15.117:0/3699123091 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a8010430 con 0x7fa5b81045c0 2024-06-03T03:23:02.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.458+0000 7fa5bca00700 1 -- 172.21.15.117:0/3699123091 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5b81a77c0 con 0x7fa5b81045c0 2024-06-03T03:23:02.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.458+0000 7fa5bca00700 1 -- 172.21.15.117:0/3699123091 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5b8069180 con 0x7fa5b81045c0 2024-06-03T03:23:02.538 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.461+0000 7fa5afe00700 1 -- 172.21.15.117:0/3699123091 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fa5a800c7c0 con 0x7fa5b81045c0 2024-06-03T03:23:02.538 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.462+0000 7fa5afe00700 1 --2- 172.21.15.117:0/3699123091 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fa59c0a2c20 0x7fa59c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:02.538 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.462+0000 7fa5afe00700 1 -- 172.21.15.117:0/3699123091 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7fa5a80cf7d0 con 0x7fa5b81045c0 2024-06-03T03:23:02.538 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.462+0000 7fa5b6a00700 1 --2- 172.21.15.117:0/3699123091 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fa59c0a2c20 0x7fa59c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:02.538 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.463+0000 7fa5b6a00700 1 --2- 172.21.15.117:0/3699123091 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fa59c0a2c20 0x7fa59c0a50e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa5b8109bc0 tx=0x7fa5a4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:23:02.538 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.464+0000 7fa5afe00700 1 -- 172.21.15.117:0/3699123091 <== mon.2 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa5a809c3a0 con 0x7fa5b81045c0 2024-06-03T03:23:02.708 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.706+0000 7fa5bca00700 1 -- 172.21.15.117:0/3699123091 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fa5b81a7a50 con 0x7fa5b81045c0 2024-06-03T03:23:02.708 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.707+0000 7fa5afe00700 1 -- 172.21.15.117:0/3699123091 <== mon.2 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa5a809bca0 con 0x7fa5b81045c0 2024-06-03T03:23:02.709 INFO:teuthology.orchestra.run.smithi117.stdout:34359738486 2024-06-03T03:23:02.711 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.709+0000 7fa5bca00700 1 -- 172.21.15.117:0/3699123091 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fa59c0a2c20 msgr2=0x7fa59c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:02.711 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.709+0000 7fa5bca00700 1 --2- 172.21.15.117:0/3699123091 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fa59c0a2c20 0x7fa59c0a50e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa5b8109bc0 tx=0x7fa5a4009260 comp rx=0 tx=0).stop 2024-06-03T03:23:02.711 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.710+0000 7fa5bca00700 1 -- 172.21.15.117:0/3699123091 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa5b81045c0 msgr2=0x7fa5b81a08f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:02.711 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.710+0000 7fa5bca00700 1 --2- 172.21.15.117:0/3699123091 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa5b81045c0 0x7fa5b81a08f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa5a800e9c0 tx=0x7fa5a800ecd0 comp rx=0 tx=0).stop 2024-06-03T03:23:02.712 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.710+0000 7fa5bca00700 1 -- 172.21.15.117:0/3699123091 shutdown_connections 2024-06-03T03:23:02.712 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.710+0000 7fa5bca00700 1 --2- 172.21.15.117:0/3699123091 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fa59c0a2c20 0x7fa59c0a50e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:02.712 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.710+0000 7fa5bca00700 1 --2- 172.21.15.117:0/3699123091 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5b810a650 0x7fa5b819ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:02.712 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.710+0000 7fa5bca00700 1 --2- 172.21.15.117:0/3699123091 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa5b81045c0 0x7fa5b81a08f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:02.712 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.710+0000 7fa5bca00700 1 --2- 172.21.15.117:0/3699123091 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5b8104f80 0x7fa5b81a0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:02.712 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.710+0000 7fa5bca00700 1 -- 172.21.15.117:0/3699123091 >> 172.21.15.117:0/3699123091 conn(0x7fa5b81000b0 msgr2=0x7fa5b8101590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:23:02.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.710+0000 7fa5bca00700 1 -- 172.21.15.117:0/3699123091 shutdown_connections 2024-06-03T03:23:02.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.710+0000 7fa5bca00700 1 -- 172.21.15.117:0/3699123091 wait complete. 2024-06-03T03:23:03.215 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:23:03 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/3699123091' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-06-03T03:23:03.215 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:23:03 smithi117 ceph-mon[31649]: pgmap v438: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 12 op/s 2024-06-03T03:23:03.218 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574954 got 55834574961 for osd.1 2024-06-03T03:23:03.218 DEBUG:teuthology.parallel:result is None 2024-06-03T03:23:03.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:23:03 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/3699123091' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-06-03T03:23:03.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:23:03 smithi151 ceph-mon[37043]: pgmap v438: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 12 op/s 2024-06-03T03:23:03.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:23:03 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/3699123091' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-06-03T03:23:03.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:23:03 smithi183 ceph-mon[37326]: pgmap v438: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 12 op/s 2024-06-03T03:23:03.542 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.952+0000 7fc72b200700 1 -- 172.21.15.117:0/1003489869 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc72406b3c0 msgr2=0x7fc72410f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:03.542 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.952+0000 7fc72b200700 1 --2- 172.21.15.117:0/1003489869 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc72406b3c0 0x7fc72410f310 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc71c009f90 tx=0x7fc71c00b330 comp rx=0 tx=0).stop 2024-06-03T03:23:03.542 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.952+0000 7fc72b200700 1 -- 172.21.15.117:0/1003489869 shutdown_connections 2024-06-03T03:23:03.542 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.952+0000 7fc72b200700 1 --2- 172.21.15.117:0/1003489869 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc72406b3c0 0x7fc72410f310 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:03.542 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.952+0000 7fc72b200700 1 --2- 172.21.15.117:0/1003489869 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7241084f0 0x7fc7241088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:03.542 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.952+0000 7fc72b200700 1 --2- 172.21.15.117:0/1003489869 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc72406aa00 0x7fc72406ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:03.542 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.952+0000 7fc72b200700 1 -- 172.21.15.117:0/1003489869 >> 172.21.15.117:0/1003489869 conn(0x7fc724078300 msgr2=0x7fc724078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:23:03.542 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.953+0000 7fc72b200700 1 -- 172.21.15.117:0/1003489869 shutdown_connections 2024-06-03T03:23:03.542 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.953+0000 7fc72b200700 1 -- 172.21.15.117:0/1003489869 wait complete. 2024-06-03T03:23:03.542 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.954+0000 7fc72b200700 1 Processor -- start 2024-06-03T03:23:03.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.954+0000 7fc72b200700 1 -- start start 2024-06-03T03:23:03.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.954+0000 7fc72b200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc72406aa00 0x7fc72419c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:03.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.954+0000 7fc72b200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc72406b3c0 0x7fc72419cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:03.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.954+0000 7fc72b200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7241084f0 0x7fc724196910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:03.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.955+0000 7fc72b200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc7241105f0 con 0x7fc7241084f0 2024-06-03T03:23:03.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.955+0000 7fc72b200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc724110470 con 0x7fc72406aa00 2024-06-03T03:23:03.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.955+0000 7fc72b200700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc724110770 con 0x7fc72406b3c0 2024-06-03T03:23:03.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.955+0000 7fc729200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7241084f0 0x7fc724196910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:03.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.955+0000 7fc729200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7241084f0 0x7fc724196910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:36792/0 (socket says 172.21.15.117:36792) 2024-06-03T03:23:03.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.955+0000 7fc729200700 1 -- 172.21.15.117:0/2211030619 learned_addr learned my addr 172.21.15.117:0/2211030619 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:23:03.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.955+0000 7fc723e00700 1 --2- 172.21.15.117:0/2211030619 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc72406aa00 0x7fc72419c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:03.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.955+0000 7fc71be00700 1 --2- 172.21.15.117:0/2211030619 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc72406b3c0 0x7fc72419cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:03.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.955+0000 7fc729200700 1 -- 172.21.15.117:0/2211030619 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc72406b3c0 msgr2=0x7fc72419cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:03.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.955+0000 7fc729200700 1 --2- 172.21.15.117:0/2211030619 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc72406b3c0 0x7fc72419cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:03.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.955+0000 7fc729200700 1 -- 172.21.15.117:0/2211030619 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc72406aa00 msgr2=0x7fc72419c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:03.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.955+0000 7fc729200700 1 --2- 172.21.15.117:0/2211030619 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc72406aa00 0x7fc72419c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:03.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.955+0000 7fc729200700 1 -- 172.21.15.117:0/2211030619 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc724197070 con 0x7fc7241084f0 2024-06-03T03:23:03.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.955+0000 7fc723e00700 1 --2- 172.21.15.117:0/2211030619 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc72406aa00 0x7fc72419c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:23:03.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.956+0000 7fc729200700 1 --2- 172.21.15.117:0/2211030619 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7241084f0 0x7fc724196910 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc71c009550 tx=0x7fc71c00cfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:23:03.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.956+0000 7fc721600700 1 -- 172.21.15.117:0/2211030619 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc71c021070 con 0x7fc7241084f0 2024-06-03T03:23:03.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.956+0000 7fc721600700 1 -- 172.21.15.117:0/2211030619 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc71c007ac0 con 0x7fc7241084f0 2024-06-03T03:23:03.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.956+0000 7fc721600700 1 -- 172.21.15.117:0/2211030619 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc71c0052a0 con 0x7fc7241084f0 2024-06-03T03:23:03.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.956+0000 7fc72b200700 1 -- 172.21.15.117:0/2211030619 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc724197300 con 0x7fc7241084f0 2024-06-03T03:23:03.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.956+0000 7fc72b200700 1 -- 172.21.15.117:0/2211030619 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7241a35d0 con 0x7fc7241084f0 2024-06-03T03:23:03.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.957+0000 7fc72b200700 1 -- 172.21.15.117:0/2211030619 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc724103bb0 con 0x7fc7241084f0 2024-06-03T03:23:03.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.959+0000 7fc721600700 1 -- 172.21.15.117:0/2211030619 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fc71c024070 con 0x7fc7241084f0 2024-06-03T03:23:03.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.960+0000 7fc721600700 1 --2- 172.21.15.117:0/2211030619 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fc7080a7030 0x7fc7080a94f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:03.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.960+0000 7fc721600700 1 -- 172.21.15.117:0/2211030619 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7fc71c00f020 con 0x7fc7241084f0 2024-06-03T03:23:03.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.961+0000 7fc723e00700 1 --2- 172.21.15.117:0/2211030619 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fc7080a7030 0x7fc7080a94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:03.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.961+0000 7fc723e00700 1 --2- 172.21.15.117:0/2211030619 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fc7080a7030 0x7fc7080a94f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc7240ff9b0 tx=0x7fc714009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:23:03.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:02.965+0000 7fc721600700 1 -- 172.21.15.117:0/2211030619 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc71c02b020 con 0x7fc7241084f0 2024-06-03T03:23:03.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:03.193+0000 7fc72b200700 1 -- 172.21.15.117:0/2211030619 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fc72402dca0 con 0x7fc7241084f0 2024-06-03T03:23:03.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:03.194+0000 7fc721600700 1 -- 172.21.15.117:0/2211030619 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc71c09d3c0 con 0x7fc7241084f0 2024-06-03T03:23:03.546 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:03.197+0000 7fc72b200700 1 -- 172.21.15.117:0/2211030619 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fc7080a7030 msgr2=0x7fc7080a94f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:03.546 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:03.197+0000 7fc72b200700 1 --2- 172.21.15.117:0/2211030619 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fc7080a7030 0x7fc7080a94f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc7240ff9b0 tx=0x7fc714009310 comp rx=0 tx=0).stop 2024-06-03T03:23:03.546 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:03.197+0000 7fc72b200700 1 -- 172.21.15.117:0/2211030619 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7241084f0 msgr2=0x7fc724196910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:03.546 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:03.197+0000 7fc72b200700 1 --2- 172.21.15.117:0/2211030619 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7241084f0 0x7fc724196910 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc71c009550 tx=0x7fc71c00cfa0 comp rx=0 tx=0).stop 2024-06-03T03:23:03.546 INFO:teuthology.orchestra.run.smithi117.stdout:77309411436 2024-06-03T03:23:03.546 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:03.197+0000 7fc72b200700 1 -- 172.21.15.117:0/2211030619 shutdown_connections 2024-06-03T03:23:03.546 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:03.197+0000 7fc72b200700 1 --2- 172.21.15.117:0/2211030619 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fc7080a7030 0x7fc7080a94f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:03.546 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:03.197+0000 7fc72b200700 1 --2- 172.21.15.117:0/2211030619 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7241084f0 0x7fc724196910 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:03.547 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:03.197+0000 7fc72b200700 1 --2- 172.21.15.117:0/2211030619 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc72406aa00 0x7fc72419c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:03.547 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:03.197+0000 7fc72b200700 1 --2- 172.21.15.117:0/2211030619 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc72406b3c0 0x7fc72419cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:03.547 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:03.197+0000 7fc72b200700 1 -- 172.21.15.117:0/2211030619 >> 172.21.15.117:0/2211030619 conn(0x7fc724078300 msgr2=0x7fc724100050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:23:03.547 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:03.197+0000 7fc72b200700 1 -- 172.21.15.117:0/2211030619 shutdown_connections 2024-06-03T03:23:03.547 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:03.197+0000 7fc72b200700 1 -- 172.21.15.117:0/2211030619 wait complete. 2024-06-03T03:23:04.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:23:04 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/2211030619' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-06-03T03:23:04.320 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411429 got 77309411436 for osd.2 2024-06-03T03:23:04.321 DEBUG:teuthology.parallel:result is None 2024-06-03T03:23:04.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:23:04 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/2211030619' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-06-03T03:23:04.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:23:04 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/2211030619' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-06-03T03:23:04.957 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645368 got 309237645374 for osd.10 2024-06-03T03:23:04.957 DEBUG:teuthology.parallel:result is None 2024-06-03T03:23:05.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:23:05 smithi117 ceph-mon[31649]: pgmap v439: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2024-06-03T03:23:05.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:23:05 smithi151 ceph-mon[37043]: pgmap v439: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2024-06-03T03:23:05.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:23:05 smithi183 ceph-mon[37326]: pgmap v439: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2024-06-03T03:23:05.723 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920859 got 141733920864 for osd.4 2024-06-03T03:23:05.724 DEBUG:teuthology.parallel:result is None 2024-06-03T03:23:06.109 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495696 got 197568495701 for osd.6 2024-06-03T03:23:06.109 DEBUG:teuthology.parallel:result is None 2024-06-03T03:23:07.403 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:23:07.442 INFO:tasks.cephadm.ceph_manager.ceph:need seq 343597383728 got 343597383736 for osd.11 2024-06-03T03:23:07.442 DEBUG:teuthology.parallel:result is None 2024-06-03T03:23:07.841 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738480 got 34359738486 for osd.0 2024-06-03T03:23:07.841 DEBUG:teuthology.parallel:result is None 2024-06-03T03:23:08.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:23:07 smithi117 ceph-mon[31649]: pgmap v440: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2024-06-03T03:23:08.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:23:07 smithi183 ceph-mon[37326]: pgmap v440: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2024-06-03T03:23:08.388 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070532 got 253403070538 for osd.8 2024-06-03T03:23:08.388 DEBUG:teuthology.parallel:result is None 2024-06-03T03:23:08.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:23:07 smithi151 ceph-mon[37043]: pgmap v440: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2024-06-03T03:23:10.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:23:09 smithi183 ceph-mon[37326]: pgmap v441: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 12 op/s 2024-06-03T03:23:10.331 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:23:09 smithi117 ceph-mon[31649]: pgmap v441: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 12 op/s 2024-06-03T03:23:10.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:23:09 smithi151 ceph-mon[37043]: pgmap v441: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 12 op/s 2024-06-03T03:23:11.175 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.096+0000 7faa33a00700 1 -- 172.21.15.117:0/3170783939 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa2c074fe0 msgr2=0x7faa2c10e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:11.175 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.096+0000 7faa33a00700 1 --2- 172.21.15.117:0/3170783939 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa2c074fe0 0x7faa2c10e950 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7faa18009990 tx=0x7faa18009ca0 comp rx=0 tx=0).stop 2024-06-03T03:23:11.175 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.097+0000 7faa33a00700 1 -- 172.21.15.117:0/3170783939 shutdown_connections 2024-06-03T03:23:11.176 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.097+0000 7faa33a00700 1 --2- 172.21.15.117:0/3170783939 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faa2c10ee90 0x7faa2c111280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:11.176 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.097+0000 7faa33a00700 1 --2- 172.21.15.117:0/3170783939 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faa2c0746c0 0x7faa2c074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:11.176 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.097+0000 7faa33a00700 1 --2- 172.21.15.117:0/3170783939 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa2c074fe0 0x7faa2c10e950 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:11.176 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.097+0000 7faa33a00700 1 -- 172.21.15.117:0/3170783939 >> 172.21.15.117:0/3170783939 conn(0x7faa2c0795f0 msgr2=0x7faa2c079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:23:11.176 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.097+0000 7faa33a00700 1 -- 172.21.15.117:0/3170783939 shutdown_connections 2024-06-03T03:23:11.176 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.097+0000 7faa33a00700 1 -- 172.21.15.117:0/3170783939 wait complete. 2024-06-03T03:23:11.176 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.098+0000 7faa33a00700 1 Processor -- start 2024-06-03T03:23:11.176 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.099+0000 7faa33a00700 1 -- start start 2024-06-03T03:23:11.176 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.099+0000 7faa33a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faa2c0746c0 0x7faa2c19aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:11.176 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.099+0000 7faa33a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa2c074fe0 0x7faa2c19b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:11.176 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.099+0000 7faa33a00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faa2c10ee90 0x7faa2c19f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:11.176 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.099+0000 7faa33a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7faa2c1149a0 con 0x7faa2c074fe0 2024-06-03T03:23:11.176 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.099+0000 7faa33a00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7faa2c114820 con 0x7faa2c0746c0 2024-06-03T03:23:11.177 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.099+0000 7faa33a00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7faa2c114b20 con 0x7faa2c10ee90 2024-06-03T03:23:11.177 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.099+0000 7faa2be00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa2c074fe0 0x7faa2c19b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:11.177 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.100+0000 7faa2be00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa2c074fe0 0x7faa2c19b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40784/0 (socket says 172.21.15.117:40784) 2024-06-03T03:23:11.177 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.100+0000 7faa2be00700 1 -- 172.21.15.117:0/607476659 learned_addr learned my addr 172.21.15.117:0/607476659 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:23:11.177 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.100+0000 7faa31000700 1 --2- 172.21.15.117:0/607476659 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faa2c0746c0 0x7faa2c19aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:11.177 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.100+0000 7faa31a00700 1 --2- 172.21.15.117:0/607476659 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faa2c10ee90 0x7faa2c19f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:11.177 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.100+0000 7faa2be00700 1 -- 172.21.15.117:0/607476659 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faa2c10ee90 msgr2=0x7faa2c19f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:11.177 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.100+0000 7faa2be00700 1 --2- 172.21.15.117:0/607476659 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faa2c10ee90 0x7faa2c19f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:11.177 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.100+0000 7faa2be00700 1 -- 172.21.15.117:0/607476659 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faa2c0746c0 msgr2=0x7faa2c19aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:11.177 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.100+0000 7faa2be00700 1 --2- 172.21.15.117:0/607476659 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faa2c0746c0 0x7faa2c19aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:11.177 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.100+0000 7faa2be00700 1 -- 172.21.15.117:0/607476659 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa2c19fec0 con 0x7faa2c074fe0 2024-06-03T03:23:11.177 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.100+0000 7faa2be00700 1 --2- 172.21.15.117:0/607476659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa2c074fe0 0x7faa2c19b3f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7faa18000c00 tx=0x7faa180047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:23:11.177 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.101+0000 7faa29600700 1 -- 172.21.15.117:0/607476659 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa18027070 con 0x7faa2c074fe0 2024-06-03T03:23:11.178 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.101+0000 7faa29600700 1 -- 172.21.15.117:0/607476659 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa18003780 con 0x7faa2c074fe0 2024-06-03T03:23:11.178 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.101+0000 7faa29600700 1 -- 172.21.15.117:0/607476659 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa18022db0 con 0x7faa2c074fe0 2024-06-03T03:23:11.178 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.101+0000 7faa33a00700 1 -- 172.21.15.117:0/607476659 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa2c1a0150 con 0x7faa2c074fe0 2024-06-03T03:23:11.178 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.101+0000 7faa33a00700 1 -- 172.21.15.117:0/607476659 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa2c1a7a30 con 0x7faa2c074fe0 2024-06-03T03:23:11.178 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.102+0000 7faa33a00700 1 -- 172.21.15.117:0/607476659 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa2c069180 con 0x7faa2c074fe0 2024-06-03T03:23:11.178 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.104+0000 7faa29600700 1 -- 172.21.15.117:0/607476659 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7faa1802a070 con 0x7faa2c074fe0 2024-06-03T03:23:11.178 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.106+0000 7faa29600700 1 --2- 172.21.15.117:0/607476659 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7faa140a2c20 0x7faa140a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:11.178 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.106+0000 7faa29600700 1 -- 172.21.15.117:0/607476659 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7faa180d00f0 con 0x7faa2c074fe0 2024-06-03T03:23:11.178 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.107+0000 7faa31000700 1 --2- 172.21.15.117:0/607476659 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7faa140a2c20 0x7faa140a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:11.178 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.107+0000 7faa31000700 1 --2- 172.21.15.117:0/607476659 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7faa140a2c20 0x7faa140a50e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7faa20007be0 tx=0x7faa200073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:23:11.178 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.110+0000 7faa29600700 1 -- 172.21.15.117:0/607476659 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7faa18013070 con 0x7faa2c074fe0 2024-06-03T03:23:11.292 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.220+0000 7fc061a00700 1 -- 172.21.15.117:0/33027728 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc05c102620 msgr2=0x7fc05c102a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:11.292 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.220+0000 7fc061a00700 1 --2- 172.21.15.117:0/33027728 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc05c102620 0x7fc05c102a00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc04c0098f0 tx=0x7fc04c009c00 comp rx=0 tx=0).stop 2024-06-03T03:23:11.292 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.221+0000 7fc061a00700 1 -- 172.21.15.117:0/33027728 shutdown_connections 2024-06-03T03:23:11.292 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.221+0000 7fc061a00700 1 --2- 172.21.15.117:0/33027728 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc05c107700 0x7fc05c10f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:11.292 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.221+0000 7fc061a00700 1 --2- 172.21.15.117:0/33027728 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc05c102620 0x7fc05c102a00 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:11.292 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.221+0000 7fc061a00700 1 --2- 172.21.15.117:0/33027728 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc05c102fd0 0x7fc05c107020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:11.292 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.221+0000 7fc061a00700 1 -- 172.21.15.117:0/33027728 >> 172.21.15.117:0/33027728 conn(0x7fc05c0fdf10 msgr2=0x7fc05c100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:23:11.292 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.221+0000 7fc061a00700 1 -- 172.21.15.117:0/33027728 shutdown_connections 2024-06-03T03:23:11.293 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.221+0000 7fc061a00700 1 -- 172.21.15.117:0/33027728 wait complete. 2024-06-03T03:23:11.293 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.222+0000 7fc061a00700 1 Processor -- start 2024-06-03T03:23:11.293 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.222+0000 7fc061a00700 1 -- start start 2024-06-03T03:23:11.293 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.222+0000 7fc061a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc05c102620 0x7fc05c19af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:11.293 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.222+0000 7fc061a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc05c102fd0 0x7fc05c19b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:11.293 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.222+0000 7fc061a00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc05c107700 0x7fc05c19f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:11.293 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.222+0000 7fc061a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc05c075110 con 0x7fc05c102fd0 2024-06-03T03:23:11.293 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.223+0000 7fc061a00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc05c074f90 con 0x7fc05c102620 2024-06-03T03:23:11.293 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.223+0000 7fc061a00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc05c075290 con 0x7fc05c107700 2024-06-03T03:23:11.293 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.223+0000 7fc05b400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc05c102fd0 0x7fc05c19b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:11.293 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.223+0000 7fc05be00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc05c102620 0x7fc05c19af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:11.294 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.223+0000 7fc05b400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc05c102fd0 0x7fc05c19b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40798/0 (socket says 172.21.15.117:40798) 2024-06-03T03:23:11.294 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.223+0000 7fc05b400700 1 -- 172.21.15.117:0/3175240264 learned_addr learned my addr 172.21.15.117:0/3175240264 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:23:11.294 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.223+0000 7fc061000700 1 --2- 172.21.15.117:0/3175240264 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc05c107700 0x7fc05c19f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:11.294 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.223+0000 7fc05b400700 1 -- 172.21.15.117:0/3175240264 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc05c107700 msgr2=0x7fc05c19f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:11.294 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.223+0000 7fc05b400700 1 --2- 172.21.15.117:0/3175240264 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc05c107700 0x7fc05c19f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:11.294 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.223+0000 7fc05b400700 1 -- 172.21.15.117:0/3175240264 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc05c102620 msgr2=0x7fc05c19af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:11.294 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.223+0000 7fc05b400700 1 --2- 172.21.15.117:0/3175240264 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc05c102620 0x7fc05c19af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:11.294 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.223+0000 7fc05b400700 1 -- 172.21.15.117:0/3175240264 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc05c19fda0 con 0x7fc05c102fd0 2024-06-03T03:23:11.294 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.224+0000 7fc05b400700 1 --2- 172.21.15.117:0/3175240264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc05c102fd0 0x7fc05c19b4c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc04800e970 tx=0x7fc04800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:23:11.294 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.224+0000 7fc058c00700 1 -- 172.21.15.117:0/3175240264 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc048004a30 con 0x7fc05c102fd0 2024-06-03T03:23:11.294 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.224+0000 7fc061a00700 1 -- 172.21.15.117:0/3175240264 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc05c1a0090 con 0x7fc05c102fd0 2024-06-03T03:23:11.294 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.224+0000 7fc058c00700 1 -- 172.21.15.117:0/3175240264 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc04800ccc0 con 0x7fc05c102fd0 2024-06-03T03:23:11.294 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.224+0000 7fc058c00700 1 -- 172.21.15.117:0/3175240264 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc048005650 con 0x7fc05c102fd0 2024-06-03T03:23:11.294 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.224+0000 7fc061a00700 1 -- 172.21.15.117:0/3175240264 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc05c1a7950 con 0x7fc05c102fd0 2024-06-03T03:23:11.295 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.225+0000 7fc061a00700 1 -- 172.21.15.117:0/3175240264 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc05c069180 con 0x7fc05c102fd0 2024-06-03T03:23:11.295 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.227+0000 7fc058c00700 1 -- 172.21.15.117:0/3175240264 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fc048004bd0 con 0x7fc05c102fd0 2024-06-03T03:23:11.295 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.230+0000 7fc058c00700 1 --2- 172.21.15.117:0/3175240264 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fc0400a2c20 0x7fc0400a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:11.295 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.230+0000 7fc058c00700 1 -- 172.21.15.117:0/3175240264 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7fc0480cf840 con 0x7fc05c102fd0 2024-06-03T03:23:11.295 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.230+0000 7fc05be00700 1 --2- 172.21.15.117:0/3175240264 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fc0400a2c20 0x7fc0400a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:11.295 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.231+0000 7fc05be00700 1 --2- 172.21.15.117:0/3175240264 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fc0400a2c20 0x7fc0400a50e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc05c19be30 tx=0x7fc04c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:23:11.295 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.232+0000 7fc058c00700 1 -- 172.21.15.117:0/3175240264 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc048053850 con 0x7fc05c102fd0 2024-06-03T03:23:11.343 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.341+0000 7faa33a00700 1 -- 172.21.15.117:0/607476659 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7faa2c02dca0 con 0x7faa2c074fe0 2024-06-03T03:23:11.343 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.342+0000 7faa29600700 1 -- 172.21.15.117:0/607476659 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7faa1809ccf0 con 0x7faa2c074fe0 2024-06-03T03:23:11.344 INFO:teuthology.orchestra.run.smithi117.stdout:287762808902 2024-06-03T03:23:11.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.345+0000 7faa33a00700 1 -- 172.21.15.117:0/607476659 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7faa140a2c20 msgr2=0x7faa140a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:11.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.345+0000 7faa33a00700 1 --2- 172.21.15.117:0/607476659 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7faa140a2c20 0x7faa140a50e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7faa20007be0 tx=0x7faa200073d0 comp rx=0 tx=0).stop 2024-06-03T03:23:11.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.345+0000 7faa33a00700 1 -- 172.21.15.117:0/607476659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa2c074fe0 msgr2=0x7faa2c19b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:11.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.345+0000 7faa33a00700 1 --2- 172.21.15.117:0/607476659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa2c074fe0 0x7faa2c19b3f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7faa18000c00 tx=0x7faa180047b0 comp rx=0 tx=0).stop 2024-06-03T03:23:11.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.345+0000 7faa33a00700 1 -- 172.21.15.117:0/607476659 shutdown_connections 2024-06-03T03:23:11.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.345+0000 7faa33a00700 1 --2- 172.21.15.117:0/607476659 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7faa140a2c20 0x7faa140a50e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:11.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.345+0000 7faa33a00700 1 --2- 172.21.15.117:0/607476659 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faa2c10ee90 0x7faa2c19f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:11.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.345+0000 7faa33a00700 1 --2- 172.21.15.117:0/607476659 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faa2c0746c0 0x7faa2c19aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:11.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.345+0000 7faa33a00700 1 --2- 172.21.15.117:0/607476659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa2c074fe0 0x7faa2c19b3f0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:11.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.345+0000 7faa33a00700 1 -- 172.21.15.117:0/607476659 >> 172.21.15.117:0/607476659 conn(0x7faa2c0795f0 msgr2=0x7faa2c0ff6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:23:11.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.345+0000 7faa33a00700 1 -- 172.21.15.117:0/607476659 shutdown_connections 2024-06-03T03:23:11.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.345+0000 7faa33a00700 1 -- 172.21.15.117:0/607476659 wait complete. 2024-06-03T03:23:11.470 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.468+0000 7fc061a00700 1 -- 172.21.15.117:0/3175240264 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fc05c19c100 con 0x7fc05c102fd0 2024-06-03T03:23:11.470 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.468+0000 7fc058c00700 1 -- 172.21.15.117:0/3175240264 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fc05c19c100 con 0x7fc05c102fd0 2024-06-03T03:23:11.471 INFO:teuthology.orchestra.run.smithi117.stdout:107374182504 2024-06-03T03:23:11.474 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.473+0000 7fc061a00700 1 -- 172.21.15.117:0/3175240264 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fc0400a2c20 msgr2=0x7fc0400a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:11.474 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.473+0000 7fc061a00700 1 --2- 172.21.15.117:0/3175240264 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fc0400a2c20 0x7fc0400a50e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc05c19be30 tx=0x7fc04c01a040 comp rx=0 tx=0).stop 2024-06-03T03:23:11.474 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.473+0000 7fc061a00700 1 -- 172.21.15.117:0/3175240264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc05c102fd0 msgr2=0x7fc05c19b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:11.475 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.473+0000 7fc061a00700 1 --2- 172.21.15.117:0/3175240264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc05c102fd0 0x7fc05c19b4c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc04800e970 tx=0x7fc04800ec80 comp rx=0 tx=0).stop 2024-06-03T03:23:11.475 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.473+0000 7fc061a00700 1 -- 172.21.15.117:0/3175240264 shutdown_connections 2024-06-03T03:23:11.475 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.473+0000 7fc061a00700 1 --2- 172.21.15.117:0/3175240264 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fc0400a2c20 0x7fc0400a50e0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:11.475 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.473+0000 7fc061a00700 1 --2- 172.21.15.117:0/3175240264 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc05c107700 0x7fc05c19f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:11.475 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.473+0000 7fc061a00700 1 --2- 172.21.15.117:0/3175240264 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc05c102620 0x7fc05c19af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:11.475 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.473+0000 7fc061a00700 1 --2- 172.21.15.117:0/3175240264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc05c102fd0 0x7fc05c19b4c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:11.475 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.473+0000 7fc061a00700 1 -- 172.21.15.117:0/3175240264 >> 172.21.15.117:0/3175240264 conn(0x7fc05c0fdf10 msgr2=0x7fc05c106000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:23:11.475 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.473+0000 7fc061a00700 1 -- 172.21.15.117:0/3175240264 shutdown_connections 2024-06-03T03:23:11.475 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:11.473+0000 7fc061a00700 1 -- 172.21.15.117:0/3175240264 wait complete. 2024-06-03T03:23:12.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:23:11 smithi117 ceph-mon[31649]: pgmap v442: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:23:12.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:23:11 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/607476659' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-06-03T03:23:12.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:23:11 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/3175240264' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-06-03T03:23:12.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:23:11 smithi183 ceph-mon[37326]: pgmap v442: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:23:12.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:23:11 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/607476659' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-06-03T03:23:12.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:23:11 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/3175240264' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-06-03T03:23:12.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:23:11 smithi151 ceph-mon[37043]: pgmap v442: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:23:12.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:23:11 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/607476659' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-06-03T03:23:12.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:23:11 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/3175240264' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-06-03T03:23:12.464 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808895 got 287762808902 for osd.9 2024-06-03T03:23:12.465 DEBUG:teuthology.parallel:result is None 2024-06-03T03:23:13.115 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182498 got 107374182504 for osd.3 2024-06-03T03:23:13.116 DEBUG:teuthology.parallel:result is None 2024-06-03T03:23:13.116 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-06-03T03:23:13.116 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-03T03:23:13.567 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:23:14.065 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:23:13 smithi117 ceph-mon[31649]: pgmap v443: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-03T03:23:14.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:23:13 smithi183 ceph-mon[37326]: pgmap v443: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-03T03:23:14.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:23:13 smithi151 ceph-mon[37043]: pgmap v443: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-03T03:23:16.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:16.012+0000 7ffaf3c00700 1 -- 172.21.15.117:0/4023838408 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffaec1085d0 msgr2=0x7ffaec1089b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:16.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:16.012+0000 7ffaf3c00700 1 --2- 172.21.15.117:0/4023838408 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffaec1085d0 0x7ffaec1089b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ffae0009990 tx=0x7ffae0009ca0 comp rx=0 tx=0).stop 2024-06-03T03:23:16.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:16.012+0000 7ffaf3c00700 1 -- 172.21.15.117:0/4023838408 shutdown_connections 2024-06-03T03:23:16.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:16.012+0000 7ffaf3c00700 1 --2- 172.21.15.117:0/4023838408 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffaec102ef0 0x7ffaec10f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:16.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:16.012+0000 7ffaf3c00700 1 --2- 172.21.15.117:0/4023838408 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffaec1085d0 0x7ffaec1089b0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:16.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:16.012+0000 7ffaf3c00700 1 --2- 172.21.15.117:0/4023838408 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffaec102530 0x7ffaec1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:16.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:16.012+0000 7ffaf3c00700 1 -- 172.21.15.117:0/4023838408 >> 172.21.15.117:0/4023838408 conn(0x7ffaec079640 msgr2=0x7ffaec079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:23:16.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:16.012+0000 7ffaf3c00700 1 -- 172.21.15.117:0/4023838408 shutdown_connections 2024-06-03T03:23:16.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:16.012+0000 7ffaf3c00700 1 -- 172.21.15.117:0/4023838408 wait complete. 2024-06-03T03:23:16.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:16.013+0000 7ffaf3c00700 1 Processor -- start 2024-06-03T03:23:16.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:16.013+0000 7ffaf3c00700 1 -- start start 2024-06-03T03:23:16.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:16.013+0000 7ffaf3c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffaec102530 0x7ffaec19e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:16.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:16.014+0000 7ffaf3c00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffaec102ef0 0x7ffaec19ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:16.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:16.014+0000 7ffaf3c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffaec1085d0 0x7ffaec198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:16.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:16.014+0000 7ffaf3c00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ffaec112810 con 0x7ffaec1085d0 2024-06-03T03:23:16.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:16.014+0000 7ffaf3c00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ffaec112690 con 0x7ffaec102530 2024-06-03T03:23:16.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:16.014+0000 7ffaf3c00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ffaec112990 con 0x7ffaec102ef0 2024-06-03T03:23:16.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:16.014+0000 7ffaf1c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffaec1085d0 0x7ffaec198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:16.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:16.014+0000 7ffaf1200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffaec102530 0x7ffaec19e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:16.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:16.014+0000 7ffaf1200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffaec102530 0x7ffaec19e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:37058/0 (socket says 172.21.15.117:37058) 2024-06-03T03:23:16.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:16.014+0000 7ffaebe00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffaec102ef0 0x7ffaec19ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:16.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:16.014+0000 7ffaf1200700 1 -- 172.21.15.117:0/2489180629 learned_addr learned my addr 172.21.15.117:0/2489180629 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:23:16.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:16.014+0000 7ffaf1200700 1 -- 172.21.15.117:0/2489180629 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffaec102ef0 msgr2=0x7ffaec19ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:16.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:16.014+0000 7ffaf1200700 1 --2- 172.21.15.117:0/2489180629 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffaec102ef0 0x7ffaec19ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:16.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:16.014+0000 7ffaf1200700 1 -- 172.21.15.117:0/2489180629 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffaec1085d0 msgr2=0x7ffaec198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:16.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:16.014+0000 7ffaf1200700 1 --2- 172.21.15.117:0/2489180629 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffaec1085d0 0x7ffaec198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:16.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:16.014+0000 7ffaf1200700 1 -- 172.21.15.117:0/2489180629 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaec199170 con 0x7ffaec102530 2024-06-03T03:23:16.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:16.015+0000 7ffaebe00700 1 --2- 172.21.15.117:0/2489180629 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffaec102ef0 0x7ffaec19ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:23:16.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:16.015+0000 7ffaf1200700 1 --2- 172.21.15.117:0/2489180629 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffaec102530 0x7ffaec19e850 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ffae00095b0 tx=0x7ffae0005590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:23:16.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:16.015+0000 7ffae9600700 1 -- 172.21.15.117:0/2489180629 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae0026070 con 0x7ffaec102530 2024-06-03T03:23:16.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:16.015+0000 7ffaf3c00700 1 -- 172.21.15.117:0/2489180629 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffaec199400 con 0x7ffaec102530 2024-06-03T03:23:16.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:16.015+0000 7ffae9600700 1 -- 172.21.15.117:0/2489180629 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffae000fe10 con 0x7ffaec102530 2024-06-03T03:23:16.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:16.015+0000 7ffae9600700 1 -- 172.21.15.117:0/2489180629 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae00173f0 con 0x7ffaec102530 2024-06-03T03:23:16.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:16.015+0000 7ffaf3c00700 1 -- 172.21.15.117:0/2489180629 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffaec1a5720 con 0x7ffaec102530 2024-06-03T03:23:16.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:16.017+0000 7ffaf3c00700 1 -- 172.21.15.117:0/2489180629 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffaec10aa60 con 0x7ffaec102530 2024-06-03T03:23:16.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:16.019+0000 7ffae9600700 1 -- 172.21.15.117:0/2489180629 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7ffae0029070 con 0x7ffaec102530 2024-06-03T03:23:16.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:16.020+0000 7ffae9600700 1 --2- 172.21.15.117:0/2489180629 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7ffad40a2c20 0x7ffad40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:16.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:16.020+0000 7ffae9600700 1 -- 172.21.15.117:0/2489180629 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7ffae00d05e0 con 0x7ffaec102530 2024-06-03T03:23:16.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:16.020+0000 7ffaebe00700 1 --2- 172.21.15.117:0/2489180629 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7ffad40a2c20 0x7ffad40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:16.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:16.021+0000 7ffaebe00700 1 --2- 172.21.15.117:0/2489180629 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7ffad40a2c20 0x7ffad40a50e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ffaec079cb0 tx=0x7ffad8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:23:16.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:16.023+0000 7ffae9600700 1 -- 172.21.15.117:0/2489180629 <== mon.2 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ffae0013070 con 0x7ffaec102530 2024-06-03T03:23:16.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:23:15 smithi183 ceph-mon[37326]: pgmap v444: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-06-03T03:23:16.242 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:16.241+0000 7ffaf3c00700 1 -- 172.21.15.117:0/2489180629 --> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ffaec199f40 con 0x7ffad40a2c20 2024-06-03T03:23:16.242 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:23:15 smithi117 ceph-mon[31649]: pgmap v444: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-06-03T03:23:16.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:16.273+0000 7ffae9600700 1 -- 172.21.15.117:0/2489180629 <== mgr.14152 v2:172.21.15.117:6800/1398160887 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340822 (secure 0 0 0) 0x7ffaec199f40 con 0x7ffad40a2c20 2024-06-03T03:23:16.278 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T03:23:16.283 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:16.282+0000 7ffaf3c00700 1 -- 172.21.15.117:0/2489180629 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7ffad40a2c20 msgr2=0x7ffad40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:16.283 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:16.282+0000 7ffaf3c00700 1 --2- 172.21.15.117:0/2489180629 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7ffad40a2c20 0x7ffad40a50e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ffaec079cb0 tx=0x7ffad8009260 comp rx=0 tx=0).stop 2024-06-03T03:23:16.283 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:16.282+0000 7ffaf3c00700 1 -- 172.21.15.117:0/2489180629 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffaec102530 msgr2=0x7ffaec19e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:16.283 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:16.282+0000 7ffaf3c00700 1 --2- 172.21.15.117:0/2489180629 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffaec102530 0x7ffaec19e850 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ffae00095b0 tx=0x7ffae0005590 comp rx=0 tx=0).stop 2024-06-03T03:23:16.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:16.282+0000 7ffaf3c00700 1 -- 172.21.15.117:0/2489180629 shutdown_connections 2024-06-03T03:23:16.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:16.282+0000 7ffaf3c00700 1 --2- 172.21.15.117:0/2489180629 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7ffad40a2c20 0x7ffad40a50e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:16.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:16.282+0000 7ffaf3c00700 1 --2- 172.21.15.117:0/2489180629 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffaec1085d0 0x7ffaec198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:16.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:16.282+0000 7ffaf3c00700 1 --2- 172.21.15.117:0/2489180629 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffaec102530 0x7ffaec19e850 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:16.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:16.282+0000 7ffaf3c00700 1 --2- 172.21.15.117:0/2489180629 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffaec102ef0 0x7ffaec19ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:16.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:16.282+0000 7ffaf3c00700 1 -- 172.21.15.117:0/2489180629 >> 172.21.15.117:0/2489180629 conn(0x7ffaec079640 msgr2=0x7ffaec101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:23:16.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:16.282+0000 7ffaf3c00700 1 -- 172.21.15.117:0/2489180629 shutdown_connections 2024-06-03T03:23:16.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:16.282+0000 7ffaf3c00700 1 -- 172.21.15.117:0/2489180629 wait complete. 2024-06-03T03:23:16.284 INFO:teuthology.orchestra.run.smithi117.stderr:dumped all 2024-06-03T03:23:16.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:23:15 smithi151 ceph-mon[37043]: pgmap v444: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-06-03T03:23:17.171 INFO:teuthology.orchestra.run.smithi117.stdout:{"pg_ready":true,"pg_map":{"version":444,"stamp":"2024-06-03T03:23:14.897883+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":666759,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1416,"num_read_kb":835,"num_write":447,"num_write_kb":3095,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":865,"ondisk_log_size":865,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":370932,"kb_used_data":13460,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124570892,"statfs":{"total":1151940427776,"available":1151560593408,"internally_reserved":0,"allocated":13783040,"data_stored":8813387,"data_compressed":501066,"data_compressed_allocated":688128,"data_compressed_original":2875392,"omap_allocated":19142,"internal_metadata":365933882},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":109,"num_read_kb":54,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005757"},"pg_stats":[{"pgid":"3.3f","version":"99'8","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.832729+0000","last_change":"2024-06-03T03:19:41.626030+0000","last_active":"2024-06-03T03:22:54.832729+0000","last_peered":"2024-06-03T03:22:54.832729+0000","last_clean":"2024-06-03T03:22:54.832729+0000","last_became_active":"2024-06-03T03:19:41.625681+0000","last_became_peered":"2024-06-03T03:19:41.625681+0000","last_unstale":"2024-06-03T03:22:54.832729+0000","last_undegraded":"2024-06-03T03:22:54.832729+0000","last_fullsized":"2024-06-03T03:22:54.832729+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:35:46.396360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"99'4","reported_seq":42,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.830161+0000","last_change":"2024-06-03T03:19:41.619668+0000","last_active":"2024-06-03T03:22:54.830161+0000","last_peered":"2024-06-03T03:22:54.830161+0000","last_clean":"2024-06-03T03:22:54.830161+0000","last_became_active":"2024-06-03T03:19:41.619318+0000","last_became_peered":"2024-06-03T03:19:41.619318+0000","last_unstale":"2024-06-03T03:22:54.830161+0000","last_undegraded":"2024-06-03T03:22:54.830161+0000","last_fullsized":"2024-06-03T03:22:54.830161+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:07:21.656045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"99'7","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.832604+0000","last_change":"2024-06-03T03:19:41.623689+0000","last_active":"2024-06-03T03:22:54.832604+0000","last_peered":"2024-06-03T03:22:54.832604+0000","last_clean":"2024-06-03T03:22:54.832604+0000","last_became_active":"2024-06-03T03:19:41.623027+0000","last_became_peered":"2024-06-03T03:19:41.623027+0000","last_unstale":"2024-06-03T03:22:54.832604+0000","last_undegraded":"2024-06-03T03:22:54.832604+0000","last_fullsized":"2024-06-03T03:22:54.832604+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:18:05.584397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"99'8","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.814911+0000","last_change":"2024-06-03T03:19:41.603802+0000","last_active":"2024-06-03T03:22:54.814911+0000","last_peered":"2024-06-03T03:22:54.814911+0000","last_clean":"2024-06-03T03:22:54.814911+0000","last_became_active":"2024-06-03T03:19:41.603470+0000","last_became_peered":"2024-06-03T03:19:41.603470+0000","last_unstale":"2024-06-03T03:22:54.814911+0000","last_undegraded":"2024-06-03T03:22:54.814911+0000","last_fullsized":"2024-06-03T03:22:54.814911+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:56:10.636108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"99'4","reported_seq":42,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.826160+0000","last_change":"2024-06-03T03:19:41.636436+0000","last_active":"2024-06-03T03:22:54.826160+0000","last_peered":"2024-06-03T03:22:54.826160+0000","last_clean":"2024-06-03T03:22:54.826160+0000","last_became_active":"2024-06-03T03:19:41.636150+0000","last_became_peered":"2024-06-03T03:19:41.636150+0000","last_unstale":"2024-06-03T03:22:54.826160+0000","last_undegraded":"2024-06-03T03:22:54.826160+0000","last_fullsized":"2024-06-03T03:22:54.826160+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:02:10.928251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.808943+0000","last_change":"2024-06-03T03:19:41.622426+0000","last_active":"2024-06-03T03:22:54.808943+0000","last_peered":"2024-06-03T03:22:54.808943+0000","last_clean":"2024-06-03T03:22:54.808943+0000","last_became_active":"2024-06-03T03:19:41.622092+0000","last_became_peered":"2024-06-03T03:19:41.622092+0000","last_unstale":"2024-06-03T03:22:54.808943+0000","last_undegraded":"2024-06-03T03:22:54.808943+0000","last_fullsized":"2024-06-03T03:22:54.808943+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T14:08:52.666771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"99'5","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.825525+0000","last_change":"2024-06-03T03:19:41.608303+0000","last_active":"2024-06-03T03:22:54.825525+0000","last_peered":"2024-06-03T03:22:54.825525+0000","last_clean":"2024-06-03T03:22:54.825525+0000","last_became_active":"2024-06-03T03:19:41.607929+0000","last_became_peered":"2024-06-03T03:19:41.607929+0000","last_unstale":"2024-06-03T03:22:54.825525+0000","last_undegraded":"2024-06-03T03:22:54.825525+0000","last_fullsized":"2024-06-03T03:22:54.825525+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:40:05.643531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"99'9","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.822761+0000","last_change":"2024-06-03T03:19:41.606722+0000","last_active":"2024-06-03T03:22:54.822761+0000","last_peered":"2024-06-03T03:22:54.822761+0000","last_clean":"2024-06-03T03:22:54.822761+0000","last_became_active":"2024-06-03T03:19:41.606131+0000","last_became_peered":"2024-06-03T03:19:41.606131+0000","last_unstale":"2024-06-03T03:22:54.822761+0000","last_undegraded":"2024-06-03T03:22:54.822761+0000","last_fullsized":"2024-06-03T03:22:54.822761+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:09:17.306848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"99'11","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.813384+0000","last_change":"2024-06-03T03:19:41.609287+0000","last_active":"2024-06-03T03:22:54.813384+0000","last_peered":"2024-06-03T03:22:54.813384+0000","last_clean":"2024-06-03T03:22:54.813384+0000","last_became_active":"2024-06-03T03:19:41.608521+0000","last_became_peered":"2024-06-03T03:19:41.608521+0000","last_unstale":"2024-06-03T03:22:54.813384+0000","last_undegraded":"2024-06-03T03:22:54.813384+0000","last_fullsized":"2024-06-03T03:22:54.813384+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:35:37.128667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"99'16","reported_seq":55,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.814582+0000","last_change":"2024-06-03T03:19:41.605193+0000","last_active":"2024-06-03T03:22:54.814582+0000","last_peered":"2024-06-03T03:22:54.814582+0000","last_clean":"2024-06-03T03:22:54.814582+0000","last_became_active":"2024-06-03T03:19:41.604787+0000","last_became_peered":"2024-06-03T03:19:41.604787+0000","last_unstale":"2024-06-03T03:22:54.814582+0000","last_undegraded":"2024-06-03T03:22:54.814582+0000","last_fullsized":"2024-06-03T03:22:54.814582+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:02:48.334599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.817541+0000","last_change":"2024-06-03T03:19:41.611522+0000","last_active":"2024-06-03T03:22:54.817541+0000","last_peered":"2024-06-03T03:22:54.817541+0000","last_clean":"2024-06-03T03:22:54.817541+0000","last_became_active":"2024-06-03T03:19:41.610932+0000","last_became_peered":"2024-06-03T03:19:41.610932+0000","last_unstale":"2024-06-03T03:22:54.817541+0000","last_undegraded":"2024-06-03T03:22:54.817541+0000","last_fullsized":"2024-06-03T03:22:54.817541+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:35:44.702791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"99'7","reported_seq":23,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.815400+0000","last_change":"2024-06-03T03:20:25.950663+0000","last_active":"2024-06-03T03:22:54.815400+0000","last_peered":"2024-06-03T03:22:54.815400+0000","last_clean":"2024-06-03T03:22:54.815400+0000","last_became_active":"2024-06-03T03:20:25.949962+0000","last_became_peered":"2024-06-03T03:20:25.949962+0000","last_unstale":"2024-06-03T03:22:54.815400+0000","last_undegraded":"2024-06-03T03:22:54.815400+0000","last_fullsized":"2024-06-03T03:22:54.815400+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:55:33.972567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"99'10","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.839057+0000","last_change":"2024-06-03T03:19:41.633464+0000","last_active":"2024-06-03T03:22:54.839057+0000","last_peered":"2024-06-03T03:22:54.839057+0000","last_clean":"2024-06-03T03:22:54.839057+0000","last_became_active":"2024-06-03T03:19:41.630625+0000","last_became_peered":"2024-06-03T03:19:41.630625+0000","last_unstale":"2024-06-03T03:22:54.839057+0000","last_undegraded":"2024-06-03T03:22:54.839057+0000","last_fullsized":"2024-06-03T03:22:54.839057+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:23:28.799058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"99'7","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.831827+0000","last_change":"2024-06-03T03:19:41.627234+0000","last_active":"2024-06-03T03:22:54.831827+0000","last_peered":"2024-06-03T03:22:54.831827+0000","last_clean":"2024-06-03T03:22:54.831827+0000","last_became_active":"2024-06-03T03:19:41.626643+0000","last_became_peered":"2024-06-03T03:19:41.626643+0000","last_unstale":"2024-06-03T03:22:54.831827+0000","last_undegraded":"2024-06-03T03:22:54.831827+0000","last_fullsized":"2024-06-03T03:22:54.831827+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:37:44.126101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"99'7","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.831738+0000","last_change":"2024-06-03T03:19:41.621833+0000","last_active":"2024-06-03T03:22:54.831738+0000","last_peered":"2024-06-03T03:22:54.831738+0000","last_clean":"2024-06-03T03:22:54.831738+0000","last_became_active":"2024-06-03T03:19:41.620689+0000","last_became_peered":"2024-06-03T03:19:41.620689+0000","last_unstale":"2024-06-03T03:22:54.831738+0000","last_undegraded":"2024-06-03T03:22:54.831738+0000","last_fullsized":"2024-06-03T03:22:54.831738+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:39:31.188372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.830605+0000","last_change":"2024-06-03T03:19:41.624072+0000","last_active":"2024-06-03T03:22:54.830605+0000","last_peered":"2024-06-03T03:22:54.830605+0000","last_clean":"2024-06-03T03:22:54.830605+0000","last_became_active":"2024-06-03T03:19:41.623689+0000","last_became_peered":"2024-06-03T03:19:41.623689+0000","last_unstale":"2024-06-03T03:22:54.830605+0000","last_undegraded":"2024-06-03T03:22:54.830605+0000","last_fullsized":"2024-06-03T03:22:54.830605+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:38:24.075256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"99'8","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.832972+0000","last_change":"2024-06-03T03:19:41.626145+0000","last_active":"2024-06-03T03:22:54.832972+0000","last_peered":"2024-06-03T03:22:54.832972+0000","last_clean":"2024-06-03T03:22:54.832972+0000","last_became_active":"2024-06-03T03:19:41.625826+0000","last_became_peered":"2024-06-03T03:19:41.625826+0000","last_unstale":"2024-06-03T03:22:54.832972+0000","last_undegraded":"2024-06-03T03:22:54.832972+0000","last_fullsized":"2024-06-03T03:22:54.832972+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:04:10.996134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"99'10","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.820262+0000","last_change":"2024-06-03T03:19:41.615754+0000","last_active":"2024-06-03T03:22:54.820262+0000","last_peered":"2024-06-03T03:22:54.820262+0000","last_clean":"2024-06-03T03:22:54.820262+0000","last_became_active":"2024-06-03T03:19:41.614577+0000","last_became_peered":"2024-06-03T03:19:41.614577+0000","last_unstale":"2024-06-03T03:22:54.820262+0000","last_undegraded":"2024-06-03T03:22:54.820262+0000","last_fullsized":"2024-06-03T03:22:54.820262+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:45:50.084332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"99'7","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.826210+0000","last_change":"2024-06-03T03:19:41.633000+0000","last_active":"2024-06-03T03:22:54.826210+0000","last_peered":"2024-06-03T03:22:54.826210+0000","last_clean":"2024-06-03T03:22:54.826210+0000","last_became_active":"2024-06-03T03:19:41.632731+0000","last_became_peered":"2024-06-03T03:19:41.632731+0000","last_unstale":"2024-06-03T03:22:54.826210+0000","last_undegraded":"2024-06-03T03:22:54.826210+0000","last_fullsized":"2024-06-03T03:22:54.826210+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:10:29.990256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":36,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.052465+0000","last_change":"2024-06-03T03:20:26.133763+0000","last_active":"2024-06-03T03:21:26.052465+0000","last_peered":"2024-06-03T03:21:26.052465+0000","last_clean":"2024-06-03T03:21:26.052465+0000","last_became_active":"2024-06-03T03:20:26.133252+0000","last_became_peered":"2024-06-03T03:20:26.133252+0000","last_unstale":"2024-06-03T03:21:26.052465+0000","last_undegraded":"2024-06-03T03:21:26.052465+0000","last_fullsized":"2024-06-03T03:21:26.052465+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:33:21.461038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"97'13","reported_seq":182,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:23:09.285203+0000","last_change":"2024-06-03T03:19:36.542545+0000","last_active":"2024-06-03T03:23:09.285203+0000","last_peered":"2024-06-03T03:23:09.285203+0000","last_clean":"2024-06-03T03:23:09.285203+0000","last_became_active":"2024-06-03T03:19:36.541682+0000","last_became_peered":"2024-06-03T03:19:36.541682+0000","last_unstale":"2024-06-03T03:23:09.285203+0000","last_undegraded":"2024-06-03T03:23:09.285203+0000","last_fullsized":"2024-06-03T03:23:09.285203+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:06:23.909253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":124,"num_read_kb":124,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"99'5","reported_seq":43,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.815844+0000","last_change":"2024-06-03T03:19:41.605848+0000","last_active":"2024-06-03T03:22:54.815844+0000","last_peered":"2024-06-03T03:22:54.815844+0000","last_clean":"2024-06-03T03:22:54.815844+0000","last_became_active":"2024-06-03T03:19:41.603131+0000","last_became_peered":"2024-06-03T03:19:41.603131+0000","last_unstale":"2024-06-03T03:22:54.815844+0000","last_undegraded":"2024-06-03T03:22:54.815844+0000","last_fullsized":"2024-06-03T03:22:54.815844+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:07:38.931055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.980786+0000","last_change":"2024-06-03T03:19:56.923180+0000","last_active":"2024-06-03T03:21:25.980786+0000","last_peered":"2024-06-03T03:21:25.980786+0000","last_clean":"2024-06-03T03:21:25.980786+0000","last_became_active":"2024-06-03T03:19:56.922726+0000","last_became_peered":"2024-06-03T03:19:56.922726+0000","last_unstale":"2024-06-03T03:21:25.980786+0000","last_undegraded":"2024-06-03T03:21:25.980786+0000","last_fullsized":"2024-06-03T03:21:25.980786+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:18:45.727680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"97'2","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.058990+0000","last_change":"2024-06-03T03:19:36.746805+0000","last_active":"2024-06-03T03:21:26.058990+0000","last_peered":"2024-06-03T03:21:26.058990+0000","last_clean":"2024-06-03T03:21:26.058990+0000","last_became_active":"2024-06-03T03:19:36.745747+0000","last_became_peered":"2024-06-03T03:19:36.745747+0000","last_unstale":"2024-06-03T03:21:26.058990+0000","last_undegraded":"2024-06-03T03:21:26.058990+0000","last_fullsized":"2024-06-03T03:21:26.058990+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:54:28.427708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"99'6","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.832686+0000","last_change":"2024-06-03T03:19:41.630582+0000","last_active":"2024-06-03T03:22:54.832686+0000","last_peered":"2024-06-03T03:22:54.832686+0000","last_clean":"2024-06-03T03:22:54.832686+0000","last_became_active":"2024-06-03T03:19:41.630238+0000","last_became_peered":"2024-06-03T03:19:41.630238+0000","last_unstale":"2024-06-03T03:22:54.832686+0000","last_undegraded":"2024-06-03T03:22:54.832686+0000","last_fullsized":"2024-06-03T03:22:54.832686+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:49:58.039697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.980892+0000","last_change":"2024-06-03T03:19:56.925142+0000","last_active":"2024-06-03T03:21:25.980892+0000","last_peered":"2024-06-03T03:21:25.980892+0000","last_clean":"2024-06-03T03:21:25.980892+0000","last_became_active":"2024-06-03T03:19:56.924760+0000","last_became_peered":"2024-06-03T03:19:56.924760+0000","last_unstale":"2024-06-03T03:21:25.980892+0000","last_undegraded":"2024-06-03T03:21:25.980892+0000","last_fullsized":"2024-06-03T03:21:25.980892+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:11:27.537077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"97'12","reported_seq":57,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.974082+0000","last_change":"2024-06-03T03:19:36.543344+0000","last_active":"2024-06-03T03:21:25.974082+0000","last_peered":"2024-06-03T03:21:25.974082+0000","last_clean":"2024-06-03T03:21:25.974082+0000","last_became_active":"2024-06-03T03:19:36.542837+0000","last_became_peered":"2024-06-03T03:19:36.542837+0000","last_unstale":"2024-06-03T03:21:25.974082+0000","last_undegraded":"2024-06-03T03:21:25.974082+0000","last_fullsized":"2024-06-03T03:21:25.974082+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:55:05.451521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.812572+0000","last_change":"2024-06-03T03:19:41.629820+0000","last_active":"2024-06-03T03:22:54.812572+0000","last_peered":"2024-06-03T03:22:54.812572+0000","last_clean":"2024-06-03T03:22:54.812572+0000","last_became_active":"2024-06-03T03:19:41.629439+0000","last_became_peered":"2024-06-03T03:19:41.629439+0000","last_unstale":"2024-06-03T03:22:54.812572+0000","last_undegraded":"2024-06-03T03:22:54.812572+0000","last_fullsized":"2024-06-03T03:22:54.812572+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:05:27.438308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.980800+0000","last_change":"2024-06-03T03:19:56.923214+0000","last_active":"2024-06-03T03:21:25.980800+0000","last_peered":"2024-06-03T03:21:25.980800+0000","last_clean":"2024-06-03T03:21:25.980800+0000","last_became_active":"2024-06-03T03:19:56.922751+0000","last_became_peered":"2024-06-03T03:19:56.922751+0000","last_unstale":"2024-06-03T03:21:25.980800+0000","last_undegraded":"2024-06-03T03:21:25.980800+0000","last_fullsized":"2024-06-03T03:21:25.980800+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:05:59.249818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"97'5","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.973910+0000","last_change":"2024-06-03T03:19:36.538141+0000","last_active":"2024-06-03T03:21:25.973910+0000","last_peered":"2024-06-03T03:21:25.973910+0000","last_clean":"2024-06-03T03:21:25.973910+0000","last_became_active":"2024-06-03T03:19:36.537786+0000","last_became_peered":"2024-06-03T03:19:36.537786+0000","last_unstale":"2024-06-03T03:21:25.973910+0000","last_undegraded":"2024-06-03T03:21:25.973910+0000","last_fullsized":"2024-06-03T03:21:25.973910+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T15:03:32.859097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"99'15","reported_seq":54,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.816371+0000","last_change":"2024-06-03T03:19:41.608471+0000","last_active":"2024-06-03T03:22:54.816371+0000","last_peered":"2024-06-03T03:22:54.816371+0000","last_clean":"2024-06-03T03:22:54.816371+0000","last_became_active":"2024-06-03T03:19:41.608096+0000","last_became_peered":"2024-06-03T03:19:41.608096+0000","last_unstale":"2024-06-03T03:22:54.816371+0000","last_undegraded":"2024-06-03T03:22:54.816371+0000","last_fullsized":"2024-06-03T03:22:54.816371+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:31:10.541803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.053026+0000","last_change":"2024-06-03T03:19:56.915540+0000","last_active":"2024-06-03T03:21:26.053026+0000","last_peered":"2024-06-03T03:21:26.053026+0000","last_clean":"2024-06-03T03:21:26.053026+0000","last_became_active":"2024-06-03T03:19:56.915017+0000","last_became_peered":"2024-06-03T03:19:56.915017+0000","last_unstale":"2024-06-03T03:21:26.053026+0000","last_undegraded":"2024-06-03T03:21:26.053026+0000","last_fullsized":"2024-06-03T03:21:26.053026+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:56:23.158344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"99'7","reported_seq":181,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:23:11.312646+0000","last_change":"2024-06-03T03:19:36.542337+0000","last_active":"2024-06-03T03:23:11.312646+0000","last_peered":"2024-06-03T03:23:11.312646+0000","last_clean":"2024-06-03T03:23:11.312646+0000","last_became_active":"2024-06-03T03:19:36.541990+0000","last_became_peered":"2024-06-03T03:19:36.541990+0000","last_unstale":"2024-06-03T03:23:11.312646+0000","last_undegraded":"2024-06-03T03:23:11.312646+0000","last_fullsized":"2024-06-03T03:23:11.312646+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:08:15.656321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1153,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":129,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"99'7","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.815200+0000","last_change":"2024-06-03T03:19:41.616293+0000","last_active":"2024-06-03T03:22:54.815200+0000","last_peered":"2024-06-03T03:22:54.815200+0000","last_clean":"2024-06-03T03:22:54.815200+0000","last_became_active":"2024-06-03T03:19:41.615946+0000","last_became_peered":"2024-06-03T03:19:41.615946+0000","last_unstale":"2024-06-03T03:22:54.815200+0000","last_undegraded":"2024-06-03T03:22:54.815200+0000","last_fullsized":"2024-06-03T03:22:54.815200+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:18:43.835977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":36,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.053631+0000","last_change":"2024-06-03T03:20:26.146237+0000","last_active":"2024-06-03T03:21:26.053631+0000","last_peered":"2024-06-03T03:21:26.053631+0000","last_clean":"2024-06-03T03:21:26.053631+0000","last_became_active":"2024-06-03T03:20:26.145731+0000","last_became_peered":"2024-06-03T03:20:26.145731+0000","last_unstale":"2024-06-03T03:21:26.053631+0000","last_undegraded":"2024-06-03T03:21:26.053631+0000","last_fullsized":"2024-06-03T03:21:26.053631+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:15:29.723833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"97'7","reported_seq":52,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.053550+0000","last_change":"2024-06-03T03:19:36.546501+0000","last_active":"2024-06-03T03:21:26.053550+0000","last_peered":"2024-06-03T03:21:26.053550+0000","last_clean":"2024-06-03T03:21:26.053550+0000","last_became_active":"2024-06-03T03:19:36.546079+0000","last_became_peered":"2024-06-03T03:19:36.546079+0000","last_unstale":"2024-06-03T03:21:26.053550+0000","last_undegraded":"2024-06-03T03:21:26.053550+0000","last_fullsized":"2024-06-03T03:21:26.053550+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:16:39.473141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"99'8","reported_seq":53,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.829062+0000","last_change":"2024-06-03T03:20:26.149410+0000","last_active":"2024-06-03T03:22:54.829062+0000","last_peered":"2024-06-03T03:22:54.829062+0000","last_clean":"2024-06-03T03:22:54.829062+0000","last_became_active":"2024-06-03T03:20:26.149084+0000","last_became_peered":"2024-06-03T03:20:26.149084+0000","last_unstale":"2024-06-03T03:22:54.829062+0000","last_undegraded":"2024-06-03T03:22:54.829062+0000","last_fullsized":"2024-06-03T03:22:54.829062+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:06:22.316449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.054192+0000","last_change":"2024-06-03T03:19:56.926498+0000","last_active":"2024-06-03T03:21:26.054192+0000","last_peered":"2024-06-03T03:21:26.054192+0000","last_clean":"2024-06-03T03:21:26.054192+0000","last_became_active":"2024-06-03T03:19:56.926101+0000","last_became_peered":"2024-06-03T03:19:56.926101+0000","last_unstale":"2024-06-03T03:21:26.054192+0000","last_undegraded":"2024-06-03T03:21:26.054192+0000","last_fullsized":"2024-06-03T03:21:26.054192+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:00:12.024443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"97'3","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.977399+0000","last_change":"2024-06-03T03:19:36.553151+0000","last_active":"2024-06-03T03:21:25.977399+0000","last_peered":"2024-06-03T03:21:25.977399+0000","last_clean":"2024-06-03T03:21:25.977399+0000","last_became_active":"2024-06-03T03:19:36.551914+0000","last_became_peered":"2024-06-03T03:19:36.551914+0000","last_unstale":"2024-06-03T03:21:25.977399+0000","last_undegraded":"2024-06-03T03:21:25.977399+0000","last_fullsized":"2024-06-03T03:21:25.977399+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:00:42.053922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"99'8","reported_seq":24,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.824233+0000","last_change":"2024-06-03T03:20:26.147909+0000","last_active":"2024-06-03T03:22:54.824233+0000","last_peered":"2024-06-03T03:22:54.824233+0000","last_clean":"2024-06-03T03:22:54.824233+0000","last_became_active":"2024-06-03T03:20:26.146448+0000","last_became_peered":"2024-06-03T03:20:26.146448+0000","last_unstale":"2024-06-03T03:22:54.824233+0000","last_undegraded":"2024-06-03T03:22:54.824233+0000","last_fullsized":"2024-06-03T03:22:54.824233+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:43:30.878472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.976693+0000","last_change":"2024-06-03T03:19:56.941910+0000","last_active":"2024-06-03T03:21:25.976693+0000","last_peered":"2024-06-03T03:21:25.976693+0000","last_clean":"2024-06-03T03:21:25.976693+0000","last_became_active":"2024-06-03T03:19:56.940963+0000","last_became_peered":"2024-06-03T03:19:56.940963+0000","last_unstale":"2024-06-03T03:21:25.976693+0000","last_undegraded":"2024-06-03T03:21:25.976693+0000","last_fullsized":"2024-06-03T03:21:25.976693+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T14:18:32.109231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"97'3","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.052977+0000","last_change":"2024-06-03T03:19:36.538928+0000","last_active":"2024-06-03T03:21:26.052977+0000","last_peered":"2024-06-03T03:21:26.052977+0000","last_clean":"2024-06-03T03:21:26.052977+0000","last_became_active":"2024-06-03T03:19:36.538580+0000","last_became_peered":"2024-06-03T03:19:36.538580+0000","last_unstale":"2024-06-03T03:21:26.052977+0000","last_undegraded":"2024-06-03T03:21:26.052977+0000","last_fullsized":"2024-06-03T03:21:26.052977+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:08:05.906257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"99'11","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.829145+0000","last_change":"2024-06-03T03:19:41.623051+0000","last_active":"2024-06-03T03:22:54.829145+0000","last_peered":"2024-06-03T03:22:54.829145+0000","last_clean":"2024-06-03T03:22:54.829145+0000","last_became_active":"2024-06-03T03:19:41.622656+0000","last_became_peered":"2024-06-03T03:19:41.622656+0000","last_unstale":"2024-06-03T03:22:54.829145+0000","last_undegraded":"2024-06-03T03:22:54.829145+0000","last_fullsized":"2024-06-03T03:22:54.829145+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:11:34.587723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.056378+0000","last_change":"2024-06-03T03:19:56.921215+0000","last_active":"2024-06-03T03:21:26.056378+0000","last_peered":"2024-06-03T03:21:26.056378+0000","last_clean":"2024-06-03T03:21:26.056378+0000","last_became_active":"2024-06-03T03:19:56.920658+0000","last_became_peered":"2024-06-03T03:19:56.920658+0000","last_unstale":"2024-06-03T03:21:26.056378+0000","last_undegraded":"2024-06-03T03:21:26.056378+0000","last_fullsized":"2024-06-03T03:21:26.056378+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T14:04:54.163818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"99'5","reported_seq":298,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:23:11.237190+0000","last_change":"2024-06-03T03:19:36.545245+0000","last_active":"2024-06-03T03:23:11.237190+0000","last_peered":"2024-06-03T03:23:11.237190+0000","last_clean":"2024-06-03T03:23:11.237190+0000","last_became_active":"2024-06-03T03:19:36.544821+0000","last_became_peered":"2024-06-03T03:19:36.544821+0000","last_unstale":"2024-06-03T03:23:11.237190+0000","last_undegraded":"2024-06-03T03:23:11.237190+0000","last_fullsized":"2024-06-03T03:23:11.237190+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:55:24.011925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":248,"num_read_kb":248,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"99'10","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.822416+0000","last_change":"2024-06-03T03:19:41.616384+0000","last_active":"2024-06-03T03:22:54.822416+0000","last_peered":"2024-06-03T03:22:54.822416+0000","last_clean":"2024-06-03T03:22:54.822416+0000","last_became_active":"2024-06-03T03:19:41.616034+0000","last_became_peered":"2024-06-03T03:19:41.616034+0000","last_unstale":"2024-06-03T03:22:54.822416+0000","last_undegraded":"2024-06-03T03:22:54.822416+0000","last_fullsized":"2024-06-03T03:22:54.822416+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:10:06.823375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.054539+0000","last_change":"2024-06-03T03:19:56.933045+0000","last_active":"2024-06-03T03:21:26.054539+0000","last_peered":"2024-06-03T03:21:26.054539+0000","last_clean":"2024-06-03T03:21:26.054539+0000","last_became_active":"2024-06-03T03:19:56.928956+0000","last_became_peered":"2024-06-03T03:19:56.928956+0000","last_unstale":"2024-06-03T03:21:26.054539+0000","last_undegraded":"2024-06-03T03:21:26.054539+0000","last_fullsized":"2024-06-03T03:21:26.054539+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:37:44.270511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"97'4","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.053400+0000","last_change":"2024-06-03T03:19:36.541559+0000","last_active":"2024-06-03T03:21:26.053400+0000","last_peered":"2024-06-03T03:21:26.053400+0000","last_clean":"2024-06-03T03:21:26.053400+0000","last_became_active":"2024-06-03T03:19:36.541166+0000","last_became_peered":"2024-06-03T03:19:36.541166+0000","last_unstale":"2024-06-03T03:21:26.053400+0000","last_undegraded":"2024-06-03T03:21:26.053400+0000","last_fullsized":"2024-06-03T03:21:26.053400+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T14:49:26.847713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"99'9","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.832063+0000","last_change":"2024-06-03T03:19:41.629789+0000","last_active":"2024-06-03T03:22:54.832063+0000","last_peered":"2024-06-03T03:22:54.832063+0000","last_clean":"2024-06-03T03:22:54.832063+0000","last_became_active":"2024-06-03T03:19:41.628026+0000","last_became_peered":"2024-06-03T03:19:41.628026+0000","last_unstale":"2024-06-03T03:22:54.832063+0000","last_undegraded":"2024-06-03T03:22:54.832063+0000","last_fullsized":"2024-06-03T03:22:54.832063+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:56:26.411907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":36,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.056621+0000","last_change":"2024-06-03T03:21:26.056114+0000","last_active":"2024-06-03T03:21:26.056621+0000","last_peered":"2024-06-03T03:21:26.056621+0000","last_clean":"2024-06-03T03:21:26.056621+0000","last_became_active":"2024-06-03T03:21:26.055505+0000","last_became_peered":"2024-06-03T03:21:26.055505+0000","last_unstale":"2024-06-03T03:21:26.056621+0000","last_undegraded":"2024-06-03T03:21:26.056621+0000","last_fullsized":"2024-06-03T03:21:26.056621+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:22:55.722566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,10],"acting":[4,1,2,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"97'5","reported_seq":51,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.053544+0000","last_change":"2024-06-03T03:19:36.744930+0000","last_active":"2024-06-03T03:21:26.053544+0000","last_peered":"2024-06-03T03:21:26.053544+0000","last_clean":"2024-06-03T03:21:26.053544+0000","last_became_active":"2024-06-03T03:19:36.744567+0000","last_became_peered":"2024-06-03T03:19:36.744567+0000","last_unstale":"2024-06-03T03:21:26.053544+0000","last_undegraded":"2024-06-03T03:21:26.053544+0000","last_fullsized":"2024-06-03T03:21:26.053544+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:35:31.927211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"99'11","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.812434+0000","last_change":"2024-06-03T03:19:41.607604+0000","last_active":"2024-06-03T03:22:54.812434+0000","last_peered":"2024-06-03T03:22:54.812434+0000","last_clean":"2024-06-03T03:22:54.812434+0000","last_became_active":"2024-06-03T03:19:41.607148+0000","last_became_peered":"2024-06-03T03:19:41.607148+0000","last_unstale":"2024-06-03T03:22:54.812434+0000","last_undegraded":"2024-06-03T03:22:54.812434+0000","last_fullsized":"2024-06-03T03:22:54.812434+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:44:44.037836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.056250+0000","last_change":"2024-06-03T03:19:56.928689+0000","last_active":"2024-06-03T03:21:26.056250+0000","last_peered":"2024-06-03T03:21:26.056250+0000","last_clean":"2024-06-03T03:21:26.056250+0000","last_became_active":"2024-06-03T03:19:56.924955+0000","last_became_peered":"2024-06-03T03:19:56.924955+0000","last_unstale":"2024-06-03T03:21:26.056250+0000","last_undegraded":"2024-06-03T03:21:26.056250+0000","last_fullsized":"2024-06-03T03:21:26.056250+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:11:07.413430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"99'42","reported_seq":347,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:23:11.237178+0000","last_change":"2024-06-03T03:19:36.539688+0000","last_active":"2024-06-03T03:23:11.237178+0000","last_peered":"2024-06-03T03:23:11.237178+0000","last_clean":"2024-06-03T03:23:11.237178+0000","last_became_active":"2024-06-03T03:19:36.539302+0000","last_became_peered":"2024-06-03T03:19:36.539302+0000","last_unstale":"2024-06-03T03:23:11.237178+0000","last_undegraded":"2024-06-03T03:23:11.237178+0000","last_fullsized":"2024-06-03T03:23:11.237178+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:28:52.790227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69204,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":260,"num_read_kb":198,"num_write":44,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"99'6","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.815310+0000","last_change":"2024-06-03T03:19:41.608202+0000","last_active":"2024-06-03T03:22:54.815310+0000","last_peered":"2024-06-03T03:22:54.815310+0000","last_clean":"2024-06-03T03:22:54.815310+0000","last_became_active":"2024-06-03T03:19:41.607891+0000","last_became_peered":"2024-06-03T03:19:41.607891+0000","last_unstale":"2024-06-03T03:22:54.815310+0000","last_undegraded":"2024-06-03T03:22:54.815310+0000","last_fullsized":"2024-06-03T03:22:54.815310+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T14:10:44.042892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.054202+0000","last_change":"2024-06-03T03:19:56.923547+0000","last_active":"2024-06-03T03:21:26.054202+0000","last_peered":"2024-06-03T03:21:26.054202+0000","last_clean":"2024-06-03T03:21:26.054202+0000","last_became_active":"2024-06-03T03:19:56.923046+0000","last_became_peered":"2024-06-03T03:19:56.923046+0000","last_unstale":"2024-06-03T03:21:26.054202+0000","last_undegraded":"2024-06-03T03:21:26.054202+0000","last_fullsized":"2024-06-03T03:21:26.054202+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:32:19.438793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"97'9","reported_seq":55,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.053870+0000","last_change":"2024-06-03T03:19:36.547024+0000","last_active":"2024-06-03T03:21:26.053870+0000","last_peered":"2024-06-03T03:21:26.053870+0000","last_clean":"2024-06-03T03:21:26.053870+0000","last_became_active":"2024-06-03T03:19:36.546663+0000","last_became_peered":"2024-06-03T03:19:36.546663+0000","last_unstale":"2024-06-03T03:21:26.053870+0000","last_undegraded":"2024-06-03T03:21:26.053870+0000","last_fullsized":"2024-06-03T03:21:26.053870+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:58:41.446730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"99'9","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.830440+0000","last_change":"2024-06-03T03:19:41.630119+0000","last_active":"2024-06-03T03:22:54.830440+0000","last_peered":"2024-06-03T03:22:54.830440+0000","last_clean":"2024-06-03T03:22:54.830440+0000","last_became_active":"2024-06-03T03:19:41.629563+0000","last_became_peered":"2024-06-03T03:19:41.629563+0000","last_unstale":"2024-06-03T03:22:54.830440+0000","last_undegraded":"2024-06-03T03:22:54.830440+0000","last_fullsized":"2024-06-03T03:22:54.830440+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:49:33.983471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.053797+0000","last_change":"2024-06-03T03:19:56.941451+0000","last_active":"2024-06-03T03:21:26.053797+0000","last_peered":"2024-06-03T03:21:26.053797+0000","last_clean":"2024-06-03T03:21:26.053797+0000","last_became_active":"2024-06-03T03:19:56.941003+0000","last_became_peered":"2024-06-03T03:19:56.941003+0000","last_unstale":"2024-06-03T03:21:26.053797+0000","last_undegraded":"2024-06-03T03:21:26.053797+0000","last_fullsized":"2024-06-03T03:21:26.053797+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:28:58.505495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"97'5","reported_seq":51,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.053452+0000","last_change":"2024-06-03T03:19:36.547128+0000","last_active":"2024-06-03T03:21:26.053452+0000","last_peered":"2024-06-03T03:21:26.053452+0000","last_clean":"2024-06-03T03:21:26.053452+0000","last_became_active":"2024-06-03T03:19:36.546576+0000","last_became_peered":"2024-06-03T03:19:36.546576+0000","last_unstale":"2024-06-03T03:21:26.053452+0000","last_undegraded":"2024-06-03T03:21:26.053452+0000","last_fullsized":"2024-06-03T03:21:26.053452+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:23:30.196543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"99'15","reported_seq":54,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.836443+0000","last_change":"2024-06-03T03:19:41.634941+0000","last_active":"2024-06-03T03:22:54.836443+0000","last_peered":"2024-06-03T03:22:54.836443+0000","last_clean":"2024-06-03T03:22:54.836443+0000","last_became_active":"2024-06-03T03:19:41.634561+0000","last_became_peered":"2024-06-03T03:19:41.634561+0000","last_unstale":"2024-06-03T03:22:54.836443+0000","last_undegraded":"2024-06-03T03:22:54.836443+0000","last_fullsized":"2024-06-03T03:22:54.836443+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:00:27.258078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.056748+0000","last_change":"2024-06-03T03:19:56.934612+0000","last_active":"2024-06-03T03:21:26.056748+0000","last_peered":"2024-06-03T03:21:26.056748+0000","last_clean":"2024-06-03T03:21:26.056748+0000","last_became_active":"2024-06-03T03:19:56.930212+0000","last_became_peered":"2024-06-03T03:19:56.930212+0000","last_unstale":"2024-06-03T03:21:26.056748+0000","last_undegraded":"2024-06-03T03:21:26.056748+0000","last_fullsized":"2024-06-03T03:21:26.056748+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:42:38.490095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"97'5","reported_seq":176,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:23:11.289638+0000","last_change":"2024-06-03T03:19:36.542052+0000","last_active":"2024-06-03T03:23:11.289638+0000","last_peered":"2024-06-03T03:23:11.289638+0000","last_clean":"2024-06-03T03:23:11.289638+0000","last_became_active":"2024-06-03T03:19:36.540987+0000","last_became_peered":"2024-06-03T03:19:36.540987+0000","last_unstale":"2024-06-03T03:23:11.289638+0000","last_undegraded":"2024-06-03T03:23:11.289638+0000","last_fullsized":"2024-06-03T03:23:11.289638+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T15:18:48.821400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":128,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.818221+0000","last_change":"2024-06-03T03:19:41.607610+0000","last_active":"2024-06-03T03:22:54.818221+0000","last_peered":"2024-06-03T03:22:54.818221+0000","last_clean":"2024-06-03T03:22:54.818221+0000","last_became_active":"2024-06-03T03:19:41.607219+0000","last_became_peered":"2024-06-03T03:19:41.607219+0000","last_unstale":"2024-06-03T03:22:54.818221+0000","last_undegraded":"2024-06-03T03:22:54.818221+0000","last_fullsized":"2024-06-03T03:22:54.818221+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:06:20.707623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.979965+0000","last_change":"2024-06-03T03:19:56.936440+0000","last_active":"2024-06-03T03:21:25.979965+0000","last_peered":"2024-06-03T03:21:25.979965+0000","last_clean":"2024-06-03T03:21:25.979965+0000","last_became_active":"2024-06-03T03:19:56.936048+0000","last_became_peered":"2024-06-03T03:19:56.936048+0000","last_unstale":"2024-06-03T03:21:25.979965+0000","last_undegraded":"2024-06-03T03:21:25.979965+0000","last_fullsized":"2024-06-03T03:21:25.979965+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:22:15.758950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"97'17","reported_seq":62,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.973400+0000","last_change":"2024-06-03T03:19:36.736105+0000","last_active":"2024-06-03T03:21:25.973400+0000","last_peered":"2024-06-03T03:21:25.973400+0000","last_clean":"2024-06-03T03:21:25.973400+0000","last_became_active":"2024-06-03T03:19:36.735663+0000","last_became_peered":"2024-06-03T03:19:36.735663+0000","last_unstale":"2024-06-03T03:21:25.973400+0000","last_undegraded":"2024-06-03T03:21:25.973400+0000","last_fullsized":"2024-06-03T03:21:25.973400+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:41:46.216923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.813317+0000","last_change":"2024-06-03T03:19:41.609758+0000","last_active":"2024-06-03T03:22:54.813317+0000","last_peered":"2024-06-03T03:22:54.813317+0000","last_clean":"2024-06-03T03:22:54.813317+0000","last_became_active":"2024-06-03T03:19:41.609421+0000","last_became_peered":"2024-06-03T03:19:41.609421+0000","last_unstale":"2024-06-03T03:22:54.813317+0000","last_undegraded":"2024-06-03T03:22:54.813317+0000","last_fullsized":"2024-06-03T03:22:54.813317+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:45:54.919352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.052841+0000","last_change":"2024-06-03T03:19:56.925422+0000","last_active":"2024-06-03T03:21:26.052841+0000","last_peered":"2024-06-03T03:21:26.052841+0000","last_clean":"2024-06-03T03:21:26.052841+0000","last_became_active":"2024-06-03T03:19:56.924891+0000","last_became_peered":"2024-06-03T03:19:56.924891+0000","last_unstale":"2024-06-03T03:21:26.052841+0000","last_undegraded":"2024-06-03T03:21:26.052841+0000","last_fullsized":"2024-06-03T03:21:26.052841+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:34:59.986256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"97'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.973924+0000","last_change":"2024-06-03T03:19:36.540605+0000","last_active":"2024-06-03T03:21:25.973924+0000","last_peered":"2024-06-03T03:21:25.973924+0000","last_clean":"2024-06-03T03:21:25.973924+0000","last_became_active":"2024-06-03T03:19:36.540180+0000","last_became_peered":"2024-06-03T03:19:36.540180+0000","last_unstale":"2024-06-03T03:21:25.973924+0000","last_undegraded":"2024-06-03T03:21:25.973924+0000","last_fullsized":"2024-06-03T03:21:25.973924+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:01:26.556726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"99'5","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.825595+0000","last_change":"2024-06-03T03:20:25.904215+0000","last_active":"2024-06-03T03:22:54.825595+0000","last_peered":"2024-06-03T03:22:54.825595+0000","last_clean":"2024-06-03T03:22:54.825595+0000","last_became_active":"2024-06-03T03:20:25.903894+0000","last_became_peered":"2024-06-03T03:20:25.903894+0000","last_unstale":"2024-06-03T03:22:54.825595+0000","last_undegraded":"2024-06-03T03:22:54.825595+0000","last_fullsized":"2024-06-03T03:22:54.825595+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:02:07.587159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.973525+0000","last_change":"2024-06-03T03:19:56.910041+0000","last_active":"2024-06-03T03:21:25.973525+0000","last_peered":"2024-06-03T03:21:25.973525+0000","last_clean":"2024-06-03T03:21:25.973525+0000","last_became_active":"2024-06-03T03:19:56.909591+0000","last_became_peered":"2024-06-03T03:19:56.909591+0000","last_unstale":"2024-06-03T03:21:25.973525+0000","last_undegraded":"2024-06-03T03:21:25.973525+0000","last_fullsized":"2024-06-03T03:21:25.973525+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:42:20.016838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"97'1","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.055897+0000","last_change":"2024-06-03T03:19:36.741731+0000","last_active":"2024-06-03T03:21:26.055897+0000","last_peered":"2024-06-03T03:21:26.055897+0000","last_clean":"2024-06-03T03:21:26.055897+0000","last_became_active":"2024-06-03T03:19:36.741329+0000","last_became_peered":"2024-06-03T03:19:36.741329+0000","last_unstale":"2024-06-03T03:21:26.055897+0000","last_undegraded":"2024-06-03T03:21:26.055897+0000","last_fullsized":"2024-06-03T03:21:26.055897+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:02:45.778368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"99'4","reported_seq":43,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.825877+0000","last_change":"2024-06-03T03:19:41.632225+0000","last_active":"2024-06-03T03:22:54.825877+0000","last_peered":"2024-06-03T03:22:54.825877+0000","last_clean":"2024-06-03T03:22:54.825877+0000","last_became_active":"2024-06-03T03:19:41.631851+0000","last_became_peered":"2024-06-03T03:19:41.631851+0000","last_unstale":"2024-06-03T03:22:54.825877+0000","last_undegraded":"2024-06-03T03:22:54.825877+0000","last_fullsized":"2024-06-03T03:22:54.825877+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:52:23.364233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.977987+0000","last_change":"2024-06-03T03:19:56.934200+0000","last_active":"2024-06-03T03:21:25.977987+0000","last_peered":"2024-06-03T03:21:25.977987+0000","last_clean":"2024-06-03T03:21:25.977987+0000","last_became_active":"2024-06-03T03:19:56.932998+0000","last_became_peered":"2024-06-03T03:19:56.932998+0000","last_unstale":"2024-06-03T03:21:25.977987+0000","last_undegraded":"2024-06-03T03:21:25.977987+0000","last_fullsized":"2024-06-03T03:21:25.977987+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:45:28.916023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"97'5","reported_seq":178,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:23:13.293425+0000","last_change":"2024-06-03T03:19:36.546630+0000","last_active":"2024-06-03T03:23:13.293425+0000","last_peered":"2024-06-03T03:23:13.293425+0000","last_clean":"2024-06-03T03:23:13.293425+0000","last_became_active":"2024-06-03T03:19:36.546248+0000","last_became_peered":"2024-06-03T03:19:36.546248+0000","last_unstale":"2024-06-03T03:23:13.293425+0000","last_undegraded":"2024-06-03T03:23:13.293425+0000","last_fullsized":"2024-06-03T03:23:13.293425+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T15:09:50.323113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":128,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.830992+0000","last_change":"2024-06-03T03:19:41.624584+0000","last_active":"2024-06-03T03:22:54.830992+0000","last_peered":"2024-06-03T03:22:54.830992+0000","last_clean":"2024-06-03T03:22:54.830992+0000","last_became_active":"2024-06-03T03:19:41.624216+0000","last_became_peered":"2024-06-03T03:19:41.624216+0000","last_unstale":"2024-06-03T03:22:54.830992+0000","last_undegraded":"2024-06-03T03:22:54.830992+0000","last_fullsized":"2024-06-03T03:22:54.830992+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:22:09.645341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.979986+0000","last_change":"2024-06-03T03:19:56.937422+0000","last_active":"2024-06-03T03:21:25.979986+0000","last_peered":"2024-06-03T03:21:25.979986+0000","last_clean":"2024-06-03T03:21:25.979986+0000","last_became_active":"2024-06-03T03:19:56.934075+0000","last_became_peered":"2024-06-03T03:19:56.934075+0000","last_unstale":"2024-06-03T03:21:25.979986+0000","last_undegraded":"2024-06-03T03:21:25.979986+0000","last_fullsized":"2024-06-03T03:21:25.979986+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:25:45.273161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.814063+0000","last_change":"2024-06-03T03:19:41.589595+0000","last_active":"2024-06-03T03:22:54.814063+0000","last_peered":"2024-06-03T03:22:54.814063+0000","last_clean":"2024-06-03T03:22:54.814063+0000","last_became_active":"2024-06-03T03:19:41.589251+0000","last_became_peered":"2024-06-03T03:19:41.589251+0000","last_unstale":"2024-06-03T03:22:54.814063+0000","last_undegraded":"2024-06-03T03:22:54.814063+0000","last_fullsized":"2024-06-03T03:22:54.814063+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:51:40.417386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"97'5","reported_seq":175,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:23:11.289114+0000","last_change":"2024-06-03T03:19:36.737523+0000","last_active":"2024-06-03T03:23:11.289114+0000","last_peered":"2024-06-03T03:23:11.289114+0000","last_clean":"2024-06-03T03:23:11.289114+0000","last_became_active":"2024-06-03T03:19:36.736698+0000","last_became_peered":"2024-06-03T03:19:36.736698+0000","last_unstale":"2024-06-03T03:23:11.289114+0000","last_undegraded":"2024-06-03T03:23:11.289114+0000","last_fullsized":"2024-06-03T03:23:11.289114+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:14:46.330549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":126,"num_read_kb":126,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"97'1","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.052814+0000","last_change":"2024-06-03T03:19:56.924421+0000","last_active":"2024-06-03T03:21:26.052814+0000","last_peered":"2024-06-03T03:21:26.052814+0000","last_clean":"2024-06-03T03:21:26.052814+0000","last_became_active":"2024-06-03T03:19:56.924043+0000","last_became_peered":"2024-06-03T03:19:56.924043+0000","last_unstale":"2024-06-03T03:21:26.052814+0000","last_undegraded":"2024-06-03T03:21:26.052814+0000","last_fullsized":"2024-06-03T03:21:26.052814+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:00:47.674717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"99'8","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.826754+0000","last_change":"2024-06-03T03:19:41.613800+0000","last_active":"2024-06-03T03:22:54.826754+0000","last_peered":"2024-06-03T03:22:54.826754+0000","last_clean":"2024-06-03T03:22:54.826754+0000","last_became_active":"2024-06-03T03:19:41.613387+0000","last_became_peered":"2024-06-03T03:19:41.613387+0000","last_unstale":"2024-06-03T03:22:54.826754+0000","last_undegraded":"2024-06-03T03:22:54.826754+0000","last_fullsized":"2024-06-03T03:22:54.826754+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:04:04.785023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"97'6","reported_seq":53,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.980987+0000","last_change":"2024-06-03T03:19:36.544518+0000","last_active":"2024-06-03T03:21:25.980987+0000","last_peered":"2024-06-03T03:21:25.980987+0000","last_clean":"2024-06-03T03:21:25.980987+0000","last_became_active":"2024-06-03T03:19:36.544056+0000","last_became_peered":"2024-06-03T03:19:36.544056+0000","last_unstale":"2024-06-03T03:21:25.980987+0000","last_undegraded":"2024-06-03T03:21:25.980987+0000","last_fullsized":"2024-06-03T03:21:25.980987+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:33:25.012986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":367,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.052323+0000","last_change":"2024-06-03T03:18:05.481351+0000","last_active":"2024-06-03T03:21:26.052323+0000","last_peered":"2024-06-03T03:21:26.052323+0000","last_clean":"2024-06-03T03:21:26.052323+0000","last_became_active":"2024-06-03T03:18:05.480929+0000","last_became_peered":"2024-06-03T03:18:05.480929+0000","last_unstale":"2024-06-03T03:21:26.052323+0000","last_undegraded":"2024-06-03T03:21:26.052323+0000","last_fullsized":"2024-06-03T03:21:26.052323+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:14:09.558597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:14:09.558597+0000","last_clean_scrub_stamp":"2024-06-03T03:14:09.558597+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:54:45.563733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.054351+0000","last_change":"2024-06-03T03:19:56.931867+0000","last_active":"2024-06-03T03:21:26.054351+0000","last_peered":"2024-06-03T03:21:26.054351+0000","last_clean":"2024-06-03T03:21:26.054351+0000","last_became_active":"2024-06-03T03:19:56.931538+0000","last_became_peered":"2024-06-03T03:19:56.931538+0000","last_unstale":"2024-06-03T03:21:26.054351+0000","last_undegraded":"2024-06-03T03:21:26.054351+0000","last_fullsized":"2024-06-03T03:21:26.054351+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:55:15.144192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"99'5","reported_seq":43,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.813951+0000","last_change":"2024-06-03T03:19:41.622713+0000","last_active":"2024-06-03T03:22:54.813951+0000","last_peered":"2024-06-03T03:22:54.813951+0000","last_clean":"2024-06-03T03:22:54.813951+0000","last_became_active":"2024-06-03T03:19:41.622363+0000","last_became_peered":"2024-06-03T03:19:41.622363+0000","last_unstale":"2024-06-03T03:22:54.813951+0000","last_undegraded":"2024-06-03T03:22:54.813951+0000","last_fullsized":"2024-06-03T03:22:54.813951+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:43:57.671797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"97'4","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.052450+0000","last_change":"2024-06-03T03:19:36.539663+0000","last_active":"2024-06-03T03:21:26.052450+0000","last_peered":"2024-06-03T03:21:26.052450+0000","last_clean":"2024-06-03T03:21:26.052450+0000","last_became_active":"2024-06-03T03:19:36.539320+0000","last_became_peered":"2024-06-03T03:19:36.539320+0000","last_unstale":"2024-06-03T03:21:26.052450+0000","last_undegraded":"2024-06-03T03:21:26.052450+0000","last_fullsized":"2024-06-03T03:21:26.052450+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:36:50.699231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.053232+0000","last_change":"2024-06-03T03:19:56.938231+0000","last_active":"2024-06-03T03:21:26.053232+0000","last_peered":"2024-06-03T03:21:26.053232+0000","last_clean":"2024-06-03T03:21:26.053232+0000","last_became_active":"2024-06-03T03:19:56.937840+0000","last_became_peered":"2024-06-03T03:19:56.937840+0000","last_unstale":"2024-06-03T03:21:26.053232+0000","last_undegraded":"2024-06-03T03:21:26.053232+0000","last_fullsized":"2024-06-03T03:21:26.053232+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:02:33.916980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"97'4","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.973969+0000","last_change":"2024-06-03T03:19:36.737519+0000","last_active":"2024-06-03T03:21:25.973969+0000","last_peered":"2024-06-03T03:21:25.973969+0000","last_clean":"2024-06-03T03:21:25.973969+0000","last_became_active":"2024-06-03T03:19:36.737191+0000","last_became_peered":"2024-06-03T03:19:36.737191+0000","last_unstale":"2024-06-03T03:21:25.973969+0000","last_undegraded":"2024-06-03T03:21:25.973969+0000","last_fullsized":"2024-06-03T03:21:25.973969+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:50:21.389739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"99'9","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.817499+0000","last_change":"2024-06-03T03:19:41.633113+0000","last_active":"2024-06-03T03:22:54.817499+0000","last_peered":"2024-06-03T03:22:54.817499+0000","last_clean":"2024-06-03T03:22:54.817499+0000","last_became_active":"2024-06-03T03:19:41.632186+0000","last_became_peered":"2024-06-03T03:19:41.632186+0000","last_unstale":"2024-06-03T03:22:54.817499+0000","last_undegraded":"2024-06-03T03:22:54.817499+0000","last_fullsized":"2024-06-03T03:22:54.817499+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T15:12:26.835061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.057204+0000","last_change":"2024-06-03T03:19:56.928603+0000","last_active":"2024-06-03T03:21:26.057204+0000","last_peered":"2024-06-03T03:21:26.057204+0000","last_clean":"2024-06-03T03:21:26.057204+0000","last_became_active":"2024-06-03T03:19:56.928175+0000","last_became_peered":"2024-06-03T03:19:56.928175+0000","last_unstale":"2024-06-03T03:21:26.057204+0000","last_undegraded":"2024-06-03T03:21:26.057204+0000","last_fullsized":"2024-06-03T03:21:26.057204+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:26:03.531091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"99'19","reported_seq":64,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:45.881135+0000","last_change":"2024-06-03T03:19:36.550577+0000","last_active":"2024-06-03T03:21:45.881135+0000","last_peered":"2024-06-03T03:21:45.881135+0000","last_clean":"2024-06-03T03:21:45.881135+0000","last_became_active":"2024-06-03T03:19:36.549968+0000","last_became_peered":"2024-06-03T03:19:36.549968+0000","last_unstale":"2024-06-03T03:21:45.881135+0000","last_undegraded":"2024-06-03T03:21:45.881135+0000","last_fullsized":"2024-06-03T03:21:45.881135+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:22:43.892002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"99'5","reported_seq":43,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.825404+0000","last_change":"2024-06-03T03:19:41.613883+0000","last_active":"2024-06-03T03:22:54.825404+0000","last_peered":"2024-06-03T03:22:54.825404+0000","last_clean":"2024-06-03T03:22:54.825404+0000","last_became_active":"2024-06-03T03:19:41.613472+0000","last_became_peered":"2024-06-03T03:19:41.613472+0000","last_unstale":"2024-06-03T03:22:54.825404+0000","last_undegraded":"2024-06-03T03:22:54.825404+0000","last_fullsized":"2024-06-03T03:22:54.825404+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T14:38:42.044534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.052273+0000","last_change":"2024-06-03T03:19:56.914487+0000","last_active":"2024-06-03T03:21:26.052273+0000","last_peered":"2024-06-03T03:21:26.052273+0000","last_clean":"2024-06-03T03:21:26.052273+0000","last_became_active":"2024-06-03T03:19:56.913904+0000","last_became_peered":"2024-06-03T03:19:56.913904+0000","last_unstale":"2024-06-03T03:21:26.052273+0000","last_undegraded":"2024-06-03T03:21:26.052273+0000","last_fullsized":"2024-06-03T03:21:26.052273+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T15:13:24.084034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"97'7","reported_seq":52,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.981407+0000","last_change":"2024-06-03T03:19:36.671355+0000","last_active":"2024-06-03T03:21:25.981407+0000","last_peered":"2024-06-03T03:21:25.981407+0000","last_clean":"2024-06-03T03:21:25.981407+0000","last_became_active":"2024-06-03T03:19:36.670926+0000","last_became_peered":"2024-06-03T03:19:36.670926+0000","last_unstale":"2024-06-03T03:21:25.981407+0000","last_undegraded":"2024-06-03T03:21:25.981407+0000","last_fullsized":"2024-06-03T03:21:25.981407+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:38:50.914238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"99'7","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.833522+0000","last_change":"2024-06-03T03:19:41.629054+0000","last_active":"2024-06-03T03:22:54.833522+0000","last_peered":"2024-06-03T03:22:54.833522+0000","last_clean":"2024-06-03T03:22:54.833522+0000","last_became_active":"2024-06-03T03:19:41.627118+0000","last_became_peered":"2024-06-03T03:19:41.627118+0000","last_unstale":"2024-06-03T03:22:54.833522+0000","last_undegraded":"2024-06-03T03:22:54.833522+0000","last_fullsized":"2024-06-03T03:22:54.833522+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:59:40.645002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":36,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.059189+0000","last_change":"2024-06-03T03:21:26.058741+0000","last_active":"2024-06-03T03:21:26.059189+0000","last_peered":"2024-06-03T03:21:26.059189+0000","last_clean":"2024-06-03T03:21:26.059189+0000","last_became_active":"2024-06-03T03:21:26.058271+0000","last_became_peered":"2024-06-03T03:21:26.058271+0000","last_unstale":"2024-06-03T03:21:26.059189+0000","last_undegraded":"2024-06-03T03:21:26.059189+0000","last_fullsized":"2024-06-03T03:21:26.059189+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:31:32.359893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"97'2","reported_seq":53,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.052741+0000","last_change":"2024-06-03T03:20:26.132994+0000","last_active":"2024-06-03T03:21:26.052741+0000","last_peered":"2024-06-03T03:21:26.052741+0000","last_clean":"2024-06-03T03:21:26.052741+0000","last_became_active":"2024-06-03T03:20:26.132574+0000","last_became_peered":"2024-06-03T03:20:26.132574+0000","last_unstale":"2024-06-03T03:21:26.052741+0000","last_undegraded":"2024-06-03T03:21:26.052741+0000","last_fullsized":"2024-06-03T03:21:26.052741+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T15:12:33.599154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"99'2","reported_seq":40,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.791726+0000","last_change":"2024-06-03T03:19:41.603630+0000","last_active":"2024-06-03T03:22:54.791726+0000","last_peered":"2024-06-03T03:22:54.791726+0000","last_clean":"2024-06-03T03:22:54.791726+0000","last_became_active":"2024-06-03T03:19:41.603301+0000","last_became_peered":"2024-06-03T03:19:41.603301+0000","last_unstale":"2024-06-03T03:22:54.791726+0000","last_undegraded":"2024-06-03T03:22:54.791726+0000","last_fullsized":"2024-06-03T03:22:54.791726+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:20:33.076622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.057638+0000","last_change":"2024-06-03T03:19:56.929023+0000","last_active":"2024-06-03T03:21:26.057638+0000","last_peered":"2024-06-03T03:21:26.057638+0000","last_clean":"2024-06-03T03:21:26.057638+0000","last_became_active":"2024-06-03T03:19:56.928454+0000","last_became_peered":"2024-06-03T03:19:56.928454+0000","last_unstale":"2024-06-03T03:21:26.057638+0000","last_undegraded":"2024-06-03T03:21:26.057638+0000","last_fullsized":"2024-06-03T03:21:26.057638+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:17:31.755649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"97'7","reported_seq":51,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.056909+0000","last_change":"2024-06-03T03:19:36.551413+0000","last_active":"2024-06-03T03:21:26.056909+0000","last_peered":"2024-06-03T03:21:26.056909+0000","last_clean":"2024-06-03T03:21:26.056909+0000","last_became_active":"2024-06-03T03:19:36.550622+0000","last_became_peered":"2024-06-03T03:19:36.550622+0000","last_unstale":"2024-06-03T03:21:26.056909+0000","last_undegraded":"2024-06-03T03:21:26.056909+0000","last_fullsized":"2024-06-03T03:21:26.056909+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:50:09.472960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"99'9","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.830598+0000","last_change":"2024-06-03T03:19:41.624871+0000","last_active":"2024-06-03T03:22:54.830598+0000","last_peered":"2024-06-03T03:22:54.830598+0000","last_clean":"2024-06-03T03:22:54.830598+0000","last_became_active":"2024-06-03T03:19:41.624638+0000","last_became_peered":"2024-06-03T03:19:41.624638+0000","last_unstale":"2024-06-03T03:22:54.830598+0000","last_undegraded":"2024-06-03T03:22:54.830598+0000","last_fullsized":"2024-06-03T03:22:54.830598+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:21:16.193817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"99'1","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:29.282055+0000","last_change":"2024-06-03T03:19:56.924638+0000","last_active":"2024-06-03T03:21:29.282055+0000","last_peered":"2024-06-03T03:21:29.282055+0000","last_clean":"2024-06-03T03:21:29.282055+0000","last_became_active":"2024-06-03T03:19:56.924200+0000","last_became_peered":"2024-06-03T03:19:56.924200+0000","last_unstale":"2024-06-03T03:21:29.282055+0000","last_undegraded":"2024-06-03T03:21:29.282055+0000","last_fullsized":"2024-06-03T03:21:29.282055+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:38:01.138253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"97'3","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.052712+0000","last_change":"2024-06-03T03:19:36.546974+0000","last_active":"2024-06-03T03:21:26.052712+0000","last_peered":"2024-06-03T03:21:26.052712+0000","last_clean":"2024-06-03T03:21:26.052712+0000","last_became_active":"2024-06-03T03:19:36.546433+0000","last_became_peered":"2024-06-03T03:19:36.546433+0000","last_unstale":"2024-06-03T03:21:26.052712+0000","last_undegraded":"2024-06-03T03:21:26.052712+0000","last_fullsized":"2024-06-03T03:21:26.052712+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:41:46.980327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"99'8","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.833961+0000","last_change":"2024-06-03T03:19:41.622118+0000","last_active":"2024-06-03T03:22:54.833961+0000","last_peered":"2024-06-03T03:22:54.833961+0000","last_clean":"2024-06-03T03:22:54.833961+0000","last_became_active":"2024-06-03T03:19:41.620789+0000","last_became_peered":"2024-06-03T03:19:41.620789+0000","last_unstale":"2024-06-03T03:22:54.833961+0000","last_undegraded":"2024-06-03T03:22:54.833961+0000","last_fullsized":"2024-06-03T03:22:54.833961+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:00:36.232238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.981089+0000","last_change":"2024-06-03T03:19:56.921299+0000","last_active":"2024-06-03T03:21:25.981089+0000","last_peered":"2024-06-03T03:21:25.981089+0000","last_clean":"2024-06-03T03:21:25.981089+0000","last_became_active":"2024-06-03T03:19:56.920870+0000","last_became_peered":"2024-06-03T03:19:56.920870+0000","last_unstale":"2024-06-03T03:21:25.981089+0000","last_undegraded":"2024-06-03T03:21:25.981089+0000","last_fullsized":"2024-06-03T03:21:25.981089+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:10:15.062381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"97'4","reported_seq":23,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.053109+0000","last_change":"2024-06-03T03:20:25.903279+0000","last_active":"2024-06-03T03:21:26.053109+0000","last_peered":"2024-06-03T03:21:26.053109+0000","last_clean":"2024-06-03T03:21:26.053109+0000","last_became_active":"2024-06-03T03:20:25.900311+0000","last_became_peered":"2024-06-03T03:20:25.900311+0000","last_unstale":"2024-06-03T03:21:26.053109+0000","last_undegraded":"2024-06-03T03:21:26.053109+0000","last_fullsized":"2024-06-03T03:21:26.053109+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:25:49.708581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"99'5","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.836116+0000","last_change":"2024-06-03T03:19:41.628659+0000","last_active":"2024-06-03T03:22:54.836116+0000","last_peered":"2024-06-03T03:22:54.836116+0000","last_clean":"2024-06-03T03:22:54.836116+0000","last_became_active":"2024-06-03T03:19:41.628291+0000","last_became_peered":"2024-06-03T03:19:41.628291+0000","last_unstale":"2024-06-03T03:22:54.836116+0000","last_undegraded":"2024-06-03T03:22:54.836116+0000","last_fullsized":"2024-06-03T03:22:54.836116+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:26:49.579669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.052963+0000","last_change":"2024-06-03T03:19:56.922766+0000","last_active":"2024-06-03T03:21:26.052963+0000","last_peered":"2024-06-03T03:21:26.052963+0000","last_clean":"2024-06-03T03:21:26.052963+0000","last_became_active":"2024-06-03T03:19:56.922310+0000","last_became_peered":"2024-06-03T03:19:56.922310+0000","last_unstale":"2024-06-03T03:21:26.052963+0000","last_undegraded":"2024-06-03T03:21:26.052963+0000","last_fullsized":"2024-06-03T03:21:26.052963+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T14:35:53.133519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"97'5","reported_seq":148,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:23:11.237711+0000","last_change":"2024-06-03T03:20:26.133562+0000","last_active":"2024-06-03T03:23:11.237711+0000","last_peered":"2024-06-03T03:23:11.237711+0000","last_clean":"2024-06-03T03:23:11.237711+0000","last_became_active":"2024-06-03T03:20:26.133074+0000","last_became_peered":"2024-06-03T03:20:26.133074+0000","last_unstale":"2024-06-03T03:23:11.237711+0000","last_undegraded":"2024-06-03T03:23:11.237711+0000","last_fullsized":"2024-06-03T03:23:11.237711+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:51:33.717584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":127,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.817652+0000","last_change":"2024-06-03T03:19:41.611450+0000","last_active":"2024-06-03T03:22:54.817652+0000","last_peered":"2024-06-03T03:22:54.817652+0000","last_clean":"2024-06-03T03:22:54.817652+0000","last_became_active":"2024-06-03T03:19:41.610214+0000","last_became_peered":"2024-06-03T03:19:41.610214+0000","last_unstale":"2024-06-03T03:22:54.817652+0000","last_undegraded":"2024-06-03T03:22:54.817652+0000","last_fullsized":"2024-06-03T03:22:54.817652+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:01:01.521917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.975566+0000","last_change":"2024-06-03T03:19:56.935686+0000","last_active":"2024-06-03T03:21:25.975566+0000","last_peered":"2024-06-03T03:21:25.975566+0000","last_clean":"2024-06-03T03:21:25.975566+0000","last_became_active":"2024-06-03T03:19:56.935314+0000","last_became_peered":"2024-06-03T03:19:56.935314+0000","last_unstale":"2024-06-03T03:21:25.975566+0000","last_undegraded":"2024-06-03T03:21:25.975566+0000","last_fullsized":"2024-06-03T03:21:25.975566+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T15:10:16.966321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"97'3","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.053921+0000","last_change":"2024-06-03T03:19:36.547756+0000","last_active":"2024-06-03T03:21:26.053921+0000","last_peered":"2024-06-03T03:21:26.053921+0000","last_clean":"2024-06-03T03:21:26.053921+0000","last_became_active":"2024-06-03T03:19:36.547350+0000","last_became_peered":"2024-06-03T03:19:36.547350+0000","last_unstale":"2024-06-03T03:21:26.053921+0000","last_undegraded":"2024-06-03T03:21:26.053921+0000","last_fullsized":"2024-06-03T03:21:26.053921+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:57:39.214380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.827896+0000","last_change":"2024-06-03T03:19:41.621083+0000","last_active":"2024-06-03T03:22:54.827896+0000","last_peered":"2024-06-03T03:22:54.827896+0000","last_clean":"2024-06-03T03:22:54.827896+0000","last_became_active":"2024-06-03T03:19:41.620719+0000","last_became_peered":"2024-06-03T03:19:41.620719+0000","last_unstale":"2024-06-03T03:22:54.827896+0000","last_undegraded":"2024-06-03T03:22:54.827896+0000","last_fullsized":"2024-06-03T03:22:54.827896+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:50:35.007025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.052302+0000","last_change":"2024-06-03T03:19:56.915406+0000","last_active":"2024-06-03T03:21:26.052302+0000","last_peered":"2024-06-03T03:21:26.052302+0000","last_clean":"2024-06-03T03:21:26.052302+0000","last_became_active":"2024-06-03T03:19:56.914918+0000","last_became_peered":"2024-06-03T03:19:56.914918+0000","last_unstale":"2024-06-03T03:21:26.052302+0000","last_undegraded":"2024-06-03T03:21:26.052302+0000","last_fullsized":"2024-06-03T03:21:26.052302+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:42:59.324011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"97'5","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.979623+0000","last_change":"2024-06-03T03:19:36.549502+0000","last_active":"2024-06-03T03:21:25.979623+0000","last_peered":"2024-06-03T03:21:25.979623+0000","last_clean":"2024-06-03T03:21:25.979623+0000","last_became_active":"2024-06-03T03:19:36.547538+0000","last_became_peered":"2024-06-03T03:19:36.547538+0000","last_unstale":"2024-06-03T03:21:25.979623+0000","last_undegraded":"2024-06-03T03:21:25.979623+0000","last_fullsized":"2024-06-03T03:21:25.979623+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:28:50.558069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.815219+0000","last_change":"2024-06-03T03:19:41.607441+0000","last_active":"2024-06-03T03:22:54.815219+0000","last_peered":"2024-06-03T03:22:54.815219+0000","last_clean":"2024-06-03T03:22:54.815219+0000","last_became_active":"2024-06-03T03:19:41.607086+0000","last_became_peered":"2024-06-03T03:19:41.607086+0000","last_unstale":"2024-06-03T03:22:54.815219+0000","last_undegraded":"2024-06-03T03:22:54.815219+0000","last_fullsized":"2024-06-03T03:22:54.815219+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T14:53:50.091872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"99'9","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.819487+0000","last_change":"2024-06-03T03:19:41.607004+0000","last_active":"2024-06-03T03:22:54.819487+0000","last_peered":"2024-06-03T03:22:54.819487+0000","last_clean":"2024-06-03T03:22:54.819487+0000","last_became_active":"2024-06-03T03:19:41.606684+0000","last_became_peered":"2024-06-03T03:19:41.606684+0000","last_unstale":"2024-06-03T03:22:54.819487+0000","last_undegraded":"2024-06-03T03:22:54.819487+0000","last_fullsized":"2024-06-03T03:22:54.819487+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:48:55.320597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"99'12","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.810392+0000","last_change":"2024-06-03T03:19:41.589426+0000","last_active":"2024-06-03T03:22:54.810392+0000","last_peered":"2024-06-03T03:22:54.810392+0000","last_clean":"2024-06-03T03:22:54.810392+0000","last_became_active":"2024-06-03T03:19:41.589091+0000","last_became_peered":"2024-06-03T03:19:41.589091+0000","last_unstale":"2024-06-03T03:22:54.810392+0000","last_undegraded":"2024-06-03T03:22:54.810392+0000","last_fullsized":"2024-06-03T03:22:54.810392+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:40:34.313214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"99'8","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.832429+0000","last_change":"2024-06-03T03:19:41.626149+0000","last_active":"2024-06-03T03:22:54.832429+0000","last_peered":"2024-06-03T03:22:54.832429+0000","last_clean":"2024-06-03T03:22:54.832429+0000","last_became_active":"2024-06-03T03:19:41.625710+0000","last_became_peered":"2024-06-03T03:19:41.625710+0000","last_unstale":"2024-06-03T03:22:54.832429+0000","last_undegraded":"2024-06-03T03:22:54.832429+0000","last_fullsized":"2024-06-03T03:22:54.832429+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:15:15.220975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"99'7","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.834957+0000","last_change":"2024-06-03T03:19:41.636585+0000","last_active":"2024-06-03T03:22:54.834957+0000","last_peered":"2024-06-03T03:22:54.834957+0000","last_clean":"2024-06-03T03:22:54.834957+0000","last_became_active":"2024-06-03T03:19:41.636241+0000","last_became_peered":"2024-06-03T03:19:41.636241+0000","last_unstale":"2024-06-03T03:22:54.834957+0000","last_undegraded":"2024-06-03T03:22:54.834957+0000","last_fullsized":"2024-06-03T03:22:54.834957+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:19:28.548297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"99'4","reported_seq":42,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.834106+0000","last_change":"2024-06-03T03:19:41.629530+0000","last_active":"2024-06-03T03:22:54.834106+0000","last_peered":"2024-06-03T03:22:54.834106+0000","last_clean":"2024-06-03T03:22:54.834106+0000","last_became_active":"2024-06-03T03:19:41.629197+0000","last_became_peered":"2024-06-03T03:19:41.629197+0000","last_unstale":"2024-06-03T03:22:54.834106+0000","last_undegraded":"2024-06-03T03:22:54.834106+0000","last_fullsized":"2024-06-03T03:22:54.834106+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:12:02.807445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"99'6","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.818988+0000","last_change":"2024-06-03T03:19:41.606995+0000","last_active":"2024-06-03T03:22:54.818988+0000","last_peered":"2024-06-03T03:22:54.818988+0000","last_clean":"2024-06-03T03:22:54.818988+0000","last_became_active":"2024-06-03T03:19:41.606651+0000","last_became_peered":"2024-06-03T03:19:41.606651+0000","last_unstale":"2024-06-03T03:22:54.818988+0000","last_undegraded":"2024-06-03T03:22:54.818988+0000","last_fullsized":"2024-06-03T03:22:54.818988+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:07:11.751606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"99'15","reported_seq":53,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.810018+0000","last_change":"2024-06-03T03:19:41.609848+0000","last_active":"2024-06-03T03:22:54.810018+0000","last_peered":"2024-06-03T03:22:54.810018+0000","last_clean":"2024-06-03T03:22:54.810018+0000","last_became_active":"2024-06-03T03:19:41.609544+0000","last_became_peered":"2024-06-03T03:19:41.609544+0000","last_unstale":"2024-06-03T03:22:54.810018+0000","last_undegraded":"2024-06-03T03:22:54.810018+0000","last_fullsized":"2024-06-03T03:22:54.810018+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:12:00.804638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"99'2","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.810429+0000","last_change":"2024-06-03T03:20:25.912049+0000","last_active":"2024-06-03T03:22:54.810429+0000","last_peered":"2024-06-03T03:22:54.810429+0000","last_clean":"2024-06-03T03:22:54.810429+0000","last_became_active":"2024-06-03T03:20:25.910664+0000","last_became_peered":"2024-06-03T03:20:25.910664+0000","last_unstale":"2024-06-03T03:22:54.810429+0000","last_undegraded":"2024-06-03T03:22:54.810429+0000","last_fullsized":"2024-06-03T03:22:54.810429+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:47:19.282569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.828352+0000","last_change":"2024-06-03T03:19:41.610800+0000","last_active":"2024-06-03T03:22:54.828352+0000","last_peered":"2024-06-03T03:22:54.828352+0000","last_clean":"2024-06-03T03:22:54.828352+0000","last_became_active":"2024-06-03T03:19:41.610457+0000","last_became_peered":"2024-06-03T03:19:41.610457+0000","last_unstale":"2024-06-03T03:22:54.828352+0000","last_undegraded":"2024-06-03T03:22:54.828352+0000","last_fullsized":"2024-06-03T03:22:54.828352+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:31:06.415159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"99'8","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.821273+0000","last_change":"2024-06-03T03:19:41.617020+0000","last_active":"2024-06-03T03:22:54.821273+0000","last_peered":"2024-06-03T03:22:54.821273+0000","last_clean":"2024-06-03T03:22:54.821273+0000","last_became_active":"2024-06-03T03:19:41.616614+0000","last_became_peered":"2024-06-03T03:19:41.616614+0000","last_unstale":"2024-06-03T03:22:54.821273+0000","last_undegraded":"2024-06-03T03:22:54.821273+0000","last_fullsized":"2024-06-03T03:22:54.821273+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:35:32.657685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"99'10","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.828102+0000","last_change":"2024-06-03T03:19:41.622785+0000","last_active":"2024-06-03T03:22:54.828102+0000","last_peered":"2024-06-03T03:22:54.828102+0000","last_clean":"2024-06-03T03:22:54.828102+0000","last_became_active":"2024-06-03T03:19:41.622407+0000","last_became_peered":"2024-06-03T03:19:41.622407+0000","last_unstale":"2024-06-03T03:22:54.828102+0000","last_undegraded":"2024-06-03T03:22:54.828102+0000","last_fullsized":"2024-06-03T03:22:54.828102+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:53:50.804889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"99'11","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.838842+0000","last_change":"2024-06-03T03:19:41.633429+0000","last_active":"2024-06-03T03:22:54.838842+0000","last_peered":"2024-06-03T03:22:54.838842+0000","last_clean":"2024-06-03T03:22:54.838842+0000","last_became_active":"2024-06-03T03:19:41.631028+0000","last_became_peered":"2024-06-03T03:19:41.631028+0000","last_unstale":"2024-06-03T03:22:54.838842+0000","last_undegraded":"2024-06-03T03:22:54.838842+0000","last_fullsized":"2024-06-03T03:22:54.838842+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:37:10.666591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"99'8","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.818347+0000","last_change":"2024-06-03T03:19:41.606204+0000","last_active":"2024-06-03T03:22:54.818347+0000","last_peered":"2024-06-03T03:22:54.818347+0000","last_clean":"2024-06-03T03:22:54.818347+0000","last_became_active":"2024-06-03T03:19:41.605756+0000","last_became_peered":"2024-06-03T03:19:41.605756+0000","last_unstale":"2024-06-03T03:22:54.818347+0000","last_undegraded":"2024-06-03T03:22:54.818347+0000","last_fullsized":"2024-06-03T03:22:54.818347+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:24:32.507426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":245318,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":76157,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1290,"num_read_kb":726,"num_write":264,"num_write_kb":155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":228471,"data_compressed":5580,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":226,"ondisk_log_size":226,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383740,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1101824,"data_stored":570375,"data_compressed":32353,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645377,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32400,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712752,"statfs":{"total":95995035648,"available":95961858048,"internally_reserved":0,"allocated":1253376,"data_stored":1226395,"data_compressed":69963,"data_compressed_allocated":94208,"data_compressed_original":688128,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808902,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32280,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712872,"statfs":{"total":95995035648,"available":95961980928,"internally_reserved":0,"allocated":1130496,"data_stored":574401,"data_compressed":32353,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070541,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32272,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712880,"statfs":{"total":95995035648,"available":95961989120,"internally_reserved":0,"allocated":1122304,"data_stored":570683,"data_compressed":32353,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332179,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32324,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712828,"statfs":{"total":95995035648,"available":95961935872,"internally_reserved":0,"allocated":1175552,"data_stored":1167320,"data_compressed":68103,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495705,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1101824,"data_stored":565809,"data_compressed":32353,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574964,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28160,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716992,"statfs":{"total":95995035648,"available":95966199808,"internally_reserved":0,"allocated":1171456,"data_stored":636043,"data_compressed":34213,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738489,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1126400,"data_stored":567265,"data_compressed":32353,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411439,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36492,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708660,"statfs":{"total":95995035648,"available":95957667840,"internally_reserved":0,"allocated":1183744,"data_stored":635624,"data_compressed":34213,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182505,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1114112,"data_stored":565517,"data_compressed":32353,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920868,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28108,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717044,"statfs":{"total":95995035648,"available":95966253056,"internally_reserved":0,"allocated":1118208,"data_stored":571053,"data_compressed":32353,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757343,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32332,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712820,"statfs":{"total":95995035648,"available":95961927680,"internally_reserved":0,"allocated":1183744,"data_stored":1162902,"data_compressed":68103,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34106,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34106,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34106,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35750,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35750,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35750,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35750,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":70956,"data_compressed":1860,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":70537,"data_compressed":1860,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":70940,"data_compressed":1860,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-03T03:23:17.177 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-03T03:23:17.681 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:23:18.183 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:23:17 smithi117 ceph-mon[31649]: from='client.24902 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-03T03:23:18.183 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:23:17 smithi117 ceph-mon[31649]: pgmap v445: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-06-03T03:23:18.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:23:17 smithi183 ceph-mon[37326]: from='client.24902 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-03T03:23:18.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:23:17 smithi183 ceph-mon[37326]: pgmap v445: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-06-03T03:23:18.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:23:17 smithi151 ceph-mon[37043]: from='client.24902 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-03T03:23:18.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:23:17 smithi151 ceph-mon[37043]: pgmap v445: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-06-03T03:23:20.040 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:23:19 smithi117 ceph-mon[31649]: pgmap v446: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-06-03T03:23:20.060 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:20.059+0000 7f8f85600700 1 -- 172.21.15.117:0/1246823827 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f800746c0 msgr2=0x7f8f80074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:20.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:20.059+0000 7f8f85600700 1 --2- 172.21.15.117:0/1246823827 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f800746c0 0x7f8f80074aa0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8f70009990 tx=0x7f8f70009ca0 comp rx=0 tx=0).stop 2024-06-03T03:23:20.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:20.059+0000 7f8f85600700 1 -- 172.21.15.117:0/1246823827 shutdown_connections 2024-06-03T03:23:20.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:20.059+0000 7f8f85600700 1 --2- 172.21.15.117:0/1246823827 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f8010ee90 0x7f8f80111280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:20.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:20.059+0000 7f8f85600700 1 --2- 172.21.15.117:0/1246823827 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f800746c0 0x7f8f80074aa0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:20.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:20.059+0000 7f8f85600700 1 --2- 172.21.15.117:0/1246823827 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f80074fe0 0x7f8f8010e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:20.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:20.059+0000 7f8f85600700 1 -- 172.21.15.117:0/1246823827 >> 172.21.15.117:0/1246823827 conn(0x7f8f800795f0 msgr2=0x7f8f80079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:23:20.062 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:20.059+0000 7f8f85600700 1 -- 172.21.15.117:0/1246823827 shutdown_connections 2024-06-03T03:23:20.062 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:20.059+0000 7f8f85600700 1 -- 172.21.15.117:0/1246823827 wait complete. 2024-06-03T03:23:20.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:20.060+0000 7f8f85600700 1 Processor -- start 2024-06-03T03:23:20.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:20.061+0000 7f8f85600700 1 -- start start 2024-06-03T03:23:20.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:20.061+0000 7f8f85600700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f800746c0 0x7f8f8019ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:20.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:20.061+0000 7f8f85600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f80074fe0 0x7f8f8019b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:20.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:20.061+0000 7f8f85600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f8010ee90 0x7f8f8019f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:20.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:20.061+0000 7f8f85600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8f80114990 con 0x7f8f80074fe0 2024-06-03T03:23:20.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:20.061+0000 7f8f85600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8f80114810 con 0x7f8f8010ee90 2024-06-03T03:23:20.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:20.061+0000 7f8f85600700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8f80114b10 con 0x7f8f800746c0 2024-06-03T03:23:20.065 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:20.061+0000 7f8f77e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f80074fe0 0x7f8f8019b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:20.065 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:20.061+0000 7f8f77e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f80074fe0 0x7f8f8019b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:42284/0 (socket says 172.21.15.117:42284) 2024-06-03T03:23:20.065 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:20.061+0000 7f8f77e00700 1 -- 172.21.15.117:0/4267366626 learned_addr learned my addr 172.21.15.117:0/4267366626 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:23:20.065 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:20.061+0000 7f8f7f400700 1 --2- 172.21.15.117:0/4267366626 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f8010ee90 0x7f8f8019f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:20.065 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:20.062+0000 7f8f7ea00700 1 --2- 172.21.15.117:0/4267366626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f800746c0 0x7f8f8019ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:20.065 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:20.062+0000 7f8f77e00700 1 -- 172.21.15.117:0/4267366626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f800746c0 msgr2=0x7f8f8019ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:20.065 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:20.062+0000 7f8f77e00700 1 --2- 172.21.15.117:0/4267366626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f800746c0 0x7f8f8019ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:20.065 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:20.062+0000 7f8f77e00700 1 -- 172.21.15.117:0/4267366626 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f8010ee90 msgr2=0x7f8f8019f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:20.065 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:20.062+0000 7f8f77e00700 1 --2- 172.21.15.117:0/4267366626 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f8010ee90 0x7f8f8019f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:20.065 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:20.062+0000 7f8f77e00700 1 -- 172.21.15.117:0/4267366626 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f8019fdb0 con 0x7f8f80074fe0 2024-06-03T03:23:20.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:20.062+0000 7f8f7f400700 1 --2- 172.21.15.117:0/4267366626 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f8010ee90 0x7f8f8019f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:23:20.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:20.062+0000 7f8f77e00700 1 --2- 172.21.15.117:0/4267366626 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f80074fe0 0x7f8f8019b2e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8f6800ea50 tx=0x7f8f6800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:23:20.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:20.063+0000 7f8f77400700 1 -- 172.21.15.117:0/4267366626 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f6800cbd0 con 0x7f8f80074fe0 2024-06-03T03:23:20.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:20.063+0000 7f8f77400700 1 -- 172.21.15.117:0/4267366626 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f6800cd70 con 0x7f8f80074fe0 2024-06-03T03:23:20.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:20.063+0000 7f8f85600700 1 -- 172.21.15.117:0/4267366626 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f801a00a0 con 0x7f8f80074fe0 2024-06-03T03:23:20.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:20.063+0000 7f8f77400700 1 -- 172.21.15.117:0/4267366626 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f68018750 con 0x7f8f80074fe0 2024-06-03T03:23:20.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:20.063+0000 7f8f85600700 1 -- 172.21.15.117:0/4267366626 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f801a7920 con 0x7f8f80074fe0 2024-06-03T03:23:20.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:20.064+0000 7f8f85600700 1 -- 172.21.15.117:0/4267366626 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f80069180 con 0x7f8f80074fe0 2024-06-03T03:23:20.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:20.066+0000 7f8f77400700 1 -- 172.21.15.117:0/4267366626 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f8f6801f020 con 0x7f8f80074fe0 2024-06-03T03:23:20.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:20.068+0000 7f8f77400700 1 --2- 172.21.15.117:0/4267366626 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f8f640a2ca0 0x7f8f640a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:20.070 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:20.069+0000 7f8f77400700 1 -- 172.21.15.117:0/4267366626 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f8f680cfc20 con 0x7f8f80074fe0 2024-06-03T03:23:20.070 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:20.069+0000 7f8f7ea00700 1 --2- 172.21.15.117:0/4267366626 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f8f640a2ca0 0x7f8f640a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:20.070 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:20.070+0000 7f8f7ea00700 1 --2- 172.21.15.117:0/4267366626 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f8f640a2ca0 0x7f8f640a5160 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8f7000b2e0 tx=0x7f8f70005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:23:20.072 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:20.071+0000 7f8f77400700 1 -- 172.21.15.117:0/4267366626 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8f68026070 con 0x7f8f80074fe0 2024-06-03T03:23:20.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:23:19 smithi183 ceph-mon[37326]: pgmap v446: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-06-03T03:23:20.292 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:20.290+0000 7f8f85600700 1 -- 172.21.15.117:0/4267366626 --> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8f800ffd70 con 0x7f8f640a2ca0 2024-06-03T03:23:20.302 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:20.301+0000 7f8f77400700 1 -- 172.21.15.117:0/4267366626 <== mgr.14152 v2:172.21.15.117:6800/1398160887 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340822 (secure 0 0 0) 0x7f8f800ffd70 con 0x7f8f640a2ca0 2024-06-03T03:23:20.303 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T03:23:20.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:20.306+0000 7f8f85600700 1 -- 172.21.15.117:0/4267366626 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f8f640a2ca0 msgr2=0x7f8f640a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:20.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:20.306+0000 7f8f85600700 1 --2- 172.21.15.117:0/4267366626 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f8f640a2ca0 0x7f8f640a5160 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8f7000b2e0 tx=0x7f8f70005960 comp rx=0 tx=0).stop 2024-06-03T03:23:20.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:20.306+0000 7f8f85600700 1 -- 172.21.15.117:0/4267366626 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f80074fe0 msgr2=0x7f8f8019b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:20.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:20.306+0000 7f8f85600700 1 --2- 172.21.15.117:0/4267366626 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f80074fe0 0x7f8f8019b2e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8f6800ea50 tx=0x7f8f6800ed60 comp rx=0 tx=0).stop 2024-06-03T03:23:20.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:20.307+0000 7f8f85600700 1 -- 172.21.15.117:0/4267366626 shutdown_connections 2024-06-03T03:23:20.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:20.307+0000 7f8f85600700 1 --2- 172.21.15.117:0/4267366626 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f8f640a2ca0 0x7f8f640a5160 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:20.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:20.307+0000 7f8f85600700 1 --2- 172.21.15.117:0/4267366626 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f8010ee90 0x7f8f8019f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:20.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:20.307+0000 7f8f85600700 1 --2- 172.21.15.117:0/4267366626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f800746c0 0x7f8f8019ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:20.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:20.307+0000 7f8f85600700 1 --2- 172.21.15.117:0/4267366626 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f80074fe0 0x7f8f8019b2e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:20.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:20.307+0000 7f8f85600700 1 -- 172.21.15.117:0/4267366626 >> 172.21.15.117:0/4267366626 conn(0x7f8f800795f0 msgr2=0x7f8f800ff660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:23:20.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:20.307+0000 7f8f85600700 1 -- 172.21.15.117:0/4267366626 shutdown_connections 2024-06-03T03:23:20.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:20.307+0000 7f8f85600700 1 -- 172.21.15.117:0/4267366626 wait complete. 2024-06-03T03:23:20.311 INFO:teuthology.orchestra.run.smithi117.stderr:dumped all 2024-06-03T03:23:20.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:23:19 smithi151 ceph-mon[37043]: pgmap v446: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-06-03T03:23:21.137 INFO:teuthology.orchestra.run.smithi117.stdout:{"pg_ready":true,"pg_map":{"version":446,"stamp":"2024-06-03T03:23:18.899798+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":666759,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1461,"num_read_kb":860,"num_write":447,"num_write_kb":3095,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":865,"ondisk_log_size":865,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":370932,"kb_used_data":13460,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124570892,"statfs":{"total":1151940427776,"available":1151560593408,"internally_reserved":0,"allocated":13783040,"data_stored":8813387,"data_compressed":501066,"data_compressed_allocated":688128,"data_compressed_original":2875392,"omap_allocated":19142,"internal_metadata":365933882},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":139,"num_read_kb":69,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005776"},"pg_stats":[{"pgid":"3.3f","version":"99'8","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.832729+0000","last_change":"2024-06-03T03:19:41.626030+0000","last_active":"2024-06-03T03:22:54.832729+0000","last_peered":"2024-06-03T03:22:54.832729+0000","last_clean":"2024-06-03T03:22:54.832729+0000","last_became_active":"2024-06-03T03:19:41.625681+0000","last_became_peered":"2024-06-03T03:19:41.625681+0000","last_unstale":"2024-06-03T03:22:54.832729+0000","last_undegraded":"2024-06-03T03:22:54.832729+0000","last_fullsized":"2024-06-03T03:22:54.832729+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:35:46.396360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"99'4","reported_seq":42,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.830161+0000","last_change":"2024-06-03T03:19:41.619668+0000","last_active":"2024-06-03T03:22:54.830161+0000","last_peered":"2024-06-03T03:22:54.830161+0000","last_clean":"2024-06-03T03:22:54.830161+0000","last_became_active":"2024-06-03T03:19:41.619318+0000","last_became_peered":"2024-06-03T03:19:41.619318+0000","last_unstale":"2024-06-03T03:22:54.830161+0000","last_undegraded":"2024-06-03T03:22:54.830161+0000","last_fullsized":"2024-06-03T03:22:54.830161+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:07:21.656045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"99'7","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.832604+0000","last_change":"2024-06-03T03:19:41.623689+0000","last_active":"2024-06-03T03:22:54.832604+0000","last_peered":"2024-06-03T03:22:54.832604+0000","last_clean":"2024-06-03T03:22:54.832604+0000","last_became_active":"2024-06-03T03:19:41.623027+0000","last_became_peered":"2024-06-03T03:19:41.623027+0000","last_unstale":"2024-06-03T03:22:54.832604+0000","last_undegraded":"2024-06-03T03:22:54.832604+0000","last_fullsized":"2024-06-03T03:22:54.832604+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:18:05.584397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"99'8","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.814911+0000","last_change":"2024-06-03T03:19:41.603802+0000","last_active":"2024-06-03T03:22:54.814911+0000","last_peered":"2024-06-03T03:22:54.814911+0000","last_clean":"2024-06-03T03:22:54.814911+0000","last_became_active":"2024-06-03T03:19:41.603470+0000","last_became_peered":"2024-06-03T03:19:41.603470+0000","last_unstale":"2024-06-03T03:22:54.814911+0000","last_undegraded":"2024-06-03T03:22:54.814911+0000","last_fullsized":"2024-06-03T03:22:54.814911+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:56:10.636108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"99'4","reported_seq":42,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.826160+0000","last_change":"2024-06-03T03:19:41.636436+0000","last_active":"2024-06-03T03:22:54.826160+0000","last_peered":"2024-06-03T03:22:54.826160+0000","last_clean":"2024-06-03T03:22:54.826160+0000","last_became_active":"2024-06-03T03:19:41.636150+0000","last_became_peered":"2024-06-03T03:19:41.636150+0000","last_unstale":"2024-06-03T03:22:54.826160+0000","last_undegraded":"2024-06-03T03:22:54.826160+0000","last_fullsized":"2024-06-03T03:22:54.826160+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:02:10.928251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.808943+0000","last_change":"2024-06-03T03:19:41.622426+0000","last_active":"2024-06-03T03:22:54.808943+0000","last_peered":"2024-06-03T03:22:54.808943+0000","last_clean":"2024-06-03T03:22:54.808943+0000","last_became_active":"2024-06-03T03:19:41.622092+0000","last_became_peered":"2024-06-03T03:19:41.622092+0000","last_unstale":"2024-06-03T03:22:54.808943+0000","last_undegraded":"2024-06-03T03:22:54.808943+0000","last_fullsized":"2024-06-03T03:22:54.808943+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T14:08:52.666771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"99'5","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.825525+0000","last_change":"2024-06-03T03:19:41.608303+0000","last_active":"2024-06-03T03:22:54.825525+0000","last_peered":"2024-06-03T03:22:54.825525+0000","last_clean":"2024-06-03T03:22:54.825525+0000","last_became_active":"2024-06-03T03:19:41.607929+0000","last_became_peered":"2024-06-03T03:19:41.607929+0000","last_unstale":"2024-06-03T03:22:54.825525+0000","last_undegraded":"2024-06-03T03:22:54.825525+0000","last_fullsized":"2024-06-03T03:22:54.825525+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:40:05.643531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"99'9","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.822761+0000","last_change":"2024-06-03T03:19:41.606722+0000","last_active":"2024-06-03T03:22:54.822761+0000","last_peered":"2024-06-03T03:22:54.822761+0000","last_clean":"2024-06-03T03:22:54.822761+0000","last_became_active":"2024-06-03T03:19:41.606131+0000","last_became_peered":"2024-06-03T03:19:41.606131+0000","last_unstale":"2024-06-03T03:22:54.822761+0000","last_undegraded":"2024-06-03T03:22:54.822761+0000","last_fullsized":"2024-06-03T03:22:54.822761+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:09:17.306848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"99'11","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.813384+0000","last_change":"2024-06-03T03:19:41.609287+0000","last_active":"2024-06-03T03:22:54.813384+0000","last_peered":"2024-06-03T03:22:54.813384+0000","last_clean":"2024-06-03T03:22:54.813384+0000","last_became_active":"2024-06-03T03:19:41.608521+0000","last_became_peered":"2024-06-03T03:19:41.608521+0000","last_unstale":"2024-06-03T03:22:54.813384+0000","last_undegraded":"2024-06-03T03:22:54.813384+0000","last_fullsized":"2024-06-03T03:22:54.813384+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:35:37.128667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"99'16","reported_seq":55,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.814582+0000","last_change":"2024-06-03T03:19:41.605193+0000","last_active":"2024-06-03T03:22:54.814582+0000","last_peered":"2024-06-03T03:22:54.814582+0000","last_clean":"2024-06-03T03:22:54.814582+0000","last_became_active":"2024-06-03T03:19:41.604787+0000","last_became_peered":"2024-06-03T03:19:41.604787+0000","last_unstale":"2024-06-03T03:22:54.814582+0000","last_undegraded":"2024-06-03T03:22:54.814582+0000","last_fullsized":"2024-06-03T03:22:54.814582+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:02:48.334599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.817541+0000","last_change":"2024-06-03T03:19:41.611522+0000","last_active":"2024-06-03T03:22:54.817541+0000","last_peered":"2024-06-03T03:22:54.817541+0000","last_clean":"2024-06-03T03:22:54.817541+0000","last_became_active":"2024-06-03T03:19:41.610932+0000","last_became_peered":"2024-06-03T03:19:41.610932+0000","last_unstale":"2024-06-03T03:22:54.817541+0000","last_undegraded":"2024-06-03T03:22:54.817541+0000","last_fullsized":"2024-06-03T03:22:54.817541+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:35:44.702791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"99'7","reported_seq":23,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.815400+0000","last_change":"2024-06-03T03:20:25.950663+0000","last_active":"2024-06-03T03:22:54.815400+0000","last_peered":"2024-06-03T03:22:54.815400+0000","last_clean":"2024-06-03T03:22:54.815400+0000","last_became_active":"2024-06-03T03:20:25.949962+0000","last_became_peered":"2024-06-03T03:20:25.949962+0000","last_unstale":"2024-06-03T03:22:54.815400+0000","last_undegraded":"2024-06-03T03:22:54.815400+0000","last_fullsized":"2024-06-03T03:22:54.815400+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:55:33.972567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"99'10","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.839057+0000","last_change":"2024-06-03T03:19:41.633464+0000","last_active":"2024-06-03T03:22:54.839057+0000","last_peered":"2024-06-03T03:22:54.839057+0000","last_clean":"2024-06-03T03:22:54.839057+0000","last_became_active":"2024-06-03T03:19:41.630625+0000","last_became_peered":"2024-06-03T03:19:41.630625+0000","last_unstale":"2024-06-03T03:22:54.839057+0000","last_undegraded":"2024-06-03T03:22:54.839057+0000","last_fullsized":"2024-06-03T03:22:54.839057+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:23:28.799058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"99'7","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.831827+0000","last_change":"2024-06-03T03:19:41.627234+0000","last_active":"2024-06-03T03:22:54.831827+0000","last_peered":"2024-06-03T03:22:54.831827+0000","last_clean":"2024-06-03T03:22:54.831827+0000","last_became_active":"2024-06-03T03:19:41.626643+0000","last_became_peered":"2024-06-03T03:19:41.626643+0000","last_unstale":"2024-06-03T03:22:54.831827+0000","last_undegraded":"2024-06-03T03:22:54.831827+0000","last_fullsized":"2024-06-03T03:22:54.831827+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:37:44.126101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"99'7","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.831738+0000","last_change":"2024-06-03T03:19:41.621833+0000","last_active":"2024-06-03T03:22:54.831738+0000","last_peered":"2024-06-03T03:22:54.831738+0000","last_clean":"2024-06-03T03:22:54.831738+0000","last_became_active":"2024-06-03T03:19:41.620689+0000","last_became_peered":"2024-06-03T03:19:41.620689+0000","last_unstale":"2024-06-03T03:22:54.831738+0000","last_undegraded":"2024-06-03T03:22:54.831738+0000","last_fullsized":"2024-06-03T03:22:54.831738+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:39:31.188372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.830605+0000","last_change":"2024-06-03T03:19:41.624072+0000","last_active":"2024-06-03T03:22:54.830605+0000","last_peered":"2024-06-03T03:22:54.830605+0000","last_clean":"2024-06-03T03:22:54.830605+0000","last_became_active":"2024-06-03T03:19:41.623689+0000","last_became_peered":"2024-06-03T03:19:41.623689+0000","last_unstale":"2024-06-03T03:22:54.830605+0000","last_undegraded":"2024-06-03T03:22:54.830605+0000","last_fullsized":"2024-06-03T03:22:54.830605+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:38:24.075256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"99'8","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.832972+0000","last_change":"2024-06-03T03:19:41.626145+0000","last_active":"2024-06-03T03:22:54.832972+0000","last_peered":"2024-06-03T03:22:54.832972+0000","last_clean":"2024-06-03T03:22:54.832972+0000","last_became_active":"2024-06-03T03:19:41.625826+0000","last_became_peered":"2024-06-03T03:19:41.625826+0000","last_unstale":"2024-06-03T03:22:54.832972+0000","last_undegraded":"2024-06-03T03:22:54.832972+0000","last_fullsized":"2024-06-03T03:22:54.832972+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:04:10.996134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"99'10","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.820262+0000","last_change":"2024-06-03T03:19:41.615754+0000","last_active":"2024-06-03T03:22:54.820262+0000","last_peered":"2024-06-03T03:22:54.820262+0000","last_clean":"2024-06-03T03:22:54.820262+0000","last_became_active":"2024-06-03T03:19:41.614577+0000","last_became_peered":"2024-06-03T03:19:41.614577+0000","last_unstale":"2024-06-03T03:22:54.820262+0000","last_undegraded":"2024-06-03T03:22:54.820262+0000","last_fullsized":"2024-06-03T03:22:54.820262+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:45:50.084332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"99'7","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.826210+0000","last_change":"2024-06-03T03:19:41.633000+0000","last_active":"2024-06-03T03:22:54.826210+0000","last_peered":"2024-06-03T03:22:54.826210+0000","last_clean":"2024-06-03T03:22:54.826210+0000","last_became_active":"2024-06-03T03:19:41.632731+0000","last_became_peered":"2024-06-03T03:19:41.632731+0000","last_unstale":"2024-06-03T03:22:54.826210+0000","last_undegraded":"2024-06-03T03:22:54.826210+0000","last_fullsized":"2024-06-03T03:22:54.826210+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:10:29.990256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":36,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.052465+0000","last_change":"2024-06-03T03:20:26.133763+0000","last_active":"2024-06-03T03:21:26.052465+0000","last_peered":"2024-06-03T03:21:26.052465+0000","last_clean":"2024-06-03T03:21:26.052465+0000","last_became_active":"2024-06-03T03:20:26.133252+0000","last_became_peered":"2024-06-03T03:20:26.133252+0000","last_unstale":"2024-06-03T03:21:26.052465+0000","last_undegraded":"2024-06-03T03:21:26.052465+0000","last_fullsized":"2024-06-03T03:21:26.052465+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:33:21.461038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"97'13","reported_seq":187,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:23:14.294689+0000","last_change":"2024-06-03T03:19:36.542545+0000","last_active":"2024-06-03T03:23:14.294689+0000","last_peered":"2024-06-03T03:23:14.294689+0000","last_clean":"2024-06-03T03:23:14.294689+0000","last_became_active":"2024-06-03T03:19:36.541682+0000","last_became_peered":"2024-06-03T03:19:36.541682+0000","last_unstale":"2024-06-03T03:23:14.294689+0000","last_undegraded":"2024-06-03T03:23:14.294689+0000","last_fullsized":"2024-06-03T03:23:14.294689+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:06:23.909253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":129,"num_read_kb":129,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"99'5","reported_seq":43,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.815844+0000","last_change":"2024-06-03T03:19:41.605848+0000","last_active":"2024-06-03T03:22:54.815844+0000","last_peered":"2024-06-03T03:22:54.815844+0000","last_clean":"2024-06-03T03:22:54.815844+0000","last_became_active":"2024-06-03T03:19:41.603131+0000","last_became_peered":"2024-06-03T03:19:41.603131+0000","last_unstale":"2024-06-03T03:22:54.815844+0000","last_undegraded":"2024-06-03T03:22:54.815844+0000","last_fullsized":"2024-06-03T03:22:54.815844+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:07:38.931055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.980786+0000","last_change":"2024-06-03T03:19:56.923180+0000","last_active":"2024-06-03T03:21:25.980786+0000","last_peered":"2024-06-03T03:21:25.980786+0000","last_clean":"2024-06-03T03:21:25.980786+0000","last_became_active":"2024-06-03T03:19:56.922726+0000","last_became_peered":"2024-06-03T03:19:56.922726+0000","last_unstale":"2024-06-03T03:21:25.980786+0000","last_undegraded":"2024-06-03T03:21:25.980786+0000","last_fullsized":"2024-06-03T03:21:25.980786+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:18:45.727680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"97'2","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.058990+0000","last_change":"2024-06-03T03:19:36.746805+0000","last_active":"2024-06-03T03:21:26.058990+0000","last_peered":"2024-06-03T03:21:26.058990+0000","last_clean":"2024-06-03T03:21:26.058990+0000","last_became_active":"2024-06-03T03:19:36.745747+0000","last_became_peered":"2024-06-03T03:19:36.745747+0000","last_unstale":"2024-06-03T03:21:26.058990+0000","last_undegraded":"2024-06-03T03:21:26.058990+0000","last_fullsized":"2024-06-03T03:21:26.058990+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:54:28.427708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"99'6","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.832686+0000","last_change":"2024-06-03T03:19:41.630582+0000","last_active":"2024-06-03T03:22:54.832686+0000","last_peered":"2024-06-03T03:22:54.832686+0000","last_clean":"2024-06-03T03:22:54.832686+0000","last_became_active":"2024-06-03T03:19:41.630238+0000","last_became_peered":"2024-06-03T03:19:41.630238+0000","last_unstale":"2024-06-03T03:22:54.832686+0000","last_undegraded":"2024-06-03T03:22:54.832686+0000","last_fullsized":"2024-06-03T03:22:54.832686+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:49:58.039697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.980892+0000","last_change":"2024-06-03T03:19:56.925142+0000","last_active":"2024-06-03T03:21:25.980892+0000","last_peered":"2024-06-03T03:21:25.980892+0000","last_clean":"2024-06-03T03:21:25.980892+0000","last_became_active":"2024-06-03T03:19:56.924760+0000","last_became_peered":"2024-06-03T03:19:56.924760+0000","last_unstale":"2024-06-03T03:21:25.980892+0000","last_undegraded":"2024-06-03T03:21:25.980892+0000","last_fullsized":"2024-06-03T03:21:25.980892+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:11:27.537077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"97'12","reported_seq":57,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.974082+0000","last_change":"2024-06-03T03:19:36.543344+0000","last_active":"2024-06-03T03:21:25.974082+0000","last_peered":"2024-06-03T03:21:25.974082+0000","last_clean":"2024-06-03T03:21:25.974082+0000","last_became_active":"2024-06-03T03:19:36.542837+0000","last_became_peered":"2024-06-03T03:19:36.542837+0000","last_unstale":"2024-06-03T03:21:25.974082+0000","last_undegraded":"2024-06-03T03:21:25.974082+0000","last_fullsized":"2024-06-03T03:21:25.974082+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:55:05.451521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.812572+0000","last_change":"2024-06-03T03:19:41.629820+0000","last_active":"2024-06-03T03:22:54.812572+0000","last_peered":"2024-06-03T03:22:54.812572+0000","last_clean":"2024-06-03T03:22:54.812572+0000","last_became_active":"2024-06-03T03:19:41.629439+0000","last_became_peered":"2024-06-03T03:19:41.629439+0000","last_unstale":"2024-06-03T03:22:54.812572+0000","last_undegraded":"2024-06-03T03:22:54.812572+0000","last_fullsized":"2024-06-03T03:22:54.812572+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:05:27.438308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.980800+0000","last_change":"2024-06-03T03:19:56.923214+0000","last_active":"2024-06-03T03:21:25.980800+0000","last_peered":"2024-06-03T03:21:25.980800+0000","last_clean":"2024-06-03T03:21:25.980800+0000","last_became_active":"2024-06-03T03:19:56.922751+0000","last_became_peered":"2024-06-03T03:19:56.922751+0000","last_unstale":"2024-06-03T03:21:25.980800+0000","last_undegraded":"2024-06-03T03:21:25.980800+0000","last_fullsized":"2024-06-03T03:21:25.980800+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:05:59.249818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"97'5","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.973910+0000","last_change":"2024-06-03T03:19:36.538141+0000","last_active":"2024-06-03T03:21:25.973910+0000","last_peered":"2024-06-03T03:21:25.973910+0000","last_clean":"2024-06-03T03:21:25.973910+0000","last_became_active":"2024-06-03T03:19:36.537786+0000","last_became_peered":"2024-06-03T03:19:36.537786+0000","last_unstale":"2024-06-03T03:21:25.973910+0000","last_undegraded":"2024-06-03T03:21:25.973910+0000","last_fullsized":"2024-06-03T03:21:25.973910+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T15:03:32.859097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"99'15","reported_seq":54,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.816371+0000","last_change":"2024-06-03T03:19:41.608471+0000","last_active":"2024-06-03T03:22:54.816371+0000","last_peered":"2024-06-03T03:22:54.816371+0000","last_clean":"2024-06-03T03:22:54.816371+0000","last_became_active":"2024-06-03T03:19:41.608096+0000","last_became_peered":"2024-06-03T03:19:41.608096+0000","last_unstale":"2024-06-03T03:22:54.816371+0000","last_undegraded":"2024-06-03T03:22:54.816371+0000","last_fullsized":"2024-06-03T03:22:54.816371+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:31:10.541803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.053026+0000","last_change":"2024-06-03T03:19:56.915540+0000","last_active":"2024-06-03T03:21:26.053026+0000","last_peered":"2024-06-03T03:21:26.053026+0000","last_clean":"2024-06-03T03:21:26.053026+0000","last_became_active":"2024-06-03T03:19:56.915017+0000","last_became_peered":"2024-06-03T03:19:56.915017+0000","last_unstale":"2024-06-03T03:21:26.053026+0000","last_undegraded":"2024-06-03T03:21:26.053026+0000","last_fullsized":"2024-06-03T03:21:26.053026+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:56:23.158344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"99'7","reported_seq":186,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:23:16.322672+0000","last_change":"2024-06-03T03:19:36.542337+0000","last_active":"2024-06-03T03:23:16.322672+0000","last_peered":"2024-06-03T03:23:16.322672+0000","last_clean":"2024-06-03T03:23:16.322672+0000","last_became_active":"2024-06-03T03:19:36.541990+0000","last_became_peered":"2024-06-03T03:19:36.541990+0000","last_unstale":"2024-06-03T03:23:16.322672+0000","last_undegraded":"2024-06-03T03:23:16.322672+0000","last_fullsized":"2024-06-03T03:23:16.322672+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:08:15.656321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1153,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":134,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"99'7","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.815200+0000","last_change":"2024-06-03T03:19:41.616293+0000","last_active":"2024-06-03T03:22:54.815200+0000","last_peered":"2024-06-03T03:22:54.815200+0000","last_clean":"2024-06-03T03:22:54.815200+0000","last_became_active":"2024-06-03T03:19:41.615946+0000","last_became_peered":"2024-06-03T03:19:41.615946+0000","last_unstale":"2024-06-03T03:22:54.815200+0000","last_undegraded":"2024-06-03T03:22:54.815200+0000","last_fullsized":"2024-06-03T03:22:54.815200+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:18:43.835977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":36,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.053631+0000","last_change":"2024-06-03T03:20:26.146237+0000","last_active":"2024-06-03T03:21:26.053631+0000","last_peered":"2024-06-03T03:21:26.053631+0000","last_clean":"2024-06-03T03:21:26.053631+0000","last_became_active":"2024-06-03T03:20:26.145731+0000","last_became_peered":"2024-06-03T03:20:26.145731+0000","last_unstale":"2024-06-03T03:21:26.053631+0000","last_undegraded":"2024-06-03T03:21:26.053631+0000","last_fullsized":"2024-06-03T03:21:26.053631+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:15:29.723833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"97'7","reported_seq":52,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.053550+0000","last_change":"2024-06-03T03:19:36.546501+0000","last_active":"2024-06-03T03:21:26.053550+0000","last_peered":"2024-06-03T03:21:26.053550+0000","last_clean":"2024-06-03T03:21:26.053550+0000","last_became_active":"2024-06-03T03:19:36.546079+0000","last_became_peered":"2024-06-03T03:19:36.546079+0000","last_unstale":"2024-06-03T03:21:26.053550+0000","last_undegraded":"2024-06-03T03:21:26.053550+0000","last_fullsized":"2024-06-03T03:21:26.053550+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:16:39.473141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"99'8","reported_seq":53,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.829062+0000","last_change":"2024-06-03T03:20:26.149410+0000","last_active":"2024-06-03T03:22:54.829062+0000","last_peered":"2024-06-03T03:22:54.829062+0000","last_clean":"2024-06-03T03:22:54.829062+0000","last_became_active":"2024-06-03T03:20:26.149084+0000","last_became_peered":"2024-06-03T03:20:26.149084+0000","last_unstale":"2024-06-03T03:22:54.829062+0000","last_undegraded":"2024-06-03T03:22:54.829062+0000","last_fullsized":"2024-06-03T03:22:54.829062+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:06:22.316449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.054192+0000","last_change":"2024-06-03T03:19:56.926498+0000","last_active":"2024-06-03T03:21:26.054192+0000","last_peered":"2024-06-03T03:21:26.054192+0000","last_clean":"2024-06-03T03:21:26.054192+0000","last_became_active":"2024-06-03T03:19:56.926101+0000","last_became_peered":"2024-06-03T03:19:56.926101+0000","last_unstale":"2024-06-03T03:21:26.054192+0000","last_undegraded":"2024-06-03T03:21:26.054192+0000","last_fullsized":"2024-06-03T03:21:26.054192+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:00:12.024443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"97'3","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.977399+0000","last_change":"2024-06-03T03:19:36.553151+0000","last_active":"2024-06-03T03:21:25.977399+0000","last_peered":"2024-06-03T03:21:25.977399+0000","last_clean":"2024-06-03T03:21:25.977399+0000","last_became_active":"2024-06-03T03:19:36.551914+0000","last_became_peered":"2024-06-03T03:19:36.551914+0000","last_unstale":"2024-06-03T03:21:25.977399+0000","last_undegraded":"2024-06-03T03:21:25.977399+0000","last_fullsized":"2024-06-03T03:21:25.977399+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:00:42.053922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"99'8","reported_seq":24,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.824233+0000","last_change":"2024-06-03T03:20:26.147909+0000","last_active":"2024-06-03T03:22:54.824233+0000","last_peered":"2024-06-03T03:22:54.824233+0000","last_clean":"2024-06-03T03:22:54.824233+0000","last_became_active":"2024-06-03T03:20:26.146448+0000","last_became_peered":"2024-06-03T03:20:26.146448+0000","last_unstale":"2024-06-03T03:22:54.824233+0000","last_undegraded":"2024-06-03T03:22:54.824233+0000","last_fullsized":"2024-06-03T03:22:54.824233+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:43:30.878472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.976693+0000","last_change":"2024-06-03T03:19:56.941910+0000","last_active":"2024-06-03T03:21:25.976693+0000","last_peered":"2024-06-03T03:21:25.976693+0000","last_clean":"2024-06-03T03:21:25.976693+0000","last_became_active":"2024-06-03T03:19:56.940963+0000","last_became_peered":"2024-06-03T03:19:56.940963+0000","last_unstale":"2024-06-03T03:21:25.976693+0000","last_undegraded":"2024-06-03T03:21:25.976693+0000","last_fullsized":"2024-06-03T03:21:25.976693+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T14:18:32.109231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"97'3","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.052977+0000","last_change":"2024-06-03T03:19:36.538928+0000","last_active":"2024-06-03T03:21:26.052977+0000","last_peered":"2024-06-03T03:21:26.052977+0000","last_clean":"2024-06-03T03:21:26.052977+0000","last_became_active":"2024-06-03T03:19:36.538580+0000","last_became_peered":"2024-06-03T03:19:36.538580+0000","last_unstale":"2024-06-03T03:21:26.052977+0000","last_undegraded":"2024-06-03T03:21:26.052977+0000","last_fullsized":"2024-06-03T03:21:26.052977+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:08:05.906257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"99'11","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.829145+0000","last_change":"2024-06-03T03:19:41.623051+0000","last_active":"2024-06-03T03:22:54.829145+0000","last_peered":"2024-06-03T03:22:54.829145+0000","last_clean":"2024-06-03T03:22:54.829145+0000","last_became_active":"2024-06-03T03:19:41.622656+0000","last_became_peered":"2024-06-03T03:19:41.622656+0000","last_unstale":"2024-06-03T03:22:54.829145+0000","last_undegraded":"2024-06-03T03:22:54.829145+0000","last_fullsized":"2024-06-03T03:22:54.829145+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:11:34.587723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.056378+0000","last_change":"2024-06-03T03:19:56.921215+0000","last_active":"2024-06-03T03:21:26.056378+0000","last_peered":"2024-06-03T03:21:26.056378+0000","last_clean":"2024-06-03T03:21:26.056378+0000","last_became_active":"2024-06-03T03:19:56.920658+0000","last_became_peered":"2024-06-03T03:19:56.920658+0000","last_unstale":"2024-06-03T03:21:26.056378+0000","last_undegraded":"2024-06-03T03:21:26.056378+0000","last_fullsized":"2024-06-03T03:21:26.056378+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T14:04:54.163818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"99'5","reported_seq":308,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:23:16.243936+0000","last_change":"2024-06-03T03:19:36.545245+0000","last_active":"2024-06-03T03:23:16.243936+0000","last_peered":"2024-06-03T03:23:16.243936+0000","last_clean":"2024-06-03T03:23:16.243936+0000","last_became_active":"2024-06-03T03:19:36.544821+0000","last_became_peered":"2024-06-03T03:19:36.544821+0000","last_unstale":"2024-06-03T03:23:16.243936+0000","last_undegraded":"2024-06-03T03:23:16.243936+0000","last_fullsized":"2024-06-03T03:23:16.243936+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:55:24.011925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":258,"num_read_kb":258,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"99'10","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.822416+0000","last_change":"2024-06-03T03:19:41.616384+0000","last_active":"2024-06-03T03:22:54.822416+0000","last_peered":"2024-06-03T03:22:54.822416+0000","last_clean":"2024-06-03T03:22:54.822416+0000","last_became_active":"2024-06-03T03:19:41.616034+0000","last_became_peered":"2024-06-03T03:19:41.616034+0000","last_unstale":"2024-06-03T03:22:54.822416+0000","last_undegraded":"2024-06-03T03:22:54.822416+0000","last_fullsized":"2024-06-03T03:22:54.822416+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:10:06.823375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.054539+0000","last_change":"2024-06-03T03:19:56.933045+0000","last_active":"2024-06-03T03:21:26.054539+0000","last_peered":"2024-06-03T03:21:26.054539+0000","last_clean":"2024-06-03T03:21:26.054539+0000","last_became_active":"2024-06-03T03:19:56.928956+0000","last_became_peered":"2024-06-03T03:19:56.928956+0000","last_unstale":"2024-06-03T03:21:26.054539+0000","last_undegraded":"2024-06-03T03:21:26.054539+0000","last_fullsized":"2024-06-03T03:21:26.054539+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:37:44.270511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"97'4","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.053400+0000","last_change":"2024-06-03T03:19:36.541559+0000","last_active":"2024-06-03T03:21:26.053400+0000","last_peered":"2024-06-03T03:21:26.053400+0000","last_clean":"2024-06-03T03:21:26.053400+0000","last_became_active":"2024-06-03T03:19:36.541166+0000","last_became_peered":"2024-06-03T03:19:36.541166+0000","last_unstale":"2024-06-03T03:21:26.053400+0000","last_undegraded":"2024-06-03T03:21:26.053400+0000","last_fullsized":"2024-06-03T03:21:26.053400+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T14:49:26.847713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"99'9","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.832063+0000","last_change":"2024-06-03T03:19:41.629789+0000","last_active":"2024-06-03T03:22:54.832063+0000","last_peered":"2024-06-03T03:22:54.832063+0000","last_clean":"2024-06-03T03:22:54.832063+0000","last_became_active":"2024-06-03T03:19:41.628026+0000","last_became_peered":"2024-06-03T03:19:41.628026+0000","last_unstale":"2024-06-03T03:22:54.832063+0000","last_undegraded":"2024-06-03T03:22:54.832063+0000","last_fullsized":"2024-06-03T03:22:54.832063+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:56:26.411907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":36,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.056621+0000","last_change":"2024-06-03T03:21:26.056114+0000","last_active":"2024-06-03T03:21:26.056621+0000","last_peered":"2024-06-03T03:21:26.056621+0000","last_clean":"2024-06-03T03:21:26.056621+0000","last_became_active":"2024-06-03T03:21:26.055505+0000","last_became_peered":"2024-06-03T03:21:26.055505+0000","last_unstale":"2024-06-03T03:21:26.056621+0000","last_undegraded":"2024-06-03T03:21:26.056621+0000","last_fullsized":"2024-06-03T03:21:26.056621+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:22:55.722566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,10],"acting":[4,1,2,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"97'5","reported_seq":51,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.053544+0000","last_change":"2024-06-03T03:19:36.744930+0000","last_active":"2024-06-03T03:21:26.053544+0000","last_peered":"2024-06-03T03:21:26.053544+0000","last_clean":"2024-06-03T03:21:26.053544+0000","last_became_active":"2024-06-03T03:19:36.744567+0000","last_became_peered":"2024-06-03T03:19:36.744567+0000","last_unstale":"2024-06-03T03:21:26.053544+0000","last_undegraded":"2024-06-03T03:21:26.053544+0000","last_fullsized":"2024-06-03T03:21:26.053544+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:35:31.927211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"99'11","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.812434+0000","last_change":"2024-06-03T03:19:41.607604+0000","last_active":"2024-06-03T03:22:54.812434+0000","last_peered":"2024-06-03T03:22:54.812434+0000","last_clean":"2024-06-03T03:22:54.812434+0000","last_became_active":"2024-06-03T03:19:41.607148+0000","last_became_peered":"2024-06-03T03:19:41.607148+0000","last_unstale":"2024-06-03T03:22:54.812434+0000","last_undegraded":"2024-06-03T03:22:54.812434+0000","last_fullsized":"2024-06-03T03:22:54.812434+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:44:44.037836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.056250+0000","last_change":"2024-06-03T03:19:56.928689+0000","last_active":"2024-06-03T03:21:26.056250+0000","last_peered":"2024-06-03T03:21:26.056250+0000","last_clean":"2024-06-03T03:21:26.056250+0000","last_became_active":"2024-06-03T03:19:56.924955+0000","last_became_peered":"2024-06-03T03:19:56.924955+0000","last_unstale":"2024-06-03T03:21:26.056250+0000","last_undegraded":"2024-06-03T03:21:26.056250+0000","last_fullsized":"2024-06-03T03:21:26.056250+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:11:07.413430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"99'42","reported_seq":357,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:23:16.243779+0000","last_change":"2024-06-03T03:19:36.539688+0000","last_active":"2024-06-03T03:23:16.243779+0000","last_peered":"2024-06-03T03:23:16.243779+0000","last_clean":"2024-06-03T03:23:16.243779+0000","last_became_active":"2024-06-03T03:19:36.539302+0000","last_became_peered":"2024-06-03T03:19:36.539302+0000","last_unstale":"2024-06-03T03:23:16.243779+0000","last_undegraded":"2024-06-03T03:23:16.243779+0000","last_fullsized":"2024-06-03T03:23:16.243779+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:28:52.790227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69204,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":270,"num_read_kb":203,"num_write":44,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"99'6","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.815310+0000","last_change":"2024-06-03T03:19:41.608202+0000","last_active":"2024-06-03T03:22:54.815310+0000","last_peered":"2024-06-03T03:22:54.815310+0000","last_clean":"2024-06-03T03:22:54.815310+0000","last_became_active":"2024-06-03T03:19:41.607891+0000","last_became_peered":"2024-06-03T03:19:41.607891+0000","last_unstale":"2024-06-03T03:22:54.815310+0000","last_undegraded":"2024-06-03T03:22:54.815310+0000","last_fullsized":"2024-06-03T03:22:54.815310+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T14:10:44.042892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.054202+0000","last_change":"2024-06-03T03:19:56.923547+0000","last_active":"2024-06-03T03:21:26.054202+0000","last_peered":"2024-06-03T03:21:26.054202+0000","last_clean":"2024-06-03T03:21:26.054202+0000","last_became_active":"2024-06-03T03:19:56.923046+0000","last_became_peered":"2024-06-03T03:19:56.923046+0000","last_unstale":"2024-06-03T03:21:26.054202+0000","last_undegraded":"2024-06-03T03:21:26.054202+0000","last_fullsized":"2024-06-03T03:21:26.054202+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:32:19.438793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"97'9","reported_seq":55,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.053870+0000","last_change":"2024-06-03T03:19:36.547024+0000","last_active":"2024-06-03T03:21:26.053870+0000","last_peered":"2024-06-03T03:21:26.053870+0000","last_clean":"2024-06-03T03:21:26.053870+0000","last_became_active":"2024-06-03T03:19:36.546663+0000","last_became_peered":"2024-06-03T03:19:36.546663+0000","last_unstale":"2024-06-03T03:21:26.053870+0000","last_undegraded":"2024-06-03T03:21:26.053870+0000","last_fullsized":"2024-06-03T03:21:26.053870+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:58:41.446730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"99'9","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.830440+0000","last_change":"2024-06-03T03:19:41.630119+0000","last_active":"2024-06-03T03:22:54.830440+0000","last_peered":"2024-06-03T03:22:54.830440+0000","last_clean":"2024-06-03T03:22:54.830440+0000","last_became_active":"2024-06-03T03:19:41.629563+0000","last_became_peered":"2024-06-03T03:19:41.629563+0000","last_unstale":"2024-06-03T03:22:54.830440+0000","last_undegraded":"2024-06-03T03:22:54.830440+0000","last_fullsized":"2024-06-03T03:22:54.830440+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:49:33.983471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.053797+0000","last_change":"2024-06-03T03:19:56.941451+0000","last_active":"2024-06-03T03:21:26.053797+0000","last_peered":"2024-06-03T03:21:26.053797+0000","last_clean":"2024-06-03T03:21:26.053797+0000","last_became_active":"2024-06-03T03:19:56.941003+0000","last_became_peered":"2024-06-03T03:19:56.941003+0000","last_unstale":"2024-06-03T03:21:26.053797+0000","last_undegraded":"2024-06-03T03:21:26.053797+0000","last_fullsized":"2024-06-03T03:21:26.053797+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:28:58.505495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"97'5","reported_seq":51,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.053452+0000","last_change":"2024-06-03T03:19:36.547128+0000","last_active":"2024-06-03T03:21:26.053452+0000","last_peered":"2024-06-03T03:21:26.053452+0000","last_clean":"2024-06-03T03:21:26.053452+0000","last_became_active":"2024-06-03T03:19:36.546576+0000","last_became_peered":"2024-06-03T03:19:36.546576+0000","last_unstale":"2024-06-03T03:21:26.053452+0000","last_undegraded":"2024-06-03T03:21:26.053452+0000","last_fullsized":"2024-06-03T03:21:26.053452+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:23:30.196543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"99'15","reported_seq":54,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.836443+0000","last_change":"2024-06-03T03:19:41.634941+0000","last_active":"2024-06-03T03:22:54.836443+0000","last_peered":"2024-06-03T03:22:54.836443+0000","last_clean":"2024-06-03T03:22:54.836443+0000","last_became_active":"2024-06-03T03:19:41.634561+0000","last_became_peered":"2024-06-03T03:19:41.634561+0000","last_unstale":"2024-06-03T03:22:54.836443+0000","last_undegraded":"2024-06-03T03:22:54.836443+0000","last_fullsized":"2024-06-03T03:22:54.836443+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:00:27.258078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.056748+0000","last_change":"2024-06-03T03:19:56.934612+0000","last_active":"2024-06-03T03:21:26.056748+0000","last_peered":"2024-06-03T03:21:26.056748+0000","last_clean":"2024-06-03T03:21:26.056748+0000","last_became_active":"2024-06-03T03:19:56.930212+0000","last_became_peered":"2024-06-03T03:19:56.930212+0000","last_unstale":"2024-06-03T03:21:26.056748+0000","last_undegraded":"2024-06-03T03:21:26.056748+0000","last_fullsized":"2024-06-03T03:21:26.056748+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:42:38.490095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"97'5","reported_seq":181,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:23:16.298951+0000","last_change":"2024-06-03T03:19:36.542052+0000","last_active":"2024-06-03T03:23:16.298951+0000","last_peered":"2024-06-03T03:23:16.298951+0000","last_clean":"2024-06-03T03:23:16.298951+0000","last_became_active":"2024-06-03T03:19:36.540987+0000","last_became_peered":"2024-06-03T03:19:36.540987+0000","last_unstale":"2024-06-03T03:23:16.298951+0000","last_undegraded":"2024-06-03T03:23:16.298951+0000","last_fullsized":"2024-06-03T03:23:16.298951+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T15:18:48.821400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":133,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.818221+0000","last_change":"2024-06-03T03:19:41.607610+0000","last_active":"2024-06-03T03:22:54.818221+0000","last_peered":"2024-06-03T03:22:54.818221+0000","last_clean":"2024-06-03T03:22:54.818221+0000","last_became_active":"2024-06-03T03:19:41.607219+0000","last_became_peered":"2024-06-03T03:19:41.607219+0000","last_unstale":"2024-06-03T03:22:54.818221+0000","last_undegraded":"2024-06-03T03:22:54.818221+0000","last_fullsized":"2024-06-03T03:22:54.818221+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:06:20.707623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.979965+0000","last_change":"2024-06-03T03:19:56.936440+0000","last_active":"2024-06-03T03:21:25.979965+0000","last_peered":"2024-06-03T03:21:25.979965+0000","last_clean":"2024-06-03T03:21:25.979965+0000","last_became_active":"2024-06-03T03:19:56.936048+0000","last_became_peered":"2024-06-03T03:19:56.936048+0000","last_unstale":"2024-06-03T03:21:25.979965+0000","last_undegraded":"2024-06-03T03:21:25.979965+0000","last_fullsized":"2024-06-03T03:21:25.979965+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:22:15.758950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"97'17","reported_seq":62,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.973400+0000","last_change":"2024-06-03T03:19:36.736105+0000","last_active":"2024-06-03T03:21:25.973400+0000","last_peered":"2024-06-03T03:21:25.973400+0000","last_clean":"2024-06-03T03:21:25.973400+0000","last_became_active":"2024-06-03T03:19:36.735663+0000","last_became_peered":"2024-06-03T03:19:36.735663+0000","last_unstale":"2024-06-03T03:21:25.973400+0000","last_undegraded":"2024-06-03T03:21:25.973400+0000","last_fullsized":"2024-06-03T03:21:25.973400+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:41:46.216923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.813317+0000","last_change":"2024-06-03T03:19:41.609758+0000","last_active":"2024-06-03T03:22:54.813317+0000","last_peered":"2024-06-03T03:22:54.813317+0000","last_clean":"2024-06-03T03:22:54.813317+0000","last_became_active":"2024-06-03T03:19:41.609421+0000","last_became_peered":"2024-06-03T03:19:41.609421+0000","last_unstale":"2024-06-03T03:22:54.813317+0000","last_undegraded":"2024-06-03T03:22:54.813317+0000","last_fullsized":"2024-06-03T03:22:54.813317+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:45:54.919352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.052841+0000","last_change":"2024-06-03T03:19:56.925422+0000","last_active":"2024-06-03T03:21:26.052841+0000","last_peered":"2024-06-03T03:21:26.052841+0000","last_clean":"2024-06-03T03:21:26.052841+0000","last_became_active":"2024-06-03T03:19:56.924891+0000","last_became_peered":"2024-06-03T03:19:56.924891+0000","last_unstale":"2024-06-03T03:21:26.052841+0000","last_undegraded":"2024-06-03T03:21:26.052841+0000","last_fullsized":"2024-06-03T03:21:26.052841+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:34:59.986256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"97'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.973924+0000","last_change":"2024-06-03T03:19:36.540605+0000","last_active":"2024-06-03T03:21:25.973924+0000","last_peered":"2024-06-03T03:21:25.973924+0000","last_clean":"2024-06-03T03:21:25.973924+0000","last_became_active":"2024-06-03T03:19:36.540180+0000","last_became_peered":"2024-06-03T03:19:36.540180+0000","last_unstale":"2024-06-03T03:21:25.973924+0000","last_undegraded":"2024-06-03T03:21:25.973924+0000","last_fullsized":"2024-06-03T03:21:25.973924+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:01:26.556726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"99'5","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.825595+0000","last_change":"2024-06-03T03:20:25.904215+0000","last_active":"2024-06-03T03:22:54.825595+0000","last_peered":"2024-06-03T03:22:54.825595+0000","last_clean":"2024-06-03T03:22:54.825595+0000","last_became_active":"2024-06-03T03:20:25.903894+0000","last_became_peered":"2024-06-03T03:20:25.903894+0000","last_unstale":"2024-06-03T03:22:54.825595+0000","last_undegraded":"2024-06-03T03:22:54.825595+0000","last_fullsized":"2024-06-03T03:22:54.825595+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:02:07.587159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.973525+0000","last_change":"2024-06-03T03:19:56.910041+0000","last_active":"2024-06-03T03:21:25.973525+0000","last_peered":"2024-06-03T03:21:25.973525+0000","last_clean":"2024-06-03T03:21:25.973525+0000","last_became_active":"2024-06-03T03:19:56.909591+0000","last_became_peered":"2024-06-03T03:19:56.909591+0000","last_unstale":"2024-06-03T03:21:25.973525+0000","last_undegraded":"2024-06-03T03:21:25.973525+0000","last_fullsized":"2024-06-03T03:21:25.973525+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:42:20.016838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"97'1","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.055897+0000","last_change":"2024-06-03T03:19:36.741731+0000","last_active":"2024-06-03T03:21:26.055897+0000","last_peered":"2024-06-03T03:21:26.055897+0000","last_clean":"2024-06-03T03:21:26.055897+0000","last_became_active":"2024-06-03T03:19:36.741329+0000","last_became_peered":"2024-06-03T03:19:36.741329+0000","last_unstale":"2024-06-03T03:21:26.055897+0000","last_undegraded":"2024-06-03T03:21:26.055897+0000","last_fullsized":"2024-06-03T03:21:26.055897+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:02:45.778368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"99'4","reported_seq":43,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.825877+0000","last_change":"2024-06-03T03:19:41.632225+0000","last_active":"2024-06-03T03:22:54.825877+0000","last_peered":"2024-06-03T03:22:54.825877+0000","last_clean":"2024-06-03T03:22:54.825877+0000","last_became_active":"2024-06-03T03:19:41.631851+0000","last_became_peered":"2024-06-03T03:19:41.631851+0000","last_unstale":"2024-06-03T03:22:54.825877+0000","last_undegraded":"2024-06-03T03:22:54.825877+0000","last_fullsized":"2024-06-03T03:22:54.825877+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:52:23.364233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.977987+0000","last_change":"2024-06-03T03:19:56.934200+0000","last_active":"2024-06-03T03:21:25.977987+0000","last_peered":"2024-06-03T03:21:25.977987+0000","last_clean":"2024-06-03T03:21:25.977987+0000","last_became_active":"2024-06-03T03:19:56.932998+0000","last_became_peered":"2024-06-03T03:19:56.932998+0000","last_unstale":"2024-06-03T03:21:25.977987+0000","last_undegraded":"2024-06-03T03:21:25.977987+0000","last_fullsized":"2024-06-03T03:21:25.977987+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:45:28.916023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"97'5","reported_seq":178,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:23:13.293425+0000","last_change":"2024-06-03T03:19:36.546630+0000","last_active":"2024-06-03T03:23:13.293425+0000","last_peered":"2024-06-03T03:23:13.293425+0000","last_clean":"2024-06-03T03:23:13.293425+0000","last_became_active":"2024-06-03T03:19:36.546248+0000","last_became_peered":"2024-06-03T03:19:36.546248+0000","last_unstale":"2024-06-03T03:23:13.293425+0000","last_undegraded":"2024-06-03T03:23:13.293425+0000","last_fullsized":"2024-06-03T03:23:13.293425+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T15:09:50.323113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":128,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.830992+0000","last_change":"2024-06-03T03:19:41.624584+0000","last_active":"2024-06-03T03:22:54.830992+0000","last_peered":"2024-06-03T03:22:54.830992+0000","last_clean":"2024-06-03T03:22:54.830992+0000","last_became_active":"2024-06-03T03:19:41.624216+0000","last_became_peered":"2024-06-03T03:19:41.624216+0000","last_unstale":"2024-06-03T03:22:54.830992+0000","last_undegraded":"2024-06-03T03:22:54.830992+0000","last_fullsized":"2024-06-03T03:22:54.830992+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:22:09.645341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.979986+0000","last_change":"2024-06-03T03:19:56.937422+0000","last_active":"2024-06-03T03:21:25.979986+0000","last_peered":"2024-06-03T03:21:25.979986+0000","last_clean":"2024-06-03T03:21:25.979986+0000","last_became_active":"2024-06-03T03:19:56.934075+0000","last_became_peered":"2024-06-03T03:19:56.934075+0000","last_unstale":"2024-06-03T03:21:25.979986+0000","last_undegraded":"2024-06-03T03:21:25.979986+0000","last_fullsized":"2024-06-03T03:21:25.979986+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:25:45.273161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.814063+0000","last_change":"2024-06-03T03:19:41.589595+0000","last_active":"2024-06-03T03:22:54.814063+0000","last_peered":"2024-06-03T03:22:54.814063+0000","last_clean":"2024-06-03T03:22:54.814063+0000","last_became_active":"2024-06-03T03:19:41.589251+0000","last_became_peered":"2024-06-03T03:19:41.589251+0000","last_unstale":"2024-06-03T03:22:54.814063+0000","last_undegraded":"2024-06-03T03:22:54.814063+0000","last_fullsized":"2024-06-03T03:22:54.814063+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:51:40.417386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"97'5","reported_seq":180,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:23:16.298401+0000","last_change":"2024-06-03T03:19:36.737523+0000","last_active":"2024-06-03T03:23:16.298401+0000","last_peered":"2024-06-03T03:23:16.298401+0000","last_clean":"2024-06-03T03:23:16.298401+0000","last_became_active":"2024-06-03T03:19:36.736698+0000","last_became_peered":"2024-06-03T03:19:36.736698+0000","last_unstale":"2024-06-03T03:23:16.298401+0000","last_undegraded":"2024-06-03T03:23:16.298401+0000","last_fullsized":"2024-06-03T03:23:16.298401+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:14:46.330549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":131,"num_read_kb":131,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"97'1","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.052814+0000","last_change":"2024-06-03T03:19:56.924421+0000","last_active":"2024-06-03T03:21:26.052814+0000","last_peered":"2024-06-03T03:21:26.052814+0000","last_clean":"2024-06-03T03:21:26.052814+0000","last_became_active":"2024-06-03T03:19:56.924043+0000","last_became_peered":"2024-06-03T03:19:56.924043+0000","last_unstale":"2024-06-03T03:21:26.052814+0000","last_undegraded":"2024-06-03T03:21:26.052814+0000","last_fullsized":"2024-06-03T03:21:26.052814+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:00:47.674717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"99'8","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.826754+0000","last_change":"2024-06-03T03:19:41.613800+0000","last_active":"2024-06-03T03:22:54.826754+0000","last_peered":"2024-06-03T03:22:54.826754+0000","last_clean":"2024-06-03T03:22:54.826754+0000","last_became_active":"2024-06-03T03:19:41.613387+0000","last_became_peered":"2024-06-03T03:19:41.613387+0000","last_unstale":"2024-06-03T03:22:54.826754+0000","last_undegraded":"2024-06-03T03:22:54.826754+0000","last_fullsized":"2024-06-03T03:22:54.826754+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:04:04.785023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"97'6","reported_seq":53,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.980987+0000","last_change":"2024-06-03T03:19:36.544518+0000","last_active":"2024-06-03T03:21:25.980987+0000","last_peered":"2024-06-03T03:21:25.980987+0000","last_clean":"2024-06-03T03:21:25.980987+0000","last_became_active":"2024-06-03T03:19:36.544056+0000","last_became_peered":"2024-06-03T03:19:36.544056+0000","last_unstale":"2024-06-03T03:21:25.980987+0000","last_undegraded":"2024-06-03T03:21:25.980987+0000","last_fullsized":"2024-06-03T03:21:25.980987+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:33:25.012986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":367,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.052323+0000","last_change":"2024-06-03T03:18:05.481351+0000","last_active":"2024-06-03T03:21:26.052323+0000","last_peered":"2024-06-03T03:21:26.052323+0000","last_clean":"2024-06-03T03:21:26.052323+0000","last_became_active":"2024-06-03T03:18:05.480929+0000","last_became_peered":"2024-06-03T03:18:05.480929+0000","last_unstale":"2024-06-03T03:21:26.052323+0000","last_undegraded":"2024-06-03T03:21:26.052323+0000","last_fullsized":"2024-06-03T03:21:26.052323+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:14:09.558597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:14:09.558597+0000","last_clean_scrub_stamp":"2024-06-03T03:14:09.558597+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:54:45.563733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.054351+0000","last_change":"2024-06-03T03:19:56.931867+0000","last_active":"2024-06-03T03:21:26.054351+0000","last_peered":"2024-06-03T03:21:26.054351+0000","last_clean":"2024-06-03T03:21:26.054351+0000","last_became_active":"2024-06-03T03:19:56.931538+0000","last_became_peered":"2024-06-03T03:19:56.931538+0000","last_unstale":"2024-06-03T03:21:26.054351+0000","last_undegraded":"2024-06-03T03:21:26.054351+0000","last_fullsized":"2024-06-03T03:21:26.054351+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:55:15.144192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"99'5","reported_seq":43,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.813951+0000","last_change":"2024-06-03T03:19:41.622713+0000","last_active":"2024-06-03T03:22:54.813951+0000","last_peered":"2024-06-03T03:22:54.813951+0000","last_clean":"2024-06-03T03:22:54.813951+0000","last_became_active":"2024-06-03T03:19:41.622363+0000","last_became_peered":"2024-06-03T03:19:41.622363+0000","last_unstale":"2024-06-03T03:22:54.813951+0000","last_undegraded":"2024-06-03T03:22:54.813951+0000","last_fullsized":"2024-06-03T03:22:54.813951+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:43:57.671797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"97'4","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.052450+0000","last_change":"2024-06-03T03:19:36.539663+0000","last_active":"2024-06-03T03:21:26.052450+0000","last_peered":"2024-06-03T03:21:26.052450+0000","last_clean":"2024-06-03T03:21:26.052450+0000","last_became_active":"2024-06-03T03:19:36.539320+0000","last_became_peered":"2024-06-03T03:19:36.539320+0000","last_unstale":"2024-06-03T03:21:26.052450+0000","last_undegraded":"2024-06-03T03:21:26.052450+0000","last_fullsized":"2024-06-03T03:21:26.052450+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:36:50.699231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.053232+0000","last_change":"2024-06-03T03:19:56.938231+0000","last_active":"2024-06-03T03:21:26.053232+0000","last_peered":"2024-06-03T03:21:26.053232+0000","last_clean":"2024-06-03T03:21:26.053232+0000","last_became_active":"2024-06-03T03:19:56.937840+0000","last_became_peered":"2024-06-03T03:19:56.937840+0000","last_unstale":"2024-06-03T03:21:26.053232+0000","last_undegraded":"2024-06-03T03:21:26.053232+0000","last_fullsized":"2024-06-03T03:21:26.053232+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:02:33.916980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"97'4","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.973969+0000","last_change":"2024-06-03T03:19:36.737519+0000","last_active":"2024-06-03T03:21:25.973969+0000","last_peered":"2024-06-03T03:21:25.973969+0000","last_clean":"2024-06-03T03:21:25.973969+0000","last_became_active":"2024-06-03T03:19:36.737191+0000","last_became_peered":"2024-06-03T03:19:36.737191+0000","last_unstale":"2024-06-03T03:21:25.973969+0000","last_undegraded":"2024-06-03T03:21:25.973969+0000","last_fullsized":"2024-06-03T03:21:25.973969+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:50:21.389739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"99'9","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.817499+0000","last_change":"2024-06-03T03:19:41.633113+0000","last_active":"2024-06-03T03:22:54.817499+0000","last_peered":"2024-06-03T03:22:54.817499+0000","last_clean":"2024-06-03T03:22:54.817499+0000","last_became_active":"2024-06-03T03:19:41.632186+0000","last_became_peered":"2024-06-03T03:19:41.632186+0000","last_unstale":"2024-06-03T03:22:54.817499+0000","last_undegraded":"2024-06-03T03:22:54.817499+0000","last_fullsized":"2024-06-03T03:22:54.817499+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T15:12:26.835061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.057204+0000","last_change":"2024-06-03T03:19:56.928603+0000","last_active":"2024-06-03T03:21:26.057204+0000","last_peered":"2024-06-03T03:21:26.057204+0000","last_clean":"2024-06-03T03:21:26.057204+0000","last_became_active":"2024-06-03T03:19:56.928175+0000","last_became_peered":"2024-06-03T03:19:56.928175+0000","last_unstale":"2024-06-03T03:21:26.057204+0000","last_undegraded":"2024-06-03T03:21:26.057204+0000","last_fullsized":"2024-06-03T03:21:26.057204+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:26:03.531091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"99'19","reported_seq":64,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:45.881135+0000","last_change":"2024-06-03T03:19:36.550577+0000","last_active":"2024-06-03T03:21:45.881135+0000","last_peered":"2024-06-03T03:21:45.881135+0000","last_clean":"2024-06-03T03:21:45.881135+0000","last_became_active":"2024-06-03T03:19:36.549968+0000","last_became_peered":"2024-06-03T03:19:36.549968+0000","last_unstale":"2024-06-03T03:21:45.881135+0000","last_undegraded":"2024-06-03T03:21:45.881135+0000","last_fullsized":"2024-06-03T03:21:45.881135+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:22:43.892002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"99'5","reported_seq":43,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.825404+0000","last_change":"2024-06-03T03:19:41.613883+0000","last_active":"2024-06-03T03:22:54.825404+0000","last_peered":"2024-06-03T03:22:54.825404+0000","last_clean":"2024-06-03T03:22:54.825404+0000","last_became_active":"2024-06-03T03:19:41.613472+0000","last_became_peered":"2024-06-03T03:19:41.613472+0000","last_unstale":"2024-06-03T03:22:54.825404+0000","last_undegraded":"2024-06-03T03:22:54.825404+0000","last_fullsized":"2024-06-03T03:22:54.825404+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T14:38:42.044534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.052273+0000","last_change":"2024-06-03T03:19:56.914487+0000","last_active":"2024-06-03T03:21:26.052273+0000","last_peered":"2024-06-03T03:21:26.052273+0000","last_clean":"2024-06-03T03:21:26.052273+0000","last_became_active":"2024-06-03T03:19:56.913904+0000","last_became_peered":"2024-06-03T03:19:56.913904+0000","last_unstale":"2024-06-03T03:21:26.052273+0000","last_undegraded":"2024-06-03T03:21:26.052273+0000","last_fullsized":"2024-06-03T03:21:26.052273+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T15:13:24.084034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"97'7","reported_seq":52,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.981407+0000","last_change":"2024-06-03T03:19:36.671355+0000","last_active":"2024-06-03T03:21:25.981407+0000","last_peered":"2024-06-03T03:21:25.981407+0000","last_clean":"2024-06-03T03:21:25.981407+0000","last_became_active":"2024-06-03T03:19:36.670926+0000","last_became_peered":"2024-06-03T03:19:36.670926+0000","last_unstale":"2024-06-03T03:21:25.981407+0000","last_undegraded":"2024-06-03T03:21:25.981407+0000","last_fullsized":"2024-06-03T03:21:25.981407+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:38:50.914238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"99'7","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.833522+0000","last_change":"2024-06-03T03:19:41.629054+0000","last_active":"2024-06-03T03:22:54.833522+0000","last_peered":"2024-06-03T03:22:54.833522+0000","last_clean":"2024-06-03T03:22:54.833522+0000","last_became_active":"2024-06-03T03:19:41.627118+0000","last_became_peered":"2024-06-03T03:19:41.627118+0000","last_unstale":"2024-06-03T03:22:54.833522+0000","last_undegraded":"2024-06-03T03:22:54.833522+0000","last_fullsized":"2024-06-03T03:22:54.833522+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:59:40.645002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":36,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.059189+0000","last_change":"2024-06-03T03:21:26.058741+0000","last_active":"2024-06-03T03:21:26.059189+0000","last_peered":"2024-06-03T03:21:26.059189+0000","last_clean":"2024-06-03T03:21:26.059189+0000","last_became_active":"2024-06-03T03:21:26.058271+0000","last_became_peered":"2024-06-03T03:21:26.058271+0000","last_unstale":"2024-06-03T03:21:26.059189+0000","last_undegraded":"2024-06-03T03:21:26.059189+0000","last_fullsized":"2024-06-03T03:21:26.059189+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:31:32.359893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"97'2","reported_seq":53,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.052741+0000","last_change":"2024-06-03T03:20:26.132994+0000","last_active":"2024-06-03T03:21:26.052741+0000","last_peered":"2024-06-03T03:21:26.052741+0000","last_clean":"2024-06-03T03:21:26.052741+0000","last_became_active":"2024-06-03T03:20:26.132574+0000","last_became_peered":"2024-06-03T03:20:26.132574+0000","last_unstale":"2024-06-03T03:21:26.052741+0000","last_undegraded":"2024-06-03T03:21:26.052741+0000","last_fullsized":"2024-06-03T03:21:26.052741+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T15:12:33.599154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"99'2","reported_seq":40,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.791726+0000","last_change":"2024-06-03T03:19:41.603630+0000","last_active":"2024-06-03T03:22:54.791726+0000","last_peered":"2024-06-03T03:22:54.791726+0000","last_clean":"2024-06-03T03:22:54.791726+0000","last_became_active":"2024-06-03T03:19:41.603301+0000","last_became_peered":"2024-06-03T03:19:41.603301+0000","last_unstale":"2024-06-03T03:22:54.791726+0000","last_undegraded":"2024-06-03T03:22:54.791726+0000","last_fullsized":"2024-06-03T03:22:54.791726+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:20:33.076622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.057638+0000","last_change":"2024-06-03T03:19:56.929023+0000","last_active":"2024-06-03T03:21:26.057638+0000","last_peered":"2024-06-03T03:21:26.057638+0000","last_clean":"2024-06-03T03:21:26.057638+0000","last_became_active":"2024-06-03T03:19:56.928454+0000","last_became_peered":"2024-06-03T03:19:56.928454+0000","last_unstale":"2024-06-03T03:21:26.057638+0000","last_undegraded":"2024-06-03T03:21:26.057638+0000","last_fullsized":"2024-06-03T03:21:26.057638+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:17:31.755649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"97'7","reported_seq":51,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.056909+0000","last_change":"2024-06-03T03:19:36.551413+0000","last_active":"2024-06-03T03:21:26.056909+0000","last_peered":"2024-06-03T03:21:26.056909+0000","last_clean":"2024-06-03T03:21:26.056909+0000","last_became_active":"2024-06-03T03:19:36.550622+0000","last_became_peered":"2024-06-03T03:19:36.550622+0000","last_unstale":"2024-06-03T03:21:26.056909+0000","last_undegraded":"2024-06-03T03:21:26.056909+0000","last_fullsized":"2024-06-03T03:21:26.056909+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:50:09.472960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"99'9","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.830598+0000","last_change":"2024-06-03T03:19:41.624871+0000","last_active":"2024-06-03T03:22:54.830598+0000","last_peered":"2024-06-03T03:22:54.830598+0000","last_clean":"2024-06-03T03:22:54.830598+0000","last_became_active":"2024-06-03T03:19:41.624638+0000","last_became_peered":"2024-06-03T03:19:41.624638+0000","last_unstale":"2024-06-03T03:22:54.830598+0000","last_undegraded":"2024-06-03T03:22:54.830598+0000","last_fullsized":"2024-06-03T03:22:54.830598+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:21:16.193817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"99'1","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:29.282055+0000","last_change":"2024-06-03T03:19:56.924638+0000","last_active":"2024-06-03T03:21:29.282055+0000","last_peered":"2024-06-03T03:21:29.282055+0000","last_clean":"2024-06-03T03:21:29.282055+0000","last_became_active":"2024-06-03T03:19:56.924200+0000","last_became_peered":"2024-06-03T03:19:56.924200+0000","last_unstale":"2024-06-03T03:21:29.282055+0000","last_undegraded":"2024-06-03T03:21:29.282055+0000","last_fullsized":"2024-06-03T03:21:29.282055+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:38:01.138253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"97'3","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.052712+0000","last_change":"2024-06-03T03:19:36.546974+0000","last_active":"2024-06-03T03:21:26.052712+0000","last_peered":"2024-06-03T03:21:26.052712+0000","last_clean":"2024-06-03T03:21:26.052712+0000","last_became_active":"2024-06-03T03:19:36.546433+0000","last_became_peered":"2024-06-03T03:19:36.546433+0000","last_unstale":"2024-06-03T03:21:26.052712+0000","last_undegraded":"2024-06-03T03:21:26.052712+0000","last_fullsized":"2024-06-03T03:21:26.052712+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:41:46.980327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"99'8","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.833961+0000","last_change":"2024-06-03T03:19:41.622118+0000","last_active":"2024-06-03T03:22:54.833961+0000","last_peered":"2024-06-03T03:22:54.833961+0000","last_clean":"2024-06-03T03:22:54.833961+0000","last_became_active":"2024-06-03T03:19:41.620789+0000","last_became_peered":"2024-06-03T03:19:41.620789+0000","last_unstale":"2024-06-03T03:22:54.833961+0000","last_undegraded":"2024-06-03T03:22:54.833961+0000","last_fullsized":"2024-06-03T03:22:54.833961+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:00:36.232238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.981089+0000","last_change":"2024-06-03T03:19:56.921299+0000","last_active":"2024-06-03T03:21:25.981089+0000","last_peered":"2024-06-03T03:21:25.981089+0000","last_clean":"2024-06-03T03:21:25.981089+0000","last_became_active":"2024-06-03T03:19:56.920870+0000","last_became_peered":"2024-06-03T03:19:56.920870+0000","last_unstale":"2024-06-03T03:21:25.981089+0000","last_undegraded":"2024-06-03T03:21:25.981089+0000","last_fullsized":"2024-06-03T03:21:25.981089+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:10:15.062381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"97'4","reported_seq":23,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.053109+0000","last_change":"2024-06-03T03:20:25.903279+0000","last_active":"2024-06-03T03:21:26.053109+0000","last_peered":"2024-06-03T03:21:26.053109+0000","last_clean":"2024-06-03T03:21:26.053109+0000","last_became_active":"2024-06-03T03:20:25.900311+0000","last_became_peered":"2024-06-03T03:20:25.900311+0000","last_unstale":"2024-06-03T03:21:26.053109+0000","last_undegraded":"2024-06-03T03:21:26.053109+0000","last_fullsized":"2024-06-03T03:21:26.053109+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:25:49.708581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"99'5","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.836116+0000","last_change":"2024-06-03T03:19:41.628659+0000","last_active":"2024-06-03T03:22:54.836116+0000","last_peered":"2024-06-03T03:22:54.836116+0000","last_clean":"2024-06-03T03:22:54.836116+0000","last_became_active":"2024-06-03T03:19:41.628291+0000","last_became_peered":"2024-06-03T03:19:41.628291+0000","last_unstale":"2024-06-03T03:22:54.836116+0000","last_undegraded":"2024-06-03T03:22:54.836116+0000","last_fullsized":"2024-06-03T03:22:54.836116+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:26:49.579669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.052963+0000","last_change":"2024-06-03T03:19:56.922766+0000","last_active":"2024-06-03T03:21:26.052963+0000","last_peered":"2024-06-03T03:21:26.052963+0000","last_clean":"2024-06-03T03:21:26.052963+0000","last_became_active":"2024-06-03T03:19:56.922310+0000","last_became_peered":"2024-06-03T03:19:56.922310+0000","last_unstale":"2024-06-03T03:21:26.052963+0000","last_undegraded":"2024-06-03T03:21:26.052963+0000","last_fullsized":"2024-06-03T03:21:26.052963+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T14:35:53.133519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"97'5","reported_seq":153,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:23:16.244585+0000","last_change":"2024-06-03T03:20:26.133562+0000","last_active":"2024-06-03T03:23:16.244585+0000","last_peered":"2024-06-03T03:23:16.244585+0000","last_clean":"2024-06-03T03:23:16.244585+0000","last_became_active":"2024-06-03T03:20:26.133074+0000","last_became_peered":"2024-06-03T03:20:26.133074+0000","last_unstale":"2024-06-03T03:23:16.244585+0000","last_undegraded":"2024-06-03T03:23:16.244585+0000","last_fullsized":"2024-06-03T03:23:16.244585+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:51:33.717584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":132,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.817652+0000","last_change":"2024-06-03T03:19:41.611450+0000","last_active":"2024-06-03T03:22:54.817652+0000","last_peered":"2024-06-03T03:22:54.817652+0000","last_clean":"2024-06-03T03:22:54.817652+0000","last_became_active":"2024-06-03T03:19:41.610214+0000","last_became_peered":"2024-06-03T03:19:41.610214+0000","last_unstale":"2024-06-03T03:22:54.817652+0000","last_undegraded":"2024-06-03T03:22:54.817652+0000","last_fullsized":"2024-06-03T03:22:54.817652+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:01:01.521917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.975566+0000","last_change":"2024-06-03T03:19:56.935686+0000","last_active":"2024-06-03T03:21:25.975566+0000","last_peered":"2024-06-03T03:21:25.975566+0000","last_clean":"2024-06-03T03:21:25.975566+0000","last_became_active":"2024-06-03T03:19:56.935314+0000","last_became_peered":"2024-06-03T03:19:56.935314+0000","last_unstale":"2024-06-03T03:21:25.975566+0000","last_undegraded":"2024-06-03T03:21:25.975566+0000","last_fullsized":"2024-06-03T03:21:25.975566+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T15:10:16.966321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"97'3","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.053921+0000","last_change":"2024-06-03T03:19:36.547756+0000","last_active":"2024-06-03T03:21:26.053921+0000","last_peered":"2024-06-03T03:21:26.053921+0000","last_clean":"2024-06-03T03:21:26.053921+0000","last_became_active":"2024-06-03T03:19:36.547350+0000","last_became_peered":"2024-06-03T03:19:36.547350+0000","last_unstale":"2024-06-03T03:21:26.053921+0000","last_undegraded":"2024-06-03T03:21:26.053921+0000","last_fullsized":"2024-06-03T03:21:26.053921+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:57:39.214380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.827896+0000","last_change":"2024-06-03T03:19:41.621083+0000","last_active":"2024-06-03T03:22:54.827896+0000","last_peered":"2024-06-03T03:22:54.827896+0000","last_clean":"2024-06-03T03:22:54.827896+0000","last_became_active":"2024-06-03T03:19:41.620719+0000","last_became_peered":"2024-06-03T03:19:41.620719+0000","last_unstale":"2024-06-03T03:22:54.827896+0000","last_undegraded":"2024-06-03T03:22:54.827896+0000","last_fullsized":"2024-06-03T03:22:54.827896+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:50:35.007025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:26.052302+0000","last_change":"2024-06-03T03:19:56.915406+0000","last_active":"2024-06-03T03:21:26.052302+0000","last_peered":"2024-06-03T03:21:26.052302+0000","last_clean":"2024-06-03T03:21:26.052302+0000","last_became_active":"2024-06-03T03:19:56.914918+0000","last_became_peered":"2024-06-03T03:19:56.914918+0000","last_unstale":"2024-06-03T03:21:26.052302+0000","last_undegraded":"2024-06-03T03:21:26.052302+0000","last_fullsized":"2024-06-03T03:21:26.052302+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:42:59.324011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"97'5","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:25.979623+0000","last_change":"2024-06-03T03:19:36.549502+0000","last_active":"2024-06-03T03:21:25.979623+0000","last_peered":"2024-06-03T03:21:25.979623+0000","last_clean":"2024-06-03T03:21:25.979623+0000","last_became_active":"2024-06-03T03:19:36.547538+0000","last_became_peered":"2024-06-03T03:19:36.547538+0000","last_unstale":"2024-06-03T03:21:25.979623+0000","last_undegraded":"2024-06-03T03:21:25.979623+0000","last_fullsized":"2024-06-03T03:21:25.979623+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:28:50.558069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.815219+0000","last_change":"2024-06-03T03:19:41.607441+0000","last_active":"2024-06-03T03:22:54.815219+0000","last_peered":"2024-06-03T03:22:54.815219+0000","last_clean":"2024-06-03T03:22:54.815219+0000","last_became_active":"2024-06-03T03:19:41.607086+0000","last_became_peered":"2024-06-03T03:19:41.607086+0000","last_unstale":"2024-06-03T03:22:54.815219+0000","last_undegraded":"2024-06-03T03:22:54.815219+0000","last_fullsized":"2024-06-03T03:22:54.815219+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T14:53:50.091872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"99'9","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.819487+0000","last_change":"2024-06-03T03:19:41.607004+0000","last_active":"2024-06-03T03:22:54.819487+0000","last_peered":"2024-06-03T03:22:54.819487+0000","last_clean":"2024-06-03T03:22:54.819487+0000","last_became_active":"2024-06-03T03:19:41.606684+0000","last_became_peered":"2024-06-03T03:19:41.606684+0000","last_unstale":"2024-06-03T03:22:54.819487+0000","last_undegraded":"2024-06-03T03:22:54.819487+0000","last_fullsized":"2024-06-03T03:22:54.819487+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:48:55.320597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"99'12","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.810392+0000","last_change":"2024-06-03T03:19:41.589426+0000","last_active":"2024-06-03T03:22:54.810392+0000","last_peered":"2024-06-03T03:22:54.810392+0000","last_clean":"2024-06-03T03:22:54.810392+0000","last_became_active":"2024-06-03T03:19:41.589091+0000","last_became_peered":"2024-06-03T03:19:41.589091+0000","last_unstale":"2024-06-03T03:22:54.810392+0000","last_undegraded":"2024-06-03T03:22:54.810392+0000","last_fullsized":"2024-06-03T03:22:54.810392+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:40:34.313214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"99'8","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.832429+0000","last_change":"2024-06-03T03:19:41.626149+0000","last_active":"2024-06-03T03:22:54.832429+0000","last_peered":"2024-06-03T03:22:54.832429+0000","last_clean":"2024-06-03T03:22:54.832429+0000","last_became_active":"2024-06-03T03:19:41.625710+0000","last_became_peered":"2024-06-03T03:19:41.625710+0000","last_unstale":"2024-06-03T03:22:54.832429+0000","last_undegraded":"2024-06-03T03:22:54.832429+0000","last_fullsized":"2024-06-03T03:22:54.832429+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:15:15.220975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"99'7","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.834957+0000","last_change":"2024-06-03T03:19:41.636585+0000","last_active":"2024-06-03T03:22:54.834957+0000","last_peered":"2024-06-03T03:22:54.834957+0000","last_clean":"2024-06-03T03:22:54.834957+0000","last_became_active":"2024-06-03T03:19:41.636241+0000","last_became_peered":"2024-06-03T03:19:41.636241+0000","last_unstale":"2024-06-03T03:22:54.834957+0000","last_undegraded":"2024-06-03T03:22:54.834957+0000","last_fullsized":"2024-06-03T03:22:54.834957+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:19:28.548297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"99'4","reported_seq":42,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.834106+0000","last_change":"2024-06-03T03:19:41.629530+0000","last_active":"2024-06-03T03:22:54.834106+0000","last_peered":"2024-06-03T03:22:54.834106+0000","last_clean":"2024-06-03T03:22:54.834106+0000","last_became_active":"2024-06-03T03:19:41.629197+0000","last_became_peered":"2024-06-03T03:19:41.629197+0000","last_unstale":"2024-06-03T03:22:54.834106+0000","last_undegraded":"2024-06-03T03:22:54.834106+0000","last_fullsized":"2024-06-03T03:22:54.834106+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:12:02.807445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"99'6","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.818988+0000","last_change":"2024-06-03T03:19:41.606995+0000","last_active":"2024-06-03T03:22:54.818988+0000","last_peered":"2024-06-03T03:22:54.818988+0000","last_clean":"2024-06-03T03:22:54.818988+0000","last_became_active":"2024-06-03T03:19:41.606651+0000","last_became_peered":"2024-06-03T03:19:41.606651+0000","last_unstale":"2024-06-03T03:22:54.818988+0000","last_undegraded":"2024-06-03T03:22:54.818988+0000","last_fullsized":"2024-06-03T03:22:54.818988+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:07:11.751606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"99'15","reported_seq":53,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.810018+0000","last_change":"2024-06-03T03:19:41.609848+0000","last_active":"2024-06-03T03:22:54.810018+0000","last_peered":"2024-06-03T03:22:54.810018+0000","last_clean":"2024-06-03T03:22:54.810018+0000","last_became_active":"2024-06-03T03:19:41.609544+0000","last_became_peered":"2024-06-03T03:19:41.609544+0000","last_unstale":"2024-06-03T03:22:54.810018+0000","last_undegraded":"2024-06-03T03:22:54.810018+0000","last_fullsized":"2024-06-03T03:22:54.810018+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:12:00.804638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"99'2","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.810429+0000","last_change":"2024-06-03T03:20:25.912049+0000","last_active":"2024-06-03T03:22:54.810429+0000","last_peered":"2024-06-03T03:22:54.810429+0000","last_clean":"2024-06-03T03:22:54.810429+0000","last_became_active":"2024-06-03T03:20:25.910664+0000","last_became_peered":"2024-06-03T03:20:25.910664+0000","last_unstale":"2024-06-03T03:22:54.810429+0000","last_undegraded":"2024-06-03T03:22:54.810429+0000","last_fullsized":"2024-06-03T03:22:54.810429+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:47:19.282569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.828352+0000","last_change":"2024-06-03T03:19:41.610800+0000","last_active":"2024-06-03T03:22:54.828352+0000","last_peered":"2024-06-03T03:22:54.828352+0000","last_clean":"2024-06-03T03:22:54.828352+0000","last_became_active":"2024-06-03T03:19:41.610457+0000","last_became_peered":"2024-06-03T03:19:41.610457+0000","last_unstale":"2024-06-03T03:22:54.828352+0000","last_undegraded":"2024-06-03T03:22:54.828352+0000","last_fullsized":"2024-06-03T03:22:54.828352+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:31:06.415159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"99'8","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.821273+0000","last_change":"2024-06-03T03:19:41.617020+0000","last_active":"2024-06-03T03:22:54.821273+0000","last_peered":"2024-06-03T03:22:54.821273+0000","last_clean":"2024-06-03T03:22:54.821273+0000","last_became_active":"2024-06-03T03:19:41.616614+0000","last_became_peered":"2024-06-03T03:19:41.616614+0000","last_unstale":"2024-06-03T03:22:54.821273+0000","last_undegraded":"2024-06-03T03:22:54.821273+0000","last_fullsized":"2024-06-03T03:22:54.821273+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:35:32.657685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"99'10","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.828102+0000","last_change":"2024-06-03T03:19:41.622785+0000","last_active":"2024-06-03T03:22:54.828102+0000","last_peered":"2024-06-03T03:22:54.828102+0000","last_clean":"2024-06-03T03:22:54.828102+0000","last_became_active":"2024-06-03T03:19:41.622407+0000","last_became_peered":"2024-06-03T03:19:41.622407+0000","last_unstale":"2024-06-03T03:22:54.828102+0000","last_undegraded":"2024-06-03T03:22:54.828102+0000","last_fullsized":"2024-06-03T03:22:54.828102+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:53:50.804889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"99'11","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.838842+0000","last_change":"2024-06-03T03:19:41.633429+0000","last_active":"2024-06-03T03:22:54.838842+0000","last_peered":"2024-06-03T03:22:54.838842+0000","last_clean":"2024-06-03T03:22:54.838842+0000","last_became_active":"2024-06-03T03:19:41.631028+0000","last_became_peered":"2024-06-03T03:19:41.631028+0000","last_unstale":"2024-06-03T03:22:54.838842+0000","last_undegraded":"2024-06-03T03:22:54.838842+0000","last_fullsized":"2024-06-03T03:22:54.838842+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:37:10.666591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"99'8","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.818347+0000","last_change":"2024-06-03T03:19:41.606204+0000","last_active":"2024-06-03T03:22:54.818347+0000","last_peered":"2024-06-03T03:22:54.818347+0000","last_clean":"2024-06-03T03:22:54.818347+0000","last_became_active":"2024-06-03T03:19:41.605756+0000","last_became_peered":"2024-06-03T03:19:41.605756+0000","last_unstale":"2024-06-03T03:22:54.818347+0000","last_undegraded":"2024-06-03T03:22:54.818347+0000","last_fullsized":"2024-06-03T03:22:54.818347+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:24:32.507426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":245318,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":76157,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1335,"num_read_kb":751,"num_write":264,"num_write_kb":155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":228471,"data_compressed":5580,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":226,"ondisk_log_size":226,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383740,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1101824,"data_stored":570375,"data_compressed":32353,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645378,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32400,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712752,"statfs":{"total":95995035648,"available":95961858048,"internally_reserved":0,"allocated":1253376,"data_stored":1226395,"data_compressed":69963,"data_compressed_allocated":94208,"data_compressed_original":688128,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808903,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32280,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712872,"statfs":{"total":95995035648,"available":95961980928,"internally_reserved":0,"allocated":1130496,"data_stored":574401,"data_compressed":32353,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070542,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32272,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712880,"statfs":{"total":95995035648,"available":95961989120,"internally_reserved":0,"allocated":1122304,"data_stored":570683,"data_compressed":32353,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332179,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32324,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712828,"statfs":{"total":95995035648,"available":95961935872,"internally_reserved":0,"allocated":1175552,"data_stored":1167320,"data_compressed":68103,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495706,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1101824,"data_stored":565809,"data_compressed":32353,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574965,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28160,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716992,"statfs":{"total":95995035648,"available":95966199808,"internally_reserved":0,"allocated":1171456,"data_stored":636043,"data_compressed":34213,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738489,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1126400,"data_stored":567265,"data_compressed":32353,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411439,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36492,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708660,"statfs":{"total":95995035648,"available":95957667840,"internally_reserved":0,"allocated":1183744,"data_stored":635624,"data_compressed":34213,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182506,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1114112,"data_stored":565517,"data_compressed":32353,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920869,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28108,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717044,"statfs":{"total":95995035648,"available":95966253056,"internally_reserved":0,"allocated":1118208,"data_stored":571053,"data_compressed":32353,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757344,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32332,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712820,"statfs":{"total":95995035648,"available":95961927680,"internally_reserved":0,"allocated":1183744,"data_stored":1162902,"data_compressed":68103,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34106,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34106,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34106,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35750,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35750,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35750,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35750,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":70956,"data_compressed":1860,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":70537,"data_compressed":1860,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":70940,"data_compressed":1860,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-03T03:23:21.142 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-06-03T03:23:21.142 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-06-03T03:23:21.142 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-06-03T03:23:21.142 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph health --format=json 2024-06-03T03:23:21.631 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:23:22.052 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:23:21 smithi117 ceph-mon[31649]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-03T03:23:22.052 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:23:21 smithi117 ceph-mon[31649]: pgmap v447: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-06-03T03:23:22.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:23:21 smithi151 ceph-mon[37043]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-03T03:23:22.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:23:21 smithi151 ceph-mon[37043]: pgmap v447: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-06-03T03:23:22.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:23:21 smithi183 ceph-mon[37326]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-03T03:23:22.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:23:21 smithi183 ceph-mon[37326]: pgmap v447: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-06-03T03:23:24.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:24.025+0000 7f8a7b200700 1 -- 172.21.15.117:0/4036766727 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a741045b0 msgr2=0x7f8a74104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:24.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:24.025+0000 7f8a7b200700 1 --2- 172.21.15.117:0/4036766727 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a741045b0 0x7f8a74104a30 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8a640098f0 tx=0x7f8a64009c00 comp rx=0 tx=0).stop 2024-06-03T03:23:24.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:24.025+0000 7f8a7b200700 1 -- 172.21.15.117:0/4036766727 shutdown_connections 2024-06-03T03:23:24.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:24.025+0000 7f8a7b200700 1 --2- 172.21.15.117:0/4036766727 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a74104f70 0x7f8a74111430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:24.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:24.025+0000 7f8a7b200700 1 --2- 172.21.15.117:0/4036766727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a7410a660 0x7f8a7410aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:24.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:24.025+0000 7f8a7b200700 1 --2- 172.21.15.117:0/4036766727 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a741045b0 0x7f8a74104a30 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:24.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:24.025+0000 7f8a7b200700 1 -- 172.21.15.117:0/4036766727 >> 172.21.15.117:0/4036766727 conn(0x7f8a74100080 msgr2=0x7f8a741024a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:23:24.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:24.025+0000 7f8a7b200700 1 -- 172.21.15.117:0/4036766727 shutdown_connections 2024-06-03T03:23:24.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:24.025+0000 7f8a7b200700 1 -- 172.21.15.117:0/4036766727 wait complete. 2024-06-03T03:23:24.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:24.026+0000 7f8a7b200700 1 Processor -- start 2024-06-03T03:23:24.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:24.026+0000 7f8a7b200700 1 -- start start 2024-06-03T03:23:24.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:24.027+0000 7f8a7b200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a741045b0 0x7f8a7419ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:24.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:24.027+0000 7f8a7b200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a74104f70 0x7f8a7419b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:24.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:24.027+0000 7f8a7b200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a7410a660 0x7f8a7419f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:24.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:24.027+0000 7f8a7b200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8a74114920 con 0x7f8a741045b0 2024-06-03T03:23:24.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:24.027+0000 7f8a7b200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8a741147a0 con 0x7f8a74104f70 2024-06-03T03:23:24.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:24.027+0000 7f8a7b200700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8a74114aa0 con 0x7f8a7410a660 2024-06-03T03:23:24.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:24.027+0000 7f8a79e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a741045b0 0x7f8a7419ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:24.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:24.027+0000 7f8a79400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a74104f70 0x7f8a7419b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:24.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:24.027+0000 7f8a79e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a741045b0 0x7f8a7419ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:42302/0 (socket says 172.21.15.117:42302) 2024-06-03T03:23:24.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:24.027+0000 7f8a79e00700 1 -- 172.21.15.117:0/3925487258 learned_addr learned my addr 172.21.15.117:0/3925487258 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:23:24.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:24.027+0000 7f8a7a800700 1 --2- 172.21.15.117:0/3925487258 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a7410a660 0x7f8a7419f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:24.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:24.027+0000 7f8a79400700 1 -- 172.21.15.117:0/3925487258 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a7410a660 msgr2=0x7f8a7419f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:24.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:24.027+0000 7f8a79400700 1 --2- 172.21.15.117:0/3925487258 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a7410a660 0x7f8a7419f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:24.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:24.027+0000 7f8a79400700 1 -- 172.21.15.117:0/3925487258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a741045b0 msgr2=0x7f8a7419ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:24.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:24.027+0000 7f8a79400700 1 --2- 172.21.15.117:0/3925487258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a741045b0 0x7f8a7419ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:24.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:24.027+0000 7f8a79400700 1 -- 172.21.15.117:0/3925487258 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a7419fd20 con 0x7f8a74104f70 2024-06-03T03:23:24.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:24.028+0000 7f8a79400700 1 --2- 172.21.15.117:0/3925487258 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a74104f70 0x7f8a7419b250 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8a64000c00 tx=0x7f8a640094b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:23:24.030 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:24.028+0000 7f8a6ea00700 1 -- 172.21.15.117:0/3925487258 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a64025070 con 0x7f8a74104f70 2024-06-03T03:23:24.030 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:24.028+0000 7f8a6ea00700 1 -- 172.21.15.117:0/3925487258 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a6400ba40 con 0x7f8a74104f70 2024-06-03T03:23:24.030 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:24.028+0000 7f8a7b200700 1 -- 172.21.15.117:0/3925487258 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a7419ffb0 con 0x7f8a74104f70 2024-06-03T03:23:24.030 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:24.028+0000 7f8a6ea00700 1 -- 172.21.15.117:0/3925487258 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a64004c10 con 0x7f8a74104f70 2024-06-03T03:23:24.030 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:24.028+0000 7f8a7b200700 1 -- 172.21.15.117:0/3925487258 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a741a7890 con 0x7f8a74104f70 2024-06-03T03:23:24.030 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:24.029+0000 7f8a7b200700 1 -- 172.21.15.117:0/3925487258 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a74105cd0 con 0x7f8a74104f70 2024-06-03T03:23:24.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:24.031+0000 7f8a6ea00700 1 -- 172.21.15.117:0/3925487258 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f8a64028070 con 0x7f8a74104f70 2024-06-03T03:23:24.033 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:24.032+0000 7f8a6ea00700 1 --2- 172.21.15.117:0/3925487258 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f8a5c0a2ca0 0x7f8a5c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:24.033 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:24.033+0000 7f8a6ea00700 1 -- 172.21.15.117:0/3925487258 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f8a640cfa70 con 0x7f8a74104f70 2024-06-03T03:23:24.033 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:24.033+0000 7f8a79e00700 1 --2- 172.21.15.117:0/3925487258 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f8a5c0a2ca0 0x7f8a5c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:24.034 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:24.033+0000 7f8a79e00700 1 --2- 172.21.15.117:0/3925487258 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f8a5c0a2ca0 0x7f8a5c0a5160 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8a68005950 tx=0x7f8a68009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:23:24.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:24.037+0000 7f8a6ea00700 1 -- 172.21.15.117:0/3925487258 <== mon.2 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8a6405c850 con 0x7f8a74104f70 2024-06-03T03:23:24.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:24.311+0000 7f8a7b200700 1 -- 172.21.15.117:0/3925487258 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f8a74069180 con 0x7f8a74104f70 2024-06-03T03:23:24.313 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:23:23 smithi117 ceph-mon[31649]: pgmap v448: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 11 op/s 2024-06-03T03:23:24.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:24.312+0000 7f8a6ea00700 1 -- 172.21.15.117:0/3925487258 <== mon.2 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f8a64013030 con 0x7f8a74104f70 2024-06-03T03:23:24.313 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T03:23:24.313 INFO:teuthology.orchestra.run.smithi117.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-06-03T03:23:24.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:24.316+0000 7f8a7b200700 1 -- 172.21.15.117:0/3925487258 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f8a5c0a2ca0 msgr2=0x7f8a5c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:24.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:24.316+0000 7f8a7b200700 1 --2- 172.21.15.117:0/3925487258 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f8a5c0a2ca0 0x7f8a5c0a5160 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8a68005950 tx=0x7f8a68009310 comp rx=0 tx=0).stop 2024-06-03T03:23:24.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:24.316+0000 7f8a7b200700 1 -- 172.21.15.117:0/3925487258 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a74104f70 msgr2=0x7f8a7419b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:24.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:24.316+0000 7f8a7b200700 1 --2- 172.21.15.117:0/3925487258 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a74104f70 0x7f8a7419b250 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8a64000c00 tx=0x7f8a640094b0 comp rx=0 tx=0).stop 2024-06-03T03:23:24.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:24.317+0000 7f8a7b200700 1 -- 172.21.15.117:0/3925487258 shutdown_connections 2024-06-03T03:23:24.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:24.317+0000 7f8a7b200700 1 --2- 172.21.15.117:0/3925487258 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f8a5c0a2ca0 0x7f8a5c0a5160 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:24.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:24.317+0000 7f8a7b200700 1 --2- 172.21.15.117:0/3925487258 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a7410a660 0x7f8a7419f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:24.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:24.317+0000 7f8a7b200700 1 --2- 172.21.15.117:0/3925487258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a741045b0 0x7f8a7419ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:24.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:24.317+0000 7f8a7b200700 1 --2- 172.21.15.117:0/3925487258 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a74104f70 0x7f8a7419b250 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:24.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:24.317+0000 7f8a7b200700 1 -- 172.21.15.117:0/3925487258 >> 172.21.15.117:0/3925487258 conn(0x7f8a74100080 msgr2=0x7f8a74101730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:23:24.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:24.317+0000 7f8a7b200700 1 -- 172.21.15.117:0/3925487258 shutdown_connections 2024-06-03T03:23:24.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:24.317+0000 7f8a7b200700 1 -- 172.21.15.117:0/3925487258 wait complete. 2024-06-03T03:23:24.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:23:23 smithi151 ceph-mon[37043]: pgmap v448: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 11 op/s 2024-06-03T03:23:24.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:23:23 smithi183 ceph-mon[37326]: pgmap v448: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 11 op/s 2024-06-03T03:23:25.164 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:23:24 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/3925487258' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-06-03T03:23:25.224 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-06-03T03:23:25.225 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-03T03:23:25.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:23:24 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/3925487258' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-06-03T03:23:25.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:23:24 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/3925487258' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-06-03T03:23:25.662 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:23:26.128 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:23:26 smithi117 ceph-mon[31649]: pgmap v449: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-06-03T03:23:26.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:23:26 smithi151 ceph-mon[37043]: pgmap v449: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-06-03T03:23:26.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:23:26 smithi183 ceph-mon[37326]: pgmap v449: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-06-03T03:23:27.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:27.986+0000 7fbd98e00700 1 -- 172.21.15.117:0/3562699756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd9410eaa0 msgr2=0x7fbd9410ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:27.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:27.986+0000 7fbd98e00700 1 --2- 172.21.15.117:0/3562699756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd9410eaa0 0x7fbd9410ee80 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fbd84009990 tx=0x7fbd84009ca0 comp rx=0 tx=0).stop 2024-06-03T03:23:27.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:27.986+0000 7fbd98e00700 1 -- 172.21.15.117:0/3562699756 shutdown_connections 2024-06-03T03:23:27.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:27.986+0000 7fbd98e00700 1 --2- 172.21.15.117:0/3562699756 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbd94109460 0x7fbd94115970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:27.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:27.986+0000 7fbd98e00700 1 --2- 172.21.15.117:0/3562699756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd9410eaa0 0x7fbd9410ee80 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:27.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:27.986+0000 7fbd98e00700 1 --2- 172.21.15.117:0/3562699756 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbd94108aa0 0x7fbd94108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:27.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:27.986+0000 7fbd98e00700 1 -- 172.21.15.117:0/3562699756 >> 172.21.15.117:0/3562699756 conn(0x7fbd94075c60 msgr2=0x7fbd941072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:23:27.988 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:27.986+0000 7fbd98e00700 1 -- 172.21.15.117:0/3562699756 shutdown_connections 2024-06-03T03:23:27.988 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:27.986+0000 7fbd98e00700 1 -- 172.21.15.117:0/3562699756 wait complete. 2024-06-03T03:23:27.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:27.987+0000 7fbd98e00700 1 Processor -- start 2024-06-03T03:23:27.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:27.987+0000 7fbd98e00700 1 -- start start 2024-06-03T03:23:27.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:27.988+0000 7fbd98e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd94108aa0 0x7fbd9419f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:27.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:27.988+0000 7fbd98e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbd94109460 0x7fbd9419f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:27.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:27.988+0000 7fbd98e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbd9410eaa0 0x7fbd941a3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:27.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:27.988+0000 7fbd98e00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbd94118de0 con 0x7fbd94108aa0 2024-06-03T03:23:27.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:27.988+0000 7fbd98e00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbd94118c60 con 0x7fbd94109460 2024-06-03T03:23:27.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:27.988+0000 7fbd98e00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbd94118f60 con 0x7fbd9410eaa0 2024-06-03T03:23:27.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:27.988+0000 7fbd91e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd94108aa0 0x7fbd9419f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:27.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:27.988+0000 7fbd91e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd94108aa0 0x7fbd9419f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:54102/0 (socket says 172.21.15.117:54102) 2024-06-03T03:23:27.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:27.988+0000 7fbd91e00700 1 -- 172.21.15.117:0/3613069156 learned_addr learned my addr 172.21.15.117:0/3613069156 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:23:27.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:27.988+0000 7fbd91400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbd94109460 0x7fbd9419f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:27.990 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:27.988+0000 7fbd92800700 1 --2- 172.21.15.117:0/3613069156 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbd9410eaa0 0x7fbd941a3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:27.990 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:27.988+0000 7fbd91400700 1 -- 172.21.15.117:0/3613069156 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbd9410eaa0 msgr2=0x7fbd941a3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:27.990 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:27.988+0000 7fbd91400700 1 --2- 172.21.15.117:0/3613069156 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbd9410eaa0 0x7fbd941a3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:27.991 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:27.988+0000 7fbd91400700 1 -- 172.21.15.117:0/3613069156 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd94108aa0 msgr2=0x7fbd9419f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:27.991 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:27.988+0000 7fbd91400700 1 --2- 172.21.15.117:0/3613069156 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd94108aa0 0x7fbd9419f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:27.991 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:27.988+0000 7fbd91400700 1 -- 172.21.15.117:0/3613069156 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd941a4200 con 0x7fbd94109460 2024-06-03T03:23:27.991 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:27.989+0000 7fbd91e00700 1 --2- 172.21.15.117:0/3613069156 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd94108aa0 0x7fbd9419f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:23:27.991 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:27.989+0000 7fbd91400700 1 --2- 172.21.15.117:0/3613069156 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbd94109460 0x7fbd9419f730 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fbd8000e9f0 tx=0x7fbd8000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:23:27.991 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:27.989+0000 7fbd8aa00700 1 -- 172.21.15.117:0/3613069156 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd8000c9c0 con 0x7fbd94109460 2024-06-03T03:23:27.991 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:27.989+0000 7fbd8aa00700 1 -- 172.21.15.117:0/3613069156 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd8000cb60 con 0x7fbd94109460 2024-06-03T03:23:27.991 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:27.989+0000 7fbd98e00700 1 -- 172.21.15.117:0/3613069156 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd941a44f0 con 0x7fbd94109460 2024-06-03T03:23:27.991 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:27.989+0000 7fbd8aa00700 1 -- 172.21.15.117:0/3613069156 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd80018680 con 0x7fbd94109460 2024-06-03T03:23:27.991 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:27.989+0000 7fbd98e00700 1 -- 172.21.15.117:0/3613069156 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd941abd70 con 0x7fbd94109460 2024-06-03T03:23:27.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:27.990+0000 7fbd98e00700 1 -- 172.21.15.117:0/3613069156 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd94069180 con 0x7fbd94109460 2024-06-03T03:23:27.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:27.993+0000 7fbd8aa00700 1 -- 172.21.15.117:0/3613069156 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fbd80010430 con 0x7fbd94109460 2024-06-03T03:23:27.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:27.994+0000 7fbd8aa00700 1 --2- 172.21.15.117:0/3613069156 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fbd780a2c20 0x7fbd780a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:27.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:27.995+0000 7fbd8aa00700 1 -- 172.21.15.117:0/3613069156 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7fbd800cf880 con 0x7fbd94109460 2024-06-03T03:23:27.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:27.995+0000 7fbd91e00700 1 --2- 172.21.15.117:0/3613069156 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fbd780a2c20 0x7fbd780a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:27.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:27.995+0000 7fbd91e00700 1 --2- 172.21.15.117:0/3613069156 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fbd780a2c20 0x7fbd780a50e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbd84006010 tx=0x7fbd84009570 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:23:27.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:27.996+0000 7fbd8aa00700 1 -- 172.21.15.117:0/3613069156 <== mon.2 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fbd8009c3f0 con 0x7fbd94109460 2024-06-03T03:23:28.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:28.267+0000 7fbd98e00700 1 -- 172.21.15.117:0/3613069156 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbd9402dca0 con 0x7fbd94109460 2024-06-03T03:23:28.269 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:23:28 smithi117 ceph-mon[31649]: pgmap v450: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-03T03:23:28.270 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:28.268+0000 7fbd8aa00700 1 -- 172.21.15.117:0/3613069156 <== mon.2 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11955 (secure 0 0 0) 0x7fbd8009bcf0 con 0x7fbd94109460 2024-06-03T03:23:28.270 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T03:23:28.270 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3353001119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3353001119},{"type":"v1","addr":"172.21.15.117:6837","nonce":3353001119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14388,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6839/405782304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":405782304},{"type":"v1","addr":"172.21.15.117:6839","nonce":405782304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-03T03:19:44.629822+0000","modified":"2024-06-03T03:21:12.085445+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24517,"mds_1":24512,"mds_2":24503,"mds_3":24499,"mds_4":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24431":{"gid":24431,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6833/3355662416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3355662416},{"type":"v1","addr":"172.21.15.151:6833","nonce":3355662416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/2429725085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2429725085},{"type":"v1","addr":"172.21.15.183:6833","nonce":2429725085}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/31153579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":31153579},{"type":"v1","addr":"172.21.15.151:6835","nonce":31153579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"j","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/2884460088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2884460088},{"type":"v1","addr":"172.21.15.117:6835","nonce":2884460088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/2392407848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":2392407848},{"type":"v1","addr":"172.21.15.151:6837","nonce":2392407848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":32,"addr":"172.21.15.183:6835/1009004320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1009004320},{"type":"v1","addr":"172.21.15.183:6835","nonce":1009004320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.183:6837/2697518526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":2697518526},{"type":"v1","addr":"172.21.15.183:6837","nonce":2697518526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"a","rank":2,"incarnation":22,"state":"up:active","state_seq":27,"addr":"172.21.15.117:6841/1212242967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6840","nonce":1212242967},{"type":"v1","addr":"172.21.15.117:6841","nonce":1212242967}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.151:6839/3623444616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":3623444616},{"type":"v1","addr":"172.21.15.151:6839","nonce":3623444616}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":10,"addr":"172.21.15.183:6839/1439615083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1439615083},{"type":"v1","addr":"172.21.15.183:6839","nonce":1439615083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24517,"qdb_cluster":[24503,24517,24512,24487,24499]},"id":1}]} 2024-06-03T03:23:28.272 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:28.271+0000 7fbd98e00700 1 -- 172.21.15.117:0/3613069156 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fbd780a2c20 msgr2=0x7fbd780a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:28.272 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:28.271+0000 7fbd98e00700 1 --2- 172.21.15.117:0/3613069156 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fbd780a2c20 0x7fbd780a50e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbd84006010 tx=0x7fbd84009570 comp rx=0 tx=0).stop 2024-06-03T03:23:28.272 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:28.271+0000 7fbd98e00700 1 -- 172.21.15.117:0/3613069156 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbd94109460 msgr2=0x7fbd9419f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:28.272 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:28.271+0000 7fbd98e00700 1 --2- 172.21.15.117:0/3613069156 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbd94109460 0x7fbd9419f730 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fbd8000e9f0 tx=0x7fbd8000ed00 comp rx=0 tx=0).stop 2024-06-03T03:23:28.272 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:28.271+0000 7fbd98e00700 1 -- 172.21.15.117:0/3613069156 shutdown_connections 2024-06-03T03:23:28.272 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:28.271+0000 7fbd98e00700 1 --2- 172.21.15.117:0/3613069156 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fbd780a2c20 0x7fbd780a50e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:28.272 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:28.271+0000 7fbd98e00700 1 --2- 172.21.15.117:0/3613069156 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbd9410eaa0 0x7fbd941a3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:28.272 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:28.271+0000 7fbd98e00700 1 --2- 172.21.15.117:0/3613069156 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd94108aa0 0x7fbd9419f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:28.273 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:28.271+0000 7fbd98e00700 1 --2- 172.21.15.117:0/3613069156 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbd94109460 0x7fbd9419f730 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:28.273 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:28.271+0000 7fbd98e00700 1 -- 172.21.15.117:0/3613069156 >> 172.21.15.117:0/3613069156 conn(0x7fbd94075c60 msgr2=0x7fbd940761e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:23:28.273 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:28.272+0000 7fbd98e00700 1 -- 172.21.15.117:0/3613069156 shutdown_connections 2024-06-03T03:23:28.273 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:28.272+0000 7fbd98e00700 1 -- 172.21.15.117:0/3613069156 wait complete. 2024-06-03T03:23:28.274 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 30 2024-06-03T03:23:28.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:23:28 smithi151 ceph-mon[37043]: pgmap v450: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-03T03:23:28.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:23:28 smithi183 ceph-mon[37326]: pgmap v450: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-03T03:23:29.298 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:23:29 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/3613069156' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T03:23:29.299 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-03T03:23:29.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:23:29 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/3613069156' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T03:23:29.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:23:29 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/3613069156' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T03:23:29.740 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:23:30.172 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:23:30 smithi117 ceph-mon[31649]: pgmap v451: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-03T03:23:30.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:23:30 smithi151 ceph-mon[37043]: pgmap v451: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-03T03:23:30.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:23:30 smithi183 ceph-mon[37326]: pgmap v451: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-03T03:23:32.174 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:23:32 smithi117 ceph-mon[31649]: pgmap v452: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:23:32.232 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:32.231+0000 7f0a2a600700 1 -- 172.21.15.117:0/542402537 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a24101220 msgr2=0x7f0a24108270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:32.233 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:32.231+0000 7f0a2a600700 1 --2- 172.21.15.117:0/542402537 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a24101220 0x7f0a24108270 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0a100099c0 tx=0x7f0a10009cd0 comp rx=0 tx=0).stop 2024-06-03T03:23:32.233 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:32.231+0000 7f0a2a600700 1 -- 172.21.15.117:0/542402537 shutdown_connections 2024-06-03T03:23:32.233 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:32.231+0000 7f0a2a600700 1 --2- 172.21.15.117:0/542402537 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a24108950 0x7f0a2410f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:32.233 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:32.231+0000 7f0a2a600700 1 --2- 172.21.15.117:0/542402537 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0a24100870 0x7f0a24100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:32.233 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:32.231+0000 7f0a2a600700 1 --2- 172.21.15.117:0/542402537 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a24101220 0x7f0a24108270 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:32.233 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:32.231+0000 7f0a2a600700 1 -- 172.21.15.117:0/542402537 >> 172.21.15.117:0/542402537 conn(0x7f0a24077590 msgr2=0x7f0a240fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:23:32.233 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:32.231+0000 7f0a2a600700 1 -- 172.21.15.117:0/542402537 shutdown_connections 2024-06-03T03:23:32.233 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:32.231+0000 7f0a2a600700 1 -- 172.21.15.117:0/542402537 wait complete. 2024-06-03T03:23:32.233 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:32.232+0000 7f0a2a600700 1 Processor -- start 2024-06-03T03:23:32.235 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:32.232+0000 7f0a2a600700 1 -- start start 2024-06-03T03:23:32.235 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:32.233+0000 7f0a2a600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a24100870 0x7f0a2419e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:32.235 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:32.233+0000 7f0a2a600700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0a24101220 0x7f0a2419ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:32.235 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:32.233+0000 7f0a2a600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a24108950 0x7f0a24198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:32.235 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:32.233+0000 7f0a2a600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0a241127c0 con 0x7f0a24108950 2024-06-03T03:23:32.235 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:32.233+0000 7f0a2a600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0a24112640 con 0x7f0a24100870 2024-06-03T03:23:32.236 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:32.233+0000 7f0a2a600700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0a24112940 con 0x7f0a24101220 2024-06-03T03:23:32.236 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:32.233+0000 7f0a23400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a24100870 0x7f0a2419e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:32.236 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:32.233+0000 7f0a23400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a24100870 0x7f0a2419e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:49490/0 (socket says 172.21.15.117:49490) 2024-06-03T03:23:32.236 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:32.233+0000 7f0a23e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a24108950 0x7f0a24198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:32.236 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:32.233+0000 7f0a23400700 1 -- 172.21.15.117:0/1639661888 learned_addr learned my addr 172.21.15.117:0/1639661888 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:23:32.236 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:32.233+0000 7f0a22a00700 1 --2- 172.21.15.117:0/1639661888 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0a24101220 0x7f0a2419ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:32.236 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:32.234+0000 7f0a23400700 1 -- 172.21.15.117:0/1639661888 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0a24101220 msgr2=0x7f0a2419ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:32.236 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:32.234+0000 7f0a23400700 1 --2- 172.21.15.117:0/1639661888 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0a24101220 0x7f0a2419ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:32.236 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:32.234+0000 7f0a23400700 1 -- 172.21.15.117:0/1639661888 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a24108950 msgr2=0x7f0a24198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:32.236 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:32.234+0000 7f0a23400700 1 --2- 172.21.15.117:0/1639661888 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a24108950 0x7f0a24198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:32.236 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:32.234+0000 7f0a23400700 1 -- 172.21.15.117:0/1639661888 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a24199240 con 0x7f0a24100870 2024-06-03T03:23:32.236 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:32.234+0000 7f0a23e00700 1 --2- 172.21.15.117:0/1639661888 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a24108950 0x7f0a24198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:23:32.236 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:32.234+0000 7f0a23400700 1 --2- 172.21.15.117:0/1639661888 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a24100870 0x7f0a2419e920 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0a1400d950 tx=0x7f0a1400dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:23:32.236 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:32.234+0000 7f0a1be00700 1 -- 172.21.15.117:0/1639661888 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a14016070 con 0x7f0a24100870 2024-06-03T03:23:32.237 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:32.234+0000 7f0a1be00700 1 -- 172.21.15.117:0/1639661888 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a1400fd70 con 0x7f0a24100870 2024-06-03T03:23:32.237 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:32.234+0000 7f0a2a600700 1 -- 172.21.15.117:0/1639661888 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a24199530 con 0x7f0a24100870 2024-06-03T03:23:32.237 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:32.234+0000 7f0a1be00700 1 -- 172.21.15.117:0/1639661888 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a1400e610 con 0x7f0a24100870 2024-06-03T03:23:32.237 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:32.234+0000 7f0a2a600700 1 -- 172.21.15.117:0/1639661888 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a241a57f0 con 0x7f0a24100870 2024-06-03T03:23:32.237 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:32.235+0000 7f0a2a600700 1 -- 172.21.15.117:0/1639661888 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a24069180 con 0x7f0a24100870 2024-06-03T03:23:32.237 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:32.237+0000 7f0a1be00700 1 -- 172.21.15.117:0/1639661888 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f0a14019070 con 0x7f0a24100870 2024-06-03T03:23:32.239 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:32.238+0000 7f0a1be00700 1 --2- 172.21.15.117:0/1639661888 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f0a080a2ca0 0x7f0a080a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:32.239 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:32.239+0000 7f0a1be00700 1 -- 172.21.15.117:0/1639661888 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f0a140cee60 con 0x7f0a24100870 2024-06-03T03:23:32.239 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:32.239+0000 7f0a22a00700 1 --2- 172.21.15.117:0/1639661888 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f0a080a2ca0 0x7f0a080a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:32.240 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:32.239+0000 7f0a22a00700 1 --2- 172.21.15.117:0/1639661888 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f0a080a2ca0 0x7f0a080a5160 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0a10000c00 tx=0x7f0a10005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:23:32.244 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:32.243+0000 7f0a1be00700 1 -- 172.21.15.117:0/1639661888 <== mon.2 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0a1409b9d0 con 0x7f0a24100870 2024-06-03T03:23:32.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:23:32 smithi151 ceph-mon[37043]: pgmap v452: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:23:32.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:23:32 smithi183 ceph-mon[37326]: pgmap v452: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:23:32.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:32.519+0000 7f0a2a600700 1 -- 172.21.15.117:0/1639661888 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0a2402dca0 con 0x7f0a24100870 2024-06-03T03:23:32.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:32.521+0000 7f0a1be00700 1 -- 172.21.15.117:0/1639661888 <== mon.2 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11955 (secure 0 0 0) 0x7f0a1409b2d0 con 0x7f0a24100870 2024-06-03T03:23:32.522 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T03:23:32.522 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3353001119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3353001119},{"type":"v1","addr":"172.21.15.117:6837","nonce":3353001119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14388,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6839/405782304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":405782304},{"type":"v1","addr":"172.21.15.117:6839","nonce":405782304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-03T03:19:44.629822+0000","modified":"2024-06-03T03:21:12.085445+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24517,"mds_1":24512,"mds_2":24503,"mds_3":24499,"mds_4":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24431":{"gid":24431,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6833/3355662416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3355662416},{"type":"v1","addr":"172.21.15.151:6833","nonce":3355662416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/2429725085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2429725085},{"type":"v1","addr":"172.21.15.183:6833","nonce":2429725085}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/31153579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":31153579},{"type":"v1","addr":"172.21.15.151:6835","nonce":31153579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"j","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/2884460088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2884460088},{"type":"v1","addr":"172.21.15.117:6835","nonce":2884460088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/2392407848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":2392407848},{"type":"v1","addr":"172.21.15.151:6837","nonce":2392407848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":32,"addr":"172.21.15.183:6835/1009004320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1009004320},{"type":"v1","addr":"172.21.15.183:6835","nonce":1009004320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.183:6837/2697518526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":2697518526},{"type":"v1","addr":"172.21.15.183:6837","nonce":2697518526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"a","rank":2,"incarnation":22,"state":"up:active","state_seq":27,"addr":"172.21.15.117:6841/1212242967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6840","nonce":1212242967},{"type":"v1","addr":"172.21.15.117:6841","nonce":1212242967}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.151:6839/3623444616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":3623444616},{"type":"v1","addr":"172.21.15.151:6839","nonce":3623444616}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":10,"addr":"172.21.15.183:6839/1439615083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1439615083},{"type":"v1","addr":"172.21.15.183:6839","nonce":1439615083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24517,"qdb_cluster":[24503,24517,24512,24487,24499]},"id":1}]} 2024-06-03T03:23:32.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:32.524+0000 7f0a2a600700 1 -- 172.21.15.117:0/1639661888 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f0a080a2ca0 msgr2=0x7f0a080a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:32.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:32.524+0000 7f0a2a600700 1 --2- 172.21.15.117:0/1639661888 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f0a080a2ca0 0x7f0a080a5160 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0a10000c00 tx=0x7f0a10005fb0 comp rx=0 tx=0).stop 2024-06-03T03:23:32.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:32.524+0000 7f0a2a600700 1 -- 172.21.15.117:0/1639661888 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a24100870 msgr2=0x7f0a2419e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:32.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:32.524+0000 7f0a2a600700 1 --2- 172.21.15.117:0/1639661888 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a24100870 0x7f0a2419e920 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0a1400d950 tx=0x7f0a1400dc60 comp rx=0 tx=0).stop 2024-06-03T03:23:32.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:32.524+0000 7f0a2a600700 1 -- 172.21.15.117:0/1639661888 shutdown_connections 2024-06-03T03:23:32.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:32.524+0000 7f0a2a600700 1 --2- 172.21.15.117:0/1639661888 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f0a080a2ca0 0x7f0a080a5160 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:32.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:32.524+0000 7f0a2a600700 1 --2- 172.21.15.117:0/1639661888 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a24108950 0x7f0a24198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:32.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:32.524+0000 7f0a2a600700 1 --2- 172.21.15.117:0/1639661888 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a24100870 0x7f0a2419e920 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:32.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:32.524+0000 7f0a2a600700 1 --2- 172.21.15.117:0/1639661888 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0a24101220 0x7f0a2419ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:32.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:32.524+0000 7f0a2a600700 1 -- 172.21.15.117:0/1639661888 >> 172.21.15.117:0/1639661888 conn(0x7f0a24077590 msgr2=0x7f0a240ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:23:32.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:32.525+0000 7f0a2a600700 1 -- 172.21.15.117:0/1639661888 shutdown_connections 2024-06-03T03:23:32.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:32.525+0000 7f0a2a600700 1 -- 172.21.15.117:0/1639661888 wait complete. 2024-06-03T03:23:32.526 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 30 2024-06-03T03:23:33.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:23:33 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/1639661888' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T03:23:33.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:23:33 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/1639661888' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T03:23:33.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:23:33 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/1639661888' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T03:23:33.661 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-03T03:23:34.100 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:23:34.406 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:23:34 smithi117 ceph-mon[31649]: pgmap v453: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-06-03T03:23:34.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:23:34 smithi151 ceph-mon[37043]: pgmap v453: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-06-03T03:23:34.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:23:34 smithi183 ceph-mon[37326]: pgmap v453: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-06-03T03:23:35.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:23:35 smithi151 ceph-mon[37043]: pgmap v454: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:23:35.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:23:35 smithi117 ceph-mon[31649]: pgmap v454: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:23:35.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:23:35 smithi183 ceph-mon[37326]: pgmap v454: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:23:36.494 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:36.493+0000 7fdfcb200700 1 -- 172.21.15.117:0/2236774720 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdfcc103ea0 msgr2=0x7fdfcc10c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:36.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:36.493+0000 7fdfcb200700 1 --2- 172.21.15.117:0/2236774720 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdfcc103ea0 0x7fdfcc10c390 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdfb8009990 tx=0x7fdfb8009ca0 comp rx=0 tx=0).stop 2024-06-03T03:23:36.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:36.493+0000 7fdfcb200700 1 -- 172.21.15.117:0/2236774720 shutdown_connections 2024-06-03T03:23:36.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:36.493+0000 7fdfcb200700 1 --2- 172.21.15.117:0/2236774720 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdfcc10c9b0 0x7fdfcc10ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:36.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:36.493+0000 7fdfcb200700 1 --2- 172.21.15.117:0/2236774720 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfcc103580 0x7fdfcc103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:36.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:36.493+0000 7fdfcb200700 1 --2- 172.21.15.117:0/2236774720 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdfcc103ea0 0x7fdfcc10c390 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:36.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:36.493+0000 7fdfcb200700 1 -- 172.21.15.117:0/2236774720 >> 172.21.15.117:0/2236774720 conn(0x7fdfcc0fcdb0 msgr2=0x7fdfcc0ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:23:36.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:36.494+0000 7fdfcb200700 1 -- 172.21.15.117:0/2236774720 shutdown_connections 2024-06-03T03:23:36.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:36.494+0000 7fdfcb200700 1 -- 172.21.15.117:0/2236774720 wait complete. 2024-06-03T03:23:36.497 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:36.495+0000 7fdfcb200700 1 Processor -- start 2024-06-03T03:23:36.497 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:36.495+0000 7fdfcb200700 1 -- start start 2024-06-03T03:23:36.497 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:36.495+0000 7fdfcb200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfcc103580 0x7fdfcc1011e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:36.497 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:36.495+0000 7fdfcb200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdfcc103ea0 0x7fdfcc101720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:36.497 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:36.495+0000 7fdfcb200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdfcc10c9b0 0x7fdfcc101c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:36.498 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:36.495+0000 7fdfcb200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdfcc112750 con 0x7fdfcc103580 2024-06-03T03:23:36.498 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:36.495+0000 7fdfcb200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdfcc1125d0 con 0x7fdfcc103ea0 2024-06-03T03:23:36.498 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:36.495+0000 7fdfcb200700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdfcc1128d0 con 0x7fdfcc10c9b0 2024-06-03T03:23:36.498 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:36.496+0000 7fdfc9e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfcc103580 0x7fdfcc1011e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:36.498 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:36.496+0000 7fdfc9e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfcc103580 0x7fdfcc1011e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:54144/0 (socket says 172.21.15.117:54144) 2024-06-03T03:23:36.498 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:36.496+0000 7fdfc9e00700 1 -- 172.21.15.117:0/2624748701 learned_addr learned my addr 172.21.15.117:0/2624748701 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:23:36.498 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:36.496+0000 7fdfc9400700 1 --2- 172.21.15.117:0/2624748701 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdfcc103ea0 0x7fdfcc101720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:36.498 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:36.496+0000 7fdfca800700 1 --2- 172.21.15.117:0/2624748701 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdfcc10c9b0 0x7fdfcc101c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:36.498 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:36.496+0000 7fdfc9e00700 1 -- 172.21.15.117:0/2624748701 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdfcc10c9b0 msgr2=0x7fdfcc101c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:36.498 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:36.496+0000 7fdfc9e00700 1 --2- 172.21.15.117:0/2624748701 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdfcc10c9b0 0x7fdfcc101c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:36.498 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:36.496+0000 7fdfc9e00700 1 -- 172.21.15.117:0/2624748701 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdfcc103ea0 msgr2=0x7fdfcc101720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:36.498 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:36.496+0000 7fdfc9e00700 1 --2- 172.21.15.117:0/2624748701 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdfcc103ea0 0x7fdfcc101720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:36.499 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:36.496+0000 7fdfc9e00700 1 -- 172.21.15.117:0/2624748701 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfcc1a7140 con 0x7fdfcc103580 2024-06-03T03:23:36.499 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:36.496+0000 7fdfc9400700 1 --2- 172.21.15.117:0/2624748701 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdfcc103ea0 0x7fdfcc101720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:23:36.499 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:36.496+0000 7fdfca800700 1 --2- 172.21.15.117:0/2624748701 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdfcc10c9b0 0x7fdfcc101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:23:36.499 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:36.497+0000 7fdfc9e00700 1 --2- 172.21.15.117:0/2624748701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfcc103580 0x7fdfcc1011e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc00e9a0 tx=0x7fdfbc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:23:36.499 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:36.497+0000 7fdfc2a00700 1 -- 172.21.15.117:0/2624748701 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc00ca10 con 0x7fdfcc103580 2024-06-03T03:23:36.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:36.497+0000 7fdfcb200700 1 -- 172.21.15.117:0/2624748701 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfcc1a7430 con 0x7fdfcc103580 2024-06-03T03:23:36.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:36.497+0000 7fdfc2a00700 1 -- 172.21.15.117:0/2624748701 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfbc00cbb0 con 0x7fdfcc103580 2024-06-03T03:23:36.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:36.497+0000 7fdfcb200700 1 -- 172.21.15.117:0/2624748701 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfcc1a7990 con 0x7fdfcc103580 2024-06-03T03:23:36.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:36.497+0000 7fdfc2a00700 1 -- 172.21.15.117:0/2624748701 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc018620 con 0x7fdfcc103580 2024-06-03T03:23:36.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:36.498+0000 7fdfcb200700 1 -- 172.21.15.117:0/2624748701 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfcc069180 con 0x7fdfcc103580 2024-06-03T03:23:36.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:36.500+0000 7fdfc2a00700 1 -- 172.21.15.117:0/2624748701 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fdfbc010430 con 0x7fdfcc103580 2024-06-03T03:23:36.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:36.501+0000 7fdfc2a00700 1 --2- 172.21.15.117:0/2624748701 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fdfb00a2c20 0x7fdfb00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:36.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:36.502+0000 7fdfc2a00700 1 -- 172.21.15.117:0/2624748701 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7fdfbc0cf5e0 con 0x7fdfcc103580 2024-06-03T03:23:36.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:36.502+0000 7fdfc9400700 1 --2- 172.21.15.117:0/2624748701 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fdfb00a2c20 0x7fdfb00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:36.503 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:36.503+0000 7fdfc9400700 1 --2- 172.21.15.117:0/2624748701 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fdfb00a2c20 0x7fdfb00a50e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc102c20 tx=0x7fdfb8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:23:36.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:36.507+0000 7fdfc2a00700 1 -- 172.21.15.117:0/2624748701 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fdfbc09c150 con 0x7fdfcc103580 2024-06-03T03:23:36.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:36.730+0000 7fdfcb200700 1 -- 172.21.15.117:0/2624748701 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdfcc02dca0 con 0x7fdfcc103580 2024-06-03T03:23:36.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:36.732+0000 7fdfc2a00700 1 -- 172.21.15.117:0/2624748701 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+24807 (secure 0 0 0) 0x7fdfbc09ba50 con 0x7fdfcc103580 2024-06-03T03:23:36.734 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T03:23:36.734 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":99,"fsid":"a0e8473a-2156-11ef-bc9b-c7b262605968","created":"2024-06-03T03:10:08.446859+0000","modified":"2024-06-03T03:21:25.966405+0000","last_up_change":"2024-06-03T03:18:28.736904+0000","last_in_change":"2024-06-03T03:18:08.485711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-03T03:14:09.370986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-03T03:19:35.044683+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-03T03:19:40.097569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-03T03:19:55.394480+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8fefa089-9401-4a1c-b9fb-fe30c63e6a1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":4286644492},{"type":"v1","addr":"172.21.15.117:6803","nonce":4286644492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":4286644492},{"type":"v1","addr":"172.21.15.117:6805","nonce":4286644492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":4286644492},{"type":"v1","addr":"172.21.15.117:6809","nonce":4286644492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":4286644492},{"type":"v1","addr":"172.21.15.117:6807","nonce":4286644492}]},"public_addr":"172.21.15.117:6803/4286644492","cluster_addr":"172.21.15.117:6805/4286644492","heartbeat_back_addr":"172.21.15.117:6809/4286644492","heartbeat_front_addr":"172.21.15.117:6807/4286644492","state":["exists","up"]},{"osd":1,"uuid":"5b12b371-f155-4b67-a470-25cbe59a2999","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":262491158},{"type":"v1","addr":"172.21.15.151:6801","nonce":262491158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":262491158},{"type":"v1","addr":"172.21.15.151:6803","nonce":262491158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":262491158},{"type":"v1","addr":"172.21.15.151:6807","nonce":262491158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":262491158},{"type":"v1","addr":"172.21.15.151:6805","nonce":262491158}]},"public_addr":"172.21.15.151:6801/262491158","cluster_addr":"172.21.15.151:6803/262491158","heartbeat_back_addr":"172.21.15.151:6807/262491158","heartbeat_front_addr":"172.21.15.151:6805/262491158","state":["exists","up"]},{"osd":2,"uuid":"dbc17818-c54c-4ecf-b656-03578b95d8c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":3542453623},{"type":"v1","addr":"172.21.15.183:6801","nonce":3542453623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":3542453623},{"type":"v1","addr":"172.21.15.183:6803","nonce":3542453623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":3542453623},{"type":"v1","addr":"172.21.15.183:6807","nonce":3542453623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":3542453623},{"type":"v1","addr":"172.21.15.183:6805","nonce":3542453623}]},"public_addr":"172.21.15.183:6801/3542453623","cluster_addr":"172.21.15.183:6803/3542453623","heartbeat_back_addr":"172.21.15.183:6807/3542453623","heartbeat_front_addr":"172.21.15.183:6805/3542453623","state":["exists","up"]},{"osd":3,"uuid":"ca08d2ac-b1d6-4a49-9ae8-e3ec146899a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":589012315},{"type":"v1","addr":"172.21.15.117:6811","nonce":589012315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":589012315},{"type":"v1","addr":"172.21.15.117:6813","nonce":589012315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":589012315},{"type":"v1","addr":"172.21.15.117:6817","nonce":589012315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":589012315},{"type":"v1","addr":"172.21.15.117:6815","nonce":589012315}]},"public_addr":"172.21.15.117:6811/589012315","cluster_addr":"172.21.15.117:6813/589012315","heartbeat_back_addr":"172.21.15.117:6817/589012315","heartbeat_front_addr":"172.21.15.117:6815/589012315","state":["exists","up"]},{"osd":4,"uuid":"810d6c7a-023f-4930-82af-bc2bbfd647da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":1239048647},{"type":"v1","addr":"172.21.15.151:6809","nonce":1239048647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":1239048647},{"type":"v1","addr":"172.21.15.151:6811","nonce":1239048647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":1239048647},{"type":"v1","addr":"172.21.15.151:6815","nonce":1239048647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":1239048647},{"type":"v1","addr":"172.21.15.151:6813","nonce":1239048647}]},"public_addr":"172.21.15.151:6809/1239048647","cluster_addr":"172.21.15.151:6811/1239048647","heartbeat_back_addr":"172.21.15.151:6815/1239048647","heartbeat_front_addr":"172.21.15.151:6813/1239048647","state":["exists","up"]},{"osd":5,"uuid":"8ab6a5f0-34bb-48d8-9471-a7dc836e486e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":2985874686},{"type":"v1","addr":"172.21.15.183:6809","nonce":2985874686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":2985874686},{"type":"v1","addr":"172.21.15.183:6811","nonce":2985874686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":2985874686},{"type":"v1","addr":"172.21.15.183:6815","nonce":2985874686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":2985874686},{"type":"v1","addr":"172.21.15.183:6813","nonce":2985874686}]},"public_addr":"172.21.15.183:6809/2985874686","cluster_addr":"172.21.15.183:6811/2985874686","heartbeat_back_addr":"172.21.15.183:6815/2985874686","heartbeat_front_addr":"172.21.15.183:6813/2985874686","state":["exists","up"]},{"osd":6,"uuid":"78d15739-e9e0-4f19-ae5f-ee8527a5b527","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4250583193},{"type":"v1","addr":"172.21.15.117:6819","nonce":4250583193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":4250583193},{"type":"v1","addr":"172.21.15.117:6821","nonce":4250583193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":4250583193},{"type":"v1","addr":"172.21.15.117:6825","nonce":4250583193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4250583193},{"type":"v1","addr":"172.21.15.117:6823","nonce":4250583193}]},"public_addr":"172.21.15.117:6819/4250583193","cluster_addr":"172.21.15.117:6821/4250583193","heartbeat_back_addr":"172.21.15.117:6825/4250583193","heartbeat_front_addr":"172.21.15.117:6823/4250583193","state":["exists","up"]},{"osd":7,"uuid":"3724fe1f-f4d7-443b-aea8-ff3ab02bfdc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":386094645},{"type":"v1","addr":"172.21.15.151:6817","nonce":386094645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":386094645},{"type":"v1","addr":"172.21.15.151:6819","nonce":386094645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":386094645},{"type":"v1","addr":"172.21.15.151:6823","nonce":386094645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":386094645},{"type":"v1","addr":"172.21.15.151:6821","nonce":386094645}]},"public_addr":"172.21.15.151:6817/386094645","cluster_addr":"172.21.15.151:6819/386094645","heartbeat_back_addr":"172.21.15.151:6823/386094645","heartbeat_front_addr":"172.21.15.151:6821/386094645","state":["exists","up"]},{"osd":8,"uuid":"006ac092-f015-448e-a0c0-f7b0c4597a18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1829748222},{"type":"v1","addr":"172.21.15.183:6817","nonce":1829748222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1829748222},{"type":"v1","addr":"172.21.15.183:6819","nonce":1829748222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1829748222},{"type":"v1","addr":"172.21.15.183:6823","nonce":1829748222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1829748222},{"type":"v1","addr":"172.21.15.183:6821","nonce":1829748222}]},"public_addr":"172.21.15.183:6817/1829748222","cluster_addr":"172.21.15.183:6819/1829748222","heartbeat_back_addr":"172.21.15.183:6823/1829748222","heartbeat_front_addr":"172.21.15.183:6821/1829748222","state":["exists","up"]},{"osd":9,"uuid":"00a970e6-2502-459e-975c-badac93c9bd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2538315141},{"type":"v1","addr":"172.21.15.117:6827","nonce":2538315141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2538315141},{"type":"v1","addr":"172.21.15.117:6829","nonce":2538315141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2538315141},{"type":"v1","addr":"172.21.15.117:6833","nonce":2538315141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2538315141},{"type":"v1","addr":"172.21.15.117:6831","nonce":2538315141}]},"public_addr":"172.21.15.117:6827/2538315141","cluster_addr":"172.21.15.117:6829/2538315141","heartbeat_back_addr":"172.21.15.117:6833/2538315141","heartbeat_front_addr":"172.21.15.117:6831/2538315141","state":["exists","up"]},{"osd":10,"uuid":"ba2333cd-32a3-41ef-9ea5-287c707f512b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":3801045133},{"type":"v1","addr":"172.21.15.151:6825","nonce":3801045133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3801045133},{"type":"v1","addr":"172.21.15.151:6827","nonce":3801045133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3801045133},{"type":"v1","addr":"172.21.15.151:6831","nonce":3801045133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3801045133},{"type":"v1","addr":"172.21.15.151:6829","nonce":3801045133}]},"public_addr":"172.21.15.151:6825/3801045133","cluster_addr":"172.21.15.151:6827/3801045133","heartbeat_back_addr":"172.21.15.151:6831/3801045133","heartbeat_front_addr":"172.21.15.151:6829/3801045133","state":["exists","up"]},{"osd":11,"uuid":"1556d654-e6fa-42c1-b399-aa53f6837a1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3513899696},{"type":"v1","addr":"172.21.15.183:6825","nonce":3513899696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3513899696},{"type":"v1","addr":"172.21.15.183:6827","nonce":3513899696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3513899696},{"type":"v1","addr":"172.21.15.183:6831","nonce":3513899696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3513899696},{"type":"v1","addr":"172.21.15.183:6829","nonce":3513899696}]},"public_addr":"172.21.15.183:6825/3513899696","cluster_addr":"172.21.15.183:6827/3513899696","heartbeat_back_addr":"172.21.15.183:6831/3513899696","heartbeat_front_addr":"172.21.15.183:6829/3513899696","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:13:17.174041+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:13:42.237001+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:14:06.273315+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:14:39.912564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:15:06.063202+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:15:30.424818+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:16:05.623554+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:16:32.926393+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:16:58.165899+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:17:32.897477+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:18:00.720866+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:18:26.474441+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.117:0/186309008":"2024-06-04T03:11:24.655516+0000","172.21.15.117:6801/2142313590":"2024-06-04T03:11:24.655516+0000","172.21.15.117:6800/2142313590":"2024-06-04T03:11:24.655516+0000","172.21.15.117:0/2399725156":"2024-06-04T03:11:24.655516+0000","172.21.15.117:0/1453232061":"2024-06-04T03:10:50.552899+0000","172.21.15.117:0/4281674974":"2024-06-04T03:10:50.552899+0000","172.21.15.117:6800/2424554985":"2024-06-04T03:10:50.552899+0000","172.21.15.117:0/3946365225":"2024-06-04T03:11:24.655516+0000","172.21.15.117:0/1146162354":"2024-06-04T03:10:50.552899+0000","172.21.15.117:6801/2424554985":"2024-06-04T03:10:50.552899+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-03T03:23:36.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:36.736+0000 7fdfcb200700 1 -- 172.21.15.117:0/2624748701 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fdfb00a2c20 msgr2=0x7fdfb00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:36.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:36.736+0000 7fdfcb200700 1 --2- 172.21.15.117:0/2624748701 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fdfb00a2c20 0x7fdfb00a50e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc102c20 tx=0x7fdfb8005fd0 comp rx=0 tx=0).stop 2024-06-03T03:23:36.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:36.736+0000 7fdfcb200700 1 -- 172.21.15.117:0/2624748701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfcc103580 msgr2=0x7fdfcc1011e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:36.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:36.736+0000 7fdfcb200700 1 --2- 172.21.15.117:0/2624748701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfcc103580 0x7fdfcc1011e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc00e9a0 tx=0x7fdfbc00ecb0 comp rx=0 tx=0).stop 2024-06-03T03:23:36.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:36.736+0000 7fdfcb200700 1 -- 172.21.15.117:0/2624748701 shutdown_connections 2024-06-03T03:23:36.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:36.736+0000 7fdfcb200700 1 --2- 172.21.15.117:0/2624748701 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fdfb00a2c20 0x7fdfb00a50e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:36.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:36.736+0000 7fdfcb200700 1 --2- 172.21.15.117:0/2624748701 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdfcc10c9b0 0x7fdfcc101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:36.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:36.736+0000 7fdfcb200700 1 --2- 172.21.15.117:0/2624748701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfcc103580 0x7fdfcc1011e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:36.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:36.736+0000 7fdfcb200700 1 --2- 172.21.15.117:0/2624748701 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdfcc103ea0 0x7fdfcc101720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:36.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:36.736+0000 7fdfcb200700 1 -- 172.21.15.117:0/2624748701 >> 172.21.15.117:0/2624748701 conn(0x7fdfcc0fcdb0 msgr2=0x7fdfcc0fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:23:36.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:36.736+0000 7fdfcb200700 1 -- 172.21.15.117:0/2624748701 shutdown_connections 2024-06-03T03:23:36.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:36.736+0000 7fdfcb200700 1 -- 172.21.15.117:0/2624748701 wait complete. 2024-06-03T03:23:37.127 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:23:36 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/2624748701' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T03:23:37.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:23:36 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/2624748701' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T03:23:37.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:23:36 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/2624748701' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T03:23:37.672 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-03T03:23:37.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:23:37 smithi117 ceph-mon[31649]: pgmap v455: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-03T03:23:38.119 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:23:38.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:23:37 smithi151 ceph-mon[37043]: pgmap v455: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-03T03:23:38.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:23:37 smithi183 ceph-mon[37326]: pgmap v455: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-03T03:23:40.202 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:23:39 smithi117 ceph-mon[31649]: pgmap v456: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-03T03:23:40.231 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:23:39 smithi183 ceph-mon[37326]: pgmap v456: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-03T03:23:40.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:23:39 smithi151 ceph-mon[37043]: pgmap v456: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-03T03:23:40.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:40.494+0000 7f675a400700 1 -- 172.21.15.117:0/2397105220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6754104680 msgr2=0x7f6754104b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:40.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:40.494+0000 7f675a400700 1 --2- 172.21.15.117:0/2397105220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6754104680 0x7f6754104b00 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6740009990 tx=0x7f6740009ca0 comp rx=0 tx=0).stop 2024-06-03T03:23:40.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:40.494+0000 7f675a400700 1 -- 172.21.15.117:0/2397105220 shutdown_connections 2024-06-03T03:23:40.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:40.494+0000 7f675a400700 1 --2- 172.21.15.117:0/2397105220 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6754105040 0x7f6754111550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:40.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:40.494+0000 7f675a400700 1 --2- 172.21.15.117:0/2397105220 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f675410a680 0x7f675410aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:40.497 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:40.494+0000 7f675a400700 1 --2- 172.21.15.117:0/2397105220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6754104680 0x7f6754104b00 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:40.497 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:40.494+0000 7f675a400700 1 -- 172.21.15.117:0/2397105220 >> 172.21.15.117:0/2397105220 conn(0x7f6754100150 msgr2=0x7f6754102570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:23:40.497 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:40.495+0000 7f675a400700 1 -- 172.21.15.117:0/2397105220 shutdown_connections 2024-06-03T03:23:40.497 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:40.495+0000 7f675a400700 1 -- 172.21.15.117:0/2397105220 wait complete. 2024-06-03T03:23:40.499 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:40.496+0000 7f675a400700 1 Processor -- start 2024-06-03T03:23:40.499 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:40.496+0000 7f675a400700 1 -- start start 2024-06-03T03:23:40.499 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:40.497+0000 7f675a400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6754104680 0x7f675419ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:40.499 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:40.497+0000 7f675a400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6754105040 0x7f675419b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:40.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:40.497+0000 7f675a400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f675410a680 0x7f675419f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:40.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:40.497+0000 7f675a400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6754114a60 con 0x7f6754104680 2024-06-03T03:23:40.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:40.497+0000 7f675a400700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f67541148e0 con 0x7f6754105040 2024-06-03T03:23:40.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:40.497+0000 7f675a400700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6754114be0 con 0x7f675410a680 2024-06-03T03:23:40.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:40.497+0000 7f6753400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6754104680 0x7f675419ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:40.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:40.497+0000 7f6752a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6754105040 0x7f675419b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:40.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:40.497+0000 7f6752a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6754105040 0x7f675419b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:59200/0 (socket says 172.21.15.117:59200) 2024-06-03T03:23:40.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:40.497+0000 7f6753400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6754104680 0x7f675419ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:45378/0 (socket says 172.21.15.117:45378) 2024-06-03T03:23:40.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:40.497+0000 7f6752a00700 1 -- 172.21.15.117:0/3349782602 learned_addr learned my addr 172.21.15.117:0/3349782602 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:23:40.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:40.497+0000 7f6753e00700 1 --2- 172.21.15.117:0/3349782602 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f675410a680 0x7f675419f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:40.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:40.498+0000 7f6752a00700 1 -- 172.21.15.117:0/3349782602 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f675410a680 msgr2=0x7f675419f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:40.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:40.498+0000 7f6752a00700 1 --2- 172.21.15.117:0/3349782602 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f675410a680 0x7f675419f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:40.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:40.498+0000 7f6752a00700 1 -- 172.21.15.117:0/3349782602 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6754104680 msgr2=0x7f675419ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:40.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:40.498+0000 7f6752a00700 1 --2- 172.21.15.117:0/3349782602 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6754104680 0x7f675419ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:40.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:40.498+0000 7f6752a00700 1 -- 172.21.15.117:0/3349782602 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f675419fe30 con 0x7f6754105040 2024-06-03T03:23:40.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:40.498+0000 7f6753400700 1 --2- 172.21.15.117:0/3349782602 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6754104680 0x7f675419ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:23:40.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:40.498+0000 7f6752a00700 1 --2- 172.21.15.117:0/3349782602 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6754105040 0x7f675419b3b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6740000c00 tx=0x7f674000bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:23:40.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:40.498+0000 7f674be00700 1 -- 172.21.15.117:0/3349782602 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6740027070 con 0x7f6754105040 2024-06-03T03:23:40.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:40.498+0000 7f675a400700 1 -- 172.21.15.117:0/3349782602 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67541a00c0 con 0x7f6754105040 2024-06-03T03:23:40.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:40.498+0000 7f674be00700 1 -- 172.21.15.117:0/3349782602 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67400056d0 con 0x7f6754105040 2024-06-03T03:23:40.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:40.498+0000 7f674be00700 1 -- 172.21.15.117:0/3349782602 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6740022e10 con 0x7f6754105040 2024-06-03T03:23:40.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:40.498+0000 7f675a400700 1 -- 172.21.15.117:0/3349782602 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67541a79a0 con 0x7f6754105040 2024-06-03T03:23:40.503 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:40.499+0000 7f675a400700 1 -- 172.21.15.117:0/3349782602 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6754069180 con 0x7f6754105040 2024-06-03T03:23:40.503 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:40.502+0000 7f674be00700 1 -- 172.21.15.117:0/3349782602 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f674002a070 con 0x7f6754105040 2024-06-03T03:23:40.503 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:40.503+0000 7f674be00700 1 --2- 172.21.15.117:0/3349782602 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f67380a2c20 0x7f67380a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:40.504 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:40.503+0000 7f674be00700 1 -- 172.21.15.117:0/3349782602 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f67400d06d0 con 0x7f6754105040 2024-06-03T03:23:40.504 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:40.503+0000 7f6753400700 1 --2- 172.21.15.117:0/3349782602 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f67380a2c20 0x7f67380a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:40.504 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:40.504+0000 7f6753400700 1 --2- 172.21.15.117:0/3349782602 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f67380a2c20 0x7f67380a50e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6744009fd0 tx=0x7f6744009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:23:40.506 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:40.505+0000 7f674be00700 1 -- 172.21.15.117:0/3349782602 <== mon.2 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6740013070 con 0x7f6754105040 2024-06-03T03:23:40.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:40.774+0000 7f675a400700 1 -- 172.21.15.117:0/3349782602 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f675419c180 con 0x7f6754105040 2024-06-03T03:23:40.777 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:40.776+0000 7f674be00700 1 -- 172.21.15.117:0/3349782602 <== mon.2 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11955 (secure 0 0 0) 0x7f674009d2d0 con 0x7f6754105040 2024-06-03T03:23:40.777 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T03:23:40.777 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3353001119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3353001119},{"type":"v1","addr":"172.21.15.117:6837","nonce":3353001119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14388,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6839/405782304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":405782304},{"type":"v1","addr":"172.21.15.117:6839","nonce":405782304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-03T03:19:44.629822+0000","modified":"2024-06-03T03:21:12.085445+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24517,"mds_1":24512,"mds_2":24503,"mds_3":24499,"mds_4":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24431":{"gid":24431,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6833/3355662416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3355662416},{"type":"v1","addr":"172.21.15.151:6833","nonce":3355662416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/2429725085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2429725085},{"type":"v1","addr":"172.21.15.183:6833","nonce":2429725085}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/31153579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":31153579},{"type":"v1","addr":"172.21.15.151:6835","nonce":31153579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"j","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/2884460088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2884460088},{"type":"v1","addr":"172.21.15.117:6835","nonce":2884460088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/2392407848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":2392407848},{"type":"v1","addr":"172.21.15.151:6837","nonce":2392407848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":32,"addr":"172.21.15.183:6835/1009004320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1009004320},{"type":"v1","addr":"172.21.15.183:6835","nonce":1009004320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.183:6837/2697518526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":2697518526},{"type":"v1","addr":"172.21.15.183:6837","nonce":2697518526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"a","rank":2,"incarnation":22,"state":"up:active","state_seq":27,"addr":"172.21.15.117:6841/1212242967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6840","nonce":1212242967},{"type":"v1","addr":"172.21.15.117:6841","nonce":1212242967}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.151:6839/3623444616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":3623444616},{"type":"v1","addr":"172.21.15.151:6839","nonce":3623444616}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":10,"addr":"172.21.15.183:6839/1439615083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1439615083},{"type":"v1","addr":"172.21.15.183:6839","nonce":1439615083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24517,"qdb_cluster":[24503,24517,24512,24487,24499]},"id":1}]} 2024-06-03T03:23:40.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:40.779+0000 7f675a400700 1 -- 172.21.15.117:0/3349782602 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f67380a2c20 msgr2=0x7f67380a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:40.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:40.779+0000 7f675a400700 1 --2- 172.21.15.117:0/3349782602 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f67380a2c20 0x7f67380a50e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6744009fd0 tx=0x7f6744009260 comp rx=0 tx=0).stop 2024-06-03T03:23:40.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:40.779+0000 7f675a400700 1 -- 172.21.15.117:0/3349782602 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6754105040 msgr2=0x7f675419b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:40.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:40.779+0000 7f675a400700 1 --2- 172.21.15.117:0/3349782602 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6754105040 0x7f675419b3b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6740000c00 tx=0x7f674000bab0 comp rx=0 tx=0).stop 2024-06-03T03:23:40.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:40.779+0000 7f675a400700 1 -- 172.21.15.117:0/3349782602 shutdown_connections 2024-06-03T03:23:40.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:40.779+0000 7f675a400700 1 --2- 172.21.15.117:0/3349782602 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f67380a2c20 0x7f67380a50e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:40.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:40.779+0000 7f675a400700 1 --2- 172.21.15.117:0/3349782602 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f675410a680 0x7f675419f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:40.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:40.779+0000 7f675a400700 1 --2- 172.21.15.117:0/3349782602 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6754104680 0x7f675419ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:40.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:40.779+0000 7f675a400700 1 --2- 172.21.15.117:0/3349782602 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6754105040 0x7f675419b3b0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:40.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:40.779+0000 7f675a400700 1 -- 172.21.15.117:0/3349782602 >> 172.21.15.117:0/3349782602 conn(0x7f6754100150 msgr2=0x7f6754100750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:23:40.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:40.779+0000 7f675a400700 1 -- 172.21.15.117:0/3349782602 shutdown_connections 2024-06-03T03:23:40.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:40.779+0000 7f675a400700 1 -- 172.21.15.117:0/3349782602 wait complete. 2024-06-03T03:23:40.782 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 30 2024-06-03T03:23:41.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:23:40 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/3349782602' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T03:23:41.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:23:40 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/3349782602' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T03:23:41.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:23:40 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/3349782602' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T03:23:41.680 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 30, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-06-03T03:19:44.629822+0000', 'modified': '2024-06-03T03:21:12.085445+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24517, 'mds_1': 24512, 'mds_2': 24503, 'mds_3': 24499, 'mds_4': 24487}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24431': {'gid': 24431, 'name': 'k', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.151:6833/3355662416', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6832', 'nonce': 3355662416}, {'type': 'v1', 'addr': '172.21.15.151:6833', 'nonce': 3355662416}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24445': {'gid': 24445, 'name': 'l', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.183:6833/2429725085', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6832', 'nonce': 2429725085}, {'type': 'v1', 'addr': '172.21.15.183:6833', 'nonce': 2429725085}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24461': {'gid': 24461, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.151:6835/31153579', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6834', 'nonce': 31153579}, {'type': 'v1', 'addr': '172.21.15.151:6835', 'nonce': 31153579}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24466': {'gid': 24466, 'name': 'j', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.117:6835/2884460088', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 2884460088}, {'type': 'v1', 'addr': '172.21.15.117:6835', 'nonce': 2884460088}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24479': {'gid': 24479, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.151:6837/2392407848', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6836', 'nonce': 2392407848}, {'type': 'v1', 'addr': '172.21.15.151:6837', 'nonce': 2392407848}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24487': {'gid': 24487, 'name': 'f', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 32, 'addr': '172.21.15.183:6835/1009004320', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6834', 'nonce': 1009004320}, {'type': 'v1', 'addr': '172.21.15.183:6835', 'nonce': 1009004320}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24499': {'gid': 24499, 'name': 'c', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 31, 'addr': '172.21.15.183:6837/2697518526', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6836', 'nonce': 2697518526}, {'type': 'v1', 'addr': '172.21.15.183:6837', 'nonce': 2697518526}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24503': {'gid': 24503, 'name': 'a', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 27, 'addr': '172.21.15.117:6841/1212242967', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6840', 'nonce': 1212242967}, {'type': 'v1', 'addr': '172.21.15.117:6841', 'nonce': 1212242967}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24512': {'gid': 24512, 'name': 'b', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 26, 'addr': '172.21.15.151:6839/3623444616', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6838', 'nonce': 3623444616}, {'type': 'v1', 'addr': '172.21.15.151:6839', 'nonce': 3623444616}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24517': {'gid': 24517, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.183:6839/1439615083', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6838', 'nonce': 1439615083}, {'type': 'v1', 'addr': '172.21.15.183:6839', 'nonce': 1439615083}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24517, 'qdb_cluster': [24503, 24517, 24512, 24487, 24499]} 2024-06-03T03:23:41.680 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-06-03T03:23:41.680 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph tell mds.f status 2024-06-03T03:23:42.135 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:23:42.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:23:41 smithi117 ceph-mon[31649]: pgmap v457: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:23:42.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:23:41 smithi183 ceph-mon[37326]: pgmap v457: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:23:42.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:23:41 smithi151 ceph-mon[37043]: pgmap v457: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:23:44.181 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:23:43 smithi117 ceph-mon[31649]: pgmap v458: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-06-03T03:23:44.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:23:43 smithi151 ceph-mon[37043]: pgmap v458: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-06-03T03:23:44.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:23:43 smithi183 ceph-mon[37326]: pgmap v458: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-06-03T03:23:44.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.715+0000 7f82db600700 1 -- 172.21.15.117:0/3446305976 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82d4100e30 msgr2=0x7f82d4101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:44.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.715+0000 7f82db600700 1 --2- 172.21.15.117:0/3446305976 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82d4100e30 0x7f82d4101290 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f82c0009990 tx=0x7f82c0009ca0 comp rx=0 tx=0).stop 2024-06-03T03:23:44.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.715+0000 7f82db600700 1 -- 172.21.15.117:0/3446305976 shutdown_connections 2024-06-03T03:23:44.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.715+0000 7f82db600700 1 --2- 172.21.15.117:0/3446305976 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f82d41017d0 0x7f82d410f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:44.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.715+0000 7f82db600700 1 --2- 172.21.15.117:0/3446305976 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82d4107570 0x7f82d4107950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:44.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.715+0000 7f82db600700 1 --2- 172.21.15.117:0/3446305976 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82d4100e30 0x7f82d4101290 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:44.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.715+0000 7f82db600700 1 -- 172.21.15.117:0/3446305976 >> 172.21.15.117:0/3446305976 conn(0x7f82d4076680 msgr2=0x7f82d4076a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:23:44.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.715+0000 7f82db600700 1 -- 172.21.15.117:0/3446305976 shutdown_connections 2024-06-03T03:23:44.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.715+0000 7f82db600700 1 -- 172.21.15.117:0/3446305976 wait complete. 2024-06-03T03:23:44.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.716+0000 7f82db600700 1 Processor -- start 2024-06-03T03:23:44.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.717+0000 7f82db600700 1 -- start start 2024-06-03T03:23:44.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.717+0000 7f82db600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f82d4100e30 0x7f82d419aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:44.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.717+0000 7f82db600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82d41017d0 0x7f82d419b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:44.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.717+0000 7f82db600700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82d4107570 0x7f82d419f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:44.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.717+0000 7f82db600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f82d41127d0 con 0x7f82d41017d0 2024-06-03T03:23:44.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.717+0000 7f82db600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f82d4112650 con 0x7f82d4100e30 2024-06-03T03:23:44.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.717+0000 7f82db600700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f82d4112950 con 0x7f82d4107570 2024-06-03T03:23:44.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.717+0000 7f82d3e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82d41017d0 0x7f82d419b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:44.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.717+0000 7f82d8c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f82d4100e30 0x7f82d419aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:44.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.717+0000 7f82d9600700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82d4107570 0x7f82d419f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:44.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.717+0000 7f82d8c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f82d4100e30 0x7f82d419aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:59224/0 (socket says 172.21.15.117:59224) 2024-06-03T03:23:44.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.717+0000 7f82d8c00700 1 -- 172.21.15.117:0/2338418987 learned_addr learned my addr 172.21.15.117:0/2338418987 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:23:44.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.718+0000 7f82d8c00700 1 -- 172.21.15.117:0/2338418987 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82d4107570 msgr2=0x7f82d419f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:44.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.718+0000 7f82d8c00700 1 --2- 172.21.15.117:0/2338418987 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82d4107570 0x7f82d419f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:44.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.718+0000 7f82d8c00700 1 -- 172.21.15.117:0/2338418987 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82d41017d0 msgr2=0x7f82d419b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:44.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.718+0000 7f82d8c00700 1 --2- 172.21.15.117:0/2338418987 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82d41017d0 0x7f82d419b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:44.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.718+0000 7f82d8c00700 1 -- 172.21.15.117:0/2338418987 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82d419feb0 con 0x7f82d4100e30 2024-06-03T03:23:44.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.718+0000 7f82d8c00700 1 --2- 172.21.15.117:0/2338418987 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f82d4100e30 0x7f82d419aea0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f82c800e950 tx=0x7f82c800ec60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:23:44.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.718+0000 7f82d1600700 1 -- 172.21.15.117:0/2338418987 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c8004230 con 0x7f82d4100e30 2024-06-03T03:23:44.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.718+0000 7f82db600700 1 -- 172.21.15.117:0/2338418987 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82d41a01a0 con 0x7f82d4100e30 2024-06-03T03:23:44.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.719+0000 7f82db600700 1 -- 172.21.15.117:0/2338418987 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82d41a7a20 con 0x7f82d4100e30 2024-06-03T03:23:44.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.719+0000 7f82d1600700 1 -- 172.21.15.117:0/2338418987 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82c80043d0 con 0x7f82d4100e30 2024-06-03T03:23:44.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.719+0000 7f82d1600700 1 -- 172.21.15.117:0/2338418987 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c8003d60 con 0x7f82d4100e30 2024-06-03T03:23:44.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.722+0000 7f82d1600700 1 -- 172.21.15.117:0/2338418987 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f82c800c7e0 con 0x7f82d4100e30 2024-06-03T03:23:44.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.723+0000 7f82e5e9fb80 1 Processor -- start 2024-06-03T03:23:44.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.723+0000 7f82d1600700 1 --2- 172.21.15.117:0/2338418987 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f82bc0a2c20 0x7f82bc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:44.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.723+0000 7f82e5e9fb80 1 -- start start 2024-06-03T03:23:44.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.723+0000 7f82d1600700 1 -- 172.21.15.117:0/2338418987 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f82c80cf670 con 0x7f82d4100e30 2024-06-03T03:23:44.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.724+0000 7f82d9600700 1 --2- 172.21.15.117:0/2338418987 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f82bc0a2c20 0x7f82bc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:44.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.724+0000 7f82e5e9fb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b9ad63df0 0x559b9ade7020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:44.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.724+0000 7f82e5e9fb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559b9ade7560 0x559b9ade79e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:44.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.724+0000 7f82e5e9fb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b9adf22b0 0x559b9adf46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:44.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.724+0000 7f82e5e9fb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559b9acf69c0 con 0x559b9adf22b0 2024-06-03T03:23:44.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.724+0000 7f82e5e9fb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559b9ad39ab0 con 0x559b9ade7560 2024-06-03T03:23:44.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.724+0000 7f82e5e9fb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559b9acf48a0 con 0x559b9ad63df0 2024-06-03T03:23:44.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.724+0000 7f82d8c00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b9ad63df0 0x559b9ade7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:44.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.724+0000 7f82d8c00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b9ad63df0 0x559b9ade7020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:45918/0 (socket says 172.21.15.117:45918) 2024-06-03T03:23:44.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.724+0000 7f82d8c00700 1 -- 172.21.15.117:0/2969380793 learned_addr learned my addr 172.21.15.117:0/2969380793 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:23:44.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.724+0000 7f82d3e00700 1 --2- 172.21.15.117:0/2969380793 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559b9ade7560 0x559b9ade79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:44.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.724+0000 7f82d9600700 1 --2- 172.21.15.117:0/2969380793 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b9adf22b0 0x559b9adf46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:44.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.725+0000 7f82d8c00700 1 -- 172.21.15.117:0/2969380793 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559b9ade7560 msgr2=0x559b9ade79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:23:44.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.725+0000 7f82d8c00700 1 --2- 172.21.15.117:0/2969380793 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559b9ade7560 0x559b9ade79e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:44.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.725+0000 7f82d8c00700 1 -- 172.21.15.117:0/2969380793 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b9adf22b0 msgr2=0x559b9adf46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:23:44.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.725+0000 7f82d8c00700 1 --2- 172.21.15.117:0/2969380793 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b9adf22b0 0x559b9adf46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:44.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.725+0000 7f82d9600700 1 --2- 172.21.15.117:0/2338418987 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f82bc0a2c20 0x7f82bc0a50e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f82d4108160 tx=0x7f82cc009920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:23:44.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.725+0000 7f82d8c00700 1 -- 172.21.15.117:0/2969380793 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b9ace1f40 con 0x559b9ad63df0 2024-06-03T03:23:44.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.725+0000 7f82d8c00700 1 --2- 172.21.15.117:0/2969380793 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b9ad63df0 0x559b9ade7020 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f82c80a4220 tx=0x7f82c809b6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b73c51ad553e2dcc server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:23:44.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.725+0000 7f82c6000700 1 -- 172.21.15.117:0/2969380793 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c8003680 con 0x559b9ad63df0 2024-06-03T03:23:44.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.725+0000 7f82c6000700 1 -- 172.21.15.117:0/2969380793 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82c80b4070 con 0x559b9ad63df0 2024-06-03T03:23:44.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.725+0000 7f82c6000700 1 -- 172.21.15.117:0/2969380793 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c809bd40 con 0x559b9ad63df0 2024-06-03T03:23:44.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.726+0000 7f82e5e9fb80 1 -- 172.21.15.117:0/2969380793 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b9ad63df0 msgr2=0x559b9ade7020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:44.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.726+0000 7f82e5e9fb80 1 --2- 172.21.15.117:0/2969380793 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b9ad63df0 0x559b9ade7020 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f82c80a4220 tx=0x7f82c809b6a0 comp rx=0 tx=0).stop 2024-06-03T03:23:44.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.726+0000 7f82e5e9fb80 1 -- 172.21.15.117:0/2969380793 shutdown_connections 2024-06-03T03:23:44.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.726+0000 7f82e5e9fb80 1 --2- 172.21.15.117:0/2969380793 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b9adf22b0 0x559b9adf46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:44.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.726+0000 7f82e5e9fb80 1 --2- 172.21.15.117:0/2969380793 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b9ad63df0 0x559b9ade7020 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:44.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.726+0000 7f82e5e9fb80 1 --2- 172.21.15.117:0/2969380793 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559b9ade7560 0x559b9ade79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:44.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.726+0000 7f82e5e9fb80 1 -- 172.21.15.117:0/2969380793 >> 172.21.15.117:0/2969380793 conn(0x559b9ad14120 msgr2=0x559b9ade3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:23:44.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.726+0000 7f82e5e9fb80 1 -- 172.21.15.117:0/2969380793 shutdown_connections 2024-06-03T03:23:44.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.726+0000 7f82e5e9fb80 1 -- 172.21.15.117:0/2969380793 wait complete. 2024-06-03T03:23:44.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.727+0000 7f82e5e9fb80 1 -- start start 2024-06-03T03:23:44.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.727+0000 7f82e5e9fb80 1 Processor -- start 2024-06-03T03:23:44.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.728+0000 7f82e5e9fb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b9ad63df0 0x559b9adf1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:44.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.728+0000 7f82e5e9fb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559b9ade7560 0x559b9adeed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:44.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.728+0000 7f82e5e9fb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b9adf22b0 0x559b9adef250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:44.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.728+0000 7f82e5e9fb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559b9acd5f10 con 0x559b9ad63df0 2024-06-03T03:23:44.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.728+0000 7f82e5e9fb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559b9ad35a70 con 0x559b9ade7560 2024-06-03T03:23:44.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.728+0000 7f82e5e9fb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559b9adf4f10 con 0x559b9adf22b0 2024-06-03T03:23:44.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.728+0000 7f82e5e9fb80 10 client.0 _ll_register_callbacks cb 0x559b9acd36d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:23:44.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.728+0000 7f82d8c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b9ad63df0 0x559b9adf1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:44.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.728+0000 7f82d9600700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b9adf22b0 0x559b9adef250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:44.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.728+0000 7f82d8c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b9ad63df0 0x559b9adf1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:45424/0 (socket says 172.21.15.117:45424) 2024-06-03T03:23:44.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.728+0000 7f82d8c00700 1 -- 172.21.15.117:0/555027985 learned_addr learned my addr 172.21.15.117:0/555027985 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:23:44.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.728+0000 7f82d3e00700 1 --2- 172.21.15.117:0/555027985 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559b9ade7560 0x559b9adeed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:44.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.729+0000 7f82d8c00700 1 -- 172.21.15.117:0/555027985 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b9adf22b0 msgr2=0x559b9adef250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:23:44.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.729+0000 7f82d8c00700 1 --2- 172.21.15.117:0/555027985 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b9adf22b0 0x559b9adef250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:44.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.729+0000 7f82d8c00700 1 -- 172.21.15.117:0/555027985 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559b9ade7560 msgr2=0x559b9adeed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:23:44.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.729+0000 7f82d8c00700 1 --2- 172.21.15.117:0/555027985 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559b9ade7560 0x559b9adeed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:44.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.729+0000 7f82d8c00700 1 -- 172.21.15.117:0/555027985 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b9ac88880 con 0x559b9ad63df0 2024-06-03T03:23:44.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.729+0000 7f82d8c00700 1 --2- 172.21.15.117:0/555027985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b9ad63df0 0x559b9adf1a10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f82c809b7e0 tx=0x7f82c80097e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40f8fb8db0b368a5 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:23:44.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.729+0000 7f82baa00700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-06-03T03:23:44.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.729+0000 7f82baa00700 1 -- 172.21.15.117:0/555027985 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c809fe90 con 0x559b9ad63df0 2024-06-03T03:23:44.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.729+0000 7f82baa00700 1 -- 172.21.15.117:0/555027985 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f82c80ba2f0 con 0x559b9ad63df0 2024-06-03T03:23:44.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.729+0000 7f82e5e9fb80 1 -- 172.21.15.117:0/555027985 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82c80b4070 con 0x559b9ad63df0 2024-06-03T03:23:44.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.729+0000 7f82baa00700 1 -- 172.21.15.117:0/555027985 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f82c80ba4f0 con 0x559b9ad63df0 2024-06-03T03:23:44.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.729+0000 7f82e5e9fb80 10 client.14838 fetch_fsmap learned FSMap version 30 2024-06-03T03:23:44.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.729+0000 7f82e5e9fb80 1 -- 172.21.15.117:0/555027985 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559b9adff6b0 con 0x559b9ad63df0 2024-06-03T03:23:44.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.730+0000 7f82baa00700 1 -- 172.21.15.117:0/555027985 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82c80b4070 con 0x559b9ad63df0 2024-06-03T03:23:44.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.730+0000 7f82baa00700 1 -- 172.21.15.117:0/555027985 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f82c80b4ce0 con 0x559b9ad63df0 2024-06-03T03:23:44.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.730+0000 7f82e5e9fb80 10 client.14838 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T03:23:44.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.730+0000 7f82e5e9fb80 10 client.14838 resolve_mds: resolved name 'f' to daemon mds.f 2024-06-03T03:23:44.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.730+0000 7f82e5e9fb80 20 client.14838 populate_metadata read hostname 'smithi117' 2024-06-03T03:23:44.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.730+0000 7f82e5e9fb80 1 --2- 172.21.15.117:0/555027985 >> [v2:172.21.15.183:6834/1009004320,v1:172.21.15.183:6835/1009004320] conn(0x559b9ae017c0 0x559b9ae02c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:44.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.730+0000 7f82e5e9fb80 4 client.14838 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T03:23:44.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.730+0000 7f82e5e9fb80 1 -- 172.21.15.117:0/555027985 --> [v2:172.21.15.183:6834/1009004320,v1:172.21.15.183:6835/1009004320] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559b9ad46fb0 con 0x559b9ae017c0 2024-06-03T03:23:44.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.730+0000 7f82d3e00700 1 --2- 172.21.15.117:0/555027985 >> [v2:172.21.15.183:6834/1009004320,v1:172.21.15.183:6835/1009004320] conn(0x559b9ae017c0 0x559b9ae02c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:44.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.731+0000 7f82d3e00700 1 --2- 172.21.15.117:0/555027985 >> [v2:172.21.15.183:6834/1009004320,v1:172.21.15.183:6835/1009004320] conn(0x559b9ae017c0 0x559b9ae02c30 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bdc40fda2ecc3481 server_cookie=541b0b1ca682289b in_seq=0 out_seq=0 2024-06-03T03:23:44.735 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.731+0000 7f82baa00700 10 client.14838 ms_handle_connect on v2:172.21.15.183:6834/1009004320 2024-06-03T03:23:44.735 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.732+0000 7f82baa00700 1 -- 172.21.15.117:0/555027985 <== mds.4 v2:172.21.15.183:6834/1009004320 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559b9ad46fb0 con 0x559b9ae017c0 2024-06-03T03:23:44.735 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.732+0000 7f82baa00700 10 client.14838 handle_command_reply: tid=0 2024-06-03T03:23:44.735 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.732+0000 7f82e5e9fb80 1 client.14838 shutdown 2024-06-03T03:23:44.735 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.732+0000 7f82e5e9fb80 1 -- 172.21.15.117:0/555027985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b9ad63df0 msgr2=0x559b9adf1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:44.735 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.732+0000 7f82e5e9fb80 1 --2- 172.21.15.117:0/555027985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b9ad63df0 0x559b9adf1a10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f82c809b7e0 tx=0x7f82c80097e0 comp rx=0 tx=0).stop 2024-06-03T03:23:44.735 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.732+0000 7f82e5e9fb80 1 -- 172.21.15.117:0/555027985 shutdown_connections 2024-06-03T03:23:44.735 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.732+0000 7f82e5e9fb80 1 --2- 172.21.15.117:0/555027985 >> [v2:172.21.15.183:6834/1009004320,v1:172.21.15.183:6835/1009004320] conn(0x559b9ae017c0 0x559b9ae02c30 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:44.735 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.732+0000 7f82e5e9fb80 1 --2- 172.21.15.117:0/555027985 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b9adf22b0 0x559b9adef250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:44.735 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.732+0000 7f82e5e9fb80 1 --2- 172.21.15.117:0/555027985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b9ad63df0 0x559b9adf1a10 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:44.735 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.732+0000 7f82e5e9fb80 1 --2- 172.21.15.117:0/555027985 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559b9ade7560 0x559b9adeed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:44.735 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.732+0000 7f82e5e9fb80 1 -- 172.21.15.117:0/555027985 >> 172.21.15.117:0/555027985 conn(0x559b9ad14120 msgr2=0x559b9ade26f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:23:44.735 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.732+0000 7f82baa00700 0 client.14838 ms_handle_reset on v2:172.21.15.183:6834/1009004320 2024-06-03T03:23:44.736 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.733+0000 7f82e5e9fb80 1 -- 172.21.15.117:0/555027985 shutdown_connections 2024-06-03T03:23:44.736 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.733+0000 7f82e5e9fb80 1 -- 172.21.15.117:0/555027985 wait complete. 2024-06-03T03:23:44.736 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.733+0000 7f82e5e9fb80 20 client.14838 trim_cache size 0 max 16384 2024-06-03T03:23:44.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.757+0000 7f82e5e9fb80 1 Processor -- start 2024-06-03T03:23:44.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.757+0000 7f82e5e9fb80 1 -- start start 2024-06-03T03:23:44.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.757+0000 7f82e5e9fb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b9ad63df0 0x559b9ae00a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:44.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.757+0000 7f82e5e9fb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b9ade7560 0x559b9adef280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:44.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.757+0000 7f82e5e9fb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559b9adf22b0 0x559b9adf36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:44.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.757+0000 7f82e5e9fb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559b9ae00f70 con 0x559b9ad63df0 2024-06-03T03:23:44.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.757+0000 7f82e5e9fb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559b9ae01640 con 0x559b9adf22b0 2024-06-03T03:23:44.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.757+0000 7f82e5e9fb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559b9ae01d10 con 0x559b9ade7560 2024-06-03T03:23:44.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.758+0000 7f82d3e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b9ade7560 0x559b9adef280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:44.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.758+0000 7f82d8c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b9ad63df0 0x559b9ae00a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:44.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.758+0000 7f82d3e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b9ade7560 0x559b9adef280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:45936/0 (socket says 172.21.15.117:45936) 2024-06-03T03:23:44.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.758+0000 7f82d3e00700 1 -- 172.21.15.117:0/309125581 learned_addr learned my addr 172.21.15.117:0/309125581 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:23:44.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.758+0000 7f82d9600700 1 --2- 172.21.15.117:0/309125581 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559b9adf22b0 0x559b9adf36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:44.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.758+0000 7f82d3e00700 1 -- 172.21.15.117:0/309125581 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559b9adf22b0 msgr2=0x559b9adf36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:23:44.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.758+0000 7f82d3e00700 1 --2- 172.21.15.117:0/309125581 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559b9adf22b0 0x559b9adf36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:44.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.758+0000 7f82d3e00700 1 -- 172.21.15.117:0/309125581 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b9ad63df0 msgr2=0x559b9ae00a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:23:44.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.758+0000 7f82d3e00700 1 --2- 172.21.15.117:0/309125581 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b9ad63df0 0x559b9ae00a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:44.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.758+0000 7f82d3e00700 1 -- 172.21.15.117:0/309125581 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b9ae010f0 con 0x559b9ade7560 2024-06-03T03:23:44.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.758+0000 7f82d9600700 1 --2- 172.21.15.117:0/309125581 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559b9adf22b0 0x559b9adf36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T03:23:44.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.758+0000 7f82d3e00700 1 --2- 172.21.15.117:0/309125581 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b9ade7560 0x559b9adef280 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f82c00098e0 tx=0x7f82c0005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5573ebd58531ca77 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:23:44.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.759+0000 7f82c5600700 1 -- 172.21.15.117:0/309125581 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c000bea0 con 0x559b9ade7560 2024-06-03T03:23:44.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.759+0000 7f82c5600700 1 -- 172.21.15.117:0/309125581 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82c0003680 con 0x559b9ade7560 2024-06-03T03:23:44.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.759+0000 7f82c5600700 1 -- 172.21.15.117:0/309125581 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c000f6c0 con 0x559b9ade7560 2024-06-03T03:23:44.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.759+0000 7f82e5e9fb80 1 -- 172.21.15.117:0/309125581 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b9ade7560 msgr2=0x559b9adef280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:44.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.759+0000 7f82e5e9fb80 1 --2- 172.21.15.117:0/309125581 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b9ade7560 0x559b9adef280 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f82c00098e0 tx=0x7f82c0005fd0 comp rx=0 tx=0).stop 2024-06-03T03:23:44.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.759+0000 7f82e5e9fb80 1 -- 172.21.15.117:0/309125581 shutdown_connections 2024-06-03T03:23:44.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.759+0000 7f82e5e9fb80 1 --2- 172.21.15.117:0/309125581 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559b9adf22b0 0x559b9adf36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:44.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.759+0000 7f82e5e9fb80 1 --2- 172.21.15.117:0/309125581 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b9ad63df0 0x559b9ae00a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:44.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.759+0000 7f82e5e9fb80 1 --2- 172.21.15.117:0/309125581 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b9ade7560 0x559b9adef280 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:44.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.759+0000 7f82e5e9fb80 1 -- 172.21.15.117:0/309125581 >> 172.21.15.117:0/309125581 conn(0x559b9ad14120 msgr2=0x559b9ade2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:23:44.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.760+0000 7f82e5e9fb80 1 -- 172.21.15.117:0/309125581 shutdown_connections 2024-06-03T03:23:44.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.760+0000 7f82e5e9fb80 1 -- 172.21.15.117:0/309125581 wait complete. 2024-06-03T03:23:44.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.760+0000 7f82e5e9fb80 1 -- start start 2024-06-03T03:23:44.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.760+0000 7f82e5e9fb80 1 Processor -- start 2024-06-03T03:23:44.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.761+0000 7f82e5e9fb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b9ad63df0 0x559b9ae0c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:44.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.761+0000 7f82e5e9fb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559b9ade7560 0x559b9adf31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:44.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.761+0000 7f82e5e9fb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b9adf22b0 0x559b9adf3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:44.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.761+0000 7f82e5e9fb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559b9ae02ab0 con 0x559b9ad63df0 2024-06-03T03:23:44.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.761+0000 7f82e5e9fb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559b9ae023e0 con 0x559b9ade7560 2024-06-03T03:23:44.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.761+0000 7f82e5e9fb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559b9ae03180 con 0x559b9adf22b0 2024-06-03T03:23:44.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.761+0000 7f82e5e9fb80 10 client.0 _ll_register_callbacks cb 0x559b9acd36d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:23:44.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.761+0000 7f82d8c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b9ad63df0 0x559b9ae0c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:44.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.761+0000 7f82d9600700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b9adf22b0 0x559b9adf3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:44.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.761+0000 7f82d9600700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b9adf22b0 0x559b9adf3700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:45948/0 (socket says 172.21.15.117:45948) 2024-06-03T03:23:44.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.761+0000 7f82d9600700 1 -- 172.21.15.117:0/3118587000 learned_addr learned my addr 172.21.15.117:0/3118587000 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:23:44.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.761+0000 7f82d3e00700 1 --2- 172.21.15.117:0/3118587000 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559b9ade7560 0x559b9adf31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:44.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.762+0000 7f82d9600700 1 -- 172.21.15.117:0/3118587000 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559b9ade7560 msgr2=0x559b9adf31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:23:44.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.762+0000 7f82d9600700 1 --2- 172.21.15.117:0/3118587000 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559b9ade7560 0x559b9adf31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:44.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.762+0000 7f82d9600700 1 -- 172.21.15.117:0/3118587000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b9ad63df0 msgr2=0x559b9ae0c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:23:44.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.762+0000 7f82d9600700 1 --2- 172.21.15.117:0/3118587000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b9ad63df0 0x559b9ae0c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:44.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.762+0000 7f82d9600700 1 -- 172.21.15.117:0/3118587000 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b9adf3c40 con 0x559b9adf22b0 2024-06-03T03:23:44.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.762+0000 7f82d8c00700 1 --2- 172.21.15.117:0/3118587000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b9ad63df0 0x559b9ae0c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:23:44.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.762+0000 7f82d9600700 1 --2- 172.21.15.117:0/3118587000 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b9adf22b0 0x559b9adf3700 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f82cc019f40 tx=0x7f82cc004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30053fd4912ee57b server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:23:44.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.762+0000 7f82baa00700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-06-03T03:23:44.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.762+0000 7f82baa00700 1 -- 172.21.15.117:0/3118587000 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82cc01d070 con 0x559b9adf22b0 2024-06-03T03:23:44.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.762+0000 7f82e5e9fb80 1 -- 172.21.15.117:0/3118587000 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82c0003680 con 0x559b9adf22b0 2024-06-03T03:23:44.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.763+0000 7f82baa00700 1 -- 172.21.15.117:0/3118587000 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f82cc003c00 con 0x559b9adf22b0 2024-06-03T03:23:44.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.763+0000 7f82baa00700 1 -- 172.21.15.117:0/3118587000 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f82cc035050 con 0x559b9adf22b0 2024-06-03T03:23:44.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.763+0000 7f82e5e9fb80 10 client.24985 fetch_fsmap learned FSMap version 30 2024-06-03T03:23:44.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.763+0000 7f82e5e9fb80 1 -- 172.21.15.117:0/3118587000 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559b9ae08e40 con 0x559b9adf22b0 2024-06-03T03:23:44.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.763+0000 7f82baa00700 1 -- 172.21.15.117:0/3118587000 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82c0003680 con 0x559b9adf22b0 2024-06-03T03:23:44.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.763+0000 7f82baa00700 1 -- 172.21.15.117:0/3118587000 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f82cc020040 con 0x559b9adf22b0 2024-06-03T03:23:44.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.763+0000 7f82e5e9fb80 10 client.24985 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T03:23:44.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.764+0000 7f82e5e9fb80 10 client.24985 resolve_mds: resolved name 'f' to daemon mds.f 2024-06-03T03:23:44.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.764+0000 7f82e5e9fb80 20 client.24985 populate_metadata read hostname 'smithi117' 2024-06-03T03:23:44.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.764+0000 7f82e5e9fb80 1 --2- 172.21.15.117:0/3118587000 >> [v2:172.21.15.183:6834/1009004320,v1:172.21.15.183:6835/1009004320] conn(0x559b9ae71a10 0x559b9ae70720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:44.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.764+0000 7f82e5e9fb80 4 client.24985 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-03T03:23:44.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.764+0000 7f82e5e9fb80 1 -- 172.21.15.117:0/3118587000 --> [v2:172.21.15.183:6834/1009004320,v1:172.21.15.183:6835/1009004320] -- command(tid 0: {"prefix": "status"}) -- 0x559b9ad41530 con 0x559b9ae71a10 2024-06-03T03:23:44.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.764+0000 7f82d8c00700 1 --2- 172.21.15.117:0/3118587000 >> [v2:172.21.15.183:6834/1009004320,v1:172.21.15.183:6835/1009004320] conn(0x559b9ae71a10 0x559b9ae70720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:44.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.765+0000 7f82d8c00700 1 --2- 172.21.15.117:0/3118587000 >> [v2:172.21.15.183:6834/1009004320,v1:172.21.15.183:6835/1009004320] conn(0x559b9ae71a10 0x559b9ae70720 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a1b268b9092ba387 server_cookie=18bf636e173462d5 in_seq=0 out_seq=0 2024-06-03T03:23:44.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.765+0000 7f82baa00700 10 client.24985 ms_handle_connect on v2:172.21.15.183:6834/1009004320 2024-06-03T03:23:44.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.765+0000 7f82baa00700 1 -- 172.21.15.117:0/3118587000 <== mds.4 v2:172.21.15.183:6834/1009004320 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x559b9ad41530 con 0x559b9ae71a10 2024-06-03T03:23:44.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.765+0000 7f82baa00700 10 client.24985 handle_command_reply: tid=0 2024-06-03T03:23:44.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.765+0000 7f82e5e9fb80 1 client.24985 shutdown 2024-06-03T03:23:44.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.765+0000 7f82e5e9fb80 1 -- 172.21.15.117:0/3118587000 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b9adf22b0 msgr2=0x559b9adf3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:44.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.765+0000 7f82e5e9fb80 1 --2- 172.21.15.117:0/3118587000 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b9adf22b0 0x559b9adf3700 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f82cc019f40 tx=0x7f82cc004b40 comp rx=0 tx=0).stop 2024-06-03T03:23:44.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.765+0000 7f82e5e9fb80 1 -- 172.21.15.117:0/3118587000 shutdown_connections 2024-06-03T03:23:44.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.765+0000 7f82e5e9fb80 1 --2- 172.21.15.117:0/3118587000 >> [v2:172.21.15.183:6834/1009004320,v1:172.21.15.183:6835/1009004320] conn(0x559b9ae71a10 0x559b9ae70720 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:44.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.765+0000 7f82e5e9fb80 1 --2- 172.21.15.117:0/3118587000 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b9adf22b0 0x559b9adf3700 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:44.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.765+0000 7f82e5e9fb80 1 --2- 172.21.15.117:0/3118587000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b9ad63df0 0x559b9ae0c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:44.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.765+0000 7f82baa00700 0 client.24985 ms_handle_reset on v2:172.21.15.183:6834/1009004320 2024-06-03T03:23:44.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.765+0000 7f82e5e9fb80 1 --2- 172.21.15.117:0/3118587000 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559b9ade7560 0x559b9adf31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:44.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.765+0000 7f82e5e9fb80 1 -- 172.21.15.117:0/3118587000 >> 172.21.15.117:0/3118587000 conn(0x559b9ad14120 msgr2=0x559b9ade1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:23:44.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.765+0000 7f82e5e9fb80 1 -- 172.21.15.117:0/3118587000 shutdown_connections 2024-06-03T03:23:44.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.765+0000 7f82e5e9fb80 1 -- 172.21.15.117:0/3118587000 wait complete. 2024-06-03T03:23:44.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.765+0000 7f82e5e9fb80 20 client.24985 trim_cache size 0 max 16384 2024-06-03T03:23:44.770 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-06-03T03:23:44.770 INFO:teuthology.orchestra.run.smithi117.stdout: "cluster_fsid": "a0e8473a-2156-11ef-bc9b-c7b262605968", 2024-06-03T03:23:44.770 INFO:teuthology.orchestra.run.smithi117.stdout: "whoami": 4, 2024-06-03T03:23:44.770 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 24487, 2024-06-03T03:23:44.770 INFO:teuthology.orchestra.run.smithi117.stdout: "want_state": "up:active", 2024-06-03T03:23:44.770 INFO:teuthology.orchestra.run.smithi117.stdout: "state": "up:active", 2024-06-03T03:23:44.770 INFO:teuthology.orchestra.run.smithi117.stdout: "fs_name": "cephfs", 2024-06-03T03:23:44.770 INFO:teuthology.orchestra.run.smithi117.stdout: "rank_uptime": 158.74969320599999, 2024-06-03T03:23:44.770 INFO:teuthology.orchestra.run.smithi117.stdout: "mdsmap_epoch": 30, 2024-06-03T03:23:44.770 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_epoch": 99, 2024-06-03T03:23:44.770 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_epoch_barrier": 97, 2024-06-03T03:23:44.771 INFO:teuthology.orchestra.run.smithi117.stdout: "uptime": 280.95036749299999 2024-06-03T03:23:44.771 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-06-03T03:23:44.771 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.766+0000 7f82db600700 1 -- 172.21.15.117:0/2338418987 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f82bc0a2c20 msgr2=0x7f82bc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:44.771 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.766+0000 7f82db600700 1 --2- 172.21.15.117:0/2338418987 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f82bc0a2c20 0x7f82bc0a50e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f82d4108160 tx=0x7f82cc009920 comp rx=0 tx=0).stop 2024-06-03T03:23:44.772 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.766+0000 7f82db600700 1 -- 172.21.15.117:0/2338418987 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f82d4100e30 msgr2=0x7f82d419aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:44.772 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.766+0000 7f82db600700 1 --2- 172.21.15.117:0/2338418987 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f82d4100e30 0x7f82d419aea0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f82c800e950 tx=0x7f82c800ec60 comp rx=0 tx=0).stop 2024-06-03T03:23:44.772 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.766+0000 7f82db600700 1 -- 172.21.15.117:0/2338418987 shutdown_connections 2024-06-03T03:23:44.772 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.766+0000 7f82db600700 1 --2- 172.21.15.117:0/2338418987 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f82bc0a2c20 0x7f82bc0a50e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:44.772 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.766+0000 7f82db600700 1 --2- 172.21.15.117:0/2338418987 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82d4107570 0x7f82d419f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:44.772 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.766+0000 7f82db600700 1 --2- 172.21.15.117:0/2338418987 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f82d4100e30 0x7f82d419aea0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:44.772 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.766+0000 7f82db600700 1 --2- 172.21.15.117:0/2338418987 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82d41017d0 0x7f82d419b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:44.772 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.766+0000 7f82db600700 1 -- 172.21.15.117:0/2338418987 >> 172.21.15.117:0/2338418987 conn(0x7f82d4076680 msgr2=0x7f82d40ff910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:23:44.772 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.766+0000 7f82db600700 1 -- 172.21.15.117:0/2338418987 shutdown_connections 2024-06-03T03:23:44.772 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:44.766+0000 7f82db600700 1 -- 172.21.15.117:0/2338418987 wait complete. 2024-06-03T03:23:45.919 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph tell mds.c status 2024-06-03T03:23:46.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:23:45 smithi117 ceph-mon[31649]: pgmap v459: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:23:46.362 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:23:46.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:23:45 smithi151 ceph-mon[37043]: pgmap v459: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:23:46.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:23:45 smithi183 ceph-mon[37326]: pgmap v459: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:23:48.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:23:47 smithi117 ceph-mon[31649]: pgmap v460: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-03T03:23:48.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:23:47 smithi151 ceph-mon[37043]: pgmap v460: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-03T03:23:48.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:23:47 smithi183 ceph-mon[37326]: pgmap v460: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-03T03:23:48.797 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.796+0000 7f256ca00700 1 -- 172.21.15.117:0/1438798550 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f256810d8a0 msgr2=0x7f256810dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:48.797 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.796+0000 7f256ca00700 1 --2- 172.21.15.117:0/1438798550 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f256810d8a0 0x7f256810dc80 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2558009990 tx=0x7f2558009ca0 comp rx=0 tx=0).stop 2024-06-03T03:23:48.797 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.796+0000 7f256ca00700 1 -- 172.21.15.117:0/1438798550 shutdown_connections 2024-06-03T03:23:48.797 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.796+0000 7f256ca00700 1 --2- 172.21.15.117:0/1438798550 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2568109390 0x7f2568115830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:48.797 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.796+0000 7f256ca00700 1 --2- 172.21.15.117:0/1438798550 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f256810d8a0 0x7f256810dc80 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:48.797 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.796+0000 7f256ca00700 1 --2- 172.21.15.117:0/1438798550 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f25681089d0 0x7f2568108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:48.797 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.796+0000 7f256ca00700 1 -- 172.21.15.117:0/1438798550 >> 172.21.15.117:0/1438798550 conn(0x7f2568078220 msgr2=0x7f25681072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:23:48.797 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.796+0000 7f256ca00700 1 -- 172.21.15.117:0/1438798550 shutdown_connections 2024-06-03T03:23:48.797 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.796+0000 7f256ca00700 1 -- 172.21.15.117:0/1438798550 wait complete. 2024-06-03T03:23:48.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.797+0000 7f256ca00700 1 Processor -- start 2024-06-03T03:23:48.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.797+0000 7f256ca00700 1 -- start start 2024-06-03T03:23:48.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.797+0000 7f256ca00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25681089d0 0x7f256819f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:48.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.797+0000 7f256ca00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2568109390 0x7f256819f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:48.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.798+0000 7f256ca00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f256810d8a0 0x7f25681a3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:48.799 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.798+0000 7f256ca00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2568118d00 con 0x7f25681089d0 2024-06-03T03:23:48.799 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.798+0000 7f256ca00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2568118b80 con 0x7f256810d8a0 2024-06-03T03:23:48.799 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.798+0000 7f256ca00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2568118e80 con 0x7f2568109390 2024-06-03T03:23:48.799 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.798+0000 7f2566800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25681089d0 0x7f256819f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:48.799 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.798+0000 7f2566800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25681089d0 0x7f256819f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34768/0 (socket says 172.21.15.117:34768) 2024-06-03T03:23:48.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.798+0000 7f2566800700 1 -- 172.21.15.117:0/3510788054 learned_addr learned my addr 172.21.15.117:0/3510788054 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:23:48.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.798+0000 7f255de00700 1 --2- 172.21.15.117:0/3510788054 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2568109390 0x7f256819f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:48.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.798+0000 7f2567200700 1 --2- 172.21.15.117:0/3510788054 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f256810d8a0 0x7f25681a3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:48.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.798+0000 7f2566800700 1 -- 172.21.15.117:0/3510788054 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2568109390 msgr2=0x7f256819f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:48.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.798+0000 7f2566800700 1 --2- 172.21.15.117:0/3510788054 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2568109390 0x7f256819f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:48.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.798+0000 7f2566800700 1 -- 172.21.15.117:0/3510788054 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f256810d8a0 msgr2=0x7f25681a3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:48.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.798+0000 7f2566800700 1 --2- 172.21.15.117:0/3510788054 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f256810d8a0 0x7f25681a3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:48.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.798+0000 7f2566800700 1 -- 172.21.15.117:0/3510788054 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25681a4200 con 0x7f25681089d0 2024-06-03T03:23:48.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.799+0000 7f2566800700 1 --2- 172.21.15.117:0/3510788054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25681089d0 0x7f256819f1f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2558000c00 tx=0x7f255800ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:23:48.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.799+0000 7f255fe00700 1 -- 172.21.15.117:0/3510788054 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2558027070 con 0x7f25681089d0 2024-06-03T03:23:48.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.799+0000 7f255fe00700 1 -- 172.21.15.117:0/3510788054 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2558003d70 con 0x7f25681089d0 2024-06-03T03:23:48.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.799+0000 7f256ca00700 1 -- 172.21.15.117:0/3510788054 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25681a4490 con 0x7f25681089d0 2024-06-03T03:23:48.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.799+0000 7f255fe00700 1 -- 172.21.15.117:0/3510788054 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f255800f460 con 0x7f25681089d0 2024-06-03T03:23:48.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.799+0000 7f256ca00700 1 -- 172.21.15.117:0/3510788054 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25681abd70 con 0x7f25681089d0 2024-06-03T03:23:48.804 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.802+0000 7f255fe00700 1 -- 172.21.15.117:0/3510788054 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f255802a070 con 0x7f25681089d0 2024-06-03T03:23:48.804 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.804+0000 7f255fe00700 1 --2- 172.21.15.117:0/3510788054 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f25480a2c20 0x7f25480a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:48.805 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.804+0000 7f255fe00700 1 -- 172.21.15.117:0/3510788054 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f25580d0360 con 0x7f25681089d0 2024-06-03T03:23:48.805 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.804+0000 7f2577d53b80 1 Processor -- start 2024-06-03T03:23:48.806 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.804+0000 7f255de00700 1 --2- 172.21.15.117:0/3510788054 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f25480a2c20 0x7f25480a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:48.806 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.804+0000 7f2577d53b80 1 -- start start 2024-06-03T03:23:48.807 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.804+0000 7f2577d53b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55976f7ebdf0 0x55976f8713d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:48.807 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.804+0000 7f2577d53b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55976f871910 0x55976f871d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:48.807 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.804+0000 7f2577d53b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55976f87a520 0x55976f87c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:48.807 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.804+0000 7f2577d53b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55976f77e9c0 con 0x55976f87a520 2024-06-03T03:23:48.807 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.805+0000 7f2577d53b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55976f7c1ab0 con 0x55976f871910 2024-06-03T03:23:48.807 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.805+0000 7f2577d53b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55976f77c8a0 con 0x55976f7ebdf0 2024-06-03T03:23:48.807 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.805+0000 7f2567200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55976f87a520 0x55976f87c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:48.807 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.805+0000 7f2567200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55976f87a520 0x55976f87c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34780/0 (socket says 172.21.15.117:34780) 2024-06-03T03:23:48.807 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.805+0000 7f2567200700 1 -- 172.21.15.117:0/1516102384 learned_addr learned my addr 172.21.15.117:0/1516102384 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:23:48.807 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.805+0000 7f255de00700 1 --2- 172.21.15.117:0/1516102384 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55976f871910 0x55976f871d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:48.807 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.805+0000 7f2566800700 1 --2- 172.21.15.117:0/1516102384 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55976f7ebdf0 0x55976f8713d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:48.807 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.805+0000 7f255de00700 1 --2- 172.21.15.117:0/3510788054 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f25480a2c20 0x7f25480a50e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2568108520 tx=0x7f25500073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:23:48.808 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.805+0000 7f255de00700 1 -- 172.21.15.117:0/1516102384 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55976f7ebdf0 msgr2=0x55976f8713d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:23:48.808 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.805+0000 7f255de00700 1 --2- 172.21.15.117:0/1516102384 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55976f7ebdf0 0x55976f8713d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:48.808 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.805+0000 7f255de00700 1 -- 172.21.15.117:0/1516102384 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55976f87a520 msgr2=0x55976f87c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:23:48.808 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.805+0000 7f255de00700 1 --2- 172.21.15.117:0/1516102384 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55976f87a520 0x55976f87c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:48.808 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.805+0000 7f255de00700 1 -- 172.21.15.117:0/1516102384 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55976f769f40 con 0x55976f871910 2024-06-03T03:23:48.808 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.806+0000 7f255de00700 1 --2- 172.21.15.117:0/1516102384 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55976f871910 0x55976f871d90 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2550010d10 tx=0x7f2550013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bbb1fb2254f80d31 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:23:48.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.806+0000 7f2557e00700 1 -- 172.21.15.117:0/1516102384 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2550011ad0 con 0x55976f871910 2024-06-03T03:23:48.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.806+0000 7f2557e00700 1 -- 172.21.15.117:0/1516102384 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2550011c70 con 0x55976f871910 2024-06-03T03:23:48.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.806+0000 7f2557e00700 1 -- 172.21.15.117:0/1516102384 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2550026620 con 0x55976f871910 2024-06-03T03:23:48.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.807+0000 7f2577d53b80 1 -- 172.21.15.117:0/1516102384 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55976f871910 msgr2=0x55976f871d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:48.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.807+0000 7f2577d53b80 1 --2- 172.21.15.117:0/1516102384 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55976f871910 0x55976f871d90 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2550010d10 tx=0x7f2550013000 comp rx=0 tx=0).stop 2024-06-03T03:23:48.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.807+0000 7f2577d53b80 1 -- 172.21.15.117:0/1516102384 shutdown_connections 2024-06-03T03:23:48.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.807+0000 7f2577d53b80 1 --2- 172.21.15.117:0/1516102384 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55976f87a520 0x55976f87c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:48.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.807+0000 7f2577d53b80 1 --2- 172.21.15.117:0/1516102384 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55976f7ebdf0 0x55976f8713d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:48.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.807+0000 7f2577d53b80 1 --2- 172.21.15.117:0/1516102384 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55976f871910 0x55976f871d90 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:48.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.807+0000 7f2577d53b80 1 -- 172.21.15.117:0/1516102384 >> 172.21.15.117:0/1516102384 conn(0x55976f79c120 msgr2=0x55976f86b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:23:48.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.807+0000 7f2577d53b80 1 -- 172.21.15.117:0/1516102384 shutdown_connections 2024-06-03T03:23:48.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.807+0000 7f2577d53b80 1 -- 172.21.15.117:0/1516102384 wait complete. 2024-06-03T03:23:48.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.808+0000 7f2577d53b80 1 -- start start 2024-06-03T03:23:48.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.808+0000 7f2577d53b80 1 Processor -- start 2024-06-03T03:23:48.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.809+0000 7f2577d53b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55976f7ebdf0 0x55976f879cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:48.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.809+0000 7f2577d53b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55976f871910 0x55976f876fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:48.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.809+0000 7f2577d53b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55976f87a520 0x55976f877500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:48.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.809+0000 7f2577d53b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55976f75df10 con 0x55976f871910 2024-06-03T03:23:48.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.809+0000 7f2577d53b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55976f7bda70 con 0x55976f7ebdf0 2024-06-03T03:23:48.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.809+0000 7f2577d53b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55976f87d190 con 0x55976f87a520 2024-06-03T03:23:48.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.809+0000 7f2577d53b80 10 client.0 _ll_register_callbacks cb 0x55976f75b6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:23:48.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.809+0000 7f255de00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55976f871910 0x55976f876fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:48.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.809+0000 7f2566800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55976f7ebdf0 0x55976f879cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:48.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.809+0000 7f255de00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55976f871910 0x55976f876fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34788/0 (socket says 172.21.15.117:34788) 2024-06-03T03:23:48.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.809+0000 7f255de00700 1 -- 172.21.15.117:0/1738057426 learned_addr learned my addr 172.21.15.117:0/1738057426 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:23:48.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.809+0000 7f2567200700 1 --2- 172.21.15.117:0/1738057426 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55976f87a520 0x55976f877500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:48.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.809+0000 7f255de00700 1 -- 172.21.15.117:0/1738057426 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55976f87a520 msgr2=0x55976f877500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:23:48.814 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.809+0000 7f255de00700 1 --2- 172.21.15.117:0/1738057426 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55976f87a520 0x55976f877500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:48.814 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.809+0000 7f255de00700 1 -- 172.21.15.117:0/1738057426 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55976f7ebdf0 msgr2=0x55976f879cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:23:48.814 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.809+0000 7f255de00700 1 --2- 172.21.15.117:0/1738057426 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55976f7ebdf0 0x55976f879cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:48.814 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.809+0000 7f255de00700 1 -- 172.21.15.117:0/1738057426 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55976f710880 con 0x55976f871910 2024-06-03T03:23:48.814 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.809+0000 7f2566800700 1 --2- 172.21.15.117:0/1738057426 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55976f7ebdf0 0x55976f879cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:23:48.814 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.810+0000 7f255de00700 1 --2- 172.21.15.117:0/1738057426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55976f871910 0x55976f876fc0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f25500059d0 tx=0x7f2550011b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4477a7d705b0b03 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:23:48.814 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.810+0000 7f2554c00700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-06-03T03:23:48.814 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.810+0000 7f2554c00700 1 -- 172.21.15.117:0/1738057426 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f255001e830 con 0x55976f871910 2024-06-03T03:23:48.814 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.810+0000 7f2554c00700 1 -- 172.21.15.117:0/1738057426 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f2550005050 con 0x55976f871910 2024-06-03T03:23:48.814 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.810+0000 7f2577d53b80 1 -- 172.21.15.117:0/1738057426 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2550011c70 con 0x55976f871910 2024-06-03T03:23:48.814 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.810+0000 7f2554c00700 1 -- 172.21.15.117:0/1738057426 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f255003c050 con 0x55976f871910 2024-06-03T03:23:48.814 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.810+0000 7f2577d53b80 10 client.14856 fetch_fsmap learned FSMap version 30 2024-06-03T03:23:48.814 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.810+0000 7f2577d53b80 1 -- 172.21.15.117:0/1738057426 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55976f8878e0 con 0x55976f871910 2024-06-03T03:23:48.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.811+0000 7f2554c00700 1 -- 172.21.15.117:0/1738057426 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2550011c70 con 0x55976f871910 2024-06-03T03:23:48.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.811+0000 7f2554c00700 1 -- 172.21.15.117:0/1738057426 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f2550037030 con 0x55976f871910 2024-06-03T03:23:48.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.811+0000 7f2577d53b80 10 client.14856 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T03:23:48.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.811+0000 7f2577d53b80 10 client.14856 resolve_mds: resolved name 'c' to daemon mds.c 2024-06-03T03:23:48.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.811+0000 7f2577d53b80 20 client.14856 populate_metadata read hostname 'smithi117' 2024-06-03T03:23:48.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.811+0000 7f2577d53b80 1 --2- 172.21.15.117:0/1738057426 >> [v2:172.21.15.183:6836/2697518526,v1:172.21.15.183:6837/2697518526] conn(0x55976f889ad0 0x55976f88af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:48.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.811+0000 7f2577d53b80 4 client.14856 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T03:23:48.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.811+0000 7f2577d53b80 1 -- 172.21.15.117:0/1738057426 --> [v2:172.21.15.183:6836/2697518526,v1:172.21.15.183:6837/2697518526] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55976f7cefb0 con 0x55976f889ad0 2024-06-03T03:23:48.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.812+0000 7f2566800700 1 --2- 172.21.15.117:0/1738057426 >> [v2:172.21.15.183:6836/2697518526,v1:172.21.15.183:6837/2697518526] conn(0x55976f889ad0 0x55976f88af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:48.816 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.812+0000 7f2566800700 1 --2- 172.21.15.117:0/1738057426 >> [v2:172.21.15.183:6836/2697518526,v1:172.21.15.183:6837/2697518526] conn(0x55976f889ad0 0x55976f88af40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=418d233f2bd6b70c server_cookie=1ee24f183be4c7ea in_seq=0 out_seq=0 2024-06-03T03:23:48.816 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.812+0000 7f2554c00700 10 client.14856 ms_handle_connect on v2:172.21.15.183:6836/2697518526 2024-06-03T03:23:48.816 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.813+0000 7f2554c00700 1 -- 172.21.15.117:0/1738057426 <== mds.3 v2:172.21.15.183:6836/2697518526 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55976f7cefb0 con 0x55976f889ad0 2024-06-03T03:23:48.817 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.813+0000 7f2554c00700 10 client.14856 handle_command_reply: tid=0 2024-06-03T03:23:48.817 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.813+0000 7f2577d53b80 1 client.14856 shutdown 2024-06-03T03:23:48.817 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.814+0000 7f2577d53b80 1 -- 172.21.15.117:0/1738057426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55976f871910 msgr2=0x55976f876fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:48.817 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.814+0000 7f2577d53b80 1 --2- 172.21.15.117:0/1738057426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55976f871910 0x55976f876fc0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f25500059d0 tx=0x7f2550011b80 comp rx=0 tx=0).stop 2024-06-03T03:23:48.817 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.814+0000 7f2577d53b80 1 -- 172.21.15.117:0/1738057426 shutdown_connections 2024-06-03T03:23:48.817 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.814+0000 7f2577d53b80 1 --2- 172.21.15.117:0/1738057426 >> [v2:172.21.15.183:6836/2697518526,v1:172.21.15.183:6837/2697518526] conn(0x55976f889ad0 0x55976f88af40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:48.817 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.814+0000 7f2577d53b80 1 --2- 172.21.15.117:0/1738057426 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55976f87a520 0x55976f877500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:48.817 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.814+0000 7f2577d53b80 1 --2- 172.21.15.117:0/1738057426 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55976f7ebdf0 0x55976f879cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:48.817 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.814+0000 7f2577d53b80 1 --2- 172.21.15.117:0/1738057426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55976f871910 0x55976f876fc0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:48.817 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.814+0000 7f2577d53b80 1 -- 172.21.15.117:0/1738057426 >> 172.21.15.117:0/1738057426 conn(0x55976f79c120 msgr2=0x55976f86b3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:23:48.817 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.814+0000 7f2554c00700 0 client.14856 ms_handle_reset on v2:172.21.15.183:6836/2697518526 2024-06-03T03:23:48.817 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.814+0000 7f2577d53b80 1 -- 172.21.15.117:0/1738057426 shutdown_connections 2024-06-03T03:23:48.817 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.814+0000 7f2577d53b80 1 -- 172.21.15.117:0/1738057426 wait complete. 2024-06-03T03:23:48.818 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.814+0000 7f2577d53b80 20 client.14856 trim_cache size 0 max 16384 2024-06-03T03:23:48.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.838+0000 7f2577d53b80 1 Processor -- start 2024-06-03T03:23:48.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.838+0000 7f2577d53b80 1 -- start start 2024-06-03T03:23:48.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.838+0000 7f2577d53b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55976f7ebdf0 0x55976f877530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:48.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.838+0000 7f2577d53b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55976f871910 0x55976f8758b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:48.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.838+0000 7f2577d53b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55976f87a520 0x55976f875df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:48.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.838+0000 7f2577d53b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55976f889280 con 0x55976f871910 2024-06-03T03:23:48.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.838+0000 7f2577d53b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55976f889950 con 0x55976f87a520 2024-06-03T03:23:48.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.838+0000 7f2577d53b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55976f88a020 con 0x55976f7ebdf0 2024-06-03T03:23:48.840 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.839+0000 7f255de00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55976f871910 0x55976f8758b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:48.841 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.839+0000 7f255de00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55976f871910 0x55976f8758b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34792/0 (socket says 172.21.15.117:34792) 2024-06-03T03:23:48.841 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.839+0000 7f255de00700 1 -- 172.21.15.117:0/3402376618 learned_addr learned my addr 172.21.15.117:0/3402376618 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:23:48.841 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.839+0000 7f2566800700 1 --2- 172.21.15.117:0/3402376618 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55976f7ebdf0 0x55976f877530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:48.841 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.839+0000 7f2567200700 1 --2- 172.21.15.117:0/3402376618 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55976f87a520 0x55976f875df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:48.841 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.839+0000 7f255de00700 1 -- 172.21.15.117:0/3402376618 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55976f7ebdf0 msgr2=0x55976f877530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:23:48.841 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.839+0000 7f255de00700 1 --2- 172.21.15.117:0/3402376618 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55976f7ebdf0 0x55976f877530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:48.841 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.839+0000 7f255de00700 1 -- 172.21.15.117:0/3402376618 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55976f87a520 msgr2=0x55976f875df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:23:48.841 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.839+0000 7f255de00700 1 --2- 172.21.15.117:0/3402376618 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55976f87a520 0x55976f875df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:48.841 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.839+0000 7f255de00700 1 -- 172.21.15.117:0/3402376618 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55976f870520 con 0x55976f871910 2024-06-03T03:23:48.841 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.839+0000 7f2566800700 1 --2- 172.21.15.117:0/3402376618 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55976f7ebdf0 0x55976f877530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:23:48.841 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.839+0000 7f255de00700 1 --2- 172.21.15.117:0/3402376618 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55976f871910 0x55976f8758b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2550007a30 tx=0x7f2550011d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2ee59d910e2b80b server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:23:48.841 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.840+0000 7f2557400700 1 -- 172.21.15.117:0/3402376618 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2550045070 con 0x55976f871910 2024-06-03T03:23:48.841 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.840+0000 7f2557400700 1 -- 172.21.15.117:0/3402376618 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f255002f710 con 0x55976f871910 2024-06-03T03:23:48.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.840+0000 7f2557400700 1 -- 172.21.15.117:0/3402376618 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2550004910 con 0x55976f871910 2024-06-03T03:23:48.843 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.841+0000 7f2577d53b80 1 -- 172.21.15.117:0/3402376618 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55976f871910 msgr2=0x55976f8758b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:48.843 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.841+0000 7f2577d53b80 1 --2- 172.21.15.117:0/3402376618 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55976f871910 0x55976f8758b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2550007a30 tx=0x7f2550011d80 comp rx=0 tx=0).stop 2024-06-03T03:23:48.843 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.841+0000 7f2577d53b80 1 -- 172.21.15.117:0/3402376618 shutdown_connections 2024-06-03T03:23:48.844 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.841+0000 7f2577d53b80 1 --2- 172.21.15.117:0/3402376618 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55976f87a520 0x55976f875df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:48.844 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.841+0000 7f2577d53b80 1 --2- 172.21.15.117:0/3402376618 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55976f7ebdf0 0x55976f877530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:48.844 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.841+0000 7f2577d53b80 1 --2- 172.21.15.117:0/3402376618 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55976f871910 0x55976f8758b0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:48.844 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.841+0000 7f2577d53b80 1 -- 172.21.15.117:0/3402376618 >> 172.21.15.117:0/3402376618 conn(0x55976f79c120 msgr2=0x55976f86b300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:23:48.844 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.841+0000 7f2577d53b80 1 -- 172.21.15.117:0/3402376618 shutdown_connections 2024-06-03T03:23:48.844 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.841+0000 7f2577d53b80 1 -- 172.21.15.117:0/3402376618 wait complete. 2024-06-03T03:23:48.844 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.841+0000 7f2577d53b80 1 -- start start 2024-06-03T03:23:48.844 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.841+0000 7f2577d53b80 1 Processor -- start 2024-06-03T03:23:48.844 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.842+0000 7f2577d53b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55976f7ebdf0 0x55976f873370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:48.844 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.842+0000 7f2577d53b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55976f871910 0x55976f8738b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:48.844 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.842+0000 7f2577d53b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55976f87a520 0x55976f873df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:48.844 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.842+0000 7f2577d53b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55976f88adc0 con 0x55976f7ebdf0 2024-06-03T03:23:48.844 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.842+0000 7f2577d53b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55976f88a6f0 con 0x55976f87a520 2024-06-03T03:23:48.845 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.842+0000 7f2577d53b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55976f88b490 con 0x55976f871910 2024-06-03T03:23:48.845 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.842+0000 7f2577d53b80 10 client.0 _ll_register_callbacks cb 0x55976f75b6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:23:48.845 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.842+0000 7f2566800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55976f7ebdf0 0x55976f873370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:48.845 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.842+0000 7f255de00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55976f871910 0x55976f8738b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:48.845 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.842+0000 7f255de00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55976f871910 0x55976f8738b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:48022/0 (socket says 172.21.15.117:48022) 2024-06-03T03:23:48.845 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.842+0000 7f255de00700 1 -- 172.21.15.117:0/2741371191 learned_addr learned my addr 172.21.15.117:0/2741371191 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:23:48.845 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.842+0000 7f2566800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55976f7ebdf0 0x55976f873370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34804/0 (socket says 172.21.15.117:34804) 2024-06-03T03:23:48.845 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.842+0000 7f2567200700 1 --2- 172.21.15.117:0/2741371191 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55976f87a520 0x55976f873df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:48.845 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.843+0000 7f2566800700 1 -- 172.21.15.117:0/2741371191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55976f871910 msgr2=0x55976f8738b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:23:48.845 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.843+0000 7f2566800700 1 --2- 172.21.15.117:0/2741371191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55976f871910 0x55976f8738b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:48.845 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.843+0000 7f2566800700 1 -- 172.21.15.117:0/2741371191 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55976f87a520 msgr2=0x55976f873df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:23:48.845 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.843+0000 7f2566800700 1 --2- 172.21.15.117:0/2741371191 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55976f87a520 0x55976f873df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:48.845 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.843+0000 7f2566800700 1 -- 172.21.15.117:0/2741371191 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55976f893660 con 0x55976f7ebdf0 2024-06-03T03:23:48.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.843+0000 7f2566800700 1 --2- 172.21.15.117:0/2741371191 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55976f7ebdf0 0x55976f873370 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2558006010 tx=0x7f2558009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa386d7c1458d5b server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:23:48.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.843+0000 7f2554c00700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-06-03T03:23:48.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.843+0000 7f2554c00700 1 -- 172.21.15.117:0/2741371191 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f255809cf60 con 0x55976f7ebdf0 2024-06-03T03:23:48.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.843+0000 7f2577d53b80 1 -- 172.21.15.117:0/2741371191 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f255002f710 con 0x55976f7ebdf0 2024-06-03T03:23:48.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.843+0000 7f2554c00700 1 -- 172.21.15.117:0/2741371191 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f25580d00b0 con 0x55976f7ebdf0 2024-06-03T03:23:48.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.844+0000 7f2554c00700 1 -- 172.21.15.117:0/2741371191 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f255809c860 con 0x55976f7ebdf0 2024-06-03T03:23:48.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.844+0000 7f2577d53b80 10 client.14862 fetch_fsmap learned FSMap version 30 2024-06-03T03:23:48.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.844+0000 7f2577d53b80 1 -- 172.21.15.117:0/2741371191 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55976f8948d0 con 0x55976f7ebdf0 2024-06-03T03:23:48.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.844+0000 7f2554c00700 1 -- 172.21.15.117:0/2741371191 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f255002f710 con 0x55976f7ebdf0 2024-06-03T03:23:48.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.844+0000 7f2554c00700 1 -- 172.21.15.117:0/2741371191 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f25580a4fb0 con 0x55976f7ebdf0 2024-06-03T03:23:48.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.844+0000 7f2577d53b80 10 client.14862 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T03:23:48.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.844+0000 7f2577d53b80 10 client.14862 resolve_mds: resolved name 'c' to daemon mds.c 2024-06-03T03:23:48.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.844+0000 7f2577d53b80 20 client.14862 populate_metadata read hostname 'smithi117' 2024-06-03T03:23:48.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.844+0000 7f2577d53b80 1 --2- 172.21.15.117:0/2741371191 >> [v2:172.21.15.183:6836/2697518526,v1:172.21.15.183:6837/2697518526] conn(0x55976f8f94c0 0x55976f8f81d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:48.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.844+0000 7f2577d53b80 4 client.14862 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-03T03:23:48.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.844+0000 7f2577d53b80 1 -- 172.21.15.117:0/2741371191 --> [v2:172.21.15.183:6836/2697518526,v1:172.21.15.183:6837/2697518526] -- command(tid 0: {"prefix": "status"}) -- 0x55976f7c9530 con 0x55976f8f94c0 2024-06-03T03:23:48.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.845+0000 7f255de00700 1 --2- 172.21.15.117:0/2741371191 >> [v2:172.21.15.183:6836/2697518526,v1:172.21.15.183:6837/2697518526] conn(0x55976f8f94c0 0x55976f8f81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:48.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.845+0000 7f255de00700 1 --2- 172.21.15.117:0/2741371191 >> [v2:172.21.15.183:6836/2697518526,v1:172.21.15.183:6837/2697518526] conn(0x55976f8f94c0 0x55976f8f81d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ad9743f415823fc8 server_cookie=3d2ee4c3ae9f2eb7 in_seq=0 out_seq=0 2024-06-03T03:23:48.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.845+0000 7f2554c00700 10 client.14862 ms_handle_connect on v2:172.21.15.183:6836/2697518526 2024-06-03T03:23:48.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.845+0000 7f2554c00700 1 -- 172.21.15.117:0/2741371191 <== mds.3 v2:172.21.15.183:6836/2697518526 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55976f7c9530 con 0x55976f8f94c0 2024-06-03T03:23:48.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.845+0000 7f2554c00700 10 client.14862 handle_command_reply: tid=0 2024-06-03T03:23:48.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.845+0000 7f2577d53b80 1 client.14862 shutdown 2024-06-03T03:23:48.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.845+0000 7f2577d53b80 1 -- 172.21.15.117:0/2741371191 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55976f7ebdf0 msgr2=0x55976f873370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:48.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.845+0000 7f2577d53b80 1 --2- 172.21.15.117:0/2741371191 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55976f7ebdf0 0x55976f873370 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2558006010 tx=0x7f2558009ca0 comp rx=0 tx=0).stop 2024-06-03T03:23:48.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.846+0000 7f2577d53b80 1 -- 172.21.15.117:0/2741371191 shutdown_connections 2024-06-03T03:23:48.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.846+0000 7f2577d53b80 1 --2- 172.21.15.117:0/2741371191 >> [v2:172.21.15.183:6836/2697518526,v1:172.21.15.183:6837/2697518526] conn(0x55976f8f94c0 0x55976f8f81d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:48.850 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.846+0000 7f2577d53b80 1 --2- 172.21.15.117:0/2741371191 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55976f87a520 0x55976f873df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:48.850 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.846+0000 7f2577d53b80 1 --2- 172.21.15.117:0/2741371191 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55976f7ebdf0 0x55976f873370 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:48.850 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.846+0000 7f2577d53b80 1 --2- 172.21.15.117:0/2741371191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55976f871910 0x55976f8738b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:48.850 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.846+0000 7f2577d53b80 1 -- 172.21.15.117:0/2741371191 >> 172.21.15.117:0/2741371191 conn(0x55976f79c120 msgr2=0x55976f875830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:23:48.850 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.846+0000 7f2554c00700 0 client.14862 ms_handle_reset on v2:172.21.15.183:6836/2697518526 2024-06-03T03:23:48.850 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.846+0000 7f2577d53b80 1 -- 172.21.15.117:0/2741371191 shutdown_connections 2024-06-03T03:23:48.850 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.846+0000 7f2577d53b80 1 -- 172.21.15.117:0/2741371191 wait complete. 2024-06-03T03:23:48.850 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.846+0000 7f2577d53b80 20 client.14862 trim_cache size 0 max 16384 2024-06-03T03:23:48.850 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-06-03T03:23:48.850 INFO:teuthology.orchestra.run.smithi117.stdout: "cluster_fsid": "a0e8473a-2156-11ef-bc9b-c7b262605968", 2024-06-03T03:23:48.851 INFO:teuthology.orchestra.run.smithi117.stdout: "whoami": 3, 2024-06-03T03:23:48.851 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 24499, 2024-06-03T03:23:48.851 INFO:teuthology.orchestra.run.smithi117.stdout: "want_state": "up:active", 2024-06-03T03:23:48.851 INFO:teuthology.orchestra.run.smithi117.stdout: "state": "up:active", 2024-06-03T03:23:48.851 INFO:teuthology.orchestra.run.smithi117.stdout: "fs_name": "cephfs", 2024-06-03T03:23:48.851 INFO:teuthology.orchestra.run.smithi117.stdout: "rank_uptime": 163.84241958499999, 2024-06-03T03:23:48.851 INFO:teuthology.orchestra.run.smithi117.stdout: "mdsmap_epoch": 30, 2024-06-03T03:23:48.851 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_epoch": 99, 2024-06-03T03:23:48.851 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_epoch_barrier": 97, 2024-06-03T03:23:48.851 INFO:teuthology.orchestra.run.smithi117.stdout: "uptime": 282.124211618 2024-06-03T03:23:48.851 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-06-03T03:23:48.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.847+0000 7f256ca00700 1 -- 172.21.15.117:0/3510788054 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f25480a2c20 msgr2=0x7f25480a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:48.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.847+0000 7f256ca00700 1 --2- 172.21.15.117:0/3510788054 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f25480a2c20 0x7f25480a50e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2568108520 tx=0x7f25500073d0 comp rx=0 tx=0).stop 2024-06-03T03:23:48.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.847+0000 7f256ca00700 1 -- 172.21.15.117:0/3510788054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25681089d0 msgr2=0x7f256819f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:48.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.847+0000 7f256ca00700 1 --2- 172.21.15.117:0/3510788054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25681089d0 0x7f256819f1f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2558000c00 tx=0x7f255800ba90 comp rx=0 tx=0).stop 2024-06-03T03:23:48.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.847+0000 7f256ca00700 1 -- 172.21.15.117:0/3510788054 shutdown_connections 2024-06-03T03:23:48.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.847+0000 7f256ca00700 1 --2- 172.21.15.117:0/3510788054 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f25480a2c20 0x7f25480a50e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:48.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.847+0000 7f256ca00700 1 --2- 172.21.15.117:0/3510788054 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f256810d8a0 0x7f25681a3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:48.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.847+0000 7f256ca00700 1 --2- 172.21.15.117:0/3510788054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25681089d0 0x7f256819f1f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:48.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.847+0000 7f256ca00700 1 --2- 172.21.15.117:0/3510788054 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2568109390 0x7f256819f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:48.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.847+0000 7f256ca00700 1 -- 172.21.15.117:0/3510788054 >> 172.21.15.117:0/3510788054 conn(0x7f2568078220 msgr2=0x7f2568078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:23:48.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.847+0000 7f256ca00700 1 -- 172.21.15.117:0/3510788054 shutdown_connections 2024-06-03T03:23:48.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:48.847+0000 7f256ca00700 1 -- 172.21.15.117:0/3510788054 wait complete. 2024-06-03T03:23:49.898 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph tell mds.a status 2024-06-03T03:23:50.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:23:49 smithi117 ceph-mon[31649]: pgmap v461: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-03T03:23:50.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:23:49 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:23:50.344 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:23:50.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:23:49 smithi151 ceph-mon[37043]: pgmap v461: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-03T03:23:50.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:23:49 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:23:50.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:23:49 smithi183 ceph-mon[37326]: pgmap v461: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-03T03:23:50.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:23:49 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:23:51.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:23:51 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:23:51.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:23:51 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:23:51.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:23:51 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:23:51.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:23:51 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:23:51.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:23:51 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:23:51.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:23:51 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:23:51.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:23:51 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:23:51.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:23:51 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:23:51.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:23:51 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:23:52.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:23:52 smithi117 ceph-mon[31649]: pgmap v462: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:23:52.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:23:52 smithi151 ceph-mon[37043]: pgmap v462: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:23:52.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:23:52 smithi183 ceph-mon[37326]: pgmap v462: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:23:52.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.862+0000 7f2d12600700 1 -- 172.21.15.117:0/2308696808 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d0c10a6b0 msgr2=0x7f2d0c10aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:52.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.862+0000 7f2d12600700 1 --2- 172.21.15.117:0/2308696808 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d0c10a6b0 0x7f2d0c10aa90 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc0098f0 tx=0x7f2cfc009c00 comp rx=0 tx=0).stop 2024-06-03T03:23:52.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.862+0000 7f2d12600700 1 -- 172.21.15.117:0/2308696808 shutdown_connections 2024-06-03T03:23:52.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.862+0000 7f2d12600700 1 --2- 172.21.15.117:0/2308696808 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2d0c105070 0x7f2d0c111580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:52.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.862+0000 7f2d12600700 1 --2- 172.21.15.117:0/2308696808 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d0c10a6b0 0x7f2d0c10aa90 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:52.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.862+0000 7f2d12600700 1 --2- 172.21.15.117:0/2308696808 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d0c1046b0 0x7f2d0c104b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:52.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.862+0000 7f2d12600700 1 -- 172.21.15.117:0/2308696808 >> 172.21.15.117:0/2308696808 conn(0x7f2d0c1001a0 msgr2=0x7f2d0c1025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:23:52.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.862+0000 7f2d12600700 1 -- 172.21.15.117:0/2308696808 shutdown_connections 2024-06-03T03:23:52.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.862+0000 7f2d12600700 1 -- 172.21.15.117:0/2308696808 wait complete. 2024-06-03T03:23:52.866 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.864+0000 7f2d12600700 1 Processor -- start 2024-06-03T03:23:52.866 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.865+0000 7f2d12600700 1 -- start start 2024-06-03T03:23:52.867 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.865+0000 7f2d12600700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2d0c1046b0 0x7f2d0c19ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:52.867 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.865+0000 7f2d12600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d0c105070 0x7f2d0c19b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:52.867 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.865+0000 7f2d12600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d0c10a6b0 0x7f2d0c19f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:52.868 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.865+0000 7f2d12600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2d0c114a00 con 0x7f2d0c105070 2024-06-03T03:23:52.868 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.865+0000 7f2d12600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2d0c114880 con 0x7f2d0c10a6b0 2024-06-03T03:23:52.868 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.865+0000 7f2d12600700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2d0c114b80 con 0x7f2d0c1046b0 2024-06-03T03:23:52.868 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.866+0000 7f2d0b400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2d0c1046b0 0x7f2d0c19ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:52.868 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.866+0000 7f2d0b400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2d0c1046b0 0x7f2d0c19ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:48040/0 (socket says 172.21.15.117:48040) 2024-06-03T03:23:52.868 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.866+0000 7f2d0b400700 1 -- 172.21.15.117:0/3913040662 learned_addr learned my addr 172.21.15.117:0/3913040662 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:23:52.868 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.866+0000 7f2d0aa00700 1 --2- 172.21.15.117:0/3913040662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d0c105070 0x7f2d0c19b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:52.868 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.866+0000 7f2d0be00700 1 --2- 172.21.15.117:0/3913040662 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d0c10a6b0 0x7f2d0c19f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:52.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.866+0000 7f2d0b400700 1 -- 172.21.15.117:0/3913040662 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d0c10a6b0 msgr2=0x7f2d0c19f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:52.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.866+0000 7f2d0b400700 1 --2- 172.21.15.117:0/3913040662 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d0c10a6b0 0x7f2d0c19f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:52.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.866+0000 7f2d0b400700 1 -- 172.21.15.117:0/3913040662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d0c105070 msgr2=0x7f2d0c19b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:52.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.866+0000 7f2d0b400700 1 --2- 172.21.15.117:0/3913040662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d0c105070 0x7f2d0c19b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:52.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.866+0000 7f2d0b400700 1 -- 172.21.15.117:0/3913040662 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d0c19fe60 con 0x7f2d0c1046b0 2024-06-03T03:23:52.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.867+0000 7f2d0b400700 1 --2- 172.21.15.117:0/3913040662 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2d0c1046b0 0x7f2d0c19ae50 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc009230 tx=0x7f2cfc00fa70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:23:52.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.867+0000 7f2d03e00700 1 -- 172.21.15.117:0/3913040662 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc005140 con 0x7f2d0c1046b0 2024-06-03T03:23:52.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.867+0000 7f2d12600700 1 -- 172.21.15.117:0/3913040662 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d0c1a00f0 con 0x7f2d0c1046b0 2024-06-03T03:23:52.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.867+0000 7f2d03e00700 1 -- 172.21.15.117:0/3913040662 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cfc00b560 con 0x7f2d0c1046b0 2024-06-03T03:23:52.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.867+0000 7f2d03e00700 1 -- 172.21.15.117:0/3913040662 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc003a50 con 0x7f2d0c1046b0 2024-06-03T03:23:52.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.867+0000 7f2d12600700 1 -- 172.21.15.117:0/3913040662 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d0c1a79d0 con 0x7f2d0c1046b0 2024-06-03T03:23:52.873 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.871+0000 7f2d03e00700 1 -- 172.21.15.117:0/3913040662 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f2cfc029060 con 0x7f2d0c1046b0 2024-06-03T03:23:52.874 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.872+0000 7f2d1cedfb80 1 Processor -- start 2024-06-03T03:23:52.874 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.872+0000 7f2d1cedfb80 1 -- start start 2024-06-03T03:23:52.874 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.873+0000 7f2d1cedfb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559932983df0 0x559932a07020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:52.875 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.873+0000 7f2d1cedfb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559932a07560 0x559932a079e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:52.875 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.873+0000 7f2d1cedfb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559932a122b0 0x559932a146a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:52.875 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.873+0000 7f2d1cedfb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5599329169c0 con 0x559932a07560 2024-06-03T03:23:52.875 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.873+0000 7f2d1cedfb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559932959ab0 con 0x559932983df0 2024-06-03T03:23:52.875 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.873+0000 7f2d1cedfb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5599329148a0 con 0x559932a122b0 2024-06-03T03:23:52.875 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.873+0000 7f2d0be00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559932983df0 0x559932a07020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:52.875 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.873+0000 7f2d0be00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559932983df0 0x559932a07020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:41444/0 (socket says 172.21.15.117:41444) 2024-06-03T03:23:52.875 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.873+0000 7f2d0be00700 1 -- 172.21.15.117:0/241205854 learned_addr learned my addr 172.21.15.117:0/241205854 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:23:52.876 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.873+0000 7f2d03e00700 1 --2- 172.21.15.117:0/3913040662 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f2cf00a2c20 0x7f2cf00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:52.876 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.873+0000 7f2d0b400700 1 --2- 172.21.15.117:0/241205854 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559932a07560 0x559932a079e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:52.876 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.873+0000 7f2d0aa00700 1 --2- 172.21.15.117:0/241205854 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559932a122b0 0x559932a146a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:52.876 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.874+0000 7f2d0be00700 1 -- 172.21.15.117:0/241205854 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559932a122b0 msgr2=0x559932a146a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:23:52.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.874+0000 7f2d0be00700 1 --2- 172.21.15.117:0/241205854 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559932a122b0 0x559932a146a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:52.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.874+0000 7f2d0be00700 1 -- 172.21.15.117:0/241205854 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559932a07560 msgr2=0x559932a079e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:23:52.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.874+0000 7f2d0be00700 1 --2- 172.21.15.117:0/241205854 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559932a07560 0x559932a079e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:52.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.874+0000 7f2d0be00700 1 -- 172.21.15.117:0/241205854 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559932901f40 con 0x559932983df0 2024-06-03T03:23:52.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.874+0000 7f2d03e00700 1 -- 172.21.15.117:0/3913040662 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f2cfc0cfc60 con 0x7f2d0c1046b0 2024-06-03T03:23:52.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.874+0000 7f2d0be00700 1 --2- 172.21.15.117:0/3913040662 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f2cf00a2c20 0x7f2cf00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:52.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.874+0000 7f2d0be00700 1 --2- 172.21.15.117:0/241205854 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559932983df0 0x559932a07020 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2d04011790 tx=0x7f2d04011aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e650c75ce961f644 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:23:52.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.874+0000 7f2d00c00700 1 -- 172.21.15.117:0/241205854 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0401c040 con 0x559932983df0 2024-06-03T03:23:52.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.874+0000 7f2d00c00700 1 -- 172.21.15.117:0/241205854 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d04018b60 con 0x559932983df0 2024-06-03T03:23:52.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.874+0000 7f2d00c00700 1 -- 172.21.15.117:0/241205854 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d04023a90 con 0x559932983df0 2024-06-03T03:23:52.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.875+0000 7f2d1cedfb80 1 -- 172.21.15.117:0/241205854 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559932983df0 msgr2=0x559932a07020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:52.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.875+0000 7f2d1cedfb80 1 --2- 172.21.15.117:0/241205854 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559932983df0 0x559932a07020 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2d04011790 tx=0x7f2d04011aa0 comp rx=0 tx=0).stop 2024-06-03T03:23:52.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.875+0000 7f2d0be00700 1 --2- 172.21.15.117:0/3913040662 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f2cf00a2c20 0x7f2cf00a50e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2d04022000 tx=0x7f2d0401f000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:23:52.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.875+0000 7f2d1cedfb80 1 -- 172.21.15.117:0/241205854 shutdown_connections 2024-06-03T03:23:52.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.875+0000 7f2d1cedfb80 1 --2- 172.21.15.117:0/241205854 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559932a122b0 0x559932a146a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:52.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.875+0000 7f2d1cedfb80 1 --2- 172.21.15.117:0/241205854 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559932983df0 0x559932a07020 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:52.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.875+0000 7f2d1cedfb80 1 --2- 172.21.15.117:0/241205854 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559932a07560 0x559932a079e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:52.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.875+0000 7f2d1cedfb80 1 -- 172.21.15.117:0/241205854 >> 172.21.15.117:0/241205854 conn(0x559932934120 msgr2=0x559932a03460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:23:52.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.875+0000 7f2d1cedfb80 1 -- 172.21.15.117:0/241205854 shutdown_connections 2024-06-03T03:23:52.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.876+0000 7f2d1cedfb80 1 -- 172.21.15.117:0/241205854 wait complete. 2024-06-03T03:23:52.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.876+0000 7f2d1cedfb80 1 -- start start 2024-06-03T03:23:52.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.876+0000 7f2d1cedfb80 1 Processor -- start 2024-06-03T03:23:52.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.877+0000 7f2d1cedfb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559932983df0 0x559932a11a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:52.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.877+0000 7f2d1cedfb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559932a07560 0x559932a0ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:52.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.877+0000 7f2d1cedfb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559932a122b0 0x559932a0f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:52.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.877+0000 7f2d1cedfb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5599328f5f10 con 0x559932983df0 2024-06-03T03:23:52.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.877+0000 7f2d1cedfb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559932955a70 con 0x559932a122b0 2024-06-03T03:23:52.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.877+0000 7f2d1cedfb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559932a14f70 con 0x559932a07560 2024-06-03T03:23:52.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.877+0000 7f2d1cedfb80 10 client.0 _ll_register_callbacks cb 0x5599328f36d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:23:52.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.877+0000 7f2d0b400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559932983df0 0x559932a11a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:52.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.877+0000 7f2d0b400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559932983df0 0x559932a11a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34846/0 (socket says 172.21.15.117:34846) 2024-06-03T03:23:52.882 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.877+0000 7f2d0b400700 1 -- 172.21.15.117:0/486512950 learned_addr learned my addr 172.21.15.117:0/486512950 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:23:52.882 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.877+0000 7f2d0aa00700 1 --2- 172.21.15.117:0/486512950 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559932a07560 0x559932a0ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:52.882 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.877+0000 7f2d0be00700 1 --2- 172.21.15.117:0/486512950 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559932a122b0 0x559932a0f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:52.882 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.878+0000 7f2d0b400700 1 -- 172.21.15.117:0/486512950 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559932a07560 msgr2=0x559932a0ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:23:52.882 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.878+0000 7f2d0b400700 1 --2- 172.21.15.117:0/486512950 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559932a07560 0x559932a0ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:52.882 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.878+0000 7f2d0b400700 1 -- 172.21.15.117:0/486512950 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559932a122b0 msgr2=0x559932a0f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:23:52.882 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.878+0000 7f2d0b400700 1 --2- 172.21.15.117:0/486512950 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559932a122b0 0x559932a0f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:52.882 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.878+0000 7f2d0b400700 1 -- 172.21.15.117:0/486512950 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599328a8880 con 0x559932983df0 2024-06-03T03:23:52.882 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.878+0000 7f2d0aa00700 1 --2- 172.21.15.117:0/486512950 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559932a07560 0x559932a0ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T03:23:52.882 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.878+0000 7f2d0b400700 1 --2- 172.21.15.117:0/486512950 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559932983df0 0x559932a11a70 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc005e90 tx=0x7f2cfc0095a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acefed25c3b77beb server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:23:52.882 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.878+0000 7f2cf5600700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-06-03T03:23:52.883 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.878+0000 7f2cf5600700 1 -- 172.21.15.117:0/486512950 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc0d4040 con 0x559932983df0 2024-06-03T03:23:52.883 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.878+0000 7f2cf5600700 1 -- 172.21.15.117:0/486512950 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f2cfc0a00b0 con 0x559932983df0 2024-06-03T03:23:52.883 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.878+0000 7f2d1cedfb80 1 -- 172.21.15.117:0/486512950 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d04018b60 con 0x559932983df0 2024-06-03T03:23:52.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.879+0000 7f2cf5600700 1 -- 172.21.15.117:0/486512950 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2cfc0b50d0 con 0x559932983df0 2024-06-03T03:23:52.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.879+0000 7f2d1cedfb80 10 client.14874 fetch_fsmap learned FSMap version 30 2024-06-03T03:23:52.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.879+0000 7f2d1cedfb80 1 -- 172.21.15.117:0/486512950 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559932a1f6f0 con 0x559932983df0 2024-06-03T03:23:52.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.879+0000 7f2cf5600700 1 -- 172.21.15.117:0/486512950 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d04018b60 con 0x559932983df0 2024-06-03T03:23:52.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.879+0000 7f2cf5600700 1 -- 172.21.15.117:0/486512950 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f2cfc09c3c0 con 0x559932983df0 2024-06-03T03:23:52.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.879+0000 7f2d1cedfb80 10 client.14874 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T03:23:52.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.879+0000 7f2d1cedfb80 10 client.14874 resolve_mds: resolved name 'a' to daemon mds.a 2024-06-03T03:23:52.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.879+0000 7f2d1cedfb80 20 client.14874 populate_metadata read hostname 'smithi117' 2024-06-03T03:23:52.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.879+0000 7f2d1cedfb80 1 --2- 172.21.15.117:0/486512950 >> [v2:172.21.15.117:6840/1212242967,v1:172.21.15.117:6841/1212242967] conn(0x559932a21d90 0x559932a23200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:52.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.879+0000 7f2d1cedfb80 4 client.14874 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T03:23:52.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.879+0000 7f2d1cedfb80 1 -- 172.21.15.117:0/486512950 --> [v2:172.21.15.117:6840/1212242967,v1:172.21.15.117:6841/1212242967] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559932966fb0 con 0x559932a21d90 2024-06-03T03:23:52.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.880+0000 7f2d0aa00700 1 --2- 172.21.15.117:0/486512950 >> [v2:172.21.15.117:6840/1212242967,v1:172.21.15.117:6841/1212242967] conn(0x559932a21d90 0x559932a23200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:52.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.880+0000 7f2d0aa00700 1 --2- 172.21.15.117:0/486512950 >> [v2:172.21.15.117:6840/1212242967,v1:172.21.15.117:6841/1212242967] conn(0x559932a21d90 0x559932a23200 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a28dde1a95ac7717 server_cookie=c848b6969a7728ef in_seq=0 out_seq=0 2024-06-03T03:23:52.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.880+0000 7f2cf5600700 10 client.14874 ms_handle_connect on v2:172.21.15.117:6840/1212242967 2024-06-03T03:23:52.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.881+0000 7f2cf5600700 1 -- 172.21.15.117:0/486512950 <== mds.2 v2:172.21.15.117:6840/1212242967 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559932966fb0 con 0x559932a21d90 2024-06-03T03:23:52.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.881+0000 7f2cf5600700 10 client.14874 handle_command_reply: tid=0 2024-06-03T03:23:52.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.881+0000 7f2d1cedfb80 1 client.14874 shutdown 2024-06-03T03:23:52.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.881+0000 7f2d1cedfb80 1 -- 172.21.15.117:0/486512950 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559932983df0 msgr2=0x559932a11a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:52.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.881+0000 7f2d1cedfb80 1 --2- 172.21.15.117:0/486512950 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559932983df0 0x559932a11a70 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc005e90 tx=0x7f2cfc0095a0 comp rx=0 tx=0).stop 2024-06-03T03:23:52.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.882+0000 7f2d1cedfb80 1 -- 172.21.15.117:0/486512950 shutdown_connections 2024-06-03T03:23:52.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.882+0000 7f2d1cedfb80 1 --2- 172.21.15.117:0/486512950 >> [v2:172.21.15.117:6840/1212242967,v1:172.21.15.117:6841/1212242967] conn(0x559932a21d90 0x559932a23200 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:52.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.882+0000 7f2d1cedfb80 1 --2- 172.21.15.117:0/486512950 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559932a122b0 0x559932a0f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:52.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.882+0000 7f2d1cedfb80 1 --2- 172.21.15.117:0/486512950 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559932983df0 0x559932a11a70 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:52.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.882+0000 7f2d1cedfb80 1 --2- 172.21.15.117:0/486512950 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559932a07560 0x559932a0ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:52.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.882+0000 7f2d1cedfb80 1 -- 172.21.15.117:0/486512950 >> 172.21.15.117:0/486512950 conn(0x559932934120 msgr2=0x559932a026f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:23:52.887 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.882+0000 7f2cf5600700 0 client.14874 ms_handle_reset on v2:172.21.15.117:6840/1212242967 2024-06-03T03:23:52.887 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.882+0000 7f2d1cedfb80 1 -- 172.21.15.117:0/486512950 shutdown_connections 2024-06-03T03:23:52.887 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.882+0000 7f2d1cedfb80 1 -- 172.21.15.117:0/486512950 wait complete. 2024-06-03T03:23:52.887 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.882+0000 7f2d1cedfb80 20 client.14874 trim_cache size 0 max 16384 2024-06-03T03:23:52.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.905+0000 7f2d1cedfb80 1 Processor -- start 2024-06-03T03:23:52.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.905+0000 7f2d1cedfb80 1 -- start start 2024-06-03T03:23:52.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.906+0000 7f2d1cedfb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559932983df0 0x559932a0f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:52.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.906+0000 7f2d1cedfb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559932a07560 0x559932a115b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:52.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.906+0000 7f2d1cedfb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559932a122b0 0x559932a136a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:52.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.906+0000 7f2d1cedfb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559932a21540 con 0x559932a07560 2024-06-03T03:23:52.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.906+0000 7f2d1cedfb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559932a21c10 con 0x559932983df0 2024-06-03T03:23:52.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.906+0000 7f2d1cedfb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559932a222e0 con 0x559932a122b0 2024-06-03T03:23:52.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.906+0000 7f2d0b400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559932983df0 0x559932a0f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:52.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.906+0000 7f2d0aa00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559932a07560 0x559932a115b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:52.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.906+0000 7f2d0b400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559932983df0 0x559932a0f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:41462/0 (socket says 172.21.15.117:41462) 2024-06-03T03:23:52.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.906+0000 7f2d0b400700 1 -- 172.21.15.117:0/1199969805 learned_addr learned my addr 172.21.15.117:0/1199969805 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:23:52.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.906+0000 7f2d0aa00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559932a07560 0x559932a115b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34862/0 (socket says 172.21.15.117:34862) 2024-06-03T03:23:52.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.906+0000 7f2d0be00700 1 --2- 172.21.15.117:0/1199969805 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559932a122b0 0x559932a136a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:52.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.907+0000 7f2d0b400700 1 -- 172.21.15.117:0/1199969805 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559932a122b0 msgr2=0x559932a136a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:23:52.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.907+0000 7f2d0b400700 1 --2- 172.21.15.117:0/1199969805 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559932a122b0 0x559932a136a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:52.910 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.907+0000 7f2d0b400700 1 -- 172.21.15.117:0/1199969805 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559932a07560 msgr2=0x559932a115b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:23:52.910 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.907+0000 7f2d0b400700 1 --2- 172.21.15.117:0/1199969805 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559932a07560 0x559932a115b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:52.910 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.907+0000 7f2d0b400700 1 -- 172.21.15.117:0/1199969805 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559932a09f80 con 0x559932983df0 2024-06-03T03:23:52.910 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.907+0000 7f2d0b400700 1 --2- 172.21.15.117:0/1199969805 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559932983df0 0x559932a0f2e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc00c010 tx=0x7f2cfc096b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa3d5e1e1ce94a7c server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:23:52.912 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.907+0000 7f2cf7e00700 1 -- 172.21.15.117:0/1199969805 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc0a4b00 con 0x559932983df0 2024-06-03T03:23:52.912 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.907+0000 7f2cf7e00700 1 -- 172.21.15.117:0/1199969805 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cfc0a0510 con 0x559932983df0 2024-06-03T03:23:52.912 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.907+0000 7f2cf7e00700 1 -- 172.21.15.117:0/1199969805 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc0a00b0 con 0x559932983df0 2024-06-03T03:23:52.912 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.908+0000 7f2d1cedfb80 1 -- 172.21.15.117:0/1199969805 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559932983df0 msgr2=0x559932a0f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:52.912 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.908+0000 7f2d1cedfb80 1 --2- 172.21.15.117:0/1199969805 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559932983df0 0x559932a0f2e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc00c010 tx=0x7f2cfc096b50 comp rx=0 tx=0).stop 2024-06-03T03:23:52.912 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.908+0000 7f2d1cedfb80 1 -- 172.21.15.117:0/1199969805 shutdown_connections 2024-06-03T03:23:52.912 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.908+0000 7f2d1cedfb80 1 --2- 172.21.15.117:0/1199969805 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559932a122b0 0x559932a136a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:52.913 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.908+0000 7f2d1cedfb80 1 --2- 172.21.15.117:0/1199969805 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559932983df0 0x559932a0f2e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:52.913 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.908+0000 7f2d1cedfb80 1 --2- 172.21.15.117:0/1199969805 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559932a07560 0x559932a115b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:52.913 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.908+0000 7f2d1cedfb80 1 -- 172.21.15.117:0/1199969805 >> 172.21.15.117:0/1199969805 conn(0x559932934120 msgr2=0x559932a0d660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:23:52.913 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.908+0000 7f2d1cedfb80 1 -- 172.21.15.117:0/1199969805 shutdown_connections 2024-06-03T03:23:52.913 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.908+0000 7f2d1cedfb80 1 -- 172.21.15.117:0/1199969805 wait complete. 2024-06-03T03:23:52.913 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.909+0000 7f2d1cedfb80 1 -- start start 2024-06-03T03:23:52.913 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.909+0000 7f2d1cedfb80 1 Processor -- start 2024-06-03T03:23:52.913 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.910+0000 7f2d1cedfb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559932983df0 0x559932a2cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:52.913 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.910+0000 7f2d1cedfb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559932a07560 0x559932a023e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:52.913 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.910+0000 7f2d1cedfb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559932a122b0 0x559932a02920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:52.913 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.910+0000 7f2d1cedfb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559932a23080 con 0x559932983df0 2024-06-03T03:23:52.913 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.910+0000 7f2d1cedfb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559932a229b0 con 0x559932a122b0 2024-06-03T03:23:52.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.910+0000 7f2d1cedfb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559932a23750 con 0x559932a07560 2024-06-03T03:23:52.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.910+0000 7f2d1cedfb80 10 client.0 _ll_register_callbacks cb 0x5599328f36d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:23:52.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.910+0000 7f2d0b400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559932983df0 0x559932a2cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:52.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.910+0000 7f2d0be00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559932a122b0 0x559932a02920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:52.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.910+0000 7f2d0b400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559932983df0 0x559932a2cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34872/0 (socket says 172.21.15.117:34872) 2024-06-03T03:23:52.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.910+0000 7f2d0b400700 1 -- 172.21.15.117:0/1235410777 learned_addr learned my addr 172.21.15.117:0/1235410777 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:23:52.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.910+0000 7f2d0aa00700 1 --2- 172.21.15.117:0/1235410777 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559932a07560 0x559932a023e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:52.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.910+0000 7f2d0b400700 1 -- 172.21.15.117:0/1235410777 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559932a07560 msgr2=0x559932a023e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:23:52.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.910+0000 7f2d0b400700 1 --2- 172.21.15.117:0/1235410777 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559932a07560 0x559932a023e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:52.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.910+0000 7f2d0b400700 1 -- 172.21.15.117:0/1235410777 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559932a122b0 msgr2=0x559932a02920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:23:52.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.910+0000 7f2d0b400700 1 --2- 172.21.15.117:0/1235410777 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559932a122b0 0x559932a02920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:52.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.910+0000 7f2d0b400700 1 -- 172.21.15.117:0/1235410777 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559932a02fd0 con 0x559932983df0 2024-06-03T03:23:52.915 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.911+0000 7f2d0b400700 1 --2- 172.21.15.117:0/1235410777 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559932983df0 0x559932a2cbe0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc0098c0 tx=0x7f2cfc0bc2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=652e720d03cb4eca server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:23:52.915 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.911+0000 7f2cf5600700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-06-03T03:23:52.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.911+0000 7f2cf5600700 1 -- 172.21.15.117:0/1235410777 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc09c160 con 0x559932983df0 2024-06-03T03:23:52.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.911+0000 7f2cf5600700 1 -- 172.21.15.117:0/1235410777 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f2cfc09b6a0 con 0x559932983df0 2024-06-03T03:23:52.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.911+0000 7f2d1cedfb80 1 -- 172.21.15.117:0/1235410777 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2cfc0a0510 con 0x559932983df0 2024-06-03T03:23:52.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.911+0000 7f2cf5600700 1 -- 172.21.15.117:0/1235410777 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2cfc09b8a0 con 0x559932983df0 2024-06-03T03:23:52.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.911+0000 7f2d1cedfb80 10 client.14880 fetch_fsmap learned FSMap version 30 2024-06-03T03:23:52.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.911+0000 7f2d1cedfb80 1 -- 172.21.15.117:0/1235410777 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559932a02fd0 con 0x559932983df0 2024-06-03T03:23:52.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.911+0000 7f2cf5600700 1 -- 172.21.15.117:0/1235410777 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cfc0a0510 con 0x559932983df0 2024-06-03T03:23:52.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.912+0000 7f2cf5600700 1 -- 172.21.15.117:0/1235410777 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f2cfc09b020 con 0x559932983df0 2024-06-03T03:23:52.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.912+0000 7f2d1cedfb80 10 client.14880 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T03:23:52.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.912+0000 7f2d1cedfb80 10 client.14880 resolve_mds: resolved name 'a' to daemon mds.a 2024-06-03T03:23:52.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.912+0000 7f2d1cedfb80 20 client.14880 populate_metadata read hostname 'smithi117' 2024-06-03T03:23:52.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.912+0000 7f2d1cedfb80 1 --2- 172.21.15.117:0/1235410777 >> [v2:172.21.15.117:6840/1212242967,v1:172.21.15.117:6841/1212242967] conn(0x559932a91a70 0x559932a90780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:52.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.912+0000 7f2d1cedfb80 4 client.14880 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-03T03:23:52.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.912+0000 7f2d1cedfb80 1 -- 172.21.15.117:0/1235410777 --> [v2:172.21.15.117:6840/1212242967,v1:172.21.15.117:6841/1212242967] -- command(tid 0: {"prefix": "status"}) -- 0x559932961530 con 0x559932a91a70 2024-06-03T03:23:52.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.912+0000 7f2d0aa00700 1 --2- 172.21.15.117:0/1235410777 >> [v2:172.21.15.117:6840/1212242967,v1:172.21.15.117:6841/1212242967] conn(0x559932a91a70 0x559932a90780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:52.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.913+0000 7f2cf5600700 10 client.14880 ms_handle_connect on v2:172.21.15.117:6840/1212242967 2024-06-03T03:23:52.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.913+0000 7f2d0aa00700 1 --2- 172.21.15.117:0/1235410777 >> [v2:172.21.15.117:6840/1212242967,v1:172.21.15.117:6841/1212242967] conn(0x559932a91a70 0x559932a90780 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9b48620ced1b6995 server_cookie=9219ef1880db07fd in_seq=0 out_seq=0 2024-06-03T03:23:52.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.913+0000 7f2cf5600700 1 -- 172.21.15.117:0/1235410777 <== mds.2 v2:172.21.15.117:6840/1212242967 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x559932961530 con 0x559932a91a70 2024-06-03T03:23:52.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.913+0000 7f2cf5600700 10 client.14880 handle_command_reply: tid=0 2024-06-03T03:23:52.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.913+0000 7f2d1cedfb80 1 client.14880 shutdown 2024-06-03T03:23:52.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.913+0000 7f2d1cedfb80 1 -- 172.21.15.117:0/1235410777 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559932983df0 msgr2=0x559932a2cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:52.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.913+0000 7f2d1cedfb80 1 --2- 172.21.15.117:0/1235410777 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559932983df0 0x559932a2cbe0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc0098c0 tx=0x7f2cfc0bc2e0 comp rx=0 tx=0).stop 2024-06-03T03:23:52.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.913+0000 7f2d1cedfb80 1 -- 172.21.15.117:0/1235410777 shutdown_connections 2024-06-03T03:23:52.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.913+0000 7f2d1cedfb80 1 --2- 172.21.15.117:0/1235410777 >> [v2:172.21.15.117:6840/1212242967,v1:172.21.15.117:6841/1212242967] conn(0x559932a91a70 0x559932a90780 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:52.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.913+0000 7f2d1cedfb80 1 --2- 172.21.15.117:0/1235410777 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559932a122b0 0x559932a02920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:52.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.913+0000 7f2d1cedfb80 1 --2- 172.21.15.117:0/1235410777 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559932983df0 0x559932a2cbe0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:52.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.913+0000 7f2d1cedfb80 1 --2- 172.21.15.117:0/1235410777 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559932a07560 0x559932a023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:52.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.913+0000 7f2d1cedfb80 1 -- 172.21.15.117:0/1235410777 >> 172.21.15.117:0/1235410777 conn(0x559932934120 msgr2=0x559932a08f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:23:52.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.913+0000 7f2cf5600700 0 client.14880 ms_handle_reset on v2:172.21.15.117:6840/1212242967 2024-06-03T03:23:52.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.914+0000 7f2d1cedfb80 1 -- 172.21.15.117:0/1235410777 shutdown_connections 2024-06-03T03:23:52.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.914+0000 7f2d1cedfb80 1 -- 172.21.15.117:0/1235410777 wait complete. 2024-06-03T03:23:52.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.914+0000 7f2d1cedfb80 20 client.14880 trim_cache size 0 max 16384 2024-06-03T03:23:52.919 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-06-03T03:23:52.919 INFO:teuthology.orchestra.run.smithi117.stdout: "cluster_fsid": "a0e8473a-2156-11ef-bc9b-c7b262605968", 2024-06-03T03:23:52.920 INFO:teuthology.orchestra.run.smithi117.stdout: "whoami": 2, 2024-06-03T03:23:52.920 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 24503, 2024-06-03T03:23:52.920 INFO:teuthology.orchestra.run.smithi117.stdout: "want_state": "up:active", 2024-06-03T03:23:52.920 INFO:teuthology.orchestra.run.smithi117.stdout: "state": "up:active", 2024-06-03T03:23:52.920 INFO:teuthology.orchestra.run.smithi117.stdout: "fs_name": "cephfs", 2024-06-03T03:23:52.920 INFO:teuthology.orchestra.run.smithi117.stdout: "rank_uptime": 168.923761039, 2024-06-03T03:23:52.920 INFO:teuthology.orchestra.run.smithi117.stdout: "mdsmap_epoch": 30, 2024-06-03T03:23:52.920 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_epoch": 99, 2024-06-03T03:23:52.920 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_epoch_barrier": 97, 2024-06-03T03:23:52.920 INFO:teuthology.orchestra.run.smithi117.stdout: "uptime": 270.89413329400003 2024-06-03T03:23:52.920 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-06-03T03:23:52.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.914+0000 7f2d12600700 1 -- 172.21.15.117:0/3913040662 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f2cf00a2c20 msgr2=0x7f2cf00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:52.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.914+0000 7f2d12600700 1 --2- 172.21.15.117:0/3913040662 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f2cf00a2c20 0x7f2cf00a50e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2d04022000 tx=0x7f2d0401f000 comp rx=0 tx=0).stop 2024-06-03T03:23:52.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.914+0000 7f2d12600700 1 -- 172.21.15.117:0/3913040662 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2d0c1046b0 msgr2=0x7f2d0c19ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:52.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.914+0000 7f2d12600700 1 --2- 172.21.15.117:0/3913040662 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2d0c1046b0 0x7f2d0c19ae50 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc009230 tx=0x7f2cfc00fa70 comp rx=0 tx=0).stop 2024-06-03T03:23:52.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.915+0000 7f2d12600700 1 -- 172.21.15.117:0/3913040662 shutdown_connections 2024-06-03T03:23:52.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.915+0000 7f2d12600700 1 --2- 172.21.15.117:0/3913040662 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f2cf00a2c20 0x7f2cf00a50e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:52.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.915+0000 7f2d12600700 1 --2- 172.21.15.117:0/3913040662 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d0c10a6b0 0x7f2d0c19f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:52.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.915+0000 7f2d12600700 1 --2- 172.21.15.117:0/3913040662 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2d0c1046b0 0x7f2d0c19ae50 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:52.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.915+0000 7f2d12600700 1 --2- 172.21.15.117:0/3913040662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d0c105070 0x7f2d0c19b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:52.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.915+0000 7f2d12600700 1 -- 172.21.15.117:0/3913040662 >> 172.21.15.117:0/3913040662 conn(0x7f2d0c1001a0 msgr2=0x7f2d0c1008a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:23:52.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.915+0000 7f2d12600700 1 -- 172.21.15.117:0/3913040662 shutdown_connections 2024-06-03T03:23:52.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:52.915+0000 7f2d12600700 1 -- 172.21.15.117:0/3913040662 wait complete. 2024-06-03T03:23:53.793 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph tell mds.b status 2024-06-03T03:23:54.243 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:23:54.388 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:23:54 smithi117 ceph-mon[31649]: pgmap v463: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-06-03T03:23:54.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:23:54 smithi151 ceph-mon[37043]: pgmap v463: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-06-03T03:23:54.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:23:54 smithi183 ceph-mon[37326]: pgmap v463: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-06-03T03:23:56.189 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:23:56 smithi117 ceph-mon[31649]: pgmap v464: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:23:56.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:23:56 smithi151 ceph-mon[37043]: pgmap v464: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:23:56.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:23:56 smithi183 ceph-mon[37326]: pgmap v464: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:23:56.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.716+0000 7f6228a00700 1 -- 172.21.15.117:0/1045041038 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6224105070 msgr2=0x7f6224111580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:56.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.716+0000 7f6228a00700 1 --2- 172.21.15.117:0/1045041038 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6224105070 0x7f6224111580 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f621c00b0a0 tx=0x7f621c00b3b0 comp rx=0 tx=0).stop 2024-06-03T03:23:56.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.716+0000 7f6228a00700 1 -- 172.21.15.117:0/1045041038 shutdown_connections 2024-06-03T03:23:56.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.716+0000 7f6228a00700 1 --2- 172.21.15.117:0/1045041038 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6224105070 0x7f6224111580 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:56.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.716+0000 7f6228a00700 1 --2- 172.21.15.117:0/1045041038 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f622410a6b0 0x7f622410aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:56.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.716+0000 7f6228a00700 1 --2- 172.21.15.117:0/1045041038 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f62241046b0 0x7f6224104b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:56.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.716+0000 7f6228a00700 1 -- 172.21.15.117:0/1045041038 >> 172.21.15.117:0/1045041038 conn(0x7f62241001a0 msgr2=0x7f62241025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:23:56.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.716+0000 7f6228a00700 1 -- 172.21.15.117:0/1045041038 shutdown_connections 2024-06-03T03:23:56.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.717+0000 7f6228a00700 1 -- 172.21.15.117:0/1045041038 wait complete. 2024-06-03T03:23:56.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.718+0000 7f6228a00700 1 Processor -- start 2024-06-03T03:23:56.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.718+0000 7f6228a00700 1 -- start start 2024-06-03T03:23:56.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.718+0000 7f6228a00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f62241046b0 0x7f622419ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:56.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.718+0000 7f6228a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6224105070 0x7f622419b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:56.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.718+0000 7f6228a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f622410a6b0 0x7f622419f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:56.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.718+0000 7f6228a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6224114a40 con 0x7f622410a6b0 2024-06-03T03:23:56.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.718+0000 7f6228a00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f62241148c0 con 0x7f6224105070 2024-06-03T03:23:56.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.718+0000 7f6228a00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6224114bc0 con 0x7f62241046b0 2024-06-03T03:23:56.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.719+0000 7f6221e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f62241046b0 0x7f622419ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:56.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.719+0000 7f6222800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f622410a6b0 0x7f622419f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:56.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.719+0000 7f6221e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f62241046b0 0x7f622419ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:48116/0 (socket says 172.21.15.117:48116) 2024-06-03T03:23:56.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.719+0000 7f6221e00700 1 -- 172.21.15.117:0/1532898147 learned_addr learned my addr 172.21.15.117:0/1532898147 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:23:56.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.719+0000 7f6221400700 1 --2- 172.21.15.117:0/1532898147 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6224105070 0x7f622419b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:56.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.719+0000 7f6221e00700 1 -- 172.21.15.117:0/1532898147 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6224105070 msgr2=0x7f622419b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:56.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.719+0000 7f6221e00700 1 --2- 172.21.15.117:0/1532898147 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6224105070 0x7f622419b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:56.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.719+0000 7f6221e00700 1 -- 172.21.15.117:0/1532898147 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f622410a6b0 msgr2=0x7f622419f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:56.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.719+0000 7f6221e00700 1 --2- 172.21.15.117:0/1532898147 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f622410a6b0 0x7f622419f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:56.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.719+0000 7f6221e00700 1 -- 172.21.15.117:0/1532898147 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f622419fda0 con 0x7f62241046b0 2024-06-03T03:23:56.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.719+0000 7f6222800700 1 --2- 172.21.15.117:0/1532898147 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f622410a6b0 0x7f622419f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T03:23:56.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.720+0000 7f6221e00700 1 --2- 172.21.15.117:0/1532898147 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f62241046b0 0x7f622419ad90 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f621400e9a0 tx=0x7f621400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:23:56.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.720+0000 7f621aa00700 1 -- 172.21.15.117:0/1532898147 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f621400cbd0 con 0x7f62241046b0 2024-06-03T03:23:56.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.720+0000 7f6228a00700 1 -- 172.21.15.117:0/1532898147 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62241a0090 con 0x7f62241046b0 2024-06-03T03:23:56.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.720+0000 7f621aa00700 1 -- 172.21.15.117:0/1532898147 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f621400cd70 con 0x7f62241046b0 2024-06-03T03:23:56.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.720+0000 7f621aa00700 1 -- 172.21.15.117:0/1532898147 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6214018750 con 0x7f62241046b0 2024-06-03T03:23:56.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.720+0000 7f6228a00700 1 -- 172.21.15.117:0/1532898147 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62241a7910 con 0x7f62241046b0 2024-06-03T03:23:56.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.724+0000 7f621aa00700 1 -- 172.21.15.117:0/1532898147 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f62140188f0 con 0x7f62241046b0 2024-06-03T03:23:56.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.725+0000 7f623340db80 1 Processor -- start 2024-06-03T03:23:56.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.725+0000 7f623340db80 1 -- start start 2024-06-03T03:23:56.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.726+0000 7f623340db80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562186ca9df0 0x562186d2d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:56.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.726+0000 7f623340db80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562186d2d560 0x562186d2d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:56.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.726+0000 7f623340db80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562186d382b0 0x562186d3a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:56.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.726+0000 7f623340db80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562186c3c9c0 con 0x562186d382b0 2024-06-03T03:23:56.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.726+0000 7f623340db80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562186c7fab0 con 0x562186ca9df0 2024-06-03T03:23:56.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.726+0000 7f621aa00700 1 --2- 172.21.15.117:0/1532898147 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f62080a2c20 0x7f62080a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:56.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.726+0000 7f623340db80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562186c3a8a0 con 0x562186d2d560 2024-06-03T03:23:56.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.726+0000 7f621aa00700 1 -- 172.21.15.117:0/1532898147 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f62140cf6b0 con 0x7f62241046b0 2024-06-03T03:23:56.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.726+0000 7f6221e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562186d2d560 0x562186d2d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:56.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.726+0000 7f6221e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562186d2d560 0x562186d2d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:48120/0 (socket says 172.21.15.117:48120) 2024-06-03T03:23:56.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.726+0000 7f6221e00700 1 -- 172.21.15.117:0/3378801388 learned_addr learned my addr 172.21.15.117:0/3378801388 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:23:56.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.726+0000 7f6222800700 1 --2- 172.21.15.117:0/1532898147 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f62080a2c20 0x7f62080a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:56.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.726+0000 7f6222800700 1 --2- 172.21.15.117:0/3378801388 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562186ca9df0 0x562186d2d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:56.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.727+0000 7f6221400700 1 --2- 172.21.15.117:0/3378801388 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562186d382b0 0x562186d3a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:56.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.727+0000 7f6221e00700 1 -- 172.21.15.117:0/3378801388 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562186ca9df0 msgr2=0x562186d2d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:23:56.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.727+0000 7f6221e00700 1 --2- 172.21.15.117:0/3378801388 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562186ca9df0 0x562186d2d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:56.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.727+0000 7f6221e00700 1 -- 172.21.15.117:0/3378801388 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562186d382b0 msgr2=0x562186d3a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:23:56.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.727+0000 7f6221e00700 1 --2- 172.21.15.117:0/3378801388 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562186d382b0 0x562186d3a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:56.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.727+0000 7f6221e00700 1 -- 172.21.15.117:0/3378801388 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562186c27f40 con 0x562186d2d560 2024-06-03T03:23:56.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.727+0000 7f6221400700 1 --2- 172.21.15.117:0/3378801388 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562186d382b0 0x562186d3a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T03:23:56.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.727+0000 7f6221e00700 1 --2- 172.21.15.117:0/3378801388 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562186d2d560 0x562186d2d9e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f62140a4260 tx=0x7f621409b5b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91fa3d2f3fa75f83 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:23:56.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.727+0000 7f6222800700 1 --2- 172.21.15.117:0/1532898147 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f62080a2c20 0x7f62080a50e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x562186d2a360 tx=0x7f621c00b3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:23:56.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.727+0000 7f620f400700 1 -- 172.21.15.117:0/3378801388 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f621409fed0 con 0x562186d2d560 2024-06-03T03:23:56.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.727+0000 7f620f400700 1 -- 172.21.15.117:0/3378801388 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f621409ce50 con 0x562186d2d560 2024-06-03T03:23:56.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.727+0000 7f620f400700 1 -- 172.21.15.117:0/3378801388 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f621409bd80 con 0x562186d2d560 2024-06-03T03:23:56.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.728+0000 7f623340db80 1 -- 172.21.15.117:0/3378801388 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562186d2d560 msgr2=0x562186d2d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:56.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.728+0000 7f623340db80 1 --2- 172.21.15.117:0/3378801388 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562186d2d560 0x562186d2d9e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f62140a4260 tx=0x7f621409b5b0 comp rx=0 tx=0).stop 2024-06-03T03:23:56.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.728+0000 7f623340db80 1 -- 172.21.15.117:0/3378801388 shutdown_connections 2024-06-03T03:23:56.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.728+0000 7f623340db80 1 --2- 172.21.15.117:0/3378801388 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562186d382b0 0x562186d3a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:56.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.728+0000 7f623340db80 1 --2- 172.21.15.117:0/3378801388 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562186ca9df0 0x562186d2d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:56.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.728+0000 7f623340db80 1 --2- 172.21.15.117:0/3378801388 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562186d2d560 0x562186d2d9e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:56.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.728+0000 7f623340db80 1 -- 172.21.15.117:0/3378801388 >> 172.21.15.117:0/3378801388 conn(0x562186c5a120 msgr2=0x562186d29460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:23:56.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.728+0000 7f623340db80 1 -- 172.21.15.117:0/3378801388 shutdown_connections 2024-06-03T03:23:56.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.728+0000 7f623340db80 1 -- 172.21.15.117:0/3378801388 wait complete. 2024-06-03T03:23:56.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.729+0000 7f623340db80 1 -- start start 2024-06-03T03:23:56.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.729+0000 7f623340db80 1 Processor -- start 2024-06-03T03:23:56.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.730+0000 7f623340db80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562186ca9df0 0x562186d37a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:56.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.730+0000 7f623340db80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562186d2d560 0x562186d34d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:56.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.730+0000 7f623340db80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562186d382b0 0x562186d35280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:56.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.730+0000 7f623340db80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562186c1bf10 con 0x562186d382b0 2024-06-03T03:23:56.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.730+0000 7f623340db80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562186c7ba70 con 0x562186ca9df0 2024-06-03T03:23:56.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.730+0000 7f623340db80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562186d3af10 con 0x562186d2d560 2024-06-03T03:23:56.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.730+0000 7f623340db80 10 client.0 _ll_register_callbacks cb 0x562186c196d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:23:56.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.730+0000 7f6222800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562186d382b0 0x562186d35280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:56.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.730+0000 7f6221e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562186ca9df0 0x562186d37a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:56.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.730+0000 7f6221e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562186ca9df0 0x562186d37a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:41508/0 (socket says 172.21.15.117:41508) 2024-06-03T03:23:56.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.730+0000 7f6222800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562186d382b0 0x562186d35280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34906/0 (socket says 172.21.15.117:34906) 2024-06-03T03:23:56.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.730+0000 7f6221e00700 1 -- 172.21.15.117:0/3987259215 learned_addr learned my addr 172.21.15.117:0/3987259215 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:23:56.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.730+0000 7f6221400700 1 --2- 172.21.15.117:0/3987259215 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562186d2d560 0x562186d34d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:56.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.730+0000 7f6221e00700 1 -- 172.21.15.117:0/3987259215 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562186d2d560 msgr2=0x562186d34d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:23:56.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.730+0000 7f6221e00700 1 --2- 172.21.15.117:0/3987259215 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562186d2d560 0x562186d34d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:56.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.730+0000 7f6221e00700 1 -- 172.21.15.117:0/3987259215 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562186d382b0 msgr2=0x562186d35280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:23:56.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.730+0000 7f6221e00700 1 --2- 172.21.15.117:0/3987259215 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562186d382b0 0x562186d35280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:56.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.730+0000 7f6221e00700 1 -- 172.21.15.117:0/3987259215 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562186bce880 con 0x562186ca9df0 2024-06-03T03:23:56.736 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.731+0000 7f6221e00700 1 --2- 172.21.15.117:0/3987259215 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562186ca9df0 0x562186d37a40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f621400e970 tx=0x7f621400c6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46c5cc762ccecb4d server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:23:56.736 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.731+0000 7f6203e00700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-06-03T03:23:56.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.731+0000 7f6203e00700 1 -- 172.21.15.117:0/3987259215 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62140b8070 con 0x562186ca9df0 2024-06-03T03:23:56.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.731+0000 7f623340db80 1 -- 172.21.15.117:0/3987259215 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f621409ce50 con 0x562186ca9df0 2024-06-03T03:23:56.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.731+0000 7f6203e00700 1 -- 172.21.15.117:0/3987259215 <== mon.2 v2:172.21.15.151:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f62140109a0 con 0x562186ca9df0 2024-06-03T03:23:56.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.731+0000 7f6203e00700 1 -- 172.21.15.117:0/3987259215 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f621409fed0 con 0x562186ca9df0 2024-06-03T03:23:56.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.731+0000 7f623340db80 10 client.25004 fetch_fsmap learned FSMap version 30 2024-06-03T03:23:56.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.731+0000 7f623340db80 1 -- 172.21.15.117:0/3987259215 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562186d455d0 con 0x562186ca9df0 2024-06-03T03:23:56.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.732+0000 7f6203e00700 1 -- 172.21.15.117:0/3987259215 <== mon.2 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f621409ce50 con 0x562186ca9df0 2024-06-03T03:23:56.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.732+0000 7f6203e00700 1 -- 172.21.15.117:0/3987259215 <== mon.2 v2:172.21.15.151:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f62140a4300 con 0x562186ca9df0 2024-06-03T03:23:56.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.732+0000 7f623340db80 10 client.25004 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T03:23:56.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.732+0000 7f623340db80 10 client.25004 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-03T03:23:56.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.732+0000 7f623340db80 20 client.25004 populate_metadata read hostname 'smithi117' 2024-06-03T03:23:56.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.732+0000 7f623340db80 1 --2- 172.21.15.117:0/3987259215 >> [v2:172.21.15.151:6838/3623444616,v1:172.21.15.151:6839/3623444616] conn(0x562186d46920 0x562186d48b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:56.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.732+0000 7f623340db80 4 client.25004 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T03:23:56.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.732+0000 7f623340db80 1 -- 172.21.15.117:0/3987259215 --> [v2:172.21.15.151:6838/3623444616,v1:172.21.15.151:6839/3623444616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562186c8cfb0 con 0x562186d46920 2024-06-03T03:23:56.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.733+0000 7f6221400700 1 --2- 172.21.15.117:0/3987259215 >> [v2:172.21.15.151:6838/3623444616,v1:172.21.15.151:6839/3623444616] conn(0x562186d46920 0x562186d48b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:56.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.733+0000 7f6221400700 1 --2- 172.21.15.117:0/3987259215 >> [v2:172.21.15.151:6838/3623444616,v1:172.21.15.151:6839/3623444616] conn(0x562186d46920 0x562186d48b20 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1dbc1b13da3b6fef server_cookie=a591377e2b623b71 in_seq=0 out_seq=0 2024-06-03T03:23:56.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.733+0000 7f6203e00700 10 client.25004 ms_handle_connect on v2:172.21.15.151:6838/3623444616 2024-06-03T03:23:56.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.734+0000 7f6203e00700 1 -- 172.21.15.117:0/3987259215 <== mds.1 v2:172.21.15.151:6838/3623444616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562186c8cfb0 con 0x562186d46920 2024-06-03T03:23:56.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.734+0000 7f6203e00700 10 client.25004 handle_command_reply: tid=0 2024-06-03T03:23:56.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.734+0000 7f623340db80 1 client.25004 shutdown 2024-06-03T03:23:56.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.734+0000 7f623340db80 1 -- 172.21.15.117:0/3987259215 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562186ca9df0 msgr2=0x562186d37a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:56.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.734+0000 7f623340db80 1 --2- 172.21.15.117:0/3987259215 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562186ca9df0 0x562186d37a40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f621400e970 tx=0x7f621400c6e0 comp rx=0 tx=0).stop 2024-06-03T03:23:56.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.734+0000 7f623340db80 1 -- 172.21.15.117:0/3987259215 shutdown_connections 2024-06-03T03:23:56.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.734+0000 7f623340db80 1 --2- 172.21.15.117:0/3987259215 >> [v2:172.21.15.151:6838/3623444616,v1:172.21.15.151:6839/3623444616] conn(0x562186d46920 0x562186d48b20 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:56.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.734+0000 7f623340db80 1 --2- 172.21.15.117:0/3987259215 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562186d382b0 0x562186d35280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:56.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.734+0000 7f623340db80 1 --2- 172.21.15.117:0/3987259215 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562186ca9df0 0x562186d37a40 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:56.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.734+0000 7f6203e00700 0 client.25004 ms_handle_reset on v2:172.21.15.151:6838/3623444616 2024-06-03T03:23:56.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.734+0000 7f623340db80 1 --2- 172.21.15.117:0/3987259215 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562186d2d560 0x562186d34d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:56.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.734+0000 7f623340db80 1 -- 172.21.15.117:0/3987259215 >> 172.21.15.117:0/3987259215 conn(0x562186c5a120 msgr2=0x562186d286f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:23:56.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.735+0000 7f623340db80 1 -- 172.21.15.117:0/3987259215 shutdown_connections 2024-06-03T03:23:56.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.735+0000 7f623340db80 1 -- 172.21.15.117:0/3987259215 wait complete. 2024-06-03T03:23:56.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.735+0000 7f623340db80 20 client.25004 trim_cache size 0 max 16384 2024-06-03T03:23:56.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.757+0000 7f623340db80 1 Processor -- start 2024-06-03T03:23:56.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.758+0000 7f623340db80 1 -- start start 2024-06-03T03:23:56.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.758+0000 7f623340db80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562186ca9df0 0x562186d37580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:56.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.758+0000 7f623340db80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562186d2d560 0x562186d396a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:56.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.758+0000 7f623340db80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562186d382b0 0x562186d39be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:56.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.758+0000 7f623340db80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562186d46e60 con 0x562186d382b0 2024-06-03T03:23:56.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.758+0000 7f623340db80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562186d47530 con 0x562186d2d560 2024-06-03T03:23:56.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.758+0000 7f623340db80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562186d47c00 con 0x562186ca9df0 2024-06-03T03:23:56.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.758+0000 7f6222800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562186d382b0 0x562186d39be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:56.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.758+0000 7f6221400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562186d2d560 0x562186d396a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:56.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.758+0000 7f6221400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562186d2d560 0x562186d396a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:41516/0 (socket says 172.21.15.117:41516) 2024-06-03T03:23:56.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.758+0000 7f6221400700 1 -- 172.21.15.117:0/1782961010 learned_addr learned my addr 172.21.15.117:0/1782961010 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:23:56.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.759+0000 7f6221e00700 1 --2- 172.21.15.117:0/1782961010 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562186ca9df0 0x562186d37580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:56.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.759+0000 7f6221400700 1 -- 172.21.15.117:0/1782961010 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562186ca9df0 msgr2=0x562186d37580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:23:56.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.759+0000 7f6221400700 1 --2- 172.21.15.117:0/1782961010 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562186ca9df0 0x562186d37580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:56.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.759+0000 7f6221400700 1 -- 172.21.15.117:0/1782961010 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562186d382b0 msgr2=0x562186d39be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:23:56.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.759+0000 7f6221400700 1 --2- 172.21.15.117:0/1782961010 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562186d382b0 0x562186d39be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:56.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.759+0000 7f6221400700 1 -- 172.21.15.117:0/1782961010 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562186d20dc0 con 0x562186d2d560 2024-06-03T03:23:56.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.759+0000 7f6221400700 1 --2- 172.21.15.117:0/1782961010 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562186d2d560 0x562186d396a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6210004790 tx=0x7f6210004320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11025a941f083de1 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:23:56.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.759+0000 7f620ea00700 1 -- 172.21.15.117:0/1782961010 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62100039a0 con 0x562186d2d560 2024-06-03T03:23:56.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.759+0000 7f620ea00700 1 -- 172.21.15.117:0/1782961010 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6210003b40 con 0x562186d2d560 2024-06-03T03:23:56.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.759+0000 7f620ea00700 1 -- 172.21.15.117:0/1782961010 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6210012690 con 0x562186d2d560 2024-06-03T03:23:56.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.760+0000 7f623340db80 1 -- 172.21.15.117:0/1782961010 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562186d2d560 msgr2=0x562186d396a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:56.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.760+0000 7f623340db80 1 --2- 172.21.15.117:0/1782961010 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562186d2d560 0x562186d396a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6210004790 tx=0x7f6210004320 comp rx=0 tx=0).stop 2024-06-03T03:23:56.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.760+0000 7f623340db80 1 -- 172.21.15.117:0/1782961010 shutdown_connections 2024-06-03T03:23:56.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.760+0000 7f623340db80 1 --2- 172.21.15.117:0/1782961010 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562186d382b0 0x562186d39be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:56.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.760+0000 7f623340db80 1 --2- 172.21.15.117:0/1782961010 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562186ca9df0 0x562186d37580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:56.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.760+0000 7f623340db80 1 --2- 172.21.15.117:0/1782961010 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562186d2d560 0x562186d396a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:56.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.760+0000 7f623340db80 1 -- 172.21.15.117:0/1782961010 >> 172.21.15.117:0/1782961010 conn(0x562186c5a120 msgr2=0x562186d33d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:23:56.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.760+0000 7f623340db80 1 -- 172.21.15.117:0/1782961010 shutdown_connections 2024-06-03T03:23:56.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.760+0000 7f623340db80 1 -- 172.21.15.117:0/1782961010 wait complete. 2024-06-03T03:23:56.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.761+0000 7f623340db80 1 -- start start 2024-06-03T03:23:56.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.761+0000 7f623340db80 1 Processor -- start 2024-06-03T03:23:56.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.762+0000 7f623340db80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562186ca9df0 0x562186d28ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:56.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.762+0000 7f623340db80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562186d2d560 0x562186d29020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:56.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.762+0000 7f623340db80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562186d382b0 0x562186d29560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:56.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.762+0000 7f623340db80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562186d489a0 con 0x562186d382b0 2024-06-03T03:23:56.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.762+0000 7f623340db80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562186d482d0 con 0x562186ca9df0 2024-06-03T03:23:56.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.762+0000 7f623340db80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562186d49070 con 0x562186d2d560 2024-06-03T03:23:56.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.762+0000 7f623340db80 10 client.0 _ll_register_callbacks cb 0x562186c196d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:23:56.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.762+0000 7f6221e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562186ca9df0 0x562186d28ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:56.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.762+0000 7f6221e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562186ca9df0 0x562186d28ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:41520/0 (socket says 172.21.15.117:41520) 2024-06-03T03:23:56.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.762+0000 7f6221e00700 1 -- 172.21.15.117:0/3252809794 learned_addr learned my addr 172.21.15.117:0/3252809794 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:23:56.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.762+0000 7f6222800700 1 --2- 172.21.15.117:0/3252809794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562186d382b0 0x562186d29560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:56.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.762+0000 7f6221400700 1 --2- 172.21.15.117:0/3252809794 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562186d2d560 0x562186d29020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:56.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.762+0000 7f6221e00700 1 -- 172.21.15.117:0/3252809794 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562186d2d560 msgr2=0x562186d29020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:23:56.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.762+0000 7f6221e00700 1 --2- 172.21.15.117:0/3252809794 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562186d2d560 0x562186d29020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:56.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.762+0000 7f6221e00700 1 -- 172.21.15.117:0/3252809794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562186d382b0 msgr2=0x562186d29560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:23:56.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.762+0000 7f6221e00700 1 --2- 172.21.15.117:0/3252809794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562186d382b0 0x562186d29560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:56.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.762+0000 7f6221e00700 1 -- 172.21.15.117:0/3252809794 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562186d4e850 con 0x562186ca9df0 2024-06-03T03:23:56.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.762+0000 7f6221400700 1 --2- 172.21.15.117:0/3252809794 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562186d2d560 0x562186d29020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T03:23:56.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.762+0000 7f6222800700 1 --2- 172.21.15.117:0/3252809794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562186d382b0 0x562186d29560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T03:23:56.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.763+0000 7f6221e00700 1 --2- 172.21.15.117:0/3252809794 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562186ca9df0 0x562186d28ae0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f621409b820 tx=0x7f62140cce40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5631f352f9f52e01 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:23:56.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.763+0000 7f6203e00700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-06-03T03:23:56.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.763+0000 7f6203e00700 1 -- 172.21.15.117:0/3252809794 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62140b8070 con 0x562186ca9df0 2024-06-03T03:23:56.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.763+0000 7f623340db80 1 -- 172.21.15.117:0/3252809794 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6210003b40 con 0x562186ca9df0 2024-06-03T03:23:56.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.763+0000 7f6203e00700 1 -- 172.21.15.117:0/3252809794 <== mon.2 v2:172.21.15.151:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f62140ba310 con 0x562186ca9df0 2024-06-03T03:23:56.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.763+0000 7f6203e00700 1 -- 172.21.15.117:0/3252809794 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6214097870 con 0x562186ca9df0 2024-06-03T03:23:56.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.763+0000 7f623340db80 10 client.25010 fetch_fsmap learned FSMap version 30 2024-06-03T03:23:56.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.763+0000 7f623340db80 1 -- 172.21.15.117:0/3252809794 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562186d4e850 con 0x562186ca9df0 2024-06-03T03:23:56.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.763+0000 7f6203e00700 1 -- 172.21.15.117:0/3252809794 <== mon.2 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6210003b40 con 0x562186ca9df0 2024-06-03T03:23:56.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.763+0000 7f6203e00700 1 -- 172.21.15.117:0/3252809794 <== mon.2 v2:172.21.15.151:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f6214097a10 con 0x562186ca9df0 2024-06-03T03:23:56.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.764+0000 7f623340db80 10 client.25010 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T03:23:56.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.764+0000 7f623340db80 10 client.25010 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-03T03:23:56.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.764+0000 7f623340db80 20 client.25010 populate_metadata read hostname 'smithi117' 2024-06-03T03:23:56.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.764+0000 7f623340db80 1 --2- 172.21.15.117:0/3252809794 >> [v2:172.21.15.151:6838/3623444616,v1:172.21.15.151:6839/3623444616] conn(0x562186dc03e0 0x562186dbf0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:23:56.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.764+0000 7f623340db80 4 client.25010 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-03T03:23:56.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.764+0000 7f623340db80 1 -- 172.21.15.117:0/3252809794 --> [v2:172.21.15.151:6838/3623444616,v1:172.21.15.151:6839/3623444616] -- command(tid 0: {"prefix": "status"}) -- 0x562186c87530 con 0x562186dc03e0 2024-06-03T03:23:56.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.764+0000 7f6221400700 1 --2- 172.21.15.117:0/3252809794 >> [v2:172.21.15.151:6838/3623444616,v1:172.21.15.151:6839/3623444616] conn(0x562186dc03e0 0x562186dbf0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:23:56.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.765+0000 7f6221400700 1 --2- 172.21.15.117:0/3252809794 >> [v2:172.21.15.151:6838/3623444616,v1:172.21.15.151:6839/3623444616] conn(0x562186dc03e0 0x562186dbf0f0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7cdc3a21c969fc26 server_cookie=946be4011cdecf13 in_seq=0 out_seq=0 2024-06-03T03:23:56.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.765+0000 7f6203e00700 10 client.25010 ms_handle_connect on v2:172.21.15.151:6838/3623444616 2024-06-03T03:23:56.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.765+0000 7f6203e00700 1 -- 172.21.15.117:0/3252809794 <== mds.1 v2:172.21.15.151:6838/3623444616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x562186c87530 con 0x562186dc03e0 2024-06-03T03:23:56.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.765+0000 7f6203e00700 10 client.25010 handle_command_reply: tid=0 2024-06-03T03:23:56.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.765+0000 7f623340db80 1 client.25010 shutdown 2024-06-03T03:23:56.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.765+0000 7f623340db80 1 -- 172.21.15.117:0/3252809794 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562186ca9df0 msgr2=0x562186d28ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:56.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.765+0000 7f623340db80 1 --2- 172.21.15.117:0/3252809794 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562186ca9df0 0x562186d28ae0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f621409b820 tx=0x7f62140cce40 comp rx=0 tx=0).stop 2024-06-03T03:23:56.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.765+0000 7f623340db80 1 -- 172.21.15.117:0/3252809794 shutdown_connections 2024-06-03T03:23:56.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.765+0000 7f623340db80 1 --2- 172.21.15.117:0/3252809794 >> [v2:172.21.15.151:6838/3623444616,v1:172.21.15.151:6839/3623444616] conn(0x562186dc03e0 0x562186dbf0f0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:56.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.765+0000 7f623340db80 1 --2- 172.21.15.117:0/3252809794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562186d382b0 0x562186d29560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:56.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.765+0000 7f623340db80 1 --2- 172.21.15.117:0/3252809794 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562186ca9df0 0x562186d28ae0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:56.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.765+0000 7f623340db80 1 --2- 172.21.15.117:0/3252809794 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562186d2d560 0x562186d29020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:56.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.765+0000 7f6203e00700 0 client.25010 ms_handle_reset on v2:172.21.15.151:6838/3623444616 2024-06-03T03:23:56.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.765+0000 7f623340db80 1 -- 172.21.15.117:0/3252809794 >> 172.21.15.117:0/3252809794 conn(0x562186c5a120 msgr2=0x562186d2ef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:23:56.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.765+0000 7f623340db80 1 -- 172.21.15.117:0/3252809794 shutdown_connections 2024-06-03T03:23:56.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.765+0000 7f623340db80 1 -- 172.21.15.117:0/3252809794 wait complete. 2024-06-03T03:23:56.771 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.765+0000 7f623340db80 20 client.25010 trim_cache size 0 max 16384 2024-06-03T03:23:56.771 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-06-03T03:23:56.771 INFO:teuthology.orchestra.run.smithi117.stdout: "cluster_fsid": "a0e8473a-2156-11ef-bc9b-c7b262605968", 2024-06-03T03:23:56.771 INFO:teuthology.orchestra.run.smithi117.stdout: "whoami": 1, 2024-06-03T03:23:56.771 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 24512, 2024-06-03T03:23:56.771 INFO:teuthology.orchestra.run.smithi117.stdout: "want_state": "up:active", 2024-06-03T03:23:56.771 INFO:teuthology.orchestra.run.smithi117.stdout: "state": "up:active", 2024-06-03T03:23:56.771 INFO:teuthology.orchestra.run.smithi117.stdout: "fs_name": "cephfs", 2024-06-03T03:23:56.771 INFO:teuthology.orchestra.run.smithi117.stdout: "rank_uptime": 173.788975153, 2024-06-03T03:23:56.771 INFO:teuthology.orchestra.run.smithi117.stdout: "mdsmap_epoch": 30, 2024-06-03T03:23:56.772 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_epoch": 99, 2024-06-03T03:23:56.772 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_epoch_barrier": 97, 2024-06-03T03:23:56.772 INFO:teuthology.orchestra.run.smithi117.stdout: "uptime": 271.93805362199998 2024-06-03T03:23:56.772 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-06-03T03:23:56.772 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.766+0000 7f6228a00700 1 -- 172.21.15.117:0/1532898147 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f62080a2c20 msgr2=0x7f62080a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:56.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.766+0000 7f6228a00700 1 --2- 172.21.15.117:0/1532898147 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f62080a2c20 0x7f62080a50e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x562186d2a360 tx=0x7f621c00b3b0 comp rx=0 tx=0).stop 2024-06-03T03:23:56.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.766+0000 7f6228a00700 1 -- 172.21.15.117:0/1532898147 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f62241046b0 msgr2=0x7f622419ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:23:56.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.766+0000 7f6228a00700 1 --2- 172.21.15.117:0/1532898147 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f62241046b0 0x7f622419ad90 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f621400e9a0 tx=0x7f621400ecb0 comp rx=0 tx=0).stop 2024-06-03T03:23:56.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.766+0000 7f6228a00700 1 -- 172.21.15.117:0/1532898147 shutdown_connections 2024-06-03T03:23:56.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.766+0000 7f6228a00700 1 --2- 172.21.15.117:0/1532898147 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f62080a2c20 0x7f62080a50e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:56.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.766+0000 7f6228a00700 1 --2- 172.21.15.117:0/1532898147 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f622410a6b0 0x7f622419f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:56.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.766+0000 7f6228a00700 1 --2- 172.21.15.117:0/1532898147 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f62241046b0 0x7f622419ad90 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:56.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.766+0000 7f6228a00700 1 --2- 172.21.15.117:0/1532898147 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6224105070 0x7f622419b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:23:56.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.766+0000 7f6228a00700 1 -- 172.21.15.117:0/1532898147 >> 172.21.15.117:0/1532898147 conn(0x7f62241001a0 msgr2=0x7f6224100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:23:56.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.766+0000 7f6228a00700 1 -- 172.21.15.117:0/1532898147 shutdown_connections 2024-06-03T03:23:56.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:23:56.767+0000 7f6228a00700 1 -- 172.21.15.117:0/1532898147 wait complete. 2024-06-03T03:23:57.866 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph tell mds.i status 2024-06-03T03:23:58.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:23:58 smithi117 ceph-mon[31649]: pgmap v465: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-03T03:23:58.315 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:23:58.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:23:58 smithi151 ceph-mon[37043]: pgmap v465: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-03T03:23:58.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:23:58 smithi183 ceph-mon[37326]: pgmap v465: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-03T03:24:00.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:24:00 smithi117 ceph-mon[31649]: pgmap v466: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-03T03:24:00.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:24:00 smithi151 ceph-mon[37043]: pgmap v466: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-03T03:24:00.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:24:00 smithi183 ceph-mon[37326]: pgmap v466: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-03T03:24:00.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.799+0000 7f085ae00700 1 -- 172.21.15.117:0/596919953 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0854102eb0 msgr2=0x7f0854103330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:00.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.799+0000 7f085ae00700 1 --2- 172.21.15.117:0/596919953 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0854102eb0 0x7f0854103330 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f08400099c0 tx=0x7f0840009cd0 comp rx=0 tx=0).stop 2024-06-03T03:24:00.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.799+0000 7f085ae00700 1 -- 172.21.15.117:0/596919953 shutdown_connections 2024-06-03T03:24:00.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.799+0000 7f085ae00700 1 --2- 172.21.15.117:0/596919953 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0854103870 0x7f085410e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:00.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.799+0000 7f085ae00700 1 --2- 172.21.15.117:0/596919953 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f08541015c0 0x7f08541019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:00.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.799+0000 7f085ae00700 1 --2- 172.21.15.117:0/596919953 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0854102eb0 0x7f0854103330 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:00.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.799+0000 7f085ae00700 1 -- 172.21.15.117:0/596919953 >> 172.21.15.117:0/596919953 conn(0x7f08540fcd70 msgr2=0x7f08540ff190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:24:00.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.799+0000 7f085ae00700 1 -- 172.21.15.117:0/596919953 shutdown_connections 2024-06-03T03:24:00.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.800+0000 7f085ae00700 1 -- 172.21.15.117:0/596919953 wait complete. 2024-06-03T03:24:00.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.801+0000 7f085ae00700 1 Processor -- start 2024-06-03T03:24:00.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.801+0000 7f085ae00700 1 -- start start 2024-06-03T03:24:00.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.802+0000 7f085ae00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f08541015c0 0x7f08540fd220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:00.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.802+0000 7f085ae00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0854102eb0 0x7f08540fd760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:00.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.802+0000 7f085ae00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0854103870 0x7f08540fdca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:00.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.802+0000 7f085ae00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f08541116f0 con 0x7f0854102eb0 2024-06-03T03:24:00.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.802+0000 7f085ae00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0854111570 con 0x7f0854103870 2024-06-03T03:24:00.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.802+0000 7f085ae00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0854111870 con 0x7f08541015c0 2024-06-03T03:24:00.804 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.802+0000 7f0859000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0854102eb0 0x7f08540fd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:00.804 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.802+0000 7f085a400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0854103870 0x7f08540fdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:00.804 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.802+0000 7f0859000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0854102eb0 0x7f08540fd760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:57620/0 (socket says 172.21.15.117:57620) 2024-06-03T03:24:00.805 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.802+0000 7f0859000700 1 -- 172.21.15.117:0/3555382027 learned_addr learned my addr 172.21.15.117:0/3555382027 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:24:00.805 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.802+0000 7f0859a00700 1 --2- 172.21.15.117:0/3555382027 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f08541015c0 0x7f08540fd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:00.805 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.802+0000 7f085a400700 1 -- 172.21.15.117:0/3555382027 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f08541015c0 msgr2=0x7f08540fd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:00.805 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.802+0000 7f085a400700 1 --2- 172.21.15.117:0/3555382027 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f08541015c0 0x7f08540fd220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:00.805 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.802+0000 7f085a400700 1 -- 172.21.15.117:0/3555382027 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0854102eb0 msgr2=0x7f08540fd760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:00.805 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.802+0000 7f085a400700 1 --2- 172.21.15.117:0/3555382027 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0854102eb0 0x7f08540fd760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:00.805 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.803+0000 7f085a400700 1 -- 172.21.15.117:0/3555382027 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08541a7110 con 0x7f0854103870 2024-06-03T03:24:00.805 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.803+0000 7f085a400700 1 --2- 172.21.15.117:0/3555382027 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0854103870 0x7f08540fdca0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f084c00efc0 tx=0x7f084c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:24:00.806 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.803+0000 7f0852000700 1 -- 172.21.15.117:0/3555382027 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084c0092e0 con 0x7f0854103870 2024-06-03T03:24:00.806 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.803+0000 7f085ae00700 1 -- 172.21.15.117:0/3555382027 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08541a73a0 con 0x7f0854103870 2024-06-03T03:24:00.806 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.803+0000 7f0852000700 1 -- 172.21.15.117:0/3555382027 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f084c004bb0 con 0x7f0854103870 2024-06-03T03:24:00.806 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.803+0000 7f0852000700 1 -- 172.21.15.117:0/3555382027 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084c004280 con 0x7f0854103870 2024-06-03T03:24:00.806 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.803+0000 7f085ae00700 1 -- 172.21.15.117:0/3555382027 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08541a78e0 con 0x7f0854103870 2024-06-03T03:24:00.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.808+0000 7f0852000700 1 -- 172.21.15.117:0/3555382027 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f084c01b030 con 0x7f0854103870 2024-06-03T03:24:00.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.809+0000 7f0866cd8b80 1 Processor -- start 2024-06-03T03:24:00.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.809+0000 7f0866cd8b80 1 -- start start 2024-06-03T03:24:00.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.809+0000 7f0866cd8b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a62e693df0 0x55a62e7193d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:00.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.809+0000 7f0852000700 1 --2- 172.21.15.117:0/3555382027 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f083c0a2c20 0x7f083c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:00.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.809+0000 7f0866cd8b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a62e719910 0x55a62e719d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:00.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.809+0000 7f0866cd8b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a62e722520 0x55a62e724910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:00.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.809+0000 7f0866cd8b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a62e6269c0 con 0x55a62e719910 2024-06-03T03:24:00.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.809+0000 7f0866cd8b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a62e669ab0 con 0x55a62e722520 2024-06-03T03:24:00.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.809+0000 7f0866cd8b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a62e6248a0 con 0x55a62e693df0 2024-06-03T03:24:00.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.810+0000 7f0852000700 1 -- 172.21.15.117:0/3555382027 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f084c0ceea0 con 0x7f0854103870 2024-06-03T03:24:00.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.810+0000 7f0859a00700 1 --2- 172.21.15.117:0/3555382027 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f083c0a2c20 0x7f083c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:00.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.810+0000 7f0859000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a62e693df0 0x55a62e7193d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:00.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.810+0000 7f085a400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a62e722520 0x55a62e724910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:00.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.810+0000 7f0859000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a62e719910 0x55a62e719d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:00.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.810+0000 7f085a400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a62e722520 0x55a62e724910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:41866/0 (socket says 172.21.15.117:41866) 2024-06-03T03:24:00.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.810+0000 7f085a400700 1 -- 172.21.15.117:0/206629814 learned_addr learned my addr 172.21.15.117:0/206629814 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:24:00.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.810+0000 7f085a400700 1 -- 172.21.15.117:0/206629814 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a62e693df0 msgr2=0x55a62e7193d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:24:00.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.810+0000 7f085a400700 1 --2- 172.21.15.117:0/206629814 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a62e693df0 0x55a62e7193d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:00.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.810+0000 7f085a400700 1 -- 172.21.15.117:0/206629814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a62e719910 msgr2=0x55a62e719d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:24:00.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.810+0000 7f085a400700 1 --2- 172.21.15.117:0/206629814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a62e719910 0x55a62e719d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:00.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.810+0000 7f085a400700 1 -- 172.21.15.117:0/206629814 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a62e611f40 con 0x55a62e722520 2024-06-03T03:24:00.814 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.810+0000 7f0859000700 1 --2- 172.21.15.117:0/206629814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a62e719910 0x55a62e719d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:24:00.814 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.811+0000 7f0859a00700 1 --2- 172.21.15.117:0/3555382027 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f083c0a2c20 0x7f083c0a50e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0848009c60 tx=0x7f0848009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:24:00.814 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.811+0000 7f085a400700 1 --2- 172.21.15.117:0/206629814 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a62e722520 0x55a62e724910 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f084c097d70 tx=0x7f084c00cad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0ff1108af760f4b server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:24:00.814 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.811+0000 7f0846a00700 1 -- 172.21.15.117:0/206629814 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084c09aee0 con 0x55a62e722520 2024-06-03T03:24:00.814 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.811+0000 7f0846a00700 1 -- 172.21.15.117:0/206629814 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f084c09baa0 con 0x55a62e722520 2024-06-03T03:24:00.814 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.811+0000 7f0846a00700 1 -- 172.21.15.117:0/206629814 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084c004d60 con 0x55a62e722520 2024-06-03T03:24:00.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.812+0000 7f0866cd8b80 1 -- 172.21.15.117:0/206629814 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a62e722520 msgr2=0x55a62e724910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:00.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.812+0000 7f0866cd8b80 1 --2- 172.21.15.117:0/206629814 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a62e722520 0x55a62e724910 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f084c097d70 tx=0x7f084c00cad0 comp rx=0 tx=0).stop 2024-06-03T03:24:00.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.812+0000 7f0866cd8b80 1 -- 172.21.15.117:0/206629814 shutdown_connections 2024-06-03T03:24:00.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.812+0000 7f0866cd8b80 1 --2- 172.21.15.117:0/206629814 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a62e722520 0x55a62e724910 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:00.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.812+0000 7f0866cd8b80 1 --2- 172.21.15.117:0/206629814 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a62e693df0 0x55a62e7193d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:00.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.812+0000 7f0866cd8b80 1 --2- 172.21.15.117:0/206629814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a62e719910 0x55a62e719d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:00.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.812+0000 7f0866cd8b80 1 -- 172.21.15.117:0/206629814 >> 172.21.15.117:0/206629814 conn(0x55a62e644120 msgr2=0x55a62e7137d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:24:00.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.812+0000 7f0866cd8b80 1 -- 172.21.15.117:0/206629814 shutdown_connections 2024-06-03T03:24:00.816 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.812+0000 7f0866cd8b80 1 -- 172.21.15.117:0/206629814 wait complete. 2024-06-03T03:24:00.816 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.813+0000 7f0866cd8b80 1 -- start start 2024-06-03T03:24:00.816 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.813+0000 7f0866cd8b80 1 Processor -- start 2024-06-03T03:24:00.817 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.814+0000 7f0866cd8b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a62e693df0 0x55a62e721d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:00.817 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.814+0000 7f0866cd8b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a62e719910 0x55a62e71f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:00.817 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.814+0000 7f0866cd8b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a62e722520 0x55a62e71f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:00.817 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.814+0000 7f0866cd8b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a62e605f10 con 0x55a62e693df0 2024-06-03T03:24:00.817 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.814+0000 7f0866cd8b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a62e665a70 con 0x55a62e722520 2024-06-03T03:24:00.817 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.814+0000 7f0866cd8b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a62e7250d0 con 0x55a62e719910 2024-06-03T03:24:00.817 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.814+0000 7f0866cd8b80 10 client.0 _ll_register_callbacks cb 0x55a62e6036d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:24:00.818 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.814+0000 7f0859a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a62e693df0 0x55a62e721d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:00.818 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.814+0000 7f0859a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a62e693df0 0x55a62e721d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:57642/0 (socket says 172.21.15.117:57642) 2024-06-03T03:24:00.818 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.814+0000 7f0859a00700 1 -- 172.21.15.117:0/2919934803 learned_addr learned my addr 172.21.15.117:0/2919934803 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:24:00.818 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.814+0000 7f085a400700 1 --2- 172.21.15.117:0/2919934803 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a62e722520 0x55a62e71f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:00.818 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.814+0000 7f0859000700 1 --2- 172.21.15.117:0/2919934803 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a62e719910 0x55a62e71f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:00.818 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.815+0000 7f0859a00700 1 -- 172.21.15.117:0/2919934803 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a62e719910 msgr2=0x55a62e71f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:24:00.818 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.815+0000 7f0859a00700 1 --2- 172.21.15.117:0/2919934803 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a62e719910 0x55a62e71f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:00.818 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.815+0000 7f0859a00700 1 -- 172.21.15.117:0/2919934803 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a62e722520 msgr2=0x55a62e71f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:24:00.818 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.815+0000 7f0859a00700 1 --2- 172.21.15.117:0/2919934803 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a62e722520 0x55a62e71f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:00.818 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.815+0000 7f0859a00700 1 -- 172.21.15.117:0/2919934803 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a62e5b8880 con 0x55a62e693df0 2024-06-03T03:24:00.818 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.815+0000 7f0859000700 1 --2- 172.21.15.117:0/2919934803 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a62e719910 0x55a62e71f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T03:24:00.818 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.815+0000 7f0859a00700 1 --2- 172.21.15.117:0/2919934803 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a62e693df0 0x55a62e721d20 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f084800eb10 tx=0x7f08480047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cfdf1e3d430fde4 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:24:00.818 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.815+0000 7f083b400700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-06-03T03:24:00.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.816+0000 7f083b400700 1 -- 172.21.15.117:0/2919934803 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0848004220 con 0x55a62e693df0 2024-06-03T03:24:00.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.816+0000 7f083b400700 1 -- 172.21.15.117:0/2919934803 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f08480043c0 con 0x55a62e693df0 2024-06-03T03:24:00.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.816+0000 7f0866cd8b80 1 -- 172.21.15.117:0/2919934803 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f084c09baa0 con 0x55a62e693df0 2024-06-03T03:24:00.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.816+0000 7f083b400700 1 -- 172.21.15.117:0/2919934803 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0848030050 con 0x55a62e693df0 2024-06-03T03:24:00.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.816+0000 7f0866cd8b80 10 client.14907 fetch_fsmap learned FSMap version 30 2024-06-03T03:24:00.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.816+0000 7f0866cd8b80 1 -- 172.21.15.117:0/2919934803 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a62e72f8e0 con 0x55a62e693df0 2024-06-03T03:24:00.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.816+0000 7f083b400700 1 -- 172.21.15.117:0/2919934803 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f084c09baa0 con 0x55a62e693df0 2024-06-03T03:24:00.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.817+0000 7f083b400700 1 -- 172.21.15.117:0/2919934803 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f0848027430 con 0x55a62e693df0 2024-06-03T03:24:00.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.817+0000 7f0866cd8b80 10 client.14907 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T03:24:00.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.817+0000 7f0866cd8b80 10 client.14907 resolve_mds: resolved name 'i' to daemon mds.i 2024-06-03T03:24:00.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.817+0000 7f0866cd8b80 20 client.14907 populate_metadata read hostname 'smithi117' 2024-06-03T03:24:00.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.817+0000 7f0866cd8b80 1 --2- 172.21.15.117:0/2919934803 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55a62e731ef0 0x55a62e733360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:00.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.817+0000 7f0866cd8b80 4 client.14907 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T03:24:00.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.817+0000 7f0866cd8b80 1 -- 172.21.15.117:0/2919934803 --> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a62e676fb0 con 0x55a62e731ef0 2024-06-03T03:24:00.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.817+0000 7f0859000700 1 --2- 172.21.15.117:0/2919934803 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55a62e731ef0 0x55a62e733360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:00.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.818+0000 7f0859000700 1 --2- 172.21.15.117:0/2919934803 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55a62e731ef0 0x55a62e733360 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56e54ec27597ab74 server_cookie=f7c595803aa1c8ef in_seq=0 out_seq=0 2024-06-03T03:24:00.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.818+0000 7f083b400700 10 client.14907 ms_handle_connect on v2:172.21.15.183:6838/1439615083 2024-06-03T03:24:00.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.819+0000 7f083b400700 1 -- 172.21.15.117:0/2919934803 <== mds.0 v2:172.21.15.183:6838/1439615083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a62e676fb0 con 0x55a62e731ef0 2024-06-03T03:24:00.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.819+0000 7f083b400700 10 client.14907 handle_command_reply: tid=0 2024-06-03T03:24:00.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.819+0000 7f0866cd8b80 1 client.14907 shutdown 2024-06-03T03:24:00.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.819+0000 7f0866cd8b80 1 -- 172.21.15.117:0/2919934803 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a62e693df0 msgr2=0x55a62e721d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:00.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.819+0000 7f0866cd8b80 1 --2- 172.21.15.117:0/2919934803 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a62e693df0 0x55a62e721d20 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f084800eb10 tx=0x7f08480047c0 comp rx=0 tx=0).stop 2024-06-03T03:24:00.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.819+0000 7f0866cd8b80 1 -- 172.21.15.117:0/2919934803 shutdown_connections 2024-06-03T03:24:00.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.819+0000 7f0866cd8b80 1 --2- 172.21.15.117:0/2919934803 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55a62e731ef0 0x55a62e733360 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:00.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.819+0000 7f0866cd8b80 1 --2- 172.21.15.117:0/2919934803 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a62e722520 0x55a62e71f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:00.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.819+0000 7f0866cd8b80 1 --2- 172.21.15.117:0/2919934803 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a62e693df0 0x55a62e721d20 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:00.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.819+0000 7f0866cd8b80 1 --2- 172.21.15.117:0/2919934803 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a62e719910 0x55a62e71f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:00.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.819+0000 7f0866cd8b80 1 -- 172.21.15.117:0/2919934803 >> 172.21.15.117:0/2919934803 conn(0x55a62e644120 msgr2=0x55a62e7133c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:24:00.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.819+0000 7f083b400700 0 client.14907 ms_handle_reset on v2:172.21.15.183:6838/1439615083 2024-06-03T03:24:00.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.819+0000 7f0866cd8b80 1 -- 172.21.15.117:0/2919934803 shutdown_connections 2024-06-03T03:24:00.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.819+0000 7f0866cd8b80 1 -- 172.21.15.117:0/2919934803 wait complete. 2024-06-03T03:24:00.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.819+0000 7f0866cd8b80 20 client.14907 trim_cache size 0 max 16384 2024-06-03T03:24:00.845 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.843+0000 7f0866cd8b80 1 Processor -- start 2024-06-03T03:24:00.845 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.843+0000 7f0866cd8b80 1 -- start start 2024-06-03T03:24:00.845 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.843+0000 7f0866cd8b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a62e693df0 0x55a62e71d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:00.845 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.843+0000 7f0866cd8b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a62e719910 0x55a62e71de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:00.845 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.843+0000 7f0866cd8b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a62e722520 0x55a62e71c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:00.845 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.843+0000 7f0866cd8b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a62e7316a0 con 0x55a62e693df0 2024-06-03T03:24:00.845 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.843+0000 7f0866cd8b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a62e731d70 con 0x55a62e719910 2024-06-03T03:24:00.845 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.843+0000 7f0866cd8b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a62e732440 con 0x55a62e722520 2024-06-03T03:24:00.846 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.844+0000 7f0859a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a62e693df0 0x55a62e71d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:00.846 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.844+0000 7f085a400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a62e722520 0x55a62e71c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:00.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.844+0000 7f0859a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a62e693df0 0x55a62e71d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:57644/0 (socket says 172.21.15.117:57644) 2024-06-03T03:24:00.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.844+0000 7f0859000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a62e719910 0x55a62e71de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:00.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.844+0000 7f0859a00700 1 -- 172.21.15.117:0/210657903 learned_addr learned my addr 172.21.15.117:0/210657903 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:24:00.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.844+0000 7f085a400700 1 -- 172.21.15.117:0/210657903 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a62e719910 msgr2=0x55a62e71de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:24:00.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.844+0000 7f085a400700 1 --2- 172.21.15.117:0/210657903 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a62e719910 0x55a62e71de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:00.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.844+0000 7f085a400700 1 -- 172.21.15.117:0/210657903 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a62e693df0 msgr2=0x55a62e71d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:24:00.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.844+0000 7f085a400700 1 --2- 172.21.15.117:0/210657903 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a62e693df0 0x55a62e71d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:00.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.844+0000 7f085a400700 1 -- 172.21.15.117:0/210657903 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a62e718e30 con 0x55a62e722520 2024-06-03T03:24:00.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.844+0000 7f0859000700 1 --2- 172.21.15.117:0/210657903 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a62e719910 0x55a62e71de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:24:00.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.844+0000 7f0859a00700 1 --2- 172.21.15.117:0/210657903 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a62e693df0 0x55a62e71d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:24:00.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.845+0000 7f085a400700 1 --2- 172.21.15.117:0/210657903 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a62e722520 0x55a62e71c2c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f084c09b330 tx=0x7f084c064040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eee2454f56e37ea7 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:24:00.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.845+0000 7f0846000700 1 -- 172.21.15.117:0/210657903 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084c0635b0 con 0x55a62e722520 2024-06-03T03:24:00.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.845+0000 7f0846000700 1 -- 172.21.15.117:0/210657903 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f084c09baa0 con 0x55a62e722520 2024-06-03T03:24:00.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.845+0000 7f0846000700 1 -- 172.21.15.117:0/210657903 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084c064120 con 0x55a62e722520 2024-06-03T03:24:00.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.846+0000 7f0866cd8b80 1 -- 172.21.15.117:0/210657903 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a62e722520 msgr2=0x55a62e71c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:00.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.846+0000 7f0866cd8b80 1 --2- 172.21.15.117:0/210657903 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a62e722520 0x55a62e71c2c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f084c09b330 tx=0x7f084c064040 comp rx=0 tx=0).stop 2024-06-03T03:24:00.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.846+0000 7f0866cd8b80 1 -- 172.21.15.117:0/210657903 shutdown_connections 2024-06-03T03:24:00.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.846+0000 7f0866cd8b80 1 --2- 172.21.15.117:0/210657903 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a62e722520 0x55a62e71c2c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:00.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.846+0000 7f0866cd8b80 1 --2- 172.21.15.117:0/210657903 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a62e693df0 0x55a62e71d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:00.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.846+0000 7f0866cd8b80 1 --2- 172.21.15.117:0/210657903 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a62e719910 0x55a62e71de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:00.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.846+0000 7f0866cd8b80 1 -- 172.21.15.117:0/210657903 >> 172.21.15.117:0/210657903 conn(0x55a62e644120 msgr2=0x55a62e71a410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:24:00.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.846+0000 7f0866cd8b80 1 -- 172.21.15.117:0/210657903 shutdown_connections 2024-06-03T03:24:00.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.846+0000 7f0866cd8b80 1 -- 172.21.15.117:0/210657903 wait complete. 2024-06-03T03:24:00.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.846+0000 7f0866cd8b80 1 -- start start 2024-06-03T03:24:00.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.847+0000 7f0866cd8b80 1 Processor -- start 2024-06-03T03:24:00.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.847+0000 7f0866cd8b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a62e693df0 0x55a62e73cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:00.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.847+0000 7f0866cd8b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a62e719910 0x55a62e71ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:00.850 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.847+0000 7f0866cd8b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a62e722520 0x55a62e71bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:00.850 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.847+0000 7f0866cd8b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a62e7331e0 con 0x55a62e722520 2024-06-03T03:24:00.850 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.847+0000 7f0866cd8b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a62e732b10 con 0x55a62e693df0 2024-06-03T03:24:00.850 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.847+0000 7f0866cd8b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a62e7338b0 con 0x55a62e719910 2024-06-03T03:24:00.850 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.847+0000 7f0866cd8b80 10 client.0 _ll_register_callbacks cb 0x55a62e6036d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:24:00.851 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.847+0000 7f0859000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a62e719910 0x55a62e71ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:00.851 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.847+0000 7f0859a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a62e693df0 0x55a62e73cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:00.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.848+0000 7f0859a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a62e693df0 0x55a62e73cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:41894/0 (socket says 172.21.15.117:41894) 2024-06-03T03:24:00.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.848+0000 7f085a400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a62e722520 0x55a62e71bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:00.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.848+0000 7f0859a00700 1 -- 172.21.15.117:0/225688109 learned_addr learned my addr 172.21.15.117:0/225688109 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:24:00.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.848+0000 7f0859000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a62e719910 0x55a62e71ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:38790/0 (socket says 172.21.15.117:38790) 2024-06-03T03:24:00.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.848+0000 7f0859a00700 1 -- 172.21.15.117:0/225688109 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a62e719910 msgr2=0x55a62e71ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:24:00.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.848+0000 7f0859a00700 1 --2- 172.21.15.117:0/225688109 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a62e719910 0x55a62e71ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:00.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.848+0000 7f0859a00700 1 -- 172.21.15.117:0/225688109 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a62e722520 msgr2=0x55a62e71bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:24:00.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.848+0000 7f0859a00700 1 --2- 172.21.15.117:0/225688109 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a62e722520 0x55a62e71bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:00.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.848+0000 7f0859a00700 1 -- 172.21.15.117:0/225688109 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a62e71c4f0 con 0x55a62e693df0 2024-06-03T03:24:00.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.848+0000 7f0859a00700 1 --2- 172.21.15.117:0/225688109 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a62e693df0 0x55a62e73cbc0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0848027970 tx=0x7f08480276c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1fb660157de328e5 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:24:00.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.848+0000 7f083b400700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-06-03T03:24:00.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.849+0000 7f083b400700 1 -- 172.21.15.117:0/225688109 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0848027c00 con 0x55a62e693df0 2024-06-03T03:24:00.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.849+0000 7f0866cd8b80 1 -- 172.21.15.117:0/225688109 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f084c09baa0 con 0x55a62e693df0 2024-06-03T03:24:00.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.849+0000 7f083b400700 1 -- 172.21.15.117:0/225688109 <== mon.2 v2:172.21.15.151:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f084801c940 con 0x55a62e693df0 2024-06-03T03:24:00.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.849+0000 7f083b400700 1 -- 172.21.15.117:0/225688109 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0848042050 con 0x55a62e693df0 2024-06-03T03:24:00.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.849+0000 7f0866cd8b80 10 client.25028 fetch_fsmap learned FSMap version 30 2024-06-03T03:24:00.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.849+0000 7f0866cd8b80 1 -- 172.21.15.117:0/225688109 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a62e73b4b0 con 0x55a62e693df0 2024-06-03T03:24:00.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.849+0000 7f083b400700 1 -- 172.21.15.117:0/225688109 <== mon.2 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f084c09baa0 con 0x55a62e693df0 2024-06-03T03:24:00.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.850+0000 7f083b400700 1 -- 172.21.15.117:0/225688109 <== mon.2 v2:172.21.15.151:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f08480037a0 con 0x55a62e693df0 2024-06-03T03:24:00.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.850+0000 7f0866cd8b80 10 client.25028 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T03:24:00.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.850+0000 7f0866cd8b80 10 client.25028 resolve_mds: resolved name 'i' to daemon mds.i 2024-06-03T03:24:00.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.850+0000 7f0866cd8b80 20 client.25028 populate_metadata read hostname 'smithi117' 2024-06-03T03:24:00.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.850+0000 7f0866cd8b80 1 --2- 172.21.15.117:0/225688109 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55a62e7a20b0 0x55a62e7a0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:00.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.850+0000 7f0866cd8b80 4 client.25028 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-03T03:24:00.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.850+0000 7f0866cd8b80 1 -- 172.21.15.117:0/225688109 --> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] -- command(tid 0: {"prefix": "status"}) -- 0x55a62e671530 con 0x55a62e7a20b0 2024-06-03T03:24:00.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.850+0000 7f0859000700 1 --2- 172.21.15.117:0/225688109 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55a62e7a20b0 0x55a62e7a0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:00.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.851+0000 7f0859000700 1 --2- 172.21.15.117:0/225688109 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55a62e7a20b0 0x55a62e7a0dc0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41a18fd2254d529f server_cookie=24f381cc49cf922b in_seq=0 out_seq=0 2024-06-03T03:24:00.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.851+0000 7f083b400700 10 client.25028 ms_handle_connect on v2:172.21.15.183:6838/1439615083 2024-06-03T03:24:00.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.851+0000 7f083b400700 1 -- 172.21.15.117:0/225688109 <== mds.0 v2:172.21.15.183:6838/1439615083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55a62e671530 con 0x55a62e7a20b0 2024-06-03T03:24:00.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.851+0000 7f083b400700 10 client.25028 handle_command_reply: tid=0 2024-06-03T03:24:00.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.851+0000 7f0866cd8b80 1 client.25028 shutdown 2024-06-03T03:24:00.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.851+0000 7f0866cd8b80 1 -- 172.21.15.117:0/225688109 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a62e693df0 msgr2=0x55a62e73cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:00.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.851+0000 7f0866cd8b80 1 --2- 172.21.15.117:0/225688109 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a62e693df0 0x55a62e73cbc0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0848027970 tx=0x7f08480276c0 comp rx=0 tx=0).stop 2024-06-03T03:24:00.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.851+0000 7f0866cd8b80 1 -- 172.21.15.117:0/225688109 shutdown_connections 2024-06-03T03:24:00.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.851+0000 7f0866cd8b80 1 --2- 172.21.15.117:0/225688109 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55a62e7a20b0 0x55a62e7a0dc0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:00.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.851+0000 7f0866cd8b80 1 --2- 172.21.15.117:0/225688109 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a62e722520 0x55a62e71bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:00.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.851+0000 7f0866cd8b80 1 --2- 172.21.15.117:0/225688109 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a62e693df0 0x55a62e73cbc0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:00.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.851+0000 7f083b400700 0 client.25028 ms_handle_reset on v2:172.21.15.183:6838/1439615083 2024-06-03T03:24:00.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.851+0000 7f0866cd8b80 1 --2- 172.21.15.117:0/225688109 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a62e719910 0x55a62e71ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:00.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.851+0000 7f0866cd8b80 1 -- 172.21.15.117:0/225688109 >> 172.21.15.117:0/225688109 conn(0x55a62e644120 msgr2=0x55a62e71d870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:24:00.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.852+0000 7f0866cd8b80 1 -- 172.21.15.117:0/225688109 shutdown_connections 2024-06-03T03:24:00.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.852+0000 7f0866cd8b80 1 -- 172.21.15.117:0/225688109 wait complete. 2024-06-03T03:24:00.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.852+0000 7f0866cd8b80 20 client.25028 trim_cache size 0 max 16384 2024-06-03T03:24:00.856 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-06-03T03:24:00.856 INFO:teuthology.orchestra.run.smithi117.stdout: "cluster_fsid": "a0e8473a-2156-11ef-bc9b-c7b262605968", 2024-06-03T03:24:00.856 INFO:teuthology.orchestra.run.smithi117.stdout: "whoami": 0, 2024-06-03T03:24:00.856 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 24517, 2024-06-03T03:24:00.856 INFO:teuthology.orchestra.run.smithi117.stdout: "want_state": "up:active", 2024-06-03T03:24:00.857 INFO:teuthology.orchestra.run.smithi117.stdout: "state": "up:active", 2024-06-03T03:24:00.857 INFO:teuthology.orchestra.run.smithi117.stdout: "fs_name": "cephfs", 2024-06-03T03:24:00.857 INFO:teuthology.orchestra.run.smithi117.stdout: "rank_uptime": 255.227958202, 2024-06-03T03:24:00.857 INFO:teuthology.orchestra.run.smithi117.stdout: "mdsmap_epoch": 30, 2024-06-03T03:24:00.857 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_epoch": 99, 2024-06-03T03:24:00.857 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_epoch_barrier": 97, 2024-06-03T03:24:00.857 INFO:teuthology.orchestra.run.smithi117.stdout: "uptime": 288.40881506900001 2024-06-03T03:24:00.857 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-06-03T03:24:00.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.853+0000 7f085ae00700 1 -- 172.21.15.117:0/3555382027 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f083c0a2c20 msgr2=0x7f083c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:00.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.853+0000 7f085ae00700 1 --2- 172.21.15.117:0/3555382027 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f083c0a2c20 0x7f083c0a50e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0848009c60 tx=0x7f0848009260 comp rx=0 tx=0).stop 2024-06-03T03:24:00.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.853+0000 7f085ae00700 1 -- 172.21.15.117:0/3555382027 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0854103870 msgr2=0x7f08540fdca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:00.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.853+0000 7f085ae00700 1 --2- 172.21.15.117:0/3555382027 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0854103870 0x7f08540fdca0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f084c00efc0 tx=0x7f084c00c3a0 comp rx=0 tx=0).stop 2024-06-03T03:24:00.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.853+0000 7f085ae00700 1 -- 172.21.15.117:0/3555382027 shutdown_connections 2024-06-03T03:24:00.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.853+0000 7f085ae00700 1 --2- 172.21.15.117:0/3555382027 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f083c0a2c20 0x7f083c0a50e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:00.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.853+0000 7f085ae00700 1 --2- 172.21.15.117:0/3555382027 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0854103870 0x7f08540fdca0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:00.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.853+0000 7f085ae00700 1 --2- 172.21.15.117:0/3555382027 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f08541015c0 0x7f08540fd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:00.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.853+0000 7f085ae00700 1 --2- 172.21.15.117:0/3555382027 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0854102eb0 0x7f08540fd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:00.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.853+0000 7f085ae00700 1 -- 172.21.15.117:0/3555382027 >> 172.21.15.117:0/3555382027 conn(0x7f08540fcd70 msgr2=0x7f085406ac80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:24:00.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.853+0000 7f085ae00700 1 -- 172.21.15.117:0/3555382027 shutdown_connections 2024-06-03T03:24:00.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:00.853+0000 7f085ae00700 1 -- 172.21.15.117:0/3555382027 wait complete. 2024-06-03T03:24:01.802 INFO:tasks.cephadm:Setup complete, yielding 2024-06-03T03:24:01.802 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-06-03T03:24:01.816 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi117.front.sepia.ceph.com 2024-06-03T03:24:01.816 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch status' 2024-06-03T03:24:02.229 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:24:02.380 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:24:02 smithi117 ceph-mon[31649]: pgmap v467: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:24:02.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:24:02 smithi151 ceph-mon[37043]: pgmap v467: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:24:02.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:24:02 smithi183 ceph-mon[37326]: pgmap v467: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:24:03.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:24:03 smithi117 ceph-mon[31649]: pgmap v468: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-06-03T03:24:03.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:24:03 smithi151 ceph-mon[37043]: pgmap v468: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-06-03T03:24:03.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:24:03 smithi183 ceph-mon[37326]: pgmap v468: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-06-03T03:24:04.625 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:04.624+0000 7fab9d600700 1 -- 172.21.15.117:0/3945360134 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab98103700 msgr2=0x7fab98103b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:04.625 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:04.624+0000 7fab9d600700 1 --2- 172.21.15.117:0/3945360134 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab98103700 0x7fab98103b60 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fab800099c0 tx=0x7fab80009cd0 comp rx=0 tx=0).stop 2024-06-03T03:24:04.625 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:04.624+0000 7fab9d600700 1 -- 172.21.15.117:0/3945360134 shutdown_connections 2024-06-03T03:24:04.626 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:04.624+0000 7fab9d600700 1 --2- 172.21.15.117:0/3945360134 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fab981040a0 0x7fab9810a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:04.626 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:04.624+0000 7fab9d600700 1 --2- 172.21.15.117:0/3945360134 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab98102500 0x7fab98102920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:04.626 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:04.624+0000 7fab9d600700 1 --2- 172.21.15.117:0/3945360134 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab98103700 0x7fab98103b60 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:04.626 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:04.624+0000 7fab9d600700 1 -- 172.21.15.117:0/3945360134 >> 172.21.15.117:0/3945360134 conn(0x7fab980fda60 msgr2=0x7fab980ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:24:04.626 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:04.624+0000 7fab9d600700 1 -- 172.21.15.117:0/3945360134 shutdown_connections 2024-06-03T03:24:04.626 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:04.624+0000 7fab9d600700 1 -- 172.21.15.117:0/3945360134 wait complete. 2024-06-03T03:24:04.627 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:04.625+0000 7fab9d600700 1 Processor -- start 2024-06-03T03:24:04.627 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:04.626+0000 7fab9d600700 1 -- start start 2024-06-03T03:24:04.628 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:04.626+0000 7fab9d600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab98102500 0x7fab98076230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:04.628 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:04.626+0000 7fab9d600700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fab98103700 0x7fab98074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:04.628 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:04.626+0000 7fab9d600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab981040a0 0x7fab98074dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:04.628 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:04.626+0000 7fab9d600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fab9810e140 con 0x7fab98102500 2024-06-03T03:24:04.628 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:04.626+0000 7fab9d600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fab9810dfc0 con 0x7fab981040a0 2024-06-03T03:24:04.628 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:04.626+0000 7fab9d600700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fab9810e2c0 con 0x7fab98103700 2024-06-03T03:24:04.628 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:04.626+0000 7fab96a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab98102500 0x7fab98076230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:04.628 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:04.626+0000 7fab97400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab981040a0 0x7fab98074dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:04.628 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:04.626+0000 7fab96a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab98102500 0x7fab98076230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:57660/0 (socket says 172.21.15.117:57660) 2024-06-03T03:24:04.628 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:04.626+0000 7fab96a00700 1 -- 172.21.15.117:0/701896790 learned_addr learned my addr 172.21.15.117:0/701896790 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:24:04.628 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:04.626+0000 7fab8e000700 1 --2- 172.21.15.117:0/701896790 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fab98103700 0x7fab98074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:04.629 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:04.627+0000 7fab96a00700 1 -- 172.21.15.117:0/701896790 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fab98103700 msgr2=0x7fab98074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:04.629 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:04.627+0000 7fab96a00700 1 --2- 172.21.15.117:0/701896790 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fab98103700 0x7fab98074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:04.629 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:04.627+0000 7fab96a00700 1 -- 172.21.15.117:0/701896790 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab981040a0 msgr2=0x7fab98074dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:04.629 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:04.627+0000 7fab96a00700 1 --2- 172.21.15.117:0/701896790 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab981040a0 0x7fab98074dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:04.629 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:04.627+0000 7fab96a00700 1 -- 172.21.15.117:0/701896790 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab98075670 con 0x7fab98102500 2024-06-03T03:24:04.629 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:04.627+0000 7fab96a00700 1 --2- 172.21.15.117:0/701896790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab98102500 0x7fab98076230 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fab8800efc0 tx=0x7fab8800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:24:04.630 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:04.627+0000 7fab8fe00700 1 -- 172.21.15.117:0/701896790 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab880092e0 con 0x7fab98102500 2024-06-03T03:24:04.630 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:04.627+0000 7fab8fe00700 1 -- 172.21.15.117:0/701896790 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab8800c9c0 con 0x7fab98102500 2024-06-03T03:24:04.630 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:04.627+0000 7fab8fe00700 1 -- 172.21.15.117:0/701896790 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab88004610 con 0x7fab98102500 2024-06-03T03:24:04.630 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:04.627+0000 7fab9d600700 1 -- 172.21.15.117:0/701896790 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab981a2830 con 0x7fab98102500 2024-06-03T03:24:04.630 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:04.628+0000 7fab9d600700 1 -- 172.21.15.117:0/701896790 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab981a2cf0 con 0x7fab98102500 2024-06-03T03:24:04.630 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:04.629+0000 7fab9d600700 1 -- 172.21.15.117:0/701896790 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab98069180 con 0x7fab98102500 2024-06-03T03:24:04.631 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:04.631+0000 7fab8fe00700 1 -- 172.21.15.117:0/701896790 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fab8801b030 con 0x7fab98102500 2024-06-03T03:24:04.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:04.632+0000 7fab8fe00700 1 --2- 172.21.15.117:0/701896790 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fab7c0a2c20 0x7fab7c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:04.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:04.632+0000 7fab8fe00700 1 -- 172.21.15.117:0/701896790 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7fab880cf370 con 0x7fab98102500 2024-06-03T03:24:04.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:04.632+0000 7fab8e000700 1 --2- 172.21.15.117:0/701896790 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fab7c0a2c20 0x7fab7c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:04.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:04.633+0000 7fab8e000700 1 --2- 172.21.15.117:0/701896790 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fab7c0a2c20 0x7fab7c0a50e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fab80000c00 tx=0x7fab80005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:24:04.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:04.637+0000 7fab8fe00700 1 -- 172.21.15.117:0/701896790 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fab88022020 con 0x7fab98102500 2024-06-03T03:24:04.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:04.883+0000 7fab9d600700 1 -- 172.21.15.117:0/701896790 --> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7fab980628d0 con 0x7fab7c0a2c20 2024-06-03T03:24:04.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:04.885+0000 7fab8fe00700 1 -- 172.21.15.117:0/701896790 <== mgr.14152 v2:172.21.15.117:6800/1398160887 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7fab980628d0 con 0x7fab7c0a2c20 2024-06-03T03:24:04.887 INFO:teuthology.orchestra.run.smithi117.stdout:Backend: cephadm 2024-06-03T03:24:04.887 INFO:teuthology.orchestra.run.smithi117.stdout:Available: Yes 2024-06-03T03:24:04.887 INFO:teuthology.orchestra.run.smithi117.stdout:Paused: No 2024-06-03T03:24:04.889 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:04.888+0000 7fab9d600700 1 -- 172.21.15.117:0/701896790 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fab7c0a2c20 msgr2=0x7fab7c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:04.889 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:04.888+0000 7fab9d600700 1 --2- 172.21.15.117:0/701896790 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fab7c0a2c20 0x7fab7c0a50e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fab80000c00 tx=0x7fab80005fb0 comp rx=0 tx=0).stop 2024-06-03T03:24:04.889 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:04.888+0000 7fab9d600700 1 -- 172.21.15.117:0/701896790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab98102500 msgr2=0x7fab98076230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:04.889 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:04.888+0000 7fab9d600700 1 --2- 172.21.15.117:0/701896790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab98102500 0x7fab98076230 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fab8800efc0 tx=0x7fab8800c3a0 comp rx=0 tx=0).stop 2024-06-03T03:24:04.890 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:04.889+0000 7fab9d600700 1 -- 172.21.15.117:0/701896790 shutdown_connections 2024-06-03T03:24:04.890 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:04.889+0000 7fab9d600700 1 --2- 172.21.15.117:0/701896790 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fab7c0a2c20 0x7fab7c0a50e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:04.890 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:04.889+0000 7fab9d600700 1 --2- 172.21.15.117:0/701896790 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab981040a0 0x7fab98074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:04.890 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:04.889+0000 7fab9d600700 1 --2- 172.21.15.117:0/701896790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab98102500 0x7fab98076230 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:04.890 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:04.889+0000 7fab9d600700 1 --2- 172.21.15.117:0/701896790 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fab98103700 0x7fab98074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:04.890 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:04.889+0000 7fab9d600700 1 -- 172.21.15.117:0/701896790 >> 172.21.15.117:0/701896790 conn(0x7fab980fda60 msgr2=0x7fab98106a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:24:04.890 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:04.889+0000 7fab9d600700 1 -- 172.21.15.117:0/701896790 shutdown_connections 2024-06-03T03:24:04.890 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:04.889+0000 7fab9d600700 1 -- 172.21.15.117:0/701896790 wait complete. 2024-06-03T03:24:05.851 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch ps' 2024-06-03T03:24:06.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:24:05 smithi117 ceph-mon[31649]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:24:06.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:24:05 smithi117 ceph-mon[31649]: pgmap v469: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:24:06.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:24:05 smithi183 ceph-mon[37326]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:24:06.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:24:05 smithi183 ceph-mon[37326]: pgmap v469: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:24:06.297 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:24:06.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:24:05 smithi151 ceph-mon[37043]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:24:06.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:24:05 smithi151 ceph-mon[37043]: pgmap v469: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:24:08.139 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:24:07 smithi117 ceph-mon[31649]: pgmap v470: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-03T03:24:08.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:24:07 smithi151 ceph-mon[37043]: pgmap v470: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-03T03:24:08.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:24:07 smithi183 ceph-mon[37326]: pgmap v470: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-03T03:24:08.712 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:08.711+0000 7f2505200700 1 -- 172.21.15.117:0/541652054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2500074ef0 msgr2=0x7f2500075370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:08.712 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:08.711+0000 7f2505200700 1 --2- 172.21.15.117:0/541652054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2500074ef0 0x7f2500075370 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f24ec009990 tx=0x7f24ec009ca0 comp rx=0 tx=0).stop 2024-06-03T03:24:08.712 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:08.711+0000 7f2505200700 1 -- 172.21.15.117:0/541652054 shutdown_connections 2024-06-03T03:24:08.712 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:08.711+0000 7f2505200700 1 --2- 172.21.15.117:0/541652054 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f250010a3c0 0x7f250010c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:08.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:08.711+0000 7f2505200700 1 --2- 172.21.15.117:0/541652054 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f25000764c0 0x7f2500074920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:08.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:08.711+0000 7f2505200700 1 --2- 172.21.15.117:0/541652054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2500074ef0 0x7f2500075370 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:08.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:08.711+0000 7f2505200700 1 -- 172.21.15.117:0/541652054 >> 172.21.15.117:0/541652054 conn(0x7f25000fda90 msgr2=0x7f25000ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:24:08.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:08.711+0000 7f2505200700 1 -- 172.21.15.117:0/541652054 shutdown_connections 2024-06-03T03:24:08.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:08.711+0000 7f2505200700 1 -- 172.21.15.117:0/541652054 wait complete. 2024-06-03T03:24:08.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:08.712+0000 7f2505200700 1 Processor -- start 2024-06-03T03:24:08.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:08.712+0000 7f2505200700 1 -- start start 2024-06-03T03:24:08.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:08.713+0000 7f2505200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2500074ef0 0x7f250019a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:08.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:08.713+0000 7f2505200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25000764c0 0x7f250019aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:08.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:08.713+0000 7f2505200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f250010a3c0 0x7f25001a1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:08.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:08.713+0000 7f2505200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2500110300 con 0x7f25000764c0 2024-06-03T03:24:08.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:08.713+0000 7f2505200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2500110180 con 0x7f250010a3c0 2024-06-03T03:24:08.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:08.713+0000 7f2505200700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2500110480 con 0x7f2500074ef0 2024-06-03T03:24:08.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:08.713+0000 7f24fe800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f250010a3c0 0x7f25001a1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:08.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:08.713+0000 7f24fd400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25000764c0 0x7f250019aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:08.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:08.713+0000 7f24fde00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2500074ef0 0x7f250019a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:08.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:08.713+0000 7f24fe800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f250010a3c0 0x7f25001a1f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:56282/0 (socket says 172.21.15.117:56282) 2024-06-03T03:24:08.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:08.713+0000 7f24fe800700 1 -- 172.21.15.117:0/540314795 learned_addr learned my addr 172.21.15.117:0/540314795 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:24:08.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:08.713+0000 7f24fde00700 1 -- 172.21.15.117:0/540314795 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f250010a3c0 msgr2=0x7f25001a1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:08.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:08.713+0000 7f24fde00700 1 --2- 172.21.15.117:0/540314795 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f250010a3c0 0x7f25001a1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:08.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:08.713+0000 7f24fde00700 1 -- 172.21.15.117:0/540314795 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25000764c0 msgr2=0x7f250019aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:08.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:08.713+0000 7f24fde00700 1 --2- 172.21.15.117:0/540314795 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25000764c0 0x7f250019aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:08.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:08.713+0000 7f24fde00700 1 -- 172.21.15.117:0/540314795 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25001a25c0 con 0x7f2500074ef0 2024-06-03T03:24:08.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:08.714+0000 7f24fde00700 1 --2- 172.21.15.117:0/540314795 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2500074ef0 0x7f250019a960 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f24f000e9f0 tx=0x7f24f000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:24:08.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:08.714+0000 7f24f6a00700 1 -- 172.21.15.117:0/540314795 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f000c9c0 con 0x7f2500074ef0 2024-06-03T03:24:08.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:08.714+0000 7f24f6a00700 1 -- 172.21.15.117:0/540314795 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24f0010430 con 0x7f2500074ef0 2024-06-03T03:24:08.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:08.714+0000 7f2505200700 1 -- 172.21.15.117:0/540314795 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25001a28b0 con 0x7f2500074ef0 2024-06-03T03:24:08.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:08.714+0000 7f24f6a00700 1 -- 172.21.15.117:0/540314795 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f0018680 con 0x7f2500074ef0 2024-06-03T03:24:08.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:08.714+0000 7f2505200700 1 -- 172.21.15.117:0/540314795 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25001a2df0 con 0x7f2500074ef0 2024-06-03T03:24:08.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:08.715+0000 7f2505200700 1 -- 172.21.15.117:0/540314795 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2500069180 con 0x7f2500074ef0 2024-06-03T03:24:08.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:08.717+0000 7f24f6a00700 1 -- 172.21.15.117:0/540314795 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f24f0010950 con 0x7f2500074ef0 2024-06-03T03:24:08.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:08.719+0000 7f24f6a00700 1 --2- 172.21.15.117:0/540314795 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f24e40a2c20 0x7f24e40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:08.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:08.719+0000 7f24f6a00700 1 -- 172.21.15.117:0/540314795 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f24f00cf980 con 0x7f2500074ef0 2024-06-03T03:24:08.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:08.719+0000 7f24fd400700 1 --2- 172.21.15.117:0/540314795 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f24e40a2c20 0x7f24e40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:08.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:08.720+0000 7f24fd400700 1 --2- 172.21.15.117:0/540314795 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f24e40a2c20 0x7f24e40a50e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f24ec00b7b0 tx=0x7f24ec00b880 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:24:08.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:08.722+0000 7f24f6a00700 1 -- 172.21.15.117:0/540314795 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f24f009c4f0 con 0x7f2500074ef0 2024-06-03T03:24:08.956 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:08.955+0000 7f2505200700 1 -- 172.21.15.117:0/540314795 --> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f25000628d0 con 0x7f24e40a2c20 2024-06-03T03:24:08.976 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:08.975+0000 7f24f6a00700 1 -- 172.21.15.117:0/540314795 <== mgr.14152 v2:172.21.15.117:6800/1398160887 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7f25000628d0 con 0x7f24e40a2c20 2024-06-03T03:24:08.976 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-03T03:24:08.976 INFO:teuthology.orchestra.run.smithi117.stdout:mds.a smithi117 running (4m) 2m ago 4m 16.0M - 19.0.0-2946-g3d89accf b045ac545f82 85a7fbdf4fdf 2024-06-03T03:24:08.976 INFO:teuthology.orchestra.run.smithi117.stdout:mds.b smithi151 running (4m) 2m ago 4m 18.0M - 19.0.0-2946-g3d89accf b045ac545f82 d52b90d82890 2024-06-03T03:24:08.977 INFO:teuthology.orchestra.run.smithi117.stdout:mds.c smithi183 running (5m) 2m ago 5m 16.7M - 19.0.0-2946-g3d89accf b045ac545f82 b98b31aef053 2024-06-03T03:24:08.977 INFO:teuthology.orchestra.run.smithi117.stdout:mds.d smithi117 running (5m) 2m ago 5m 13.0M - 19.0.0-2946-g3d89accf b045ac545f82 d731bf2a4312 2024-06-03T03:24:08.977 INFO:teuthology.orchestra.run.smithi117.stdout:mds.e smithi151 running (5m) 2m ago 5m 15.6M - 19.0.0-2946-g3d89accf b045ac545f82 427fca501166 2024-06-03T03:24:08.977 INFO:teuthology.orchestra.run.smithi117.stdout:mds.f smithi183 running (5m) 2m ago 5m 17.4M - 19.0.0-2946-g3d89accf b045ac545f82 9477330b4194 2024-06-03T03:24:08.977 INFO:teuthology.orchestra.run.smithi117.stdout:mds.g smithi117 running (4m) 2m ago 4m 12.7M - 19.0.0-2946-g3d89accf b045ac545f82 7ea4d8f5f974 2024-06-03T03:24:08.977 INFO:teuthology.orchestra.run.smithi117.stdout:mds.h smithi151 running (4m) 2m ago 4m 16.6M - 19.0.0-2946-g3d89accf b045ac545f82 ec4dc6e6e019 2024-06-03T03:24:08.977 INFO:teuthology.orchestra.run.smithi117.stdout:mds.i smithi183 running (4m) 2m ago 4m 28.5M - 19.0.0-2946-g3d89accf b045ac545f82 71fa4d2bfe2c 2024-06-03T03:24:08.977 INFO:teuthology.orchestra.run.smithi117.stdout:mds.j smithi117 running (5m) 2m ago 5m 15.1M - 19.0.0-2946-g3d89accf b045ac545f82 acc516aa313c 2024-06-03T03:24:08.977 INFO:teuthology.orchestra.run.smithi117.stdout:mds.k smithi151 running (5m) 2m ago 5m 21.1M - 19.0.0-2946-g3d89accf b045ac545f82 8f37978860c8 2024-06-03T03:24:08.977 INFO:teuthology.orchestra.run.smithi117.stdout:mds.l smithi183 running (5m) 2m ago 5m 16.6M - 19.0.0-2946-g3d89accf b045ac545f82 148d60d50a9a 2024-06-03T03:24:08.977 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.x smithi117 *:9283,8765 running (13m) 2m ago 13m 517M - 19.0.0-2946-g3d89accf b045ac545f82 31f67206095a 2024-06-03T03:24:08.977 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.y smithi151 *:8443,8765 running (11m) 2m ago 11m 456M - 19.0.0-2946-g3d89accf b045ac545f82 11784dd35c97 2024-06-03T03:24:08.978 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.z smithi183 *:8443,8765 running (11m) 2m ago 11m 453M - 19.0.0-2946-g3d89accf b045ac545f82 275464c5a737 2024-06-03T03:24:08.978 INFO:teuthology.orchestra.run.smithi117.stdout:mon.a smithi117 running (13m) 2m ago 14m 48.9M 2048M 19.0.0-2946-g3d89accf b045ac545f82 92ba733bffb8 2024-06-03T03:24:08.978 INFO:teuthology.orchestra.run.smithi117.stdout:mon.b smithi151 running (11m) 2m ago 11m 42.0M 2048M 19.0.0-2946-g3d89accf b045ac545f82 193aee5b1d93 2024-06-03T03:24:08.978 INFO:teuthology.orchestra.run.smithi117.stdout:mon.c smithi183 running (11m) 2m ago 11m 42.7M 2048M 19.0.0-2946-g3d89accf b045ac545f82 24b9dbb1fddb 2024-06-03T03:24:08.978 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (10m) 2m ago 10m 45.7M 4272M 19.0.0-2946-g3d89accf b045ac545f82 242d07237290 2024-06-03T03:24:08.978 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi151 running (10m) 2m ago 10m 47.9M 4272M 19.0.0-2946-g3d89accf b045ac545f82 ac8f6d836aee 2024-06-03T03:24:08.978 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi183 running (10m) 2m ago 10m 59.1M 4272M 19.0.0-2946-g3d89accf b045ac545f82 d6f5c0d7780d 2024-06-03T03:24:08.978 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi117 running (9m) 2m ago 9m 43.4M 4272M 19.0.0-2946-g3d89accf b045ac545f82 c78df18b338a 2024-06-03T03:24:08.978 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi151 running (9m) 2m ago 9m 45.5M 4272M 19.0.0-2946-g3d89accf b045ac545f82 dc4177a89dc4 2024-06-03T03:24:08.978 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi183 running (8m) 2m ago 8m 58.9M 4272M 19.0.0-2946-g3d89accf b045ac545f82 fdaef9fa776d 2024-06-03T03:24:08.978 INFO:teuthology.orchestra.run.smithi117.stdout:osd.6 smithi117 running (8m) 2m ago 8m 43.6M 4272M 19.0.0-2946-g3d89accf b045ac545f82 85b94ed18f70 2024-06-03T03:24:08.978 INFO:teuthology.orchestra.run.smithi117.stdout:osd.7 smithi151 running (7m) 2m ago 7m 59.2M 4272M 19.0.0-2946-g3d89accf b045ac545f82 3872c2f9de02 2024-06-03T03:24:08.979 INFO:teuthology.orchestra.run.smithi117.stdout:osd.8 smithi183 running (7m) 2m ago 7m 57.2M 4272M 19.0.0-2946-g3d89accf b045ac545f82 a3da1a0d4954 2024-06-03T03:24:08.979 INFO:teuthology.orchestra.run.smithi117.stdout:osd.9 smithi117 running (6m) 2m ago 6m 53.8M 4272M 19.0.0-2946-g3d89accf b045ac545f82 ce935bdb182e 2024-06-03T03:24:08.979 INFO:teuthology.orchestra.run.smithi117.stdout:osd.10 smithi151 running (6m) 2m ago 6m 56.3M 4272M 19.0.0-2946-g3d89accf b045ac545f82 a90a223158ba 2024-06-03T03:24:08.979 INFO:teuthology.orchestra.run.smithi117.stdout:osd.11 smithi183 running (5m) 2m ago 5m 51.6M 4272M 19.0.0-2946-g3d89accf b045ac545f82 e810848ad39a 2024-06-03T03:24:08.980 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:08.979+0000 7f2505200700 1 -- 172.21.15.117:0/540314795 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f24e40a2c20 msgr2=0x7f24e40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:08.980 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:08.979+0000 7f2505200700 1 --2- 172.21.15.117:0/540314795 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f24e40a2c20 0x7f24e40a50e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f24ec00b7b0 tx=0x7f24ec00b880 comp rx=0 tx=0).stop 2024-06-03T03:24:08.980 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:08.979+0000 7f2505200700 1 -- 172.21.15.117:0/540314795 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2500074ef0 msgr2=0x7f250019a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:08.980 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:08.979+0000 7f2505200700 1 --2- 172.21.15.117:0/540314795 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2500074ef0 0x7f250019a960 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f24f000e9f0 tx=0x7f24f000ed00 comp rx=0 tx=0).stop 2024-06-03T03:24:08.981 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:08.979+0000 7f2505200700 1 -- 172.21.15.117:0/540314795 shutdown_connections 2024-06-03T03:24:08.981 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:08.979+0000 7f2505200700 1 --2- 172.21.15.117:0/540314795 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f24e40a2c20 0x7f24e40a50e0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:08.981 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:08.979+0000 7f2505200700 1 --2- 172.21.15.117:0/540314795 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f250010a3c0 0x7f25001a1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:08.981 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:08.979+0000 7f2505200700 1 --2- 172.21.15.117:0/540314795 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2500074ef0 0x7f250019a960 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:08.981 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:08.979+0000 7f2505200700 1 --2- 172.21.15.117:0/540314795 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25000764c0 0x7f250019aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:08.981 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:08.979+0000 7f2505200700 1 -- 172.21.15.117:0/540314795 >> 172.21.15.117:0/540314795 conn(0x7f25000fda90 msgr2=0x7f25001062c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:24:08.981 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:08.980+0000 7f2505200700 1 -- 172.21.15.117:0/540314795 shutdown_connections 2024-06-03T03:24:08.981 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:08.980+0000 7f2505200700 1 -- 172.21.15.117:0/540314795 wait complete. 2024-06-03T03:24:10.004 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch ls' 2024-06-03T03:24:10.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:24:09 smithi117 ceph-mon[31649]: pgmap v471: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-03T03:24:10.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:24:09 smithi117 ceph-mon[31649]: from='client.25069 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:24:10.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:24:09 smithi151 ceph-mon[37043]: pgmap v471: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-03T03:24:10.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:24:09 smithi151 ceph-mon[37043]: from='client.25069 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:24:10.453 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:24:10.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:24:09 smithi183 ceph-mon[37326]: pgmap v471: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-03T03:24:10.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:24:09 smithi183 ceph-mon[37326]: from='client.25069 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:24:12.284 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:24:11 smithi183 ceph-mon[37326]: pgmap v472: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:24:12.377 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:24:11 smithi117 ceph-mon[31649]: pgmap v472: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:24:12.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:24:11 smithi151 ceph-mon[37043]: pgmap v472: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:24:12.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:12.939+0000 7f9efca00700 1 -- 172.21.15.117:0/999708034 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9ef8104ce0 msgr2=0x7f9ef810cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:12.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:12.939+0000 7f9efca00700 1 --2- 172.21.15.117:0/999708034 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9ef8104ce0 0x7f9ef810cb70 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9ef000b0d0 tx=0x7f9ef000b3e0 comp rx=0 tx=0).stop 2024-06-03T03:24:12.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:12.939+0000 7f9efca00700 1 -- 172.21.15.117:0/999708034 shutdown_connections 2024-06-03T03:24:12.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:12.939+0000 7f9efca00700 1 --2- 172.21.15.117:0/999708034 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9ef8104ce0 0x7f9ef810cb70 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:12.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:12.939+0000 7f9efca00700 1 --2- 172.21.15.117:0/999708034 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9ef8105e00 0x7f9ef8106220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:12.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:12.939+0000 7f9efca00700 1 --2- 172.21.15.117:0/999708034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ef8104340 0x7f9ef81047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:12.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:12.939+0000 7f9efca00700 1 -- 172.21.15.117:0/999708034 >> 172.21.15.117:0/999708034 conn(0x7f9ef80ffce0 msgr2=0x7f9ef8102100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:24:12.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:12.939+0000 7f9efca00700 1 -- 172.21.15.117:0/999708034 shutdown_connections 2024-06-03T03:24:12.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:12.939+0000 7f9efca00700 1 -- 172.21.15.117:0/999708034 wait complete. 2024-06-03T03:24:12.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:12.940+0000 7f9efca00700 1 Processor -- start 2024-06-03T03:24:12.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:12.940+0000 7f9efca00700 1 -- start start 2024-06-03T03:24:12.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:12.941+0000 7f9efca00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ef8104340 0x7f9ef819aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:12.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:12.941+0000 7f9efca00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9ef8104ce0 0x7f9ef819afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:12.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:12.941+0000 7f9efca00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9ef8105e00 0x7f9ef81a2020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:12.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:12.941+0000 7f9efca00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef8110460 con 0x7f9ef8104340 2024-06-03T03:24:12.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:12.941+0000 7f9efca00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef81102e0 con 0x7f9ef8105e00 2024-06-03T03:24:12.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:12.941+0000 7f9efca00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef81105e0 con 0x7f9ef8104ce0 2024-06-03T03:24:12.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:12.941+0000 7f9ef6800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9ef8105e00 0x7f9ef81a2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:12.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:12.941+0000 7f9ef5e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ef8104340 0x7f9ef819aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:12.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:12.941+0000 7f9ef5400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9ef8104ce0 0x7f9ef819afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:12.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:12.941+0000 7f9ef6800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9ef8105e00 0x7f9ef81a2020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:56302/0 (socket says 172.21.15.117:56302) 2024-06-03T03:24:12.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:12.941+0000 7f9ef6800700 1 -- 172.21.15.117:0/3335168209 learned_addr learned my addr 172.21.15.117:0/3335168209 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:24:12.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:12.941+0000 7f9ef6800700 1 -- 172.21.15.117:0/3335168209 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9ef8104ce0 msgr2=0x7f9ef819afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:12.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:12.941+0000 7f9ef6800700 1 --2- 172.21.15.117:0/3335168209 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9ef8104ce0 0x7f9ef819afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:12.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:12.941+0000 7f9ef6800700 1 -- 172.21.15.117:0/3335168209 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ef8104340 msgr2=0x7f9ef819aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:12.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:12.941+0000 7f9ef6800700 1 --2- 172.21.15.117:0/3335168209 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ef8104340 0x7f9ef819aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:12.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:12.942+0000 7f9ef6800700 1 -- 172.21.15.117:0/3335168209 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ef81a26c0 con 0x7f9ef8105e00 2024-06-03T03:24:12.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:12.942+0000 7f9ef6800700 1 --2- 172.21.15.117:0/3335168209 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9ef8105e00 0x7f9ef81a2020 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9ef000b770 tx=0x7f9ef00079a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:24:12.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:12.942+0000 7f9eeea00700 1 -- 172.21.15.117:0/3335168209 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef0003c00 con 0x7f9ef8105e00 2024-06-03T03:24:12.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:12.942+0000 7f9efca00700 1 -- 172.21.15.117:0/3335168209 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ef81a2950 con 0x7f9ef8105e00 2024-06-03T03:24:12.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:12.942+0000 7f9eeea00700 1 -- 172.21.15.117:0/3335168209 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ef0003da0 con 0x7f9ef8105e00 2024-06-03T03:24:12.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:12.942+0000 7f9eeea00700 1 -- 172.21.15.117:0/3335168209 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef001b880 con 0x7f9ef8105e00 2024-06-03T03:24:12.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:12.942+0000 7f9efca00700 1 -- 172.21.15.117:0/3335168209 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ef81a2da0 con 0x7f9ef8105e00 2024-06-03T03:24:12.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:12.944+0000 7f9efca00700 1 -- 172.21.15.117:0/3335168209 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ef8069180 con 0x7f9ef8105e00 2024-06-03T03:24:12.946 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:12.946+0000 7f9eeea00700 1 -- 172.21.15.117:0/3335168209 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f9ef001ba40 con 0x7f9ef8105e00 2024-06-03T03:24:12.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:12.947+0000 7f9eeea00700 1 --2- 172.21.15.117:0/3335168209 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f9edc0a2c20 0x7f9edc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:12.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:12.948+0000 7f9eeea00700 1 -- 172.21.15.117:0/3335168209 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f9ef00d0920 con 0x7f9ef8105e00 2024-06-03T03:24:12.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:12.948+0000 7f9ef5e00700 1 --2- 172.21.15.117:0/3335168209 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f9edc0a2c20 0x7f9edc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:12.949 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:12.948+0000 7f9ef5e00700 1 --2- 172.21.15.117:0/3335168209 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f9edc0a2c20 0x7f9edc0a50e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9ef8105780 tx=0x7f9ee8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:24:12.950 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:12.950+0000 7f9eeea00700 1 -- 172.21.15.117:0/3335168209 <== mon.2 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9ef0024350 con 0x7f9ef8105e00 2024-06-03T03:24:13.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:13.185+0000 7f9efca00700 1 -- 172.21.15.117:0/3335168209 --> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f9ef80628d0 con 0x7f9edc0a2c20 2024-06-03T03:24:13.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:13.202+0000 7f9eeea00700 1 -- 172.21.15.117:0/3335168209 <== mgr.14152 v2:172.21.15.117:6800/1398160887 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f9ef80628d0 con 0x7f9edc0a2c20 2024-06-03T03:24:13.203 INFO:teuthology.orchestra.run.smithi117.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-06-03T03:24:13.203 INFO:teuthology.orchestra.run.smithi117.stdout:mds.all 12/12 2m ago 5m smithi117=a;smithi117=d;smithi117=g;smithi117=j;smithi151=b;smithi151=e;smithi151=h;smithi151=k;smithi183=c;smithi183=f;smithi183=i;smithi183=l;count:12 2024-06-03T03:24:13.203 INFO:teuthology.orchestra.run.smithi117.stdout:mgr 3/3 2m ago 11m smithi117=x;smithi151=y;smithi183=z;count:3 2024-06-03T03:24:13.203 INFO:teuthology.orchestra.run.smithi117.stdout:mon 3/3 2m ago 11m smithi117:172.21.15.117=a;smithi151:172.21.15.151=b;smithi183:172.21.15.183=c;count:3 2024-06-03T03:24:13.203 INFO:teuthology.orchestra.run.smithi117.stdout:osd 12 2m ago - 2024-06-03T03:24:13.206 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:13.205+0000 7f9efca00700 1 -- 172.21.15.117:0/3335168209 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f9edc0a2c20 msgr2=0x7f9edc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:13.206 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:13.205+0000 7f9efca00700 1 --2- 172.21.15.117:0/3335168209 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f9edc0a2c20 0x7f9edc0a50e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9ef8105780 tx=0x7f9ee8009260 comp rx=0 tx=0).stop 2024-06-03T03:24:13.206 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:13.205+0000 7f9efca00700 1 -- 172.21.15.117:0/3335168209 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9ef8105e00 msgr2=0x7f9ef81a2020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:13.206 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:13.205+0000 7f9efca00700 1 --2- 172.21.15.117:0/3335168209 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9ef8105e00 0x7f9ef81a2020 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9ef000b770 tx=0x7f9ef00079a0 comp rx=0 tx=0).stop 2024-06-03T03:24:13.206 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:13.205+0000 7f9efca00700 1 -- 172.21.15.117:0/3335168209 shutdown_connections 2024-06-03T03:24:13.206 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:13.205+0000 7f9efca00700 1 --2- 172.21.15.117:0/3335168209 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f9edc0a2c20 0x7f9edc0a50e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:13.206 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:13.205+0000 7f9efca00700 1 --2- 172.21.15.117:0/3335168209 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9ef8105e00 0x7f9ef81a2020 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:13.206 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:13.205+0000 7f9efca00700 1 --2- 172.21.15.117:0/3335168209 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ef8104340 0x7f9ef819aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:13.206 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:13.205+0000 7f9efca00700 1 --2- 172.21.15.117:0/3335168209 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9ef8104ce0 0x7f9ef819afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:13.206 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:13.205+0000 7f9efca00700 1 -- 172.21.15.117:0/3335168209 >> 172.21.15.117:0/3335168209 conn(0x7f9ef80ffce0 msgr2=0x7f9ef8108b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:24:13.206 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:13.205+0000 7f9efca00700 1 -- 172.21.15.117:0/3335168209 shutdown_connections 2024-06-03T03:24:13.207 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:13.205+0000 7f9efca00700 1 -- 172.21.15.117:0/3335168209 wait complete. 2024-06-03T03:24:14.113 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch host ls' 2024-06-03T03:24:14.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:24:13 smithi151 ceph-mon[37043]: pgmap v473: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-06-03T03:24:14.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:24:13 smithi117 ceph-mon[31649]: pgmap v473: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-06-03T03:24:14.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:24:13 smithi183 ceph-mon[37326]: pgmap v473: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-06-03T03:24:14.562 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:24:15.298 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:24:15 smithi117 ceph-mon[31649]: from='client.25046 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:24:15.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:24:15 smithi151 ceph-mon[37043]: from='client.25046 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:24:15.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:24:15 smithi183 ceph-mon[37326]: from='client.25046 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:24:16.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:24:16 smithi117 ceph-mon[31649]: pgmap v474: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:24:16.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:24:16 smithi151 ceph-mon[37043]: pgmap v474: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:24:16.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:24:16 smithi183 ceph-mon[37326]: pgmap v474: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:24:17.183 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:17.182+0000 7fc5d4c00700 1 -- 172.21.15.117:0/3654879207 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5d0109d30 msgr2=0x7fc5d010a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:17.184 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:17.182+0000 7fc5d4c00700 1 --2- 172.21.15.117:0/3654879207 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5d0109d30 0x7fc5d010a190 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc0099c0 tx=0x7fc5bc009cd0 comp rx=0 tx=0).stop 2024-06-03T03:24:17.184 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:17.182+0000 7fc5d4c00700 1 -- 172.21.15.117:0/3654879207 shutdown_connections 2024-06-03T03:24:17.184 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:17.182+0000 7fc5d4c00700 1 --2- 172.21.15.117:0/3654879207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc5d010a6d0 0x7fc5d0110f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:17.184 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:17.182+0000 7fc5d4c00700 1 --2- 172.21.15.117:0/3654879207 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc5d0108b30 0x7fc5d0108f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:17.184 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:17.182+0000 7fc5d4c00700 1 --2- 172.21.15.117:0/3654879207 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5d0109d30 0x7fc5d010a190 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:17.184 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:17.182+0000 7fc5d4c00700 1 -- 172.21.15.117:0/3654879207 >> 172.21.15.117:0/3654879207 conn(0x7fc5d0077b00 msgr2=0x7fc5d0079f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:24:17.184 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:17.182+0000 7fc5d4c00700 1 -- 172.21.15.117:0/3654879207 shutdown_connections 2024-06-03T03:24:17.184 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:17.182+0000 7fc5d4c00700 1 -- 172.21.15.117:0/3654879207 wait complete. 2024-06-03T03:24:17.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:17.183+0000 7fc5d4c00700 1 Processor -- start 2024-06-03T03:24:17.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:17.184+0000 7fc5d4c00700 1 -- start start 2024-06-03T03:24:17.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:17.184+0000 7fc5d4c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc5d0108b30 0x7fc5d019ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:17.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:17.184+0000 7fc5d4c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5d0109d30 0x7fc5d019f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:17.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:17.184+0000 7fc5d4c00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc5d010a6d0 0x7fc5d01a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:17.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:17.184+0000 7fc5d4c00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d0114770 con 0x7fc5d0109d30 2024-06-03T03:24:17.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:17.184+0000 7fc5d4c00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d01145f0 con 0x7fc5d0108b30 2024-06-03T03:24:17.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:17.184+0000 7fc5d4c00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d01148f0 con 0x7fc5d010a6d0 2024-06-03T03:24:17.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:17.184+0000 7fc5ce800700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc5d010a6d0 0x7fc5d01a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:17.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:17.184+0000 7fc5cde00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc5d0108b30 0x7fc5d019ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:17.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:17.184+0000 7fc5cd400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5d0109d30 0x7fc5d019f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:17.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:17.184+0000 7fc5cde00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc5d0108b30 0x7fc5d019ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:36398/0 (socket says 172.21.15.117:36398) 2024-06-03T03:24:17.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:17.184+0000 7fc5ce800700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc5d010a6d0 0x7fc5d01a62a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:48558/0 (socket says 172.21.15.117:48558) 2024-06-03T03:24:17.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:17.184+0000 7fc5cde00700 1 -- 172.21.15.117:0/3241693495 learned_addr learned my addr 172.21.15.117:0/3241693495 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:24:17.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:17.185+0000 7fc5cde00700 1 -- 172.21.15.117:0/3241693495 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc5d010a6d0 msgr2=0x7fc5d01a62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:17.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:17.185+0000 7fc5cde00700 1 --2- 172.21.15.117:0/3241693495 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc5d010a6d0 0x7fc5d01a62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:17.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:17.185+0000 7fc5cde00700 1 -- 172.21.15.117:0/3241693495 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5d0109d30 msgr2=0x7fc5d019f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:17.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:17.185+0000 7fc5cde00700 1 --2- 172.21.15.117:0/3241693495 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5d0109d30 0x7fc5d019f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:17.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:17.185+0000 7fc5cde00700 1 -- 172.21.15.117:0/3241693495 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5d01a6970 con 0x7fc5d0108b30 2024-06-03T03:24:17.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:17.185+0000 7fc5ce800700 1 --2- 172.21.15.117:0/3241693495 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc5d010a6d0 0x7fc5d01a62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:24:17.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:17.185+0000 7fc5cd400700 1 --2- 172.21.15.117:0/3241693495 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5d0109d30 0x7fc5d019f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:24:17.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:17.185+0000 7fc5cde00700 1 --2- 172.21.15.117:0/3241693495 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc5d0108b30 0x7fc5d019ed20 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc5c000e970 tx=0x7fc5c000ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:24:17.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:17.186+0000 7fc5c6a00700 1 -- 172.21.15.117:0/3241693495 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c0004220 con 0x7fc5d0108b30 2024-06-03T03:24:17.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:17.186+0000 7fc5c6a00700 1 -- 172.21.15.117:0/3241693495 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5c00043c0 con 0x7fc5d0108b30 2024-06-03T03:24:17.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:17.186+0000 7fc5d4c00700 1 -- 172.21.15.117:0/3241693495 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5d01a6c60 con 0x7fc5d0108b30 2024-06-03T03:24:17.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:17.186+0000 7fc5c6a00700 1 -- 172.21.15.117:0/3241693495 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c0003dc0 con 0x7fc5d0108b30 2024-06-03T03:24:17.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:17.186+0000 7fc5d4c00700 1 -- 172.21.15.117:0/3241693495 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5d01a71a0 con 0x7fc5d0108b30 2024-06-03T03:24:17.189 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:17.187+0000 7fc5d4c00700 1 -- 172.21.15.117:0/3241693495 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5d0069180 con 0x7fc5d0108b30 2024-06-03T03:24:17.189 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:17.189+0000 7fc5c6a00700 1 -- 172.21.15.117:0/3241693495 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fc5c000c7e0 con 0x7fc5d0108b30 2024-06-03T03:24:17.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:17.190+0000 7fc5c6a00700 1 --2- 172.21.15.117:0/3241693495 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fc5b40a2ca0 0x7fc5b40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:17.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:17.190+0000 7fc5c6a00700 1 -- 172.21.15.117:0/3241693495 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7fc5c00cf590 con 0x7fc5d0108b30 2024-06-03T03:24:17.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:17.190+0000 7fc5cd400700 1 --2- 172.21.15.117:0/3241693495 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fc5b40a2ca0 0x7fc5b40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:17.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:17.191+0000 7fc5cd400700 1 --2- 172.21.15.117:0/3241693495 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fc5b40a2ca0 0x7fc5b40a5160 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc000c00 tx=0x7fc5bc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:24:17.195 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:17.195+0000 7fc5c6a00700 1 -- 172.21.15.117:0/3241693495 <== mon.2 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc5c009c100 con 0x7fc5d0108b30 2024-06-03T03:24:17.424 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:17.423+0000 7fc5d4c00700 1 -- 172.21.15.117:0/3241693495 --> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7fc5d00628d0 con 0x7fc5b40a2ca0 2024-06-03T03:24:17.427 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:17.426+0000 7fc5c6a00700 1 -- 172.21.15.117:0/3241693495 <== mgr.14152 v2:172.21.15.117:6800/1398160887 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7fc5d00628d0 con 0x7fc5b40a2ca0 2024-06-03T03:24:17.427 INFO:teuthology.orchestra.run.smithi117.stdout:HOST ADDR LABELS STATUS 2024-06-03T03:24:17.427 INFO:teuthology.orchestra.run.smithi117.stdout:smithi117 172.21.15.117 2024-06-03T03:24:17.427 INFO:teuthology.orchestra.run.smithi117.stdout:smithi151 172.21.15.151 2024-06-03T03:24:17.427 INFO:teuthology.orchestra.run.smithi117.stdout:smithi183 172.21.15.183 2024-06-03T03:24:17.428 INFO:teuthology.orchestra.run.smithi117.stdout:3 hosts in cluster 2024-06-03T03:24:17.431 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:17.430+0000 7fc5d4c00700 1 -- 172.21.15.117:0/3241693495 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fc5b40a2ca0 msgr2=0x7fc5b40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:17.431 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:17.430+0000 7fc5d4c00700 1 --2- 172.21.15.117:0/3241693495 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fc5b40a2ca0 0x7fc5b40a5160 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc000c00 tx=0x7fc5bc005fb0 comp rx=0 tx=0).stop 2024-06-03T03:24:17.431 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:17.430+0000 7fc5d4c00700 1 -- 172.21.15.117:0/3241693495 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc5d0108b30 msgr2=0x7fc5d019ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:17.432 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:17.430+0000 7fc5d4c00700 1 --2- 172.21.15.117:0/3241693495 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc5d0108b30 0x7fc5d019ed20 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc5c000e970 tx=0x7fc5c000ec80 comp rx=0 tx=0).stop 2024-06-03T03:24:17.432 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:17.430+0000 7fc5d4c00700 1 -- 172.21.15.117:0/3241693495 shutdown_connections 2024-06-03T03:24:17.432 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:17.430+0000 7fc5d4c00700 1 --2- 172.21.15.117:0/3241693495 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fc5b40a2ca0 0x7fc5b40a5160 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:17.432 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:17.430+0000 7fc5d4c00700 1 --2- 172.21.15.117:0/3241693495 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc5d010a6d0 0x7fc5d01a62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:17.432 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:17.430+0000 7fc5d4c00700 1 --2- 172.21.15.117:0/3241693495 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc5d0108b30 0x7fc5d019ed20 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:17.432 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:17.431+0000 7fc5d4c00700 1 --2- 172.21.15.117:0/3241693495 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5d0109d30 0x7fc5d019f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:17.432 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:17.431+0000 7fc5d4c00700 1 -- 172.21.15.117:0/3241693495 >> 172.21.15.117:0/3241693495 conn(0x7fc5d0077b00 msgr2=0x7fc5d010cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:24:17.432 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:17.431+0000 7fc5d4c00700 1 -- 172.21.15.117:0/3241693495 shutdown_connections 2024-06-03T03:24:17.432 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:17.431+0000 7fc5d4c00700 1 -- 172.21.15.117:0/3241693495 wait complete. 2024-06-03T03:24:18.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:24:18 smithi117 ceph-mon[31649]: pgmap v475: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-03T03:24:18.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:24:18 smithi151 ceph-mon[37043]: pgmap v475: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-03T03:24:18.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:24:18 smithi183 ceph-mon[37326]: pgmap v475: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-03T03:24:18.529 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch device ls' 2024-06-03T03:24:18.962 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:24:19.296 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:24:19 smithi117 ceph-mon[31649]: from='client.25052 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:24:19.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:24:19 smithi151 ceph-mon[37043]: from='client.25052 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:24:19.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:24:19 smithi183 ceph-mon[37326]: from='client.25052 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:24:20.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:24:20 smithi151 ceph-mon[37043]: pgmap v476: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-03T03:24:20.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:24:20 smithi117 ceph-mon[31649]: pgmap v476: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-03T03:24:20.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:24:20 smithi183 ceph-mon[37326]: pgmap v476: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-03T03:24:21.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:21.343+0000 7f178b000700 1 -- 172.21.15.117:0/1327622642 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1784104630 msgr2=0x7f1784104a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:21.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:21.343+0000 7f178b000700 1 --2- 172.21.15.117:0/1327622642 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1784104630 0x7f1784104a50 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f17780098f0 tx=0x7f1778009c00 comp rx=0 tx=0).stop 2024-06-03T03:24:21.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:21.343+0000 7f178b000700 1 -- 172.21.15.117:0/1327622642 shutdown_connections 2024-06-03T03:24:21.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:21.343+0000 7f178b000700 1 --2- 172.21.15.117:0/1327622642 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f17841061d0 0x7f178410ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:21.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:21.343+0000 7f178b000700 1 --2- 172.21.15.117:0/1327622642 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1784104630 0x7f1784104a50 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:21.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:21.343+0000 7f178b000700 1 --2- 172.21.15.117:0/1327622642 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1784105830 0x7f1784105c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:21.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:21.343+0000 7f178b000700 1 -- 172.21.15.117:0/1327622642 >> 172.21.15.117:0/1327622642 conn(0x7f17840ffbd0 msgr2=0x7f1784102010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:24:21.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:21.344+0000 7f178b000700 1 -- 172.21.15.117:0/1327622642 shutdown_connections 2024-06-03T03:24:21.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:21.344+0000 7f178b000700 1 -- 172.21.15.117:0/1327622642 wait complete. 2024-06-03T03:24:21.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:21.345+0000 7f178b000700 1 Processor -- start 2024-06-03T03:24:21.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:21.345+0000 7f178b000700 1 -- start start 2024-06-03T03:24:21.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:21.345+0000 7f178b000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1784104630 0x7f178419a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:21.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:21.345+0000 7f178b000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1784105830 0x7f178419adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:21.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:21.345+0000 7f178b000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f17841061d0 0x7f17841a1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:21.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:21.345+0000 7f178b000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1784110220 con 0x7f1784105830 2024-06-03T03:24:21.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:21.345+0000 7f178b000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f17841100a0 con 0x7f17841061d0 2024-06-03T03:24:21.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:21.345+0000 7f178b000700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f17841103a0 con 0x7f1784104630 2024-06-03T03:24:21.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:21.346+0000 7f1789200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1784105830 0x7f178419adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:21.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:21.346+0000 7f1789200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1784105830 0x7f178419adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:57180/0 (socket says 172.21.15.117:57180) 2024-06-03T03:24:21.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:21.346+0000 7f1789200700 1 -- 172.21.15.117:0/1815577213 learned_addr learned my addr 172.21.15.117:0/1815577213 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:24:21.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:21.346+0000 7f1789c00700 1 --2- 172.21.15.117:0/1815577213 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1784104630 0x7f178419a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:21.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:21.346+0000 7f178a600700 1 --2- 172.21.15.117:0/1815577213 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f17841061d0 0x7f17841a1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:21.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:21.346+0000 7f1789200700 1 -- 172.21.15.117:0/1815577213 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1784104630 msgr2=0x7f178419a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:21.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:21.346+0000 7f1789200700 1 --2- 172.21.15.117:0/1815577213 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1784104630 0x7f178419a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:21.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:21.346+0000 7f1789200700 1 -- 172.21.15.117:0/1815577213 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f17841061d0 msgr2=0x7f17841a1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:21.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:21.346+0000 7f1789200700 1 --2- 172.21.15.117:0/1815577213 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f17841061d0 0x7f17841a1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:21.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:21.346+0000 7f1789200700 1 -- 172.21.15.117:0/1815577213 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17841a24e0 con 0x7f1784105830 2024-06-03T03:24:21.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:21.346+0000 7f1789c00700 1 --2- 172.21.15.117:0/1815577213 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1784104630 0x7f178419a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:24:21.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:21.347+0000 7f1789200700 1 --2- 172.21.15.117:0/1815577213 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1784105830 0x7f178419adc0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f177000e9f0 tx=0x7f177000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:24:21.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:21.347+0000 7f1782000700 1 -- 172.21.15.117:0/1815577213 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1770004a30 con 0x7f1784105830 2024-06-03T03:24:21.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:21.347+0000 7f1782000700 1 -- 172.21.15.117:0/1815577213 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1770005500 con 0x7f1784105830 2024-06-03T03:24:21.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:21.347+0000 7f178b000700 1 -- 172.21.15.117:0/1815577213 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17841a27d0 con 0x7f1784105830 2024-06-03T03:24:21.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:21.347+0000 7f1782000700 1 -- 172.21.15.117:0/1815577213 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f177000cd60 con 0x7f1784105830 2024-06-03T03:24:21.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:21.347+0000 7f178b000700 1 -- 172.21.15.117:0/1815577213 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17841a2d10 con 0x7f1784105830 2024-06-03T03:24:21.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:21.348+0000 7f178b000700 1 -- 172.21.15.117:0/1815577213 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1784069180 con 0x7f1784105830 2024-06-03T03:24:21.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:21.350+0000 7f1782000700 1 -- 172.21.15.117:0/1815577213 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f1770004c60 con 0x7f1784105830 2024-06-03T03:24:21.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:21.351+0000 7f1782000700 1 --2- 172.21.15.117:0/1815577213 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f176c0a2c20 0x7f176c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:21.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:21.352+0000 7f1782000700 1 -- 172.21.15.117:0/1815577213 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f17700cf590 con 0x7f1784105830 2024-06-03T03:24:21.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:21.352+0000 7f1789c00700 1 --2- 172.21.15.117:0/1815577213 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f176c0a2c20 0x7f176c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:21.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:21.352+0000 7f1789c00700 1 --2- 172.21.15.117:0/1815577213 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f176c0a2c20 0x7f176c0a50e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1778009810 tx=0x7f177801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:24:21.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:21.356+0000 7f1782000700 1 -- 172.21.15.117:0/1815577213 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f177009c100 con 0x7f1784105830 2024-06-03T03:24:21.587 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:21.586+0000 7f178b000700 1 -- 172.21.15.117:0/1815577213 --> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f17840628d0 con 0x7f176c0a2c20 2024-06-03T03:24:21.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:21.589+0000 7f1782000700 1 -- 172.21.15.117:0/1815577213 <== mgr.14152 v2:172.21.15.117:6800/1398160887 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f17840628d0 con 0x7f176c0a2c20 2024-06-03T03:24:21.590 INFO:teuthology.orchestra.run.smithi117.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-06-03T03:24:21.591 INFO:teuthology.orchestra.run.smithi117.stdout:smithi117 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400V6400BGN 372G No 6m ago Has a FileSystem, LVM detected 2024-06-03T03:24:21.591 INFO:teuthology.orchestra.run.smithi117.stdout:smithi151 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300JJ400BGN 372G No 6m ago Has a FileSystem, LVM detected 2024-06-03T03:24:21.591 INFO:teuthology.orchestra.run.smithi117.stdout:smithi183 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620500QJ400BGN 372G No 5m ago Has a FileSystem, LVM detected 2024-06-03T03:24:21.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:21.592+0000 7f178b000700 1 -- 172.21.15.117:0/1815577213 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f176c0a2c20 msgr2=0x7f176c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:21.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:21.592+0000 7f178b000700 1 --2- 172.21.15.117:0/1815577213 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f176c0a2c20 0x7f176c0a50e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1778009810 tx=0x7f177801a040 comp rx=0 tx=0).stop 2024-06-03T03:24:21.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:21.592+0000 7f178b000700 1 -- 172.21.15.117:0/1815577213 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1784105830 msgr2=0x7f178419adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:21.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:21.592+0000 7f178b000700 1 --2- 172.21.15.117:0/1815577213 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1784105830 0x7f178419adc0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f177000e9f0 tx=0x7f177000ed00 comp rx=0 tx=0).stop 2024-06-03T03:24:21.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:21.592+0000 7f178b000700 1 -- 172.21.15.117:0/1815577213 shutdown_connections 2024-06-03T03:24:21.594 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:21.592+0000 7f178b000700 1 --2- 172.21.15.117:0/1815577213 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f176c0a2c20 0x7f176c0a50e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:21.594 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:21.592+0000 7f178b000700 1 --2- 172.21.15.117:0/1815577213 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f17841061d0 0x7f17841a1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:21.594 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:21.592+0000 7f178b000700 1 --2- 172.21.15.117:0/1815577213 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1784104630 0x7f178419a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:21.594 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:21.592+0000 7f178b000700 1 --2- 172.21.15.117:0/1815577213 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1784105830 0x7f178419adc0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:21.594 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:21.592+0000 7f178b000700 1 -- 172.21.15.117:0/1815577213 >> 172.21.15.117:0/1815577213 conn(0x7f17840ffbd0 msgr2=0x7f1784108bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:24:21.594 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:21.593+0000 7f178b000700 1 -- 172.21.15.117:0/1815577213 shutdown_connections 2024-06-03T03:24:21.594 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:21.593+0000 7f178b000700 1 -- 172.21.15.117:0/1815577213 wait complete. 2024-06-03T03:24:22.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:24:22 smithi151 ceph-mon[37043]: pgmap v477: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:24:22.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:24:22 smithi117 ceph-mon[31649]: pgmap v477: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:24:22.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:24:22 smithi183 ceph-mon[37326]: pgmap v477: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:24:22.584 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-06-03T03:24:22.594 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi117.front.sepia.ceph.com 2024-06-03T03:24:22.594 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- bash -c 'ceph fs dump' 2024-06-03T03:24:23.064 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:24:23.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:24:23 smithi151 ceph-mon[37043]: from='client.14943 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:24:23.421 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:24:23 smithi117 ceph-mon[31649]: from='client.14943 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:24:23.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:24:23 smithi183 ceph-mon[37326]: from='client.14943 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:24:24.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:24:24 smithi151 ceph-mon[37043]: pgmap v478: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-06-03T03:24:24.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:24:24 smithi117 ceph-mon[31649]: pgmap v478: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-06-03T03:24:24.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:24:24 smithi183 ceph-mon[37326]: pgmap v478: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-06-03T03:24:25.343 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:24:25 smithi117 ceph-mon[31649]: pgmap v479: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-03T03:24:25.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:25.370+0000 7f0830400700 1 -- 172.21.15.117:0/134460415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0828105930 msgr2=0x7f0828105d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:25.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:25.370+0000 7f0830400700 1 --2- 172.21.15.117:0/134460415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0828105930 0x7f0828105d90 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f08140099c0 tx=0x7f0814009cd0 comp rx=0 tx=0).stop 2024-06-03T03:24:25.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:25.370+0000 7f0830400700 1 -- 172.21.15.117:0/134460415 shutdown_connections 2024-06-03T03:24:25.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:25.370+0000 7f0830400700 1 --2- 172.21.15.117:0/134460415 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f08281062d0 0x7f082810cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:25.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:25.370+0000 7f0830400700 1 --2- 172.21.15.117:0/134460415 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0828104730 0x7f0828104b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:25.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:25.370+0000 7f0830400700 1 --2- 172.21.15.117:0/134460415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0828105930 0x7f0828105d90 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:25.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:25.371+0000 7f0830400700 1 -- 172.21.15.117:0/134460415 >> 172.21.15.117:0/134460415 conn(0x7f08280ffcf0 msgr2=0x7f0828102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:24:25.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:25.371+0000 7f0830400700 1 -- 172.21.15.117:0/134460415 shutdown_connections 2024-06-03T03:24:25.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:25.371+0000 7f0830400700 1 -- 172.21.15.117:0/134460415 wait complete. 2024-06-03T03:24:25.373 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:25.372+0000 7f0830400700 1 Processor -- start 2024-06-03T03:24:25.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:25.372+0000 7f0830400700 1 -- start start 2024-06-03T03:24:25.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:25.372+0000 7f0830400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0828104730 0x7f082819a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:25.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:25.372+0000 7f0830400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0828105930 0x7f082819ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:25.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:25.372+0000 7f0830400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08281062d0 0x7f08281a1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:25.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:25.372+0000 7f0830400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0828110370 con 0x7f08281062d0 2024-06-03T03:24:25.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:25.372+0000 7f0830400700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f08281101f0 con 0x7f0828105930 2024-06-03T03:24:25.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:25.372+0000 7f0830400700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f08281104f0 con 0x7f0828104730 2024-06-03T03:24:25.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:25.373+0000 7f082e400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08281062d0 0x7f08281a1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:25.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:25.373+0000 7f082da00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0828104730 0x7f082819a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:25.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:25.373+0000 7f082d000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0828105930 0x7f082819ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:25.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:25.373+0000 7f082e400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08281062d0 0x7f08281a1ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:57208/0 (socket says 172.21.15.117:57208) 2024-06-03T03:24:25.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:25.373+0000 7f082e400700 1 -- 172.21.15.117:0/1427906863 learned_addr learned my addr 172.21.15.117:0/1427906863 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:24:25.375 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:25.373+0000 7f082da00700 1 -- 172.21.15.117:0/1427906863 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0828105930 msgr2=0x7f082819ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:25.375 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:25.373+0000 7f082da00700 1 --2- 172.21.15.117:0/1427906863 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0828105930 0x7f082819ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:25.375 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:25.373+0000 7f082da00700 1 -- 172.21.15.117:0/1427906863 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08281062d0 msgr2=0x7f08281a1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:25.375 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:25.373+0000 7f082da00700 1 --2- 172.21.15.117:0/1427906863 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08281062d0 0x7f08281a1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:25.375 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:25.373+0000 7f082da00700 1 -- 172.21.15.117:0/1427906863 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08281a2560 con 0x7f0828104730 2024-06-03T03:24:25.375 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:25.373+0000 7f082e400700 1 --2- 172.21.15.117:0/1427906863 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08281062d0 0x7f08281a1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:24:25.376 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:25.374+0000 7f082da00700 1 --2- 172.21.15.117:0/1427906863 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0828104730 0x7f082819a900 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f081c00e9f0 tx=0x7f081c00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:24:25.376 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:25.374+0000 7f0826000700 1 -- 172.21.15.117:0/1427906863 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f081c004220 con 0x7f0828104730 2024-06-03T03:24:25.376 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:25.374+0000 7f0826000700 1 -- 172.21.15.117:0/1427906863 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f081c0043c0 con 0x7f0828104730 2024-06-03T03:24:25.376 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:25.374+0000 7f0830400700 1 -- 172.21.15.117:0/1427906863 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08281a2850 con 0x7f0828104730 2024-06-03T03:24:25.376 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:25.374+0000 7f0826000700 1 -- 172.21.15.117:0/1427906863 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f081c003cb0 con 0x7f0828104730 2024-06-03T03:24:25.376 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:25.374+0000 7f0830400700 1 -- 172.21.15.117:0/1427906863 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08281a2de0 con 0x7f0828104730 2024-06-03T03:24:25.376 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:25.375+0000 7f0830400700 1 -- 172.21.15.117:0/1427906863 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0828069180 con 0x7f0828104730 2024-06-03T03:24:25.377 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:25.377+0000 7f0826000700 1 -- 172.21.15.117:0/1427906863 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f081c00c7c0 con 0x7f0828104730 2024-06-03T03:24:25.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:25.378+0000 7f0826000700 1 --2- 172.21.15.117:0/1427906863 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f08100a2ca0 0x7f08100a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:25.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:25.379+0000 7f0826000700 1 -- 172.21.15.117:0/1427906863 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f081c0cf590 con 0x7f0828104730 2024-06-03T03:24:25.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:25.379+0000 7f082d000700 1 --2- 172.21.15.117:0/1427906863 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f08100a2ca0 0x7f08100a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:25.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:25.379+0000 7f082d000700 1 --2- 172.21.15.117:0/1427906863 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f08100a2ca0 0x7f08100a5160 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0814000c00 tx=0x7f0814005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:24:25.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:25.383+0000 7f0826000700 1 -- 172.21.15.117:0/1427906863 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f081c09c100 con 0x7f0828104730 2024-06-03T03:24:25.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:24:25 smithi151 ceph-mon[37043]: pgmap v479: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-03T03:24:25.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:24:25 smithi183 ceph-mon[37326]: pgmap v479: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-03T03:24:25.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:25.658+0000 7f0830400700 1 -- 172.21.15.117:0/1427906863 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f08280630a0 con 0x7f0828104730 2024-06-03T03:24:25.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:25.659+0000 7f0826000700 1 -- 172.21.15.117:0/1427906863 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) ==== 76+0+3107 (secure 0 0 0) 0x7f081c09ba00 con 0x7f0828104730 2024-06-03T03:24:25.660 INFO:teuthology.orchestra.run.smithi117.stdout:e30 2024-06-03T03:24:25.660 INFO:teuthology.orchestra.run.smithi117.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-06-03T03:24:25.660 INFO:teuthology.orchestra.run.smithi117.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-06-03T03:24:25.660 INFO:teuthology.orchestra.run.smithi117.stdout:legacy client fscid: 1 2024-06-03T03:24:25.660 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T03:24:25.660 INFO:teuthology.orchestra.run.smithi117.stdout:Filesystem 'cephfs' (1) 2024-06-03T03:24:25.660 INFO:teuthology.orchestra.run.smithi117.stdout:fs_name cephfs 2024-06-03T03:24:25.661 INFO:teuthology.orchestra.run.smithi117.stdout:epoch 30 2024-06-03T03:24:25.661 INFO:teuthology.orchestra.run.smithi117.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-06-03T03:24:25.661 INFO:teuthology.orchestra.run.smithi117.stdout:created 2024-06-03T03:19:44.629822+0000 2024-06-03T03:24:25.661 INFO:teuthology.orchestra.run.smithi117.stdout:modified 2024-06-03T03:21:12.085445+0000 2024-06-03T03:24:25.661 INFO:teuthology.orchestra.run.smithi117.stdout:tableserver 0 2024-06-03T03:24:25.661 INFO:teuthology.orchestra.run.smithi117.stdout:root 0 2024-06-03T03:24:25.661 INFO:teuthology.orchestra.run.smithi117.stdout:session_timeout 300 2024-06-03T03:24:25.661 INFO:teuthology.orchestra.run.smithi117.stdout:session_autoclose 300 2024-06-03T03:24:25.661 INFO:teuthology.orchestra.run.smithi117.stdout:max_file_size 1099511627776 2024-06-03T03:24:25.661 INFO:teuthology.orchestra.run.smithi117.stdout:max_xattr_size 65536 2024-06-03T03:24:25.661 INFO:teuthology.orchestra.run.smithi117.stdout:required_client_features {} 2024-06-03T03:24:25.661 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure 0 2024-06-03T03:24:25.661 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure_osd_epoch 0 2024-06-03T03:24:25.661 INFO:teuthology.orchestra.run.smithi117.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-06-03T03:24:25.662 INFO:teuthology.orchestra.run.smithi117.stdout:max_mds 5 2024-06-03T03:24:25.662 INFO:teuthology.orchestra.run.smithi117.stdout:in 0,1,2,3,4 2024-06-03T03:24:25.662 INFO:teuthology.orchestra.run.smithi117.stdout:up {0=24517,1=24512,2=24503,3=24499,4=24487} 2024-06-03T03:24:25.662 INFO:teuthology.orchestra.run.smithi117.stdout:failed 2024-06-03T03:24:25.662 INFO:teuthology.orchestra.run.smithi117.stdout:damaged 2024-06-03T03:24:25.662 INFO:teuthology.orchestra.run.smithi117.stdout:stopped 2024-06-03T03:24:25.662 INFO:teuthology.orchestra.run.smithi117.stdout:data_pools [3,4] 2024-06-03T03:24:25.662 INFO:teuthology.orchestra.run.smithi117.stdout:metadata_pool 2 2024-06-03T03:24:25.662 INFO:teuthology.orchestra.run.smithi117.stdout:inline_data disabled 2024-06-03T03:24:25.662 INFO:teuthology.orchestra.run.smithi117.stdout:balancer 2024-06-03T03:24:25.662 INFO:teuthology.orchestra.run.smithi117.stdout:bal_rank_mask -1 2024-06-03T03:24:25.662 INFO:teuthology.orchestra.run.smithi117.stdout:standby_count_wanted 0 2024-06-03T03:24:25.663 INFO:teuthology.orchestra.run.smithi117.stdout:qdb_cluster leader: 24517 members: 24503,24517,24512,24487,24499 2024-06-03T03:24:25.663 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.i{0:24517} state up:active seq 10 addr [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] compat {c=[1],r=[1],i=[1fff]}] 2024-06-03T03:24:25.663 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.h{0:24479} state up:standby-replay seq 1 addr [v2:172.21.15.151:6836/2392407848,v1:172.21.15.151:6837/2392407848] compat {c=[1],r=[1],i=[1fff]}] 2024-06-03T03:24:25.663 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.b{1:24512} state up:active seq 26 addr [v2:172.21.15.151:6838/3623444616,v1:172.21.15.151:6839/3623444616] compat {c=[1],r=[1],i=[1fff]}] 2024-06-03T03:24:25.663 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.j{1:24466} state up:standby-replay seq 1 addr [v2:172.21.15.117:6834/2884460088,v1:172.21.15.117:6835/2884460088] compat {c=[1],r=[1],i=[1fff]}] 2024-06-03T03:24:25.663 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.a{2:24503} state up:active seq 27 addr [v2:172.21.15.117:6840/1212242967,v1:172.21.15.117:6841/1212242967] compat {c=[1],r=[1],i=[1fff]}] 2024-06-03T03:24:25.663 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.e{2:24461} state up:standby-replay seq 1 addr [v2:172.21.15.151:6834/31153579,v1:172.21.15.151:6835/31153579] compat {c=[1],r=[1],i=[1fff]}] 2024-06-03T03:24:25.663 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.c{3:24499} state up:active seq 31 addr [v2:172.21.15.183:6836/2697518526,v1:172.21.15.183:6837/2697518526] compat {c=[1],r=[1],i=[1fff]}] 2024-06-03T03:24:25.663 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.l{3:24445} state up:standby-replay seq 1 addr [v2:172.21.15.183:6832/2429725085,v1:172.21.15.183:6833/2429725085] compat {c=[1],r=[1],i=[1fff]}] 2024-06-03T03:24:25.663 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.f{4:24487} state up:active seq 32 addr [v2:172.21.15.183:6834/1009004320,v1:172.21.15.183:6835/1009004320] compat {c=[1],r=[1],i=[1fff]}] 2024-06-03T03:24:25.663 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.k{4:24431} state up:standby-replay seq 1 addr [v2:172.21.15.151:6832/3355662416,v1:172.21.15.151:6833/3355662416] compat {c=[1],r=[1],i=[1fff]}] 2024-06-03T03:24:25.663 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T03:24:25.663 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T03:24:25.663 INFO:teuthology.orchestra.run.smithi117.stdout:Standby daemons: 2024-06-03T03:24:25.664 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T03:24:25.664 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.d{-1:14367} state up:standby seq 1 addr [v2:172.21.15.117:6836/3353001119,v1:172.21.15.117:6837/3353001119] compat {c=[1],r=[1],i=[1fff]}] 2024-06-03T03:24:25.664 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.g{-1:14388} state up:standby seq 1 addr [v2:172.21.15.117:6838/405782304,v1:172.21.15.117:6839/405782304] compat {c=[1],r=[1],i=[1fff]}] 2024-06-03T03:24:25.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:25.662+0000 7f0830400700 1 -- 172.21.15.117:0/1427906863 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f08100a2ca0 msgr2=0x7f08100a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:25.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:25.662+0000 7f0830400700 1 --2- 172.21.15.117:0/1427906863 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f08100a2ca0 0x7f08100a5160 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0814000c00 tx=0x7f0814005fb0 comp rx=0 tx=0).stop 2024-06-03T03:24:25.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:25.662+0000 7f0830400700 1 -- 172.21.15.117:0/1427906863 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0828104730 msgr2=0x7f082819a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:25.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:25.662+0000 7f0830400700 1 --2- 172.21.15.117:0/1427906863 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0828104730 0x7f082819a900 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f081c00e9f0 tx=0x7f081c00ed00 comp rx=0 tx=0).stop 2024-06-03T03:24:25.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:25.662+0000 7f0830400700 1 -- 172.21.15.117:0/1427906863 shutdown_connections 2024-06-03T03:24:25.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:25.662+0000 7f0830400700 1 --2- 172.21.15.117:0/1427906863 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f08100a2ca0 0x7f08100a5160 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:25.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:25.662+0000 7f0830400700 1 --2- 172.21.15.117:0/1427906863 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08281062d0 0x7f08281a1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:25.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:25.662+0000 7f0830400700 1 --2- 172.21.15.117:0/1427906863 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0828104730 0x7f082819a900 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:25.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:25.662+0000 7f0830400700 1 --2- 172.21.15.117:0/1427906863 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0828105930 0x7f082819ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:25.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:25.662+0000 7f0830400700 1 -- 172.21.15.117:0/1427906863 >> 172.21.15.117:0/1427906863 conn(0x7f08280ffcf0 msgr2=0x7f0828108bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:24:25.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:25.662+0000 7f0830400700 1 -- 172.21.15.117:0/1427906863 shutdown_connections 2024-06-03T03:24:25.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:25.662+0000 7f0830400700 1 -- 172.21.15.117:0/1427906863 wait complete. 2024-06-03T03:24:25.665 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 30 2024-06-03T03:24:26.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:24:26 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/1427906863' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-03T03:24:26.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:24:26 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/1427906863' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-03T03:24:26.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:24:26 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/1427906863' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-03T03:24:26.653 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- bash -c 'ceph osd dump' 2024-06-03T03:24:27.114 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:24:27.310 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:24:27 smithi117 ceph-mon[31649]: pgmap v480: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-03T03:24:27.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:24:27 smithi151 ceph-mon[37043]: pgmap v480: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-03T03:24:27.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:24:27 smithi183 ceph-mon[37326]: pgmap v480: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-03T03:24:29.566 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:29.565+0000 7fe2d4c00700 1 -- 172.21.15.117:0/2913239836 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe2d0074ef0 msgr2=0x7fe2d0075370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:29.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:29.565+0000 7fe2d4c00700 1 --2- 172.21.15.117:0/2913239836 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe2d0074ef0 0x7fe2d0075370 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe2b8009990 tx=0x7fe2b8009ca0 comp rx=0 tx=0).stop 2024-06-03T03:24:29.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:29.565+0000 7fe2d4c00700 1 -- 172.21.15.117:0/2913239836 shutdown_connections 2024-06-03T03:24:29.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:29.565+0000 7fe2d4c00700 1 --2- 172.21.15.117:0/2913239836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2d010a3d0 0x7fe2d010c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:29.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:29.565+0000 7fe2d4c00700 1 --2- 172.21.15.117:0/2913239836 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe2d00764c0 0x7fe2d0074920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:29.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:29.565+0000 7fe2d4c00700 1 --2- 172.21.15.117:0/2913239836 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe2d0074ef0 0x7fe2d0075370 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:29.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:29.565+0000 7fe2d4c00700 1 -- 172.21.15.117:0/2913239836 >> 172.21.15.117:0/2913239836 conn(0x7fe2d00fda60 msgr2=0x7fe2d00ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:24:29.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:29.565+0000 7fe2d4c00700 1 -- 172.21.15.117:0/2913239836 shutdown_connections 2024-06-03T03:24:29.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:29.565+0000 7fe2d4c00700 1 -- 172.21.15.117:0/2913239836 wait complete. 2024-06-03T03:24:29.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:29.567+0000 7fe2d4c00700 1 Processor -- start 2024-06-03T03:24:29.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:29.567+0000 7fe2d4c00700 1 -- start start 2024-06-03T03:24:29.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:29.568+0000 7fe2d4c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2d0074ef0 0x7fe2d019a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:29.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:29.568+0000 7fe2d4c00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe2d00764c0 0x7fe2d019aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:29.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:29.568+0000 7fe2d4c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe2d010a3d0 0x7fe2d01a1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:29.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:29.568+0000 7fe2d4c00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d0110330 con 0x7fe2d0074ef0 2024-06-03T03:24:29.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:29.568+0000 7fe2d4c00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d01101b0 con 0x7fe2d010a3d0 2024-06-03T03:24:29.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:29.568+0000 7fe2d4c00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d01104b0 con 0x7fe2d00764c0 2024-06-03T03:24:29.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:29.568+0000 7fe2cde00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2d0074ef0 0x7fe2d019a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:29.570 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:29.568+0000 7fe2c7e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe2d00764c0 0x7fe2d019aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:29.570 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:29.568+0000 7fe2c7e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe2d00764c0 0x7fe2d019aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:50670/0 (socket says 172.21.15.117:50670) 2024-06-03T03:24:29.570 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:29.568+0000 7fe2c7e00700 1 -- 172.21.15.117:0/4050282157 learned_addr learned my addr 172.21.15.117:0/4050282157 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:24:29.570 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:29.568+0000 7fe2ce800700 1 --2- 172.21.15.117:0/4050282157 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe2d010a3d0 0x7fe2d01a1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:29.570 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:29.569+0000 7fe2c7e00700 1 -- 172.21.15.117:0/4050282157 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe2d010a3d0 msgr2=0x7fe2d01a1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:29.570 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:29.569+0000 7fe2c7e00700 1 --2- 172.21.15.117:0/4050282157 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe2d010a3d0 0x7fe2d01a1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:29.570 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:29.569+0000 7fe2c7e00700 1 -- 172.21.15.117:0/4050282157 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2d0074ef0 msgr2=0x7fe2d019a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:29.570 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:29.569+0000 7fe2c7e00700 1 --2- 172.21.15.117:0/4050282157 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2d0074ef0 0x7fe2d019a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:29.570 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:29.569+0000 7fe2c7e00700 1 -- 172.21.15.117:0/4050282157 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2d01a2600 con 0x7fe2d00764c0 2024-06-03T03:24:29.570 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:29.569+0000 7fe2ce800700 1 --2- 172.21.15.117:0/4050282157 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe2d010a3d0 0x7fe2d01a1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:24:29.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:29.569+0000 7fe2c7e00700 1 --2- 172.21.15.117:0/4050282157 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe2d00764c0 0x7fe2d019aee0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe2b8000c00 tx=0x7fe2b800bb90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:24:29.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:29.569+0000 7fe2c6a00700 1 -- 172.21.15.117:0/4050282157 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b8026070 con 0x7fe2d00764c0 2024-06-03T03:24:29.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:29.569+0000 7fe2d4c00700 1 -- 172.21.15.117:0/4050282157 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2d01a2890 con 0x7fe2d00764c0 2024-06-03T03:24:29.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:29.570+0000 7fe2c6a00700 1 -- 172.21.15.117:0/4050282157 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2b8003d70 con 0x7fe2d00764c0 2024-06-03T03:24:29.572 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:29.570+0000 7fe2c6a00700 1 -- 172.21.15.117:0/4050282157 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b8021410 con 0x7fe2d00764c0 2024-06-03T03:24:29.572 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:29.570+0000 7fe2d4c00700 1 -- 172.21.15.117:0/4050282157 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2d01a2ce0 con 0x7fe2d00764c0 2024-06-03T03:24:29.572 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:29.571+0000 7fe2d4c00700 1 -- 172.21.15.117:0/4050282157 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2d0069180 con 0x7fe2d00764c0 2024-06-03T03:24:29.573 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:29.573+0000 7fe2c6a00700 1 -- 172.21.15.117:0/4050282157 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fe2b8029070 con 0x7fe2d00764c0 2024-06-03T03:24:29.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:29.574+0000 7fe2c6a00700 1 --2- 172.21.15.117:0/4050282157 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fe2b40a2c20 0x7fe2b40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:29.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:29.575+0000 7fe2c6a00700 1 -- 172.21.15.117:0/4050282157 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7fe2b80d0800 con 0x7fe2d00764c0 2024-06-03T03:24:29.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:29.575+0000 7fe2cde00700 1 --2- 172.21.15.117:0/4050282157 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fe2b40a2c20 0x7fe2b40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:29.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:29.575+0000 7fe2cde00700 1 --2- 172.21.15.117:0/4050282157 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fe2b40a2c20 0x7fe2b40a50e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe2d00768a0 tx=0x7fe2c0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:24:29.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:29.576+0000 7fe2c6a00700 1 -- 172.21.15.117:0/4050282157 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe2b8030080 con 0x7fe2d00764c0 2024-06-03T03:24:29.808 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:29.808+0000 7fe2d4c00700 1 -- 172.21.15.117:0/4050282157 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7fe2d00630a0 con 0x7fe2d00764c0 2024-06-03T03:24:29.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:29.809+0000 7fe2c6a00700 1 -- 172.21.15.117:0/4050282157 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v99) ==== 56+0+5680 (secure 0 0 0) 0x7fe2b8013070 con 0x7fe2d00764c0 2024-06-03T03:24:29.811 INFO:teuthology.orchestra.run.smithi117.stdout:epoch 99 2024-06-03T03:24:29.811 INFO:teuthology.orchestra.run.smithi117.stdout:fsid a0e8473a-2156-11ef-bc9b-c7b262605968 2024-06-03T03:24:29.811 INFO:teuthology.orchestra.run.smithi117.stdout:created 2024-06-03T03:10:08.446859+0000 2024-06-03T03:24:29.811 INFO:teuthology.orchestra.run.smithi117.stdout:modified 2024-06-03T03:21:25.966405+0000 2024-06-03T03:24:29.811 INFO:teuthology.orchestra.run.smithi117.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-06-03T03:24:29.812 INFO:teuthology.orchestra.run.smithi117.stdout:crush_version 27 2024-06-03T03:24:29.812 INFO:teuthology.orchestra.run.smithi117.stdout:full_ratio 0.95 2024-06-03T03:24:29.812 INFO:teuthology.orchestra.run.smithi117.stdout:backfillfull_ratio 0.9 2024-06-03T03:24:29.812 INFO:teuthology.orchestra.run.smithi117.stdout:nearfull_ratio 0.85 2024-06-03T03:24:29.812 INFO:teuthology.orchestra.run.smithi117.stdout:require_min_compat_client luminous 2024-06-03T03:24:29.812 INFO:teuthology.orchestra.run.smithi117.stdout:min_compat_client luminous 2024-06-03T03:24:29.812 INFO:teuthology.orchestra.run.smithi117.stdout:require_osd_release squid 2024-06-03T03:24:29.812 INFO:teuthology.orchestra.run.smithi117.stdout:stretch_mode_enabled false 2024-06-03T03:24:29.812 INFO:teuthology.orchestra.run.smithi117.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-06-03T03:24:29.812 INFO:teuthology.orchestra.run.smithi117.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 88 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-06-03T03:24:29.812 INFO:teuthology.orchestra.run.smithi117.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 88 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-06-03T03:24:29.812 INFO:teuthology.orchestra.run.smithi117.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 95 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-06-03T03:24:29.812 INFO:teuthology.orchestra.run.smithi117.stdout:max_osd 12 2024-06-03T03:24:29.813 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 up in weight 1 up_from 8 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.117:6802/4286644492,v1:172.21.15.117:6803/4286644492] [v2:172.21.15.117:6804/4286644492,v1:172.21.15.117:6805/4286644492] exists,up 8fefa089-9401-4a1c-b9fb-fe30c63e6a1d 2024-06-03T03:24:29.813 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 up in weight 1 up_from 13 up_thru 96 down_at 0 last_clean_interval [0,0) [v2:172.21.15.151:6800/262491158,v1:172.21.15.151:6801/262491158] [v2:172.21.15.151:6802/262491158,v1:172.21.15.151:6803/262491158] exists,up 5b12b371-f155-4b67-a470-25cbe59a2999 2024-06-03T03:24:29.813 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 up in weight 1 up_from 18 up_thru 96 down_at 0 last_clean_interval [0,0) [v2:172.21.15.183:6800/3542453623,v1:172.21.15.183:6801/3542453623] [v2:172.21.15.183:6802/3542453623,v1:172.21.15.183:6803/3542453623] exists,up dbc17818-c54c-4ecf-b656-03578b95d8c4 2024-06-03T03:24:29.813 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 up in weight 1 up_from 25 up_thru 96 down_at 0 last_clean_interval [0,0) [v2:172.21.15.117:6810/589012315,v1:172.21.15.117:6811/589012315] [v2:172.21.15.117:6812/589012315,v1:172.21.15.117:6813/589012315] exists,up ca08d2ac-b1d6-4a49-9ae8-e3ec146899a7 2024-06-03T03:24:29.813 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 up in weight 1 up_from 33 up_thru 98 down_at 0 last_clean_interval [0,0) [v2:172.21.15.151:6808/1239048647,v1:172.21.15.151:6809/1239048647] [v2:172.21.15.151:6810/1239048647,v1:172.21.15.151:6811/1239048647] exists,up 810d6c7a-023f-4930-82af-bc2bbfd647da 2024-06-03T03:24:29.813 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 up in weight 1 up_from 38 up_thru 96 down_at 0 last_clean_interval [0,0) [v2:172.21.15.183:6808/2985874686,v1:172.21.15.183:6809/2985874686] [v2:172.21.15.183:6810/2985874686,v1:172.21.15.183:6811/2985874686] exists,up 8ab6a5f0-34bb-48d8-9471-a7dc836e486e 2024-06-03T03:24:29.813 INFO:teuthology.orchestra.run.smithi117.stdout:osd.6 up in weight 1 up_from 46 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.117:6818/4250583193,v1:172.21.15.117:6819/4250583193] [v2:172.21.15.117:6820/4250583193,v1:172.21.15.117:6821/4250583193] exists,up 78d15739-e9e0-4f19-ae5f-ee8527a5b527 2024-06-03T03:24:29.813 INFO:teuthology.orchestra.run.smithi117.stdout:osd.7 up in weight 1 up_from 51 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.151:6816/386094645,v1:172.21.15.151:6817/386094645] [v2:172.21.15.151:6818/386094645,v1:172.21.15.151:6819/386094645] exists,up 3724fe1f-f4d7-443b-aea8-ff3ab02bfdc6 2024-06-03T03:24:29.813 INFO:teuthology.orchestra.run.smithi117.stdout:osd.8 up in weight 1 up_from 59 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.183:6816/1829748222,v1:172.21.15.183:6817/1829748222] [v2:172.21.15.183:6818/1829748222,v1:172.21.15.183:6819/1829748222] exists,up 006ac092-f015-448e-a0c0-f7b0c4597a18 2024-06-03T03:24:29.813 INFO:teuthology.orchestra.run.smithi117.stdout:osd.9 up in weight 1 up_from 67 up_thru 96 down_at 0 last_clean_interval [0,0) [v2:172.21.15.117:6826/2538315141,v1:172.21.15.117:6827/2538315141] [v2:172.21.15.117:6828/2538315141,v1:172.21.15.117:6829/2538315141] exists,up 00a970e6-2502-459e-975c-badac93c9bd0 2024-06-03T03:24:29.813 INFO:teuthology.orchestra.run.smithi117.stdout:osd.10 up in weight 1 up_from 72 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.151:6824/3801045133,v1:172.21.15.151:6825/3801045133] [v2:172.21.15.151:6826/3801045133,v1:172.21.15.151:6827/3801045133] exists,up ba2333cd-32a3-41ef-9ea5-287c707f512b 2024-06-03T03:24:29.813 INFO:teuthology.orchestra.run.smithi117.stdout:osd.11 up in weight 1 up_from 80 up_thru 96 down_at 0 last_clean_interval [0,0) [v2:172.21.15.183:6824/3513899696,v1:172.21.15.183:6825/3513899696] [v2:172.21.15.183:6826/3513899696,v1:172.21.15.183:6827/3513899696] exists,up 1556d654-e6fa-42c1-b399-aa53f6837a1e 2024-06-03T03:24:29.813 INFO:teuthology.orchestra.run.smithi117.stdout:pg_upmap_items 2.8 [2,11] 2024-06-03T03:24:29.813 INFO:teuthology.orchestra.run.smithi117.stdout:pg_upmap_items 2.b [8,11] 2024-06-03T03:24:29.814 INFO:teuthology.orchestra.run.smithi117.stdout:pg_upmap_items 2.c [2,4] 2024-06-03T03:24:29.814 INFO:teuthology.orchestra.run.smithi117.stdout:pg_upmap_items 3.e [5,9] 2024-06-03T03:24:29.814 INFO:teuthology.orchestra.run.smithi117.stdout:pg_upmap_items 3.18 [3,9] 2024-06-03T03:24:29.814 INFO:teuthology.orchestra.run.smithi117.stdout:pg_upmap_items 3.1b [4,6] 2024-06-03T03:24:29.814 INFO:teuthology.orchestra.run.smithi117.stdout:pg_upmap_items 3.27 [4,8] 2024-06-03T03:24:29.814 INFO:teuthology.orchestra.run.smithi117.stdout:pg_upmap_items 3.34 [4,9] 2024-06-03T03:24:29.814 INFO:teuthology.orchestra.run.smithi117.stdout:pg_upmap_items 4.e [5,8] 2024-06-03T03:24:29.814 INFO:teuthology.orchestra.run.smithi117.stdout:pg_upmap_items 4.13 [9,10] 2024-06-03T03:24:29.814 INFO:teuthology.orchestra.run.smithi117.stdout:pg_upmap_items 4.19 [9,8] 2024-06-03T03:24:29.814 INFO:teuthology.orchestra.run.smithi117.stdout:pg_upmap_items 4.1c [9,8] 2024-06-03T03:24:29.814 INFO:teuthology.orchestra.run.smithi117.stdout:blocklist 172.21.15.117:0/186309008 expires 2024-06-04T03:11:24.655516+0000 2024-06-03T03:24:29.814 INFO:teuthology.orchestra.run.smithi117.stdout:blocklist 172.21.15.117:6801/2142313590 expires 2024-06-04T03:11:24.655516+0000 2024-06-03T03:24:29.814 INFO:teuthology.orchestra.run.smithi117.stdout:blocklist 172.21.15.117:6800/2142313590 expires 2024-06-04T03:11:24.655516+0000 2024-06-03T03:24:29.815 INFO:teuthology.orchestra.run.smithi117.stdout:blocklist 172.21.15.117:0/2399725156 expires 2024-06-04T03:11:24.655516+0000 2024-06-03T03:24:29.815 INFO:teuthology.orchestra.run.smithi117.stdout:blocklist 172.21.15.117:0/1453232061 expires 2024-06-04T03:10:50.552899+0000 2024-06-03T03:24:29.815 INFO:teuthology.orchestra.run.smithi117.stdout:blocklist 172.21.15.117:0/4281674974 expires 2024-06-04T03:10:50.552899+0000 2024-06-03T03:24:29.815 INFO:teuthology.orchestra.run.smithi117.stdout:blocklist 172.21.15.117:6800/2424554985 expires 2024-06-04T03:10:50.552899+0000 2024-06-03T03:24:29.815 INFO:teuthology.orchestra.run.smithi117.stdout:blocklist 172.21.15.117:0/3946365225 expires 2024-06-04T03:11:24.655516+0000 2024-06-03T03:24:29.815 INFO:teuthology.orchestra.run.smithi117.stdout:blocklist 172.21.15.117:0/1146162354 expires 2024-06-04T03:10:50.552899+0000 2024-06-03T03:24:29.815 INFO:teuthology.orchestra.run.smithi117.stdout:blocklist 172.21.15.117:6801/2424554985 expires 2024-06-04T03:10:50.552899+0000 2024-06-03T03:24:29.816 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:29.813+0000 7fe2d4c00700 1 -- 172.21.15.117:0/4050282157 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fe2b40a2c20 msgr2=0x7fe2b40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:29.816 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:29.813+0000 7fe2d4c00700 1 --2- 172.21.15.117:0/4050282157 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fe2b40a2c20 0x7fe2b40a50e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe2d00768a0 tx=0x7fe2c0009260 comp rx=0 tx=0).stop 2024-06-03T03:24:29.816 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:29.813+0000 7fe2d4c00700 1 -- 172.21.15.117:0/4050282157 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe2d00764c0 msgr2=0x7fe2d019aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:29.816 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:29.813+0000 7fe2d4c00700 1 --2- 172.21.15.117:0/4050282157 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe2d00764c0 0x7fe2d019aee0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe2b8000c00 tx=0x7fe2b800bb90 comp rx=0 tx=0).stop 2024-06-03T03:24:29.816 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:29.813+0000 7fe2d4c00700 1 -- 172.21.15.117:0/4050282157 shutdown_connections 2024-06-03T03:24:29.816 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:29.813+0000 7fe2d4c00700 1 --2- 172.21.15.117:0/4050282157 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fe2b40a2c20 0x7fe2b40a50e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:29.816 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:29.813+0000 7fe2d4c00700 1 --2- 172.21.15.117:0/4050282157 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe2d010a3d0 0x7fe2d01a1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:29.816 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:29.813+0000 7fe2d4c00700 1 --2- 172.21.15.117:0/4050282157 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2d0074ef0 0x7fe2d019a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:29.816 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:29.813+0000 7fe2d4c00700 1 --2- 172.21.15.117:0/4050282157 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe2d00764c0 0x7fe2d019aee0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:29.816 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:29.813+0000 7fe2d4c00700 1 -- 172.21.15.117:0/4050282157 >> 172.21.15.117:0/4050282157 conn(0x7fe2d00fda60 msgr2=0x7fe2d01062d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:24:29.816 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:29.813+0000 7fe2d4c00700 1 -- 172.21.15.117:0/4050282157 shutdown_connections 2024-06-03T03:24:29.816 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:29.814+0000 7fe2d4c00700 1 -- 172.21.15.117:0/4050282157 wait complete. 2024-06-03T03:24:30.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:24:29 smithi117 ceph-mon[31649]: pgmap v481: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-03T03:24:30.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:24:29 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/4050282157' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-06-03T03:24:30.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:24:29 smithi151 ceph-mon[37043]: pgmap v481: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-03T03:24:30.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:24:29 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/4050282157' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-06-03T03:24:30.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:24:29 smithi183 ceph-mon[37326]: pgmap v481: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-03T03:24:30.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:24:29 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/4050282157' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-06-03T03:24:30.838 INFO:teuthology.run_tasks:Running task fs.ready... 2024-06-03T03:24:30.849 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-03T03:24:31.294 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:24:32.375 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:24:31 smithi117 ceph-mon[31649]: pgmap v482: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:24:32.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:24:31 smithi151 ceph-mon[37043]: pgmap v482: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:24:32.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:24:31 smithi183 ceph-mon[37326]: pgmap v482: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:24:33.784 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:33.783+0000 7faae7200700 1 -- 172.21.15.117:0/2727949353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faae00746c0 msgr2=0x7faae0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:33.784 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:33.783+0000 7faae7200700 1 --2- 172.21.15.117:0/2727949353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faae00746c0 0x7faae0074aa0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7faad000b0a0 tx=0x7faad000b3b0 comp rx=0 tx=0).stop 2024-06-03T03:24:33.784 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:33.783+0000 7faae7200700 1 -- 172.21.15.117:0/2727949353 shutdown_connections 2024-06-03T03:24:33.784 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:33.783+0000 7faae7200700 1 --2- 172.21.15.117:0/2727949353 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faae0113660 0x7faae0115a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:33.784 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:33.783+0000 7faae7200700 1 --2- 172.21.15.117:0/2727949353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faae00746c0 0x7faae0074aa0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:33.784 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:33.783+0000 7faae7200700 1 --2- 172.21.15.117:0/2727949353 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faae0074fe0 0x7faae0113120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:33.784 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:33.783+0000 7faae7200700 1 -- 172.21.15.117:0/2727949353 >> 172.21.15.117:0/2727949353 conn(0x7faae00795f0 msgr2=0x7faae0079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:24:33.784 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:33.783+0000 7faae7200700 1 -- 172.21.15.117:0/2727949353 shutdown_connections 2024-06-03T03:24:33.784 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:33.783+0000 7faae7200700 1 -- 172.21.15.117:0/2727949353 wait complete. 2024-06-03T03:24:33.785 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:33.784+0000 7faae7200700 1 Processor -- start 2024-06-03T03:24:33.785 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:33.784+0000 7faae7200700 1 -- start start 2024-06-03T03:24:33.785 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:33.785+0000 7faae7200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faae00746c0 0x7faae019f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:33.785 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:33.785+0000 7faae7200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faae0074fe0 0x7faae019f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:33.786 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:33.785+0000 7faae7200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faae0113660 0x7faae01a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:33.786 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:33.785+0000 7faae7200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7faae0118dc0 con 0x7faae0074fe0 2024-06-03T03:24:33.786 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:33.785+0000 7faae7200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7faae0118c40 con 0x7faae0113660 2024-06-03T03:24:33.786 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:33.785+0000 7faae7200700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7faae0118f40 con 0x7faae00746c0 2024-06-03T03:24:33.787 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:33.785+0000 7faad7400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faae0074fe0 0x7faae019f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:33.787 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:33.785+0000 7faad7400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faae0074fe0 0x7faae019f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:45260/0 (socket says 172.21.15.117:45260) 2024-06-03T03:24:33.787 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:33.785+0000 7faad7400700 1 -- 172.21.15.117:0/535247875 learned_addr learned my addr 172.21.15.117:0/535247875 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:24:33.787 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:33.785+0000 7faae5200700 1 --2- 172.21.15.117:0/535247875 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faae0113660 0x7faae01a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:33.787 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:33.785+0000 7faadfe00700 1 --2- 172.21.15.117:0/535247875 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faae00746c0 0x7faae019f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:33.787 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:33.786+0000 7faad7400700 1 -- 172.21.15.117:0/535247875 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faae00746c0 msgr2=0x7faae019f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:33.787 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:33.786+0000 7faad7400700 1 --2- 172.21.15.117:0/535247875 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faae00746c0 0x7faae019f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:33.787 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:33.786+0000 7faad7400700 1 -- 172.21.15.117:0/535247875 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faae0113660 msgr2=0x7faae01a3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:33.788 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:33.786+0000 7faad7400700 1 --2- 172.21.15.117:0/535247875 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faae0113660 0x7faae01a3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:33.788 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:33.786+0000 7faad7400700 1 -- 172.21.15.117:0/535247875 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faae01a4230 con 0x7faae0074fe0 2024-06-03T03:24:33.788 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:33.786+0000 7faae5200700 1 --2- 172.21.15.117:0/535247875 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faae0113660 0x7faae01a3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:24:33.788 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:33.786+0000 7faad7400700 1 --2- 172.21.15.117:0/535247875 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faae0074fe0 0x7faae019f7b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7faac800eb30 tx=0x7faac800ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:24:33.788 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:33.786+0000 7faadd600700 1 -- 172.21.15.117:0/535247875 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac800cbd0 con 0x7faae0074fe0 2024-06-03T03:24:33.788 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:33.786+0000 7faadd600700 1 -- 172.21.15.117:0/535247875 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faac8004500 con 0x7faae0074fe0 2024-06-03T03:24:33.788 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:33.786+0000 7faae7200700 1 -- 172.21.15.117:0/535247875 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faae01a4520 con 0x7faae0074fe0 2024-06-03T03:24:33.788 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:33.786+0000 7faadd600700 1 -- 172.21.15.117:0/535247875 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac8010430 con 0x7faae0074fe0 2024-06-03T03:24:33.788 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:33.787+0000 7faae7200700 1 -- 172.21.15.117:0/535247875 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faae01abda0 con 0x7faae0074fe0 2024-06-03T03:24:33.788 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:33.788+0000 7faae7200700 1 -- 172.21.15.117:0/535247875 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faae0069180 con 0x7faae0074fe0 2024-06-03T03:24:33.790 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:33.790+0000 7faadd600700 1 -- 172.21.15.117:0/535247875 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7faac8003f00 con 0x7faae0074fe0 2024-06-03T03:24:33.791 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:33.791+0000 7faadd600700 1 --2- 172.21.15.117:0/535247875 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7faac40a2ca0 0x7faac40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:33.792 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:33.791+0000 7faadd600700 1 -- 172.21.15.117:0/535247875 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7faac80cf710 con 0x7faae0074fe0 2024-06-03T03:24:33.792 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:33.791+0000 7faadfe00700 1 --2- 172.21.15.117:0/535247875 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7faac40a2ca0 0x7faac40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:33.792 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:33.792+0000 7faadfe00700 1 --2- 172.21.15.117:0/535247875 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7faac40a2ca0 0x7faac40a5160 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7faad0009e60 tx=0x7faad000b700 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:24:33.796 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:33.796+0000 7faadd600700 1 -- 172.21.15.117:0/535247875 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7faac809c280 con 0x7faae0074fe0 2024-06-03T03:24:34.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:34.054+0000 7faae7200700 1 -- 172.21.15.117:0/535247875 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faae01a0580 con 0x7faae0074fe0 2024-06-03T03:24:34.056 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:34.056+0000 7faadd600700 1 -- 172.21.15.117:0/535247875 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11955 (secure 0 0 0) 0x7faae01a0580 con 0x7faae0074fe0 2024-06-03T03:24:34.057 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T03:24:34.057 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3353001119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3353001119},{"type":"v1","addr":"172.21.15.117:6837","nonce":3353001119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14388,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6839/405782304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":405782304},{"type":"v1","addr":"172.21.15.117:6839","nonce":405782304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-03T03:19:44.629822+0000","modified":"2024-06-03T03:21:12.085445+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24517,"mds_1":24512,"mds_2":24503,"mds_3":24499,"mds_4":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24431":{"gid":24431,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6833/3355662416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3355662416},{"type":"v1","addr":"172.21.15.151:6833","nonce":3355662416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/2429725085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2429725085},{"type":"v1","addr":"172.21.15.183:6833","nonce":2429725085}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/31153579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":31153579},{"type":"v1","addr":"172.21.15.151:6835","nonce":31153579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"j","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/2884460088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2884460088},{"type":"v1","addr":"172.21.15.117:6835","nonce":2884460088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/2392407848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":2392407848},{"type":"v1","addr":"172.21.15.151:6837","nonce":2392407848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":32,"addr":"172.21.15.183:6835/1009004320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1009004320},{"type":"v1","addr":"172.21.15.183:6835","nonce":1009004320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.183:6837/2697518526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":2697518526},{"type":"v1","addr":"172.21.15.183:6837","nonce":2697518526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"a","rank":2,"incarnation":22,"state":"up:active","state_seq":27,"addr":"172.21.15.117:6841/1212242967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6840","nonce":1212242967},{"type":"v1","addr":"172.21.15.117:6841","nonce":1212242967}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.151:6839/3623444616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":3623444616},{"type":"v1","addr":"172.21.15.151:6839","nonce":3623444616}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":10,"addr":"172.21.15.183:6839/1439615083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1439615083},{"type":"v1","addr":"172.21.15.183:6839","nonce":1439615083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24517,"qdb_cluster":[24503,24517,24512,24487,24499]},"id":1}]} 2024-06-03T03:24:34.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:34.058+0000 7faae7200700 1 -- 172.21.15.117:0/535247875 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7faac40a2ca0 msgr2=0x7faac40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:34.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:34.058+0000 7faae7200700 1 --2- 172.21.15.117:0/535247875 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7faac40a2ca0 0x7faac40a5160 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7faad0009e60 tx=0x7faad000b700 comp rx=0 tx=0).stop 2024-06-03T03:24:34.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:34.058+0000 7faae7200700 1 -- 172.21.15.117:0/535247875 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faae0074fe0 msgr2=0x7faae019f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:34.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:34.058+0000 7faae7200700 1 --2- 172.21.15.117:0/535247875 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faae0074fe0 0x7faae019f7b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7faac800eb30 tx=0x7faac800ee40 comp rx=0 tx=0).stop 2024-06-03T03:24:34.060 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:34.059+0000 7faae7200700 1 -- 172.21.15.117:0/535247875 shutdown_connections 2024-06-03T03:24:34.060 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:34.059+0000 7faae7200700 1 --2- 172.21.15.117:0/535247875 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7faac40a2ca0 0x7faac40a5160 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:34.060 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:34.059+0000 7faae7200700 1 --2- 172.21.15.117:0/535247875 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faae0113660 0x7faae01a3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:34.060 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:34.059+0000 7faae7200700 1 --2- 172.21.15.117:0/535247875 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faae00746c0 0x7faae019f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:34.060 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:34.059+0000 7faae7200700 1 --2- 172.21.15.117:0/535247875 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faae0074fe0 0x7faae019f7b0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:34.060 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:34.059+0000 7faae7200700 1 -- 172.21.15.117:0/535247875 >> 172.21.15.117:0/535247875 conn(0x7faae00795f0 msgr2=0x7faae0112420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:24:34.060 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:34.059+0000 7faae7200700 1 -- 172.21.15.117:0/535247875 shutdown_connections 2024-06-03T03:24:34.060 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:34.059+0000 7faae7200700 1 -- 172.21.15.117:0/535247875 wait complete. 2024-06-03T03:24:34.061 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 30 2024-06-03T03:24:34.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:24:33 smithi151 ceph-mon[37043]: pgmap v483: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-06-03T03:24:34.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:24:33 smithi117 ceph-mon[31649]: pgmap v483: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-06-03T03:24:34.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:24:33 smithi183 ceph-mon[37326]: pgmap v483: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-06-03T03:24:34.977 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-03T03:24:35.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:24:35 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/535247875' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T03:24:35.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:24:35 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/535247875' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T03:24:35.425 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:24:35.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:24:35 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/535247875' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T03:24:36.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:24:36 smithi117 ceph-mon[31649]: pgmap v484: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:24:36.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:24:36 smithi151 ceph-mon[37043]: pgmap v484: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:24:36.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:24:36 smithi183 ceph-mon[37326]: pgmap v484: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:24:37.876 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:37.874+0000 7fb631400700 1 -- 172.21.15.117:0/3663313798 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb62c102740 msgr2=0x7fb62c102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:37.876 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:37.874+0000 7fb631400700 1 --2- 172.21.15.117:0/3663313798 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb62c102740 0x7fb62c102b20 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb61c0099c0 tx=0x7fb61c009cd0 comp rx=0 tx=0).stop 2024-06-03T03:24:37.876 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:37.874+0000 7fb631400700 1 -- 172.21.15.117:0/3663313798 shutdown_connections 2024-06-03T03:24:37.876 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:37.874+0000 7fb631400700 1 --2- 172.21.15.117:0/3663313798 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb62c107840 0x7fb62c10f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:37.876 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:37.874+0000 7fb631400700 1 --2- 172.21.15.117:0/3663313798 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb62c102740 0x7fb62c102b20 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:37.876 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:37.874+0000 7fb631400700 1 --2- 172.21.15.117:0/3663313798 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb62c1030f0 0x7fb62c107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:37.876 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:37.874+0000 7fb631400700 1 -- 172.21.15.117:0/3663313798 >> 172.21.15.117:0/3663313798 conn(0x7fb62c0795f0 msgr2=0x7fb62c079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:24:37.876 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:37.875+0000 7fb631400700 1 -- 172.21.15.117:0/3663313798 shutdown_connections 2024-06-03T03:24:37.876 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:37.875+0000 7fb631400700 1 -- 172.21.15.117:0/3663313798 wait complete. 2024-06-03T03:24:37.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:37.876+0000 7fb631400700 1 Processor -- start 2024-06-03T03:24:37.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:37.876+0000 7fb631400700 1 -- start start 2024-06-03T03:24:37.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:37.876+0000 7fb631400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb62c102740 0x7fb62c074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:37.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:37.876+0000 7fb631400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb62c1030f0 0x7fb62c074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:37.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:37.876+0000 7fb631400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb62c107840 0x7fb62c075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:37.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:37.876+0000 7fb631400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb62c112760 con 0x7fb62c102740 2024-06-03T03:24:37.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:37.876+0000 7fb631400700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb62c1125e0 con 0x7fb62c1030f0 2024-06-03T03:24:37.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:37.876+0000 7fb631400700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb62c1128e0 con 0x7fb62c107840 2024-06-03T03:24:37.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:37.877+0000 7fb62a800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb62c102740 0x7fb62c074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:37.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:37.877+0000 7fb62a800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb62c102740 0x7fb62c074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44654/0 (socket says 172.21.15.117:44654) 2024-06-03T03:24:37.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:37.877+0000 7fb62a800700 1 -- 172.21.15.117:0/1099432369 learned_addr learned my addr 172.21.15.117:0/1099432369 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:24:37.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:37.877+0000 7fb623e00700 1 --2- 172.21.15.117:0/1099432369 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb62c1030f0 0x7fb62c074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:37.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:37.877+0000 7fb62b200700 1 --2- 172.21.15.117:0/1099432369 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb62c107840 0x7fb62c075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:37.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:37.877+0000 7fb62a800700 1 -- 172.21.15.117:0/1099432369 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb62c107840 msgr2=0x7fb62c075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:37.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:37.877+0000 7fb62a800700 1 --2- 172.21.15.117:0/1099432369 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb62c107840 0x7fb62c075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:37.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:37.877+0000 7fb62a800700 1 -- 172.21.15.117:0/1099432369 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb62c1030f0 msgr2=0x7fb62c074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:37.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:37.877+0000 7fb62a800700 1 --2- 172.21.15.117:0/1099432369 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb62c1030f0 0x7fb62c074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:37.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:37.877+0000 7fb62a800700 1 -- 172.21.15.117:0/1099432369 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb62c1a7120 con 0x7fb62c102740 2024-06-03T03:24:37.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:37.877+0000 7fb623e00700 1 --2- 172.21.15.117:0/1099432369 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb62c1030f0 0x7fb62c074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T03:24:37.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:37.877+0000 7fb62a800700 1 --2- 172.21.15.117:0/1099432369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb62c102740 0x7fb62c074790 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb61c006010 tx=0x7fb61c003e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:24:37.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:37.878+0000 7fb623400700 1 -- 172.21.15.117:0/1099432369 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb61c009e60 con 0x7fb62c102740 2024-06-03T03:24:37.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:37.878+0000 7fb623400700 1 -- 172.21.15.117:0/1099432369 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb61c003680 con 0x7fb62c102740 2024-06-03T03:24:37.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:37.878+0000 7fb623400700 1 -- 172.21.15.117:0/1099432369 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb61c0173f0 con 0x7fb62c102740 2024-06-03T03:24:37.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:37.878+0000 7fb631400700 1 -- 172.21.15.117:0/1099432369 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb62c1a73b0 con 0x7fb62c102740 2024-06-03T03:24:37.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:37.878+0000 7fb631400700 1 -- 172.21.15.117:0/1099432369 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb62c1a77b0 con 0x7fb62c102740 2024-06-03T03:24:37.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:37.879+0000 7fb631400700 1 -- 172.21.15.117:0/1099432369 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb62c10aa80 con 0x7fb62c102740 2024-06-03T03:24:37.882 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:37.881+0000 7fb623400700 1 -- 172.21.15.117:0/1099432369 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fb61c00f660 con 0x7fb62c102740 2024-06-03T03:24:37.883 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:37.883+0000 7fb623400700 1 --2- 172.21.15.117:0/1099432369 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fb6100a2ca0 0x7fb6100a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:37.883 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:37.883+0000 7fb623400700 1 -- 172.21.15.117:0/1099432369 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7fb61c0d0180 con 0x7fb62c102740 2024-06-03T03:24:37.883 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:37.883+0000 7fb623e00700 1 --2- 172.21.15.117:0/1099432369 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fb6100a2ca0 0x7fb6100a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:37.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:37.884+0000 7fb623e00700 1 --2- 172.21.15.117:0/1099432369 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fb6100a2ca0 0x7fb6100a5160 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb62c100f60 tx=0x7fb61400b5f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:24:37.888 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:37.887+0000 7fb623400700 1 -- 172.21.15.117:0/1099432369 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb61c013070 con 0x7fb62c102740 2024-06-03T03:24:38.166 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:38.165+0000 7fb631400700 1 -- 172.21.15.117:0/1099432369 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb62c10a460 con 0x7fb62c102740 2024-06-03T03:24:38.167 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:38.167+0000 7fb623400700 1 -- 172.21.15.117:0/1099432369 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11955 (secure 0 0 0) 0x7fb61c09cd80 con 0x7fb62c102740 2024-06-03T03:24:38.168 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T03:24:38.168 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3353001119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3353001119},{"type":"v1","addr":"172.21.15.117:6837","nonce":3353001119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14388,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6839/405782304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":405782304},{"type":"v1","addr":"172.21.15.117:6839","nonce":405782304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-03T03:19:44.629822+0000","modified":"2024-06-03T03:21:12.085445+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24517,"mds_1":24512,"mds_2":24503,"mds_3":24499,"mds_4":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24431":{"gid":24431,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6833/3355662416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3355662416},{"type":"v1","addr":"172.21.15.151:6833","nonce":3355662416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/2429725085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2429725085},{"type":"v1","addr":"172.21.15.183:6833","nonce":2429725085}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/31153579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":31153579},{"type":"v1","addr":"172.21.15.151:6835","nonce":31153579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"j","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/2884460088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2884460088},{"type":"v1","addr":"172.21.15.117:6835","nonce":2884460088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/2392407848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":2392407848},{"type":"v1","addr":"172.21.15.151:6837","nonce":2392407848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":32,"addr":"172.21.15.183:6835/1009004320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1009004320},{"type":"v1","addr":"172.21.15.183:6835","nonce":1009004320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.183:6837/2697518526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":2697518526},{"type":"v1","addr":"172.21.15.183:6837","nonce":2697518526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"a","rank":2,"incarnation":22,"state":"up:active","state_seq":27,"addr":"172.21.15.117:6841/1212242967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6840","nonce":1212242967},{"type":"v1","addr":"172.21.15.117:6841","nonce":1212242967}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.151:6839/3623444616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":3623444616},{"type":"v1","addr":"172.21.15.151:6839","nonce":3623444616}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":10,"addr":"172.21.15.183:6839/1439615083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1439615083},{"type":"v1","addr":"172.21.15.183:6839","nonce":1439615083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24517,"qdb_cluster":[24503,24517,24512,24487,24499]},"id":1}]} 2024-06-03T03:24:38.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:38.170+0000 7fb631400700 1 -- 172.21.15.117:0/1099432369 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fb6100a2ca0 msgr2=0x7fb6100a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:38.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:38.170+0000 7fb631400700 1 --2- 172.21.15.117:0/1099432369 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fb6100a2ca0 0x7fb6100a5160 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb62c100f60 tx=0x7fb61400b5f0 comp rx=0 tx=0).stop 2024-06-03T03:24:38.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:38.170+0000 7fb631400700 1 -- 172.21.15.117:0/1099432369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb62c102740 msgr2=0x7fb62c074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:38.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:38.170+0000 7fb631400700 1 --2- 172.21.15.117:0/1099432369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb62c102740 0x7fb62c074790 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb61c006010 tx=0x7fb61c003e00 comp rx=0 tx=0).stop 2024-06-03T03:24:38.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:38.170+0000 7fb631400700 1 -- 172.21.15.117:0/1099432369 shutdown_connections 2024-06-03T03:24:38.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:38.170+0000 7fb631400700 1 --2- 172.21.15.117:0/1099432369 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fb6100a2ca0 0x7fb6100a5160 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:38.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:38.170+0000 7fb631400700 1 --2- 172.21.15.117:0/1099432369 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb62c107840 0x7fb62c075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:38.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:38.170+0000 7fb631400700 1 --2- 172.21.15.117:0/1099432369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb62c102740 0x7fb62c074790 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:38.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:38.170+0000 7fb631400700 1 --2- 172.21.15.117:0/1099432369 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb62c1030f0 0x7fb62c074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:38.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:38.170+0000 7fb631400700 1 -- 172.21.15.117:0/1099432369 >> 172.21.15.117:0/1099432369 conn(0x7fb62c0795f0 msgr2=0x7fb62c101c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:24:38.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:38.170+0000 7fb631400700 1 -- 172.21.15.117:0/1099432369 shutdown_connections 2024-06-03T03:24:38.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:38.170+0000 7fb631400700 1 -- 172.21.15.117:0/1099432369 wait complete. 2024-06-03T03:24:38.172 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 30 2024-06-03T03:24:38.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:24:38 smithi151 ceph-mon[37043]: pgmap v485: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-03T03:24:38.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:24:38 smithi117 ceph-mon[31649]: pgmap v485: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-03T03:24:38.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:24:38 smithi183 ceph-mon[37326]: pgmap v485: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-03T03:24:39.293 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:24:39 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/1099432369' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T03:24:39.297 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-03T03:24:39.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:24:39 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/1099432369' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T03:24:39.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:24:39 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/1099432369' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T03:24:39.747 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:24:40.149 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:24:40 smithi117 ceph-mon[31649]: pgmap v486: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-03T03:24:40.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:24:40 smithi151 ceph-mon[37043]: pgmap v486: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-03T03:24:40.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:24:40 smithi183 ceph-mon[37326]: pgmap v486: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-03T03:24:41.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:24:41 smithi117 ceph-mon[31649]: pgmap v487: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:24:41.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:24:41 smithi151 ceph-mon[37043]: pgmap v487: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:24:41.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:24:41 smithi183 ceph-mon[37326]: pgmap v487: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:24:42.322 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:42.320+0000 7f2136800700 1 -- 172.21.15.117:0/671002729 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f213006b5a0 msgr2=0x7f213010f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:42.322 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:42.320+0000 7f2136800700 1 --2- 172.21.15.117:0/671002729 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f213006b5a0 0x7f213010f320 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f212800b0a0 tx=0x7f212800b3b0 comp rx=0 tx=0).stop 2024-06-03T03:24:42.322 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:42.321+0000 7f2136800700 1 -- 172.21.15.117:0/671002729 shutdown_connections 2024-06-03T03:24:42.322 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:42.321+0000 7f2136800700 1 --2- 172.21.15.117:0/671002729 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f213006b5a0 0x7f213010f320 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:42.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:42.321+0000 7f2136800700 1 --2- 172.21.15.117:0/671002729 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2130108520 0x7f2130108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:42.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:42.321+0000 7f2136800700 1 --2- 172.21.15.117:0/671002729 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f213006abe0 0x7f213006b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:42.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:42.321+0000 7f2136800700 1 -- 172.21.15.117:0/671002729 >> 172.21.15.117:0/671002729 conn(0x7f2130078330 msgr2=0x7f2130078740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:24:42.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:42.321+0000 7f2136800700 1 -- 172.21.15.117:0/671002729 shutdown_connections 2024-06-03T03:24:42.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:42.321+0000 7f2136800700 1 -- 172.21.15.117:0/671002729 wait complete. 2024-06-03T03:24:42.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:42.322+0000 7f2136800700 1 Processor -- start 2024-06-03T03:24:42.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:42.322+0000 7f2136800700 1 -- start start 2024-06-03T03:24:42.324 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:42.322+0000 7f2136800700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f213006abe0 0x7f213019e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:42.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:42.322+0000 7f2136800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f213006b5a0 0x7f213019edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:42.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:42.322+0000 7f2136800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2130108520 0x7f2130198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:42.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:42.322+0000 7f2136800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f21301127c0 con 0x7f213006b5a0 2024-06-03T03:24:42.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:42.322+0000 7f2136800700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2130112640 con 0x7f2130108520 2024-06-03T03:24:42.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:42.322+0000 7f2136800700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2130112940 con 0x7f213006abe0 2024-06-03T03:24:42.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:42.323+0000 7f2127e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f213006b5a0 0x7f213019edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:42.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:42.323+0000 7f212f400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f213006abe0 0x7f213019e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:42.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:42.323+0000 7f2127e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f213006b5a0 0x7f213019edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44666/0 (socket says 172.21.15.117:44666) 2024-06-03T03:24:42.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:42.323+0000 7f2127e00700 1 -- 172.21.15.117:0/4075245525 learned_addr learned my addr 172.21.15.117:0/4075245525 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:24:42.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:42.323+0000 7f212fe00700 1 --2- 172.21.15.117:0/4075245525 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2130108520 0x7f2130198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:42.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:42.323+0000 7f212f400700 1 -- 172.21.15.117:0/4075245525 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2130108520 msgr2=0x7f2130198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:42.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:42.323+0000 7f212f400700 1 --2- 172.21.15.117:0/4075245525 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2130108520 0x7f2130198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:42.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:42.323+0000 7f212f400700 1 -- 172.21.15.117:0/4075245525 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f213006b5a0 msgr2=0x7f213019edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:42.326 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:42.323+0000 7f212f400700 1 --2- 172.21.15.117:0/4075245525 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f213006b5a0 0x7f213019edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:42.326 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:42.323+0000 7f212f400700 1 -- 172.21.15.117:0/4075245525 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2130199200 con 0x7f213006abe0 2024-06-03T03:24:42.326 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:42.323+0000 7f212f400700 1 --2- 172.21.15.117:0/4075245525 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f213006abe0 0x7f213019e890 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f212000e9f0 tx=0x7f212000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:24:42.326 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:42.324+0000 7f212cc00700 1 -- 172.21.15.117:0/4075245525 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2120004220 con 0x7f213006abe0 2024-06-03T03:24:42.326 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:42.324+0000 7f212cc00700 1 -- 172.21.15.117:0/4075245525 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2120004d10 con 0x7f213006abe0 2024-06-03T03:24:42.326 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:42.324+0000 7f2136800700 1 -- 172.21.15.117:0/4075245525 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21301994f0 con 0x7f213006abe0 2024-06-03T03:24:42.326 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:42.324+0000 7f212cc00700 1 -- 172.21.15.117:0/4075245525 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2120005020 con 0x7f213006abe0 2024-06-03T03:24:42.326 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:42.324+0000 7f2136800700 1 -- 172.21.15.117:0/4075245525 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21301a5760 con 0x7f213006abe0 2024-06-03T03:24:42.326 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:42.325+0000 7f2136800700 1 -- 172.21.15.117:0/4075245525 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2130103bc0 con 0x7f213006abe0 2024-06-03T03:24:42.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:42.327+0000 7f212cc00700 1 -- 172.21.15.117:0/4075245525 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f2120004750 con 0x7f213006abe0 2024-06-03T03:24:42.329 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:42.328+0000 7f212cc00700 1 --2- 172.21.15.117:0/4075245525 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f21140a2ca0 0x7f21140a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:42.329 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:42.329+0000 7f212cc00700 1 -- 172.21.15.117:0/4075245525 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f21200cf250 con 0x7f213006abe0 2024-06-03T03:24:42.329 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:42.329+0000 7f2127e00700 1 --2- 172.21.15.117:0/4075245525 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f21140a2ca0 0x7f21140a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:42.330 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:42.329+0000 7f2127e00700 1 --2- 172.21.15.117:0/4075245525 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f21140a2ca0 0x7f21140a5160 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2118009680 tx=0x7f2118009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:24:42.331 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:42.331+0000 7f212cc00700 1 -- 172.21.15.117:0/4075245525 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f212009beb0 con 0x7f213006abe0 2024-06-03T03:24:42.554 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:42.552+0000 7f2136800700 1 -- 172.21.15.117:0/4075245525 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2130199fb0 con 0x7f213006abe0 2024-06-03T03:24:42.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:42.555+0000 7f212cc00700 1 -- 172.21.15.117:0/4075245525 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+24807 (secure 0 0 0) 0x7f2130199fb0 con 0x7f213006abe0 2024-06-03T03:24:42.557 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T03:24:42.558 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":99,"fsid":"a0e8473a-2156-11ef-bc9b-c7b262605968","created":"2024-06-03T03:10:08.446859+0000","modified":"2024-06-03T03:21:25.966405+0000","last_up_change":"2024-06-03T03:18:28.736904+0000","last_in_change":"2024-06-03T03:18:08.485711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-03T03:14:09.370986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-03T03:19:35.044683+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-03T03:19:40.097569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-03T03:19:55.394480+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8fefa089-9401-4a1c-b9fb-fe30c63e6a1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":4286644492},{"type":"v1","addr":"172.21.15.117:6803","nonce":4286644492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":4286644492},{"type":"v1","addr":"172.21.15.117:6805","nonce":4286644492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":4286644492},{"type":"v1","addr":"172.21.15.117:6809","nonce":4286644492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":4286644492},{"type":"v1","addr":"172.21.15.117:6807","nonce":4286644492}]},"public_addr":"172.21.15.117:6803/4286644492","cluster_addr":"172.21.15.117:6805/4286644492","heartbeat_back_addr":"172.21.15.117:6809/4286644492","heartbeat_front_addr":"172.21.15.117:6807/4286644492","state":["exists","up"]},{"osd":1,"uuid":"5b12b371-f155-4b67-a470-25cbe59a2999","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":262491158},{"type":"v1","addr":"172.21.15.151:6801","nonce":262491158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":262491158},{"type":"v1","addr":"172.21.15.151:6803","nonce":262491158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":262491158},{"type":"v1","addr":"172.21.15.151:6807","nonce":262491158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":262491158},{"type":"v1","addr":"172.21.15.151:6805","nonce":262491158}]},"public_addr":"172.21.15.151:6801/262491158","cluster_addr":"172.21.15.151:6803/262491158","heartbeat_back_addr":"172.21.15.151:6807/262491158","heartbeat_front_addr":"172.21.15.151:6805/262491158","state":["exists","up"]},{"osd":2,"uuid":"dbc17818-c54c-4ecf-b656-03578b95d8c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":3542453623},{"type":"v1","addr":"172.21.15.183:6801","nonce":3542453623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":3542453623},{"type":"v1","addr":"172.21.15.183:6803","nonce":3542453623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":3542453623},{"type":"v1","addr":"172.21.15.183:6807","nonce":3542453623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":3542453623},{"type":"v1","addr":"172.21.15.183:6805","nonce":3542453623}]},"public_addr":"172.21.15.183:6801/3542453623","cluster_addr":"172.21.15.183:6803/3542453623","heartbeat_back_addr":"172.21.15.183:6807/3542453623","heartbeat_front_addr":"172.21.15.183:6805/3542453623","state":["exists","up"]},{"osd":3,"uuid":"ca08d2ac-b1d6-4a49-9ae8-e3ec146899a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":589012315},{"type":"v1","addr":"172.21.15.117:6811","nonce":589012315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":589012315},{"type":"v1","addr":"172.21.15.117:6813","nonce":589012315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":589012315},{"type":"v1","addr":"172.21.15.117:6817","nonce":589012315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":589012315},{"type":"v1","addr":"172.21.15.117:6815","nonce":589012315}]},"public_addr":"172.21.15.117:6811/589012315","cluster_addr":"172.21.15.117:6813/589012315","heartbeat_back_addr":"172.21.15.117:6817/589012315","heartbeat_front_addr":"172.21.15.117:6815/589012315","state":["exists","up"]},{"osd":4,"uuid":"810d6c7a-023f-4930-82af-bc2bbfd647da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":1239048647},{"type":"v1","addr":"172.21.15.151:6809","nonce":1239048647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":1239048647},{"type":"v1","addr":"172.21.15.151:6811","nonce":1239048647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":1239048647},{"type":"v1","addr":"172.21.15.151:6815","nonce":1239048647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":1239048647},{"type":"v1","addr":"172.21.15.151:6813","nonce":1239048647}]},"public_addr":"172.21.15.151:6809/1239048647","cluster_addr":"172.21.15.151:6811/1239048647","heartbeat_back_addr":"172.21.15.151:6815/1239048647","heartbeat_front_addr":"172.21.15.151:6813/1239048647","state":["exists","up"]},{"osd":5,"uuid":"8ab6a5f0-34bb-48d8-9471-a7dc836e486e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":2985874686},{"type":"v1","addr":"172.21.15.183:6809","nonce":2985874686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":2985874686},{"type":"v1","addr":"172.21.15.183:6811","nonce":2985874686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":2985874686},{"type":"v1","addr":"172.21.15.183:6815","nonce":2985874686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":2985874686},{"type":"v1","addr":"172.21.15.183:6813","nonce":2985874686}]},"public_addr":"172.21.15.183:6809/2985874686","cluster_addr":"172.21.15.183:6811/2985874686","heartbeat_back_addr":"172.21.15.183:6815/2985874686","heartbeat_front_addr":"172.21.15.183:6813/2985874686","state":["exists","up"]},{"osd":6,"uuid":"78d15739-e9e0-4f19-ae5f-ee8527a5b527","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4250583193},{"type":"v1","addr":"172.21.15.117:6819","nonce":4250583193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":4250583193},{"type":"v1","addr":"172.21.15.117:6821","nonce":4250583193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":4250583193},{"type":"v1","addr":"172.21.15.117:6825","nonce":4250583193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4250583193},{"type":"v1","addr":"172.21.15.117:6823","nonce":4250583193}]},"public_addr":"172.21.15.117:6819/4250583193","cluster_addr":"172.21.15.117:6821/4250583193","heartbeat_back_addr":"172.21.15.117:6825/4250583193","heartbeat_front_addr":"172.21.15.117:6823/4250583193","state":["exists","up"]},{"osd":7,"uuid":"3724fe1f-f4d7-443b-aea8-ff3ab02bfdc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":386094645},{"type":"v1","addr":"172.21.15.151:6817","nonce":386094645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":386094645},{"type":"v1","addr":"172.21.15.151:6819","nonce":386094645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":386094645},{"type":"v1","addr":"172.21.15.151:6823","nonce":386094645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":386094645},{"type":"v1","addr":"172.21.15.151:6821","nonce":386094645}]},"public_addr":"172.21.15.151:6817/386094645","cluster_addr":"172.21.15.151:6819/386094645","heartbeat_back_addr":"172.21.15.151:6823/386094645","heartbeat_front_addr":"172.21.15.151:6821/386094645","state":["exists","up"]},{"osd":8,"uuid":"006ac092-f015-448e-a0c0-f7b0c4597a18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1829748222},{"type":"v1","addr":"172.21.15.183:6817","nonce":1829748222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1829748222},{"type":"v1","addr":"172.21.15.183:6819","nonce":1829748222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1829748222},{"type":"v1","addr":"172.21.15.183:6823","nonce":1829748222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1829748222},{"type":"v1","addr":"172.21.15.183:6821","nonce":1829748222}]},"public_addr":"172.21.15.183:6817/1829748222","cluster_addr":"172.21.15.183:6819/1829748222","heartbeat_back_addr":"172.21.15.183:6823/1829748222","heartbeat_front_addr":"172.21.15.183:6821/1829748222","state":["exists","up"]},{"osd":9,"uuid":"00a970e6-2502-459e-975c-badac93c9bd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2538315141},{"type":"v1","addr":"172.21.15.117:6827","nonce":2538315141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2538315141},{"type":"v1","addr":"172.21.15.117:6829","nonce":2538315141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2538315141},{"type":"v1","addr":"172.21.15.117:6833","nonce":2538315141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2538315141},{"type":"v1","addr":"172.21.15.117:6831","nonce":2538315141}]},"public_addr":"172.21.15.117:6827/2538315141","cluster_addr":"172.21.15.117:6829/2538315141","heartbeat_back_addr":"172.21.15.117:6833/2538315141","heartbeat_front_addr":"172.21.15.117:6831/2538315141","state":["exists","up"]},{"osd":10,"uuid":"ba2333cd-32a3-41ef-9ea5-287c707f512b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":3801045133},{"type":"v1","addr":"172.21.15.151:6825","nonce":3801045133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3801045133},{"type":"v1","addr":"172.21.15.151:6827","nonce":3801045133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3801045133},{"type":"v1","addr":"172.21.15.151:6831","nonce":3801045133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3801045133},{"type":"v1","addr":"172.21.15.151:6829","nonce":3801045133}]},"public_addr":"172.21.15.151:6825/3801045133","cluster_addr":"172.21.15.151:6827/3801045133","heartbeat_back_addr":"172.21.15.151:6831/3801045133","heartbeat_front_addr":"172.21.15.151:6829/3801045133","state":["exists","up"]},{"osd":11,"uuid":"1556d654-e6fa-42c1-b399-aa53f6837a1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3513899696},{"type":"v1","addr":"172.21.15.183:6825","nonce":3513899696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3513899696},{"type":"v1","addr":"172.21.15.183:6827","nonce":3513899696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3513899696},{"type":"v1","addr":"172.21.15.183:6831","nonce":3513899696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3513899696},{"type":"v1","addr":"172.21.15.183:6829","nonce":3513899696}]},"public_addr":"172.21.15.183:6825/3513899696","cluster_addr":"172.21.15.183:6827/3513899696","heartbeat_back_addr":"172.21.15.183:6831/3513899696","heartbeat_front_addr":"172.21.15.183:6829/3513899696","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:13:17.174041+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:13:42.237001+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:14:06.273315+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:14:39.912564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:15:06.063202+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:15:30.424818+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:16:05.623554+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:16:32.926393+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:16:58.165899+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:17:32.897477+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:18:00.720866+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:18:26.474441+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.117:0/186309008":"2024-06-04T03:11:24.655516+0000","172.21.15.117:6801/2142313590":"2024-06-04T03:11:24.655516+0000","172.21.15.117:6800/2142313590":"2024-06-04T03:11:24.655516+0000","172.21.15.117:0/2399725156":"2024-06-04T03:11:24.655516+0000","172.21.15.117:0/1453232061":"2024-06-04T03:10:50.552899+0000","172.21.15.117:0/4281674974":"2024-06-04T03:10:50.552899+0000","172.21.15.117:6800/2424554985":"2024-06-04T03:10:50.552899+0000","172.21.15.117:0/3946365225":"2024-06-04T03:11:24.655516+0000","172.21.15.117:0/1146162354":"2024-06-04T03:10:50.552899+0000","172.21.15.117:6801/2424554985":"2024-06-04T03:10:50.552899+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-03T03:24:42.561 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:42.559+0000 7f2136800700 1 -- 172.21.15.117:0/4075245525 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f21140a2ca0 msgr2=0x7f21140a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:42.561 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:42.559+0000 7f2136800700 1 --2- 172.21.15.117:0/4075245525 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f21140a2ca0 0x7f21140a5160 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2118009680 tx=0x7f2118009260 comp rx=0 tx=0).stop 2024-06-03T03:24:42.561 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:42.559+0000 7f2136800700 1 -- 172.21.15.117:0/4075245525 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f213006abe0 msgr2=0x7f213019e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:42.561 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:42.559+0000 7f2136800700 1 --2- 172.21.15.117:0/4075245525 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f213006abe0 0x7f213019e890 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f212000e9f0 tx=0x7f212000ed00 comp rx=0 tx=0).stop 2024-06-03T03:24:42.561 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:42.559+0000 7f2136800700 1 -- 172.21.15.117:0/4075245525 shutdown_connections 2024-06-03T03:24:42.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:42.559+0000 7f2136800700 1 --2- 172.21.15.117:0/4075245525 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f21140a2ca0 0x7f21140a5160 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:42.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:42.559+0000 7f2136800700 1 --2- 172.21.15.117:0/4075245525 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2130108520 0x7f2130198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:42.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:42.559+0000 7f2136800700 1 --2- 172.21.15.117:0/4075245525 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f213006abe0 0x7f213019e890 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:42.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:42.559+0000 7f2136800700 1 --2- 172.21.15.117:0/4075245525 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f213006b5a0 0x7f213019edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:42.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:42.559+0000 7f2136800700 1 -- 172.21.15.117:0/4075245525 >> 172.21.15.117:0/4075245525 conn(0x7f2130078330 msgr2=0x7f2130100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:24:42.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:42.560+0000 7f2136800700 1 -- 172.21.15.117:0/4075245525 shutdown_connections 2024-06-03T03:24:42.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:42.560+0000 7f2136800700 1 -- 172.21.15.117:0/4075245525 wait complete. 2024-06-03T03:24:42.906 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:24:42 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/4075245525' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T03:24:42.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:24:42 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/4075245525' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T03:24:42.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:24:42 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/4075245525' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T03:24:43.608 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 30, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-06-03T03:19:44.629822+0000', 'modified': '2024-06-03T03:21:12.085445+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24517, 'mds_1': 24512, 'mds_2': 24503, 'mds_3': 24499, 'mds_4': 24487}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24431': {'gid': 24431, 'name': 'k', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.151:6833/3355662416', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6832', 'nonce': 3355662416}, {'type': 'v1', 'addr': '172.21.15.151:6833', 'nonce': 3355662416}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24445': {'gid': 24445, 'name': 'l', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.183:6833/2429725085', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6832', 'nonce': 2429725085}, {'type': 'v1', 'addr': '172.21.15.183:6833', 'nonce': 2429725085}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24461': {'gid': 24461, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.151:6835/31153579', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6834', 'nonce': 31153579}, {'type': 'v1', 'addr': '172.21.15.151:6835', 'nonce': 31153579}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24466': {'gid': 24466, 'name': 'j', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.117:6835/2884460088', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 2884460088}, {'type': 'v1', 'addr': '172.21.15.117:6835', 'nonce': 2884460088}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24479': {'gid': 24479, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.151:6837/2392407848', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6836', 'nonce': 2392407848}, {'type': 'v1', 'addr': '172.21.15.151:6837', 'nonce': 2392407848}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24487': {'gid': 24487, 'name': 'f', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 32, 'addr': '172.21.15.183:6835/1009004320', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6834', 'nonce': 1009004320}, {'type': 'v1', 'addr': '172.21.15.183:6835', 'nonce': 1009004320}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24499': {'gid': 24499, 'name': 'c', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 31, 'addr': '172.21.15.183:6837/2697518526', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6836', 'nonce': 2697518526}, {'type': 'v1', 'addr': '172.21.15.183:6837', 'nonce': 2697518526}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24503': {'gid': 24503, 'name': 'a', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 27, 'addr': '172.21.15.117:6841/1212242967', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6840', 'nonce': 1212242967}, {'type': 'v1', 'addr': '172.21.15.117:6841', 'nonce': 1212242967}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24512': {'gid': 24512, 'name': 'b', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 26, 'addr': '172.21.15.151:6839/3623444616', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6838', 'nonce': 3623444616}, {'type': 'v1', 'addr': '172.21.15.151:6839', 'nonce': 3623444616}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24517': {'gid': 24517, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.183:6839/1439615083', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6838', 'nonce': 1439615083}, {'type': 'v1', 'addr': '172.21.15.183:6839', 'nonce': 1439615083}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24517, 'qdb_cluster': [24503, 24517, 24512, 24487, 24499]} 2024-06-03T03:24:43.608 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-06-03T03:24:43.608 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph tell mds.f status 2024-06-03T03:24:43.906 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:24:43 smithi151 ceph-mon[37043]: pgmap v488: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-06-03T03:24:43.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:24:43 smithi117 ceph-mon[31649]: pgmap v488: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-06-03T03:24:43.981 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:24:43 smithi183 ceph-mon[37326]: pgmap v488: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-06-03T03:24:44.056 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:24:46.297 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:24:45 smithi117 ceph-mon[31649]: pgmap v489: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:24:46.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:24:46 smithi151 ceph-mon[37043]: pgmap v489: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:24:46.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:24:45 smithi183 ceph-mon[37326]: pgmap v489: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:24:46.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.605+0000 7f303b200700 1 -- 172.21.15.117:0/4223247561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f303c104f80 msgr2=0x7f303c111490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:46.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.605+0000 7f303b200700 1 --2- 172.21.15.117:0/4223247561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f303c104f80 0x7f303c111490 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f303400b0a0 tx=0x7f303400b3b0 comp rx=0 tx=0).stop 2024-06-03T03:24:46.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.605+0000 7f303b200700 1 -- 172.21.15.117:0/4223247561 shutdown_connections 2024-06-03T03:24:46.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.605+0000 7f303b200700 1 --2- 172.21.15.117:0/4223247561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f303c104f80 0x7f303c111490 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:46.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.605+0000 7f303b200700 1 --2- 172.21.15.117:0/4223247561 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f303c10a650 0x7f303c10aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:46.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.605+0000 7f303b200700 1 --2- 172.21.15.117:0/4223247561 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f303c1045c0 0x7f303c104a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:46.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.605+0000 7f303b200700 1 -- 172.21.15.117:0/4223247561 >> 172.21.15.117:0/4223247561 conn(0x7f303c1000b0 msgr2=0x7f303c1024d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:24:46.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.606+0000 7f303b200700 1 -- 172.21.15.117:0/4223247561 shutdown_connections 2024-06-03T03:24:46.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.606+0000 7f303b200700 1 -- 172.21.15.117:0/4223247561 wait complete. 2024-06-03T03:24:46.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.607+0000 7f303b200700 1 Processor -- start 2024-06-03T03:24:46.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.607+0000 7f303b200700 1 -- start start 2024-06-03T03:24:46.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.607+0000 7f303b200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f303c1045c0 0x7f303c1a08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:46.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.607+0000 7f303b200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f303c104f80 0x7f303c1a0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:46.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.607+0000 7f303b200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f303c10a650 0x7f303c19aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:46.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.607+0000 7f303b200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f303c114910 con 0x7f303c10a650 2024-06-03T03:24:46.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.607+0000 7f303b200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f303c114790 con 0x7f303c1045c0 2024-06-03T03:24:46.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.607+0000 7f303b200700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f303c114a90 con 0x7f303c104f80 2024-06-03T03:24:46.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.608+0000 7f3039e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f303c1045c0 0x7f303c1a08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:46.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.608+0000 7f3039e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f303c1045c0 0x7f303c1a08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:59308/0 (socket says 172.21.15.117:59308) 2024-06-03T03:24:46.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.608+0000 7f3039e00700 1 -- 172.21.15.117:0/3660844174 learned_addr learned my addr 172.21.15.117:0/3660844174 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:24:46.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.608+0000 7f3039400700 1 --2- 172.21.15.117:0/3660844174 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f303c104f80 0x7f303c1a0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:46.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.608+0000 7f303a800700 1 --2- 172.21.15.117:0/3660844174 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f303c10a650 0x7f303c19aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:46.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.608+0000 7f3039e00700 1 -- 172.21.15.117:0/3660844174 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f303c104f80 msgr2=0x7f303c1a0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:46.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.608+0000 7f3039e00700 1 --2- 172.21.15.117:0/3660844174 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f303c104f80 0x7f303c1a0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:46.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.608+0000 7f3039e00700 1 -- 172.21.15.117:0/3660844174 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f303c10a650 msgr2=0x7f303c19aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:46.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.608+0000 7f3039e00700 1 --2- 172.21.15.117:0/3660844174 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f303c10a650 0x7f303c19aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:46.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.608+0000 7f3039e00700 1 -- 172.21.15.117:0/3660844174 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f303c19b1d0 con 0x7f303c1045c0 2024-06-03T03:24:46.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.608+0000 7f3039e00700 1 --2- 172.21.15.117:0/3660844174 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f303c1045c0 0x7f303c1a08b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f302c00e950 tx=0x7f302c00ec60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:24:46.612 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.609+0000 7f3032a00700 1 -- 172.21.15.117:0/3660844174 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f302c004230 con 0x7f303c1045c0 2024-06-03T03:24:46.612 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.609+0000 7f3032a00700 1 -- 172.21.15.117:0/3660844174 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f302c0043d0 con 0x7f303c1045c0 2024-06-03T03:24:46.612 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.609+0000 7f303b200700 1 -- 172.21.15.117:0/3660844174 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f303c19b4c0 con 0x7f303c1045c0 2024-06-03T03:24:46.612 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.609+0000 7f3032a00700 1 -- 172.21.15.117:0/3660844174 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f302c003dd0 con 0x7f303c1045c0 2024-06-03T03:24:46.612 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.609+0000 7f303b200700 1 -- 172.21.15.117:0/3660844174 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f303c1a7780 con 0x7f303c1045c0 2024-06-03T03:24:46.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.612+0000 7f3032a00700 1 -- 172.21.15.117:0/3660844174 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f302c00c7e0 con 0x7f303c1045c0 2024-06-03T03:24:46.615 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.613+0000 7f304b549b80 1 Processor -- start 2024-06-03T03:24:46.615 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.613+0000 7f304b549b80 1 -- start start 2024-06-03T03:24:46.615 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.613+0000 7f3032a00700 1 --2- 172.21.15.117:0/3660844174 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f30200a2c20 0x7f30200a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:46.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.614+0000 7f304b549b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56300c004df0 0x56300c08a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:46.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.614+0000 7f3032a00700 1 -- 172.21.15.117:0/3660844174 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f302c0cf6b0 con 0x7f303c1045c0 2024-06-03T03:24:46.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.614+0000 7f304b549b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56300c08a910 0x56300c08ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:46.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.614+0000 7f304b549b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56300c093520 0x56300c095910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:46.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.614+0000 7f304b549b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56300bf979c0 con 0x56300c093520 2024-06-03T03:24:46.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.614+0000 7f304b549b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56300bfdaab0 con 0x56300c004df0 2024-06-03T03:24:46.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.614+0000 7f304b549b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56300bf958a0 con 0x56300c08a910 2024-06-03T03:24:46.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.614+0000 7f303a800700 1 --2- 172.21.15.117:0/3660844174 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f30200a2c20 0x7f30200a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:46.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.614+0000 7f303a800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56300c093520 0x56300c095910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:46.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.614+0000 7f3039e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56300c004df0 0x56300c08a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:46.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.614+0000 7f3039400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56300c08a910 0x56300c08ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:46.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.614+0000 7f3039e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56300c004df0 0x56300c08a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:59312/0 (socket says 172.21.15.117:59312) 2024-06-03T03:24:46.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.614+0000 7f3039e00700 1 -- 172.21.15.117:0/2384893940 learned_addr learned my addr 172.21.15.117:0/2384893940 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:24:46.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.614+0000 7f303a800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56300c093520 0x56300c095910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44680/0 (socket says 172.21.15.117:44680) 2024-06-03T03:24:46.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.615+0000 7f3039400700 1 -- 172.21.15.117:0/2384893940 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56300c004df0 msgr2=0x56300c08a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:24:46.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.615+0000 7f3039400700 1 --2- 172.21.15.117:0/2384893940 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56300c004df0 0x56300c08a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:46.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.615+0000 7f3039400700 1 -- 172.21.15.117:0/2384893940 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56300c093520 msgr2=0x56300c095910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:24:46.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.615+0000 7f3039400700 1 --2- 172.21.15.117:0/2384893940 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56300c093520 0x56300c095910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:46.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.615+0000 7f3039400700 1 -- 172.21.15.117:0/2384893940 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56300bf82f40 con 0x56300c08a910 2024-06-03T03:24:46.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.615+0000 7f303a800700 1 --2- 172.21.15.117:0/2384893940 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56300c093520 0x56300c095910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T03:24:46.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.615+0000 7f303a800700 1 --2- 172.21.15.117:0/3660844174 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f30200a2c20 0x7f30200a50e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3034009fd0 tx=0x7f303400bbe0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:24:46.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.615+0000 7f3039400700 1 --2- 172.21.15.117:0/2384893940 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56300c08a910 0x56300c08ad90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3028009c80 tx=0x7f3028009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50d136b36b7903e2 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:24:46.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.615+0000 7f3027400700 1 -- 172.21.15.117:0/2384893940 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3028011cb0 con 0x56300c08a910 2024-06-03T03:24:46.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.615+0000 7f3027400700 1 -- 172.21.15.117:0/2384893940 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3028011e50 con 0x56300c08a910 2024-06-03T03:24:46.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.615+0000 7f3027400700 1 -- 172.21.15.117:0/2384893940 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3028005280 con 0x56300c08a910 2024-06-03T03:24:46.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.616+0000 7f304b549b80 1 -- 172.21.15.117:0/2384893940 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56300c08a910 msgr2=0x56300c08ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:46.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.616+0000 7f304b549b80 1 --2- 172.21.15.117:0/2384893940 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56300c08a910 0x56300c08ad90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3028009c80 tx=0x7f3028009f90 comp rx=0 tx=0).stop 2024-06-03T03:24:46.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.616+0000 7f304b549b80 1 -- 172.21.15.117:0/2384893940 shutdown_connections 2024-06-03T03:24:46.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.616+0000 7f304b549b80 1 --2- 172.21.15.117:0/2384893940 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56300c093520 0x56300c095910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:46.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.616+0000 7f304b549b80 1 --2- 172.21.15.117:0/2384893940 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56300c004df0 0x56300c08a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:46.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.616+0000 7f304b549b80 1 --2- 172.21.15.117:0/2384893940 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56300c08a910 0x56300c08ad90 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:46.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.616+0000 7f304b549b80 1 -- 172.21.15.117:0/2384893940 >> 172.21.15.117:0/2384893940 conn(0x56300bfb5120 msgr2=0x56300c0847d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:24:46.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.616+0000 7f304b549b80 1 -- 172.21.15.117:0/2384893940 shutdown_connections 2024-06-03T03:24:46.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.617+0000 7f304b549b80 1 -- 172.21.15.117:0/2384893940 wait complete. 2024-06-03T03:24:46.621 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.617+0000 7f304b549b80 1 -- start start 2024-06-03T03:24:46.621 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.617+0000 7f304b549b80 1 Processor -- start 2024-06-03T03:24:46.621 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.618+0000 7f304b549b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56300c004df0 0x56300c092c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:46.621 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.618+0000 7f304b549b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56300c08a910 0x56300c08ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:46.621 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.618+0000 7f304b549b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56300c093520 0x56300c0904c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:46.621 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.618+0000 7f304b549b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56300bf76f10 con 0x56300c08a910 2024-06-03T03:24:46.621 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.618+0000 7f304b549b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56300bfd6a70 con 0x56300c004df0 2024-06-03T03:24:46.621 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.618+0000 7f304b549b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56300c096130 con 0x56300c093520 2024-06-03T03:24:46.621 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.618+0000 7f304b549b80 10 client.0 _ll_register_callbacks cb 0x56300bf746d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:24:46.622 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.618+0000 7f3039400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56300c08a910 0x56300c08ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:46.622 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.618+0000 7f3039400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56300c08a910 0x56300c08ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44686/0 (socket says 172.21.15.117:44686) 2024-06-03T03:24:46.622 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.618+0000 7f303a800700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56300c093520 0x56300c0904c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:46.622 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.618+0000 7f3039400700 1 -- 172.21.15.117:0/3056881980 learned_addr learned my addr 172.21.15.117:0/3056881980 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:24:46.622 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.618+0000 7f3039e00700 1 --2- 172.21.15.117:0/3056881980 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56300c004df0 0x56300c092c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:46.622 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.619+0000 7f303a800700 1 -- 172.21.15.117:0/3056881980 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56300c004df0 msgr2=0x56300c092c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:24:46.622 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.619+0000 7f303a800700 1 --2- 172.21.15.117:0/3056881980 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56300c004df0 0x56300c092c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:46.622 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.619+0000 7f303a800700 1 -- 172.21.15.117:0/3056881980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56300c08a910 msgr2=0x56300c08ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:24:46.622 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.619+0000 7f303a800700 1 --2- 172.21.15.117:0/3056881980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56300c08a910 0x56300c08ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:46.622 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.619+0000 7f303a800700 1 -- 172.21.15.117:0/3056881980 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56300bf29880 con 0x56300c093520 2024-06-03T03:24:46.622 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.619+0000 7f3039400700 1 --2- 172.21.15.117:0/3056881980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56300c08a910 0x56300c08ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:24:46.622 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.619+0000 7f303a800700 1 --2- 172.21.15.117:0/3056881980 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56300c093520 0x56300c0904c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f303400b0a0 tx=0x7f303400b3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71e54665bf4e488f server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:24:46.623 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.619+0000 7f301be00700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-06-03T03:24:46.623 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.620+0000 7f301be00700 1 -- 172.21.15.117:0/3056881980 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3034030060 con 0x56300c093520 2024-06-03T03:24:46.623 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.620+0000 7f301be00700 1 -- 172.21.15.117:0/3056881980 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f3034007d50 con 0x56300c093520 2024-06-03T03:24:46.623 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.620+0000 7f304b549b80 1 -- 172.21.15.117:0/3056881980 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3028011e50 con 0x56300c093520 2024-06-03T03:24:46.625 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.620+0000 7f301be00700 1 -- 172.21.15.117:0/3056881980 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f303402b440 con 0x56300c093520 2024-06-03T03:24:46.625 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.620+0000 7f304b549b80 10 client.25129 fetch_fsmap learned FSMap version 30 2024-06-03T03:24:46.625 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.620+0000 7f304b549b80 1 -- 172.21.15.117:0/3056881980 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56300c0a0850 con 0x56300c093520 2024-06-03T03:24:46.625 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.620+0000 7f301be00700 1 -- 172.21.15.117:0/3056881980 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3028011e50 con 0x56300c093520 2024-06-03T03:24:46.625 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.620+0000 7f301be00700 1 -- 172.21.15.117:0/3056881980 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f3034011070 con 0x56300c093520 2024-06-03T03:24:46.625 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.621+0000 7f304b549b80 10 client.25129 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T03:24:46.625 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.621+0000 7f304b549b80 10 client.25129 resolve_mds: resolved name 'f' to daemon mds.f 2024-06-03T03:24:46.625 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.621+0000 7f304b549b80 20 client.25129 populate_metadata read hostname 'smithi117' 2024-06-03T03:24:46.625 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.621+0000 7f304b549b80 1 --2- 172.21.15.117:0/3056881980 >> [v2:172.21.15.183:6834/1009004320,v1:172.21.15.183:6835/1009004320] conn(0x56300c0a2f00 0x56300c0a4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:46.625 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.621+0000 7f304b549b80 4 client.25129 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T03:24:46.626 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.621+0000 7f304b549b80 1 -- 172.21.15.117:0/3056881980 --> [v2:172.21.15.183:6834/1009004320,v1:172.21.15.183:6835/1009004320] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56300bfe7fb0 con 0x56300c0a2f00 2024-06-03T03:24:46.626 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.621+0000 7f3039e00700 1 --2- 172.21.15.117:0/3056881980 >> [v2:172.21.15.183:6834/1009004320,v1:172.21.15.183:6835/1009004320] conn(0x56300c0a2f00 0x56300c0a4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:46.626 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.622+0000 7f3039e00700 1 --2- 172.21.15.117:0/3056881980 >> [v2:172.21.15.183:6834/1009004320,v1:172.21.15.183:6835/1009004320] conn(0x56300c0a2f00 0x56300c0a4370 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=59fdc6e86025db9d server_cookie=e7ba1441b42b1f43 in_seq=0 out_seq=0 2024-06-03T03:24:46.626 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.622+0000 7f301be00700 10 client.25129 ms_handle_connect on v2:172.21.15.183:6834/1009004320 2024-06-03T03:24:46.626 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.622+0000 7f301be00700 1 -- 172.21.15.117:0/3056881980 <== mds.4 v2:172.21.15.183:6834/1009004320 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56300bfe7fb0 con 0x56300c0a2f00 2024-06-03T03:24:46.626 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.622+0000 7f301be00700 10 client.25129 handle_command_reply: tid=0 2024-06-03T03:24:46.626 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.622+0000 7f304b549b80 1 client.25129 shutdown 2024-06-03T03:24:46.626 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.623+0000 7f304b549b80 1 -- 172.21.15.117:0/3056881980 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56300c093520 msgr2=0x56300c0904c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:46.626 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.623+0000 7f304b549b80 1 --2- 172.21.15.117:0/3056881980 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56300c093520 0x56300c0904c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f303400b0a0 tx=0x7f303400b3b0 comp rx=0 tx=0).stop 2024-06-03T03:24:46.626 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.623+0000 7f304b549b80 1 -- 172.21.15.117:0/3056881980 shutdown_connections 2024-06-03T03:24:46.626 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.623+0000 7f304b549b80 1 --2- 172.21.15.117:0/3056881980 >> [v2:172.21.15.183:6834/1009004320,v1:172.21.15.183:6835/1009004320] conn(0x56300c0a2f00 0x56300c0a4370 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:46.626 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.623+0000 7f304b549b80 1 --2- 172.21.15.117:0/3056881980 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56300c093520 0x56300c0904c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:46.626 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.623+0000 7f304b549b80 1 --2- 172.21.15.117:0/3056881980 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56300c004df0 0x56300c092c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:46.627 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.623+0000 7f304b549b80 1 --2- 172.21.15.117:0/3056881980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56300c08a910 0x56300c08ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:46.627 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.623+0000 7f301be00700 0 client.25129 ms_handle_reset on v2:172.21.15.183:6834/1009004320 2024-06-03T03:24:46.627 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.623+0000 7f304b549b80 1 -- 172.21.15.117:0/3056881980 >> 172.21.15.117:0/3056881980 conn(0x56300bfb5120 msgr2=0x56300c0843c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:24:46.627 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.623+0000 7f304b549b80 1 -- 172.21.15.117:0/3056881980 shutdown_connections 2024-06-03T03:24:46.627 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.623+0000 7f304b549b80 1 -- 172.21.15.117:0/3056881980 wait complete. 2024-06-03T03:24:46.627 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.623+0000 7f304b549b80 20 client.25129 trim_cache size 0 max 16384 2024-06-03T03:24:46.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.646+0000 7f304b549b80 1 Processor -- start 2024-06-03T03:24:46.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.646+0000 7f304b549b80 1 -- start start 2024-06-03T03:24:46.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.646+0000 7f304b549b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56300c004df0 0x56300c083c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:46.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.646+0000 7f304b549b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56300c08a910 0x56300c0890a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:46.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.646+0000 7f304b549b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56300c093520 0x56300c0ad7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:46.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.646+0000 7f304b549b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56300c0a26b0 con 0x56300c093520 2024-06-03T03:24:46.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.646+0000 7f304b549b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56300c0a2d80 con 0x56300c08a910 2024-06-03T03:24:46.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.646+0000 7f304b549b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56300c0a3450 con 0x56300c004df0 2024-06-03T03:24:46.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.646+0000 7f3039e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56300c004df0 0x56300c083c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:46.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.647+0000 7f3039e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56300c004df0 0x56300c083c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:59562/0 (socket says 172.21.15.117:59562) 2024-06-03T03:24:46.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.647+0000 7f3039e00700 1 -- 172.21.15.117:0/2107412070 learned_addr learned my addr 172.21.15.117:0/2107412070 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:24:46.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.647+0000 7f303a800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56300c093520 0x56300c0ad7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:46.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.647+0000 7f3039400700 1 --2- 172.21.15.117:0/2107412070 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56300c08a910 0x56300c0890a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:46.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.647+0000 7f3039e00700 1 -- 172.21.15.117:0/2107412070 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56300c08a910 msgr2=0x56300c0890a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:24:46.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.647+0000 7f3039e00700 1 --2- 172.21.15.117:0/2107412070 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56300c08a910 0x56300c0890a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:46.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.647+0000 7f3039e00700 1 -- 172.21.15.117:0/2107412070 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56300c093520 msgr2=0x56300c0ad7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:24:46.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.647+0000 7f3039e00700 1 --2- 172.21.15.117:0/2107412070 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56300c093520 0x56300c0ad7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:46.650 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.647+0000 7f3039e00700 1 -- 172.21.15.117:0/2107412070 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56300c09f3d0 con 0x56300c004df0 2024-06-03T03:24:46.650 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.647+0000 7f3039e00700 1 --2- 172.21.15.117:0/2107412070 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56300c004df0 0x56300c083c70 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f302c09bab0 tx=0x7f302c0b1040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccda6d66fd22b764 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:24:46.650 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.647+0000 7f3026a00700 1 -- 172.21.15.117:0/2107412070 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f302c09c220 con 0x56300c004df0 2024-06-03T03:24:46.650 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.647+0000 7f3026a00700 1 -- 172.21.15.117:0/2107412070 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f302c09ce50 con 0x56300c004df0 2024-06-03T03:24:46.650 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.647+0000 7f3026a00700 1 -- 172.21.15.117:0/2107412070 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f302c01fe40 con 0x56300c004df0 2024-06-03T03:24:46.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.648+0000 7f304b549b80 1 -- 172.21.15.117:0/2107412070 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56300c004df0 msgr2=0x56300c083c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:46.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.648+0000 7f304b549b80 1 --2- 172.21.15.117:0/2107412070 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56300c004df0 0x56300c083c70 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f302c09bab0 tx=0x7f302c0b1040 comp rx=0 tx=0).stop 2024-06-03T03:24:46.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.648+0000 7f304b549b80 1 -- 172.21.15.117:0/2107412070 shutdown_connections 2024-06-03T03:24:46.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.648+0000 7f304b549b80 1 --2- 172.21.15.117:0/2107412070 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56300c093520 0x56300c0ad7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:46.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.648+0000 7f304b549b80 1 --2- 172.21.15.117:0/2107412070 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56300c004df0 0x56300c083c70 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:46.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.648+0000 7f304b549b80 1 --2- 172.21.15.117:0/2107412070 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56300c08a910 0x56300c0890a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:46.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.648+0000 7f304b549b80 1 -- 172.21.15.117:0/2107412070 >> 172.21.15.117:0/2107412070 conn(0x56300bfb5120 msgr2=0x56300c084300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:24:46.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.648+0000 7f304b549b80 1 -- 172.21.15.117:0/2107412070 shutdown_connections 2024-06-03T03:24:46.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.648+0000 7f304b549b80 1 -- 172.21.15.117:0/2107412070 wait complete. 2024-06-03T03:24:46.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.649+0000 7f304b549b80 1 -- start start 2024-06-03T03:24:46.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.649+0000 7f304b549b80 1 Processor -- start 2024-06-03T03:24:46.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.649+0000 7f304b549b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56300c004df0 0x56300c0adf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:46.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.649+0000 7f304b549b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56300c08a910 0x56300c0a9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:46.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.649+0000 7f304b549b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56300c093520 0x56300c0a98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:46.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.649+0000 7f304b549b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56300c0a41f0 con 0x56300c08a910 2024-06-03T03:24:46.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.649+0000 7f304b549b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56300c0a3b20 con 0x56300c093520 2024-06-03T03:24:46.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.649+0000 7f304b549b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56300c0a48c0 con 0x56300c004df0 2024-06-03T03:24:46.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.649+0000 7f304b549b80 10 client.0 _ll_register_callbacks cb 0x56300bf746d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:24:46.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.650+0000 7f3039400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56300c08a910 0x56300c0a9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:46.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.650+0000 7f3039400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56300c08a910 0x56300c0a9350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44716/0 (socket says 172.21.15.117:44716) 2024-06-03T03:24:46.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.650+0000 7f3039400700 1 -- 172.21.15.117:0/1752286713 learned_addr learned my addr 172.21.15.117:0/1752286713 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:24:46.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.650+0000 7f3039e00700 1 --2- 172.21.15.117:0/1752286713 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56300c004df0 0x56300c0adf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:46.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.650+0000 7f303a800700 1 --2- 172.21.15.117:0/1752286713 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56300c093520 0x56300c0a98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:46.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.650+0000 7f3039400700 1 -- 172.21.15.117:0/1752286713 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56300c004df0 msgr2=0x56300c0adf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:24:46.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.650+0000 7f3039400700 1 --2- 172.21.15.117:0/1752286713 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56300c004df0 0x56300c0adf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:46.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.650+0000 7f3039400700 1 -- 172.21.15.117:0/1752286713 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56300c093520 msgr2=0x56300c0a98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:24:46.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.650+0000 7f3039400700 1 --2- 172.21.15.117:0/1752286713 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56300c093520 0x56300c0a98d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:46.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.650+0000 7f3039400700 1 -- 172.21.15.117:0/1752286713 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56300c0aa110 con 0x56300c08a910 2024-06-03T03:24:46.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.650+0000 7f3039e00700 1 --2- 172.21.15.117:0/1752286713 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56300c004df0 0x56300c0adf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:24:46.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.650+0000 7f303a800700 1 --2- 172.21.15.117:0/1752286713 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56300c093520 0x56300c0a98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:24:46.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.651+0000 7f3039400700 1 --2- 172.21.15.117:0/1752286713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56300c08a910 0x56300c0a9350 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3028009bd0 tx=0x7f3028004c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=132e053d97fc519a server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:24:46.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.651+0000 7f301be00700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-06-03T03:24:46.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.651+0000 7f301be00700 1 -- 172.21.15.117:0/1752286713 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3028004260 con 0x56300c08a910 2024-06-03T03:24:46.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.651+0000 7f304b549b80 1 -- 172.21.15.117:0/1752286713 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f302c09ce50 con 0x56300c08a910 2024-06-03T03:24:46.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.651+0000 7f301be00700 1 -- 172.21.15.117:0/1752286713 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f3028004400 con 0x56300c08a910 2024-06-03T03:24:46.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.651+0000 7f301be00700 1 -- 172.21.15.117:0/1752286713 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3028035050 con 0x56300c08a910 2024-06-03T03:24:46.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.651+0000 7f304b549b80 10 client.14991 fetch_fsmap learned FSMap version 30 2024-06-03T03:24:46.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.651+0000 7f304b549b80 1 -- 172.21.15.117:0/1752286713 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56300c10e840 con 0x56300c08a910 2024-06-03T03:24:46.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.652+0000 7f301be00700 1 -- 172.21.15.117:0/1752286713 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f302c09ce50 con 0x56300c08a910 2024-06-03T03:24:46.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.652+0000 7f301be00700 1 -- 172.21.15.117:0/1752286713 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f3028018070 con 0x56300c08a910 2024-06-03T03:24:46.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.652+0000 7f304b549b80 10 client.14991 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T03:24:46.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.652+0000 7f304b549b80 10 client.14991 resolve_mds: resolved name 'f' to daemon mds.f 2024-06-03T03:24:46.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.652+0000 7f304b549b80 20 client.14991 populate_metadata read hostname 'smithi117' 2024-06-03T03:24:46.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.652+0000 7f304b549b80 1 --2- 172.21.15.117:0/1752286713 >> [v2:172.21.15.183:6834/1009004320,v1:172.21.15.183:6835/1009004320] conn(0x56300c113230 0x56300c111f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:46.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.652+0000 7f304b549b80 4 client.14991 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-03T03:24:46.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.652+0000 7f304b549b80 1 -- 172.21.15.117:0/1752286713 --> [v2:172.21.15.183:6834/1009004320,v1:172.21.15.183:6835/1009004320] -- command(tid 0: {"prefix": "status"}) -- 0x56300bfe2530 con 0x56300c113230 2024-06-03T03:24:46.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.652+0000 7f3039e00700 1 --2- 172.21.15.117:0/1752286713 >> [v2:172.21.15.183:6834/1009004320,v1:172.21.15.183:6835/1009004320] conn(0x56300c113230 0x56300c111f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:46.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.653+0000 7f301be00700 10 client.14991 ms_handle_connect on v2:172.21.15.183:6834/1009004320 2024-06-03T03:24:46.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.653+0000 7f3039e00700 1 --2- 172.21.15.117:0/1752286713 >> [v2:172.21.15.183:6834/1009004320,v1:172.21.15.183:6835/1009004320] conn(0x56300c113230 0x56300c111f40 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d41229ef98b917c0 server_cookie=4033f857271b969a in_seq=0 out_seq=0 2024-06-03T03:24:46.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.653+0000 7f301be00700 1 -- 172.21.15.117:0/1752286713 <== mds.4 v2:172.21.15.183:6834/1009004320 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x56300bfe2530 con 0x56300c113230 2024-06-03T03:24:46.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.653+0000 7f301be00700 10 client.14991 handle_command_reply: tid=0 2024-06-03T03:24:46.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.653+0000 7f304b549b80 1 client.14991 shutdown 2024-06-03T03:24:46.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.653+0000 7f304b549b80 1 -- 172.21.15.117:0/1752286713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56300c08a910 msgr2=0x56300c0a9350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:46.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.653+0000 7f304b549b80 1 --2- 172.21.15.117:0/1752286713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56300c08a910 0x56300c0a9350 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3028009bd0 tx=0x7f3028004c20 comp rx=0 tx=0).stop 2024-06-03T03:24:46.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.653+0000 7f304b549b80 1 -- 172.21.15.117:0/1752286713 shutdown_connections 2024-06-03T03:24:46.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.653+0000 7f304b549b80 1 --2- 172.21.15.117:0/1752286713 >> [v2:172.21.15.183:6834/1009004320,v1:172.21.15.183:6835/1009004320] conn(0x56300c113230 0x56300c111f40 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:46.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.653+0000 7f304b549b80 1 --2- 172.21.15.117:0/1752286713 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56300c093520 0x56300c0a98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:46.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.653+0000 7f304b549b80 1 --2- 172.21.15.117:0/1752286713 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56300c004df0 0x56300c0adf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:46.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.653+0000 7f304b549b80 1 --2- 172.21.15.117:0/1752286713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56300c08a910 0x56300c0a9350 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:46.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.653+0000 7f301be00700 0 client.14991 ms_handle_reset on v2:172.21.15.183:6834/1009004320 2024-06-03T03:24:46.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.653+0000 7f304b549b80 1 -- 172.21.15.117:0/1752286713 >> 172.21.15.117:0/1752286713 conn(0x56300bfb5120 msgr2=0x56300c08acf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:24:46.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.654+0000 7f304b549b80 1 -- 172.21.15.117:0/1752286713 shutdown_connections 2024-06-03T03:24:46.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.654+0000 7f304b549b80 1 -- 172.21.15.117:0/1752286713 wait complete. 2024-06-03T03:24:46.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.654+0000 7f304b549b80 20 client.14991 trim_cache size 0 max 16384 2024-06-03T03:24:46.658 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-06-03T03:24:46.658 INFO:teuthology.orchestra.run.smithi117.stdout: "cluster_fsid": "a0e8473a-2156-11ef-bc9b-c7b262605968", 2024-06-03T03:24:46.658 INFO:teuthology.orchestra.run.smithi117.stdout: "whoami": 4, 2024-06-03T03:24:46.658 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 24487, 2024-06-03T03:24:46.658 INFO:teuthology.orchestra.run.smithi117.stdout: "want_state": "up:active", 2024-06-03T03:24:46.659 INFO:teuthology.orchestra.run.smithi117.stdout: "state": "up:active", 2024-06-03T03:24:46.659 INFO:teuthology.orchestra.run.smithi117.stdout: "fs_name": "cephfs", 2024-06-03T03:24:46.659 INFO:teuthology.orchestra.run.smithi117.stdout: "rank_uptime": 220.63896638599999, 2024-06-03T03:24:46.659 INFO:teuthology.orchestra.run.smithi117.stdout: "mdsmap_epoch": 30, 2024-06-03T03:24:46.659 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_epoch": 99, 2024-06-03T03:24:46.659 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_epoch_barrier": 97, 2024-06-03T03:24:46.659 INFO:teuthology.orchestra.run.smithi117.stdout: "uptime": 342.83964029700002 2024-06-03T03:24:46.659 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-06-03T03:24:46.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.654+0000 7f303b200700 1 -- 172.21.15.117:0/3660844174 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f30200a2c20 msgr2=0x7f30200a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:46.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.654+0000 7f303b200700 1 --2- 172.21.15.117:0/3660844174 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f30200a2c20 0x7f30200a50e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3034009fd0 tx=0x7f303400bbe0 comp rx=0 tx=0).stop 2024-06-03T03:24:46.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.654+0000 7f303b200700 1 -- 172.21.15.117:0/3660844174 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f303c1045c0 msgr2=0x7f303c1a08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:46.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.654+0000 7f303b200700 1 --2- 172.21.15.117:0/3660844174 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f303c1045c0 0x7f303c1a08b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f302c00e950 tx=0x7f302c00ec60 comp rx=0 tx=0).stop 2024-06-03T03:24:46.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.655+0000 7f303b200700 1 -- 172.21.15.117:0/3660844174 shutdown_connections 2024-06-03T03:24:46.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.655+0000 7f303b200700 1 --2- 172.21.15.117:0/3660844174 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f30200a2c20 0x7f30200a50e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:46.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.655+0000 7f303b200700 1 --2- 172.21.15.117:0/3660844174 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f303c10a650 0x7f303c19aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:46.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.655+0000 7f303b200700 1 --2- 172.21.15.117:0/3660844174 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f303c1045c0 0x7f303c1a08b0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:46.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.655+0000 7f303b200700 1 --2- 172.21.15.117:0/3660844174 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f303c104f80 0x7f303c1a0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:46.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.655+0000 7f303b200700 1 -- 172.21.15.117:0/3660844174 >> 172.21.15.117:0/3660844174 conn(0x7f303c1000b0 msgr2=0x7f303c101550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:24:46.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.655+0000 7f303b200700 1 -- 172.21.15.117:0/3660844174 shutdown_connections 2024-06-03T03:24:46.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:46.655+0000 7f303b200700 1 -- 172.21.15.117:0/3660844174 wait complete. 2024-06-03T03:24:47.683 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph tell mds.c status 2024-06-03T03:24:48.119 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:24:48.267 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:24:48 smithi117 ceph-mon[31649]: pgmap v490: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-03T03:24:48.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:24:48 smithi151 ceph-mon[37043]: pgmap v490: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-03T03:24:48.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:24:48 smithi183 ceph-mon[37326]: pgmap v490: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-03T03:24:50.131 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:24:50 smithi117 ceph-mon[31649]: pgmap v491: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-03T03:24:50.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:24:50 smithi151 ceph-mon[37043]: pgmap v491: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-03T03:24:50.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.479+0000 7f452b600700 1 -- 172.21.15.117:0/2607861034 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4524074fe0 msgr2=0x7f452410e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:50.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.479+0000 7f452b600700 1 --2- 172.21.15.117:0/2607861034 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4524074fe0 0x7f452410e9c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4510009990 tx=0x7f4510009ca0 comp rx=0 tx=0).stop 2024-06-03T03:24:50.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.479+0000 7f452b600700 1 -- 172.21.15.117:0/2607861034 shutdown_connections 2024-06-03T03:24:50.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.479+0000 7f452b600700 1 --2- 172.21.15.117:0/2607861034 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f452410ef00 0x7f45241112f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:50.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.479+0000 7f452b600700 1 --2- 172.21.15.117:0/2607861034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45240746c0 0x7f4524074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:50.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.479+0000 7f452b600700 1 --2- 172.21.15.117:0/2607861034 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4524074fe0 0x7f452410e9c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:50.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.479+0000 7f452b600700 1 -- 172.21.15.117:0/2607861034 >> 172.21.15.117:0/2607861034 conn(0x7f4524079650 msgr2=0x7f4524079a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:24:50.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.479+0000 7f452b600700 1 -- 172.21.15.117:0/2607861034 shutdown_connections 2024-06-03T03:24:50.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.480+0000 7f452b600700 1 -- 172.21.15.117:0/2607861034 wait complete. 2024-06-03T03:24:50.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:24:50 smithi183 ceph-mon[37326]: pgmap v491: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-03T03:24:50.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.481+0000 7f452b600700 1 Processor -- start 2024-06-03T03:24:50.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.481+0000 7f452b600700 1 -- start start 2024-06-03T03:24:50.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.481+0000 7f452b600700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45240746c0 0x7f452419ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:50.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.481+0000 7f452b600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4524074fe0 0x7f452419b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:50.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.481+0000 7f452b600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f452410ef00 0x7f452419f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:50.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.481+0000 7f452b600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f45241149e0 con 0x7f4524074fe0 2024-06-03T03:24:50.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.481+0000 7f452b600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4524114860 con 0x7f452410ef00 2024-06-03T03:24:50.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.481+0000 7f452b600700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4524114b60 con 0x7f45240746c0 2024-06-03T03:24:50.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.482+0000 7f4523e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4524074fe0 0x7f452419b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:50.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.482+0000 7f4523e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4524074fe0 0x7f452419b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53032/0 (socket says 172.21.15.117:53032) 2024-06-03T03:24:50.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.482+0000 7f4523e00700 1 -- 172.21.15.117:0/1124817496 learned_addr learned my addr 172.21.15.117:0/1124817496 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:24:50.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.482+0000 7f4528c00700 1 --2- 172.21.15.117:0/1124817496 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45240746c0 0x7f452419ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:50.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.482+0000 7f4529600700 1 --2- 172.21.15.117:0/1124817496 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f452410ef00 0x7f452419f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:50.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.482+0000 7f4523e00700 1 -- 172.21.15.117:0/1124817496 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45240746c0 msgr2=0x7f452419ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:50.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.482+0000 7f4523e00700 1 --2- 172.21.15.117:0/1124817496 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45240746c0 0x7f452419ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:50.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.482+0000 7f4523e00700 1 -- 172.21.15.117:0/1124817496 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f452410ef00 msgr2=0x7f452419f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:50.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.482+0000 7f4523e00700 1 --2- 172.21.15.117:0/1124817496 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f452410ef00 0x7f452419f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:50.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.482+0000 7f4523e00700 1 -- 172.21.15.117:0/1124817496 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f452419fe60 con 0x7f4524074fe0 2024-06-03T03:24:50.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.482+0000 7f4523e00700 1 --2- 172.21.15.117:0/1124817496 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4524074fe0 0x7f452419b390 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f4510000c00 tx=0x7f451000bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:24:50.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.483+0000 7f4521600700 1 -- 172.21.15.117:0/1124817496 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4510027070 con 0x7f4524074fe0 2024-06-03T03:24:50.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.483+0000 7f452b600700 1 -- 172.21.15.117:0/1124817496 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45241a00f0 con 0x7f4524074fe0 2024-06-03T03:24:50.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.483+0000 7f4521600700 1 -- 172.21.15.117:0/1124817496 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45100056d0 con 0x7f4524074fe0 2024-06-03T03:24:50.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.483+0000 7f4521600700 1 -- 172.21.15.117:0/1124817496 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4510022e10 con 0x7f4524074fe0 2024-06-03T03:24:50.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.483+0000 7f452b600700 1 -- 172.21.15.117:0/1124817496 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45241a79d0 con 0x7f4524074fe0 2024-06-03T03:24:50.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.486+0000 7f4521600700 1 -- 172.21.15.117:0/1124817496 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f451002a070 con 0x7f4524074fe0 2024-06-03T03:24:50.489 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.488+0000 7f4535fbbb80 1 Processor -- start 2024-06-03T03:24:50.489 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.488+0000 7f4535fbbb80 1 -- start start 2024-06-03T03:24:50.490 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.488+0000 7f4521600700 1 --2- 172.21.15.117:0/1124817496 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f450c0a2ca0 0x7f450c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:50.490 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.489+0000 7f4535fbbb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccc8b62df0 0x55ccc8be6020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:50.490 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.489+0000 7f4535fbbb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ccc8be6560 0x55ccc8be69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:50.490 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.489+0000 7f4535fbbb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ccc8bf12b0 0x55ccc8bf36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:50.490 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.489+0000 7f4535fbbb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ccc8af59c0 con 0x55ccc8b62df0 2024-06-03T03:24:50.490 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.489+0000 7f4535fbbb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ccc8b38ab0 con 0x55ccc8bf12b0 2024-06-03T03:24:50.491 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.489+0000 7f4535fbbb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ccc8af38a0 con 0x55ccc8be6560 2024-06-03T03:24:50.491 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.489+0000 7f4521600700 1 -- 172.21.15.117:0/1124817496 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f45100cfff0 con 0x7f4524074fe0 2024-06-03T03:24:50.492 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.489+0000 7f4529600700 1 --2- 172.21.15.117:0/1124817496 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f450c0a2ca0 0x7f450c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:50.492 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.489+0000 7f4529600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ccc8bf12b0 0x55ccc8bf36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:50.492 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.489+0000 7f4529600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ccc8bf12b0 0x55ccc8bf36a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:60502/0 (socket says 172.21.15.117:60502) 2024-06-03T03:24:50.492 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.489+0000 7f4529600700 1 -- 172.21.15.117:0/1311258283 learned_addr learned my addr 172.21.15.117:0/1311258283 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:24:50.492 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.489+0000 7f4528c00700 1 --2- 172.21.15.117:0/1311258283 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccc8b62df0 0x55ccc8be6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:50.492 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.489+0000 7f4523e00700 1 --2- 172.21.15.117:0/1311258283 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ccc8be6560 0x55ccc8be69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:50.492 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.490+0000 7f4529600700 1 -- 172.21.15.117:0/1311258283 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ccc8be6560 msgr2=0x55ccc8be69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:24:50.492 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.490+0000 7f4529600700 1 --2- 172.21.15.117:0/1311258283 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ccc8be6560 0x55ccc8be69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:50.492 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.490+0000 7f4529600700 1 -- 172.21.15.117:0/1311258283 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccc8b62df0 msgr2=0x55ccc8be6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:24:50.492 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.490+0000 7f4529600700 1 --2- 172.21.15.117:0/1311258283 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccc8b62df0 0x55ccc8be6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:50.492 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.490+0000 7f4529600700 1 -- 172.21.15.117:0/1311258283 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccc8ae0f40 con 0x55ccc8bf12b0 2024-06-03T03:24:50.492 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.490+0000 7f4529600700 1 --2- 172.21.15.117:0/1124817496 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f450c0a2ca0 0x7f450c0a5160 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f451c013eb0 tx=0x7f451c013530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:24:50.492 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.490+0000 7f4529600700 1 --2- 172.21.15.117:0/1311258283 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ccc8bf12b0 0x55ccc8bf36a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f451c01aa50 tx=0x7f451c013f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c72aaa7d6796dd90 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:24:50.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.490+0000 7f4516000700 1 -- 172.21.15.117:0/1311258283 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f451c021070 con 0x55ccc8bf12b0 2024-06-03T03:24:50.494 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.490+0000 7f4516000700 1 -- 172.21.15.117:0/1311258283 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f451c01ba40 con 0x55ccc8bf12b0 2024-06-03T03:24:50.494 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.490+0000 7f4516000700 1 -- 172.21.15.117:0/1311258283 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f451c003df0 con 0x55ccc8bf12b0 2024-06-03T03:24:50.494 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.491+0000 7f4535fbbb80 1 -- 172.21.15.117:0/1311258283 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ccc8bf12b0 msgr2=0x55ccc8bf36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:50.494 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.491+0000 7f4535fbbb80 1 --2- 172.21.15.117:0/1311258283 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ccc8bf12b0 0x55ccc8bf36a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f451c01aa50 tx=0x7f451c013f90 comp rx=0 tx=0).stop 2024-06-03T03:24:50.494 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.491+0000 7f4535fbbb80 1 -- 172.21.15.117:0/1311258283 shutdown_connections 2024-06-03T03:24:50.494 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.491+0000 7f4535fbbb80 1 --2- 172.21.15.117:0/1311258283 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ccc8bf12b0 0x55ccc8bf36a0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:50.494 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.491+0000 7f4535fbbb80 1 --2- 172.21.15.117:0/1311258283 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccc8b62df0 0x55ccc8be6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:50.494 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.491+0000 7f4535fbbb80 1 --2- 172.21.15.117:0/1311258283 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ccc8be6560 0x55ccc8be69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:50.494 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.491+0000 7f4535fbbb80 1 -- 172.21.15.117:0/1311258283 >> 172.21.15.117:0/1311258283 conn(0x55ccc8b13120 msgr2=0x55ccc8be2460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:24:50.494 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.491+0000 7f4535fbbb80 1 -- 172.21.15.117:0/1311258283 shutdown_connections 2024-06-03T03:24:50.494 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.491+0000 7f4535fbbb80 1 -- 172.21.15.117:0/1311258283 wait complete. 2024-06-03T03:24:50.494 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.492+0000 7f4535fbbb80 1 -- start start 2024-06-03T03:24:50.494 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.492+0000 7f4535fbbb80 1 Processor -- start 2024-06-03T03:24:50.497 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.493+0000 7f4535fbbb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ccc8b62df0 0x55ccc8bf0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:50.497 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.493+0000 7f4535fbbb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ccc8be6560 0x55ccc8bedd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:50.497 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.493+0000 7f4535fbbb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccc8bf12b0 0x55ccc8bee280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:50.497 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.493+0000 7f4535fbbb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ccc8ad4f10 con 0x55ccc8bf12b0 2024-06-03T03:24:50.497 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.493+0000 7f4535fbbb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ccc8b34a70 con 0x55ccc8be6560 2024-06-03T03:24:50.497 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.493+0000 7f4535fbbb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ccc8bf3f10 con 0x55ccc8b62df0 2024-06-03T03:24:50.497 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.493+0000 7f4535fbbb80 10 client.0 _ll_register_callbacks cb 0x55ccc8ad26d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:24:50.497 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.493+0000 7f4523e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ccc8be6560 0x55ccc8bedd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:50.497 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.493+0000 7f4523e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ccc8be6560 0x55ccc8bedd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:60512/0 (socket says 172.21.15.117:60512) 2024-06-03T03:24:50.497 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.493+0000 7f4523e00700 1 -- 172.21.15.117:0/1875803632 learned_addr learned my addr 172.21.15.117:0/1875803632 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:24:50.497 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.493+0000 7f4528c00700 1 --2- 172.21.15.117:0/1875803632 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ccc8b62df0 0x55ccc8bf0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:50.497 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.493+0000 7f4529600700 1 --2- 172.21.15.117:0/1875803632 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccc8bf12b0 0x55ccc8bee280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:50.498 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.493+0000 7f4523e00700 1 -- 172.21.15.117:0/1875803632 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ccc8b62df0 msgr2=0x55ccc8bf0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:24:50.498 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.493+0000 7f4523e00700 1 --2- 172.21.15.117:0/1875803632 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ccc8b62df0 0x55ccc8bf0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:50.498 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.493+0000 7f4523e00700 1 -- 172.21.15.117:0/1875803632 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccc8bf12b0 msgr2=0x55ccc8bee280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:24:50.498 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.493+0000 7f4523e00700 1 --2- 172.21.15.117:0/1875803632 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccc8bf12b0 0x55ccc8bee280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:50.498 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.493+0000 7f4523e00700 1 -- 172.21.15.117:0/1875803632 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccc8a87880 con 0x55ccc8be6560 2024-06-03T03:24:50.498 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.494+0000 7f4523e00700 1 --2- 172.21.15.117:0/1875803632 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ccc8be6560 0x55ccc8bedd40 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f45100981d0 tx=0x7f45100227e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b482299ec8ef2b4f server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:24:50.498 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.494+0000 7f450aa00700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-06-03T03:24:50.498 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.494+0000 7f450aa00700 1 -- 172.21.15.117:0/1875803632 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45100b5060 con 0x55ccc8be6560 2024-06-03T03:24:50.498 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.494+0000 7f450aa00700 1 -- 172.21.15.117:0/1875803632 <== mon.2 v2:172.21.15.151:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f45100b5400 con 0x55ccc8be6560 2024-06-03T03:24:50.498 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.494+0000 7f4535fbbb80 1 -- 172.21.15.117:0/1875803632 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f451c01ba40 con 0x55ccc8be6560 2024-06-03T03:24:50.498 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.494+0000 7f450aa00700 1 -- 172.21.15.117:0/1875803632 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f451009be50 con 0x55ccc8be6560 2024-06-03T03:24:50.498 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.494+0000 7f4535fbbb80 10 client.25118 fetch_fsmap learned FSMap version 30 2024-06-03T03:24:50.498 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.494+0000 7f4535fbbb80 1 -- 172.21.15.117:0/1875803632 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ccc8bfe660 con 0x55ccc8be6560 2024-06-03T03:24:50.499 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.495+0000 7f450aa00700 1 -- 172.21.15.117:0/1875803632 <== mon.2 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f451c01ba40 con 0x55ccc8be6560 2024-06-03T03:24:50.499 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.495+0000 7f450aa00700 1 -- 172.21.15.117:0/1875803632 <== mon.2 v2:172.21.15.151:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f45100a4c40 con 0x55ccc8be6560 2024-06-03T03:24:50.499 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.495+0000 7f4535fbbb80 10 client.25118 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T03:24:50.499 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.495+0000 7f4535fbbb80 10 client.25118 resolve_mds: resolved name 'c' to daemon mds.c 2024-06-03T03:24:50.499 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.495+0000 7f4535fbbb80 20 client.25118 populate_metadata read hostname 'smithi117' 2024-06-03T03:24:50.499 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.495+0000 7f4535fbbb80 1 --2- 172.21.15.117:0/1875803632 >> [v2:172.21.15.183:6836/2697518526,v1:172.21.15.183:6837/2697518526] conn(0x55ccc8c00cc0 0x55ccc8c02130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:50.499 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.495+0000 7f4535fbbb80 4 client.25118 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T03:24:50.499 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.495+0000 7f4535fbbb80 1 -- 172.21.15.117:0/1875803632 --> [v2:172.21.15.183:6836/2697518526,v1:172.21.15.183:6837/2697518526] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ccc8b45fb0 con 0x55ccc8c00cc0 2024-06-03T03:24:50.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.496+0000 7f4528c00700 1 --2- 172.21.15.117:0/1875803632 >> [v2:172.21.15.183:6836/2697518526,v1:172.21.15.183:6837/2697518526] conn(0x55ccc8c00cc0 0x55ccc8c02130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:50.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.496+0000 7f4528c00700 1 --2- 172.21.15.117:0/1875803632 >> [v2:172.21.15.183:6836/2697518526,v1:172.21.15.183:6837/2697518526] conn(0x55ccc8c00cc0 0x55ccc8c02130 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9c6f2a1e9ab275f8 server_cookie=75823e379d7e2352 in_seq=0 out_seq=0 2024-06-03T03:24:50.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.496+0000 7f450aa00700 10 client.25118 ms_handle_connect on v2:172.21.15.183:6836/2697518526 2024-06-03T03:24:50.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.497+0000 7f450aa00700 1 -- 172.21.15.117:0/1875803632 <== mds.3 v2:172.21.15.183:6836/2697518526 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ccc8b45fb0 con 0x55ccc8c00cc0 2024-06-03T03:24:50.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.497+0000 7f450aa00700 10 client.25118 handle_command_reply: tid=0 2024-06-03T03:24:50.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.497+0000 7f4535fbbb80 1 client.25118 shutdown 2024-06-03T03:24:50.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.497+0000 7f4535fbbb80 1 -- 172.21.15.117:0/1875803632 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ccc8be6560 msgr2=0x55ccc8bedd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:50.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.497+0000 7f4535fbbb80 1 --2- 172.21.15.117:0/1875803632 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ccc8be6560 0x55ccc8bedd40 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f45100981d0 tx=0x7f45100227e0 comp rx=0 tx=0).stop 2024-06-03T03:24:50.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.497+0000 7f4535fbbb80 1 -- 172.21.15.117:0/1875803632 shutdown_connections 2024-06-03T03:24:50.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.497+0000 7f4535fbbb80 1 --2- 172.21.15.117:0/1875803632 >> [v2:172.21.15.183:6836/2697518526,v1:172.21.15.183:6837/2697518526] conn(0x55ccc8c00cc0 0x55ccc8c02130 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:50.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.498+0000 7f4535fbbb80 1 --2- 172.21.15.117:0/1875803632 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccc8bf12b0 0x55ccc8bee280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:50.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.498+0000 7f4535fbbb80 1 --2- 172.21.15.117:0/1875803632 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ccc8b62df0 0x55ccc8bf0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:50.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.498+0000 7f4535fbbb80 1 --2- 172.21.15.117:0/1875803632 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ccc8be6560 0x55ccc8bedd40 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:50.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.498+0000 7f450aa00700 0 client.25118 ms_handle_reset on v2:172.21.15.183:6836/2697518526 2024-06-03T03:24:50.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.498+0000 7f4535fbbb80 1 -- 172.21.15.117:0/1875803632 >> 172.21.15.117:0/1875803632 conn(0x55ccc8b13120 msgr2=0x55ccc8be16f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:24:50.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.498+0000 7f4535fbbb80 1 -- 172.21.15.117:0/1875803632 shutdown_connections 2024-06-03T03:24:50.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.498+0000 7f4535fbbb80 1 -- 172.21.15.117:0/1875803632 wait complete. 2024-06-03T03:24:50.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.498+0000 7f4535fbbb80 20 client.25118 trim_cache size 0 max 16384 2024-06-03T03:24:50.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.521+0000 7f4535fbbb80 1 Processor -- start 2024-06-03T03:24:50.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.521+0000 7f4535fbbb80 1 -- start start 2024-06-03T03:24:50.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.521+0000 7f4535fbbb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccc8b62df0 0x55ccc8be6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:50.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.521+0000 7f4535fbbb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ccc8be6560 0x55ccc8bf0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:50.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.521+0000 7f4535fbbb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ccc8bf12b0 0x55ccc8bf26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:50.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.521+0000 7f4535fbbb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ccc8c00470 con 0x55ccc8b62df0 2024-06-03T03:24:50.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.521+0000 7f4535fbbb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ccc8c00b40 con 0x55ccc8be6560 2024-06-03T03:24:50.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.521+0000 7f4535fbbb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ccc8c01210 con 0x55ccc8bf12b0 2024-06-03T03:24:50.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.522+0000 7f4528c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccc8b62df0 0x55ccc8be6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:50.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.522+0000 7f4523e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ccc8be6560 0x55ccc8bf0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:50.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.522+0000 7f4528c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccc8b62df0 0x55ccc8be6940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53060/0 (socket says 172.21.15.117:53060) 2024-06-03T03:24:50.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.522+0000 7f4528c00700 1 -- 172.21.15.117:0/854470235 learned_addr learned my addr 172.21.15.117:0/854470235 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:24:50.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.522+0000 7f4529600700 1 --2- 172.21.15.117:0/854470235 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ccc8bf12b0 0x55ccc8bf26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:50.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.522+0000 7f4523e00700 1 -- 172.21.15.117:0/854470235 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ccc8bf12b0 msgr2=0x55ccc8bf26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:24:50.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.522+0000 7f4523e00700 1 --2- 172.21.15.117:0/854470235 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ccc8bf12b0 0x55ccc8bf26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:50.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.522+0000 7f4523e00700 1 -- 172.21.15.117:0/854470235 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccc8b62df0 msgr2=0x55ccc8be6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:24:50.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.522+0000 7f4523e00700 1 --2- 172.21.15.117:0/854470235 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccc8b62df0 0x55ccc8be6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:50.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.522+0000 7f4523e00700 1 -- 172.21.15.117:0/854470235 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccc8bec2b0 con 0x55ccc8be6560 2024-06-03T03:24:50.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.522+0000 7f4528c00700 1 --2- 172.21.15.117:0/854470235 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccc8b62df0 0x55ccc8be6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:24:50.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.523+0000 7f4523e00700 1 --2- 172.21.15.117:0/854470235 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ccc8be6560 0x55ccc8bf0580 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f451000be00 tx=0x7f451009aa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=927d79110d3b13b8 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:24:50.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.523+0000 7f4515600700 1 -- 172.21.15.117:0/854470235 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f451009c4f0 con 0x55ccc8be6560 2024-06-03T03:24:50.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.523+0000 7f4515600700 1 -- 172.21.15.117:0/854470235 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45100a08a0 con 0x55ccc8be6560 2024-06-03T03:24:50.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.523+0000 7f4515600700 1 -- 172.21.15.117:0/854470235 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f451009bbf0 con 0x55ccc8be6560 2024-06-03T03:24:50.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.523+0000 7f4535fbbb80 1 -- 172.21.15.117:0/854470235 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ccc8be6560 msgr2=0x55ccc8bf0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:50.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.523+0000 7f4535fbbb80 1 --2- 172.21.15.117:0/854470235 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ccc8be6560 0x55ccc8bf0580 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f451000be00 tx=0x7f451009aa00 comp rx=0 tx=0).stop 2024-06-03T03:24:50.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.523+0000 7f4535fbbb80 1 -- 172.21.15.117:0/854470235 shutdown_connections 2024-06-03T03:24:50.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.524+0000 7f4535fbbb80 1 --2- 172.21.15.117:0/854470235 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ccc8bf12b0 0x55ccc8bf26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:50.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.524+0000 7f4535fbbb80 1 --2- 172.21.15.117:0/854470235 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccc8b62df0 0x55ccc8be6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:50.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.524+0000 7f4535fbbb80 1 --2- 172.21.15.117:0/854470235 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ccc8be6560 0x55ccc8bf0580 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:50.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.524+0000 7f4535fbbb80 1 -- 172.21.15.117:0/854470235 >> 172.21.15.117:0/854470235 conn(0x55ccc8b13120 msgr2=0x55ccc8bec630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:24:50.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.524+0000 7f4535fbbb80 1 -- 172.21.15.117:0/854470235 shutdown_connections 2024-06-03T03:24:50.528 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.524+0000 7f4535fbbb80 1 -- 172.21.15.117:0/854470235 wait complete. 2024-06-03T03:24:50.528 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.524+0000 7f4535fbbb80 1 -- start start 2024-06-03T03:24:50.528 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.524+0000 7f4535fbbb80 1 Processor -- start 2024-06-03T03:24:50.529 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.525+0000 7f4535fbbb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccc8b62df0 0x55ccc8c0b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:50.530 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.525+0000 7f4535fbbb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ccc8be6560 0x55ccc8c0baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:50.530 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.525+0000 7f4535fbbb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ccc8bf12b0 0x55ccc8c04550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:50.530 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.525+0000 7f4535fbbb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ccc8c01fb0 con 0x55ccc8b62df0 2024-06-03T03:24:50.530 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.525+0000 7f4535fbbb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ccc8c018e0 con 0x55ccc8bf12b0 2024-06-03T03:24:50.530 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.525+0000 7f4535fbbb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ccc8c02680 con 0x55ccc8be6560 2024-06-03T03:24:50.530 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.525+0000 7f4535fbbb80 10 client.0 _ll_register_callbacks cb 0x55ccc8ad26d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:24:50.530 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.525+0000 7f4528c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccc8b62df0 0x55ccc8c0b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:50.530 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.525+0000 7f4528c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccc8b62df0 0x55ccc8c0b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53068/0 (socket says 172.21.15.117:53068) 2024-06-03T03:24:50.530 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.525+0000 7f4529600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ccc8bf12b0 0x55ccc8c04550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:50.530 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.525+0000 7f4528c00700 1 -- 172.21.15.117:0/3628407327 learned_addr learned my addr 172.21.15.117:0/3628407327 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:24:50.530 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.525+0000 7f4523e00700 1 --2- 172.21.15.117:0/3628407327 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ccc8be6560 0x55ccc8c0baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:50.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.526+0000 7f4528c00700 1 -- 172.21.15.117:0/3628407327 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ccc8be6560 msgr2=0x55ccc8c0baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:24:50.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.526+0000 7f4528c00700 1 --2- 172.21.15.117:0/3628407327 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ccc8be6560 0x55ccc8c0baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:50.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.526+0000 7f4528c00700 1 -- 172.21.15.117:0/3628407327 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ccc8bf12b0 msgr2=0x55ccc8c04550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:24:50.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.526+0000 7f4528c00700 1 --2- 172.21.15.117:0/3628407327 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ccc8bf12b0 0x55ccc8c04550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:50.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.526+0000 7f4528c00700 1 -- 172.21.15.117:0/3628407327 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccc8c0c1a0 con 0x55ccc8b62df0 2024-06-03T03:24:50.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.526+0000 7f4523e00700 1 --2- 172.21.15.117:0/3628407327 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ccc8be6560 0x55ccc8c0baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:24:50.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.526+0000 7f4528c00700 1 --2- 172.21.15.117:0/3628407327 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccc8b62df0 0x55ccc8c0b5b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x55ccc8b34b80 tx=0x7f4518004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=987aac31d995c467 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:24:50.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.526+0000 7f450aa00700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-06-03T03:24:50.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.526+0000 7f450aa00700 1 -- 172.21.15.117:0/3628407327 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4518022030 con 0x55ccc8b62df0 2024-06-03T03:24:50.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.526+0000 7f450aa00700 1 -- 172.21.15.117:0/3628407327 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f4518003c20 con 0x55ccc8b62df0 2024-06-03T03:24:50.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.526+0000 7f4535fbbb80 1 -- 172.21.15.117:0/3628407327 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45100a08a0 con 0x55ccc8b62df0 2024-06-03T03:24:50.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.527+0000 7f450aa00700 1 -- 172.21.15.117:0/3628407327 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f451802f050 con 0x55ccc8b62df0 2024-06-03T03:24:50.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.527+0000 7f4535fbbb80 10 client.15009 fetch_fsmap learned FSMap version 30 2024-06-03T03:24:50.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.527+0000 7f4535fbbb80 1 -- 172.21.15.117:0/3628407327 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ccc8c75390 con 0x55ccc8b62df0 2024-06-03T03:24:50.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.527+0000 7f450aa00700 1 -- 172.21.15.117:0/3628407327 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45100a08a0 con 0x55ccc8b62df0 2024-06-03T03:24:50.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.527+0000 7f450aa00700 1 -- 172.21.15.117:0/3628407327 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f4518010530 con 0x55ccc8b62df0 2024-06-03T03:24:50.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.527+0000 7f4535fbbb80 10 client.15009 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T03:24:50.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.527+0000 7f4535fbbb80 10 client.15009 resolve_mds: resolved name 'c' to daemon mds.c 2024-06-03T03:24:50.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.527+0000 7f4535fbbb80 20 client.15009 populate_metadata read hostname 'smithi117' 2024-06-03T03:24:50.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.527+0000 7f4535fbbb80 1 --2- 172.21.15.117:0/3628407327 >> [v2:172.21.15.183:6836/2697518526,v1:172.21.15.183:6837/2697518526] conn(0x55ccc8c79fc0 0x55ccc8c78cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:50.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.527+0000 7f4535fbbb80 4 client.15009 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-03T03:24:50.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.527+0000 7f4535fbbb80 1 -- 172.21.15.117:0/3628407327 --> [v2:172.21.15.183:6836/2697518526,v1:172.21.15.183:6837/2697518526] -- command(tid 0: {"prefix": "status"}) -- 0x55ccc8b40530 con 0x55ccc8c79fc0 2024-06-03T03:24:50.533 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.528+0000 7f4523e00700 1 --2- 172.21.15.117:0/3628407327 >> [v2:172.21.15.183:6836/2697518526,v1:172.21.15.183:6837/2697518526] conn(0x55ccc8c79fc0 0x55ccc8c78cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:50.533 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.528+0000 7f4523e00700 1 --2- 172.21.15.117:0/3628407327 >> [v2:172.21.15.183:6836/2697518526,v1:172.21.15.183:6837/2697518526] conn(0x55ccc8c79fc0 0x55ccc8c78cd0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3fb057c325e32da4 server_cookie=717604b630883b3c in_seq=0 out_seq=0 2024-06-03T03:24:50.534 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.528+0000 7f450aa00700 10 client.15009 ms_handle_connect on v2:172.21.15.183:6836/2697518526 2024-06-03T03:24:50.534 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.529+0000 7f450aa00700 1 -- 172.21.15.117:0/3628407327 <== mds.3 v2:172.21.15.183:6836/2697518526 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55ccc8b40530 con 0x55ccc8c79fc0 2024-06-03T03:24:50.534 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.529+0000 7f450aa00700 10 client.15009 handle_command_reply: tid=0 2024-06-03T03:24:50.534 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.529+0000 7f4535fbbb80 1 client.15009 shutdown 2024-06-03T03:24:50.534 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.529+0000 7f4535fbbb80 1 -- 172.21.15.117:0/3628407327 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccc8b62df0 msgr2=0x55ccc8c0b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:50.534 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.529+0000 7f4535fbbb80 1 --2- 172.21.15.117:0/3628407327 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccc8b62df0 0x55ccc8c0b5b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x55ccc8b34b80 tx=0x7f4518004410 comp rx=0 tx=0).stop 2024-06-03T03:24:50.534 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.529+0000 7f4535fbbb80 1 -- 172.21.15.117:0/3628407327 shutdown_connections 2024-06-03T03:24:50.534 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.529+0000 7f4535fbbb80 1 --2- 172.21.15.117:0/3628407327 >> [v2:172.21.15.183:6836/2697518526,v1:172.21.15.183:6837/2697518526] conn(0x55ccc8c79fc0 0x55ccc8c78cd0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:50.534 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.529+0000 7f4535fbbb80 1 --2- 172.21.15.117:0/3628407327 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ccc8bf12b0 0x55ccc8c04550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:50.534 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.529+0000 7f4535fbbb80 1 --2- 172.21.15.117:0/3628407327 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccc8b62df0 0x55ccc8c0b5b0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:50.534 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.529+0000 7f4535fbbb80 1 --2- 172.21.15.117:0/3628407327 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ccc8be6560 0x55ccc8c0baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:50.534 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.529+0000 7f4535fbbb80 1 -- 172.21.15.117:0/3628407327 >> 172.21.15.117:0/3628407327 conn(0x55ccc8b13120 msgr2=0x55ccc8be0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:24:50.535 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.529+0000 7f450aa00700 0 client.15009 ms_handle_reset on v2:172.21.15.183:6836/2697518526 2024-06-03T03:24:50.535 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.529+0000 7f4535fbbb80 1 -- 172.21.15.117:0/3628407327 shutdown_connections 2024-06-03T03:24:50.535 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.529+0000 7f4535fbbb80 1 -- 172.21.15.117:0/3628407327 wait complete. 2024-06-03T03:24:50.535 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.530+0000 7f4535fbbb80 20 client.15009 trim_cache size 0 max 16384 2024-06-03T03:24:50.535 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-06-03T03:24:50.535 INFO:teuthology.orchestra.run.smithi117.stdout: "cluster_fsid": "a0e8473a-2156-11ef-bc9b-c7b262605968", 2024-06-03T03:24:50.535 INFO:teuthology.orchestra.run.smithi117.stdout: "whoami": 3, 2024-06-03T03:24:50.535 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 24499, 2024-06-03T03:24:50.535 INFO:teuthology.orchestra.run.smithi117.stdout: "want_state": "up:active", 2024-06-03T03:24:50.535 INFO:teuthology.orchestra.run.smithi117.stdout: "state": "up:active", 2024-06-03T03:24:50.536 INFO:teuthology.orchestra.run.smithi117.stdout: "fs_name": "cephfs", 2024-06-03T03:24:50.536 INFO:teuthology.orchestra.run.smithi117.stdout: "rank_uptime": 225.526891952, 2024-06-03T03:24:50.536 INFO:teuthology.orchestra.run.smithi117.stdout: "mdsmap_epoch": 30, 2024-06-03T03:24:50.536 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_epoch": 99, 2024-06-03T03:24:50.536 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_epoch_barrier": 97, 2024-06-03T03:24:50.536 INFO:teuthology.orchestra.run.smithi117.stdout: "uptime": 343.80868612199998 2024-06-03T03:24:50.536 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-06-03T03:24:50.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.530+0000 7f452b600700 1 -- 172.21.15.117:0/1124817496 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f450c0a2ca0 msgr2=0x7f450c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:50.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.530+0000 7f452b600700 1 --2- 172.21.15.117:0/1124817496 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f450c0a2ca0 0x7f450c0a5160 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f451c013eb0 tx=0x7f451c013530 comp rx=0 tx=0).stop 2024-06-03T03:24:50.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.530+0000 7f452b600700 1 -- 172.21.15.117:0/1124817496 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4524074fe0 msgr2=0x7f452419b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:50.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.530+0000 7f452b600700 1 --2- 172.21.15.117:0/1124817496 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4524074fe0 0x7f452419b390 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f4510000c00 tx=0x7f451000bab0 comp rx=0 tx=0).stop 2024-06-03T03:24:50.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.530+0000 7f452b600700 1 -- 172.21.15.117:0/1124817496 shutdown_connections 2024-06-03T03:24:50.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.530+0000 7f452b600700 1 --2- 172.21.15.117:0/1124817496 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f450c0a2ca0 0x7f450c0a5160 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:50.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.530+0000 7f452b600700 1 --2- 172.21.15.117:0/1124817496 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f452410ef00 0x7f452419f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:50.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.530+0000 7f452b600700 1 --2- 172.21.15.117:0/1124817496 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45240746c0 0x7f452419ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:50.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.530+0000 7f452b600700 1 --2- 172.21.15.117:0/1124817496 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4524074fe0 0x7f452419b390 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:50.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.530+0000 7f452b600700 1 -- 172.21.15.117:0/1124817496 >> 172.21.15.117:0/1124817496 conn(0x7f4524079650 msgr2=0x7f45240ff770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:24:50.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.531+0000 7f452b600700 1 -- 172.21.15.117:0/1124817496 shutdown_connections 2024-06-03T03:24:50.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:50.531+0000 7f452b600700 1 -- 172.21.15.117:0/1124817496 wait complete. 2024-06-03T03:24:51.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:24:51 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:24:51.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:24:51 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:24:51.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:24:51 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:24:51.621 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph tell mds.a status 2024-06-03T03:24:52.062 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:24:52.289 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:24:52 smithi117 ceph-mon[31649]: pgmap v492: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:24:52.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:24:52 smithi151 ceph-mon[37043]: pgmap v492: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:24:52.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:24:52 smithi183 ceph-mon[37326]: pgmap v492: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:24:53.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:24:53 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:24:53.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:24:53 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:24:53.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:24:53 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:24:53.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:24:53 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:24:53.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:24:53 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:24:53.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:24:53 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:24:53.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:24:53 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:24:53.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:24:53 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:24:53.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:24:53 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:24:54.246 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:24:54 smithi117 ceph-mon[31649]: pgmap v493: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-06-03T03:24:54.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:24:54 smithi151 ceph-mon[37043]: pgmap v493: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-06-03T03:24:54.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:24:54 smithi183 ceph-mon[37326]: pgmap v493: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-06-03T03:24:54.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.631+0000 7f9202a00700 1 -- 172.21.15.117:0/3207683500 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f91fc109480 msgr2=0x7f91fc115990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:54.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.631+0000 7f9202a00700 1 --2- 172.21.15.117:0/3207683500 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f91fc109480 0x7f91fc115990 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f91f400b0a0 tx=0x7f91f400b3b0 comp rx=0 tx=0).stop 2024-06-03T03:24:54.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.631+0000 7f9202a00700 1 -- 172.21.15.117:0/3207683500 shutdown_connections 2024-06-03T03:24:54.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.631+0000 7f9202a00700 1 --2- 172.21.15.117:0/3207683500 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f91fc109480 0x7f91fc115990 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:54.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.631+0000 7f9202a00700 1 --2- 172.21.15.117:0/3207683500 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f91fc10eac0 0x7f91fc10eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:54.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.631+0000 7f9202a00700 1 --2- 172.21.15.117:0/3207683500 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91fc108ac0 0x7f91fc108f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:54.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.631+0000 7f9202a00700 1 -- 172.21.15.117:0/3207683500 >> 172.21.15.117:0/3207683500 conn(0x7f91fc075c60 msgr2=0x7f91fc1072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:24:54.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.631+0000 7f9202a00700 1 -- 172.21.15.117:0/3207683500 shutdown_connections 2024-06-03T03:24:54.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.631+0000 7f9202a00700 1 -- 172.21.15.117:0/3207683500 wait complete. 2024-06-03T03:24:54.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.632+0000 7f9202a00700 1 Processor -- start 2024-06-03T03:24:54.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.633+0000 7f9202a00700 1 -- start start 2024-06-03T03:24:54.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.633+0000 7f9202a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f91fc108ac0 0x7f91fc19f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:54.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.633+0000 7f9202a00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f91fc109480 0x7f91fc19f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:54.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.633+0000 7f9202a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91fc10eac0 0x7f91fc1a3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:54.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.633+0000 7f9202a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f91fc118ea0 con 0x7f91fc10eac0 2024-06-03T03:24:54.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.633+0000 7f9202a00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f91fc118d20 con 0x7f91fc108ac0 2024-06-03T03:24:54.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.633+0000 7f9202a00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f91fc119020 con 0x7f91fc109480 2024-06-03T03:24:54.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.633+0000 7f91fbe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f91fc108ac0 0x7f91fc19f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:54.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.633+0000 7f91fbe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f91fc108ac0 0x7f91fc19f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:60562/0 (socket says 172.21.15.117:60562) 2024-06-03T03:24:54.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.633+0000 7f91fbe00700 1 -- 172.21.15.117:0/3557368853 learned_addr learned my addr 172.21.15.117:0/3557368853 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:24:54.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.633+0000 7f9200a00700 1 --2- 172.21.15.117:0/3557368853 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91fc10eac0 0x7f91fc1a3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:54.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.633+0000 7f91fb400700 1 --2- 172.21.15.117:0/3557368853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f91fc109480 0x7f91fc19f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:54.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.634+0000 7f9200a00700 1 -- 172.21.15.117:0/3557368853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f91fc109480 msgr2=0x7f91fc19f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:54.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.634+0000 7f9200a00700 1 --2- 172.21.15.117:0/3557368853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f91fc109480 0x7f91fc19f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:54.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.634+0000 7f9200a00700 1 -- 172.21.15.117:0/3557368853 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f91fc108ac0 msgr2=0x7f91fc19f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:54.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.634+0000 7f9200a00700 1 --2- 172.21.15.117:0/3557368853 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f91fc108ac0 0x7f91fc19f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:54.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.634+0000 7f9200a00700 1 -- 172.21.15.117:0/3557368853 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91fc1a4240 con 0x7f91fc10eac0 2024-06-03T03:24:54.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.634+0000 7f9200a00700 1 --2- 172.21.15.117:0/3557368853 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91fc10eac0 0x7f91fc1a3b00 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f91f400b070 tx=0x7f91f4004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:24:54.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.635+0000 7f91f8c00700 1 -- 172.21.15.117:0/3557368853 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f4021070 con 0x7f91fc10eac0 2024-06-03T03:24:54.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.635+0000 7f91f8c00700 1 -- 172.21.15.117:0/3557368853 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91f4007b80 con 0x7f91fc10eac0 2024-06-03T03:24:54.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.635+0000 7f9202a00700 1 -- 172.21.15.117:0/3557368853 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91fc1a44d0 con 0x7f91fc10eac0 2024-06-03T03:24:54.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.635+0000 7f91f8c00700 1 -- 172.21.15.117:0/3557368853 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f4005040 con 0x7f91fc10eac0 2024-06-03T03:24:54.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.635+0000 7f9202a00700 1 -- 172.21.15.117:0/3557368853 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91fc1abdb0 con 0x7f91fc10eac0 2024-06-03T03:24:54.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.638+0000 7f91f8c00700 1 -- 172.21.15.117:0/3557368853 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f91f4024070 con 0x7f91fc10eac0 2024-06-03T03:24:54.639 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.639+0000 7f920d27db80 1 Processor -- start 2024-06-03T03:24:54.640 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.639+0000 7f91f8c00700 1 --2- 172.21.15.117:0/3557368853 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f91e00a2ca0 0x7f91e00a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:54.640 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.639+0000 7f920d27db80 1 -- start start 2024-06-03T03:24:54.640 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.640+0000 7f91f8c00700 1 -- 172.21.15.117:0/3557368853 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f91f40d0250 con 0x7f91fc10eac0 2024-06-03T03:24:54.640 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.640+0000 7f91fb400700 1 --2- 172.21.15.117:0/3557368853 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f91e00a2ca0 0x7f91e00a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:54.641 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.640+0000 7f920d27db80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e68ca7b40 0x562e68ca7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:54.641 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.640+0000 7f920d27db80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e68ca8460 0x562e68ca1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:54.641 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.640+0000 7f920d27db80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e68ca21f0 0x562e68ca2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:54.641 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.640+0000 7f920d27db80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562e68bb09c0 con 0x562e68ca8460 2024-06-03T03:24:54.641 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.640+0000 7f920d27db80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562e68bf3ab0 con 0x562e68ca21f0 2024-06-03T03:24:54.641 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.640+0000 7f920d27db80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562e68bae8a0 con 0x562e68ca7b40 2024-06-03T03:24:54.642 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.640+0000 7f91fb400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e68ca8460 0x562e68ca1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:54.642 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.640+0000 7f91fbe00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e68ca7b40 0x562e68ca7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:54.642 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.640+0000 7f91fbe00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e68ca7b40 0x562e68ca7f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:34266/0 (socket says 172.21.15.117:34266) 2024-06-03T03:24:54.642 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.640+0000 7f91fbe00700 1 -- 172.21.15.117:0/1642071365 learned_addr learned my addr 172.21.15.117:0/1642071365 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:24:54.642 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.641+0000 7f9200a00700 1 --2- 172.21.15.117:0/1642071365 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e68ca21f0 0x562e68ca2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:54.642 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.641+0000 7f91fbe00700 1 -- 172.21.15.117:0/1642071365 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e68ca21f0 msgr2=0x562e68ca2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:24:54.642 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.641+0000 7f91fbe00700 1 --2- 172.21.15.117:0/1642071365 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e68ca21f0 0x562e68ca2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:54.643 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.641+0000 7f91fbe00700 1 -- 172.21.15.117:0/1642071365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e68ca8460 msgr2=0x562e68ca1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:24:54.643 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.641+0000 7f91fbe00700 1 --2- 172.21.15.117:0/1642071365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e68ca8460 0x562e68ca1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:54.643 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.641+0000 7f91fbe00700 1 -- 172.21.15.117:0/1642071365 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e68b9bf40 con 0x562e68ca7b40 2024-06-03T03:24:54.643 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.641+0000 7f91fb400700 1 --2- 172.21.15.117:0/3557368853 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f91e00a2ca0 0x7f91e00a5160 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f91fc077aa0 tx=0x7f91e8013900 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:24:54.643 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.641+0000 7f91fbe00700 1 --2- 172.21.15.117:0/1642071365 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e68ca7b40 0x562e68ca7f20 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f91ec009dd0 tx=0x7f91ec011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7f74ab078b75689 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:24:54.643 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.641+0000 7f91f1600700 1 -- 172.21.15.117:0/1642071365 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91ec004a30 con 0x562e68ca7b40 2024-06-03T03:24:54.643 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.641+0000 7f91f1600700 1 -- 172.21.15.117:0/1642071365 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91ec005500 con 0x562e68ca7b40 2024-06-03T03:24:54.643 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.641+0000 7f91f1600700 1 -- 172.21.15.117:0/1642071365 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91ec0198a0 con 0x562e68ca7b40 2024-06-03T03:24:54.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.642+0000 7f920d27db80 1 -- 172.21.15.117:0/1642071365 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e68ca7b40 msgr2=0x562e68ca7f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:54.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.642+0000 7f920d27db80 1 --2- 172.21.15.117:0/1642071365 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e68ca7b40 0x562e68ca7f20 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f91ec009dd0 tx=0x7f91ec011720 comp rx=0 tx=0).stop 2024-06-03T03:24:54.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.642+0000 7f920d27db80 1 -- 172.21.15.117:0/1642071365 shutdown_connections 2024-06-03T03:24:54.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.642+0000 7f920d27db80 1 --2- 172.21.15.117:0/1642071365 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e68ca21f0 0x562e68ca2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:54.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.642+0000 7f920d27db80 1 --2- 172.21.15.117:0/1642071365 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e68ca7b40 0x562e68ca7f20 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:54.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.642+0000 7f920d27db80 1 --2- 172.21.15.117:0/1642071365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e68ca8460 0x562e68ca1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:54.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.642+0000 7f920d27db80 1 -- 172.21.15.117:0/1642071365 >> 172.21.15.117:0/1642071365 conn(0x562e68bce120 msgr2=0x562e68c9cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:24:54.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.642+0000 7f920d27db80 1 -- 172.21.15.117:0/1642071365 shutdown_connections 2024-06-03T03:24:54.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.642+0000 7f920d27db80 1 -- 172.21.15.117:0/1642071365 wait complete. 2024-06-03T03:24:54.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.643+0000 7f920d27db80 1 -- start start 2024-06-03T03:24:54.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.643+0000 7f920d27db80 1 Processor -- start 2024-06-03T03:24:54.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.644+0000 7f920d27db80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e68ca21f0 0x562e68cae170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:54.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.644+0000 7f920d27db80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e68ca7b40 0x562e68cab470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:54.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.644+0000 7f920d27db80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e68ca8460 0x562e68cab9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:54.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.644+0000 7f920d27db80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562e68b8ff10 con 0x562e68ca7b40 2024-06-03T03:24:54.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.644+0000 7f920d27db80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562e68befa70 con 0x562e68ca21f0 2024-06-03T03:24:54.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.644+0000 7f920d27db80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562e68caee40 con 0x562e68ca8460 2024-06-03T03:24:54.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.644+0000 7f920d27db80 10 client.0 _ll_register_callbacks cb 0x562e68b89330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:24:54.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.644+0000 7f9200a00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e68ca8460 0x562e68cab9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:54.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.644+0000 7f91fbe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e68ca21f0 0x562e68cae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:54.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.644+0000 7f91fb400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e68ca7b40 0x562e68cab470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:54.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.644+0000 7f91fbe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e68ca21f0 0x562e68cae170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:60586/0 (socket says 172.21.15.117:60586) 2024-06-03T03:24:54.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.644+0000 7f9200a00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e68ca8460 0x562e68cab9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:34278/0 (socket says 172.21.15.117:34278) 2024-06-03T03:24:54.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.644+0000 7f91fbe00700 1 -- 172.21.15.117:0/2806444091 learned_addr learned my addr 172.21.15.117:0/2806444091 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:24:54.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.645+0000 7f91fbe00700 1 -- 172.21.15.117:0/2806444091 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e68ca8460 msgr2=0x562e68cab9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:24:54.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.645+0000 7f91fbe00700 1 --2- 172.21.15.117:0/2806444091 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e68ca8460 0x562e68cab9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:54.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.645+0000 7f91fbe00700 1 -- 172.21.15.117:0/2806444091 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e68ca7b40 msgr2=0x562e68cab470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:24:54.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.645+0000 7f91fbe00700 1 --2- 172.21.15.117:0/2806444091 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e68ca7b40 0x562e68cab470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:54.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.645+0000 7f91fbe00700 1 -- 172.21.15.117:0/2806444091 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e68b42880 con 0x562e68ca21f0 2024-06-03T03:24:54.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.645+0000 7f9200a00700 1 --2- 172.21.15.117:0/2806444091 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e68ca8460 0x562e68cab9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:24:54.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.645+0000 7f91fb400700 1 --2- 172.21.15.117:0/2806444091 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e68ca7b40 0x562e68cab470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:24:54.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.645+0000 7f91fbe00700 1 --2- 172.21.15.117:0/2806444091 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e68ca21f0 0x562e68cae170 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f91ec01b040 tx=0x7f91ec0051c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98f4990ae107469e server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:24:54.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.645+0000 7f91e6000700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-06-03T03:24:54.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.646+0000 7f91e6000700 1 -- 172.21.15.117:0/2806444091 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91ec026070 con 0x562e68ca21f0 2024-06-03T03:24:54.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.646+0000 7f91e6000700 1 -- 172.21.15.117:0/2806444091 <== mon.2 v2:172.21.15.151:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f91ec004460 con 0x562e68ca21f0 2024-06-03T03:24:54.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.646+0000 7f920d27db80 1 -- 172.21.15.117:0/2806444091 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91ec005500 con 0x562e68ca21f0 2024-06-03T03:24:54.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.646+0000 7f91e6000700 1 -- 172.21.15.117:0/2806444091 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f91ec032050 con 0x562e68ca21f0 2024-06-03T03:24:54.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.646+0000 7f920d27db80 10 client.25136 fetch_fsmap learned FSMap version 30 2024-06-03T03:24:54.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.646+0000 7f920d27db80 1 -- 172.21.15.117:0/2806444091 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562e68cb9590 con 0x562e68ca21f0 2024-06-03T03:24:54.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.646+0000 7f91e6000700 1 -- 172.21.15.117:0/2806444091 <== mon.2 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91ec005500 con 0x562e68ca21f0 2024-06-03T03:24:54.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.647+0000 7f91e6000700 1 -- 172.21.15.117:0/2806444091 <== mon.2 v2:172.21.15.151:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f91ec022450 con 0x562e68ca21f0 2024-06-03T03:24:54.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.647+0000 7f920d27db80 10 client.25136 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T03:24:54.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.647+0000 7f920d27db80 10 client.25136 resolve_mds: resolved name 'a' to daemon mds.a 2024-06-03T03:24:54.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.647+0000 7f920d27db80 20 client.25136 populate_metadata read hostname 'smithi117' 2024-06-03T03:24:54.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.647+0000 7f920d27db80 1 --2- 172.21.15.117:0/2806444091 >> [v2:172.21.15.117:6840/1212242967,v1:172.21.15.117:6841/1212242967] conn(0x562e68cbb9d0 0x562e68cbce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:54.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.647+0000 7f920d27db80 4 client.25136 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T03:24:54.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.647+0000 7f920d27db80 1 -- 172.21.15.117:0/2806444091 --> [v2:172.21.15.117:6840/1212242967,v1:172.21.15.117:6841/1212242967] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e68c00fb0 con 0x562e68cbb9d0 2024-06-03T03:24:54.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.647+0000 7f91fb400700 1 --2- 172.21.15.117:0/2806444091 >> [v2:172.21.15.117:6840/1212242967,v1:172.21.15.117:6841/1212242967] conn(0x562e68cbb9d0 0x562e68cbce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:54.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.648+0000 7f91fb400700 1 --2- 172.21.15.117:0/2806444091 >> [v2:172.21.15.117:6840/1212242967,v1:172.21.15.117:6841/1212242967] conn(0x562e68cbb9d0 0x562e68cbce40 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2a0d867813b8f51f server_cookie=a00b986b325b8149 in_seq=0 out_seq=0 2024-06-03T03:24:54.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.648+0000 7f91e6000700 10 client.25136 ms_handle_connect on v2:172.21.15.117:6840/1212242967 2024-06-03T03:24:54.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.648+0000 7f91e6000700 1 -- 172.21.15.117:0/2806444091 <== mds.2 v2:172.21.15.117:6840/1212242967 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562e68c00fb0 con 0x562e68cbb9d0 2024-06-03T03:24:54.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.648+0000 7f91e6000700 10 client.25136 handle_command_reply: tid=0 2024-06-03T03:24:54.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.649+0000 7f920d27db80 1 client.25136 shutdown 2024-06-03T03:24:54.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.649+0000 7f920d27db80 1 -- 172.21.15.117:0/2806444091 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e68ca21f0 msgr2=0x562e68cae170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:54.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.649+0000 7f920d27db80 1 --2- 172.21.15.117:0/2806444091 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e68ca21f0 0x562e68cae170 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f91ec01b040 tx=0x7f91ec0051c0 comp rx=0 tx=0).stop 2024-06-03T03:24:54.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.649+0000 7f920d27db80 1 -- 172.21.15.117:0/2806444091 shutdown_connections 2024-06-03T03:24:54.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.649+0000 7f920d27db80 1 --2- 172.21.15.117:0/2806444091 >> [v2:172.21.15.117:6840/1212242967,v1:172.21.15.117:6841/1212242967] conn(0x562e68cbb9d0 0x562e68cbce40 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:54.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.649+0000 7f920d27db80 1 --2- 172.21.15.117:0/2806444091 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e68ca8460 0x562e68cab9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:54.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.649+0000 7f920d27db80 1 --2- 172.21.15.117:0/2806444091 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e68ca21f0 0x562e68cae170 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:54.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.649+0000 7f91e6000700 0 client.25136 ms_handle_reset on v2:172.21.15.117:6840/1212242967 2024-06-03T03:24:54.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.649+0000 7f920d27db80 1 --2- 172.21.15.117:0/2806444091 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e68ca7b40 0x562e68cab470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:54.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.649+0000 7f920d27db80 1 -- 172.21.15.117:0/2806444091 >> 172.21.15.117:0/2806444091 conn(0x562e68bce120 msgr2=0x562e68c96740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:24:54.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.649+0000 7f920d27db80 1 -- 172.21.15.117:0/2806444091 shutdown_connections 2024-06-03T03:24:54.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.649+0000 7f920d27db80 1 -- 172.21.15.117:0/2806444091 wait complete. 2024-06-03T03:24:54.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.649+0000 7f920d27db80 20 client.25136 trim_cache size 0 max 16384 2024-06-03T03:24:54.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.672+0000 7f920d27db80 1 Processor -- start 2024-06-03T03:24:54.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.672+0000 7f920d27db80 1 -- start start 2024-06-03T03:24:54.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.673+0000 7f920d27db80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e68ca21f0 0x562e68ca1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:54.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.673+0000 7f920d27db80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e68ca7b40 0x562e68ca5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:54.674 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.673+0000 7f920d27db80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e68ca8460 0x562e68c9d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:54.674 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.673+0000 7f920d27db80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562e68cbb180 con 0x562e68ca21f0 2024-06-03T03:24:54.674 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.673+0000 7f920d27db80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562e68cbb850 con 0x562e68ca7b40 2024-06-03T03:24:54.674 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.673+0000 7f920d27db80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562e68cbbf20 con 0x562e68ca8460 2024-06-03T03:24:54.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.673+0000 7f91fbe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e68ca21f0 0x562e68ca1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:54.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.673+0000 7f91fb400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e68ca7b40 0x562e68ca5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:54.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.673+0000 7f91fb400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e68ca7b40 0x562e68ca5c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:60594/0 (socket says 172.21.15.117:60594) 2024-06-03T03:24:54.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.673+0000 7f91fb400700 1 -- 172.21.15.117:0/1976908151 learned_addr learned my addr 172.21.15.117:0/1976908151 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:24:54.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.673+0000 7f9200a00700 1 --2- 172.21.15.117:0/1976908151 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e68ca8460 0x562e68c9d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:54.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.673+0000 7f91fb400700 1 -- 172.21.15.117:0/1976908151 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e68ca8460 msgr2=0x562e68c9d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:24:54.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.673+0000 7f91fb400700 1 --2- 172.21.15.117:0/1976908151 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e68ca8460 0x562e68c9d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:54.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.673+0000 7f91fb400700 1 -- 172.21.15.117:0/1976908151 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e68ca21f0 msgr2=0x562e68ca1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:24:54.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.673+0000 7f91fb400700 1 --2- 172.21.15.117:0/1976908151 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e68ca21f0 0x562e68ca1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:54.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.673+0000 7f91fb400700 1 -- 172.21.15.117:0/1976908151 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e68ca99e0 con 0x562e68ca7b40 2024-06-03T03:24:54.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.674+0000 7f91fb400700 1 --2- 172.21.15.117:0/1976908151 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e68ca7b40 0x562e68ca5c30 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f91e8013d00 tx=0x7f91e801f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7fdcc226ed562fb server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:24:54.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.674+0000 7f91f0c00700 1 -- 172.21.15.117:0/1976908151 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e8012030 con 0x562e68ca7b40 2024-06-03T03:24:54.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.674+0000 7f91f0c00700 1 -- 172.21.15.117:0/1976908151 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91e80121d0 con 0x562e68ca7b40 2024-06-03T03:24:54.676 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.674+0000 7f91f0c00700 1 -- 172.21.15.117:0/1976908151 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e8005310 con 0x562e68ca7b40 2024-06-03T03:24:54.679 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.675+0000 7f920d27db80 1 -- 172.21.15.117:0/1976908151 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e68ca7b40 msgr2=0x562e68ca5c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:54.679 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.675+0000 7f920d27db80 1 --2- 172.21.15.117:0/1976908151 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e68ca7b40 0x562e68ca5c30 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f91e8013d00 tx=0x7f91e801f000 comp rx=0 tx=0).stop 2024-06-03T03:24:54.679 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.675+0000 7f920d27db80 1 -- 172.21.15.117:0/1976908151 shutdown_connections 2024-06-03T03:24:54.679 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.675+0000 7f920d27db80 1 --2- 172.21.15.117:0/1976908151 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e68ca8460 0x562e68c9d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:54.679 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.675+0000 7f920d27db80 1 --2- 172.21.15.117:0/1976908151 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e68ca21f0 0x562e68ca1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:54.679 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.675+0000 7f920d27db80 1 --2- 172.21.15.117:0/1976908151 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e68ca7b40 0x562e68ca5c30 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:54.679 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.675+0000 7f920d27db80 1 -- 172.21.15.117:0/1976908151 >> 172.21.15.117:0/1976908151 conn(0x562e68bce120 msgr2=0x562e68c9ca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:24:54.679 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.675+0000 7f920d27db80 1 -- 172.21.15.117:0/1976908151 shutdown_connections 2024-06-03T03:24:54.679 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.675+0000 7f920d27db80 1 -- 172.21.15.117:0/1976908151 wait complete. 2024-06-03T03:24:54.679 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.675+0000 7f920d27db80 1 -- start start 2024-06-03T03:24:54.679 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.675+0000 7f920d27db80 1 Processor -- start 2024-06-03T03:24:54.679 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.676+0000 7f920d27db80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e68ca21f0 0x562e68c960d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:54.679 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.676+0000 7f920d27db80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e68ca7b40 0x562e68c96610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:54.680 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.676+0000 7f920d27db80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e68ca8460 0x562e68cc52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:54.680 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.676+0000 7f920d27db80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562e68cbccc0 con 0x562e68ca21f0 2024-06-03T03:24:54.680 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.676+0000 7f920d27db80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562e68cbc5f0 con 0x562e68ca8460 2024-06-03T03:24:54.680 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.676+0000 7f920d27db80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562e68cbd390 con 0x562e68ca7b40 2024-06-03T03:24:54.680 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.676+0000 7f920d27db80 10 client.0 _ll_register_callbacks cb 0x562e68b89330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:24:54.680 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.676+0000 7f91fbe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e68ca21f0 0x562e68c960d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:54.680 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.676+0000 7f9200a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e68ca8460 0x562e68cc52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:54.680 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.676+0000 7f91fb400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e68ca7b40 0x562e68c96610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:54.680 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.676+0000 7f91fbe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e68ca21f0 0x562e68c960d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53122/0 (socket says 172.21.15.117:53122) 2024-06-03T03:24:54.680 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.676+0000 7f91fbe00700 1 -- 172.21.15.117:0/3213706747 learned_addr learned my addr 172.21.15.117:0/3213706747 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:24:54.680 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.677+0000 7f9200a00700 1 -- 172.21.15.117:0/3213706747 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e68ca7b40 msgr2=0x562e68c96610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:24:54.680 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.677+0000 7f9200a00700 1 --2- 172.21.15.117:0/3213706747 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e68ca7b40 0x562e68c96610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:54.680 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.677+0000 7f9200a00700 1 -- 172.21.15.117:0/3213706747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e68ca21f0 msgr2=0x562e68c960d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:24:54.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.677+0000 7f9200a00700 1 --2- 172.21.15.117:0/3213706747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e68ca21f0 0x562e68c960d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:54.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.677+0000 7f9200a00700 1 -- 172.21.15.117:0/3213706747 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e68cc57f0 con 0x562e68ca8460 2024-06-03T03:24:54.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.677+0000 7f91fbe00700 1 --2- 172.21.15.117:0/3213706747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e68ca21f0 0x562e68c960d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:24:54.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.677+0000 7f9200a00700 1 --2- 172.21.15.117:0/3213706747 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e68ca8460 0x562e68cc52b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f91f40cd960 tx=0x7f91f40a3040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=899cff3a9dff0f1c server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:24:54.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.677+0000 7f91e6000700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-06-03T03:24:54.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.677+0000 7f91e6000700 1 -- 172.21.15.117:0/3213706747 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f402b040 con 0x562e68ca8460 2024-06-03T03:24:54.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.677+0000 7f91e6000700 1 -- 172.21.15.117:0/3213706747 <== mon.2 v2:172.21.15.151:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f91f40074e0 con 0x562e68ca8460 2024-06-03T03:24:54.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.677+0000 7f920d27db80 1 -- 172.21.15.117:0/3213706747 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91e80121d0 con 0x562e68ca8460 2024-06-03T03:24:54.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.677+0000 7f91e6000700 1 -- 172.21.15.117:0/3213706747 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f91f409ce50 con 0x562e68ca8460 2024-06-03T03:24:54.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.677+0000 7f920d27db80 10 client.25142 fetch_fsmap learned FSMap version 30 2024-06-03T03:24:54.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.677+0000 7f920d27db80 1 -- 172.21.15.117:0/3213706747 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562e68cc6ad0 con 0x562e68ca8460 2024-06-03T03:24:54.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.678+0000 7f91e6000700 1 -- 172.21.15.117:0/3213706747 <== mon.2 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91e80121d0 con 0x562e68ca8460 2024-06-03T03:24:54.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.678+0000 7f91e6000700 1 -- 172.21.15.117:0/3213706747 <== mon.2 v2:172.21.15.151:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f91f4085ce0 con 0x562e68ca8460 2024-06-03T03:24:54.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.678+0000 7f920d27db80 10 client.25142 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T03:24:54.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.678+0000 7f920d27db80 10 client.25142 resolve_mds: resolved name 'a' to daemon mds.a 2024-06-03T03:24:54.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.678+0000 7f920d27db80 20 client.25142 populate_metadata read hostname 'smithi117' 2024-06-03T03:24:54.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.678+0000 7f920d27db80 1 --2- 172.21.15.117:0/3213706747 >> [v2:172.21.15.117:6840/1212242967,v1:172.21.15.117:6841/1212242967] conn(0x562e68d2b6c0 0x562e68d2a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:54.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.678+0000 7f920d27db80 4 client.25142 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-03T03:24:54.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.678+0000 7f920d27db80 1 -- 172.21.15.117:0/3213706747 --> [v2:172.21.15.117:6840/1212242967,v1:172.21.15.117:6841/1212242967] -- command(tid 0: {"prefix": "status"}) -- 0x562e68bfb530 con 0x562e68d2b6c0 2024-06-03T03:24:54.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.678+0000 7f91fbe00700 1 --2- 172.21.15.117:0/3213706747 >> [v2:172.21.15.117:6840/1212242967,v1:172.21.15.117:6841/1212242967] conn(0x562e68d2b6c0 0x562e68d2a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:54.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.679+0000 7f91e6000700 10 client.25142 ms_handle_connect on v2:172.21.15.117:6840/1212242967 2024-06-03T03:24:54.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.679+0000 7f91fbe00700 1 --2- 172.21.15.117:0/3213706747 >> [v2:172.21.15.117:6840/1212242967,v1:172.21.15.117:6841/1212242967] conn(0x562e68d2b6c0 0x562e68d2a3d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f4831f8ec7111948 server_cookie=e5e143c4b8d1f61 in_seq=0 out_seq=0 2024-06-03T03:24:54.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.679+0000 7f91e6000700 1 -- 172.21.15.117:0/3213706747 <== mds.2 v2:172.21.15.117:6840/1212242967 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x562e68bfb530 con 0x562e68d2b6c0 2024-06-03T03:24:54.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.679+0000 7f91e6000700 10 client.25142 handle_command_reply: tid=0 2024-06-03T03:24:54.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.679+0000 7f920d27db80 1 client.25142 shutdown 2024-06-03T03:24:54.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.679+0000 7f920d27db80 1 -- 172.21.15.117:0/3213706747 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e68ca8460 msgr2=0x562e68cc52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:54.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.679+0000 7f920d27db80 1 --2- 172.21.15.117:0/3213706747 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e68ca8460 0x562e68cc52b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f91f40cd960 tx=0x7f91f40a3040 comp rx=0 tx=0).stop 2024-06-03T03:24:54.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.679+0000 7f920d27db80 1 -- 172.21.15.117:0/3213706747 shutdown_connections 2024-06-03T03:24:54.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.679+0000 7f920d27db80 1 --2- 172.21.15.117:0/3213706747 >> [v2:172.21.15.117:6840/1212242967,v1:172.21.15.117:6841/1212242967] conn(0x562e68d2b6c0 0x562e68d2a3d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:54.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.679+0000 7f920d27db80 1 --2- 172.21.15.117:0/3213706747 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e68ca8460 0x562e68cc52b0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:54.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.679+0000 7f920d27db80 1 --2- 172.21.15.117:0/3213706747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e68ca21f0 0x562e68c960d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:54.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.679+0000 7f920d27db80 1 --2- 172.21.15.117:0/3213706747 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e68ca7b40 0x562e68c96610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:54.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.679+0000 7f91e6000700 0 client.25142 ms_handle_reset on v2:172.21.15.117:6840/1212242967 2024-06-03T03:24:54.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.679+0000 7f920d27db80 1 -- 172.21.15.117:0/3213706747 >> 172.21.15.117:0/3213706747 conn(0x562e68bce120 msgr2=0x562e68ca8840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:24:54.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.680+0000 7f920d27db80 1 -- 172.21.15.117:0/3213706747 shutdown_connections 2024-06-03T03:24:54.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.680+0000 7f920d27db80 1 -- 172.21.15.117:0/3213706747 wait complete. 2024-06-03T03:24:54.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.680+0000 7f920d27db80 20 client.25142 trim_cache size 0 max 16384 2024-06-03T03:24:54.685 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-06-03T03:24:54.685 INFO:teuthology.orchestra.run.smithi117.stdout: "cluster_fsid": "a0e8473a-2156-11ef-bc9b-c7b262605968", 2024-06-03T03:24:54.685 INFO:teuthology.orchestra.run.smithi117.stdout: "whoami": 2, 2024-06-03T03:24:54.685 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 24503, 2024-06-03T03:24:54.685 INFO:teuthology.orchestra.run.smithi117.stdout: "want_state": "up:active", 2024-06-03T03:24:54.685 INFO:teuthology.orchestra.run.smithi117.stdout: "state": "up:active", 2024-06-03T03:24:54.685 INFO:teuthology.orchestra.run.smithi117.stdout: "fs_name": "cephfs", 2024-06-03T03:24:54.685 INFO:teuthology.orchestra.run.smithi117.stdout: "rank_uptime": 230.688925407, 2024-06-03T03:24:54.685 INFO:teuthology.orchestra.run.smithi117.stdout: "mdsmap_epoch": 30, 2024-06-03T03:24:54.685 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_epoch": 99, 2024-06-03T03:24:54.686 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_epoch_barrier": 97, 2024-06-03T03:24:54.686 INFO:teuthology.orchestra.run.smithi117.stdout: "uptime": 332.65929809599999 2024-06-03T03:24:54.686 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-06-03T03:24:54.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.680+0000 7f9202a00700 1 -- 172.21.15.117:0/3557368853 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f91e00a2ca0 msgr2=0x7f91e00a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:54.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.680+0000 7f9202a00700 1 --2- 172.21.15.117:0/3557368853 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f91e00a2ca0 0x7f91e00a5160 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f91fc077aa0 tx=0x7f91e8013900 comp rx=0 tx=0).stop 2024-06-03T03:24:54.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.680+0000 7f9202a00700 1 -- 172.21.15.117:0/3557368853 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91fc10eac0 msgr2=0x7f91fc1a3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:54.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.680+0000 7f9202a00700 1 --2- 172.21.15.117:0/3557368853 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91fc10eac0 0x7f91fc1a3b00 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f91f400b070 tx=0x7f91f4004c80 comp rx=0 tx=0).stop 2024-06-03T03:24:54.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.681+0000 7f9202a00700 1 -- 172.21.15.117:0/3557368853 shutdown_connections 2024-06-03T03:24:54.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.681+0000 7f9202a00700 1 --2- 172.21.15.117:0/3557368853 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f91e00a2ca0 0x7f91e00a5160 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:54.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.681+0000 7f9202a00700 1 --2- 172.21.15.117:0/3557368853 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91fc10eac0 0x7f91fc1a3b00 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:54.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.681+0000 7f9202a00700 1 --2- 172.21.15.117:0/3557368853 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f91fc108ac0 0x7f91fc19f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:54.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.681+0000 7f9202a00700 1 --2- 172.21.15.117:0/3557368853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f91fc109480 0x7f91fc19f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:54.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.681+0000 7f9202a00700 1 -- 172.21.15.117:0/3557368853 >> 172.21.15.117:0/3557368853 conn(0x7f91fc075c60 msgr2=0x7f91fc0761e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:24:54.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.681+0000 7f9202a00700 1 -- 172.21.15.117:0/3557368853 shutdown_connections 2024-06-03T03:24:54.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:54.681+0000 7f9202a00700 1 -- 172.21.15.117:0/3557368853 wait complete. 2024-06-03T03:24:55.720 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph tell mds.b status 2024-06-03T03:24:56.166 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:24:56.311 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:24:56 smithi117 ceph-mon[31649]: pgmap v494: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:24:56.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:24:56 smithi151 ceph-mon[37043]: pgmap v494: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:24:56.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:24:56 smithi183 ceph-mon[37326]: pgmap v494: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:24:58.103 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:24:58 smithi117 ceph-mon[31649]: pgmap v495: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-03T03:24:58.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:24:58 smithi151 ceph-mon[37043]: pgmap v495: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-03T03:24:58.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:24:58 smithi183 ceph-mon[37326]: pgmap v495: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-03T03:24:58.672 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.671+0000 7f30d9600700 1 -- 172.21.15.117:0/2171755982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30d410a690 msgr2=0x7f30d410aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:58.672 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.671+0000 7f30d9600700 1 --2- 172.21.15.117:0/2171755982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30d410a690 0x7f30d410aa70 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f30c4009990 tx=0x7f30c4009ca0 comp rx=0 tx=0).stop 2024-06-03T03:24:58.672 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.671+0000 7f30d9600700 1 -- 172.21.15.117:0/2171755982 shutdown_connections 2024-06-03T03:24:58.672 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.671+0000 7f30d9600700 1 --2- 172.21.15.117:0/2171755982 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f30d4105050 0x7f30d4111560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:58.672 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.671+0000 7f30d9600700 1 --2- 172.21.15.117:0/2171755982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30d410a690 0x7f30d410aa70 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:58.672 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.671+0000 7f30d9600700 1 --2- 172.21.15.117:0/2171755982 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f30d4104690 0x7f30d4104b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:58.672 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.671+0000 7f30d9600700 1 -- 172.21.15.117:0/2171755982 >> 172.21.15.117:0/2171755982 conn(0x7f30d41001a0 msgr2=0x7f30d41025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:24:58.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.671+0000 7f30d9600700 1 -- 172.21.15.117:0/2171755982 shutdown_connections 2024-06-03T03:24:58.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.671+0000 7f30d9600700 1 -- 172.21.15.117:0/2171755982 wait complete. 2024-06-03T03:24:58.674 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.672+0000 7f30d9600700 1 Processor -- start 2024-06-03T03:24:58.674 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.672+0000 7f30d9600700 1 -- start start 2024-06-03T03:24:58.674 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.673+0000 7f30d9600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30d4104690 0x7f30d419aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:58.674 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.673+0000 7f30d9600700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f30d4105050 0x7f30d419b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:58.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.673+0000 7f30d9600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f30d410a690 0x7f30d419f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:58.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.673+0000 7f30d9600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f30d4114a30 con 0x7f30d4104690 2024-06-03T03:24:58.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.673+0000 7f30d9600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f30d41148b0 con 0x7f30d410a690 2024-06-03T03:24:58.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.673+0000 7f30d9600700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f30d4114bb0 con 0x7f30d4105050 2024-06-03T03:24:58.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.673+0000 7f30d2a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30d4104690 0x7f30d419aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:58.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.673+0000 7f30d2a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30d4104690 0x7f30d419aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:33386/0 (socket says 172.21.15.117:33386) 2024-06-03T03:24:58.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.673+0000 7f30d2a00700 1 -- 172.21.15.117:0/421856506 learned_addr learned my addr 172.21.15.117:0/421856506 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:24:58.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.673+0000 7f30d3400700 1 --2- 172.21.15.117:0/421856506 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f30d410a690 0x7f30d419f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:58.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.673+0000 7f30cbe00700 1 --2- 172.21.15.117:0/421856506 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f30d4105050 0x7f30d419b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:58.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.674+0000 7f30d2a00700 1 -- 172.21.15.117:0/421856506 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f30d4105050 msgr2=0x7f30d419b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:58.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.674+0000 7f30d2a00700 1 --2- 172.21.15.117:0/421856506 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f30d4105050 0x7f30d419b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:58.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.674+0000 7f30d2a00700 1 -- 172.21.15.117:0/421856506 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f30d410a690 msgr2=0x7f30d419f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:58.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.674+0000 7f30d2a00700 1 --2- 172.21.15.117:0/421856506 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f30d410a690 0x7f30d419f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:58.676 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.674+0000 7f30d2a00700 1 -- 172.21.15.117:0/421856506 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30d419fef0 con 0x7f30d4104690 2024-06-03T03:24:58.676 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.674+0000 7f30d3400700 1 --2- 172.21.15.117:0/421856506 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f30d410a690 0x7f30d419f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:24:58.676 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.674+0000 7f30d2a00700 1 --2- 172.21.15.117:0/421856506 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30d4104690 0x7f30d419aee0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f30c4000c00 tx=0x7f30c400bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:24:58.676 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.674+0000 7f30cb400700 1 -- 172.21.15.117:0/421856506 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c4027070 con 0x7f30d4104690 2024-06-03T03:24:58.676 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.674+0000 7f30cb400700 1 -- 172.21.15.117:0/421856506 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30c40056d0 con 0x7f30d4104690 2024-06-03T03:24:58.677 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.674+0000 7f30d9600700 1 -- 172.21.15.117:0/421856506 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30d41a0180 con 0x7f30d4104690 2024-06-03T03:24:58.677 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.674+0000 7f30cb400700 1 -- 172.21.15.117:0/421856506 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c4022e10 con 0x7f30d4104690 2024-06-03T03:24:58.677 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.674+0000 7f30d9600700 1 -- 172.21.15.117:0/421856506 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30d41a7a60 con 0x7f30d4104690 2024-06-03T03:24:58.678 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.677+0000 7f30cb400700 1 -- 172.21.15.117:0/421856506 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f30c402a070 con 0x7f30d4104690 2024-06-03T03:24:58.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.680+0000 7f30e4048b80 1 Processor -- start 2024-06-03T03:24:58.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.680+0000 7f30cb400700 1 --2- 172.21.15.117:0/421856506 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f30b80a2c20 0x7f30b80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:58.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.680+0000 7f30e4048b80 1 -- start start 2024-06-03T03:24:58.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.680+0000 7f30cb400700 1 -- 172.21.15.117:0/421856506 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f30c40d0090 con 0x7f30d4104690 2024-06-03T03:24:58.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.680+0000 7f30d3400700 1 --2- 172.21.15.117:0/421856506 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f30b80a2c20 0x7f30b80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:58.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.680+0000 7f30e4048b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55913625ab40 0x55913625af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:58.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.680+0000 7f30e4048b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55913625b460 0x559136254cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:58.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.680+0000 7f30e4048b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5591362551f0 0x559136255630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:58.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.680+0000 7f30e4048b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5591361639c0 con 0x5591362551f0 2024-06-03T03:24:58.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.680+0000 7f30e4048b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5591361a6ab0 con 0x55913625ab40 2024-06-03T03:24:58.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.680+0000 7f30e4048b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5591361618a0 con 0x55913625b460 2024-06-03T03:24:58.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.681+0000 7f30d2a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55913625ab40 0x55913625af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:58.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.681+0000 7f30cbe00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55913625b460 0x559136254cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:58.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.681+0000 7f30d2a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55913625ab40 0x55913625af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:56396/0 (socket says 172.21.15.117:56396) 2024-06-03T03:24:58.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.681+0000 7f30d2a00700 1 -- 172.21.15.117:0/2707942561 learned_addr learned my addr 172.21.15.117:0/2707942561 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:24:58.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.681+0000 7f30d3400700 1 --2- 172.21.15.117:0/2707942561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5591362551f0 0x559136255630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:58.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.681+0000 7f30d3400700 1 --2- 172.21.15.117:0/421856506 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f30b80a2c20 0x7f30b80a50e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f30cc003e80 tx=0x7f30cc009920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:24:58.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.681+0000 7f30d2a00700 1 -- 172.21.15.117:0/2707942561 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55913625b460 msgr2=0x559136254cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:24:58.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.681+0000 7f30d2a00700 1 --2- 172.21.15.117:0/2707942561 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55913625b460 0x559136254cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:58.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.681+0000 7f30d2a00700 1 -- 172.21.15.117:0/2707942561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5591362551f0 msgr2=0x559136255630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:24:58.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.681+0000 7f30d2a00700 1 --2- 172.21.15.117:0/2707942561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5591362551f0 0x559136255630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:58.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.681+0000 7f30d2a00700 1 -- 172.21.15.117:0/2707942561 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55913614ef40 con 0x55913625ab40 2024-06-03T03:24:58.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.682+0000 7f30d2a00700 1 --2- 172.21.15.117:0/2707942561 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55913625ab40 0x55913625af20 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f30c4098f60 tx=0x7f30c4009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f789e0d11bf5ccf server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:24:58.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.682+0000 7f30c3e00700 1 -- 172.21.15.117:0/2707942561 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c409cc90 con 0x55913625ab40 2024-06-03T03:24:58.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.682+0000 7f30c3e00700 1 -- 172.21.15.117:0/2707942561 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30c409c590 con 0x55913625ab40 2024-06-03T03:24:58.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.682+0000 7f30c3e00700 1 -- 172.21.15.117:0/2707942561 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c409ca50 con 0x55913625ab40 2024-06-03T03:24:58.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.683+0000 7f30e4048b80 1 -- 172.21.15.117:0/2707942561 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55913625ab40 msgr2=0x55913625af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:58.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.683+0000 7f30e4048b80 1 --2- 172.21.15.117:0/2707942561 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55913625ab40 0x55913625af20 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f30c4098f60 tx=0x7f30c4009ca0 comp rx=0 tx=0).stop 2024-06-03T03:24:58.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.683+0000 7f30e4048b80 1 -- 172.21.15.117:0/2707942561 shutdown_connections 2024-06-03T03:24:58.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.683+0000 7f30e4048b80 1 --2- 172.21.15.117:0/2707942561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5591362551f0 0x559136255630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:58.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.683+0000 7f30e4048b80 1 --2- 172.21.15.117:0/2707942561 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55913625ab40 0x55913625af20 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:58.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.683+0000 7f30e4048b80 1 --2- 172.21.15.117:0/2707942561 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55913625b460 0x559136254cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:58.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.683+0000 7f30e4048b80 1 -- 172.21.15.117:0/2707942561 >> 172.21.15.117:0/2707942561 conn(0x559136181120 msgr2=0x55913624ffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:24:58.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.683+0000 7f30e4048b80 1 -- 172.21.15.117:0/2707942561 shutdown_connections 2024-06-03T03:24:58.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.683+0000 7f30e4048b80 1 -- 172.21.15.117:0/2707942561 wait complete. 2024-06-03T03:24:58.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.684+0000 7f30e4048b80 1 -- start start 2024-06-03T03:24:58.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.684+0000 7f30e4048b80 1 Processor -- start 2024-06-03T03:24:58.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.684+0000 7f30e4048b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5591362551f0 0x559136261230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:58.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.684+0000 7f30e4048b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55913625ab40 0x55913625e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:58.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.684+0000 7f30e4048b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55913625b460 0x55913625ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:58.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.684+0000 7f30e4048b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559136142f10 con 0x5591362551f0 2024-06-03T03:24:58.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.684+0000 7f30e4048b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5591361a2a70 con 0x55913625ab40 2024-06-03T03:24:58.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.684+0000 7f30e4048b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559136261bf0 con 0x55913625b460 2024-06-03T03:24:58.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.684+0000 7f30e4048b80 10 client.0 _ll_register_callbacks cb 0x55913613c330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:24:58.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.685+0000 7f30d2a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5591362551f0 0x559136261230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:58.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.685+0000 7f30d2a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5591362551f0 0x559136261230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:33410/0 (socket says 172.21.15.117:33410) 2024-06-03T03:24:58.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.685+0000 7f30d2a00700 1 -- 172.21.15.117:0/1624392047 learned_addr learned my addr 172.21.15.117:0/1624392047 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:24:58.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.685+0000 7f30cbe00700 1 --2- 172.21.15.117:0/1624392047 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55913625ab40 0x55913625e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:58.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.685+0000 7f30d3400700 1 --2- 172.21.15.117:0/1624392047 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55913625b460 0x55913625ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:58.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.685+0000 7f30d2a00700 1 -- 172.21.15.117:0/1624392047 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55913625b460 msgr2=0x55913625ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:24:58.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.685+0000 7f30d2a00700 1 --2- 172.21.15.117:0/1624392047 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55913625b460 0x55913625ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:58.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.685+0000 7f30d2a00700 1 -- 172.21.15.117:0/1624392047 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55913625ab40 msgr2=0x55913625e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:24:58.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.685+0000 7f30d2a00700 1 --2- 172.21.15.117:0/1624392047 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55913625ab40 0x55913625e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:58.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.685+0000 7f30d2a00700 1 -- 172.21.15.117:0/1624392047 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591360f5880 con 0x5591362551f0 2024-06-03T03:24:58.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.685+0000 7f30d2a00700 1 --2- 172.21.15.117:0/1624392047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5591362551f0 0x559136261230 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f30c409c520 tx=0x7f30c40095f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5109d04372a2c41 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:24:58.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.685+0000 7f30c0c00700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-06-03T03:24:58.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.686+0000 7f30c0c00700 1 -- 172.21.15.117:0/1624392047 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c40b4030 con 0x5591362551f0 2024-06-03T03:24:58.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.686+0000 7f30c0c00700 1 -- 172.21.15.117:0/1624392047 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f30c40b5270 con 0x5591362551f0 2024-06-03T03:24:58.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.686+0000 7f30e4048b80 1 -- 172.21.15.117:0/1624392047 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30c409c590 con 0x5591362551f0 2024-06-03T03:24:58.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.686+0000 7f30c0c00700 1 -- 172.21.15.117:0/1624392047 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f30c40a5170 con 0x5591362551f0 2024-06-03T03:24:58.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.686+0000 7f30e4048b80 10 client.15039 fetch_fsmap learned FSMap version 30 2024-06-03T03:24:58.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.686+0000 7f30e4048b80 1 -- 172.21.15.117:0/1624392047 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55913626c320 con 0x5591362551f0 2024-06-03T03:24:58.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.686+0000 7f30c0c00700 1 -- 172.21.15.117:0/1624392047 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30c409c590 con 0x5591362551f0 2024-06-03T03:24:58.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.687+0000 7f30c0c00700 1 -- 172.21.15.117:0/1624392047 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f30c409b9e0 con 0x5591362551f0 2024-06-03T03:24:58.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.687+0000 7f30e4048b80 10 client.15039 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T03:24:58.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.687+0000 7f30e4048b80 10 client.15039 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-03T03:24:58.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.687+0000 7f30e4048b80 20 client.15039 populate_metadata read hostname 'smithi117' 2024-06-03T03:24:58.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.687+0000 7f30e4048b80 1 --2- 172.21.15.117:0/1624392047 >> [v2:172.21.15.151:6838/3623444616,v1:172.21.15.151:6839/3623444616] conn(0x55913626e770 0x55913626fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:58.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.687+0000 7f30e4048b80 4 client.15039 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T03:24:58.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.687+0000 7f30e4048b80 1 -- 172.21.15.117:0/1624392047 --> [v2:172.21.15.151:6838/3623444616,v1:172.21.15.151:6839/3623444616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5591361b3fb0 con 0x55913626e770 2024-06-03T03:24:58.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.687+0000 7f30cbe00700 1 --2- 172.21.15.117:0/1624392047 >> [v2:172.21.15.151:6838/3623444616,v1:172.21.15.151:6839/3623444616] conn(0x55913626e770 0x55913626fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:58.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.688+0000 7f30cbe00700 1 --2- 172.21.15.117:0/1624392047 >> [v2:172.21.15.151:6838/3623444616,v1:172.21.15.151:6839/3623444616] conn(0x55913626e770 0x55913626fbe0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4167ffce9ff7e6e1 server_cookie=ac04b37c2137137b in_seq=0 out_seq=0 2024-06-03T03:24:58.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.688+0000 7f30c0c00700 10 client.15039 ms_handle_connect on v2:172.21.15.151:6838/3623444616 2024-06-03T03:24:58.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.688+0000 7f30c0c00700 1 -- 172.21.15.117:0/1624392047 <== mds.1 v2:172.21.15.151:6838/3623444616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5591361b3fb0 con 0x55913626e770 2024-06-03T03:24:58.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.688+0000 7f30c0c00700 10 client.15039 handle_command_reply: tid=0 2024-06-03T03:24:58.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.689+0000 7f30e4048b80 1 client.15039 shutdown 2024-06-03T03:24:58.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.689+0000 7f30e4048b80 1 -- 172.21.15.117:0/1624392047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5591362551f0 msgr2=0x559136261230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:58.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.689+0000 7f30e4048b80 1 --2- 172.21.15.117:0/1624392047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5591362551f0 0x559136261230 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f30c409c520 tx=0x7f30c40095f0 comp rx=0 tx=0).stop 2024-06-03T03:24:58.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.689+0000 7f30e4048b80 1 -- 172.21.15.117:0/1624392047 shutdown_connections 2024-06-03T03:24:58.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.689+0000 7f30e4048b80 1 --2- 172.21.15.117:0/1624392047 >> [v2:172.21.15.151:6838/3623444616,v1:172.21.15.151:6839/3623444616] conn(0x55913626e770 0x55913626fbe0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:58.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.689+0000 7f30e4048b80 1 --2- 172.21.15.117:0/1624392047 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55913625b460 0x55913625ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:58.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.689+0000 7f30e4048b80 1 --2- 172.21.15.117:0/1624392047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5591362551f0 0x559136261230 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:58.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.689+0000 7f30e4048b80 1 --2- 172.21.15.117:0/1624392047 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55913625ab40 0x55913625e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:58.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.689+0000 7f30e4048b80 1 -- 172.21.15.117:0/1624392047 >> 172.21.15.117:0/1624392047 conn(0x559136181120 msgr2=0x559136249740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:24:58.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.689+0000 7f30c0c00700 0 client.15039 ms_handle_reset on v2:172.21.15.151:6838/3623444616 2024-06-03T03:24:58.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.689+0000 7f30e4048b80 1 -- 172.21.15.117:0/1624392047 shutdown_connections 2024-06-03T03:24:58.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.689+0000 7f30e4048b80 1 -- 172.21.15.117:0/1624392047 wait complete. 2024-06-03T03:24:58.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.689+0000 7f30e4048b80 20 client.15039 trim_cache size 0 max 16384 2024-06-03T03:24:58.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.712+0000 7f30e4048b80 1 Processor -- start 2024-06-03T03:24:58.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.712+0000 7f30e4048b80 1 -- start start 2024-06-03T03:24:58.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.712+0000 7f30e4048b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5591362551f0 0x559136254cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:58.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.712+0000 7f30e4048b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55913625ab40 0x55913625ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:58.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.712+0000 7f30e4048b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55913625b460 0x55913625d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:58.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.712+0000 7f30e4048b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55913626df20 con 0x5591362551f0 2024-06-03T03:24:58.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.712+0000 7f30e4048b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55913626e5f0 con 0x55913625b460 2024-06-03T03:24:58.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.712+0000 7f30e4048b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55913626ecc0 con 0x55913625ab40 2024-06-03T03:24:58.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.713+0000 7f30d2a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5591362551f0 0x559136254cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:58.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.713+0000 7f30cbe00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55913625ab40 0x55913625ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:58.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.713+0000 7f30d2a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5591362551f0 0x559136254cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:33424/0 (socket says 172.21.15.117:33424) 2024-06-03T03:24:58.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.713+0000 7f30d2a00700 1 -- 172.21.15.117:0/1000748963 learned_addr learned my addr 172.21.15.117:0/1000748963 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:24:58.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.713+0000 7f30d3400700 1 --2- 172.21.15.117:0/1000748963 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55913625b460 0x55913625d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:58.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.713+0000 7f30cbe00700 1 -- 172.21.15.117:0/1000748963 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55913625b460 msgr2=0x55913625d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:24:58.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.713+0000 7f30cbe00700 1 --2- 172.21.15.117:0/1000748963 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55913625b460 0x55913625d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:58.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.713+0000 7f30cbe00700 1 -- 172.21.15.117:0/1000748963 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5591362551f0 msgr2=0x559136254cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:24:58.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.713+0000 7f30cbe00700 1 --2- 172.21.15.117:0/1000748963 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5591362551f0 0x559136254cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:58.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.713+0000 7f30cbe00700 1 -- 172.21.15.117:0/1000748963 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55913625b170 con 0x55913625ab40 2024-06-03T03:24:58.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.713+0000 7f30d2a00700 1 --2- 172.21.15.117:0/1000748963 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5591362551f0 0x559136254cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:24:58.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.714+0000 7f30cbe00700 1 --2- 172.21.15.117:0/1000748963 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55913625ab40 0x55913625ec00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x55913625fb00 tx=0x7f30bc009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44ca8847864c9dd server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:24:58.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.714+0000 7f30c3400700 1 -- 172.21.15.117:0/1000748963 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30bc022070 con 0x55913625ab40 2024-06-03T03:24:58.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.714+0000 7f30c3400700 1 -- 172.21.15.117:0/1000748963 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30bc010d10 con 0x55913625ab40 2024-06-03T03:24:58.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.714+0000 7f30c3400700 1 -- 172.21.15.117:0/1000748963 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30bc005660 con 0x55913625ab40 2024-06-03T03:24:58.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.715+0000 7f30e4048b80 1 -- 172.21.15.117:0/1000748963 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55913625ab40 msgr2=0x55913625ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:58.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.715+0000 7f30e4048b80 1 --2- 172.21.15.117:0/1000748963 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55913625ab40 0x55913625ec00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x55913625fb00 tx=0x7f30bc009260 comp rx=0 tx=0).stop 2024-06-03T03:24:58.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.715+0000 7f30e4048b80 1 -- 172.21.15.117:0/1000748963 shutdown_connections 2024-06-03T03:24:58.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.715+0000 7f30e4048b80 1 --2- 172.21.15.117:0/1000748963 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55913625b460 0x55913625d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:58.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.715+0000 7f30e4048b80 1 --2- 172.21.15.117:0/1000748963 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5591362551f0 0x559136254cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:58.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.715+0000 7f30e4048b80 1 --2- 172.21.15.117:0/1000748963 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55913625ab40 0x55913625ec00 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:58.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.715+0000 7f30e4048b80 1 -- 172.21.15.117:0/1000748963 >> 172.21.15.117:0/1000748963 conn(0x559136181120 msgr2=0x55913625e4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:24:58.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.715+0000 7f30e4048b80 1 -- 172.21.15.117:0/1000748963 shutdown_connections 2024-06-03T03:24:58.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.715+0000 7f30e4048b80 1 -- 172.21.15.117:0/1000748963 wait complete. 2024-06-03T03:24:58.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.715+0000 7f30e4048b80 1 -- start start 2024-06-03T03:24:58.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.716+0000 7f30e4048b80 1 Processor -- start 2024-06-03T03:24:58.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.716+0000 7f30e4048b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5591362551f0 0x5591362606a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:58.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.716+0000 7f30e4048b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55913625ab40 0x559136260be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:58.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.716+0000 7f30e4048b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55913625b460 0x559136261120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:58.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.716+0000 7f30e4048b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55913626fa60 con 0x5591362551f0 2024-06-03T03:24:58.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.716+0000 7f30e4048b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55913626f390 con 0x55913625ab40 2024-06-03T03:24:58.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.716+0000 7f30e4048b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559136270130 con 0x55913625b460 2024-06-03T03:24:58.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.716+0000 7f30e4048b80 10 client.0 _ll_register_callbacks cb 0x55913613c330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:24:58.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.717+0000 7f30d2a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5591362551f0 0x5591362606a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:58.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.717+0000 7f30d2a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5591362551f0 0x5591362606a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:33428/0 (socket says 172.21.15.117:33428) 2024-06-03T03:24:58.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.717+0000 7f30d2a00700 1 -- 172.21.15.117:0/1768479603 learned_addr learned my addr 172.21.15.117:0/1768479603 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:24:58.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.717+0000 7f30cbe00700 1 --2- 172.21.15.117:0/1768479603 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55913625ab40 0x559136260be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:58.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.717+0000 7f30d3400700 1 --2- 172.21.15.117:0/1768479603 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55913625b460 0x559136261120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:58.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.717+0000 7f30d2a00700 1 -- 172.21.15.117:0/1768479603 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55913625b460 msgr2=0x559136261120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:24:58.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.717+0000 7f30d2a00700 1 --2- 172.21.15.117:0/1768479603 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55913625b460 0x559136261120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:58.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.717+0000 7f30d2a00700 1 -- 172.21.15.117:0/1768479603 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55913625ab40 msgr2=0x559136260be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:24:58.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.717+0000 7f30d2a00700 1 --2- 172.21.15.117:0/1768479603 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55913625ab40 0x559136260be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:58.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.717+0000 7f30d2a00700 1 -- 172.21.15.117:0/1768479603 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591362778b0 con 0x5591362551f0 2024-06-03T03:24:58.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.717+0000 7f30cbe00700 1 --2- 172.21.15.117:0/1768479603 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55913625ab40 0x559136260be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T03:24:58.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.717+0000 7f30d2a00700 1 --2- 172.21.15.117:0/1768479603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5591362551f0 0x5591362606a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f30c40989f0 tx=0x7f30c40a4ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cccb3991315e341 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:24:58.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.717+0000 7f30c0c00700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-06-03T03:24:58.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.718+0000 7f30c0c00700 1 -- 172.21.15.117:0/1768479603 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c40b4070 con 0x5591362551f0 2024-06-03T03:24:58.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.718+0000 7f30c0c00700 1 -- 172.21.15.117:0/1768479603 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f30c40b5270 con 0x5591362551f0 2024-06-03T03:24:58.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.718+0000 7f30e4048b80 1 -- 172.21.15.117:0/1768479603 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30bc010d10 con 0x5591362551f0 2024-06-03T03:24:58.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.718+0000 7f30c0c00700 1 -- 172.21.15.117:0/1768479603 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f30c4098630 con 0x5591362551f0 2024-06-03T03:24:58.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.718+0000 7f30e4048b80 10 client.15045 fetch_fsmap learned FSMap version 30 2024-06-03T03:24:58.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.718+0000 7f30e4048b80 1 -- 172.21.15.117:0/1768479603 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5591362778b0 con 0x5591362551f0 2024-06-03T03:24:58.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.718+0000 7f30c0c00700 1 -- 172.21.15.117:0/1768479603 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30bc010d10 con 0x5591362551f0 2024-06-03T03:24:58.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.718+0000 7f30c0c00700 1 -- 172.21.15.117:0/1768479603 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f30c409b450 con 0x5591362551f0 2024-06-03T03:24:58.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.718+0000 7f30e4048b80 10 client.15045 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T03:24:58.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.718+0000 7f30e4048b80 10 client.15045 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-03T03:24:58.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.718+0000 7f30e4048b80 20 client.15045 populate_metadata read hostname 'smithi117' 2024-06-03T03:24:58.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.718+0000 7f30e4048b80 1 --2- 172.21.15.117:0/1768479603 >> [v2:172.21.15.151:6838/3623444616,v1:172.21.15.151:6839/3623444616] conn(0x5591362dec20 0x5591362dd930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:24:58.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.718+0000 7f30e4048b80 4 client.15045 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-03T03:24:58.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.718+0000 7f30e4048b80 1 -- 172.21.15.117:0/1768479603 --> [v2:172.21.15.151:6838/3623444616,v1:172.21.15.151:6839/3623444616] -- command(tid 0: {"prefix": "status"}) -- 0x5591361ae530 con 0x5591362dec20 2024-06-03T03:24:58.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.719+0000 7f30cbe00700 1 --2- 172.21.15.117:0/1768479603 >> [v2:172.21.15.151:6838/3623444616,v1:172.21.15.151:6839/3623444616] conn(0x5591362dec20 0x5591362dd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:24:58.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.719+0000 7f30cbe00700 1 --2- 172.21.15.117:0/1768479603 >> [v2:172.21.15.151:6838/3623444616,v1:172.21.15.151:6839/3623444616] conn(0x5591362dec20 0x5591362dd930 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8086de83aadbebe4 server_cookie=8b10851f26e7b226 in_seq=0 out_seq=0 2024-06-03T03:24:58.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.719+0000 7f30c0c00700 10 client.15045 ms_handle_connect on v2:172.21.15.151:6838/3623444616 2024-06-03T03:24:58.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.720+0000 7f30c0c00700 1 -- 172.21.15.117:0/1768479603 <== mds.1 v2:172.21.15.151:6838/3623444616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5591361ae530 con 0x5591362dec20 2024-06-03T03:24:58.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.720+0000 7f30c0c00700 10 client.15045 handle_command_reply: tid=0 2024-06-03T03:24:58.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.720+0000 7f30e4048b80 1 client.15045 shutdown 2024-06-03T03:24:58.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.720+0000 7f30e4048b80 1 -- 172.21.15.117:0/1768479603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5591362551f0 msgr2=0x5591362606a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:58.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.720+0000 7f30e4048b80 1 --2- 172.21.15.117:0/1768479603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5591362551f0 0x5591362606a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f30c40989f0 tx=0x7f30c40a4ef0 comp rx=0 tx=0).stop 2024-06-03T03:24:58.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.720+0000 7f30e4048b80 1 -- 172.21.15.117:0/1768479603 shutdown_connections 2024-06-03T03:24:58.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.720+0000 7f30e4048b80 1 --2- 172.21.15.117:0/1768479603 >> [v2:172.21.15.151:6838/3623444616,v1:172.21.15.151:6839/3623444616] conn(0x5591362dec20 0x5591362dd930 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:58.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.720+0000 7f30e4048b80 1 --2- 172.21.15.117:0/1768479603 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55913625b460 0x559136261120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:58.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.720+0000 7f30e4048b80 1 --2- 172.21.15.117:0/1768479603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5591362551f0 0x5591362606a0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:58.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.720+0000 7f30e4048b80 1 --2- 172.21.15.117:0/1768479603 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55913625ab40 0x559136260be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:58.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.720+0000 7f30e4048b80 1 -- 172.21.15.117:0/1768479603 >> 172.21.15.117:0/1768479603 conn(0x559136181120 msgr2=0x55913625b840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:24:58.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.720+0000 7f30c0c00700 0 client.15045 ms_handle_reset on v2:172.21.15.151:6838/3623444616 2024-06-03T03:24:58.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.720+0000 7f30e4048b80 1 -- 172.21.15.117:0/1768479603 shutdown_connections 2024-06-03T03:24:58.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.720+0000 7f30e4048b80 1 -- 172.21.15.117:0/1768479603 wait complete. 2024-06-03T03:24:58.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.720+0000 7f30e4048b80 20 client.15045 trim_cache size 0 max 16384 2024-06-03T03:24:58.727 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-06-03T03:24:58.727 INFO:teuthology.orchestra.run.smithi117.stdout: "cluster_fsid": "a0e8473a-2156-11ef-bc9b-c7b262605968", 2024-06-03T03:24:58.727 INFO:teuthology.orchestra.run.smithi117.stdout: "whoami": 1, 2024-06-03T03:24:58.727 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 24512, 2024-06-03T03:24:58.727 INFO:teuthology.orchestra.run.smithi117.stdout: "want_state": "up:active", 2024-06-03T03:24:58.727 INFO:teuthology.orchestra.run.smithi117.stdout: "state": "up:active", 2024-06-03T03:24:58.728 INFO:teuthology.orchestra.run.smithi117.stdout: "fs_name": "cephfs", 2024-06-03T03:24:58.728 INFO:teuthology.orchestra.run.smithi117.stdout: "rank_uptime": 235.74278634699999, 2024-06-03T03:24:58.728 INFO:teuthology.orchestra.run.smithi117.stdout: "mdsmap_epoch": 30, 2024-06-03T03:24:58.728 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_epoch": 99, 2024-06-03T03:24:58.728 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_epoch_barrier": 97, 2024-06-03T03:24:58.728 INFO:teuthology.orchestra.run.smithi117.stdout: "uptime": 333.89186513099997 2024-06-03T03:24:58.728 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-06-03T03:24:58.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.721+0000 7f30d9600700 1 -- 172.21.15.117:0/421856506 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f30b80a2c20 msgr2=0x7f30b80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:58.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.721+0000 7f30d9600700 1 --2- 172.21.15.117:0/421856506 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f30b80a2c20 0x7f30b80a50e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f30cc003e80 tx=0x7f30cc009920 comp rx=0 tx=0).stop 2024-06-03T03:24:58.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.721+0000 7f30d9600700 1 -- 172.21.15.117:0/421856506 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30d4104690 msgr2=0x7f30d419aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:24:58.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.721+0000 7f30d9600700 1 --2- 172.21.15.117:0/421856506 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30d4104690 0x7f30d419aee0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f30c4000c00 tx=0x7f30c400bab0 comp rx=0 tx=0).stop 2024-06-03T03:24:58.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.721+0000 7f30d9600700 1 -- 172.21.15.117:0/421856506 shutdown_connections 2024-06-03T03:24:58.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.721+0000 7f30d9600700 1 --2- 172.21.15.117:0/421856506 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f30b80a2c20 0x7f30b80a50e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:58.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.721+0000 7f30d9600700 1 --2- 172.21.15.117:0/421856506 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f30d410a690 0x7f30d419f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:58.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.721+0000 7f30d9600700 1 --2- 172.21.15.117:0/421856506 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30d4104690 0x7f30d419aee0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:58.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.721+0000 7f30d9600700 1 --2- 172.21.15.117:0/421856506 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f30d4105050 0x7f30d419b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:24:58.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.721+0000 7f30d9600700 1 -- 172.21.15.117:0/421856506 >> 172.21.15.117:0/421856506 conn(0x7f30d41001a0 msgr2=0x7f30d4100740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:24:58.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.721+0000 7f30d9600700 1 -- 172.21.15.117:0/421856506 shutdown_connections 2024-06-03T03:24:58.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:24:58.721+0000 7f30d9600700 1 -- 172.21.15.117:0/421856506 wait complete. 2024-06-03T03:24:59.673 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph tell mds.i status 2024-06-03T03:25:00.119 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:25:00.301 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:25:00 smithi117 ceph-mon[31649]: pgmap v496: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-03T03:25:00.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:25:00 smithi151 ceph-mon[37043]: pgmap v496: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-03T03:25:00.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:25:00 smithi183 ceph-mon[37326]: pgmap v496: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-03T03:25:01.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:25:01 smithi151 ceph-mon[37043]: pgmap v497: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:25:01.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:25:01 smithi117 ceph-mon[31649]: pgmap v497: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:25:01.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:25:01 smithi183 ceph-mon[37326]: pgmap v497: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:25:02.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.580+0000 7fcd66e00700 1 -- 172.21.15.117:0/1762799969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd600746c0 msgr2=0x7fcd60074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:02.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.580+0000 7fcd66e00700 1 --2- 172.21.15.117:0/1762799969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd600746c0 0x7fcd60074aa0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fcd50009990 tx=0x7fcd50009ca0 comp rx=0 tx=0).stop 2024-06-03T03:25:02.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.581+0000 7fcd66e00700 1 -- 172.21.15.117:0/1762799969 shutdown_connections 2024-06-03T03:25:02.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.581+0000 7fcd66e00700 1 --2- 172.21.15.117:0/1762799969 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd6010ee90 0x7fcd60111280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:02.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.581+0000 7fcd66e00700 1 --2- 172.21.15.117:0/1762799969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd600746c0 0x7fcd60074aa0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:02.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.581+0000 7fcd66e00700 1 --2- 172.21.15.117:0/1762799969 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd60074fe0 0x7fcd6010e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:02.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.581+0000 7fcd66e00700 1 -- 172.21.15.117:0/1762799969 >> 172.21.15.117:0/1762799969 conn(0x7fcd600795f0 msgr2=0x7fcd60079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:25:02.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.581+0000 7fcd66e00700 1 -- 172.21.15.117:0/1762799969 shutdown_connections 2024-06-03T03:25:02.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.581+0000 7fcd66e00700 1 -- 172.21.15.117:0/1762799969 wait complete. 2024-06-03T03:25:02.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.582+0000 7fcd66e00700 1 Processor -- start 2024-06-03T03:25:02.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.582+0000 7fcd66e00700 1 -- start start 2024-06-03T03:25:02.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.582+0000 7fcd66e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd600746c0 0x7fcd6019ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:02.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.582+0000 7fcd66e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd60074fe0 0x7fcd6019b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:02.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.582+0000 7fcd66e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd6010ee90 0x7fcd6019f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:02.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.582+0000 7fcd66e00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcd60114990 con 0x7fcd600746c0 2024-06-03T03:25:02.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.582+0000 7fcd66e00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcd60114810 con 0x7fcd60074fe0 2024-06-03T03:25:02.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.583+0000 7fcd66e00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcd60114b10 con 0x7fcd6010ee90 2024-06-03T03:25:02.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.583+0000 7fcd5fe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd600746c0 0x7fcd6019ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:02.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.583+0000 7fcd5fe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd600746c0 0x7fcd6019ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:33446/0 (socket says 172.21.15.117:33446) 2024-06-03T03:25:02.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.583+0000 7fcd5fe00700 1 -- 172.21.15.117:0/2689513524 learned_addr learned my addr 172.21.15.117:0/2689513524 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:25:02.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.583+0000 7fcd5f400700 1 --2- 172.21.15.117:0/2689513524 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd60074fe0 0x7fcd6019b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:02.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.583+0000 7fcd64e00700 1 --2- 172.21.15.117:0/2689513524 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd6010ee90 0x7fcd6019f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:02.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.583+0000 7fcd5f400700 1 -- 172.21.15.117:0/2689513524 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd6010ee90 msgr2=0x7fcd6019f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:02.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.583+0000 7fcd5f400700 1 --2- 172.21.15.117:0/2689513524 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd6010ee90 0x7fcd6019f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:02.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.583+0000 7fcd5f400700 1 -- 172.21.15.117:0/2689513524 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd600746c0 msgr2=0x7fcd6019ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:02.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.583+0000 7fcd5f400700 1 --2- 172.21.15.117:0/2689513524 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd600746c0 0x7fcd6019ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:02.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.583+0000 7fcd5f400700 1 -- 172.21.15.117:0/2689513524 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd6019fe10 con 0x7fcd60074fe0 2024-06-03T03:25:02.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.583+0000 7fcd5fe00700 1 --2- 172.21.15.117:0/2689513524 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd600746c0 0x7fcd6019ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:25:02.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.584+0000 7fcd5f400700 1 --2- 172.21.15.117:0/2689513524 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd60074fe0 0x7fcd6019b340 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c00e9c0 tx=0x7fcd4c00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:25:02.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.584+0000 7fcd5cc00700 1 -- 172.21.15.117:0/2689513524 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c00cb70 con 0x7fcd60074fe0 2024-06-03T03:25:02.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.584+0000 7fcd5cc00700 1 -- 172.21.15.117:0/2689513524 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd4c00ee50 con 0x7fcd60074fe0 2024-06-03T03:25:02.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.584+0000 7fcd66e00700 1 -- 172.21.15.117:0/2689513524 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd601a0100 con 0x7fcd60074fe0 2024-06-03T03:25:02.587 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.584+0000 7fcd5cc00700 1 -- 172.21.15.117:0/2689513524 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c018680 con 0x7fcd60074fe0 2024-06-03T03:25:02.587 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.584+0000 7fcd66e00700 1 -- 172.21.15.117:0/2689513524 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd601a7980 con 0x7fcd60074fe0 2024-06-03T03:25:02.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.588+0000 7fcd5cc00700 1 -- 172.21.15.117:0/2689513524 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fcd4c01f020 con 0x7fcd60074fe0 2024-06-03T03:25:02.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.588+0000 7fcd717b8b80 1 Processor -- start 2024-06-03T03:25:02.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.589+0000 7fcd717b8b80 1 -- start start 2024-06-03T03:25:02.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.589+0000 7fcd717b8b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569f5500b40 0x5569f5500f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:02.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.589+0000 7fcd717b8b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569f5501460 0x5569f54facb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:02.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.589+0000 7fcd717b8b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569f54fb1f0 0x5569f54fb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:02.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.589+0000 7fcd717b8b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5569f54099c0 con 0x5569f54fb1f0 2024-06-03T03:25:02.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.589+0000 7fcd717b8b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5569f544cab0 con 0x5569f5500b40 2024-06-03T03:25:02.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.589+0000 7fcd717b8b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5569f54078a0 con 0x5569f5501460 2024-06-03T03:25:02.591 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.589+0000 7fcd5fe00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569f5501460 0x5569f54facb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:02.591 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.589+0000 7fcd64e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569f5500b40 0x5569f5500f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:02.591 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.589+0000 7fcd5fe00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569f5501460 0x5569f54facb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:50570/0 (socket says 172.21.15.117:50570) 2024-06-03T03:25:02.591 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.589+0000 7fcd5fe00700 1 -- 172.21.15.117:0/1515347458 learned_addr learned my addr 172.21.15.117:0/1515347458 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:25:02.591 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.590+0000 7fcd5f400700 1 --2- 172.21.15.117:0/1515347458 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569f54fb1f0 0x5569f54fb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:02.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.590+0000 7fcd64e00700 1 -- 172.21.15.117:0/1515347458 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569f5501460 msgr2=0x5569f54facb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:25:02.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.590+0000 7fcd64e00700 1 --2- 172.21.15.117:0/1515347458 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569f5501460 0x5569f54facb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:02.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.590+0000 7fcd64e00700 1 -- 172.21.15.117:0/1515347458 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569f54fb1f0 msgr2=0x5569f54fb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:25:02.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.590+0000 7fcd64e00700 1 --2- 172.21.15.117:0/1515347458 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569f54fb1f0 0x5569f54fb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:02.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.590+0000 7fcd64e00700 1 -- 172.21.15.117:0/1515347458 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569f53f4f40 con 0x5569f5500b40 2024-06-03T03:25:02.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.590+0000 7fcd5cc00700 1 --2- 172.21.15.117:0/2689513524 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fcd440a2c20 0x7fcd440a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:02.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.590+0000 7fcd64e00700 1 --2- 172.21.15.117:0/1515347458 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569f5500b40 0x5569f5500f20 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fcd6006b9f0 tx=0x7fcd58009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89d2da003c835cea server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:25:02.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.590+0000 7fcd5cc00700 1 -- 172.21.15.117:0/2689513524 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7fcd4c0cf2a0 con 0x7fcd60074fe0 2024-06-03T03:25:02.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.590+0000 7fcd55600700 1 -- 172.21.15.117:0/1515347458 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd58019020 con 0x5569f5500b40 2024-06-03T03:25:02.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.591+0000 7fcd55600700 1 -- 172.21.15.117:0/1515347458 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd58011e80 con 0x5569f5500b40 2024-06-03T03:25:02.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.591+0000 7fcd55600700 1 -- 172.21.15.117:0/1515347458 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd58007700 con 0x5569f5500b40 2024-06-03T03:25:02.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.591+0000 7fcd5fe00700 1 --2- 172.21.15.117:0/2689513524 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fcd440a2c20 0x7fcd440a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:02.594 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.591+0000 7fcd5fe00700 1 --2- 172.21.15.117:0/2689513524 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fcd440a2c20 0x7fcd440a50e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fcd50009750 tx=0x7fcd50005c40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:25:02.594 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.591+0000 7fcd717b8b80 1 -- 172.21.15.117:0/1515347458 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569f5500b40 msgr2=0x5569f5500f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:02.594 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.591+0000 7fcd717b8b80 1 --2- 172.21.15.117:0/1515347458 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569f5500b40 0x5569f5500f20 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fcd6006b9f0 tx=0x7fcd58009920 comp rx=0 tx=0).stop 2024-06-03T03:25:02.594 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.591+0000 7fcd717b8b80 1 -- 172.21.15.117:0/1515347458 shutdown_connections 2024-06-03T03:25:02.594 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.591+0000 7fcd717b8b80 1 --2- 172.21.15.117:0/1515347458 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569f54fb1f0 0x5569f54fb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:02.594 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.591+0000 7fcd717b8b80 1 --2- 172.21.15.117:0/1515347458 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569f5500b40 0x5569f5500f20 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:02.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.591+0000 7fcd717b8b80 1 --2- 172.21.15.117:0/1515347458 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569f5501460 0x5569f54facb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:02.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.591+0000 7fcd717b8b80 1 -- 172.21.15.117:0/1515347458 >> 172.21.15.117:0/1515347458 conn(0x5569f5427120 msgr2=0x5569f54f5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:25:02.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.592+0000 7fcd717b8b80 1 -- 172.21.15.117:0/1515347458 shutdown_connections 2024-06-03T03:25:02.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.592+0000 7fcd717b8b80 1 -- 172.21.15.117:0/1515347458 wait complete. 2024-06-03T03:25:02.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.592+0000 7fcd717b8b80 1 -- start start 2024-06-03T03:25:02.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.593+0000 7fcd717b8b80 1 Processor -- start 2024-06-03T03:25:02.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.593+0000 7fcd717b8b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569f54fb1f0 0x5569f5507250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:02.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.593+0000 7fcd717b8b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569f5500b40 0x5569f5504550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:02.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.593+0000 7fcd717b8b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569f5501460 0x5569f5504a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:02.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.593+0000 7fcd717b8b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5569f53e8f10 con 0x5569f5500b40 2024-06-03T03:25:02.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.593+0000 7fcd717b8b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5569f5448a70 con 0x5569f5501460 2024-06-03T03:25:02.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.593+0000 7fcd717b8b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5569f5507e40 con 0x5569f54fb1f0 2024-06-03T03:25:02.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.593+0000 7fcd717b8b80 10 client.0 _ll_register_callbacks cb 0x5569f53e2330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:25:02.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.594+0000 7fcd64e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569f5501460 0x5569f5504a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:02.596 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.594+0000 7fcd5fe00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569f54fb1f0 0x5569f5507250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:02.596 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.594+0000 7fcd64e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569f5501460 0x5569f5504a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:56450/0 (socket says 172.21.15.117:56450) 2024-06-03T03:25:02.596 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.594+0000 7fcd5f400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569f5500b40 0x5569f5504550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:02.596 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.594+0000 7fcd64e00700 1 -- 172.21.15.117:0/1039187226 learned_addr learned my addr 172.21.15.117:0/1039187226 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:25:02.598 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.594+0000 7fcd64e00700 1 -- 172.21.15.117:0/1039187226 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569f54fb1f0 msgr2=0x5569f5507250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:25:02.598 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.594+0000 7fcd64e00700 1 --2- 172.21.15.117:0/1039187226 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569f54fb1f0 0x5569f5507250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:02.598 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.594+0000 7fcd64e00700 1 -- 172.21.15.117:0/1039187226 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569f5500b40 msgr2=0x5569f5504550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:25:02.598 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.594+0000 7fcd64e00700 1 --2- 172.21.15.117:0/1039187226 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569f5500b40 0x5569f5504550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:02.598 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.594+0000 7fcd64e00700 1 -- 172.21.15.117:0/1039187226 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569f539b880 con 0x5569f5501460 2024-06-03T03:25:02.598 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.594+0000 7fcd5f400700 1 --2- 172.21.15.117:0/1039187226 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569f5500b40 0x5569f5504550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:25:02.598 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.594+0000 7fcd64e00700 1 --2- 172.21.15.117:0/1039187226 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569f5501460 0x5569f5504a90 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fcd58004300 tx=0x7fcd58004330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6a3cedade3748f6 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:25:02.598 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.594+0000 7fcd4a000700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-06-03T03:25:02.598 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.595+0000 7fcd4a000700 1 -- 172.21.15.117:0/1039187226 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd58019040 con 0x5569f5501460 2024-06-03T03:25:02.598 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.595+0000 7fcd4a000700 1 -- 172.21.15.117:0/1039187226 <== mon.2 v2:172.21.15.151:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7fcd58003a00 con 0x5569f5501460 2024-06-03T03:25:02.598 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.595+0000 7fcd717b8b80 1 -- 172.21.15.117:0/1039187226 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5569f5459fb0 con 0x5569f5501460 2024-06-03T03:25:02.599 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.595+0000 7fcd4a000700 1 -- 172.21.15.117:0/1039187226 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcd58031050 con 0x5569f5501460 2024-06-03T03:25:02.599 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.595+0000 7fcd717b8b80 10 client.25172 fetch_fsmap learned FSMap version 30 2024-06-03T03:25:02.599 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.595+0000 7fcd717b8b80 1 -- 172.21.15.117:0/1039187226 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5569f55125c0 con 0x5569f5501460 2024-06-03T03:25:02.599 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.595+0000 7fcd4a000700 1 -- 172.21.15.117:0/1039187226 <== mon.2 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5569f5459fb0 con 0x5569f5501460 2024-06-03T03:25:02.599 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.595+0000 7fcd4a000700 1 -- 172.21.15.117:0/1039187226 <== mon.2 v2:172.21.15.151:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fcd58028470 con 0x5569f5501460 2024-06-03T03:25:02.599 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.596+0000 7fcd717b8b80 10 client.25172 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T03:25:02.599 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.596+0000 7fcd717b8b80 10 client.25172 resolve_mds: resolved name 'i' to daemon mds.i 2024-06-03T03:25:02.599 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.596+0000 7fcd717b8b80 20 client.25172 populate_metadata read hostname 'smithi117' 2024-06-03T03:25:02.599 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.596+0000 7fcd717b8b80 1 --2- 172.21.15.117:0/1039187226 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x5569f5514810 0x5569f5515c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:02.599 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.596+0000 7fcd717b8b80 4 client.25172 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T03:25:02.599 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.596+0000 7fcd717b8b80 1 -- 172.21.15.117:0/1039187226 --> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5569f5454530 con 0x5569f5514810 2024-06-03T03:25:02.599 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.596+0000 7fcd5fe00700 1 --2- 172.21.15.117:0/1039187226 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x5569f5514810 0x5569f5515c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:02.599 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.597+0000 7fcd5fe00700 1 --2- 172.21.15.117:0/1039187226 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x5569f5514810 0x5569f5515c80 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=930fdf6021e71424 server_cookie=33156a26b9835bf9 in_seq=0 out_seq=0 2024-06-03T03:25:02.600 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.597+0000 7fcd4a000700 10 client.25172 ms_handle_connect on v2:172.21.15.183:6838/1439615083 2024-06-03T03:25:02.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.598+0000 7fcd4a000700 1 -- 172.21.15.117:0/1039187226 <== mds.0 v2:172.21.15.183:6838/1439615083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5569f5454530 con 0x5569f5514810 2024-06-03T03:25:02.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.598+0000 7fcd4a000700 10 client.25172 handle_command_reply: tid=0 2024-06-03T03:25:02.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.598+0000 7fcd717b8b80 1 client.25172 shutdown 2024-06-03T03:25:02.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.598+0000 7fcd717b8b80 1 -- 172.21.15.117:0/1039187226 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569f5501460 msgr2=0x5569f5504a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:02.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.598+0000 7fcd717b8b80 1 --2- 172.21.15.117:0/1039187226 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569f5501460 0x5569f5504a90 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fcd58004300 tx=0x7fcd58004330 comp rx=0 tx=0).stop 2024-06-03T03:25:02.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.598+0000 7fcd717b8b80 1 -- 172.21.15.117:0/1039187226 shutdown_connections 2024-06-03T03:25:02.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.598+0000 7fcd717b8b80 1 --2- 172.21.15.117:0/1039187226 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x5569f5514810 0x5569f5515c80 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:02.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.598+0000 7fcd717b8b80 1 --2- 172.21.15.117:0/1039187226 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569f5501460 0x5569f5504a90 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:02.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.598+0000 7fcd717b8b80 1 --2- 172.21.15.117:0/1039187226 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569f54fb1f0 0x5569f5507250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:02.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.598+0000 7fcd4a000700 0 client.25172 ms_handle_reset on v2:172.21.15.183:6838/1439615083 2024-06-03T03:25:02.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.598+0000 7fcd717b8b80 1 --2- 172.21.15.117:0/1039187226 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569f5500b40 0x5569f5504550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:02.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.598+0000 7fcd717b8b80 1 -- 172.21.15.117:0/1039187226 >> 172.21.15.117:0/1039187226 conn(0x5569f5427120 msgr2=0x5569f54ef740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:25:02.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.598+0000 7fcd717b8b80 1 -- 172.21.15.117:0/1039187226 shutdown_connections 2024-06-03T03:25:02.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.598+0000 7fcd717b8b80 1 -- 172.21.15.117:0/1039187226 wait complete. 2024-06-03T03:25:02.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.598+0000 7fcd717b8b80 20 client.25172 trim_cache size 0 max 16384 2024-06-03T03:25:02.621 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.621+0000 7fcd717b8b80 1 Processor -- start 2024-06-03T03:25:02.622 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.621+0000 7fcd717b8b80 1 -- start start 2024-06-03T03:25:02.624 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.621+0000 7fcd717b8b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569f54fb1f0 0x5569f54facb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:02.624 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.621+0000 7fcd717b8b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569f5500b40 0x5569f55034d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:02.624 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.621+0000 7fcd717b8b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569f5501460 0x5569f54f00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:02.624 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.621+0000 7fcd717b8b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5569f5513fc0 con 0x5569f5501460 2024-06-03T03:25:02.624 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.621+0000 7fcd717b8b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5569f5514690 con 0x5569f5500b40 2024-06-03T03:25:02.625 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.621+0000 7fcd717b8b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5569f5514d60 con 0x5569f54fb1f0 2024-06-03T03:25:02.625 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.622+0000 7fcd64e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569f5501460 0x5569f54f00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:02.625 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.622+0000 7fcd64e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569f5501460 0x5569f54f00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:33474/0 (socket says 172.21.15.117:33474) 2024-06-03T03:25:02.625 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.622+0000 7fcd64e00700 1 -- 172.21.15.117:0/758989715 learned_addr learned my addr 172.21.15.117:0/758989715 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:25:02.625 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.622+0000 7fcd5f400700 1 --2- 172.21.15.117:0/758989715 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569f5500b40 0x5569f55034d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:02.625 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.622+0000 7fcd5fe00700 1 --2- 172.21.15.117:0/758989715 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569f54fb1f0 0x5569f54facb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:02.625 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.622+0000 7fcd64e00700 1 -- 172.21.15.117:0/758989715 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569f54fb1f0 msgr2=0x5569f54facb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:25:02.625 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.622+0000 7fcd64e00700 1 --2- 172.21.15.117:0/758989715 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569f54fb1f0 0x5569f54facb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:02.625 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.622+0000 7fcd64e00700 1 -- 172.21.15.117:0/758989715 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569f5500b40 msgr2=0x5569f55034d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:25:02.625 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.622+0000 7fcd64e00700 1 --2- 172.21.15.117:0/758989715 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569f5500b40 0x5569f55034d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:02.625 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.622+0000 7fcd64e00700 1 -- 172.21.15.117:0/758989715 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569f5501170 con 0x5569f5501460 2024-06-03T03:25:02.625 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.622+0000 7fcd5f400700 1 --2- 172.21.15.117:0/758989715 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569f5500b40 0x5569f55034d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:25:02.626 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.623+0000 7fcd64e00700 1 --2- 172.21.15.117:0/758989715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569f5501460 0x5569f54f00e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fcd58004aa0 tx=0x7fcd580021f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18e343107d032f73 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:25:02.626 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.623+0000 7fcd54c00700 1 -- 172.21.15.117:0/758989715 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd58019070 con 0x5569f5501460 2024-06-03T03:25:02.626 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.623+0000 7fcd54c00700 1 -- 172.21.15.117:0/758989715 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd58023070 con 0x5569f5501460 2024-06-03T03:25:02.626 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.623+0000 7fcd54c00700 1 -- 172.21.15.117:0/758989715 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd58007500 con 0x5569f5501460 2024-06-03T03:25:02.626 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.623+0000 7fcd717b8b80 1 -- 172.21.15.117:0/758989715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569f5501460 msgr2=0x5569f54f00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:02.626 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.623+0000 7fcd717b8b80 1 --2- 172.21.15.117:0/758989715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569f5501460 0x5569f54f00e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fcd58004aa0 tx=0x7fcd580021f0 comp rx=0 tx=0).stop 2024-06-03T03:25:02.626 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.624+0000 7fcd717b8b80 1 -- 172.21.15.117:0/758989715 shutdown_connections 2024-06-03T03:25:02.626 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.624+0000 7fcd717b8b80 1 --2- 172.21.15.117:0/758989715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569f5501460 0x5569f54f00e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:02.626 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.624+0000 7fcd717b8b80 1 --2- 172.21.15.117:0/758989715 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569f54fb1f0 0x5569f54facb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:02.626 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.624+0000 7fcd717b8b80 1 --2- 172.21.15.117:0/758989715 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569f5500b40 0x5569f55034d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:02.626 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.624+0000 7fcd717b8b80 1 -- 172.21.15.117:0/758989715 >> 172.21.15.117:0/758989715 conn(0x5569f5427120 msgr2=0x5569f5503e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:25:02.626 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.624+0000 7fcd717b8b80 1 -- 172.21.15.117:0/758989715 shutdown_connections 2024-06-03T03:25:02.626 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.624+0000 7fcd717b8b80 1 -- 172.21.15.117:0/758989715 wait complete. 2024-06-03T03:25:02.631 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.624+0000 7fcd717b8b80 1 -- start start 2024-06-03T03:25:02.631 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.624+0000 7fcd717b8b80 1 Processor -- start 2024-06-03T03:25:02.631 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.625+0000 7fcd717b8b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569f54fb1f0 0x5569f551f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:02.631 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.625+0000 7fcd717b8b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569f5500b40 0x5569f5506340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:02.631 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.625+0000 7fcd717b8b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569f5501460 0x5569f5506950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:02.632 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.625+0000 7fcd717b8b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5569f5515b00 con 0x5569f54fb1f0 2024-06-03T03:25:02.632 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.625+0000 7fcd717b8b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5569f5515430 con 0x5569f5501460 2024-06-03T03:25:02.632 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.625+0000 7fcd717b8b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5569f55161d0 con 0x5569f5500b40 2024-06-03T03:25:02.632 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.625+0000 7fcd717b8b80 10 client.0 _ll_register_callbacks cb 0x5569f53e2330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:25:02.632 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.625+0000 7fcd5fe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569f54fb1f0 0x5569f551f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:02.632 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.625+0000 7fcd5fe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569f54fb1f0 0x5569f551f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:33486/0 (socket says 172.21.15.117:33486) 2024-06-03T03:25:02.632 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.625+0000 7fcd5fe00700 1 -- 172.21.15.117:0/4033995380 learned_addr learned my addr 172.21.15.117:0/4033995380 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:25:02.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.625+0000 7fcd5f400700 1 --2- 172.21.15.117:0/4033995380 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569f5500b40 0x5569f5506340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:02.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.625+0000 7fcd64e00700 1 --2- 172.21.15.117:0/4033995380 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569f5501460 0x5569f5506950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:02.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.626+0000 7fcd5f400700 1 -- 172.21.15.117:0/4033995380 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569f5501460 msgr2=0x5569f5506950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:25:02.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.626+0000 7fcd5f400700 1 --2- 172.21.15.117:0/4033995380 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569f5501460 0x5569f5506950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:02.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.626+0000 7fcd5f400700 1 -- 172.21.15.117:0/4033995380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569f54fb1f0 msgr2=0x5569f551f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:25:02.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.626+0000 7fcd5f400700 1 --2- 172.21.15.117:0/4033995380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569f54fb1f0 0x5569f551f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:02.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.626+0000 7fcd5f400700 1 -- 172.21.15.117:0/4033995380 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569f5507160 con 0x5569f5500b40 2024-06-03T03:25:02.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.626+0000 7fcd5fe00700 1 --2- 172.21.15.117:0/4033995380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569f54fb1f0 0x5569f551f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:25:02.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.626+0000 7fcd64e00700 1 --2- 172.21.15.117:0/4033995380 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569f5501460 0x5569f5506950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:25:02.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.626+0000 7fcd5f400700 1 --2- 172.21.15.117:0/4033995380 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569f5500b40 0x5569f5506340 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c0cc9b0 tx=0x7fcd4c0a41d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=907d494260b6947a server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:25:02.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.626+0000 7fcd4a000700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-06-03T03:25:02.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.626+0000 7fcd4a000700 1 -- 172.21.15.117:0/4033995380 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c09be10 con 0x5569f5500b40 2024-06-03T03:25:02.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.626+0000 7fcd717b8b80 1 -- 172.21.15.117:0/4033995380 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd58023070 con 0x5569f5500b40 2024-06-03T03:25:02.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.626+0000 7fcd4a000700 1 -- 172.21.15.117:0/4033995380 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7fcd4c0ccac0 con 0x5569f5500b40 2024-06-03T03:25:02.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.626+0000 7fcd4a000700 1 -- 172.21.15.117:0/4033995380 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcd4c09b710 con 0x5569f5500b40 2024-06-03T03:25:02.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.627+0000 7fcd717b8b80 10 client.25207 fetch_fsmap learned FSMap version 30 2024-06-03T03:25:02.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.627+0000 7fcd717b8b80 1 -- 172.21.15.117:0/4033995380 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5569f55801b0 con 0x5569f5500b40 2024-06-03T03:25:02.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.627+0000 7fcd4a000700 1 -- 172.21.15.117:0/4033995380 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd58023070 con 0x5569f5500b40 2024-06-03T03:25:02.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.627+0000 7fcd4a000700 1 -- 172.21.15.117:0/4033995380 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fcd4c0d2b30 con 0x5569f5500b40 2024-06-03T03:25:02.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.627+0000 7fcd717b8b80 10 client.25207 fetch_fsmap finished waiting for FSMap version 30 2024-06-03T03:25:02.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.627+0000 7fcd717b8b80 10 client.25207 resolve_mds: resolved name 'i' to daemon mds.i 2024-06-03T03:25:02.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.627+0000 7fcd717b8b80 20 client.25207 populate_metadata read hostname 'smithi117' 2024-06-03T03:25:02.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.627+0000 7fcd717b8b80 1 --2- 172.21.15.117:0/4033995380 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x5569f5584c90 0x5569f55839a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:02.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.627+0000 7fcd717b8b80 4 client.25207 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-03T03:25:02.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.627+0000 7fcd717b8b80 1 -- 172.21.15.117:0/4033995380 --> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] -- command(tid 0: {"prefix": "status"}) -- 0x5569f54fbca0 con 0x5569f5584c90 2024-06-03T03:25:02.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.627+0000 7fcd5fe00700 1 --2- 172.21.15.117:0/4033995380 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x5569f5584c90 0x5569f55839a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:02.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.628+0000 7fcd5fe00700 1 --2- 172.21.15.117:0/4033995380 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x5569f5584c90 0x5569f55839a0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83834c0cb3e0b38e server_cookie=4ca1b821e7f9f508 in_seq=0 out_seq=0 2024-06-03T03:25:02.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.628+0000 7fcd4a000700 10 client.25207 ms_handle_connect on v2:172.21.15.183:6838/1439615083 2024-06-03T03:25:02.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.628+0000 7fcd4a000700 1 -- 172.21.15.117:0/4033995380 <== mds.0 v2:172.21.15.183:6838/1439615083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5569f54fbca0 con 0x5569f5584c90 2024-06-03T03:25:02.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.628+0000 7fcd4a000700 10 client.25207 handle_command_reply: tid=0 2024-06-03T03:25:02.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.628+0000 7fcd717b8b80 1 client.25207 shutdown 2024-06-03T03:25:02.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.628+0000 7fcd717b8b80 1 -- 172.21.15.117:0/4033995380 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569f5500b40 msgr2=0x5569f5506340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:02.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.628+0000 7fcd717b8b80 1 --2- 172.21.15.117:0/4033995380 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569f5500b40 0x5569f5506340 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c0cc9b0 tx=0x7fcd4c0a41d0 comp rx=0 tx=0).stop 2024-06-03T03:25:02.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.628+0000 7fcd717b8b80 1 -- 172.21.15.117:0/4033995380 shutdown_connections 2024-06-03T03:25:02.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.628+0000 7fcd717b8b80 1 --2- 172.21.15.117:0/4033995380 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x5569f5584c90 0x5569f55839a0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:02.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.628+0000 7fcd717b8b80 1 --2- 172.21.15.117:0/4033995380 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569f5501460 0x5569f5506950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:02.639 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.628+0000 7fcd4a000700 0 client.25207 ms_handle_reset on v2:172.21.15.183:6838/1439615083 2024-06-03T03:25:02.639 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.628+0000 7fcd717b8b80 1 --2- 172.21.15.117:0/4033995380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569f54fb1f0 0x5569f551f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:02.639 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.628+0000 7fcd717b8b80 1 --2- 172.21.15.117:0/4033995380 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569f5500b40 0x5569f5506340 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:02.639 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.628+0000 7fcd717b8b80 1 -- 172.21.15.117:0/4033995380 >> 172.21.15.117:0/4033995380 conn(0x5569f5427120 msgr2=0x5569f54ff690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:25:02.639 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.629+0000 7fcd717b8b80 1 -- 172.21.15.117:0/4033995380 shutdown_connections 2024-06-03T03:25:02.639 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.629+0000 7fcd717b8b80 1 -- 172.21.15.117:0/4033995380 wait complete. 2024-06-03T03:25:02.639 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.629+0000 7fcd717b8b80 20 client.25207 trim_cache size 0 max 16384 2024-06-03T03:25:02.640 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-06-03T03:25:02.640 INFO:teuthology.orchestra.run.smithi117.stdout: "cluster_fsid": "a0e8473a-2156-11ef-bc9b-c7b262605968", 2024-06-03T03:25:02.641 INFO:teuthology.orchestra.run.smithi117.stdout: "whoami": 0, 2024-06-03T03:25:02.641 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 24517, 2024-06-03T03:25:02.641 INFO:teuthology.orchestra.run.smithi117.stdout: "want_state": "up:active", 2024-06-03T03:25:02.641 INFO:teuthology.orchestra.run.smithi117.stdout: "state": "up:active", 2024-06-03T03:25:02.641 INFO:teuthology.orchestra.run.smithi117.stdout: "fs_name": "cephfs", 2024-06-03T03:25:02.642 INFO:teuthology.orchestra.run.smithi117.stdout: "rank_uptime": 317.00402784400001, 2024-06-03T03:25:02.642 INFO:teuthology.orchestra.run.smithi117.stdout: "mdsmap_epoch": 30, 2024-06-03T03:25:02.642 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_epoch": 99, 2024-06-03T03:25:02.642 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_epoch_barrier": 97, 2024-06-03T03:25:02.642 INFO:teuthology.orchestra.run.smithi117.stdout: "uptime": 350.18488087200001 2024-06-03T03:25:02.642 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-06-03T03:25:02.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.629+0000 7fcd66e00700 1 -- 172.21.15.117:0/2689513524 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fcd440a2c20 msgr2=0x7fcd440a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:02.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.629+0000 7fcd66e00700 1 --2- 172.21.15.117:0/2689513524 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fcd440a2c20 0x7fcd440a50e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fcd50009750 tx=0x7fcd50005c40 comp rx=0 tx=0).stop 2024-06-03T03:25:02.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.629+0000 7fcd66e00700 1 -- 172.21.15.117:0/2689513524 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd60074fe0 msgr2=0x7fcd6019b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:02.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.629+0000 7fcd66e00700 1 --2- 172.21.15.117:0/2689513524 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd60074fe0 0x7fcd6019b340 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c00e9c0 tx=0x7fcd4c00ecd0 comp rx=0 tx=0).stop 2024-06-03T03:25:02.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.630+0000 7fcd66e00700 1 -- 172.21.15.117:0/2689513524 shutdown_connections 2024-06-03T03:25:02.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.630+0000 7fcd66e00700 1 --2- 172.21.15.117:0/2689513524 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fcd440a2c20 0x7fcd440a50e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:02.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.630+0000 7fcd66e00700 1 --2- 172.21.15.117:0/2689513524 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd6010ee90 0x7fcd6019f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:02.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.630+0000 7fcd66e00700 1 --2- 172.21.15.117:0/2689513524 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd600746c0 0x7fcd6019ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:02.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.630+0000 7fcd66e00700 1 --2- 172.21.15.117:0/2689513524 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd60074fe0 0x7fcd6019b340 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:02.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.630+0000 7fcd66e00700 1 -- 172.21.15.117:0/2689513524 >> 172.21.15.117:0/2689513524 conn(0x7fcd600795f0 msgr2=0x7fcd60100750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:25:02.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.630+0000 7fcd66e00700 1 -- 172.21.15.117:0/2689513524 shutdown_connections 2024-06-03T03:25:02.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:02.630+0000 7fcd66e00700 1 -- 172.21.15.117:0/2689513524 wait complete. 2024-06-03T03:25:03.580 INFO:teuthology.run_tasks:Running task sequential... 2024-06-03T03:25:03.590 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-06-03T03:25:03.590 INFO:teuthology.task.sequential:In sequential, running task print... 2024-06-03T03:25:03.591 INFO:teuthology.task.print:Enabling mgr modules 2024-06-03T03:25:03.591 INFO:teuthology.run_tasks:Running task kclient... 2024-06-03T03:25:03.624 INFO:tasks.kclient:Mounting kernel clients... 2024-06-03T03:25:03.625 INFO:tasks.kclient:config is {'client.0': {}} 2024-06-03T03:25:03.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T03:25:03.625 DEBUG:teuthology.orchestra.run.smithi117:> ip netns list 2024-06-03T03:25:03.683 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-06-03T03:25:03.684 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T03:25:03.684 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-03T03:25:03.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T03:25:03.768 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip link delete ceph-brx 2024-06-03T03:25:03.873 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'nowsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-06-03T03:25:03.874 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-06-03T03:25:03.874 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-06-03T03:25:03.874 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph fs ls 2024-06-03T03:25:04.304 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:25:04.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:25:04 smithi151 ceph-mon[37043]: pgmap v498: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-06-03T03:25:04.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:25:04 smithi117 ceph-mon[31649]: pgmap v498: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-06-03T03:25:04.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:25:04 smithi183 ceph-mon[37326]: pgmap v498: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-06-03T03:25:06.189 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:25:06 smithi117 ceph-mon[31649]: pgmap v499: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:25:06.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:25:06 smithi151 ceph-mon[37043]: pgmap v499: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:25:06.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:25:06 smithi183 ceph-mon[37326]: pgmap v499: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:25:06.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:06.759+0000 7f2a55200700 1 -- 172.21.15.117:0/4083253736 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2a50105070 msgr2=0x7f2a50111580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:06.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:06.759+0000 7f2a55200700 1 --2- 172.21.15.117:0/4083253736 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2a50105070 0x7f2a50111580 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2a48009f90 tx=0x7f2a4800b330 comp rx=0 tx=0).stop 2024-06-03T03:25:06.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:06.759+0000 7f2a55200700 1 -- 172.21.15.117:0/4083253736 shutdown_connections 2024-06-03T03:25:06.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:06.759+0000 7f2a55200700 1 --2- 172.21.15.117:0/4083253736 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2a50105070 0x7f2a50111580 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:06.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:06.759+0000 7f2a55200700 1 --2- 172.21.15.117:0/4083253736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a5010a6b0 0x7f2a5010aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:06.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:06.759+0000 7f2a55200700 1 --2- 172.21.15.117:0/4083253736 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a501046b0 0x7f2a50104b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:06.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:06.759+0000 7f2a55200700 1 -- 172.21.15.117:0/4083253736 >> 172.21.15.117:0/4083253736 conn(0x7f2a501001a0 msgr2=0x7f2a501025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:25:06.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:06.759+0000 7f2a55200700 1 -- 172.21.15.117:0/4083253736 shutdown_connections 2024-06-03T03:25:06.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:06.759+0000 7f2a55200700 1 -- 172.21.15.117:0/4083253736 wait complete. 2024-06-03T03:25:06.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:06.760+0000 7f2a55200700 1 Processor -- start 2024-06-03T03:25:06.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:06.761+0000 7f2a55200700 1 -- start start 2024-06-03T03:25:06.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:06.761+0000 7f2a55200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2a501046b0 0x7f2a5019ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:06.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:06.761+0000 7f2a55200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a50105070 0x7f2a5019b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:06.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:06.761+0000 7f2a55200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a5010a6b0 0x7f2a5019f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:06.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:06.761+0000 7f2a55200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2a50114a40 con 0x7f2a50105070 2024-06-03T03:25:06.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:06.761+0000 7f2a55200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2a501148c0 con 0x7f2a501046b0 2024-06-03T03:25:06.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:06.761+0000 7f2a55200700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2a50114bc0 con 0x7f2a5010a6b0 2024-06-03T03:25:06.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:06.761+0000 7f2a4d400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a50105070 0x7f2a5019b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:06.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:06.761+0000 7f2a4de00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2a501046b0 0x7f2a5019ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:06.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:06.762+0000 7f2a4d400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a50105070 0x7f2a5019b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:33510/0 (socket says 172.21.15.117:33510) 2024-06-03T03:25:06.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:06.762+0000 7f2a4d400700 1 -- 172.21.15.117:0/1513262997 learned_addr learned my addr 172.21.15.117:0/1513262997 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:25:06.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:06.762+0000 7f2a4e800700 1 --2- 172.21.15.117:0/1513262997 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a5010a6b0 0x7f2a5019f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:06.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:06.762+0000 7f2a4d400700 1 -- 172.21.15.117:0/1513262997 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a5010a6b0 msgr2=0x7f2a5019f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:06.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:06.762+0000 7f2a4d400700 1 --2- 172.21.15.117:0/1513262997 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a5010a6b0 0x7f2a5019f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:06.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:06.762+0000 7f2a4d400700 1 -- 172.21.15.117:0/1513262997 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2a501046b0 msgr2=0x7f2a5019ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:06.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:06.762+0000 7f2a4d400700 1 --2- 172.21.15.117:0/1513262997 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2a501046b0 0x7f2a5019ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:06.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:06.762+0000 7f2a4d400700 1 -- 172.21.15.117:0/1513262997 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a5019fe10 con 0x7f2a50105070 2024-06-03T03:25:06.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:06.762+0000 7f2a4de00700 1 --2- 172.21.15.117:0/1513262997 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2a501046b0 0x7f2a5019ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:25:06.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:06.762+0000 7f2a4d400700 1 --2- 172.21.15.117:0/1513262997 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a50105070 0x7f2a5019b390 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c00e9c0 tx=0x7f2a3c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:25:06.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:06.763+0000 7f2a46a00700 1 -- 172.21.15.117:0/1513262997 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a3c004a30 con 0x7f2a50105070 2024-06-03T03:25:06.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:06.763+0000 7f2a46a00700 1 -- 172.21.15.117:0/1513262997 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a3c00cca0 con 0x7f2a50105070 2024-06-03T03:25:06.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:06.763+0000 7f2a55200700 1 -- 172.21.15.117:0/1513262997 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a501a0100 con 0x7f2a50105070 2024-06-03T03:25:06.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:06.763+0000 7f2a46a00700 1 -- 172.21.15.117:0/1513262997 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a3c005610 con 0x7f2a50105070 2024-06-03T03:25:06.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:06.763+0000 7f2a55200700 1 -- 172.21.15.117:0/1513262997 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a501a7980 con 0x7f2a50105070 2024-06-03T03:25:06.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:06.764+0000 7f2a55200700 1 -- 172.21.15.117:0/1513262997 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a50069180 con 0x7f2a50105070 2024-06-03T03:25:06.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:06.767+0000 7f2a46a00700 1 -- 172.21.15.117:0/1513262997 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f2a3c004cf0 con 0x7f2a50105070 2024-06-03T03:25:06.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:06.769+0000 7f2a46a00700 1 --2- 172.21.15.117:0/1513262997 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f2a340a2c20 0x7f2a340a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:06.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:06.769+0000 7f2a46a00700 1 -- 172.21.15.117:0/1513262997 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f2a3c0cf510 con 0x7f2a50105070 2024-06-03T03:25:06.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:06.769+0000 7f2a4de00700 1 --2- 172.21.15.117:0/1513262997 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f2a340a2c20 0x7f2a340a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:06.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:06.770+0000 7f2a4de00700 1 --2- 172.21.15.117:0/1513262997 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f2a340a2c20 0x7f2a340a50e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2a40009fd0 tx=0x7f2a40009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:25:06.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:06.774+0000 7f2a46a00700 1 -- 172.21.15.117:0/1513262997 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2a3c09c080 con 0x7f2a50105070 2024-06-03T03:25:07.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:07.044+0000 7f2a55200700 1 -- 172.21.15.117:0/1513262997 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f2a5002dca0 con 0x7f2a50105070 2024-06-03T03:25:07.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:07.044+0000 7f2a46a00700 1 -- 172.21.15.117:0/1513262997 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v30) ==== 53+0+88 (secure 0 0 0) 0x7f2a3c09b980 con 0x7f2a50105070 2024-06-03T03:25:07.045 INFO:teuthology.orchestra.run.smithi117.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-06-03T03:25:07.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:07.047+0000 7f2a55200700 1 -- 172.21.15.117:0/1513262997 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f2a340a2c20 msgr2=0x7f2a340a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:07.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:07.047+0000 7f2a55200700 1 --2- 172.21.15.117:0/1513262997 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f2a340a2c20 0x7f2a340a50e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2a40009fd0 tx=0x7f2a40009260 comp rx=0 tx=0).stop 2024-06-03T03:25:07.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:07.047+0000 7f2a55200700 1 -- 172.21.15.117:0/1513262997 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a50105070 msgr2=0x7f2a5019b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:07.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:07.047+0000 7f2a55200700 1 --2- 172.21.15.117:0/1513262997 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a50105070 0x7f2a5019b390 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c00e9c0 tx=0x7f2a3c00ecd0 comp rx=0 tx=0).stop 2024-06-03T03:25:07.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:07.047+0000 7f2a55200700 1 -- 172.21.15.117:0/1513262997 shutdown_connections 2024-06-03T03:25:07.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:07.047+0000 7f2a55200700 1 --2- 172.21.15.117:0/1513262997 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f2a340a2c20 0x7f2a340a50e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:07.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:07.047+0000 7f2a55200700 1 --2- 172.21.15.117:0/1513262997 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a5010a6b0 0x7f2a5019f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:07.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:07.047+0000 7f2a55200700 1 --2- 172.21.15.117:0/1513262997 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2a501046b0 0x7f2a5019ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:07.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:07.047+0000 7f2a55200700 1 --2- 172.21.15.117:0/1513262997 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a50105070 0x7f2a5019b390 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:07.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:07.047+0000 7f2a55200700 1 -- 172.21.15.117:0/1513262997 >> 172.21.15.117:0/1513262997 conn(0x7f2a501001a0 msgr2=0x7f2a50100760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:25:07.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:07.048+0000 7f2a55200700 1 -- 172.21.15.117:0/1513262997 shutdown_connections 2024-06-03T03:25:07.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:07.048+0000 7f2a55200700 1 -- 172.21.15.117:0/1513262997 wait complete. 2024-06-03T03:25:08.006 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-06-03T03:25:08.006 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-06-03T03:25:08.006 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi117.front.sepia.ceph.com 2024-06-03T03:25:08.006 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-06-03T03:25:08.007 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-06-03T03:25:08.007 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-06-03T03:25:08.007 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-06-03T03:25:08.007 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-06-03T03:25:08.007 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-06-03T03:25:08.007 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T03:25:08.007 DEBUG:teuthology.orchestra.run.smithi117:> ip addr 2024-06-03T03:25:08.065 INFO:teuthology.orchestra.run.smithi117.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-06-03T03:25:08.066 INFO:teuthology.orchestra.run.smithi117.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-06-03T03:25:08.066 INFO:teuthology.orchestra.run.smithi117.stdout: inet 127.0.0.1/8 scope host lo 2024-06-03T03:25:08.066 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft forever preferred_lft forever 2024-06-03T03:25:08.066 INFO:teuthology.orchestra.run.smithi117.stdout: inet6 ::1/128 scope host 2024-06-03T03:25:08.066 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft forever preferred_lft forever 2024-06-03T03:25:08.066 INFO:teuthology.orchestra.run.smithi117.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-03T03:25:08.066 INFO:teuthology.orchestra.run.smithi117.stdout: link/ether 0c:c4:7a:d8:a8:8a brd ff:ff:ff:ff:ff:ff 2024-06-03T03:25:08.066 INFO:teuthology.orchestra.run.smithi117.stdout: altname enp7s0f0 2024-06-03T03:25:08.066 INFO:teuthology.orchestra.run.smithi117.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-03T03:25:08.066 INFO:teuthology.orchestra.run.smithi117.stdout: link/ether 0c:c4:7a:d8:a8:8b brd ff:ff:ff:ff:ff:ff 2024-06-03T03:25:08.066 INFO:teuthology.orchestra.run.smithi117.stdout: altname enp7s0f1 2024-06-03T03:25:08.067 INFO:teuthology.orchestra.run.smithi117.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-03T03:25:08.067 INFO:teuthology.orchestra.run.smithi117.stdout: link/ether 0c:c4:7a:8f:44:62 brd ff:ff:ff:ff:ff:ff 2024-06-03T03:25:08.067 INFO:teuthology.orchestra.run.smithi117.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-06-03T03:25:08.067 INFO:teuthology.orchestra.run.smithi117.stdout: link/ether 0c:c4:7a:8f:44:63 brd ff:ff:ff:ff:ff:ff 2024-06-03T03:25:08.067 INFO:teuthology.orchestra.run.smithi117.stdout: inet 172.21.15.117/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-06-03T03:25:08.067 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft 41689sec preferred_lft 41689sec 2024-06-03T03:25:08.067 INFO:teuthology.orchestra.run.smithi117.stdout: inet6 fe80::ec4:7aff:fe8f:4463/64 scope link 2024-06-03T03:25:08.067 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft forever preferred_lft forever 2024-06-03T03:25:08.068 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-06-03T03:25:08.068 DEBUG:teuthology.orchestra.run.smithi117:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-03T03:25:08.068 DEBUG:teuthology.orchestra.run.smithi117:> set -e 2024-06-03T03:25:08.069 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip link add name ceph-brx type bridge 2024-06-03T03:25:08.069 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip addr flush dev ceph-brx 2024-06-03T03:25:08.069 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip link set ceph-brx up 2024-06-03T03:25:08.069 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-06-03T03:25:08.069 DEBUG:teuthology.orchestra.run.smithi117:> ') 2024-06-03T03:25:08.173 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-03T03:25:08.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:25:08 smithi117 ceph-mon[31649]: pgmap v500: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-03T03:25:08.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:25:08 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/1513262997' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-03T03:25:08.304 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-03T03:25:08.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T03:25:08.315 DEBUG:teuthology.orchestra.run.smithi117:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-06-03T03:25:08.379 INFO:teuthology.orchestra.run.smithi117.stdout:1 2024-06-03T03:25:08.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T03:25:08.383 DEBUG:teuthology.orchestra.run.smithi117:> route 2024-06-03T03:25:08.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:25:08 smithi151 ceph-mon[37043]: pgmap v500: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-03T03:25:08.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:25:08 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/1513262997' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-03T03:25:08.429 INFO:teuthology.orchestra.run.smithi117.stdout:Kernel IP routing table 2024-06-03T03:25:08.429 INFO:teuthology.orchestra.run.smithi117.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-06-03T03:25:08.429 INFO:teuthology.orchestra.run.smithi117.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-06-03T03:25:08.429 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-06-03T03:25:08.429 INFO:teuthology.orchestra.run.smithi117.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-06-03T03:25:08.429 DEBUG:teuthology.orchestra.run.smithi117:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-03T03:25:08.430 DEBUG:teuthology.orchestra.run.smithi117:> set -e 2024-06-03T03:25:08.430 DEBUG:teuthology.orchestra.run.smithi117:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-06-03T03:25:08.430 DEBUG:teuthology.orchestra.run.smithi117:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-06-03T03:25:08.430 DEBUG:teuthology.orchestra.run.smithi117:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-06-03T03:25:08.430 DEBUG:teuthology.orchestra.run.smithi117:> ') 2024-06-03T03:25:08.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:25:08 smithi183 ceph-mon[37326]: pgmap v500: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-03T03:25:08.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:25:08 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/1513262997' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-03T03:25:08.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-03T03:25:08.677 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-03T03:25:08.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T03:25:08.687 DEBUG:teuthology.orchestra.run.smithi117:> ip netns list 2024-06-03T03:25:08.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T03:25:08.738 DEBUG:teuthology.orchestra.run.smithi117:> ip netns list-id 2024-06-03T03:25:08.791 DEBUG:teuthology.orchestra.run.smithi117:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-03T03:25:08.791 DEBUG:teuthology.orchestra.run.smithi117:> set -e 2024-06-03T03:25:08.791 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-06-03T03:25:08.791 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-06-03T03:25:08.791 DEBUG:teuthology.orchestra.run.smithi117:> ') 2024-06-03T03:25:08.893 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-03T03:25:08.966 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-03T03:25:08.976 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-06-03T03:25:08.976 DEBUG:teuthology.orchestra.run.smithi117:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-03T03:25:08.977 DEBUG:teuthology.orchestra.run.smithi117:> set -e 2024-06-03T03:25:08.977 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-06-03T03:25:08.977 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-06-03T03:25:08.977 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-06-03T03:25:08.977 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-06-03T03:25:08.977 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-06-03T03:25:08.977 DEBUG:teuthology.orchestra.run.smithi117:> ') 2024-06-03T03:25:09.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-03T03:25:09.233 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-03T03:25:09.244 DEBUG:teuthology.orchestra.run.smithi117:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-03T03:25:09.244 DEBUG:teuthology.orchestra.run.smithi117:> set -e 2024-06-03T03:25:09.244 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip link set brx.0 up 2024-06-03T03:25:09.244 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip link set dev brx.0 master ceph-brx 2024-06-03T03:25:09.244 DEBUG:teuthology.orchestra.run.smithi117:> ') 2024-06-03T03:25:09.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-03T03:25:09.430 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-03T03:25:09.439 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-06-03T03:25:09.439 DEBUG:teuthology.orchestra.run.smithi117:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-06-03T03:25:09.482 INFO:teuthology.orchestra.run.smithi117.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-06-03T03:25:09.482 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-06-03T03:25:09.482 DEBUG:teuthology.orchestra.run.smithi117:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-06-03T03:25:09.578 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-06-03T03:25:09.578 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-06-03T03:25:09.578 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/a202463a-d267-4cb0-ba31-bccad7941779 2024-06-03T03:25:09.578 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T03:25:09.578 DEBUG:teuthology.orchestra.run.smithi117:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/a202463a-d267-4cb0-ba31-bccad7941779 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync 2024-06-03T03:25:09.705 INFO:teuthology.orchestra.run.smithi117.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync 2024-06-03T03:25:09.706 INFO:teuthology.orchestra.run.smithi117.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync". 2024-06-03T03:25:09.706 INFO:teuthology.orchestra.run.smithi117.stdout:invalid new device string format 2024-06-03T03:25:10.163 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:25:10 smithi117 ceph-mon[31649]: pgmap v501: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-03T03:25:10.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:25:10 smithi151 ceph-mon[37043]: pgmap v501: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-03T03:25:10.440 INFO:teuthology.orchestra.run.smithi117.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync 2024-06-03T03:25:10.440 INFO:teuthology.orchestra.run.smithi117.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync". 2024-06-03T03:25:10.440 INFO:teuthology.orchestra.run.smithi117.stdout:invalid new device string format 2024-06-03T03:25:10.440 INFO:teuthology.orchestra.run.smithi117.stdout:mount.ceph: resolved to: "172.21.15.117:3300,172.21.15.151:3300,172.21.15.183:3300" 2024-06-03T03:25:10.440 INFO:teuthology.orchestra.run.smithi117.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.117:3300,172.21.15.151:3300,172.21.15.183:3300:/volumes/qa/sv_1/a202463a-d267-4cb0-ba31-bccad7941779 2024-06-03T03:25:10.440 INFO:teuthology.orchestra.run.smithi117.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync,key=0,fsid=a0e8473a-2156-11ef-bc9b-c7b262605968" will pass to kernel 2024-06-03T03:25:10.444 INFO:teuthology.orchestra.run.smithi117.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-06-03T03:25:10.445 INFO:teuthology.orchestra.run.smithi117.stdout: You just mounted a file system that supports labels which does not 2024-06-03T03:25:10.445 INFO:teuthology.orchestra.run.smithi117.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-06-03T03:25:10.445 INFO:teuthology.orchestra.run.smithi117.stdout: applications will generate AVC messages and not be allowed access to 2024-06-03T03:25:10.445 INFO:teuthology.orchestra.run.smithi117.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-06-03T03:25:10.450 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-06-03T03:25:10.450 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T03:25:10.450 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-06-03T03:25:10.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:25:10 smithi183 ceph-mon[37326]: pgmap v501: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-03T03:25:10.537 DEBUG:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-06-03T03:25:10.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:10 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-06-03T03:25:10.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-03T03:25:10.660 INFO:teuthology.orchestra.run.smithi117.stdout:client15072 2024-06-03T03:25:10.660 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-06-03T03:25:10.661 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-03T03:25:10.713 DEBUG:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-06-03T03:25:10.814 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:10 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-06-03T03:25:10.819 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-03T03:25:10.828 INFO:teuthology.orchestra.run.smithi117.stdout:client15072 2024-06-03T03:25:10.828 DEBUG:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-06-03T03:25:10.930 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:10 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-06-03T03:25:10.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-03T03:25:10.944 INFO:teuthology.orchestra.run.smithi117.stdout:client15072 2024-06-03T03:25:10.944 DEBUG:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968.client15072/status') 2024-06-03T03:25:11.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-03T03:25:11.077 INFO:teuthology.orchestra.run.smithi117.stdout:instance: client.15072 (3)192.168.0.1:0/3794197228 2024-06-03T03:25:11.077 INFO:teuthology.orchestra.run.smithi117.stdout:blocklisted: false 2024-06-03T03:25:11.077 INFO:teuthology.orchestra.run.smithi117.stderr:0+1 records in 2024-06-03T03:25:11.077 INFO:teuthology.orchestra.run.smithi117.stderr:0+1 records out 2024-06-03T03:25:11.077 INFO:teuthology.orchestra.run.smithi117.stderr:70 bytes copied, 7.5918e-05 s, 922 kB/s 2024-06-03T03:25:11.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-03T03:25:11.089 INFO:teuthology.run_tasks:Running task exec... 2024-06-03T03:25:11.112 INFO:teuthology.task.exec:Executing custom commands... 2024-06-03T03:25:11.113 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi117.front.sepia.ceph.com 2024-06-03T03:25:11.113 DEBUG:teuthology.orchestra.run.smithi117:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-06-03T03:25:11.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:11.315+0000 7f41bf000640 1 -- 172.21.15.117:0/3793737980 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f41b81057d0 msgr2=0x7f41b810c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:11.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:11.315+0000 7f41bf000640 1 --2- 172.21.15.117:0/3793737980 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f41b81057d0 0x7f41b810c060 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f41b0009860 tx=0x7f41b002a340 comp rx=0 tx=0).stop 2024-06-03T03:25:11.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:11.316+0000 7f41bf000640 1 -- 172.21.15.117:0/3793737980 shutdown_connections 2024-06-03T03:25:11.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:11.316+0000 7f41bf000640 1 --2- 172.21.15.117:0/3793737980 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f41b81057d0 0x7f41b810c060 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:11.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:11.316+0000 7f41bf000640 1 --2- 172.21.15.117:0/3793737980 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f41b8104e10 0x7f41b8105290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:11.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:11.316+0000 7f41bf000640 1 --2- 172.21.15.117:0/3793737980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41b8103c20 0x7f41b8104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:11.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:11.316+0000 7f41bf000640 1 -- 172.21.15.117:0/3793737980 >> 172.21.15.117:0/3793737980 conn(0x7f41b80ff190 msgr2=0x7f41b81015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:25:11.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:11.316+0000 7f41bf000640 1 -- 172.21.15.117:0/3793737980 shutdown_connections 2024-06-03T03:25:11.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:11.316+0000 7f41bf000640 1 -- 172.21.15.117:0/3793737980 wait complete. 2024-06-03T03:25:11.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:11.317+0000 7f41bf000640 1 Processor -- start 2024-06-03T03:25:11.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:11.317+0000 7f41bf000640 1 -- start start 2024-06-03T03:25:11.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:11.317+0000 7f41bf000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41b8103c20 0x7f41b8199e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:11.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:11.317+0000 7f41bf000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f41b8104e10 0x7f41b819a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:11.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:11.317+0000 7f41bf000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f41b81057d0 0x7f41b81a13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:11.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:11.317+0000 7f41bf000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f41b810f8a0 con 0x7f41b8103c20 2024-06-03T03:25:11.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:11.317+0000 7f41bf000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f41b810f720 con 0x7f41b81057d0 2024-06-03T03:25:11.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:11.317+0000 7f41bf000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f41b810fa20 con 0x7f41b8104e10 2024-06-03T03:25:11.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:11.317+0000 7f41b7e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41b8103c20 0x7f41b8199e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:11.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:11.317+0000 7f41b7e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41b8103c20 0x7f41b8199e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:47084/0 (socket says 172.21.15.117:47084) 2024-06-03T03:25:11.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:11.317+0000 7f41bd000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f41b81057d0 0x7f41b81a13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:11.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:11.317+0000 7f41b7e00640 1 -- 172.21.15.117:0/1361871121 learned_addr learned my addr 172.21.15.117:0/1361871121 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:25:11.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:11.317+0000 7f41b7400640 1 --2- 172.21.15.117:0/1361871121 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f41b8104e10 0x7f41b819a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:11.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:11.318+0000 7f41bd000640 1 -- 172.21.15.117:0/1361871121 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f41b8104e10 msgr2=0x7f41b819a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:11.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:11.318+0000 7f41bd000640 1 --2- 172.21.15.117:0/1361871121 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f41b8104e10 0x7f41b819a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:11.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:11.318+0000 7f41bd000640 1 -- 172.21.15.117:0/1361871121 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41b8103c20 msgr2=0x7f41b8199e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:11.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:11.318+0000 7f41bd000640 1 --2- 172.21.15.117:0/1361871121 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41b8103c20 0x7f41b8199e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:11.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:11.318+0000 7f41bd000640 1 -- 172.21.15.117:0/1361871121 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41b81a1a40 con 0x7f41b81057d0 2024-06-03T03:25:11.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:11.318+0000 7f41b7e00640 1 --2- 172.21.15.117:0/1361871121 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41b8103c20 0x7f41b8199e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:25:11.322 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:11.318+0000 7f41bd000640 1 --2- 172.21.15.117:0/1361871121 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f41b81057d0 0x7f41b81a13d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f41b002a930 tx=0x7f41b0033d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:25:11.322 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:11.318+0000 7f41b4c00640 1 -- 172.21.15.117:0/1361871121 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b0037040 con 0x7f41b81057d0 2024-06-03T03:25:11.322 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:11.318+0000 7f41b4c00640 1 -- 172.21.15.117:0/1361871121 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41b0040070 con 0x7f41b81057d0 2024-06-03T03:25:11.322 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:11.318+0000 7f41bf000640 1 -- 172.21.15.117:0/1361871121 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41b81a1cd0 con 0x7f41b81057d0 2024-06-03T03:25:11.322 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:11.318+0000 7f41b4c00640 1 -- 172.21.15.117:0/1361871121 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b00046c0 con 0x7f41b81057d0 2024-06-03T03:25:11.322 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:11.319+0000 7f41bf000640 1 -- 172.21.15.117:0/1361871121 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41b81a2190 con 0x7f41b81057d0 2024-06-03T03:25:11.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:11.320+0000 7f41bf000640 1 -- 172.21.15.117:0/1361871121 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41b810aee0 con 0x7f41b81057d0 2024-06-03T03:25:11.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:11.321+0000 7f41b4c00640 1 -- 172.21.15.117:0/1361871121 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f41b0004040 con 0x7f41b81057d0 2024-06-03T03:25:11.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:11.323+0000 7f41b4c00640 1 --2- 172.21.15.117:0/1361871121 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f41980a2cb0 0x7f41980a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:11.324 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:11.323+0000 7f41b4c00640 1 -- 172.21.15.117:0/1361871121 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f41b00ef3b0 con 0x7f41b81057d0 2024-06-03T03:25:11.324 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:11.323+0000 7f41b7e00640 1 --2- 172.21.15.117:0/1361871121 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f41980a2cb0 0x7f41980a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:11.324 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:11.324+0000 7f41b7e00640 1 --2- 172.21.15.117:0/1361871121 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f41980a2cb0 0x7f41980a5170 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f41a8011490 tx=0x7f41a8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:25:11.328 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:11.327+0000 7f41b4c00640 1 -- 172.21.15.117:0/1361871121 <== mon.2 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f41b00bbf20 con 0x7f41b81057d0 2024-06-03T03:25:11.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:11.574+0000 7f41bf000640 1 -- 172.21.15.117:0/1361871121 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f41b80625a0 con 0x7f41b81057d0 2024-06-03T03:25:12.030 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.030+0000 7f41b4c00640 1 -- 172.21.15.117:0/1361871121 <== mon.2 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v31) ==== 118+0+0 (secure 0 0 0) 0x7f41b00bb820 con 0x7f41b81057d0 2024-06-03T03:25:12.036 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.035+0000 7f41bf000640 1 -- 172.21.15.117:0/1361871121 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f41980a2cb0 msgr2=0x7f41980a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:12.036 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.035+0000 7f41bf000640 1 --2- 172.21.15.117:0/1361871121 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f41980a2cb0 0x7f41980a5170 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f41a8011490 tx=0x7f41a8009260 comp rx=0 tx=0).stop 2024-06-03T03:25:12.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.035+0000 7f41bf000640 1 -- 172.21.15.117:0/1361871121 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f41b81057d0 msgr2=0x7f41b81a13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:12.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.035+0000 7f41bf000640 1 --2- 172.21.15.117:0/1361871121 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f41b81057d0 0x7f41b81a13d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f41b002a930 tx=0x7f41b0033d90 comp rx=0 tx=0).stop 2024-06-03T03:25:12.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.035+0000 7f41bf000640 1 -- 172.21.15.117:0/1361871121 shutdown_connections 2024-06-03T03:25:12.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.035+0000 7f41bf000640 1 --2- 172.21.15.117:0/1361871121 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f41980a2cb0 0x7f41980a5170 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:12.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.035+0000 7f41bf000640 1 --2- 172.21.15.117:0/1361871121 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f41b81057d0 0x7f41b81a13d0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:12.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.035+0000 7f41bf000640 1 --2- 172.21.15.117:0/1361871121 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f41b8104e10 0x7f41b819a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:12.038 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.035+0000 7f41bf000640 1 --2- 172.21.15.117:0/1361871121 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41b8103c20 0x7f41b8199e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:12.038 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.035+0000 7f41bf000640 1 -- 172.21.15.117:0/1361871121 >> 172.21.15.117:0/1361871121 conn(0x7f41b80ff190 msgr2=0x7f41b81080b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:25:12.038 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.035+0000 7f41bf000640 1 -- 172.21.15.117:0/1361871121 shutdown_connections 2024-06-03T03:25:12.038 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.036+0000 7f41bf000640 1 -- 172.21.15.117:0/1361871121 wait complete. 2024-06-03T03:25:12.058 DEBUG:teuthology.orchestra.run.smithi117:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-06-03T03:25:12.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.260+0000 7f502b000640 1 -- 172.21.15.117:0/312977715 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5024108fa0 msgr2=0x7f5024109420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:12.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.260+0000 7f502b000640 1 --2- 172.21.15.117:0/312977715 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5024108fa0 0x7f5024109420 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5014009900 tx=0x7f501402a410 comp rx=0 tx=0).stop 2024-06-03T03:25:12.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.260+0000 7f502b000640 1 -- 172.21.15.117:0/312977715 shutdown_connections 2024-06-03T03:25:12.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.260+0000 7f502b000640 1 --2- 172.21.15.117:0/312977715 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5024109960 0x7f5024110280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:12.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.260+0000 7f502b000640 1 --2- 172.21.15.117:0/312977715 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5024108fa0 0x7f5024109420 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:12.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.260+0000 7f502b000640 1 --2- 172.21.15.117:0/312977715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5024107e50 0x7f5024108270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:12.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.260+0000 7f502b000640 1 -- 172.21.15.117:0/312977715 >> 172.21.15.117:0/312977715 conn(0x7f5024076e30 msgr2=0x7f5024079250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:25:12.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.260+0000 7f502b000640 1 -- 172.21.15.117:0/312977715 shutdown_connections 2024-06-03T03:25:12.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.260+0000 7f502b000640 1 -- 172.21.15.117:0/312977715 wait complete. 2024-06-03T03:25:12.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.261+0000 7f502b000640 1 Processor -- start 2024-06-03T03:25:12.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.261+0000 7f502b000640 1 -- start start 2024-06-03T03:25:12.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.262+0000 7f502b000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5024107e50 0x7f502419e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:12.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.262+0000 7f502b000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5024108fa0 0x7f502419e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:12.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.262+0000 7f502b000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5024109960 0x7f50241a56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:12.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.262+0000 7f502b000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5024113b20 con 0x7f5024108fa0 2024-06-03T03:25:12.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.262+0000 7f502b000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f50241139a0 con 0x7f5024107e50 2024-06-03T03:25:12.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.262+0000 7f502b000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5024113ca0 con 0x7f5024109960 2024-06-03T03:25:12.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.262+0000 7f5029200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5024108fa0 0x7f502419e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:12.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.262+0000 7f5029c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5024107e50 0x7f502419e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:12.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.262+0000 7f502a600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5024109960 0x7f50241a56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:12.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.262+0000 7f5029c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5024107e50 0x7f502419e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:35556/0 (socket says 172.21.15.117:35556) 2024-06-03T03:25:12.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.262+0000 7f5029c00640 1 -- 172.21.15.117:0/2919656708 learned_addr learned my addr 172.21.15.117:0/2919656708 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:25:12.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.263+0000 7f5029c00640 1 -- 172.21.15.117:0/2919656708 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5024109960 msgr2=0x7f50241a56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:12.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.263+0000 7f5029c00640 1 --2- 172.21.15.117:0/2919656708 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5024109960 0x7f50241a56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:12.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.263+0000 7f5029c00640 1 -- 172.21.15.117:0/2919656708 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5024108fa0 msgr2=0x7f502419e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:12.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.263+0000 7f5029c00640 1 --2- 172.21.15.117:0/2919656708 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5024108fa0 0x7f502419e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:12.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.263+0000 7f5029c00640 1 -- 172.21.15.117:0/2919656708 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50241a5d30 con 0x7f5024107e50 2024-06-03T03:25:12.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.263+0000 7f502a600640 1 --2- 172.21.15.117:0/2919656708 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5024109960 0x7f50241a56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:25:12.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.263+0000 7f5029c00640 1 --2- 172.21.15.117:0/2919656708 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5024107e50 0x7f502419e110 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f501800e9f0 tx=0x7f501800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:25:12.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.263+0000 7f5022000640 1 -- 172.21.15.117:0/2919656708 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f501800cd80 con 0x7f5024107e50 2024-06-03T03:25:12.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.263+0000 7f5022000640 1 -- 172.21.15.117:0/2919656708 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5018010910 con 0x7f5024107e50 2024-06-03T03:25:12.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.263+0000 7f5022000640 1 -- 172.21.15.117:0/2919656708 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5018018990 con 0x7f5024107e50 2024-06-03T03:25:12.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.263+0000 7f502b000640 1 -- 172.21.15.117:0/2919656708 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50241a5fc0 con 0x7f5024107e50 2024-06-03T03:25:12.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.263+0000 7f502b000640 1 -- 172.21.15.117:0/2919656708 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50241a64d0 con 0x7f5024107e50 2024-06-03T03:25:12.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.265+0000 7f502b000640 1 -- 172.21.15.117:0/2919656708 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f502410f160 con 0x7f5024107e50 2024-06-03T03:25:12.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.267+0000 7f5022000640 1 -- 172.21.15.117:0/2919656708 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f5018010ac0 con 0x7f5024107e50 2024-06-03T03:25:12.268 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.268+0000 7f5022000640 1 --2- 172.21.15.117:0/2919656708 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f500c0a2c00 0x7f500c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:12.268 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.268+0000 7f5022000640 1 -- 172.21.15.117:0/2919656708 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f50180cf6a0 con 0x7f5024107e50 2024-06-03T03:25:12.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.268+0000 7f5029200640 1 --2- 172.21.15.117:0/2919656708 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f500c0a2c00 0x7f500c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:12.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.269+0000 7f5029200640 1 --2- 172.21.15.117:0/2919656708 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f500c0a2c00 0x7f500c0a50c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f501402aa00 tx=0x7f50140315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:25:12.272 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.272+0000 7f5022000640 1 -- 172.21.15.117:0/2919656708 <== mon.2 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f501809bbc0 con 0x7f5024107e50 2024-06-03T03:25:12.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:25:12 smithi151 ceph-mon[37043]: pgmap v502: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-06-03T03:25:12.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:25:12 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/1361871121' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-06-03T03:25:12.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:25:12 smithi151 ceph-mon[37043]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-06-03T03:25:12.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:25:12 smithi117 ceph-mon[31649]: pgmap v502: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-06-03T03:25:12.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:25:12 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/1361871121' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-06-03T03:25:12.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:25:12 smithi117 ceph-mon[31649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-06-03T03:25:12.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:25:12 smithi183 ceph-mon[37326]: pgmap v502: 129 pgs: 129 active+clean; 651 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-06-03T03:25:12.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:25:12 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/1361871121' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-06-03T03:25:12.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:25:12 smithi183 ceph-mon[37326]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-06-03T03:25:12.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.500+0000 7f502b000640 1 -- 172.21.15.117:0/2919656708 --> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f5024061f40 con 0x7f500c0a2c00 2024-06-03T03:25:12.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.536+0000 7f5022000640 1 -- 172.21.15.117:0/2919656708 <== mgr.14152 v2:172.21.15.117:6800/1398160887 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f5024061f40 con 0x7f500c0a2c00 2024-06-03T03:25:12.536 INFO:teuthology.orchestra.run.smithi117.stdout:{} 2024-06-03T03:25:12.539 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.538+0000 7f502b000640 1 -- 172.21.15.117:0/2919656708 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f500c0a2c00 msgr2=0x7f500c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:12.539 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.538+0000 7f502b000640 1 --2- 172.21.15.117:0/2919656708 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f500c0a2c00 0x7f500c0a50c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f501402aa00 tx=0x7f50140315b0 comp rx=0 tx=0).stop 2024-06-03T03:25:12.540 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.538+0000 7f502b000640 1 -- 172.21.15.117:0/2919656708 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5024107e50 msgr2=0x7f502419e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:12.540 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.538+0000 7f502b000640 1 --2- 172.21.15.117:0/2919656708 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5024107e50 0x7f502419e110 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f501800e9f0 tx=0x7f501800eeb0 comp rx=0 tx=0).stop 2024-06-03T03:25:12.540 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.539+0000 7f502b000640 1 -- 172.21.15.117:0/2919656708 shutdown_connections 2024-06-03T03:25:12.540 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.539+0000 7f502b000640 1 --2- 172.21.15.117:0/2919656708 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5024109960 0x7f50241a56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:12.540 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.539+0000 7f502b000640 1 --2- 172.21.15.117:0/2919656708 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5024108fa0 0x7f502419e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:12.540 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.539+0000 7f502b000640 1 --2- 172.21.15.117:0/2919656708 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f500c0a2c00 0x7f500c0a50c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:12.540 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.539+0000 7f502b000640 1 --2- 172.21.15.117:0/2919656708 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5024107e50 0x7f502419e110 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:12.540 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.539+0000 7f502b000640 1 -- 172.21.15.117:0/2919656708 >> 172.21.15.117:0/2919656708 conn(0x7f5024076e30 msgr2=0x7f502410c330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:25:12.540 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.539+0000 7f502b000640 1 -- 172.21.15.117:0/2919656708 shutdown_connections 2024-06-03T03:25:12.540 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.539+0000 7f502b000640 1 -- 172.21.15.117:0/2919656708 wait complete. 2024-06-03T03:25:12.557 INFO:teuthology.run_tasks:Running task check-counter... 2024-06-03T03:25:12.568 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-06-03T03:25:12.582 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-06-03T03:25:12.776 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.775+0000 7f1e57e00640 1 -- 172.21.15.117:0/684698746 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e58101840 msgr2=0x7f1e58101c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:12.776 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.775+0000 7f1e57e00640 1 --2- 172.21.15.117:0/684698746 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e58101840 0x7f1e58101c60 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1e48009ca0 tx=0x7f1e4802a8c0 comp rx=0 tx=0).stop 2024-06-03T03:25:12.776 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.775+0000 7f1e57e00640 1 -- 172.21.15.117:0/684698746 shutdown_connections 2024-06-03T03:25:12.776 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.775+0000 7f1e57e00640 1 --2- 172.21.15.117:0/684698746 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e581033f0 0x7f1e58109c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:12.777 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.775+0000 7f1e57e00640 1 --2- 172.21.15.117:0/684698746 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e58102a30 0x7f1e58102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:12.777 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.775+0000 7f1e57e00640 1 --2- 172.21.15.117:0/684698746 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e58101840 0x7f1e58101c60 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:12.777 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.775+0000 7f1e57e00640 1 -- 172.21.15.117:0/684698746 >> 172.21.15.117:0/684698746 conn(0x7f1e580fcdb0 msgr2=0x7f1e580ff1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:25:12.777 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.775+0000 7f1e57e00640 1 -- 172.21.15.117:0/684698746 shutdown_connections 2024-06-03T03:25:12.777 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.775+0000 7f1e57e00640 1 -- 172.21.15.117:0/684698746 wait complete. 2024-06-03T03:25:12.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.776+0000 7f1e57e00640 1 Processor -- start 2024-06-03T03:25:12.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.776+0000 7f1e57e00640 1 -- start start 2024-06-03T03:25:12.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.777+0000 7f1e57e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e58101840 0x7f1e58199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:12.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.777+0000 7f1e57e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e58102a30 0x7f1e5819a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:12.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.777+0000 7f1e57e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e581033f0 0x7f1e581a1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:12.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.777+0000 7f1e57e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1e5810d4c0 con 0x7f1e581033f0 2024-06-03T03:25:12.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.777+0000 7f1e57e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1e5810d340 con 0x7f1e58101840 2024-06-03T03:25:12.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.777+0000 7f1e57e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1e5810d640 con 0x7f1e58102a30 2024-06-03T03:25:12.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.777+0000 7f1e57400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e581033f0 0x7f1e581a1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:12.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.777+0000 7f1e57400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e581033f0 0x7f1e581a1300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:47146/0 (socket says 172.21.15.117:47146) 2024-06-03T03:25:12.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.777+0000 7f1e57400640 1 -- 172.21.15.117:0/4293658852 learned_addr learned my addr 172.21.15.117:0/4293658852 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:25:12.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.777+0000 7f1e4e000640 1 --2- 172.21.15.117:0/4293658852 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e58102a30 0x7f1e5819a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:12.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.777+0000 7f1e56a00640 1 --2- 172.21.15.117:0/4293658852 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e58101840 0x7f1e58199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:12.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.777+0000 7f1e56a00640 1 -- 172.21.15.117:0/4293658852 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e58102a30 msgr2=0x7f1e5819a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:12.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.777+0000 7f1e56a00640 1 --2- 172.21.15.117:0/4293658852 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e58102a30 0x7f1e5819a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:12.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.777+0000 7f1e56a00640 1 -- 172.21.15.117:0/4293658852 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e581033f0 msgr2=0x7f1e581a1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:12.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.777+0000 7f1e56a00640 1 --2- 172.21.15.117:0/4293658852 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e581033f0 0x7f1e581a1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:12.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.777+0000 7f1e56a00640 1 -- 172.21.15.117:0/4293658852 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e581a1970 con 0x7f1e58101840 2024-06-03T03:25:12.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.778+0000 7f1e57400640 1 --2- 172.21.15.117:0/4293658852 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e581033f0 0x7f1e581a1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:25:12.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.778+0000 7f1e56a00640 1 --2- 172.21.15.117:0/4293658852 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e58101840 0x7f1e58199d50 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1e48009bf0 tx=0x7f1e48031ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:25:12.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.778+0000 7f1e4fe00640 1 -- 172.21.15.117:0/4293658852 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e4802fa20 con 0x7f1e58101840 2024-06-03T03:25:12.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.778+0000 7f1e57e00640 1 -- 172.21.15.117:0/4293658852 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e581a1c00 con 0x7f1e58101840 2024-06-03T03:25:12.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.778+0000 7f1e4fe00640 1 -- 172.21.15.117:0/4293658852 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e4802fbc0 con 0x7f1e58101840 2024-06-03T03:25:12.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.778+0000 7f1e4fe00640 1 -- 172.21.15.117:0/4293658852 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e48046750 con 0x7f1e58101840 2024-06-03T03:25:12.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.778+0000 7f1e57e00640 1 -- 172.21.15.117:0/4293658852 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e581a2150 con 0x7f1e58101840 2024-06-03T03:25:12.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.779+0000 7f1e57e00640 1 -- 172.21.15.117:0/4293658852 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e581a2320 con 0x7f1e58101840 2024-06-03T03:25:12.782 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.781+0000 7f1e4fe00640 1 -- 172.21.15.117:0/4293658852 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f1e4802fd70 con 0x7f1e58101840 2024-06-03T03:25:12.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.783+0000 7f1e4fe00640 1 --2- 172.21.15.117:0/4293658852 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f1e3c0a2b90 0x7f1e3c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:12.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.783+0000 7f1e4fe00640 1 -- 172.21.15.117:0/4293658852 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f1e480f0a40 con 0x7f1e58101840 2024-06-03T03:25:12.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.783+0000 7f1e4e000640 1 --2- 172.21.15.117:0/4293658852 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f1e3c0a2b90 0x7f1e3c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:12.784 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.784+0000 7f1e4e000640 1 --2- 172.21.15.117:0/4293658852 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f1e3c0a2b90 0x7f1e3c0a5050 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1e400095f0 tx=0x7f1e40005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:25:12.785 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.785+0000 7f1e4fe00640 1 -- 172.21.15.117:0/4293658852 <== mon.2 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1e48081850 con 0x7f1e58101840 2024-06-03T03:25:12.947 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:25:12 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:25:12.512+0000 7fbb97600700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-03T03:25:12.947 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:25:12 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:25:12.512+0000 7fbb97600700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-03T03:25:12.947 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:25:12 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:25:12.512+0000 7fbb97600700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-03T03:25:12.947 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:25:12 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:25:12.512+0000 7fbb97600700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-03T03:25:12.947 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:25:12 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:25:12.512+0000 7fbb97600700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-03T03:25:12.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.988+0000 7f1e57e00640 1 -- 172.21.15.117:0/4293658852 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1e581a2610 con 0x7f1e58101840 2024-06-03T03:25:12.991 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.991+0000 7f1e4fe00640 1 -- 172.21.15.117:0/4293658852 <== mon.2 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+24807 (secure 0 0 0) 0x7f1e480bd5b0 con 0x7f1e58101840 2024-06-03T03:25:12.992 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T03:25:12.992 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":99,"fsid":"a0e8473a-2156-11ef-bc9b-c7b262605968","created":"2024-06-03T03:10:08.446859+0000","modified":"2024-06-03T03:21:25.966405+0000","last_up_change":"2024-06-03T03:18:28.736904+0000","last_in_change":"2024-06-03T03:18:08.485711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-03T03:14:09.370986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-03T03:19:35.044683+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-03T03:19:40.097569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-03T03:19:55.394480+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8fefa089-9401-4a1c-b9fb-fe30c63e6a1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":4286644492},{"type":"v1","addr":"172.21.15.117:6803","nonce":4286644492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":4286644492},{"type":"v1","addr":"172.21.15.117:6805","nonce":4286644492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":4286644492},{"type":"v1","addr":"172.21.15.117:6809","nonce":4286644492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":4286644492},{"type":"v1","addr":"172.21.15.117:6807","nonce":4286644492}]},"public_addr":"172.21.15.117:6803/4286644492","cluster_addr":"172.21.15.117:6805/4286644492","heartbeat_back_addr":"172.21.15.117:6809/4286644492","heartbeat_front_addr":"172.21.15.117:6807/4286644492","state":["exists","up"]},{"osd":1,"uuid":"5b12b371-f155-4b67-a470-25cbe59a2999","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":262491158},{"type":"v1","addr":"172.21.15.151:6801","nonce":262491158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":262491158},{"type":"v1","addr":"172.21.15.151:6803","nonce":262491158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":262491158},{"type":"v1","addr":"172.21.15.151:6807","nonce":262491158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":262491158},{"type":"v1","addr":"172.21.15.151:6805","nonce":262491158}]},"public_addr":"172.21.15.151:6801/262491158","cluster_addr":"172.21.15.151:6803/262491158","heartbeat_back_addr":"172.21.15.151:6807/262491158","heartbeat_front_addr":"172.21.15.151:6805/262491158","state":["exists","up"]},{"osd":2,"uuid":"dbc17818-c54c-4ecf-b656-03578b95d8c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":3542453623},{"type":"v1","addr":"172.21.15.183:6801","nonce":3542453623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":3542453623},{"type":"v1","addr":"172.21.15.183:6803","nonce":3542453623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":3542453623},{"type":"v1","addr":"172.21.15.183:6807","nonce":3542453623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":3542453623},{"type":"v1","addr":"172.21.15.183:6805","nonce":3542453623}]},"public_addr":"172.21.15.183:6801/3542453623","cluster_addr":"172.21.15.183:6803/3542453623","heartbeat_back_addr":"172.21.15.183:6807/3542453623","heartbeat_front_addr":"172.21.15.183:6805/3542453623","state":["exists","up"]},{"osd":3,"uuid":"ca08d2ac-b1d6-4a49-9ae8-e3ec146899a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":589012315},{"type":"v1","addr":"172.21.15.117:6811","nonce":589012315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":589012315},{"type":"v1","addr":"172.21.15.117:6813","nonce":589012315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":589012315},{"type":"v1","addr":"172.21.15.117:6817","nonce":589012315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":589012315},{"type":"v1","addr":"172.21.15.117:6815","nonce":589012315}]},"public_addr":"172.21.15.117:6811/589012315","cluster_addr":"172.21.15.117:6813/589012315","heartbeat_back_addr":"172.21.15.117:6817/589012315","heartbeat_front_addr":"172.21.15.117:6815/589012315","state":["exists","up"]},{"osd":4,"uuid":"810d6c7a-023f-4930-82af-bc2bbfd647da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":1239048647},{"type":"v1","addr":"172.21.15.151:6809","nonce":1239048647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":1239048647},{"type":"v1","addr":"172.21.15.151:6811","nonce":1239048647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":1239048647},{"type":"v1","addr":"172.21.15.151:6815","nonce":1239048647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":1239048647},{"type":"v1","addr":"172.21.15.151:6813","nonce":1239048647}]},"public_addr":"172.21.15.151:6809/1239048647","cluster_addr":"172.21.15.151:6811/1239048647","heartbeat_back_addr":"172.21.15.151:6815/1239048647","heartbeat_front_addr":"172.21.15.151:6813/1239048647","state":["exists","up"]},{"osd":5,"uuid":"8ab6a5f0-34bb-48d8-9471-a7dc836e486e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":2985874686},{"type":"v1","addr":"172.21.15.183:6809","nonce":2985874686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":2985874686},{"type":"v1","addr":"172.21.15.183:6811","nonce":2985874686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":2985874686},{"type":"v1","addr":"172.21.15.183:6815","nonce":2985874686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":2985874686},{"type":"v1","addr":"172.21.15.183:6813","nonce":2985874686}]},"public_addr":"172.21.15.183:6809/2985874686","cluster_addr":"172.21.15.183:6811/2985874686","heartbeat_back_addr":"172.21.15.183:6815/2985874686","heartbeat_front_addr":"172.21.15.183:6813/2985874686","state":["exists","up"]},{"osd":6,"uuid":"78d15739-e9e0-4f19-ae5f-ee8527a5b527","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4250583193},{"type":"v1","addr":"172.21.15.117:6819","nonce":4250583193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":4250583193},{"type":"v1","addr":"172.21.15.117:6821","nonce":4250583193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":4250583193},{"type":"v1","addr":"172.21.15.117:6825","nonce":4250583193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4250583193},{"type":"v1","addr":"172.21.15.117:6823","nonce":4250583193}]},"public_addr":"172.21.15.117:6819/4250583193","cluster_addr":"172.21.15.117:6821/4250583193","heartbeat_back_addr":"172.21.15.117:6825/4250583193","heartbeat_front_addr":"172.21.15.117:6823/4250583193","state":["exists","up"]},{"osd":7,"uuid":"3724fe1f-f4d7-443b-aea8-ff3ab02bfdc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":386094645},{"type":"v1","addr":"172.21.15.151:6817","nonce":386094645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":386094645},{"type":"v1","addr":"172.21.15.151:6819","nonce":386094645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":386094645},{"type":"v1","addr":"172.21.15.151:6823","nonce":386094645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":386094645},{"type":"v1","addr":"172.21.15.151:6821","nonce":386094645}]},"public_addr":"172.21.15.151:6817/386094645","cluster_addr":"172.21.15.151:6819/386094645","heartbeat_back_addr":"172.21.15.151:6823/386094645","heartbeat_front_addr":"172.21.15.151:6821/386094645","state":["exists","up"]},{"osd":8,"uuid":"006ac092-f015-448e-a0c0-f7b0c4597a18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1829748222},{"type":"v1","addr":"172.21.15.183:6817","nonce":1829748222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1829748222},{"type":"v1","addr":"172.21.15.183:6819","nonce":1829748222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1829748222},{"type":"v1","addr":"172.21.15.183:6823","nonce":1829748222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1829748222},{"type":"v1","addr":"172.21.15.183:6821","nonce":1829748222}]},"public_addr":"172.21.15.183:6817/1829748222","cluster_addr":"172.21.15.183:6819/1829748222","heartbeat_back_addr":"172.21.15.183:6823/1829748222","heartbeat_front_addr":"172.21.15.183:6821/1829748222","state":["exists","up"]},{"osd":9,"uuid":"00a970e6-2502-459e-975c-badac93c9bd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2538315141},{"type":"v1","addr":"172.21.15.117:6827","nonce":2538315141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2538315141},{"type":"v1","addr":"172.21.15.117:6829","nonce":2538315141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2538315141},{"type":"v1","addr":"172.21.15.117:6833","nonce":2538315141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2538315141},{"type":"v1","addr":"172.21.15.117:6831","nonce":2538315141}]},"public_addr":"172.21.15.117:6827/2538315141","cluster_addr":"172.21.15.117:6829/2538315141","heartbeat_back_addr":"172.21.15.117:6833/2538315141","heartbeat_front_addr":"172.21.15.117:6831/2538315141","state":["exists","up"]},{"osd":10,"uuid":"ba2333cd-32a3-41ef-9ea5-287c707f512b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":3801045133},{"type":"v1","addr":"172.21.15.151:6825","nonce":3801045133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3801045133},{"type":"v1","addr":"172.21.15.151:6827","nonce":3801045133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3801045133},{"type":"v1","addr":"172.21.15.151:6831","nonce":3801045133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3801045133},{"type":"v1","addr":"172.21.15.151:6829","nonce":3801045133}]},"public_addr":"172.21.15.151:6825/3801045133","cluster_addr":"172.21.15.151:6827/3801045133","heartbeat_back_addr":"172.21.15.151:6831/3801045133","heartbeat_front_addr":"172.21.15.151:6829/3801045133","state":["exists","up"]},{"osd":11,"uuid":"1556d654-e6fa-42c1-b399-aa53f6837a1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3513899696},{"type":"v1","addr":"172.21.15.183:6825","nonce":3513899696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3513899696},{"type":"v1","addr":"172.21.15.183:6827","nonce":3513899696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3513899696},{"type":"v1","addr":"172.21.15.183:6831","nonce":3513899696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3513899696},{"type":"v1","addr":"172.21.15.183:6829","nonce":3513899696}]},"public_addr":"172.21.15.183:6825/3513899696","cluster_addr":"172.21.15.183:6827/3513899696","heartbeat_back_addr":"172.21.15.183:6831/3513899696","heartbeat_front_addr":"172.21.15.183:6829/3513899696","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:13:17.174041+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:13:42.237001+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:14:06.273315+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:14:39.912564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:15:06.063202+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:15:30.424818+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:16:05.623554+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:16:32.926393+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:16:58.165899+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:17:32.897477+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:18:00.720866+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:18:26.474441+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.117:0/186309008":"2024-06-04T03:11:24.655516+0000","172.21.15.117:6801/2142313590":"2024-06-04T03:11:24.655516+0000","172.21.15.117:6800/2142313590":"2024-06-04T03:11:24.655516+0000","172.21.15.117:0/2399725156":"2024-06-04T03:11:24.655516+0000","172.21.15.117:0/1453232061":"2024-06-04T03:10:50.552899+0000","172.21.15.117:0/4281674974":"2024-06-04T03:10:50.552899+0000","172.21.15.117:6800/2424554985":"2024-06-04T03:10:50.552899+0000","172.21.15.117:0/3946365225":"2024-06-04T03:11:24.655516+0000","172.21.15.117:0/1146162354":"2024-06-04T03:10:50.552899+0000","172.21.15.117:6801/2424554985":"2024-06-04T03:10:50.552899+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-03T03:25:12.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.994+0000 7f1e57e00640 1 -- 172.21.15.117:0/4293658852 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f1e3c0a2b90 msgr2=0x7f1e3c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:12.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.994+0000 7f1e57e00640 1 --2- 172.21.15.117:0/4293658852 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f1e3c0a2b90 0x7f1e3c0a5050 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1e400095f0 tx=0x7f1e40005f90 comp rx=0 tx=0).stop 2024-06-03T03:25:12.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.994+0000 7f1e57e00640 1 -- 172.21.15.117:0/4293658852 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e58101840 msgr2=0x7f1e58199d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:12.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.994+0000 7f1e57e00640 1 --2- 172.21.15.117:0/4293658852 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e58101840 0x7f1e58199d50 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1e48009bf0 tx=0x7f1e48031ee0 comp rx=0 tx=0).stop 2024-06-03T03:25:12.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.994+0000 7f1e57e00640 1 -- 172.21.15.117:0/4293658852 shutdown_connections 2024-06-03T03:25:12.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.994+0000 7f1e57e00640 1 --2- 172.21.15.117:0/4293658852 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e581033f0 0x7f1e581a1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:12.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.994+0000 7f1e57e00640 1 --2- 172.21.15.117:0/4293658852 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e58102a30 0x7f1e5819a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:12.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.994+0000 7f1e57e00640 1 --2- 172.21.15.117:0/4293658852 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f1e3c0a2b90 0x7f1e3c0a5050 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:12.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.994+0000 7f1e57e00640 1 --2- 172.21.15.117:0/4293658852 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e58101840 0x7f1e58199d50 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:12.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.994+0000 7f1e57e00640 1 -- 172.21.15.117:0/4293658852 >> 172.21.15.117:0/4293658852 conn(0x7f1e580fcdb0 msgr2=0x7f1e58073bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:25:12.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.994+0000 7f1e57e00640 1 -- 172.21.15.117:0/4293658852 shutdown_connections 2024-06-03T03:25:12.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:12.994+0000 7f1e57e00640 1 -- 172.21.15.117:0/4293658852 wait complete. 2024-06-03T03:25:13.013 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-06-03T03:14:09.370986+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-06-03T03:19:35.044683+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-06-03T03:19:40.097569+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-06-03T03:19:55.394480+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-06-03T03:25:13.013 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-06-03T03:25:13.219 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.218+0000 7f8ef6600640 1 -- 172.21.15.117:0/2263595715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ef0103c00 msgr2=0x7f8ef0104020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:13.219 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.218+0000 7f8ef6600640 1 --2- 172.21.15.117:0/2263595715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ef0103c00 0x7f8ef0104020 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0009900 tx=0x7f8ee002a3f0 comp rx=0 tx=0).stop 2024-06-03T03:25:13.219 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.218+0000 7f8ef6600640 1 -- 172.21.15.117:0/2263595715 shutdown_connections 2024-06-03T03:25:13.219 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.218+0000 7f8ef6600640 1 --2- 172.21.15.117:0/2263595715 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ef01057b0 0x7f8ef010c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:13.219 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.218+0000 7f8ef6600640 1 --2- 172.21.15.117:0/2263595715 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ef0104df0 0x7f8ef0105270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:13.219 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.218+0000 7f8ef6600640 1 --2- 172.21.15.117:0/2263595715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ef0103c00 0x7f8ef0104020 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:13.219 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.218+0000 7f8ef6600640 1 -- 172.21.15.117:0/2263595715 >> 172.21.15.117:0/2263595715 conn(0x7f8ef00ff190 msgr2=0x7f8ef01015b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:25:13.219 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.218+0000 7f8ef6600640 1 -- 172.21.15.117:0/2263595715 shutdown_connections 2024-06-03T03:25:13.220 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.218+0000 7f8ef6600640 1 -- 172.21.15.117:0/2263595715 wait complete. 2024-06-03T03:25:13.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.219+0000 7f8ef6600640 1 Processor -- start 2024-06-03T03:25:13.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.219+0000 7f8ef6600640 1 -- start start 2024-06-03T03:25:13.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.220+0000 7f8ef6600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ef0103c00 0x7f8ef0199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:13.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.220+0000 7f8ef6600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ef0104df0 0x7f8ef019a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:13.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.220+0000 7f8ef6600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ef01057b0 0x7f8ef01a14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:13.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.220+0000 7f8ef6600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef010f8a0 con 0x7f8ef0103c00 2024-06-03T03:25:13.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.220+0000 7f8ef6600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef010f720 con 0x7f8ef01057b0 2024-06-03T03:25:13.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.220+0000 7f8ef6600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef010fa20 con 0x7f8ef0104df0 2024-06-03T03:25:13.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.220+0000 7f8eef400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ef0103c00 0x7f8ef0199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:13.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.220+0000 7f8eef400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ef0103c00 0x7f8ef0199f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:47170/0 (socket says 172.21.15.117:47170) 2024-06-03T03:25:13.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.220+0000 7f8eef400640 1 -- 172.21.15.117:0/2138503705 learned_addr learned my addr 172.21.15.117:0/2138503705 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:25:13.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.220+0000 7f8eefe00640 1 --2- 172.21.15.117:0/2138503705 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ef01057b0 0x7f8ef01a14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:13.224 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.220+0000 7f8ee7e00640 1 --2- 172.21.15.117:0/2138503705 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ef0104df0 0x7f8ef019a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:13.224 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.220+0000 7f8eef400640 1 -- 172.21.15.117:0/2138503705 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ef0104df0 msgr2=0x7f8ef019a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:13.224 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.220+0000 7f8eef400640 1 --2- 172.21.15.117:0/2138503705 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ef0104df0 0x7f8ef019a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:13.224 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.220+0000 7f8eef400640 1 -- 172.21.15.117:0/2138503705 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ef01057b0 msgr2=0x7f8ef01a14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:13.224 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.220+0000 7f8eef400640 1 --2- 172.21.15.117:0/2138503705 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ef01057b0 0x7f8ef01a14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:13.224 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.220+0000 7f8eef400640 1 -- 172.21.15.117:0/2138503705 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ef01a1b60 con 0x7f8ef0103c00 2024-06-03T03:25:13.225 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.221+0000 7f8eef400640 1 --2- 172.21.15.117:0/2138503705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ef0103c00 0x7f8ef0199f40 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0031440 tx=0x7f8ee0032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:25:13.225 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.221+0000 7f8eecc00640 1 -- 172.21.15.117:0/2138503705 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee003f030 con 0x7f8ef0103c00 2024-06-03T03:25:13.225 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.221+0000 7f8eecc00640 1 -- 172.21.15.117:0/2138503705 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ee0003ce0 con 0x7f8ef0103c00 2024-06-03T03:25:13.225 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.221+0000 7f8eecc00640 1 -- 172.21.15.117:0/2138503705 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee0035070 con 0x7f8ef0103c00 2024-06-03T03:25:13.225 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.221+0000 7f8ef6600640 1 -- 172.21.15.117:0/2138503705 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ef01a1df0 con 0x7f8ef0103c00 2024-06-03T03:25:13.225 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.221+0000 7f8ef6600640 1 -- 172.21.15.117:0/2138503705 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ef01a21d0 con 0x7f8ef0103c00 2024-06-03T03:25:13.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.222+0000 7f8ef6600640 1 -- 172.21.15.117:0/2138503705 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ef010afa0 con 0x7f8ef0103c00 2024-06-03T03:25:13.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.225+0000 7f8eecc00640 1 -- 172.21.15.117:0/2138503705 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f8ee0042030 con 0x7f8ef0103c00 2024-06-03T03:25:13.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.226+0000 7f8eecc00640 1 --2- 172.21.15.117:0/2138503705 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f8ed40a2b90 0x7f8ed40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:13.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.226+0000 7f8eecc00640 1 -- 172.21.15.117:0/2138503705 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f8ee00efda0 con 0x7f8ef0103c00 2024-06-03T03:25:13.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.226+0000 7f8ee7e00640 1 --2- 172.21.15.117:0/2138503705 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f8ed40a2b90 0x7f8ed40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:13.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.227+0000 7f8ee7e00640 1 --2- 172.21.15.117:0/2138503705 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f8ed40a2b90 0x7f8ed40a5050 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8ef019b450 tx=0x7f8ed800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:25:13.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.229+0000 7f8eecc00640 1 -- 172.21.15.117:0/2138503705 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8ee00bc2c0 con 0x7f8ef0103c00 2024-06-03T03:25:13.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:25:13 smithi151 ceph-mon[37043]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-06-03T03:25:13.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:25:13 smithi151 ceph-mon[37043]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T03:25:13.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:25:13 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/4293658852' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T03:25:13.427 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.427+0000 7f8ef6600640 1 -- 172.21.15.117:0/2138503705 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8ef00626a0 con 0x7f8ef0103c00 2024-06-03T03:25:13.429 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.428+0000 7f8eecc00640 1 -- 172.21.15.117:0/2138503705 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v99) ==== 93+0+10 (secure 0 0 0) 0x7f8ee00c05c0 con 0x7f8ef0103c00 2024-06-03T03:25:13.429 INFO:teuthology.orchestra.run.smithi117.stdout:pg_num: 1 2024-06-03T03:25:13.432 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.431+0000 7f8ef6600640 1 -- 172.21.15.117:0/2138503705 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f8ed40a2b90 msgr2=0x7f8ed40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:13.432 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.431+0000 7f8ef6600640 1 --2- 172.21.15.117:0/2138503705 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f8ed40a2b90 0x7f8ed40a5050 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8ef019b450 tx=0x7f8ed800b390 comp rx=0 tx=0).stop 2024-06-03T03:25:13.432 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.431+0000 7f8ef6600640 1 -- 172.21.15.117:0/2138503705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ef0103c00 msgr2=0x7f8ef0199f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:13.432 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.431+0000 7f8ef6600640 1 --2- 172.21.15.117:0/2138503705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ef0103c00 0x7f8ef0199f40 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0031440 tx=0x7f8ee0032ce0 comp rx=0 tx=0).stop 2024-06-03T03:25:13.432 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.431+0000 7f8ef6600640 1 -- 172.21.15.117:0/2138503705 shutdown_connections 2024-06-03T03:25:13.432 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.431+0000 7f8ef6600640 1 --2- 172.21.15.117:0/2138503705 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f8ed40a2b90 0x7f8ed40a5050 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:13.432 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.431+0000 7f8ef6600640 1 --2- 172.21.15.117:0/2138503705 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ef01057b0 0x7f8ef01a14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:13.432 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.431+0000 7f8ef6600640 1 --2- 172.21.15.117:0/2138503705 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ef0104df0 0x7f8ef019a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:13.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.431+0000 7f8ef6600640 1 --2- 172.21.15.117:0/2138503705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ef0103c00 0x7f8ef0199f40 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:13.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.431+0000 7f8ef6600640 1 -- 172.21.15.117:0/2138503705 >> 172.21.15.117:0/2138503705 conn(0x7f8ef00ff190 msgr2=0x7f8ef0108170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:25:13.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.431+0000 7f8ef6600640 1 -- 172.21.15.117:0/2138503705 shutdown_connections 2024-06-03T03:25:13.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.432+0000 7f8ef6600640 1 -- 172.21.15.117:0/2138503705 wait complete. 2024-06-03T03:25:13.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:25:13 smithi117 ceph-mon[31649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-06-03T03:25:13.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:25:13 smithi117 ceph-mon[31649]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T03:25:13.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:25:13 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/4293658852' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T03:25:13.450 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-06-03T03:25:13.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:25:13 smithi183 ceph-mon[37326]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-06-03T03:25:13.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:25:13 smithi183 ceph-mon[37326]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T03:25:13.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:25:13 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/4293658852' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T03:25:13.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.647+0000 7f508d400640 1 -- 172.21.15.117:0/1395662662 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5088102330 msgr2=0x7f5088104720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:13.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.647+0000 7f508d400640 1 --2- 172.21.15.117:0/1395662662 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5088102330 0x7f5088104720 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5078009ca0 tx=0x7f507802a8c0 comp rx=0 tx=0).stop 2024-06-03T03:25:13.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.647+0000 7f508d400640 1 -- 172.21.15.117:0/1395662662 shutdown_connections 2024-06-03T03:25:13.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.647+0000 7f508d400640 1 --2- 172.21.15.117:0/1395662662 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5088107590 0x7f50881099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:13.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.647+0000 7f508d400640 1 --2- 172.21.15.117:0/1395662662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5088104c60 0x7f5088107050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:13.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.647+0000 7f508d400640 1 --2- 172.21.15.117:0/1395662662 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5088102330 0x7f5088104720 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:13.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.647+0000 7f508d400640 1 -- 172.21.15.117:0/1395662662 >> 172.21.15.117:0/1395662662 conn(0x7f50880fbd40 msgr2=0x7f50880fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:25:13.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.647+0000 7f508d400640 1 -- 172.21.15.117:0/1395662662 shutdown_connections 2024-06-03T03:25:13.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.647+0000 7f508d400640 1 -- 172.21.15.117:0/1395662662 wait complete. 2024-06-03T03:25:13.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.648+0000 7f508d400640 1 Processor -- start 2024-06-03T03:25:13.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.648+0000 7f508d400640 1 -- start start 2024-06-03T03:25:13.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.649+0000 7f508d400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5088102330 0x7f5088199de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:13.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.649+0000 7f508d400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5088104c60 0x7f508819a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:13.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.649+0000 7f508d400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5088107590 0x7f50881a1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:13.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.649+0000 7f508d400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f508810d550 con 0x7f5088107590 2024-06-03T03:25:13.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.649+0000 7f508d400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f508810d3d0 con 0x7f5088102330 2024-06-03T03:25:13.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.649+0000 7f508d400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f508810d6d0 con 0x7f5088104c60 2024-06-03T03:25:13.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.649+0000 7f508ca00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5088107590 0x7f50881a1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:13.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.649+0000 7f5087e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5088102330 0x7f5088199de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:13.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.649+0000 7f5087400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5088104c60 0x7f508819a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:13.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.649+0000 7f5087e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5088102330 0x7f5088199de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:35624/0 (socket says 172.21.15.117:35624) 2024-06-03T03:25:13.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.649+0000 7f5087e00640 1 -- 172.21.15.117:0/181310355 learned_addr learned my addr 172.21.15.117:0/181310355 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:25:13.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.649+0000 7f5087400640 1 -- 172.21.15.117:0/181310355 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5088102330 msgr2=0x7f5088199de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:13.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.649+0000 7f5087400640 1 --2- 172.21.15.117:0/181310355 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5088102330 0x7f5088199de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:13.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.649+0000 7f5087400640 1 -- 172.21.15.117:0/181310355 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5088107590 msgr2=0x7f50881a1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:13.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.649+0000 7f5087400640 1 --2- 172.21.15.117:0/181310355 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5088107590 0x7f50881a1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:13.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.649+0000 7f5087400640 1 -- 172.21.15.117:0/181310355 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50881a1a00 con 0x7f5088104c60 2024-06-03T03:25:13.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.650+0000 7f5087400640 1 --2- 172.21.15.117:0/181310355 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5088104c60 0x7f508819a320 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f507000e9c0 tx=0x7f507000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:25:13.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.650+0000 7f5084c00640 1 -- 172.21.15.117:0/181310355 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f507000ccc0 con 0x7f5088104c60 2024-06-03T03:25:13.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.650+0000 7f5084c00640 1 -- 172.21.15.117:0/181310355 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5070010910 con 0x7f5088104c60 2024-06-03T03:25:13.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.650+0000 7f508d400640 1 -- 172.21.15.117:0/181310355 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50881a1cf0 con 0x7f5088104c60 2024-06-03T03:25:13.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.650+0000 7f5084c00640 1 -- 172.21.15.117:0/181310355 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50700188c0 con 0x7f5088104c60 2024-06-03T03:25:13.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.650+0000 7f508d400640 1 -- 172.21.15.117:0/181310355 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50881a2230 con 0x7f5088104c60 2024-06-03T03:25:13.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.651+0000 7f508d400640 1 -- 172.21.15.117:0/181310355 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50881090e0 con 0x7f5088104c60 2024-06-03T03:25:13.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.654+0000 7f5084c00640 1 -- 172.21.15.117:0/181310355 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f5070010ac0 con 0x7f5088104c60 2024-06-03T03:25:13.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.655+0000 7f5084c00640 1 --2- 172.21.15.117:0/181310355 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f506c0a2c10 0x7f506c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:13.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.656+0000 7f5084c00640 1 -- 172.21.15.117:0/181310355 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f50700cfbb0 con 0x7f5088104c60 2024-06-03T03:25:13.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.656+0000 7f5087e00640 1 --2- 172.21.15.117:0/181310355 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f506c0a2c10 0x7f506c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:13.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.656+0000 7f5087e00640 1 --2- 172.21.15.117:0/181310355 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f506c0a2c10 0x7f506c0a50d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5078004af0 tx=0x7f5078004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:25:13.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.657+0000 7f5084c00640 1 -- 172.21.15.117:0/181310355 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f507009c0d0 con 0x7f5088104c60 2024-06-03T03:25:13.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.859+0000 7f508d400640 1 -- 172.21.15.117:0/181310355 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f508802d530 con 0x7f5088104c60 2024-06-03T03:25:13.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.860+0000 7f5084c00640 1 -- 172.21.15.117:0/181310355 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v99) ==== 104+0+11 (secure 0 0 0) 0x7f50700a03d0 con 0x7f5088104c60 2024-06-03T03:25:13.860 INFO:teuthology.orchestra.run.smithi117.stdout:pg_num: 32 2024-06-03T03:25:13.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.862+0000 7f508d400640 1 -- 172.21.15.117:0/181310355 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f506c0a2c10 msgr2=0x7f506c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:13.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.862+0000 7f508d400640 1 --2- 172.21.15.117:0/181310355 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f506c0a2c10 0x7f506c0a50d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5078004af0 tx=0x7f5078004a40 comp rx=0 tx=0).stop 2024-06-03T03:25:13.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.862+0000 7f508d400640 1 -- 172.21.15.117:0/181310355 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5088104c60 msgr2=0x7f508819a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:13.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.862+0000 7f508d400640 1 --2- 172.21.15.117:0/181310355 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5088104c60 0x7f508819a320 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f507000e9c0 tx=0x7f507000ee80 comp rx=0 tx=0).stop 2024-06-03T03:25:13.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.862+0000 7f508d400640 1 -- 172.21.15.117:0/181310355 shutdown_connections 2024-06-03T03:25:13.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.862+0000 7f508d400640 1 --2- 172.21.15.117:0/181310355 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5088107590 0x7f50881a1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:13.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.862+0000 7f508d400640 1 --2- 172.21.15.117:0/181310355 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5088104c60 0x7f508819a320 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:13.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.862+0000 7f508d400640 1 --2- 172.21.15.117:0/181310355 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f506c0a2c10 0x7f506c0a50d0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:13.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.862+0000 7f508d400640 1 --2- 172.21.15.117:0/181310355 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5088102330 0x7f5088199de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:13.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.862+0000 7f508d400640 1 -- 172.21.15.117:0/181310355 >> 172.21.15.117:0/181310355 conn(0x7f50880fbd40 msgr2=0x7f5088101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:25:13.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.862+0000 7f508d400640 1 -- 172.21.15.117:0/181310355 shutdown_connections 2024-06-03T03:25:13.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:13.862+0000 7f508d400640 1 -- 172.21.15.117:0/181310355 wait complete. 2024-06-03T03:25:13.883 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-06-03T03:25:14.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.097+0000 7fd257600640 1 -- 172.21.15.117:0/3129712119 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd250110220 msgr2=0x7fd250112610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:14.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.097+0000 7fd257600640 1 --2- 172.21.15.117:0/3129712119 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd250110220 0x7fd250112610 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd240009930 tx=0x7fd24002a420 comp rx=0 tx=0).stop 2024-06-03T03:25:14.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.097+0000 7fd257600640 1 -- 172.21.15.117:0/3129712119 shutdown_connections 2024-06-03T03:25:14.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.097+0000 7fd257600640 1 --2- 172.21.15.117:0/3129712119 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd250110220 0x7fd250112610 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:14.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.097+0000 7fd257600640 1 --2- 172.21.15.117:0/3129712119 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd250109300 0x7fd25010fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:14.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.097+0000 7fd257600640 1 --2- 172.21.15.117:0/3129712119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2501089a0 0x7fd250108dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:14.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.097+0000 7fd257600640 1 -- 172.21.15.117:0/3129712119 >> 172.21.15.117:0/3129712119 conn(0x7fd25006ebd0 msgr2=0x7fd250070ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:25:14.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.097+0000 7fd257600640 1 -- 172.21.15.117:0/3129712119 shutdown_connections 2024-06-03T03:25:14.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.097+0000 7fd257600640 1 -- 172.21.15.117:0/3129712119 wait complete. 2024-06-03T03:25:14.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.098+0000 7fd257600640 1 Processor -- start 2024-06-03T03:25:14.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.098+0000 7fd257600640 1 -- start start 2024-06-03T03:25:14.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.098+0000 7fd257600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2501089a0 0x7fd2501a2760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:14.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.098+0000 7fd257600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd250109300 0x7fd2501a2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:14.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.098+0000 7fd257600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd250110220 0x7fd2501a9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:14.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.098+0000 7fd257600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd250115e90 con 0x7fd2501089a0 2024-06-03T03:25:14.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.098+0000 7fd257600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd250115d10 con 0x7fd250110220 2024-06-03T03:25:14.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.098+0000 7fd257600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd250116010 con 0x7fd250109300 2024-06-03T03:25:14.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.098+0000 7fd255600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd250110220 0x7fd2501a9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:14.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.099+0000 7fd255600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd250110220 0x7fd2501a9cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:35652/0 (socket says 172.21.15.117:35652) 2024-06-03T03:25:14.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.099+0000 7fd255600640 1 -- 172.21.15.117:0/1118974792 learned_addr learned my addr 172.21.15.117:0/1118974792 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:25:14.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.099+0000 7fd254c00640 1 --2- 172.21.15.117:0/1118974792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2501089a0 0x7fd2501a2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:14.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.099+0000 7fd24fe00640 1 --2- 172.21.15.117:0/1118974792 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd250109300 0x7fd2501a2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:14.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.099+0000 7fd255600640 1 -- 172.21.15.117:0/1118974792 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd250109300 msgr2=0x7fd2501a2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:14.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.099+0000 7fd255600640 1 --2- 172.21.15.117:0/1118974792 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd250109300 0x7fd2501a2ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:14.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.099+0000 7fd255600640 1 -- 172.21.15.117:0/1118974792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2501089a0 msgr2=0x7fd2501a2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:14.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.099+0000 7fd255600640 1 --2- 172.21.15.117:0/1118974792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2501089a0 0x7fd2501a2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:14.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.099+0000 7fd255600640 1 -- 172.21.15.117:0/1118974792 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2501aa330 con 0x7fd250110220 2024-06-03T03:25:14.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.099+0000 7fd24fe00640 1 --2- 172.21.15.117:0/1118974792 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd250109300 0x7fd2501a2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:25:14.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.099+0000 7fd255600640 1 --2- 172.21.15.117:0/1118974792 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd250110220 0x7fd2501a9cc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd240009900 tx=0x7fd2400042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:25:14.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.099+0000 7fd24d600640 1 -- 172.21.15.117:0/1118974792 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd24003f030 con 0x7fd250110220 2024-06-03T03:25:14.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.100+0000 7fd24d600640 1 -- 172.21.15.117:0/1118974792 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd240003ce0 con 0x7fd250110220 2024-06-03T03:25:14.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.100+0000 7fd257600640 1 -- 172.21.15.117:0/1118974792 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2501aa5c0 con 0x7fd250110220 2024-06-03T03:25:14.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.100+0000 7fd24d600640 1 -- 172.21.15.117:0/1118974792 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd240035070 con 0x7fd250110220 2024-06-03T03:25:14.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.100+0000 7fd257600640 1 -- 172.21.15.117:0/1118974792 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2501aab00 con 0x7fd250110220 2024-06-03T03:25:14.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.101+0000 7fd257600640 1 -- 172.21.15.117:0/1118974792 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2501aad60 con 0x7fd250110220 2024-06-03T03:25:14.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.102+0000 7fd24d600640 1 -- 172.21.15.117:0/1118974792 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7fd240042030 con 0x7fd250110220 2024-06-03T03:25:14.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.104+0000 7fd24d600640 1 --2- 172.21.15.117:0/1118974792 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fd2300a2ca0 0x7fd2300a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:14.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.104+0000 7fd24d600640 1 -- 172.21.15.117:0/1118974792 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7fd2400ef9d0 con 0x7fd250110220 2024-06-03T03:25:14.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.104+0000 7fd254c00640 1 --2- 172.21.15.117:0/1118974792 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fd2300a2ca0 0x7fd2300a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:14.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.105+0000 7fd254c00640 1 --2- 172.21.15.117:0/1118974792 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fd2300a2ca0 0x7fd2300a5160 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd248009bd0 tx=0x7fd248005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:25:14.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.108+0000 7fd24d600640 1 -- 172.21.15.117:0/1118974792 <== mon.2 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd2400bbd90 con 0x7fd250110220 2024-06-03T03:25:14.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:25:14 smithi117 ceph-mon[31649]: from='client.25196 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:25:14.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:25:14 smithi117 ceph-mon[31649]: pgmap v503: 129 pgs: 129 active+clean; 652 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 11 op/s 2024-06-03T03:25:14.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:25:14 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/2138503705' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-03T03:25:14.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:25:14 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/181310355' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-03T03:25:14.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.308+0000 7fd257600640 1 -- 172.21.15.117:0/1118974792 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd2501aaf30 con 0x7fd250110220 2024-06-03T03:25:14.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.309+0000 7fd24d600640 1 -- 172.21.15.117:0/1118974792 <== mon.2 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v99) ==== 100+0+11 (secure 0 0 0) 0x7fd2400c0090 con 0x7fd250110220 2024-06-03T03:25:14.310 INFO:teuthology.orchestra.run.smithi117.stdout:pg_num: 64 2024-06-03T03:25:14.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.312+0000 7fd257600640 1 -- 172.21.15.117:0/1118974792 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fd2300a2ca0 msgr2=0x7fd2300a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:14.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.312+0000 7fd257600640 1 --2- 172.21.15.117:0/1118974792 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fd2300a2ca0 0x7fd2300a5160 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd248009bd0 tx=0x7fd248005f90 comp rx=0 tx=0).stop 2024-06-03T03:25:14.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.312+0000 7fd257600640 1 -- 172.21.15.117:0/1118974792 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd250110220 msgr2=0x7fd2501a9cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:14.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.312+0000 7fd257600640 1 --2- 172.21.15.117:0/1118974792 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd250110220 0x7fd2501a9cc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd240009900 tx=0x7fd2400042f0 comp rx=0 tx=0).stop 2024-06-03T03:25:14.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.312+0000 7fd257600640 1 -- 172.21.15.117:0/1118974792 shutdown_connections 2024-06-03T03:25:14.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.312+0000 7fd257600640 1 --2- 172.21.15.117:0/1118974792 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fd2300a2ca0 0x7fd2300a5160 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:14.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.312+0000 7fd257600640 1 --2- 172.21.15.117:0/1118974792 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd250110220 0x7fd2501a9cc0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:14.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.312+0000 7fd257600640 1 --2- 172.21.15.117:0/1118974792 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd250109300 0x7fd2501a2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:14.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.312+0000 7fd257600640 1 --2- 172.21.15.117:0/1118974792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2501089a0 0x7fd2501a2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:14.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.312+0000 7fd257600640 1 -- 172.21.15.117:0/1118974792 >> 172.21.15.117:0/1118974792 conn(0x7fd25006ebd0 msgr2=0x7fd25006efb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:25:14.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.312+0000 7fd257600640 1 -- 172.21.15.117:0/1118974792 shutdown_connections 2024-06-03T03:25:14.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.312+0000 7fd257600640 1 -- 172.21.15.117:0/1118974792 wait complete. 2024-06-03T03:25:14.332 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-06-03T03:25:14.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:25:14 smithi151 ceph-mon[37043]: from='client.25196 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:25:14.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:25:14 smithi151 ceph-mon[37043]: pgmap v503: 129 pgs: 129 active+clean; 652 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 11 op/s 2024-06-03T03:25:14.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:25:14 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/2138503705' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-03T03:25:14.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:25:14 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/181310355' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-03T03:25:14.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:25:14 smithi183 ceph-mon[37326]: from='client.25196 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T03:25:14.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:25:14 smithi183 ceph-mon[37326]: pgmap v503: 129 pgs: 129 active+clean; 652 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 11 op/s 2024-06-03T03:25:14.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:25:14 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/2138503705' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-03T03:25:14.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:25:14 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/181310355' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-03T03:25:14.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.544+0000 7f443b200640 1 -- 172.21.15.117:0/2920914738 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f443c109a30 msgr2=0x7f443c110350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:14.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.544+0000 7f443b200640 1 --2- 172.21.15.117:0/2920914738 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f443c109a30 0x7f443c110350 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4434009900 tx=0x7f443402a3f0 comp rx=0 tx=0).stop 2024-06-03T03:25:14.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.544+0000 7f443b200640 1 -- 172.21.15.117:0/2920914738 shutdown_connections 2024-06-03T03:25:14.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.544+0000 7f443b200640 1 --2- 172.21.15.117:0/2920914738 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f443c109a30 0x7f443c110350 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:14.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.544+0000 7f443b200640 1 --2- 172.21.15.117:0/2920914738 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f443c109070 0x7f443c1094f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:14.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.544+0000 7f443b200640 1 --2- 172.21.15.117:0/2920914738 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f443c107e90 0x7f443c1082b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:14.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.544+0000 7f443b200640 1 -- 172.21.15.117:0/2920914738 >> 172.21.15.117:0/2920914738 conn(0x7f443c076e70 msgr2=0x7f443c0792b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:25:14.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.544+0000 7f443b200640 1 -- 172.21.15.117:0/2920914738 shutdown_connections 2024-06-03T03:25:14.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.544+0000 7f443b200640 1 -- 172.21.15.117:0/2920914738 wait complete. 2024-06-03T03:25:14.548 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.545+0000 7f443b200640 1 Processor -- start 2024-06-03T03:25:14.548 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.545+0000 7f443b200640 1 -- start start 2024-06-03T03:25:14.548 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.545+0000 7f443b200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f443c107e90 0x7f443c19e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:14.548 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.545+0000 7f443b200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f443c109070 0x7f443c19e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:14.548 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.545+0000 7f443b200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f443c109a30 0x7f443c1a5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:14.548 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.545+0000 7f443b200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f443c113b90 con 0x7f443c109a30 2024-06-03T03:25:14.548 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.545+0000 7f443b200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f443c113a10 con 0x7f443c107e90 2024-06-03T03:25:14.548 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.545+0000 7f443b200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f443c113d10 con 0x7f443c109070 2024-06-03T03:25:14.548 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.546+0000 7f443a800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f443c109a30 0x7f443c1a5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:14.548 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.546+0000 7f4439e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f443c107e90 0x7f443c19e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:14.548 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.546+0000 7f443a800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f443c109a30 0x7f443c1a5750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:47212/0 (socket says 172.21.15.117:47212) 2024-06-03T03:25:14.548 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.546+0000 7f443a800640 1 -- 172.21.15.117:0/2905712200 learned_addr learned my addr 172.21.15.117:0/2905712200 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:25:14.549 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.546+0000 7f4439400640 1 --2- 172.21.15.117:0/2905712200 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f443c109070 0x7f443c19e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:14.549 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.546+0000 7f4439e00640 1 -- 172.21.15.117:0/2905712200 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f443c109070 msgr2=0x7f443c19e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:14.549 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.546+0000 7f4439e00640 1 --2- 172.21.15.117:0/2905712200 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f443c109070 0x7f443c19e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:14.549 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.546+0000 7f4439e00640 1 -- 172.21.15.117:0/2905712200 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f443c109a30 msgr2=0x7f443c1a5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:14.549 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.546+0000 7f4439e00640 1 --2- 172.21.15.117:0/2905712200 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f443c109a30 0x7f443c1a5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:14.549 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.546+0000 7f4439e00640 1 -- 172.21.15.117:0/2905712200 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f443c1a5dc0 con 0x7f443c107e90 2024-06-03T03:25:14.549 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.546+0000 7f4439e00640 1 --2- 172.21.15.117:0/2905712200 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f443c107e90 0x7f443c19e1a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f442c005fd0 tx=0x7f442c00ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:25:14.549 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.547+0000 7f4432a00640 1 -- 172.21.15.117:0/2905712200 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f442c003970 con 0x7f443c107e90 2024-06-03T03:25:14.549 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.547+0000 7f4432a00640 1 -- 172.21.15.117:0/2905712200 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f442c004030 con 0x7f443c107e90 2024-06-03T03:25:14.549 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.547+0000 7f4432a00640 1 -- 172.21.15.117:0/2905712200 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f442c0107f0 con 0x7f443c107e90 2024-06-03T03:25:14.549 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.547+0000 7f443b200640 1 -- 172.21.15.117:0/2905712200 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f443c1a60b0 con 0x7f443c107e90 2024-06-03T03:25:14.549 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.547+0000 7f443b200640 1 -- 172.21.15.117:0/2905712200 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f443c1a6650 con 0x7f443c107e90 2024-06-03T03:25:14.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.548+0000 7f443b200640 1 -- 172.21.15.117:0/2905712200 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f443c10f1d0 con 0x7f443c107e90 2024-06-03T03:25:14.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.550+0000 7f4432a00640 1 -- 172.21.15.117:0/2905712200 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f442c0041e0 con 0x7f443c107e90 2024-06-03T03:25:14.552 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.551+0000 7f4432a00640 1 --2- 172.21.15.117:0/2905712200 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f44200a2ca0 0x7f44200a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:14.552 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.551+0000 7f4432a00640 1 -- 172.21.15.117:0/2905712200 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f442c0cfd80 con 0x7f443c107e90 2024-06-03T03:25:14.552 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.551+0000 7f4439400640 1 --2- 172.21.15.117:0/2905712200 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f44200a2ca0 0x7f44200a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:14.552 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.552+0000 7f4439400640 1 --2- 172.21.15.117:0/2905712200 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f44200a2ca0 0x7f44200a5160 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f443c19f6b0 tx=0x7f4424005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:25:14.555 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.555+0000 7f4432a00640 1 -- 172.21.15.117:0/2905712200 <== mon.2 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f442c09c790 con 0x7f443c107e90 2024-06-03T03:25:14.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.754+0000 7f443b200640 1 -- 172.21.15.117:0/2905712200 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f443c0625a0 con 0x7f443c107e90 2024-06-03T03:25:14.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.755+0000 7f4432a00640 1 -- 172.21.15.117:0/2905712200 <== mon.2 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v99) ==== 103+0+11 (secure 0 0 0) 0x7f442c09c090 con 0x7f443c107e90 2024-06-03T03:25:14.755 INFO:teuthology.orchestra.run.smithi117.stdout:pg_num: 32 2024-06-03T03:25:14.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.757+0000 7f443b200640 1 -- 172.21.15.117:0/2905712200 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f44200a2ca0 msgr2=0x7f44200a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:14.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.757+0000 7f443b200640 1 --2- 172.21.15.117:0/2905712200 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f44200a2ca0 0x7f44200a5160 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f443c19f6b0 tx=0x7f4424005f90 comp rx=0 tx=0).stop 2024-06-03T03:25:14.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.757+0000 7f443b200640 1 -- 172.21.15.117:0/2905712200 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f443c107e90 msgr2=0x7f443c19e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:14.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.757+0000 7f443b200640 1 --2- 172.21.15.117:0/2905712200 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f443c107e90 0x7f443c19e1a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f442c005fd0 tx=0x7f442c00ef90 comp rx=0 tx=0).stop 2024-06-03T03:25:14.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.757+0000 7f443b200640 1 -- 172.21.15.117:0/2905712200 shutdown_connections 2024-06-03T03:25:14.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.757+0000 7f443b200640 1 --2- 172.21.15.117:0/2905712200 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f443c109a30 0x7f443c1a5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:14.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.757+0000 7f443b200640 1 --2- 172.21.15.117:0/2905712200 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f443c109070 0x7f443c19e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:14.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.757+0000 7f443b200640 1 --2- 172.21.15.117:0/2905712200 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f44200a2ca0 0x7f44200a5160 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:14.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.757+0000 7f443b200640 1 --2- 172.21.15.117:0/2905712200 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f443c107e90 0x7f443c19e1a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:14.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.757+0000 7f443b200640 1 -- 172.21.15.117:0/2905712200 >> 172.21.15.117:0/2905712200 conn(0x7f443c076e70 msgr2=0x7f443c10c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:25:14.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.758+0000 7f443b200640 1 -- 172.21.15.117:0/2905712200 shutdown_connections 2024-06-03T03:25:14.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:14.758+0000 7f443b200640 1 -- 172.21.15.117:0/2905712200 wait complete. 2024-06-03T03:25:14.778 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-06-03T03:25:14.778 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-03T03:25:15.214 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:25:15.364 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:25:15 smithi117 ceph-mon[31649]: mgrmap e19: x(active, since 13m), standbys: z, y 2024-06-03T03:25:15.364 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:25:15 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/1118974792' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-03T03:25:15.364 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:25:15 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/2905712200' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-06-03T03:25:15.364 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:25:15 smithi117 ceph-mon[31649]: pgmap v504: 129 pgs: 129 active+clean; 652 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 682 B/s wr, 10 op/s 2024-06-03T03:25:15.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:25:15 smithi151 ceph-mon[37043]: mgrmap e19: x(active, since 13m), standbys: z, y 2024-06-03T03:25:15.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:25:15 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/1118974792' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-03T03:25:15.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:25:15 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/2905712200' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-06-03T03:25:15.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:25:15 smithi151 ceph-mon[37043]: pgmap v504: 129 pgs: 129 active+clean; 652 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 682 B/s wr, 10 op/s 2024-06-03T03:25:15.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:25:15 smithi183 ceph-mon[37326]: mgrmap e19: x(active, since 13m), standbys: z, y 2024-06-03T03:25:15.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:25:15 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/1118974792' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-03T03:25:15.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:25:15 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/2905712200' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-06-03T03:25:15.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:25:15 smithi183 ceph-mon[37326]: pgmap v504: 129 pgs: 129 active+clean; 652 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 682 B/s wr, 10 op/s 2024-06-03T03:25:17.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:17.697+0000 7f2107200700 1 -- 172.21.15.117:0/1805359806 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f210810ec00 msgr2=0x7f2108111070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:17.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:17.697+0000 7f2107200700 1 --2- 172.21.15.117:0/1805359806 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f210810ec00 0x7f2108111070 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f210000b0a0 tx=0x7f210000b3b0 comp rx=0 tx=0).stop 2024-06-03T03:25:17.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:17.697+0000 7f2107200700 1 -- 172.21.15.117:0/1805359806 shutdown_connections 2024-06-03T03:25:17.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:17.697+0000 7f2107200700 1 --2- 172.21.15.117:0/1805359806 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f210810ec00 0x7f2108111070 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:17.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:17.697+0000 7f2107200700 1 --2- 172.21.15.117:0/1805359806 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f21080748c0 0x7f2108074ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:17.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:17.697+0000 7f2107200700 1 --2- 172.21.15.117:0/1805359806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f21080751e0 0x7f210810e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:17.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:17.697+0000 7f2107200700 1 -- 172.21.15.117:0/1805359806 >> 172.21.15.117:0/1805359806 conn(0x7f21080fdf10 msgr2=0x7f2108100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:25:17.700 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:17.698+0000 7f2107200700 1 -- 172.21.15.117:0/1805359806 shutdown_connections 2024-06-03T03:25:17.700 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:17.698+0000 7f2107200700 1 -- 172.21.15.117:0/1805359806 wait complete. 2024-06-03T03:25:17.701 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:17.699+0000 7f2107200700 1 Processor -- start 2024-06-03T03:25:17.701 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:17.699+0000 7f2107200700 1 -- start start 2024-06-03T03:25:17.701 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:17.700+0000 7f2107200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f21080748c0 0x7f210819ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:17.701 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:17.700+0000 7f2107200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f21080751e0 0x7f210819b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:17.701 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:17.700+0000 7f2107200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f210810ec00 0x7f210819f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:17.701 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:17.700+0000 7f2107200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2108114930 con 0x7f21080748c0 2024-06-03T03:25:17.701 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:17.700+0000 7f2107200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f21081147b0 con 0x7f210810ec00 2024-06-03T03:25:17.702 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:17.700+0000 7f2107200700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2108114ab0 con 0x7f21080751e0 2024-06-03T03:25:17.702 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:17.700+0000 7f2105e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f21080748c0 0x7f210819ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:17.702 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:17.700+0000 7f20fd400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f21080751e0 0x7f210819b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:17.702 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:17.700+0000 7f2105e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f21080748c0 0x7f210819ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39774/0 (socket says 172.21.15.117:39774) 2024-06-03T03:25:17.702 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:17.700+0000 7f2105e00700 1 -- 172.21.15.117:0/2254680444 learned_addr learned my addr 172.21.15.117:0/2254680444 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:25:17.702 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:17.700+0000 7f2106800700 1 --2- 172.21.15.117:0/2254680444 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f210810ec00 0x7f210819f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:17.702 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:17.700+0000 7f2105e00700 1 -- 172.21.15.117:0/2254680444 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f21080751e0 msgr2=0x7f210819b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:17.702 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:17.700+0000 7f2105e00700 1 --2- 172.21.15.117:0/2254680444 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f21080751e0 0x7f210819b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:17.702 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:17.700+0000 7f2105e00700 1 -- 172.21.15.117:0/2254680444 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f210810ec00 msgr2=0x7f210819f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:17.702 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:17.700+0000 7f2105e00700 1 --2- 172.21.15.117:0/2254680444 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f210810ec00 0x7f210819f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:17.702 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:17.701+0000 7f2105e00700 1 -- 172.21.15.117:0/2254680444 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f210819fe30 con 0x7f21080748c0 2024-06-03T03:25:17.702 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:17.701+0000 7f2105e00700 1 --2- 172.21.15.117:0/2254680444 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f21080748c0 0x7f210819ae20 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f20f800d950 tx=0x7f20f800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:25:17.703 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:17.701+0000 7f20ff400700 1 -- 172.21.15.117:0/2254680444 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f8016070 con 0x7f21080748c0 2024-06-03T03:25:17.703 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:17.701+0000 7f2107200700 1 -- 172.21.15.117:0/2254680444 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21081a0120 con 0x7f21080748c0 2024-06-03T03:25:17.703 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:17.701+0000 7f20ff400700 1 -- 172.21.15.117:0/2254680444 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20f800fd70 con 0x7f21080748c0 2024-06-03T03:25:17.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:17.701+0000 7f20ff400700 1 -- 172.21.15.117:0/2254680444 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f800e610 con 0x7f21080748c0 2024-06-03T03:25:17.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:17.701+0000 7f2107200700 1 -- 172.21.15.117:0/2254680444 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21081a79a0 con 0x7f21080748c0 2024-06-03T03:25:17.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:17.703+0000 7f2107200700 1 -- 172.21.15.117:0/2254680444 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2108069180 con 0x7f21080748c0 2024-06-03T03:25:17.705 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:17.704+0000 7f20ff400700 1 -- 172.21.15.117:0/2254680444 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f20f8019070 con 0x7f21080748c0 2024-06-03T03:25:17.706 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:17.706+0000 7f20ff400700 1 --2- 172.21.15.117:0/2254680444 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f20ec0a2d30 0x7f20ec0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:17.707 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:17.706+0000 7f20ff400700 1 -- 172.21.15.117:0/2254680444 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f20f80ceff0 con 0x7f21080748c0 2024-06-03T03:25:17.707 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:17.706+0000 7f20fd400700 1 --2- 172.21.15.117:0/2254680444 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f20ec0a2d30 0x7f20ec0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:17.707 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:17.707+0000 7f20fd400700 1 --2- 172.21.15.117:0/2254680444 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f20ec0a2d30 0x7f20ec0a51f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f21081007b0 tx=0x7f20f0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:25:17.711 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:17.711+0000 7f20ff400700 1 -- 172.21.15.117:0/2254680444 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f20f809ba00 con 0x7f21080748c0 2024-06-03T03:25:17.990 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:17.989+0000 7f2107200700 1 -- 172.21.15.117:0/2254680444 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f210819c180 con 0x7f21080748c0 2024-06-03T03:25:17.991 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:17.991+0000 7f20ff400700 1 -- 172.21.15.117:0/2254680444 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11955 (secure 0 0 0) 0x7f20f809b300 con 0x7f21080748c0 2024-06-03T03:25:17.992 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T03:25:17.992 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3353001119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3353001119},{"type":"v1","addr":"172.21.15.117:6837","nonce":3353001119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14388,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6839/405782304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":405782304},{"type":"v1","addr":"172.21.15.117:6839","nonce":405782304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-03T03:19:44.629822+0000","modified":"2024-06-03T03:25:11.576498+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24517,"mds_1":24512,"mds_2":24503,"mds_3":24499,"mds_4":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24431":{"gid":24431,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6833/3355662416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3355662416},{"type":"v1","addr":"172.21.15.151:6833","nonce":3355662416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/2429725085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2429725085},{"type":"v1","addr":"172.21.15.183:6833","nonce":2429725085}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/31153579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":31153579},{"type":"v1","addr":"172.21.15.151:6835","nonce":31153579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"j","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/2884460088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2884460088},{"type":"v1","addr":"172.21.15.117:6835","nonce":2884460088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/2392407848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":2392407848},{"type":"v1","addr":"172.21.15.151:6837","nonce":2392407848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":32,"addr":"172.21.15.183:6835/1009004320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1009004320},{"type":"v1","addr":"172.21.15.183:6835","nonce":1009004320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.183:6837/2697518526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":2697518526},{"type":"v1","addr":"172.21.15.183:6837","nonce":2697518526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"a","rank":2,"incarnation":22,"state":"up:active","state_seq":27,"addr":"172.21.15.117:6841/1212242967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6840","nonce":1212242967},{"type":"v1","addr":"172.21.15.117:6841","nonce":1212242967}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.151:6839/3623444616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":3623444616},{"type":"v1","addr":"172.21.15.151:6839","nonce":3623444616}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":10,"addr":"172.21.15.183:6839/1439615083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1439615083},{"type":"v1","addr":"172.21.15.183:6839","nonce":1439615083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24517,"qdb_cluster":[24499,24487,24512,24503,24517]},"id":1}]} 2024-06-03T03:25:17.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:17.994+0000 7f2107200700 1 -- 172.21.15.117:0/2254680444 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f20ec0a2d30 msgr2=0x7f20ec0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:17.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:17.994+0000 7f2107200700 1 --2- 172.21.15.117:0/2254680444 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f20ec0a2d30 0x7f20ec0a51f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f21081007b0 tx=0x7f20f0009260 comp rx=0 tx=0).stop 2024-06-03T03:25:17.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:17.994+0000 7f2107200700 1 -- 172.21.15.117:0/2254680444 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f21080748c0 msgr2=0x7f210819ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:17.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:17.994+0000 7f2107200700 1 --2- 172.21.15.117:0/2254680444 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f21080748c0 0x7f210819ae20 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f20f800d950 tx=0x7f20f800dc60 comp rx=0 tx=0).stop 2024-06-03T03:25:17.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:17.994+0000 7f2107200700 1 -- 172.21.15.117:0/2254680444 shutdown_connections 2024-06-03T03:25:17.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:17.994+0000 7f2107200700 1 --2- 172.21.15.117:0/2254680444 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f20ec0a2d30 0x7f20ec0a51f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:17.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:17.994+0000 7f2107200700 1 --2- 172.21.15.117:0/2254680444 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f210810ec00 0x7f210819f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:17.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:17.994+0000 7f2107200700 1 --2- 172.21.15.117:0/2254680444 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f21080748c0 0x7f210819ae20 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:17.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:17.994+0000 7f2107200700 1 --2- 172.21.15.117:0/2254680444 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f21080751e0 0x7f210819b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:17.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:17.994+0000 7f2107200700 1 -- 172.21.15.117:0/2254680444 >> 172.21.15.117:0/2254680444 conn(0x7f21080fdf10 msgr2=0x7f21080feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:25:17.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:17.994+0000 7f2107200700 1 -- 172.21.15.117:0/2254680444 shutdown_connections 2024-06-03T03:25:17.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:17.994+0000 7f2107200700 1 -- 172.21.15.117:0/2254680444 wait complete. 2024-06-03T03:25:17.996 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 31 2024-06-03T03:25:18.361 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:25:18 smithi117 ceph-mon[31649]: pgmap v505: 129 pgs: 129 active+clean; 652 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 852 B/s wr, 10 op/s 2024-06-03T03:25:18.361 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:25:18 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/2254680444' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T03:25:18.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:25:18 smithi151 ceph-mon[37043]: pgmap v505: 129 pgs: 129 active+clean; 652 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 852 B/s wr, 10 op/s 2024-06-03T03:25:18.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:25:18 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/2254680444' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T03:25:18.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:25:18 smithi183 ceph-mon[37326]: pgmap v505: 129 pgs: 129 active+clean; 652 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 852 B/s wr, 10 op/s 2024-06-03T03:25:18.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:25:18 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/2254680444' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T03:25:18.962 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-06-03T03:25:18.962 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-06-03T03:25:18.962 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-06-03T03:25:19.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.173+0000 7f5c02400640 1 -- 172.21.15.117:0/2866269142 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bfc108000 msgr2=0x7f5bfc108420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:19.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.173+0000 7f5c02400640 1 --2- 172.21.15.117:0/2866269142 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bfc108000 0x7f5bfc108420 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5bec009900 tx=0x7f5bec02a410 comp rx=0 tx=0).stop 2024-06-03T03:25:19.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.173+0000 7f5c02400640 1 -- 172.21.15.117:0/2866269142 shutdown_connections 2024-06-03T03:25:19.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.173+0000 7f5c02400640 1 --2- 172.21.15.117:0/2866269142 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5bfc109bb0 0x7f5bfc110440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:19.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.173+0000 7f5c02400640 1 --2- 172.21.15.117:0/2866269142 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5bfc1091f0 0x7f5bfc109670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:19.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.173+0000 7f5c02400640 1 --2- 172.21.15.117:0/2866269142 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bfc108000 0x7f5bfc108420 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:19.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.173+0000 7f5c02400640 1 -- 172.21.15.117:0/2866269142 >> 172.21.15.117:0/2866269142 conn(0x7f5bfc076fc0 msgr2=0x7f5bfc0793e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:25:19.175 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.173+0000 7f5c02400640 1 -- 172.21.15.117:0/2866269142 shutdown_connections 2024-06-03T03:25:19.175 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.173+0000 7f5c02400640 1 -- 172.21.15.117:0/2866269142 wait complete. 2024-06-03T03:25:19.179 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.174+0000 7f5c02400640 1 Processor -- start 2024-06-03T03:25:19.180 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.174+0000 7f5c02400640 1 -- start start 2024-06-03T03:25:19.180 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.175+0000 7f5c02400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5bfc108000 0x7f5bfc19e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:19.180 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.175+0000 7f5c02400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5bfc1091f0 0x7f5bfc19e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:19.180 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.175+0000 7f5c02400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bfc109bb0 0x7f5bfc1a58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:19.180 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.175+0000 7f5c02400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5bfc113ce0 con 0x7f5bfc109bb0 2024-06-03T03:25:19.180 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.175+0000 7f5c02400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5bfc113b60 con 0x7f5bfc108000 2024-06-03T03:25:19.180 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.175+0000 7f5c02400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5bfc113e60 con 0x7f5bfc1091f0 2024-06-03T03:25:19.180 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.175+0000 7f5bfbe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bfc109bb0 0x7f5bfc1a58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:19.180 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.175+0000 7f5bfb400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5bfc108000 0x7f5bfc19e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:19.180 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.175+0000 7f5bfbe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bfc109bb0 0x7f5bfc1a58e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39786/0 (socket says 172.21.15.117:39786) 2024-06-03T03:25:19.180 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.175+0000 7f5bfbe00640 1 -- 172.21.15.117:0/535484058 learned_addr learned my addr 172.21.15.117:0/535484058 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:25:19.180 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.175+0000 7f5bfaa00640 1 --2- 172.21.15.117:0/535484058 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5bfc1091f0 0x7f5bfc19e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:19.180 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.175+0000 7f5bfb400640 1 -- 172.21.15.117:0/535484058 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5bfc1091f0 msgr2=0x7f5bfc19e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:19.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.175+0000 7f5bfb400640 1 --2- 172.21.15.117:0/535484058 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5bfc1091f0 0x7f5bfc19e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:19.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.175+0000 7f5bfb400640 1 -- 172.21.15.117:0/535484058 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bfc109bb0 msgr2=0x7f5bfc1a58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:19.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.175+0000 7f5bfb400640 1 --2- 172.21.15.117:0/535484058 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bfc109bb0 0x7f5bfc1a58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:19.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.175+0000 7f5bfb400640 1 -- 172.21.15.117:0/535484058 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bfc1a5f50 con 0x7f5bfc108000 2024-06-03T03:25:19.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.176+0000 7f5bfb400640 1 --2- 172.21.15.117:0/535484058 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5bfc108000 0x7f5bfc19e330 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5bec009230 tx=0x7f5bec032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:25:19.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.176+0000 7f5bf3e00640 1 -- 172.21.15.117:0/535484058 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bec03f030 con 0x7f5bfc108000 2024-06-03T03:25:19.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.176+0000 7f5bf3e00640 1 -- 172.21.15.117:0/535484058 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bec003ce0 con 0x7f5bfc108000 2024-06-03T03:25:19.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.176+0000 7f5c02400640 1 -- 172.21.15.117:0/535484058 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bfc1a61e0 con 0x7f5bfc108000 2024-06-03T03:25:19.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.176+0000 7f5bf3e00640 1 -- 172.21.15.117:0/535484058 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bec035070 con 0x7f5bfc108000 2024-06-03T03:25:19.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.176+0000 7f5c02400640 1 -- 172.21.15.117:0/535484058 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bfc1a66a0 con 0x7f5bfc108000 2024-06-03T03:25:19.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.177+0000 7f5c02400640 1 -- 172.21.15.117:0/535484058 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bfc10f320 con 0x7f5bfc108000 2024-06-03T03:25:19.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.179+0000 7f5bf3e00640 1 -- 172.21.15.117:0/535484058 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f5bec042030 con 0x7f5bfc108000 2024-06-03T03:25:19.182 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.180+0000 7f5bf3e00640 1 --2- 172.21.15.117:0/535484058 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f5be00a2c20 0x7f5be00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:19.182 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.181+0000 7f5bf3e00640 1 -- 172.21.15.117:0/535484058 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f5bec0efa30 con 0x7f5bfc108000 2024-06-03T03:25:19.182 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.181+0000 7f5bfaa00640 1 --2- 172.21.15.117:0/535484058 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f5be00a2c20 0x7f5be00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:19.182 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.181+0000 7f5bfaa00640 1 --2- 172.21.15.117:0/535484058 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f5be00a2c20 0x7f5be00a50e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc19f840 tx=0x7f5be4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:25:19.185 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.185+0000 7f5bf3e00640 1 -- 172.21.15.117:0/535484058 <== mon.2 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5bec0bbdf0 con 0x7f5bfc108000 2024-06-03T03:25:19.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.382+0000 7f5c02400640 1 -- 172.21.15.117:0/535484058 --> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5bfc061de0 con 0x7f5be00a2c20 2024-06-03T03:25:19.401 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.401+0000 7f5bf3e00640 1 -- 172.21.15.117:0/535484058 <== mgr.14152 v2:172.21.15.117:6800/1398160887 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340831 (secure 0 0 0) 0x7f5bfc061de0 con 0x7f5be00a2c20 2024-06-03T03:25:19.407 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T03:25:19.409 INFO:teuthology.orchestra.run.smithi117.stderr:dumped all 2024-06-03T03:25:19.410 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.408+0000 7f5c02400640 1 -- 172.21.15.117:0/535484058 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f5be00a2c20 msgr2=0x7f5be00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:19.410 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.408+0000 7f5c02400640 1 --2- 172.21.15.117:0/535484058 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f5be00a2c20 0x7f5be00a50e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc19f840 tx=0x7f5be4005f90 comp rx=0 tx=0).stop 2024-06-03T03:25:19.410 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.408+0000 7f5c02400640 1 -- 172.21.15.117:0/535484058 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5bfc108000 msgr2=0x7f5bfc19e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:19.410 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.408+0000 7f5c02400640 1 --2- 172.21.15.117:0/535484058 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5bfc108000 0x7f5bfc19e330 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5bec009230 tx=0x7f5bec032ce0 comp rx=0 tx=0).stop 2024-06-03T03:25:19.411 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.408+0000 7f5c02400640 1 -- 172.21.15.117:0/535484058 shutdown_connections 2024-06-03T03:25:19.411 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.408+0000 7f5c02400640 1 --2- 172.21.15.117:0/535484058 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bfc109bb0 0x7f5bfc1a58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:19.411 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.408+0000 7f5c02400640 1 --2- 172.21.15.117:0/535484058 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5bfc1091f0 0x7f5bfc19e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:19.411 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.408+0000 7f5c02400640 1 --2- 172.21.15.117:0/535484058 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f5be00a2c20 0x7f5be00a50e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:19.411 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.408+0000 7f5c02400640 1 --2- 172.21.15.117:0/535484058 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5bfc108000 0x7f5bfc19e330 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:19.411 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.408+0000 7f5c02400640 1 -- 172.21.15.117:0/535484058 >> 172.21.15.117:0/535484058 conn(0x7f5bfc076fc0 msgr2=0x7f5bfc10c4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:25:19.411 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.409+0000 7f5c02400640 1 -- 172.21.15.117:0/535484058 shutdown_connections 2024-06-03T03:25:19.411 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.409+0000 7f5c02400640 1 -- 172.21.15.117:0/535484058 wait complete. 2024-06-03T03:25:19.429 INFO:teuthology.orchestra.run.smithi117.stdout:{"pg_ready":true,"pg_map":{"version":506,"stamp":"2024-06-03T03:25:18.957567+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":673245,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":2664,"num_read_kb":1470,"num_write":466,"num_write_kb":3114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":880,"ondisk_log_size":880,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":370944,"kb_used_data":13472,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124570880,"statfs":{"total":1151940427776,"available":1151560581120,"internally_reserved":0,"allocated":13795328,"data_stored":8832845,"data_compressed":501066,"data_compressed_allocated":688128,"data_compressed_original":2875392,"omap_allocated":19142,"internal_metadata":365933882},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6486,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":146,"num_read_kb":80,"num_write":19,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005661"},"pg_stats":[{"pgid":"3.3f","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.832850+0000","last_change":"2024-06-03T03:19:41.626030+0000","last_active":"2024-06-03T03:22:54.832850+0000","last_peered":"2024-06-03T03:22:54.832850+0000","last_clean":"2024-06-03T03:22:54.832850+0000","last_became_active":"2024-06-03T03:19:41.625681+0000","last_became_peered":"2024-06-03T03:19:41.625681+0000","last_unstale":"2024-06-03T03:22:54.832850+0000","last_undegraded":"2024-06-03T03:22:54.832850+0000","last_fullsized":"2024-06-03T03:22:54.832850+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:35:46.396360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"99'4","reported_seq":43,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.831429+0000","last_change":"2024-06-03T03:19:41.619668+0000","last_active":"2024-06-03T03:22:54.831429+0000","last_peered":"2024-06-03T03:22:54.831429+0000","last_clean":"2024-06-03T03:22:54.831429+0000","last_became_active":"2024-06-03T03:19:41.619318+0000","last_became_peered":"2024-06-03T03:19:41.619318+0000","last_unstale":"2024-06-03T03:22:54.831429+0000","last_undegraded":"2024-06-03T03:22:54.831429+0000","last_fullsized":"2024-06-03T03:22:54.831429+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:07:21.656045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.832725+0000","last_change":"2024-06-03T03:19:41.623689+0000","last_active":"2024-06-03T03:22:54.832725+0000","last_peered":"2024-06-03T03:22:54.832725+0000","last_clean":"2024-06-03T03:22:54.832725+0000","last_became_active":"2024-06-03T03:19:41.623027+0000","last_became_peered":"2024-06-03T03:19:41.623027+0000","last_unstale":"2024-06-03T03:22:54.832725+0000","last_undegraded":"2024-06-03T03:22:54.832725+0000","last_fullsized":"2024-06-03T03:22:54.832725+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:18:05.584397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.815076+0000","last_change":"2024-06-03T03:19:41.603802+0000","last_active":"2024-06-03T03:22:54.815076+0000","last_peered":"2024-06-03T03:22:54.815076+0000","last_clean":"2024-06-03T03:22:54.815076+0000","last_became_active":"2024-06-03T03:19:41.603470+0000","last_became_peered":"2024-06-03T03:19:41.603470+0000","last_unstale":"2024-06-03T03:22:54.815076+0000","last_undegraded":"2024-06-03T03:22:54.815076+0000","last_fullsized":"2024-06-03T03:22:54.815076+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:56:10.636108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"99'4","reported_seq":43,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.826310+0000","last_change":"2024-06-03T03:19:41.636436+0000","last_active":"2024-06-03T03:22:54.826310+0000","last_peered":"2024-06-03T03:22:54.826310+0000","last_clean":"2024-06-03T03:22:54.826310+0000","last_became_active":"2024-06-03T03:19:41.636150+0000","last_became_peered":"2024-06-03T03:19:41.636150+0000","last_unstale":"2024-06-03T03:22:54.826310+0000","last_undegraded":"2024-06-03T03:22:54.826310+0000","last_fullsized":"2024-06-03T03:22:54.826310+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:02:10.928251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"99'6","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.809082+0000","last_change":"2024-06-03T03:19:41.622426+0000","last_active":"2024-06-03T03:22:54.809082+0000","last_peered":"2024-06-03T03:22:54.809082+0000","last_clean":"2024-06-03T03:22:54.809082+0000","last_became_active":"2024-06-03T03:19:41.622092+0000","last_became_peered":"2024-06-03T03:19:41.622092+0000","last_unstale":"2024-06-03T03:22:54.809082+0000","last_undegraded":"2024-06-03T03:22:54.809082+0000","last_fullsized":"2024-06-03T03:22:54.809082+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T14:08:52.666771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"99'5","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.825682+0000","last_change":"2024-06-03T03:19:41.608303+0000","last_active":"2024-06-03T03:22:54.825682+0000","last_peered":"2024-06-03T03:22:54.825682+0000","last_clean":"2024-06-03T03:22:54.825682+0000","last_became_active":"2024-06-03T03:19:41.607929+0000","last_became_peered":"2024-06-03T03:19:41.607929+0000","last_unstale":"2024-06-03T03:22:54.825682+0000","last_undegraded":"2024-06-03T03:22:54.825682+0000","last_fullsized":"2024-06-03T03:22:54.825682+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:40:05.643531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.822841+0000","last_change":"2024-06-03T03:19:41.606722+0000","last_active":"2024-06-03T03:22:54.822841+0000","last_peered":"2024-06-03T03:22:54.822841+0000","last_clean":"2024-06-03T03:22:54.822841+0000","last_became_active":"2024-06-03T03:19:41.606131+0000","last_became_peered":"2024-06-03T03:19:41.606131+0000","last_unstale":"2024-06-03T03:22:54.822841+0000","last_undegraded":"2024-06-03T03:22:54.822841+0000","last_fullsized":"2024-06-03T03:22:54.822841+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:09:17.306848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"99'11","reported_seq":51,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.813800+0000","last_change":"2024-06-03T03:19:41.609287+0000","last_active":"2024-06-03T03:22:54.813800+0000","last_peered":"2024-06-03T03:22:54.813800+0000","last_clean":"2024-06-03T03:22:54.813800+0000","last_became_active":"2024-06-03T03:19:41.608521+0000","last_became_peered":"2024-06-03T03:19:41.608521+0000","last_unstale":"2024-06-03T03:22:54.813800+0000","last_undegraded":"2024-06-03T03:22:54.813800+0000","last_fullsized":"2024-06-03T03:22:54.813800+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:35:37.128667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"99'16","reported_seq":56,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.814705+0000","last_change":"2024-06-03T03:19:41.605193+0000","last_active":"2024-06-03T03:22:54.814705+0000","last_peered":"2024-06-03T03:22:54.814705+0000","last_clean":"2024-06-03T03:22:54.814705+0000","last_became_active":"2024-06-03T03:19:41.604787+0000","last_became_peered":"2024-06-03T03:19:41.604787+0000","last_unstale":"2024-06-03T03:22:54.814705+0000","last_undegraded":"2024-06-03T03:22:54.814705+0000","last_fullsized":"2024-06-03T03:22:54.814705+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:02:48.334599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"99'6","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.817666+0000","last_change":"2024-06-03T03:19:41.611522+0000","last_active":"2024-06-03T03:22:54.817666+0000","last_peered":"2024-06-03T03:22:54.817666+0000","last_clean":"2024-06-03T03:22:54.817666+0000","last_became_active":"2024-06-03T03:19:41.610932+0000","last_became_peered":"2024-06-03T03:19:41.610932+0000","last_unstale":"2024-06-03T03:22:54.817666+0000","last_undegraded":"2024-06-03T03:22:54.817666+0000","last_fullsized":"2024-06-03T03:22:54.817666+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:35:44.702791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"99'7","reported_seq":24,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.820810+0000","last_change":"2024-06-03T03:20:25.950663+0000","last_active":"2024-06-03T03:22:54.820810+0000","last_peered":"2024-06-03T03:22:54.820810+0000","last_clean":"2024-06-03T03:22:54.820810+0000","last_became_active":"2024-06-03T03:20:25.949962+0000","last_became_peered":"2024-06-03T03:20:25.949962+0000","last_unstale":"2024-06-03T03:22:54.820810+0000","last_undegraded":"2024-06-03T03:22:54.820810+0000","last_fullsized":"2024-06-03T03:22:54.820810+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:55:33.972567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"99'10","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.839179+0000","last_change":"2024-06-03T03:19:41.633464+0000","last_active":"2024-06-03T03:22:54.839179+0000","last_peered":"2024-06-03T03:22:54.839179+0000","last_clean":"2024-06-03T03:22:54.839179+0000","last_became_active":"2024-06-03T03:19:41.630625+0000","last_became_peered":"2024-06-03T03:19:41.630625+0000","last_unstale":"2024-06-03T03:22:54.839179+0000","last_undegraded":"2024-06-03T03:22:54.839179+0000","last_fullsized":"2024-06-03T03:22:54.839179+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:23:28.799058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.831893+0000","last_change":"2024-06-03T03:19:41.627234+0000","last_active":"2024-06-03T03:22:54.831893+0000","last_peered":"2024-06-03T03:22:54.831893+0000","last_clean":"2024-06-03T03:22:54.831893+0000","last_became_active":"2024-06-03T03:19:41.626643+0000","last_became_peered":"2024-06-03T03:19:41.626643+0000","last_unstale":"2024-06-03T03:22:54.831893+0000","last_undegraded":"2024-06-03T03:22:54.831893+0000","last_fullsized":"2024-06-03T03:22:54.831893+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:37:44.126101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.831893+0000","last_change":"2024-06-03T03:19:41.621833+0000","last_active":"2024-06-03T03:22:54.831893+0000","last_peered":"2024-06-03T03:22:54.831893+0000","last_clean":"2024-06-03T03:22:54.831893+0000","last_became_active":"2024-06-03T03:19:41.620689+0000","last_became_peered":"2024-06-03T03:19:41.620689+0000","last_unstale":"2024-06-03T03:22:54.831893+0000","last_undegraded":"2024-06-03T03:22:54.831893+0000","last_fullsized":"2024-06-03T03:22:54.831893+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:39:31.188372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"99'9","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.830743+0000","last_change":"2024-06-03T03:19:41.624072+0000","last_active":"2024-06-03T03:22:54.830743+0000","last_peered":"2024-06-03T03:22:54.830743+0000","last_clean":"2024-06-03T03:22:54.830743+0000","last_became_active":"2024-06-03T03:19:41.623689+0000","last_became_peered":"2024-06-03T03:19:41.623689+0000","last_unstale":"2024-06-03T03:22:54.830743+0000","last_undegraded":"2024-06-03T03:22:54.830743+0000","last_fullsized":"2024-06-03T03:22:54.830743+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:38:24.075256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.833104+0000","last_change":"2024-06-03T03:19:41.626145+0000","last_active":"2024-06-03T03:22:54.833104+0000","last_peered":"2024-06-03T03:22:54.833104+0000","last_clean":"2024-06-03T03:22:54.833104+0000","last_became_active":"2024-06-03T03:19:41.625826+0000","last_became_peered":"2024-06-03T03:19:41.625826+0000","last_unstale":"2024-06-03T03:22:54.833104+0000","last_undegraded":"2024-06-03T03:22:54.833104+0000","last_fullsized":"2024-06-03T03:22:54.833104+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:04:10.996134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"99'10","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.821280+0000","last_change":"2024-06-03T03:19:41.615754+0000","last_active":"2024-06-03T03:22:54.821280+0000","last_peered":"2024-06-03T03:22:54.821280+0000","last_clean":"2024-06-03T03:22:54.821280+0000","last_became_active":"2024-06-03T03:19:41.614577+0000","last_became_peered":"2024-06-03T03:19:41.614577+0000","last_unstale":"2024-06-03T03:22:54.821280+0000","last_undegraded":"2024-06-03T03:22:54.821280+0000","last_fullsized":"2024-06-03T03:22:54.821280+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:45:50.084332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.826381+0000","last_change":"2024-06-03T03:19:41.633000+0000","last_active":"2024-06-03T03:22:54.826381+0000","last_peered":"2024-06-03T03:22:54.826381+0000","last_clean":"2024-06-03T03:22:54.826381+0000","last_became_active":"2024-06-03T03:19:41.632731+0000","last_became_peered":"2024-06-03T03:19:41.632731+0000","last_unstale":"2024-06-03T03:22:54.826381+0000","last_undegraded":"2024-06-03T03:22:54.826381+0000","last_fullsized":"2024-06-03T03:22:54.826381+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:10:29.990256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:24.128512+0000","last_change":"2024-06-03T03:20:26.133763+0000","last_active":"2024-06-03T03:22:24.128512+0000","last_peered":"2024-06-03T03:22:24.128512+0000","last_clean":"2024-06-03T03:22:24.128512+0000","last_became_active":"2024-06-03T03:20:26.133252+0000","last_became_peered":"2024-06-03T03:20:26.133252+0000","last_unstale":"2024-06-03T03:22:24.128512+0000","last_undegraded":"2024-06-03T03:22:24.128512+0000","last_fullsized":"2024-06-03T03:22:24.128512+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:33:21.461038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"99'14","reported_seq":309,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:25:14.520260+0000","last_change":"2024-06-03T03:19:36.542545+0000","last_active":"2024-06-03T03:25:14.520260+0000","last_peered":"2024-06-03T03:25:14.520260+0000","last_clean":"2024-06-03T03:25:14.520260+0000","last_became_active":"2024-06-03T03:19:36.541682+0000","last_became_peered":"2024-06-03T03:19:36.541682+0000","last_unstale":"2024-06-03T03:25:14.520260+0000","last_undegraded":"2024-06-03T03:25:14.520260+0000","last_fullsized":"2024-06-03T03:25:14.520260+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:06:23.909253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":249,"num_read_kb":249,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"99'5","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.819154+0000","last_change":"2024-06-03T03:19:41.605848+0000","last_active":"2024-06-03T03:22:54.819154+0000","last_peered":"2024-06-03T03:22:54.819154+0000","last_clean":"2024-06-03T03:22:54.819154+0000","last_became_active":"2024-06-03T03:19:41.603131+0000","last_became_peered":"2024-06-03T03:19:41.603131+0000","last_unstale":"2024-06-03T03:22:54.819154+0000","last_undegraded":"2024-06-03T03:22:54.819154+0000","last_fullsized":"2024-06-03T03:22:54.819154+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:07:38.931055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:22.310371+0000","last_change":"2024-06-03T03:19:56.923180+0000","last_active":"2024-06-03T03:22:22.310371+0000","last_peered":"2024-06-03T03:22:22.310371+0000","last_clean":"2024-06-03T03:22:22.310371+0000","last_became_active":"2024-06-03T03:19:56.922726+0000","last_became_peered":"2024-06-03T03:19:56.922726+0000","last_unstale":"2024-06-03T03:22:22.310371+0000","last_undegraded":"2024-06-03T03:22:22.310371+0000","last_fullsized":"2024-06-03T03:22:22.310371+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:18:45.727680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"97'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:36.766695+0000","last_change":"2024-06-03T03:19:36.746805+0000","last_active":"2024-06-03T03:21:36.766695+0000","last_peered":"2024-06-03T03:21:36.766695+0000","last_clean":"2024-06-03T03:21:36.766695+0000","last_became_active":"2024-06-03T03:19:36.745747+0000","last_became_peered":"2024-06-03T03:19:36.745747+0000","last_unstale":"2024-06-03T03:21:36.766695+0000","last_undegraded":"2024-06-03T03:21:36.766695+0000","last_fullsized":"2024-06-03T03:21:36.766695+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:54:28.427708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.832890+0000","last_change":"2024-06-03T03:19:41.630582+0000","last_active":"2024-06-03T03:22:54.832890+0000","last_peered":"2024-06-03T03:22:54.832890+0000","last_clean":"2024-06-03T03:22:54.832890+0000","last_became_active":"2024-06-03T03:19:41.630238+0000","last_became_peered":"2024-06-03T03:19:41.630238+0000","last_unstale":"2024-06-03T03:22:54.832890+0000","last_undegraded":"2024-06-03T03:22:54.832890+0000","last_fullsized":"2024-06-03T03:22:54.832890+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:49:58.039697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:27.311089+0000","last_change":"2024-06-03T03:19:56.925142+0000","last_active":"2024-06-03T03:22:27.311089+0000","last_peered":"2024-06-03T03:22:27.311089+0000","last_clean":"2024-06-03T03:22:27.311089+0000","last_became_active":"2024-06-03T03:19:56.924760+0000","last_became_peered":"2024-06-03T03:19:56.924760+0000","last_unstale":"2024-06-03T03:22:27.311089+0000","last_undegraded":"2024-06-03T03:22:27.311089+0000","last_fullsized":"2024-06-03T03:22:27.311089+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:11:27.537077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"97'12","reported_seq":58,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:09.897381+0000","last_change":"2024-06-03T03:19:36.543344+0000","last_active":"2024-06-03T03:22:09.897381+0000","last_peered":"2024-06-03T03:22:09.897381+0000","last_clean":"2024-06-03T03:22:09.897381+0000","last_became_active":"2024-06-03T03:19:36.542837+0000","last_became_peered":"2024-06-03T03:19:36.542837+0000","last_unstale":"2024-06-03T03:22:09.897381+0000","last_undegraded":"2024-06-03T03:22:09.897381+0000","last_fullsized":"2024-06-03T03:22:09.897381+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:55:05.451521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"99'9","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.813173+0000","last_change":"2024-06-03T03:19:41.629820+0000","last_active":"2024-06-03T03:22:54.813173+0000","last_peered":"2024-06-03T03:22:54.813173+0000","last_clean":"2024-06-03T03:22:54.813173+0000","last_became_active":"2024-06-03T03:19:41.629439+0000","last_became_peered":"2024-06-03T03:19:41.629439+0000","last_unstale":"2024-06-03T03:22:54.813173+0000","last_undegraded":"2024-06-03T03:22:54.813173+0000","last_fullsized":"2024-06-03T03:22:54.813173+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:05:27.438308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:27.311072+0000","last_change":"2024-06-03T03:19:56.923214+0000","last_active":"2024-06-03T03:22:27.311072+0000","last_peered":"2024-06-03T03:22:27.311072+0000","last_clean":"2024-06-03T03:22:27.311072+0000","last_became_active":"2024-06-03T03:19:56.922751+0000","last_became_peered":"2024-06-03T03:19:56.922751+0000","last_unstale":"2024-06-03T03:22:27.311072+0000","last_undegraded":"2024-06-03T03:22:27.311072+0000","last_fullsized":"2024-06-03T03:22:27.311072+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:05:59.249818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"97'5","reported_seq":51,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:09.897329+0000","last_change":"2024-06-03T03:19:36.538141+0000","last_active":"2024-06-03T03:22:09.897329+0000","last_peered":"2024-06-03T03:22:09.897329+0000","last_clean":"2024-06-03T03:22:09.897329+0000","last_became_active":"2024-06-03T03:19:36.537786+0000","last_became_peered":"2024-06-03T03:19:36.537786+0000","last_unstale":"2024-06-03T03:22:09.897329+0000","last_undegraded":"2024-06-03T03:22:09.897329+0000","last_fullsized":"2024-06-03T03:22:09.897329+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T15:03:32.859097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"99'15","reported_seq":55,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.816490+0000","last_change":"2024-06-03T03:19:41.608471+0000","last_active":"2024-06-03T03:22:54.816490+0000","last_peered":"2024-06-03T03:22:54.816490+0000","last_clean":"2024-06-03T03:22:54.816490+0000","last_became_active":"2024-06-03T03:19:41.608096+0000","last_became_peered":"2024-06-03T03:19:41.608096+0000","last_unstale":"2024-06-03T03:22:54.816490+0000","last_undegraded":"2024-06-03T03:22:54.816490+0000","last_fullsized":"2024-06-03T03:22:54.816490+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:31:10.541803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:29.129303+0000","last_change":"2024-06-03T03:19:56.915540+0000","last_active":"2024-06-03T03:22:29.129303+0000","last_peered":"2024-06-03T03:22:29.129303+0000","last_clean":"2024-06-03T03:22:29.129303+0000","last_became_active":"2024-06-03T03:19:56.915017+0000","last_became_peered":"2024-06-03T03:19:56.915017+0000","last_unstale":"2024-06-03T03:22:29.129303+0000","last_undegraded":"2024-06-03T03:22:29.129303+0000","last_fullsized":"2024-06-03T03:22:29.129303+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:56:23.158344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"99'9","reported_seq":310,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:25:15.590058+0000","last_change":"2024-06-03T03:19:36.542337+0000","last_active":"2024-06-03T03:25:15.590058+0000","last_peered":"2024-06-03T03:25:15.590058+0000","last_clean":"2024-06-03T03:25:15.590058+0000","last_became_active":"2024-06-03T03:19:36.541990+0000","last_became_peered":"2024-06-03T03:19:36.541990+0000","last_unstale":"2024-06-03T03:25:15.590058+0000","last_undegraded":"2024-06-03T03:25:15.590058+0000","last_fullsized":"2024-06-03T03:25:15.590058+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:08:15.656321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1849,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":255,"num_read_kb":6,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.815481+0000","last_change":"2024-06-03T03:19:41.616293+0000","last_active":"2024-06-03T03:22:54.815481+0000","last_peered":"2024-06-03T03:22:54.815481+0000","last_clean":"2024-06-03T03:22:54.815481+0000","last_became_active":"2024-06-03T03:19:41.615946+0000","last_became_peered":"2024-06-03T03:19:41.615946+0000","last_unstale":"2024-06-03T03:22:54.815481+0000","last_undegraded":"2024-06-03T03:22:54.815481+0000","last_fullsized":"2024-06-03T03:22:54.815481+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:18:43.835977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:29.129278+0000","last_change":"2024-06-03T03:20:26.146237+0000","last_active":"2024-06-03T03:22:29.129278+0000","last_peered":"2024-06-03T03:22:29.129278+0000","last_clean":"2024-06-03T03:22:29.129278+0000","last_became_active":"2024-06-03T03:20:26.145731+0000","last_became_peered":"2024-06-03T03:20:26.145731+0000","last_unstale":"2024-06-03T03:22:29.129278+0000","last_undegraded":"2024-06-03T03:22:29.129278+0000","last_fullsized":"2024-06-03T03:22:29.129278+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:15:29.723833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"97'7","reported_seq":53,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:07.058194+0000","last_change":"2024-06-03T03:19:36.546501+0000","last_active":"2024-06-03T03:22:07.058194+0000","last_peered":"2024-06-03T03:22:07.058194+0000","last_clean":"2024-06-03T03:22:07.058194+0000","last_became_active":"2024-06-03T03:19:36.546079+0000","last_became_peered":"2024-06-03T03:19:36.546079+0000","last_unstale":"2024-06-03T03:22:07.058194+0000","last_undegraded":"2024-06-03T03:22:07.058194+0000","last_fullsized":"2024-06-03T03:22:07.058194+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:16:39.473141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"99'8","reported_seq":54,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.829180+0000","last_change":"2024-06-03T03:20:26.149410+0000","last_active":"2024-06-03T03:22:54.829180+0000","last_peered":"2024-06-03T03:22:54.829180+0000","last_clean":"2024-06-03T03:22:54.829180+0000","last_became_active":"2024-06-03T03:20:26.149084+0000","last_became_peered":"2024-06-03T03:20:26.149084+0000","last_unstale":"2024-06-03T03:22:54.829180+0000","last_undegraded":"2024-06-03T03:22:54.829180+0000","last_fullsized":"2024-06-03T03:22:54.829180+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:06:22.316449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:04.314994+0000","last_change":"2024-06-03T03:19:56.926498+0000","last_active":"2024-06-03T03:22:04.314994+0000","last_peered":"2024-06-03T03:22:04.314994+0000","last_clean":"2024-06-03T03:22:04.314994+0000","last_became_active":"2024-06-03T03:19:56.926101+0000","last_became_peered":"2024-06-03T03:19:56.926101+0000","last_unstale":"2024-06-03T03:22:04.314994+0000","last_undegraded":"2024-06-03T03:22:04.314994+0000","last_fullsized":"2024-06-03T03:22:04.314994+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:00:12.024443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"97'3","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:03.279571+0000","last_change":"2024-06-03T03:19:36.553151+0000","last_active":"2024-06-03T03:22:03.279571+0000","last_peered":"2024-06-03T03:22:03.279571+0000","last_clean":"2024-06-03T03:22:03.279571+0000","last_became_active":"2024-06-03T03:19:36.551914+0000","last_became_peered":"2024-06-03T03:19:36.551914+0000","last_unstale":"2024-06-03T03:22:03.279571+0000","last_undegraded":"2024-06-03T03:22:03.279571+0000","last_fullsized":"2024-06-03T03:22:03.279571+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:00:42.053922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"99'8","reported_seq":25,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.824386+0000","last_change":"2024-06-03T03:20:26.147909+0000","last_active":"2024-06-03T03:22:54.824386+0000","last_peered":"2024-06-03T03:22:54.824386+0000","last_clean":"2024-06-03T03:22:54.824386+0000","last_became_active":"2024-06-03T03:20:26.146448+0000","last_became_peered":"2024-06-03T03:20:26.146448+0000","last_unstale":"2024-06-03T03:22:54.824386+0000","last_undegraded":"2024-06-03T03:22:54.824386+0000","last_fullsized":"2024-06-03T03:22:54.824386+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:43:30.878472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:59.843167+0000","last_change":"2024-06-03T03:19:56.941910+0000","last_active":"2024-06-03T03:21:59.843167+0000","last_peered":"2024-06-03T03:21:59.843167+0000","last_clean":"2024-06-03T03:21:59.843167+0000","last_became_active":"2024-06-03T03:19:56.940963+0000","last_became_peered":"2024-06-03T03:19:56.940963+0000","last_unstale":"2024-06-03T03:21:59.843167+0000","last_undegraded":"2024-06-03T03:21:59.843167+0000","last_fullsized":"2024-06-03T03:21:59.843167+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T14:18:32.109231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"97'3","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:02.447068+0000","last_change":"2024-06-03T03:19:36.538928+0000","last_active":"2024-06-03T03:22:02.447068+0000","last_peered":"2024-06-03T03:22:02.447068+0000","last_clean":"2024-06-03T03:22:02.447068+0000","last_became_active":"2024-06-03T03:19:36.538580+0000","last_became_peered":"2024-06-03T03:19:36.538580+0000","last_unstale":"2024-06-03T03:22:02.447068+0000","last_undegraded":"2024-06-03T03:22:02.447068+0000","last_fullsized":"2024-06-03T03:22:02.447068+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:08:05.906257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"99'11","reported_seq":51,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.829295+0000","last_change":"2024-06-03T03:19:41.623051+0000","last_active":"2024-06-03T03:22:54.829295+0000","last_peered":"2024-06-03T03:22:54.829295+0000","last_clean":"2024-06-03T03:22:54.829295+0000","last_became_active":"2024-06-03T03:19:41.622656+0000","last_became_peered":"2024-06-03T03:19:41.622656+0000","last_unstale":"2024-06-03T03:22:54.829295+0000","last_undegraded":"2024-06-03T03:22:54.829295+0000","last_fullsized":"2024-06-03T03:22:54.829295+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:11:34.587723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:16.773697+0000","last_change":"2024-06-03T03:19:56.921215+0000","last_active":"2024-06-03T03:22:16.773697+0000","last_peered":"2024-06-03T03:22:16.773697+0000","last_clean":"2024-06-03T03:22:16.773697+0000","last_became_active":"2024-06-03T03:19:56.920658+0000","last_became_peered":"2024-06-03T03:19:56.920658+0000","last_unstale":"2024-06-03T03:22:16.773697+0000","last_undegraded":"2024-06-03T03:22:16.773697+0000","last_fullsized":"2024-06-03T03:22:16.773697+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T14:04:54.163818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"99'7","reported_seq":551,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:25:16.411576+0000","last_change":"2024-06-03T03:19:36.545245+0000","last_active":"2024-06-03T03:25:16.411576+0000","last_peered":"2024-06-03T03:25:16.411576+0000","last_clean":"2024-06-03T03:25:16.411576+0000","last_became_active":"2024-06-03T03:19:36.544821+0000","last_became_peered":"2024-06-03T03:19:36.544821+0000","last_unstale":"2024-06-03T03:25:16.411576+0000","last_undegraded":"2024-06-03T03:25:16.411576+0000","last_fullsized":"2024-06-03T03:25:16.411576+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:55:24.011925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":498,"num_read_kb":498,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"99'10","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.822488+0000","last_change":"2024-06-03T03:19:41.616384+0000","last_active":"2024-06-03T03:22:54.822488+0000","last_peered":"2024-06-03T03:22:54.822488+0000","last_clean":"2024-06-03T03:22:54.822488+0000","last_became_active":"2024-06-03T03:19:41.616034+0000","last_became_peered":"2024-06-03T03:19:41.616034+0000","last_unstale":"2024-06-03T03:22:54.822488+0000","last_undegraded":"2024-06-03T03:22:54.822488+0000","last_fullsized":"2024-06-03T03:22:54.822488+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:10:06.823375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:20.886218+0000","last_change":"2024-06-03T03:19:56.933045+0000","last_active":"2024-06-03T03:22:20.886218+0000","last_peered":"2024-06-03T03:22:20.886218+0000","last_clean":"2024-06-03T03:22:20.886218+0000","last_became_active":"2024-06-03T03:19:56.928956+0000","last_became_peered":"2024-06-03T03:19:56.928956+0000","last_unstale":"2024-06-03T03:22:20.886218+0000","last_undegraded":"2024-06-03T03:22:20.886218+0000","last_fullsized":"2024-06-03T03:22:20.886218+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:37:44.270511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"97'4","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:04.852741+0000","last_change":"2024-06-03T03:19:36.541559+0000","last_active":"2024-06-03T03:22:04.852741+0000","last_peered":"2024-06-03T03:22:04.852741+0000","last_clean":"2024-06-03T03:22:04.852741+0000","last_became_active":"2024-06-03T03:19:36.541166+0000","last_became_peered":"2024-06-03T03:19:36.541166+0000","last_unstale":"2024-06-03T03:22:04.852741+0000","last_undegraded":"2024-06-03T03:22:04.852741+0000","last_fullsized":"2024-06-03T03:22:04.852741+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T14:49:26.847713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.833224+0000","last_change":"2024-06-03T03:19:41.629789+0000","last_active":"2024-06-03T03:22:54.833224+0000","last_peered":"2024-06-03T03:22:54.833224+0000","last_clean":"2024-06-03T03:22:54.833224+0000","last_became_active":"2024-06-03T03:19:41.628026+0000","last_became_peered":"2024-06-03T03:19:41.628026+0000","last_unstale":"2024-06-03T03:22:54.833224+0000","last_undegraded":"2024-06-03T03:22:54.833224+0000","last_fullsized":"2024-06-03T03:22:54.833224+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:56:26.411907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:17.309900+0000","last_change":"2024-06-03T03:21:26.056114+0000","last_active":"2024-06-03T03:22:17.309900+0000","last_peered":"2024-06-03T03:22:17.309900+0000","last_clean":"2024-06-03T03:22:17.309900+0000","last_became_active":"2024-06-03T03:21:26.055505+0000","last_became_peered":"2024-06-03T03:21:26.055505+0000","last_unstale":"2024-06-03T03:22:17.309900+0000","last_undegraded":"2024-06-03T03:22:17.309900+0000","last_fullsized":"2024-06-03T03:22:17.309900+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:22:55.722566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,10],"acting":[4,1,2,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"97'5","reported_seq":52,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:02.447113+0000","last_change":"2024-06-03T03:19:36.744930+0000","last_active":"2024-06-03T03:22:02.447113+0000","last_peered":"2024-06-03T03:22:02.447113+0000","last_clean":"2024-06-03T03:22:02.447113+0000","last_became_active":"2024-06-03T03:19:36.744567+0000","last_became_peered":"2024-06-03T03:19:36.744567+0000","last_unstale":"2024-06-03T03:22:02.447113+0000","last_undegraded":"2024-06-03T03:22:02.447113+0000","last_fullsized":"2024-06-03T03:22:02.447113+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:35:31.927211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"99'11","reported_seq":51,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.812800+0000","last_change":"2024-06-03T03:19:41.607604+0000","last_active":"2024-06-03T03:22:54.812800+0000","last_peered":"2024-06-03T03:22:54.812800+0000","last_clean":"2024-06-03T03:22:54.812800+0000","last_became_active":"2024-06-03T03:19:41.607148+0000","last_became_peered":"2024-06-03T03:19:41.607148+0000","last_unstale":"2024-06-03T03:22:54.812800+0000","last_undegraded":"2024-06-03T03:22:54.812800+0000","last_fullsized":"2024-06-03T03:22:54.812800+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:44:44.037836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:20.886272+0000","last_change":"2024-06-03T03:19:56.928689+0000","last_active":"2024-06-03T03:22:20.886272+0000","last_peered":"2024-06-03T03:22:20.886272+0000","last_clean":"2024-06-03T03:22:20.886272+0000","last_became_active":"2024-06-03T03:19:56.924955+0000","last_became_peered":"2024-06-03T03:19:56.924955+0000","last_unstale":"2024-06-03T03:22:20.886272+0000","last_undegraded":"2024-06-03T03:22:20.886272+0000","last_fullsized":"2024-06-03T03:22:20.886272+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:11:07.413430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"99'46","reported_seq":604,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:25:16.411088+0000","last_change":"2024-06-03T03:19:36.539688+0000","last_active":"2024-06-03T03:25:16.411088+0000","last_peered":"2024-06-03T03:25:16.411088+0000","last_clean":"2024-06-03T03:25:16.411088+0000","last_became_active":"2024-06-03T03:19:36.539302+0000","last_became_peered":"2024-06-03T03:19:36.539302+0000","last_unstale":"2024-06-03T03:25:16.411088+0000","last_undegraded":"2024-06-03T03:25:16.411088+0000","last_fullsized":"2024-06-03T03:25:16.411088+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:28:52.790227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":74214,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":512,"num_read_kb":329,"num_write":48,"num_write_kb":95,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.815494+0000","last_change":"2024-06-03T03:19:41.608202+0000","last_active":"2024-06-03T03:22:54.815494+0000","last_peered":"2024-06-03T03:22:54.815494+0000","last_clean":"2024-06-03T03:22:54.815494+0000","last_became_active":"2024-06-03T03:19:41.607891+0000","last_became_peered":"2024-06-03T03:19:41.607891+0000","last_unstale":"2024-06-03T03:22:54.815494+0000","last_undegraded":"2024-06-03T03:22:54.815494+0000","last_fullsized":"2024-06-03T03:22:54.815494+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T14:10:44.042892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:22.449231+0000","last_change":"2024-06-03T03:19:56.923547+0000","last_active":"2024-06-03T03:22:22.449231+0000","last_peered":"2024-06-03T03:22:22.449231+0000","last_clean":"2024-06-03T03:22:22.449231+0000","last_became_active":"2024-06-03T03:19:56.923046+0000","last_became_peered":"2024-06-03T03:19:56.923046+0000","last_unstale":"2024-06-03T03:22:22.449231+0000","last_undegraded":"2024-06-03T03:22:22.449231+0000","last_fullsized":"2024-06-03T03:22:22.449231+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:32:19.438793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"97'9","reported_seq":56,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:59.125620+0000","last_change":"2024-06-03T03:19:36.547024+0000","last_active":"2024-06-03T03:21:59.125620+0000","last_peered":"2024-06-03T03:21:59.125620+0000","last_clean":"2024-06-03T03:21:59.125620+0000","last_became_active":"2024-06-03T03:19:36.546663+0000","last_became_peered":"2024-06-03T03:19:36.546663+0000","last_unstale":"2024-06-03T03:21:59.125620+0000","last_undegraded":"2024-06-03T03:21:59.125620+0000","last_fullsized":"2024-06-03T03:21:59.125620+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:58:41.446730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.830570+0000","last_change":"2024-06-03T03:19:41.630119+0000","last_active":"2024-06-03T03:22:54.830570+0000","last_peered":"2024-06-03T03:22:54.830570+0000","last_clean":"2024-06-03T03:22:54.830570+0000","last_became_active":"2024-06-03T03:19:41.629563+0000","last_became_peered":"2024-06-03T03:19:41.629563+0000","last_unstale":"2024-06-03T03:22:54.830570+0000","last_undegraded":"2024-06-03T03:22:54.830570+0000","last_fullsized":"2024-06-03T03:22:54.830570+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:49:33.983471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:19.316793+0000","last_change":"2024-06-03T03:19:56.941451+0000","last_active":"2024-06-03T03:22:19.316793+0000","last_peered":"2024-06-03T03:22:19.316793+0000","last_clean":"2024-06-03T03:22:19.316793+0000","last_became_active":"2024-06-03T03:19:56.941003+0000","last_became_peered":"2024-06-03T03:19:56.941003+0000","last_unstale":"2024-06-03T03:22:19.316793+0000","last_undegraded":"2024-06-03T03:22:19.316793+0000","last_fullsized":"2024-06-03T03:22:19.316793+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:28:58.505495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"97'5","reported_seq":52,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:57.446530+0000","last_change":"2024-06-03T03:19:36.547128+0000","last_active":"2024-06-03T03:21:57.446530+0000","last_peered":"2024-06-03T03:21:57.446530+0000","last_clean":"2024-06-03T03:21:57.446530+0000","last_became_active":"2024-06-03T03:19:36.546576+0000","last_became_peered":"2024-06-03T03:19:36.546576+0000","last_unstale":"2024-06-03T03:21:57.446530+0000","last_undegraded":"2024-06-03T03:21:57.446530+0000","last_fullsized":"2024-06-03T03:21:57.446530+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:23:30.196543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"99'15","reported_seq":55,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.836544+0000","last_change":"2024-06-03T03:19:41.634941+0000","last_active":"2024-06-03T03:22:54.836544+0000","last_peered":"2024-06-03T03:22:54.836544+0000","last_clean":"2024-06-03T03:22:54.836544+0000","last_became_active":"2024-06-03T03:19:41.634561+0000","last_became_peered":"2024-06-03T03:19:41.634561+0000","last_unstale":"2024-06-03T03:22:54.836544+0000","last_undegraded":"2024-06-03T03:22:54.836544+0000","last_fullsized":"2024-06-03T03:22:54.836544+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:00:27.258078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:25.887154+0000","last_change":"2024-06-03T03:19:56.934612+0000","last_active":"2024-06-03T03:22:25.887154+0000","last_peered":"2024-06-03T03:22:25.887154+0000","last_clean":"2024-06-03T03:22:25.887154+0000","last_became_active":"2024-06-03T03:19:56.930212+0000","last_became_peered":"2024-06-03T03:19:56.930212+0000","last_unstale":"2024-06-03T03:22:25.887154+0000","last_undegraded":"2024-06-03T03:22:25.887154+0000","last_fullsized":"2024-06-03T03:22:25.887154+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:42:38.490095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"97'5","reported_seq":301,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:25:15.522909+0000","last_change":"2024-06-03T03:19:36.542052+0000","last_active":"2024-06-03T03:25:15.522909+0000","last_peered":"2024-06-03T03:25:15.522909+0000","last_clean":"2024-06-03T03:25:15.522909+0000","last_became_active":"2024-06-03T03:19:36.540987+0000","last_became_peered":"2024-06-03T03:19:36.540987+0000","last_unstale":"2024-06-03T03:25:15.522909+0000","last_undegraded":"2024-06-03T03:25:15.522909+0000","last_fullsized":"2024-06-03T03:25:15.522909+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T15:18:48.821400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":252,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"99'6","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.818326+0000","last_change":"2024-06-03T03:19:41.607610+0000","last_active":"2024-06-03T03:22:54.818326+0000","last_peered":"2024-06-03T03:22:54.818326+0000","last_clean":"2024-06-03T03:22:54.818326+0000","last_became_active":"2024-06-03T03:19:41.607219+0000","last_became_peered":"2024-06-03T03:19:41.607219+0000","last_unstale":"2024-06-03T03:22:54.818326+0000","last_undegraded":"2024-06-03T03:22:54.818326+0000","last_fullsized":"2024-06-03T03:22:54.818326+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:06:20.707623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:23.282636+0000","last_change":"2024-06-03T03:19:56.936440+0000","last_active":"2024-06-03T03:22:23.282636+0000","last_peered":"2024-06-03T03:22:23.282636+0000","last_clean":"2024-06-03T03:22:23.282636+0000","last_became_active":"2024-06-03T03:19:56.936048+0000","last_became_peered":"2024-06-03T03:19:56.936048+0000","last_unstale":"2024-06-03T03:22:23.282636+0000","last_undegraded":"2024-06-03T03:22:23.282636+0000","last_fullsized":"2024-06-03T03:22:23.282636+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:22:15.758950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"97'17","reported_seq":63,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:59.896170+0000","last_change":"2024-06-03T03:19:36.736105+0000","last_active":"2024-06-03T03:21:59.896170+0000","last_peered":"2024-06-03T03:21:59.896170+0000","last_clean":"2024-06-03T03:21:59.896170+0000","last_became_active":"2024-06-03T03:19:36.735663+0000","last_became_peered":"2024-06-03T03:19:36.735663+0000","last_unstale":"2024-06-03T03:21:59.896170+0000","last_undegraded":"2024-06-03T03:21:59.896170+0000","last_fullsized":"2024-06-03T03:21:59.896170+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:41:46.216923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"99'6","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.814118+0000","last_change":"2024-06-03T03:19:41.609758+0000","last_active":"2024-06-03T03:22:54.814118+0000","last_peered":"2024-06-03T03:22:54.814118+0000","last_clean":"2024-06-03T03:22:54.814118+0000","last_became_active":"2024-06-03T03:19:41.609421+0000","last_became_peered":"2024-06-03T03:19:41.609421+0000","last_unstale":"2024-06-03T03:22:54.814118+0000","last_undegraded":"2024-06-03T03:22:54.814118+0000","last_fullsized":"2024-06-03T03:22:54.814118+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:45:54.919352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:07.447579+0000","last_change":"2024-06-03T03:19:56.925422+0000","last_active":"2024-06-03T03:22:07.447579+0000","last_peered":"2024-06-03T03:22:07.447579+0000","last_clean":"2024-06-03T03:22:07.447579+0000","last_became_active":"2024-06-03T03:19:56.924891+0000","last_became_peered":"2024-06-03T03:19:56.924891+0000","last_unstale":"2024-06-03T03:22:07.447579+0000","last_undegraded":"2024-06-03T03:22:07.447579+0000","last_fullsized":"2024-06-03T03:22:07.447579+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:34:59.986256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"97'2","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:54.895740+0000","last_change":"2024-06-03T03:19:36.540605+0000","last_active":"2024-06-03T03:21:54.895740+0000","last_peered":"2024-06-03T03:21:54.895740+0000","last_clean":"2024-06-03T03:21:54.895740+0000","last_became_active":"2024-06-03T03:19:36.540180+0000","last_became_peered":"2024-06-03T03:19:36.540180+0000","last_unstale":"2024-06-03T03:21:54.895740+0000","last_undegraded":"2024-06-03T03:21:54.895740+0000","last_fullsized":"2024-06-03T03:21:54.895740+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:01:26.556726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"99'5","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.825730+0000","last_change":"2024-06-03T03:20:25.904215+0000","last_active":"2024-06-03T03:22:54.825730+0000","last_peered":"2024-06-03T03:22:54.825730+0000","last_clean":"2024-06-03T03:22:54.825730+0000","last_became_active":"2024-06-03T03:20:25.903894+0000","last_became_peered":"2024-06-03T03:20:25.903894+0000","last_unstale":"2024-06-03T03:22:54.825730+0000","last_undegraded":"2024-06-03T03:22:54.825730+0000","last_fullsized":"2024-06-03T03:22:54.825730+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:02:07.587159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:04.896819+0000","last_change":"2024-06-03T03:19:56.910041+0000","last_active":"2024-06-03T03:22:04.896819+0000","last_peered":"2024-06-03T03:22:04.896819+0000","last_clean":"2024-06-03T03:22:04.896819+0000","last_became_active":"2024-06-03T03:19:56.909591+0000","last_became_peered":"2024-06-03T03:19:56.909591+0000","last_unstale":"2024-06-03T03:22:04.896819+0000","last_undegraded":"2024-06-03T03:22:04.896819+0000","last_fullsized":"2024-06-03T03:22:04.896819+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:42:20.016838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"97'1","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:40.880350+0000","last_change":"2024-06-03T03:19:36.741731+0000","last_active":"2024-06-03T03:21:40.880350+0000","last_peered":"2024-06-03T03:21:40.880350+0000","last_clean":"2024-06-03T03:21:40.880350+0000","last_became_active":"2024-06-03T03:19:36.741329+0000","last_became_peered":"2024-06-03T03:19:36.741329+0000","last_unstale":"2024-06-03T03:21:40.880350+0000","last_undegraded":"2024-06-03T03:21:40.880350+0000","last_fullsized":"2024-06-03T03:21:40.880350+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:02:45.778368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"99'4","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.825952+0000","last_change":"2024-06-03T03:19:41.632225+0000","last_active":"2024-06-03T03:22:54.825952+0000","last_peered":"2024-06-03T03:22:54.825952+0000","last_clean":"2024-06-03T03:22:54.825952+0000","last_became_active":"2024-06-03T03:19:41.631851+0000","last_became_peered":"2024-06-03T03:19:41.631851+0000","last_unstale":"2024-06-03T03:22:54.825952+0000","last_undegraded":"2024-06-03T03:22:54.825952+0000","last_fullsized":"2024-06-03T03:22:54.825952+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:52:23.364233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:03.279638+0000","last_change":"2024-06-03T03:19:56.934200+0000","last_active":"2024-06-03T03:22:03.279638+0000","last_peered":"2024-06-03T03:22:03.279638+0000","last_clean":"2024-06-03T03:22:03.279638+0000","last_became_active":"2024-06-03T03:19:56.932998+0000","last_became_peered":"2024-06-03T03:19:56.932998+0000","last_unstale":"2024-06-03T03:22:03.279638+0000","last_undegraded":"2024-06-03T03:22:03.279638+0000","last_fullsized":"2024-06-03T03:22:03.279638+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:45:28.916023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"99'6","reported_seq":301,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:25:13.519391+0000","last_change":"2024-06-03T03:19:36.546630+0000","last_active":"2024-06-03T03:25:13.519391+0000","last_peered":"2024-06-03T03:25:13.519391+0000","last_clean":"2024-06-03T03:25:13.519391+0000","last_became_active":"2024-06-03T03:19:36.546248+0000","last_became_peered":"2024-06-03T03:19:36.546248+0000","last_unstale":"2024-06-03T03:25:13.519391+0000","last_undegraded":"2024-06-03T03:25:13.519391+0000","last_fullsized":"2024-06-03T03:25:13.519391+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T15:09:50.323113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":249,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"99'7","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.831073+0000","last_change":"2024-06-03T03:19:41.624584+0000","last_active":"2024-06-03T03:22:54.831073+0000","last_peered":"2024-06-03T03:22:54.831073+0000","last_clean":"2024-06-03T03:22:54.831073+0000","last_became_active":"2024-06-03T03:19:41.624216+0000","last_became_peered":"2024-06-03T03:19:41.624216+0000","last_unstale":"2024-06-03T03:22:54.831073+0000","last_undegraded":"2024-06-03T03:22:54.831073+0000","last_fullsized":"2024-06-03T03:22:54.831073+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:22:09.645341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:08.280228+0000","last_change":"2024-06-03T03:19:56.937422+0000","last_active":"2024-06-03T03:22:08.280228+0000","last_peered":"2024-06-03T03:22:08.280228+0000","last_clean":"2024-06-03T03:22:08.280228+0000","last_became_active":"2024-06-03T03:19:56.934075+0000","last_became_peered":"2024-06-03T03:19:56.934075+0000","last_unstale":"2024-06-03T03:22:08.280228+0000","last_undegraded":"2024-06-03T03:22:08.280228+0000","last_fullsized":"2024-06-03T03:22:08.280228+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:25:45.273161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"99'8","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.814169+0000","last_change":"2024-06-03T03:19:41.589595+0000","last_active":"2024-06-03T03:22:54.814169+0000","last_peered":"2024-06-03T03:22:54.814169+0000","last_clean":"2024-06-03T03:22:54.814169+0000","last_became_active":"2024-06-03T03:19:41.589251+0000","last_became_peered":"2024-06-03T03:19:41.589251+0000","last_unstale":"2024-06-03T03:22:54.814169+0000","last_undegraded":"2024-06-03T03:22:54.814169+0000","last_fullsized":"2024-06-03T03:22:54.814169+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:51:40.417386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"97'5","reported_seq":301,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:25:16.524308+0000","last_change":"2024-06-03T03:19:36.737523+0000","last_active":"2024-06-03T03:25:16.524308+0000","last_peered":"2024-06-03T03:25:16.524308+0000","last_clean":"2024-06-03T03:25:16.524308+0000","last_became_active":"2024-06-03T03:19:36.736698+0000","last_became_peered":"2024-06-03T03:19:36.736698+0000","last_unstale":"2024-06-03T03:25:16.524308+0000","last_undegraded":"2024-06-03T03:25:16.524308+0000","last_fullsized":"2024-06-03T03:25:16.524308+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:14:46.330549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":251,"num_read_kb":251,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"97'1","reported_seq":31,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:04.852726+0000","last_change":"2024-06-03T03:19:56.924421+0000","last_active":"2024-06-03T03:22:04.852726+0000","last_peered":"2024-06-03T03:22:04.852726+0000","last_clean":"2024-06-03T03:22:04.852726+0000","last_became_active":"2024-06-03T03:19:56.924043+0000","last_became_peered":"2024-06-03T03:19:56.924043+0000","last_unstale":"2024-06-03T03:22:04.852726+0000","last_undegraded":"2024-06-03T03:22:04.852726+0000","last_fullsized":"2024-06-03T03:22:04.852726+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:00:47.674717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.826839+0000","last_change":"2024-06-03T03:19:41.613800+0000","last_active":"2024-06-03T03:22:54.826839+0000","last_peered":"2024-06-03T03:22:54.826839+0000","last_clean":"2024-06-03T03:22:54.826839+0000","last_became_active":"2024-06-03T03:19:41.613387+0000","last_became_peered":"2024-06-03T03:19:41.613387+0000","last_unstale":"2024-06-03T03:22:54.826839+0000","last_undegraded":"2024-06-03T03:22:54.826839+0000","last_fullsized":"2024-06-03T03:22:54.826839+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:04:04.785023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"97'6","reported_seq":54,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:42.305912+0000","last_change":"2024-06-03T03:19:36.544518+0000","last_active":"2024-06-03T03:21:42.305912+0000","last_peered":"2024-06-03T03:21:42.305912+0000","last_clean":"2024-06-03T03:21:42.305912+0000","last_became_active":"2024-06-03T03:19:36.544056+0000","last_became_peered":"2024-06-03T03:19:36.544056+0000","last_unstale":"2024-06-03T03:21:42.305912+0000","last_undegraded":"2024-06-03T03:21:42.305912+0000","last_fullsized":"2024-06-03T03:21:42.305912+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:33:25.012986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":368,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:39.856348+0000","last_change":"2024-06-03T03:18:05.481351+0000","last_active":"2024-06-03T03:22:39.856348+0000","last_peered":"2024-06-03T03:22:39.856348+0000","last_clean":"2024-06-03T03:22:39.856348+0000","last_became_active":"2024-06-03T03:18:05.480929+0000","last_became_peered":"2024-06-03T03:18:05.480929+0000","last_unstale":"2024-06-03T03:22:39.856348+0000","last_undegraded":"2024-06-03T03:22:39.856348+0000","last_fullsized":"2024-06-03T03:22:39.856348+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:14:09.558597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:14:09.558597+0000","last_clean_scrub_stamp":"2024-06-03T03:14:09.558597+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:54:45.563733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:07.447611+0000","last_change":"2024-06-03T03:19:56.931867+0000","last_active":"2024-06-03T03:22:07.447611+0000","last_peered":"2024-06-03T03:22:07.447611+0000","last_clean":"2024-06-03T03:22:07.447611+0000","last_became_active":"2024-06-03T03:19:56.931538+0000","last_became_peered":"2024-06-03T03:19:56.931538+0000","last_unstale":"2024-06-03T03:22:07.447611+0000","last_undegraded":"2024-06-03T03:22:07.447611+0000","last_fullsized":"2024-06-03T03:22:07.447611+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:55:15.144192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"99'5","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.814110+0000","last_change":"2024-06-03T03:19:41.622713+0000","last_active":"2024-06-03T03:22:54.814110+0000","last_peered":"2024-06-03T03:22:54.814110+0000","last_clean":"2024-06-03T03:22:54.814110+0000","last_became_active":"2024-06-03T03:19:41.622363+0000","last_became_peered":"2024-06-03T03:19:41.622363+0000","last_unstale":"2024-06-03T03:22:54.814110+0000","last_undegraded":"2024-06-03T03:22:54.814110+0000","last_fullsized":"2024-06-03T03:22:54.814110+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:43:57.671797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"97'4","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:39.850108+0000","last_change":"2024-06-03T03:19:36.539663+0000","last_active":"2024-06-03T03:21:39.850108+0000","last_peered":"2024-06-03T03:21:39.850108+0000","last_clean":"2024-06-03T03:21:39.850108+0000","last_became_active":"2024-06-03T03:19:36.539320+0000","last_became_peered":"2024-06-03T03:19:36.539320+0000","last_unstale":"2024-06-03T03:21:39.850108+0000","last_undegraded":"2024-06-03T03:21:39.850108+0000","last_fullsized":"2024-06-03T03:21:39.850108+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:36:50.699231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:04.314937+0000","last_change":"2024-06-03T03:19:56.938231+0000","last_active":"2024-06-03T03:22:04.314937+0000","last_peered":"2024-06-03T03:22:04.314937+0000","last_clean":"2024-06-03T03:22:04.314937+0000","last_became_active":"2024-06-03T03:19:56.937840+0000","last_became_peered":"2024-06-03T03:19:56.937840+0000","last_unstale":"2024-06-03T03:22:04.314937+0000","last_undegraded":"2024-06-03T03:22:04.314937+0000","last_fullsized":"2024-06-03T03:22:04.314937+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:02:33.916980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"97'4","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:44.894507+0000","last_change":"2024-06-03T03:19:36.737519+0000","last_active":"2024-06-03T03:21:44.894507+0000","last_peered":"2024-06-03T03:21:44.894507+0000","last_clean":"2024-06-03T03:21:44.894507+0000","last_became_active":"2024-06-03T03:19:36.737191+0000","last_became_peered":"2024-06-03T03:19:36.737191+0000","last_unstale":"2024-06-03T03:21:44.894507+0000","last_undegraded":"2024-06-03T03:21:44.894507+0000","last_fullsized":"2024-06-03T03:21:44.894507+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:50:21.389739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.817624+0000","last_change":"2024-06-03T03:19:41.633113+0000","last_active":"2024-06-03T03:22:54.817624+0000","last_peered":"2024-06-03T03:22:54.817624+0000","last_clean":"2024-06-03T03:22:54.817624+0000","last_became_active":"2024-06-03T03:19:41.632186+0000","last_became_peered":"2024-06-03T03:19:41.632186+0000","last_unstale":"2024-06-03T03:22:54.817624+0000","last_undegraded":"2024-06-03T03:22:54.817624+0000","last_fullsized":"2024-06-03T03:22:54.817624+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T15:12:26.835061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:56.770202+0000","last_change":"2024-06-03T03:19:56.928603+0000","last_active":"2024-06-03T03:21:56.770202+0000","last_peered":"2024-06-03T03:21:56.770202+0000","last_clean":"2024-06-03T03:21:56.770202+0000","last_became_active":"2024-06-03T03:19:56.928175+0000","last_became_peered":"2024-06-03T03:19:56.928175+0000","last_unstale":"2024-06-03T03:21:56.770202+0000","last_undegraded":"2024-06-03T03:21:56.770202+0000","last_fullsized":"2024-06-03T03:21:56.770202+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:26:03.531091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"99'21","reported_seq":67,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:25:10.676315+0000","last_change":"2024-06-03T03:19:36.550577+0000","last_active":"2024-06-03T03:25:10.676315+0000","last_peered":"2024-06-03T03:25:10.676315+0000","last_clean":"2024-06-03T03:25:10.676315+0000","last_became_active":"2024-06-03T03:19:36.549968+0000","last_became_peered":"2024-06-03T03:19:36.549968+0000","last_unstale":"2024-06-03T03:25:10.676315+0000","last_undegraded":"2024-06-03T03:25:10.676315+0000","last_fullsized":"2024-06-03T03:25:10.676315+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:22:43.892002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":38,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"99'5","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.825483+0000","last_change":"2024-06-03T03:19:41.613883+0000","last_active":"2024-06-03T03:22:54.825483+0000","last_peered":"2024-06-03T03:22:54.825483+0000","last_clean":"2024-06-03T03:22:54.825483+0000","last_became_active":"2024-06-03T03:19:41.613472+0000","last_became_peered":"2024-06-03T03:19:41.613472+0000","last_unstale":"2024-06-03T03:22:54.825483+0000","last_undegraded":"2024-06-03T03:22:54.825483+0000","last_fullsized":"2024-06-03T03:22:54.825483+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T14:38:42.044534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:04.126263+0000","last_change":"2024-06-03T03:19:56.914487+0000","last_active":"2024-06-03T03:22:04.126263+0000","last_peered":"2024-06-03T03:22:04.126263+0000","last_clean":"2024-06-03T03:22:04.126263+0000","last_became_active":"2024-06-03T03:19:56.913904+0000","last_became_peered":"2024-06-03T03:19:56.913904+0000","last_unstale":"2024-06-03T03:22:04.126263+0000","last_undegraded":"2024-06-03T03:22:04.126263+0000","last_fullsized":"2024-06-03T03:22:04.126263+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T15:13:24.084034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"97'7","reported_seq":53,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:47.306495+0000","last_change":"2024-06-03T03:19:36.671355+0000","last_active":"2024-06-03T03:21:47.306495+0000","last_peered":"2024-06-03T03:21:47.306495+0000","last_clean":"2024-06-03T03:21:47.306495+0000","last_became_active":"2024-06-03T03:19:36.670926+0000","last_became_peered":"2024-06-03T03:19:36.670926+0000","last_unstale":"2024-06-03T03:21:47.306495+0000","last_undegraded":"2024-06-03T03:21:47.306495+0000","last_fullsized":"2024-06-03T03:21:47.306495+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:38:50.914238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.835937+0000","last_change":"2024-06-03T03:19:41.629054+0000","last_active":"2024-06-03T03:22:54.835937+0000","last_peered":"2024-06-03T03:22:54.835937+0000","last_clean":"2024-06-03T03:22:54.835937+0000","last_became_active":"2024-06-03T03:19:41.627118+0000","last_became_peered":"2024-06-03T03:19:41.627118+0000","last_unstale":"2024-06-03T03:22:54.835937+0000","last_undegraded":"2024-06-03T03:22:54.835937+0000","last_fullsized":"2024-06-03T03:22:54.835937+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:59:40.645002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:12.309263+0000","last_change":"2024-06-03T03:21:26.058741+0000","last_active":"2024-06-03T03:22:12.309263+0000","last_peered":"2024-06-03T03:22:12.309263+0000","last_clean":"2024-06-03T03:22:12.309263+0000","last_became_active":"2024-06-03T03:21:26.058271+0000","last_became_peered":"2024-06-03T03:21:26.058271+0000","last_unstale":"2024-06-03T03:22:12.309263+0000","last_undegraded":"2024-06-03T03:22:12.309263+0000","last_fullsized":"2024-06-03T03:22:12.309263+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:31:32.359893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"97'2","reported_seq":54,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:50.231392+0000","last_change":"2024-06-03T03:20:26.132994+0000","last_active":"2024-06-03T03:21:50.231392+0000","last_peered":"2024-06-03T03:21:50.231392+0000","last_clean":"2024-06-03T03:21:50.231392+0000","last_became_active":"2024-06-03T03:20:26.132574+0000","last_became_peered":"2024-06-03T03:20:26.132574+0000","last_unstale":"2024-06-03T03:21:50.231392+0000","last_undegraded":"2024-06-03T03:21:50.231392+0000","last_fullsized":"2024-06-03T03:21:50.231392+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T15:12:33.599154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"99'2","reported_seq":41,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.791970+0000","last_change":"2024-06-03T03:19:41.603630+0000","last_active":"2024-06-03T03:22:54.791970+0000","last_peered":"2024-06-03T03:22:54.791970+0000","last_clean":"2024-06-03T03:22:54.791970+0000","last_became_active":"2024-06-03T03:19:41.603301+0000","last_became_peered":"2024-06-03T03:19:41.603301+0000","last_unstale":"2024-06-03T03:22:54.791970+0000","last_undegraded":"2024-06-03T03:22:54.791970+0000","last_fullsized":"2024-06-03T03:22:54.791970+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:20:33.076622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:15.885256+0000","last_change":"2024-06-03T03:19:56.929023+0000","last_active":"2024-06-03T03:22:15.885256+0000","last_peered":"2024-06-03T03:22:15.885256+0000","last_clean":"2024-06-03T03:22:15.885256+0000","last_became_active":"2024-06-03T03:19:56.928454+0000","last_became_peered":"2024-06-03T03:19:56.928454+0000","last_unstale":"2024-06-03T03:22:15.885256+0000","last_undegraded":"2024-06-03T03:22:15.885256+0000","last_fullsized":"2024-06-03T03:22:15.885256+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:17:31.755649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"97'7","reported_seq":52,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:46.768385+0000","last_change":"2024-06-03T03:19:36.551413+0000","last_active":"2024-06-03T03:21:46.768385+0000","last_peered":"2024-06-03T03:21:46.768385+0000","last_clean":"2024-06-03T03:21:46.768385+0000","last_became_active":"2024-06-03T03:19:36.550622+0000","last_became_peered":"2024-06-03T03:19:36.550622+0000","last_unstale":"2024-06-03T03:21:46.768385+0000","last_undegraded":"2024-06-03T03:21:46.768385+0000","last_fullsized":"2024-06-03T03:21:46.768385+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:50:09.472960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.830703+0000","last_change":"2024-06-03T03:19:41.624871+0000","last_active":"2024-06-03T03:22:54.830703+0000","last_peered":"2024-06-03T03:22:54.830703+0000","last_clean":"2024-06-03T03:22:54.830703+0000","last_became_active":"2024-06-03T03:19:41.624638+0000","last_became_peered":"2024-06-03T03:19:41.624638+0000","last_unstale":"2024-06-03T03:22:54.830703+0000","last_undegraded":"2024-06-03T03:22:54.830703+0000","last_fullsized":"2024-06-03T03:22:54.830703+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:21:16.193817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"99'1","reported_seq":31,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:12.309224+0000","last_change":"2024-06-03T03:19:56.924638+0000","last_active":"2024-06-03T03:22:12.309224+0000","last_peered":"2024-06-03T03:22:12.309224+0000","last_clean":"2024-06-03T03:22:12.309224+0000","last_became_active":"2024-06-03T03:19:56.924200+0000","last_became_peered":"2024-06-03T03:19:56.924200+0000","last_unstale":"2024-06-03T03:22:12.309224+0000","last_undegraded":"2024-06-03T03:22:12.309224+0000","last_fullsized":"2024-06-03T03:22:12.309224+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:38:01.138253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"99'5","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:25:11.017683+0000","last_change":"2024-06-03T03:19:36.546974+0000","last_active":"2024-06-03T03:25:11.017683+0000","last_peered":"2024-06-03T03:25:11.017683+0000","last_clean":"2024-06-03T03:25:11.017683+0000","last_became_active":"2024-06-03T03:19:36.546433+0000","last_became_peered":"2024-06-03T03:19:36.546433+0000","last_unstale":"2024-06-03T03:25:11.017683+0000","last_undegraded":"2024-06-03T03:25:11.017683+0000","last_fullsized":"2024-06-03T03:25:11.017683+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:41:46.980327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.834248+0000","last_change":"2024-06-03T03:19:41.622118+0000","last_active":"2024-06-03T03:22:54.834248+0000","last_peered":"2024-06-03T03:22:54.834248+0000","last_clean":"2024-06-03T03:22:54.834248+0000","last_became_active":"2024-06-03T03:19:41.620789+0000","last_became_peered":"2024-06-03T03:19:41.620789+0000","last_unstale":"2024-06-03T03:22:54.834248+0000","last_undegraded":"2024-06-03T03:22:54.834248+0000","last_fullsized":"2024-06-03T03:22:54.834248+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:00:36.232238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:12.309247+0000","last_change":"2024-06-03T03:19:56.921299+0000","last_active":"2024-06-03T03:22:12.309247+0000","last_peered":"2024-06-03T03:22:12.309247+0000","last_clean":"2024-06-03T03:22:12.309247+0000","last_became_active":"2024-06-03T03:19:56.920870+0000","last_became_peered":"2024-06-03T03:19:56.920870+0000","last_unstale":"2024-06-03T03:22:12.309247+0000","last_undegraded":"2024-06-03T03:22:12.309247+0000","last_fullsized":"2024-06-03T03:22:12.309247+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:10:15.062381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"97'4","reported_seq":24,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:44.319531+0000","last_change":"2024-06-03T03:20:25.903279+0000","last_active":"2024-06-03T03:22:44.319531+0000","last_peered":"2024-06-03T03:22:44.319531+0000","last_clean":"2024-06-03T03:22:44.319531+0000","last_became_active":"2024-06-03T03:20:25.900311+0000","last_became_peered":"2024-06-03T03:20:25.900311+0000","last_unstale":"2024-06-03T03:22:44.319531+0000","last_undegraded":"2024-06-03T03:22:44.319531+0000","last_fullsized":"2024-06-03T03:22:44.319531+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:25:49.708581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"99'5","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.836223+0000","last_change":"2024-06-03T03:19:41.628659+0000","last_active":"2024-06-03T03:22:54.836223+0000","last_peered":"2024-06-03T03:22:54.836223+0000","last_clean":"2024-06-03T03:22:54.836223+0000","last_became_active":"2024-06-03T03:19:41.628291+0000","last_became_peered":"2024-06-03T03:19:41.628291+0000","last_unstale":"2024-06-03T03:22:54.836223+0000","last_undegraded":"2024-06-03T03:22:54.836223+0000","last_fullsized":"2024-06-03T03:22:54.836223+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:26:49.579669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:10.233742+0000","last_change":"2024-06-03T03:19:56.922766+0000","last_active":"2024-06-03T03:22:10.233742+0000","last_peered":"2024-06-03T03:22:10.233742+0000","last_clean":"2024-06-03T03:22:10.233742+0000","last_became_active":"2024-06-03T03:19:56.922310+0000","last_became_peered":"2024-06-03T03:19:56.922310+0000","last_unstale":"2024-06-03T03:22:10.233742+0000","last_undegraded":"2024-06-03T03:22:10.233742+0000","last_fullsized":"2024-06-03T03:22:10.233742+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T14:35:53.133519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"99'6","reported_seq":275,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:25:15.410739+0000","last_change":"2024-06-03T03:20:26.133562+0000","last_active":"2024-06-03T03:25:15.410739+0000","last_peered":"2024-06-03T03:25:15.410739+0000","last_clean":"2024-06-03T03:25:15.410739+0000","last_became_active":"2024-06-03T03:20:26.133074+0000","last_became_peered":"2024-06-03T03:20:26.133074+0000","last_unstale":"2024-06-03T03:25:15.410739+0000","last_undegraded":"2024-06-03T03:25:15.410739+0000","last_fullsized":"2024-06-03T03:25:15.410739+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:51:33.717584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1560,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":252,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"99'7","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.818537+0000","last_change":"2024-06-03T03:19:41.611450+0000","last_active":"2024-06-03T03:22:54.818537+0000","last_peered":"2024-06-03T03:22:54.818537+0000","last_clean":"2024-06-03T03:22:54.818537+0000","last_became_active":"2024-06-03T03:19:41.610214+0000","last_became_peered":"2024-06-03T03:19:41.610214+0000","last_unstale":"2024-06-03T03:22:54.818537+0000","last_undegraded":"2024-06-03T03:22:54.818537+0000","last_fullsized":"2024-06-03T03:22:54.818537+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:01:01.521917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:14.845604+0000","last_change":"2024-06-03T03:19:56.935686+0000","last_active":"2024-06-03T03:22:14.845604+0000","last_peered":"2024-06-03T03:22:14.845604+0000","last_clean":"2024-06-03T03:22:14.845604+0000","last_became_active":"2024-06-03T03:19:56.935314+0000","last_became_peered":"2024-06-03T03:19:56.935314+0000","last_unstale":"2024-06-03T03:22:14.845604+0000","last_undegraded":"2024-06-03T03:22:14.845604+0000","last_fullsized":"2024-06-03T03:22:14.845604+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T15:10:16.966321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"97'3","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:52.445989+0000","last_change":"2024-06-03T03:19:36.547756+0000","last_active":"2024-06-03T03:21:52.445989+0000","last_peered":"2024-06-03T03:21:52.445989+0000","last_clean":"2024-06-03T03:21:52.445989+0000","last_became_active":"2024-06-03T03:19:36.547350+0000","last_became_peered":"2024-06-03T03:19:36.547350+0000","last_unstale":"2024-06-03T03:21:52.445989+0000","last_undegraded":"2024-06-03T03:21:52.445989+0000","last_fullsized":"2024-06-03T03:21:52.445989+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:57:39.214380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"99'6","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.828041+0000","last_change":"2024-06-03T03:19:41.621083+0000","last_active":"2024-06-03T03:22:54.828041+0000","last_peered":"2024-06-03T03:22:54.828041+0000","last_clean":"2024-06-03T03:22:54.828041+0000","last_became_active":"2024-06-03T03:19:41.620719+0000","last_became_peered":"2024-06-03T03:19:41.620719+0000","last_unstale":"2024-06-03T03:22:54.828041+0000","last_undegraded":"2024-06-03T03:22:54.828041+0000","last_fullsized":"2024-06-03T03:22:54.828041+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:50:35.007025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:07.447552+0000","last_change":"2024-06-03T03:19:56.915406+0000","last_active":"2024-06-03T03:22:07.447552+0000","last_peered":"2024-06-03T03:22:07.447552+0000","last_clean":"2024-06-03T03:22:07.447552+0000","last_became_active":"2024-06-03T03:19:56.914918+0000","last_became_peered":"2024-06-03T03:19:56.914918+0000","last_unstale":"2024-06-03T03:22:07.447552+0000","last_undegraded":"2024-06-03T03:22:07.447552+0000","last_fullsized":"2024-06-03T03:22:07.447552+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:42:59.324011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"97'5","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:53.277541+0000","last_change":"2024-06-03T03:19:36.549502+0000","last_active":"2024-06-03T03:21:53.277541+0000","last_peered":"2024-06-03T03:21:53.277541+0000","last_clean":"2024-06-03T03:21:53.277541+0000","last_became_active":"2024-06-03T03:19:36.547538+0000","last_became_peered":"2024-06-03T03:19:36.547538+0000","last_unstale":"2024-06-03T03:21:53.277541+0000","last_undegraded":"2024-06-03T03:21:53.277541+0000","last_fullsized":"2024-06-03T03:21:53.277541+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:28:50.558069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"99'9","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.815966+0000","last_change":"2024-06-03T03:19:41.607441+0000","last_active":"2024-06-03T03:22:54.815966+0000","last_peered":"2024-06-03T03:22:54.815966+0000","last_clean":"2024-06-03T03:22:54.815966+0000","last_became_active":"2024-06-03T03:19:41.607086+0000","last_became_peered":"2024-06-03T03:19:41.607086+0000","last_unstale":"2024-06-03T03:22:54.815966+0000","last_undegraded":"2024-06-03T03:22:54.815966+0000","last_fullsized":"2024-06-03T03:22:54.815966+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T14:53:50.091872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.819602+0000","last_change":"2024-06-03T03:19:41.607004+0000","last_active":"2024-06-03T03:22:54.819602+0000","last_peered":"2024-06-03T03:22:54.819602+0000","last_clean":"2024-06-03T03:22:54.819602+0000","last_became_active":"2024-06-03T03:19:41.606684+0000","last_became_peered":"2024-06-03T03:19:41.606684+0000","last_unstale":"2024-06-03T03:22:54.819602+0000","last_undegraded":"2024-06-03T03:22:54.819602+0000","last_fullsized":"2024-06-03T03:22:54.819602+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:48:55.320597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"99'12","reported_seq":51,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.817158+0000","last_change":"2024-06-03T03:19:41.589426+0000","last_active":"2024-06-03T03:22:54.817158+0000","last_peered":"2024-06-03T03:22:54.817158+0000","last_clean":"2024-06-03T03:22:54.817158+0000","last_became_active":"2024-06-03T03:19:41.589091+0000","last_became_peered":"2024-06-03T03:19:41.589091+0000","last_unstale":"2024-06-03T03:22:54.817158+0000","last_undegraded":"2024-06-03T03:22:54.817158+0000","last_fullsized":"2024-06-03T03:22:54.817158+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:40:34.313214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.832557+0000","last_change":"2024-06-03T03:19:41.626149+0000","last_active":"2024-06-03T03:22:54.832557+0000","last_peered":"2024-06-03T03:22:54.832557+0000","last_clean":"2024-06-03T03:22:54.832557+0000","last_became_active":"2024-06-03T03:19:41.625710+0000","last_became_peered":"2024-06-03T03:19:41.625710+0000","last_unstale":"2024-06-03T03:22:54.832557+0000","last_undegraded":"2024-06-03T03:22:54.832557+0000","last_fullsized":"2024-06-03T03:22:54.832557+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:15:15.220975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.838595+0000","last_change":"2024-06-03T03:19:41.636585+0000","last_active":"2024-06-03T03:22:54.838595+0000","last_peered":"2024-06-03T03:22:54.838595+0000","last_clean":"2024-06-03T03:22:54.838595+0000","last_became_active":"2024-06-03T03:19:41.636241+0000","last_became_peered":"2024-06-03T03:19:41.636241+0000","last_unstale":"2024-06-03T03:22:54.838595+0000","last_undegraded":"2024-06-03T03:22:54.838595+0000","last_fullsized":"2024-06-03T03:22:54.838595+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:19:28.548297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"99'4","reported_seq":43,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.834220+0000","last_change":"2024-06-03T03:19:41.629530+0000","last_active":"2024-06-03T03:22:54.834220+0000","last_peered":"2024-06-03T03:22:54.834220+0000","last_clean":"2024-06-03T03:22:54.834220+0000","last_became_active":"2024-06-03T03:19:41.629197+0000","last_became_peered":"2024-06-03T03:19:41.629197+0000","last_unstale":"2024-06-03T03:22:54.834220+0000","last_undegraded":"2024-06-03T03:22:54.834220+0000","last_fullsized":"2024-06-03T03:22:54.834220+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:12:02.807445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.819264+0000","last_change":"2024-06-03T03:19:41.606995+0000","last_active":"2024-06-03T03:22:54.819264+0000","last_peered":"2024-06-03T03:22:54.819264+0000","last_clean":"2024-06-03T03:22:54.819264+0000","last_became_active":"2024-06-03T03:19:41.606651+0000","last_became_peered":"2024-06-03T03:19:41.606651+0000","last_unstale":"2024-06-03T03:22:54.819264+0000","last_undegraded":"2024-06-03T03:22:54.819264+0000","last_fullsized":"2024-06-03T03:22:54.819264+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:07:11.751606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"99'15","reported_seq":54,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.810136+0000","last_change":"2024-06-03T03:19:41.609848+0000","last_active":"2024-06-03T03:22:54.810136+0000","last_peered":"2024-06-03T03:22:54.810136+0000","last_clean":"2024-06-03T03:22:54.810136+0000","last_became_active":"2024-06-03T03:19:41.609544+0000","last_became_peered":"2024-06-03T03:19:41.609544+0000","last_unstale":"2024-06-03T03:22:54.810136+0000","last_undegraded":"2024-06-03T03:22:54.810136+0000","last_fullsized":"2024-06-03T03:22:54.810136+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:12:00.804638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"99'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.810509+0000","last_change":"2024-06-03T03:20:25.912049+0000","last_active":"2024-06-03T03:22:54.810509+0000","last_peered":"2024-06-03T03:22:54.810509+0000","last_clean":"2024-06-03T03:22:54.810509+0000","last_became_active":"2024-06-03T03:20:25.910664+0000","last_became_peered":"2024-06-03T03:20:25.910664+0000","last_unstale":"2024-06-03T03:22:54.810509+0000","last_undegraded":"2024-06-03T03:22:54.810509+0000","last_fullsized":"2024-06-03T03:22:54.810509+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:47:19.282569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"99'6","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.828822+0000","last_change":"2024-06-03T03:19:41.610800+0000","last_active":"2024-06-03T03:22:54.828822+0000","last_peered":"2024-06-03T03:22:54.828822+0000","last_clean":"2024-06-03T03:22:54.828822+0000","last_became_active":"2024-06-03T03:19:41.610457+0000","last_became_peered":"2024-06-03T03:19:41.610457+0000","last_unstale":"2024-06-03T03:22:54.828822+0000","last_undegraded":"2024-06-03T03:22:54.828822+0000","last_fullsized":"2024-06-03T03:22:54.828822+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:31:06.415159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.821379+0000","last_change":"2024-06-03T03:19:41.617020+0000","last_active":"2024-06-03T03:22:54.821379+0000","last_peered":"2024-06-03T03:22:54.821379+0000","last_clean":"2024-06-03T03:22:54.821379+0000","last_became_active":"2024-06-03T03:19:41.616614+0000","last_became_peered":"2024-06-03T03:19:41.616614+0000","last_unstale":"2024-06-03T03:22:54.821379+0000","last_undegraded":"2024-06-03T03:22:54.821379+0000","last_fullsized":"2024-06-03T03:22:54.821379+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:35:32.657685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"99'10","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.828254+0000","last_change":"2024-06-03T03:19:41.622785+0000","last_active":"2024-06-03T03:22:54.828254+0000","last_peered":"2024-06-03T03:22:54.828254+0000","last_clean":"2024-06-03T03:22:54.828254+0000","last_became_active":"2024-06-03T03:19:41.622407+0000","last_became_peered":"2024-06-03T03:19:41.622407+0000","last_unstale":"2024-06-03T03:22:54.828254+0000","last_undegraded":"2024-06-03T03:22:54.828254+0000","last_fullsized":"2024-06-03T03:22:54.828254+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:53:50.804889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"99'11","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.838978+0000","last_change":"2024-06-03T03:19:41.633429+0000","last_active":"2024-06-03T03:22:54.838978+0000","last_peered":"2024-06-03T03:22:54.838978+0000","last_clean":"2024-06-03T03:22:54.838978+0000","last_became_active":"2024-06-03T03:19:41.631028+0000","last_became_peered":"2024-06-03T03:19:41.631028+0000","last_unstale":"2024-06-03T03:22:54.838978+0000","last_undegraded":"2024-06-03T03:22:54.838978+0000","last_fullsized":"2024-06-03T03:22:54.838978+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:37:10.666591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.818463+0000","last_change":"2024-06-03T03:19:41.606204+0000","last_active":"2024-06-03T03:22:54.818463+0000","last_peered":"2024-06-03T03:22:54.818463+0000","last_clean":"2024-06-03T03:22:54.818463+0000","last_became_active":"2024-06-03T03:19:41.605756+0000","last_became_peered":"2024-06-03T03:19:41.605756+0000","last_unstale":"2024-06-03T03:22:54.818463+0000","last_undegraded":"2024-06-03T03:22:54.818463+0000","last_fullsized":"2024-06-03T03:22:54.818463+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:24:32.507426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":245318,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":82643,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":2538,"num_read_kb":1361,"num_write":283,"num_write_kb":174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589824,"data_stored":247929,"data_compressed":5580,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":241,"ondisk_log_size":241,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383764,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1101824,"data_stored":570765,"data_compressed":32353,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645402,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32404,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712748,"statfs":{"total":95995035648,"available":95961853952,"internally_reserved":0,"allocated":1257472,"data_stored":1231405,"data_compressed":69963,"data_compressed_allocated":94208,"data_compressed_original":688128,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808927,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32280,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712872,"statfs":{"total":95995035648,"available":95961980928,"internally_reserved":0,"allocated":1130496,"data_stored":574401,"data_compressed":32353,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070566,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32272,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712880,"statfs":{"total":95995035648,"available":95961989120,"internally_reserved":0,"allocated":1122304,"data_stored":570683,"data_compressed":32353,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332203,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32324,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712828,"statfs":{"total":95995035648,"available":95961935872,"internally_reserved":0,"allocated":1175552,"data_stored":1168406,"data_compressed":68103,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495730,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1101824,"data_stored":566199,"data_compressed":32353,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574989,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28164,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716988,"statfs":{"total":95995035648,"available":95966195712,"internally_reserved":0,"allocated":1175552,"data_stored":641443,"data_compressed":34213,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738513,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1126400,"data_stored":567655,"data_compressed":32353,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411463,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36496,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708656,"statfs":{"total":95995035648,"available":95957663744,"internally_reserved":0,"allocated":1187840,"data_stored":641330,"data_compressed":34213,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182530,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1114112,"data_stored":565517,"data_compressed":32353,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920893,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28108,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717044,"statfs":{"total":95995035648,"available":95966253056,"internally_reserved":0,"allocated":1118208,"data_stored":571443,"data_compressed":32353,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757368,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32332,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712820,"statfs":{"total":95995035648,"available":95961927680,"internally_reserved":0,"allocated":1183744,"data_stored":1163598,"data_compressed":68103,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34106,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34106,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34106,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35750,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35750,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35750,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35750,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":76356,"data_compressed":1860,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":76243,"data_compressed":1860,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":4047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":75950,"data_compressed":1860,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-03T03:25:19.435 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-06-03T03:25:19.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.682+0000 7f44c3400640 1 -- 172.21.15.117:0/181074697 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f44bc102500 msgr2=0x7f44bc1048f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:19.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.682+0000 7f44c3400640 1 --2- 172.21.15.117:0/181074697 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f44bc102500 0x7f44bc1048f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f44b4009900 tx=0x7f44b402a410 comp rx=0 tx=0).stop 2024-06-03T03:25:19.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.682+0000 7f44c3400640 1 -- 172.21.15.117:0/181074697 shutdown_connections 2024-06-03T03:25:19.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.682+0000 7f44c3400640 1 --2- 172.21.15.117:0/181074697 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f44bc107760 0x7f44bc109b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:19.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.682+0000 7f44c3400640 1 --2- 172.21.15.117:0/181074697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44bc104e30 0x7f44bc107220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:19.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.682+0000 7f44c3400640 1 --2- 172.21.15.117:0/181074697 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f44bc102500 0x7f44bc1048f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:19.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.682+0000 7f44c3400640 1 -- 172.21.15.117:0/181074697 >> 172.21.15.117:0/181074697 conn(0x7f44bc0fbe50 msgr2=0x7f44bc0fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:25:19.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.682+0000 7f44c3400640 1 -- 172.21.15.117:0/181074697 shutdown_connections 2024-06-03T03:25:19.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.682+0000 7f44c3400640 1 -- 172.21.15.117:0/181074697 wait complete. 2024-06-03T03:25:19.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.683+0000 7f44c3400640 1 Processor -- start 2024-06-03T03:25:19.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.683+0000 7f44c3400640 1 -- start start 2024-06-03T03:25:19.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.684+0000 7f44c3400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f44bc102500 0x7f44bc199ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:19.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.684+0000 7f44c3400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44bc104e30 0x7f44bc19a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:19.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.684+0000 7f44c3400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f44bc107760 0x7f44bc1a14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:19.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.684+0000 7f44c3400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f44bc10d6a0 con 0x7f44bc104e30 2024-06-03T03:25:19.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.684+0000 7f44c3400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f44bc10d520 con 0x7f44bc102500 2024-06-03T03:25:19.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.684+0000 7f44c3400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f44bc10d820 con 0x7f44bc107760 2024-06-03T03:25:19.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.684+0000 7f44bbe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44bc104e30 0x7f44bc19a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:19.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.684+0000 7f44bbe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44bc104e30 0x7f44bc19a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39812/0 (socket says 172.21.15.117:39812) 2024-06-03T03:25:19.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.684+0000 7f44bbe00640 1 -- 172.21.15.117:0/4225399823 learned_addr learned my addr 172.21.15.117:0/4225399823 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:25:19.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.684+0000 7f44c0a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f44bc102500 0x7f44bc199ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:19.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.684+0000 7f44c1400640 1 --2- 172.21.15.117:0/4225399823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f44bc107760 0x7f44bc1a14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:19.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.685+0000 7f44bbe00640 1 -- 172.21.15.117:0/4225399823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f44bc107760 msgr2=0x7f44bc1a14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:19.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.685+0000 7f44bbe00640 1 --2- 172.21.15.117:0/4225399823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f44bc107760 0x7f44bc1a14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:19.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.685+0000 7f44bbe00640 1 -- 172.21.15.117:0/4225399823 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f44bc102500 msgr2=0x7f44bc199ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:19.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.685+0000 7f44bbe00640 1 --2- 172.21.15.117:0/4225399823 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f44bc102500 0x7f44bc199ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:19.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.685+0000 7f44bbe00640 1 -- 172.21.15.117:0/4225399823 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44bc1a1b10 con 0x7f44bc104e30 2024-06-03T03:25:19.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.685+0000 7f44c1400640 1 --2- 172.21.15.117:0/4225399823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f44bc107760 0x7f44bc1a14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:25:19.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.685+0000 7f44c0a00640 1 --2- 172.21.15.117:0/4225399823 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f44bc102500 0x7f44bc199ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:25:19.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.685+0000 7f44bbe00640 1 --2- 172.21.15.117:0/4225399823 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44bc104e30 0x7f44bc19a430 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f44a400e9c0 tx=0x7f44a400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:25:19.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.685+0000 7f44b9600640 1 -- 172.21.15.117:0/4225399823 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a400cd80 con 0x7f44bc104e30 2024-06-03T03:25:19.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.685+0000 7f44b9600640 1 -- 172.21.15.117:0/4225399823 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44a4005500 con 0x7f44bc104e30 2024-06-03T03:25:19.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.685+0000 7f44b9600640 1 -- 172.21.15.117:0/4225399823 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a4010690 con 0x7f44bc104e30 2024-06-03T03:25:19.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.685+0000 7f44c3400640 1 -- 172.21.15.117:0/4225399823 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44bc1a1e00 con 0x7f44bc104e30 2024-06-03T03:25:19.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.685+0000 7f44c3400640 1 -- 172.21.15.117:0/4225399823 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44bc1a2270 con 0x7f44bc104e30 2024-06-03T03:25:19.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.687+0000 7f44c3400640 1 -- 172.21.15.117:0/4225399823 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44bc109250 con 0x7f44bc104e30 2024-06-03T03:25:19.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.689+0000 7f44b9600640 1 -- 172.21.15.117:0/4225399823 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f44a40046c0 con 0x7f44bc104e30 2024-06-03T03:25:19.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.691+0000 7f44b9600640 1 --2- 172.21.15.117:0/4225399823 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f44a00a2ca0 0x7f44a00a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:19.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.691+0000 7f44b9600640 1 -- 172.21.15.117:0/4225399823 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f44a40cf8b0 con 0x7f44bc104e30 2024-06-03T03:25:19.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.691+0000 7f44c0a00640 1 --2- 172.21.15.117:0/4225399823 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f44a00a2ca0 0x7f44a00a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:19.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.692+0000 7f44c0a00640 1 --2- 172.21.15.117:0/4225399823 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f44a00a2ca0 0x7f44a00a5160 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f44b4031440 tx=0x7f44b40315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:25:19.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.696+0000 7f44b9600640 1 -- 172.21.15.117:0/4225399823 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f44a409c2c0 con 0x7f44bc104e30 2024-06-03T03:25:19.893 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.892+0000 7f44c3400640 1 -- 172.21.15.117:0/4225399823 --> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f44bc061de0 con 0x7f44a00a2ca0 2024-06-03T03:25:19.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.921+0000 7f44b9600640 1 -- 172.21.15.117:0/4225399823 <== mgr.14152 v2:172.21.15.117:6800/1398160887 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340831 (secure 0 0 0) 0x7f44bc061de0 con 0x7f44a00a2ca0 2024-06-03T03:25:19.925 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T03:25:19.929 INFO:teuthology.orchestra.run.smithi117.stderr:dumped all 2024-06-03T03:25:19.932 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.931+0000 7f44c3400640 1 -- 172.21.15.117:0/4225399823 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f44a00a2ca0 msgr2=0x7f44a00a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:19.932 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.931+0000 7f44c3400640 1 --2- 172.21.15.117:0/4225399823 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f44a00a2ca0 0x7f44a00a5160 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f44b4031440 tx=0x7f44b40315b0 comp rx=0 tx=0).stop 2024-06-03T03:25:19.932 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.931+0000 7f44c3400640 1 -- 172.21.15.117:0/4225399823 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44bc104e30 msgr2=0x7f44bc19a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:19.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.931+0000 7f44c3400640 1 --2- 172.21.15.117:0/4225399823 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44bc104e30 0x7f44bc19a430 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f44a400e9c0 tx=0x7f44a400ee80 comp rx=0 tx=0).stop 2024-06-03T03:25:19.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.932+0000 7f44c3400640 1 -- 172.21.15.117:0/4225399823 shutdown_connections 2024-06-03T03:25:19.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.932+0000 7f44c3400640 1 --2- 172.21.15.117:0/4225399823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f44bc107760 0x7f44bc1a14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:19.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.932+0000 7f44c3400640 1 --2- 172.21.15.117:0/4225399823 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44bc104e30 0x7f44bc19a430 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:19.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.932+0000 7f44c3400640 1 --2- 172.21.15.117:0/4225399823 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f44a00a2ca0 0x7f44a00a5160 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:19.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.932+0000 7f44c3400640 1 --2- 172.21.15.117:0/4225399823 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f44bc102500 0x7f44bc199ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:19.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.932+0000 7f44c3400640 1 -- 172.21.15.117:0/4225399823 >> 172.21.15.117:0/4225399823 conn(0x7f44bc0fbe50 msgr2=0x7f44bc101c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:25:19.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.932+0000 7f44c3400640 1 -- 172.21.15.117:0/4225399823 shutdown_connections 2024-06-03T03:25:19.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:19.932+0000 7f44c3400640 1 -- 172.21.15.117:0/4225399823 wait complete. 2024-06-03T03:25:19.951 INFO:teuthology.orchestra.run.smithi117.stdout:{"pg_ready":true,"pg_map":{"version":506,"stamp":"2024-06-03T03:25:18.957567+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":673245,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":2664,"num_read_kb":1470,"num_write":466,"num_write_kb":3114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":880,"ondisk_log_size":880,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":370944,"kb_used_data":13472,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124570880,"statfs":{"total":1151940427776,"available":1151560581120,"internally_reserved":0,"allocated":13795328,"data_stored":8832845,"data_compressed":501066,"data_compressed_allocated":688128,"data_compressed_original":2875392,"omap_allocated":19142,"internal_metadata":365933882},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6486,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":146,"num_read_kb":80,"num_write":19,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005661"},"pg_stats":[{"pgid":"3.3f","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.832850+0000","last_change":"2024-06-03T03:19:41.626030+0000","last_active":"2024-06-03T03:22:54.832850+0000","last_peered":"2024-06-03T03:22:54.832850+0000","last_clean":"2024-06-03T03:22:54.832850+0000","last_became_active":"2024-06-03T03:19:41.625681+0000","last_became_peered":"2024-06-03T03:19:41.625681+0000","last_unstale":"2024-06-03T03:22:54.832850+0000","last_undegraded":"2024-06-03T03:22:54.832850+0000","last_fullsized":"2024-06-03T03:22:54.832850+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:35:46.396360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"99'4","reported_seq":43,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.831429+0000","last_change":"2024-06-03T03:19:41.619668+0000","last_active":"2024-06-03T03:22:54.831429+0000","last_peered":"2024-06-03T03:22:54.831429+0000","last_clean":"2024-06-03T03:22:54.831429+0000","last_became_active":"2024-06-03T03:19:41.619318+0000","last_became_peered":"2024-06-03T03:19:41.619318+0000","last_unstale":"2024-06-03T03:22:54.831429+0000","last_undegraded":"2024-06-03T03:22:54.831429+0000","last_fullsized":"2024-06-03T03:22:54.831429+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:07:21.656045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.832725+0000","last_change":"2024-06-03T03:19:41.623689+0000","last_active":"2024-06-03T03:22:54.832725+0000","last_peered":"2024-06-03T03:22:54.832725+0000","last_clean":"2024-06-03T03:22:54.832725+0000","last_became_active":"2024-06-03T03:19:41.623027+0000","last_became_peered":"2024-06-03T03:19:41.623027+0000","last_unstale":"2024-06-03T03:22:54.832725+0000","last_undegraded":"2024-06-03T03:22:54.832725+0000","last_fullsized":"2024-06-03T03:22:54.832725+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:18:05.584397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.815076+0000","last_change":"2024-06-03T03:19:41.603802+0000","last_active":"2024-06-03T03:22:54.815076+0000","last_peered":"2024-06-03T03:22:54.815076+0000","last_clean":"2024-06-03T03:22:54.815076+0000","last_became_active":"2024-06-03T03:19:41.603470+0000","last_became_peered":"2024-06-03T03:19:41.603470+0000","last_unstale":"2024-06-03T03:22:54.815076+0000","last_undegraded":"2024-06-03T03:22:54.815076+0000","last_fullsized":"2024-06-03T03:22:54.815076+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:56:10.636108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"99'4","reported_seq":43,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.826310+0000","last_change":"2024-06-03T03:19:41.636436+0000","last_active":"2024-06-03T03:22:54.826310+0000","last_peered":"2024-06-03T03:22:54.826310+0000","last_clean":"2024-06-03T03:22:54.826310+0000","last_became_active":"2024-06-03T03:19:41.636150+0000","last_became_peered":"2024-06-03T03:19:41.636150+0000","last_unstale":"2024-06-03T03:22:54.826310+0000","last_undegraded":"2024-06-03T03:22:54.826310+0000","last_fullsized":"2024-06-03T03:22:54.826310+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:02:10.928251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"99'6","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.809082+0000","last_change":"2024-06-03T03:19:41.622426+0000","last_active":"2024-06-03T03:22:54.809082+0000","last_peered":"2024-06-03T03:22:54.809082+0000","last_clean":"2024-06-03T03:22:54.809082+0000","last_became_active":"2024-06-03T03:19:41.622092+0000","last_became_peered":"2024-06-03T03:19:41.622092+0000","last_unstale":"2024-06-03T03:22:54.809082+0000","last_undegraded":"2024-06-03T03:22:54.809082+0000","last_fullsized":"2024-06-03T03:22:54.809082+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T14:08:52.666771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"99'5","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.825682+0000","last_change":"2024-06-03T03:19:41.608303+0000","last_active":"2024-06-03T03:22:54.825682+0000","last_peered":"2024-06-03T03:22:54.825682+0000","last_clean":"2024-06-03T03:22:54.825682+0000","last_became_active":"2024-06-03T03:19:41.607929+0000","last_became_peered":"2024-06-03T03:19:41.607929+0000","last_unstale":"2024-06-03T03:22:54.825682+0000","last_undegraded":"2024-06-03T03:22:54.825682+0000","last_fullsized":"2024-06-03T03:22:54.825682+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:40:05.643531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.822841+0000","last_change":"2024-06-03T03:19:41.606722+0000","last_active":"2024-06-03T03:22:54.822841+0000","last_peered":"2024-06-03T03:22:54.822841+0000","last_clean":"2024-06-03T03:22:54.822841+0000","last_became_active":"2024-06-03T03:19:41.606131+0000","last_became_peered":"2024-06-03T03:19:41.606131+0000","last_unstale":"2024-06-03T03:22:54.822841+0000","last_undegraded":"2024-06-03T03:22:54.822841+0000","last_fullsized":"2024-06-03T03:22:54.822841+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:09:17.306848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"99'11","reported_seq":51,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.813800+0000","last_change":"2024-06-03T03:19:41.609287+0000","last_active":"2024-06-03T03:22:54.813800+0000","last_peered":"2024-06-03T03:22:54.813800+0000","last_clean":"2024-06-03T03:22:54.813800+0000","last_became_active":"2024-06-03T03:19:41.608521+0000","last_became_peered":"2024-06-03T03:19:41.608521+0000","last_unstale":"2024-06-03T03:22:54.813800+0000","last_undegraded":"2024-06-03T03:22:54.813800+0000","last_fullsized":"2024-06-03T03:22:54.813800+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:35:37.128667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"99'16","reported_seq":56,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.814705+0000","last_change":"2024-06-03T03:19:41.605193+0000","last_active":"2024-06-03T03:22:54.814705+0000","last_peered":"2024-06-03T03:22:54.814705+0000","last_clean":"2024-06-03T03:22:54.814705+0000","last_became_active":"2024-06-03T03:19:41.604787+0000","last_became_peered":"2024-06-03T03:19:41.604787+0000","last_unstale":"2024-06-03T03:22:54.814705+0000","last_undegraded":"2024-06-03T03:22:54.814705+0000","last_fullsized":"2024-06-03T03:22:54.814705+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:02:48.334599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"99'6","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.817666+0000","last_change":"2024-06-03T03:19:41.611522+0000","last_active":"2024-06-03T03:22:54.817666+0000","last_peered":"2024-06-03T03:22:54.817666+0000","last_clean":"2024-06-03T03:22:54.817666+0000","last_became_active":"2024-06-03T03:19:41.610932+0000","last_became_peered":"2024-06-03T03:19:41.610932+0000","last_unstale":"2024-06-03T03:22:54.817666+0000","last_undegraded":"2024-06-03T03:22:54.817666+0000","last_fullsized":"2024-06-03T03:22:54.817666+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:35:44.702791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"99'7","reported_seq":24,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.820810+0000","last_change":"2024-06-03T03:20:25.950663+0000","last_active":"2024-06-03T03:22:54.820810+0000","last_peered":"2024-06-03T03:22:54.820810+0000","last_clean":"2024-06-03T03:22:54.820810+0000","last_became_active":"2024-06-03T03:20:25.949962+0000","last_became_peered":"2024-06-03T03:20:25.949962+0000","last_unstale":"2024-06-03T03:22:54.820810+0000","last_undegraded":"2024-06-03T03:22:54.820810+0000","last_fullsized":"2024-06-03T03:22:54.820810+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:55:33.972567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"99'10","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.839179+0000","last_change":"2024-06-03T03:19:41.633464+0000","last_active":"2024-06-03T03:22:54.839179+0000","last_peered":"2024-06-03T03:22:54.839179+0000","last_clean":"2024-06-03T03:22:54.839179+0000","last_became_active":"2024-06-03T03:19:41.630625+0000","last_became_peered":"2024-06-03T03:19:41.630625+0000","last_unstale":"2024-06-03T03:22:54.839179+0000","last_undegraded":"2024-06-03T03:22:54.839179+0000","last_fullsized":"2024-06-03T03:22:54.839179+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:23:28.799058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.831893+0000","last_change":"2024-06-03T03:19:41.627234+0000","last_active":"2024-06-03T03:22:54.831893+0000","last_peered":"2024-06-03T03:22:54.831893+0000","last_clean":"2024-06-03T03:22:54.831893+0000","last_became_active":"2024-06-03T03:19:41.626643+0000","last_became_peered":"2024-06-03T03:19:41.626643+0000","last_unstale":"2024-06-03T03:22:54.831893+0000","last_undegraded":"2024-06-03T03:22:54.831893+0000","last_fullsized":"2024-06-03T03:22:54.831893+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:37:44.126101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.831893+0000","last_change":"2024-06-03T03:19:41.621833+0000","last_active":"2024-06-03T03:22:54.831893+0000","last_peered":"2024-06-03T03:22:54.831893+0000","last_clean":"2024-06-03T03:22:54.831893+0000","last_became_active":"2024-06-03T03:19:41.620689+0000","last_became_peered":"2024-06-03T03:19:41.620689+0000","last_unstale":"2024-06-03T03:22:54.831893+0000","last_undegraded":"2024-06-03T03:22:54.831893+0000","last_fullsized":"2024-06-03T03:22:54.831893+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:39:31.188372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"99'9","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.830743+0000","last_change":"2024-06-03T03:19:41.624072+0000","last_active":"2024-06-03T03:22:54.830743+0000","last_peered":"2024-06-03T03:22:54.830743+0000","last_clean":"2024-06-03T03:22:54.830743+0000","last_became_active":"2024-06-03T03:19:41.623689+0000","last_became_peered":"2024-06-03T03:19:41.623689+0000","last_unstale":"2024-06-03T03:22:54.830743+0000","last_undegraded":"2024-06-03T03:22:54.830743+0000","last_fullsized":"2024-06-03T03:22:54.830743+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:38:24.075256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.833104+0000","last_change":"2024-06-03T03:19:41.626145+0000","last_active":"2024-06-03T03:22:54.833104+0000","last_peered":"2024-06-03T03:22:54.833104+0000","last_clean":"2024-06-03T03:22:54.833104+0000","last_became_active":"2024-06-03T03:19:41.625826+0000","last_became_peered":"2024-06-03T03:19:41.625826+0000","last_unstale":"2024-06-03T03:22:54.833104+0000","last_undegraded":"2024-06-03T03:22:54.833104+0000","last_fullsized":"2024-06-03T03:22:54.833104+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:04:10.996134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"99'10","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.821280+0000","last_change":"2024-06-03T03:19:41.615754+0000","last_active":"2024-06-03T03:22:54.821280+0000","last_peered":"2024-06-03T03:22:54.821280+0000","last_clean":"2024-06-03T03:22:54.821280+0000","last_became_active":"2024-06-03T03:19:41.614577+0000","last_became_peered":"2024-06-03T03:19:41.614577+0000","last_unstale":"2024-06-03T03:22:54.821280+0000","last_undegraded":"2024-06-03T03:22:54.821280+0000","last_fullsized":"2024-06-03T03:22:54.821280+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:45:50.084332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.826381+0000","last_change":"2024-06-03T03:19:41.633000+0000","last_active":"2024-06-03T03:22:54.826381+0000","last_peered":"2024-06-03T03:22:54.826381+0000","last_clean":"2024-06-03T03:22:54.826381+0000","last_became_active":"2024-06-03T03:19:41.632731+0000","last_became_peered":"2024-06-03T03:19:41.632731+0000","last_unstale":"2024-06-03T03:22:54.826381+0000","last_undegraded":"2024-06-03T03:22:54.826381+0000","last_fullsized":"2024-06-03T03:22:54.826381+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:10:29.990256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:24.128512+0000","last_change":"2024-06-03T03:20:26.133763+0000","last_active":"2024-06-03T03:22:24.128512+0000","last_peered":"2024-06-03T03:22:24.128512+0000","last_clean":"2024-06-03T03:22:24.128512+0000","last_became_active":"2024-06-03T03:20:26.133252+0000","last_became_peered":"2024-06-03T03:20:26.133252+0000","last_unstale":"2024-06-03T03:22:24.128512+0000","last_undegraded":"2024-06-03T03:22:24.128512+0000","last_fullsized":"2024-06-03T03:22:24.128512+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:33:21.461038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"99'14","reported_seq":309,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:25:14.520260+0000","last_change":"2024-06-03T03:19:36.542545+0000","last_active":"2024-06-03T03:25:14.520260+0000","last_peered":"2024-06-03T03:25:14.520260+0000","last_clean":"2024-06-03T03:25:14.520260+0000","last_became_active":"2024-06-03T03:19:36.541682+0000","last_became_peered":"2024-06-03T03:19:36.541682+0000","last_unstale":"2024-06-03T03:25:14.520260+0000","last_undegraded":"2024-06-03T03:25:14.520260+0000","last_fullsized":"2024-06-03T03:25:14.520260+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:06:23.909253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":249,"num_read_kb":249,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"99'5","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.819154+0000","last_change":"2024-06-03T03:19:41.605848+0000","last_active":"2024-06-03T03:22:54.819154+0000","last_peered":"2024-06-03T03:22:54.819154+0000","last_clean":"2024-06-03T03:22:54.819154+0000","last_became_active":"2024-06-03T03:19:41.603131+0000","last_became_peered":"2024-06-03T03:19:41.603131+0000","last_unstale":"2024-06-03T03:22:54.819154+0000","last_undegraded":"2024-06-03T03:22:54.819154+0000","last_fullsized":"2024-06-03T03:22:54.819154+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:07:38.931055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:22.310371+0000","last_change":"2024-06-03T03:19:56.923180+0000","last_active":"2024-06-03T03:22:22.310371+0000","last_peered":"2024-06-03T03:22:22.310371+0000","last_clean":"2024-06-03T03:22:22.310371+0000","last_became_active":"2024-06-03T03:19:56.922726+0000","last_became_peered":"2024-06-03T03:19:56.922726+0000","last_unstale":"2024-06-03T03:22:22.310371+0000","last_undegraded":"2024-06-03T03:22:22.310371+0000","last_fullsized":"2024-06-03T03:22:22.310371+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:18:45.727680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"97'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:36.766695+0000","last_change":"2024-06-03T03:19:36.746805+0000","last_active":"2024-06-03T03:21:36.766695+0000","last_peered":"2024-06-03T03:21:36.766695+0000","last_clean":"2024-06-03T03:21:36.766695+0000","last_became_active":"2024-06-03T03:19:36.745747+0000","last_became_peered":"2024-06-03T03:19:36.745747+0000","last_unstale":"2024-06-03T03:21:36.766695+0000","last_undegraded":"2024-06-03T03:21:36.766695+0000","last_fullsized":"2024-06-03T03:21:36.766695+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:54:28.427708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.832890+0000","last_change":"2024-06-03T03:19:41.630582+0000","last_active":"2024-06-03T03:22:54.832890+0000","last_peered":"2024-06-03T03:22:54.832890+0000","last_clean":"2024-06-03T03:22:54.832890+0000","last_became_active":"2024-06-03T03:19:41.630238+0000","last_became_peered":"2024-06-03T03:19:41.630238+0000","last_unstale":"2024-06-03T03:22:54.832890+0000","last_undegraded":"2024-06-03T03:22:54.832890+0000","last_fullsized":"2024-06-03T03:22:54.832890+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:49:58.039697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:27.311089+0000","last_change":"2024-06-03T03:19:56.925142+0000","last_active":"2024-06-03T03:22:27.311089+0000","last_peered":"2024-06-03T03:22:27.311089+0000","last_clean":"2024-06-03T03:22:27.311089+0000","last_became_active":"2024-06-03T03:19:56.924760+0000","last_became_peered":"2024-06-03T03:19:56.924760+0000","last_unstale":"2024-06-03T03:22:27.311089+0000","last_undegraded":"2024-06-03T03:22:27.311089+0000","last_fullsized":"2024-06-03T03:22:27.311089+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:11:27.537077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"97'12","reported_seq":58,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:09.897381+0000","last_change":"2024-06-03T03:19:36.543344+0000","last_active":"2024-06-03T03:22:09.897381+0000","last_peered":"2024-06-03T03:22:09.897381+0000","last_clean":"2024-06-03T03:22:09.897381+0000","last_became_active":"2024-06-03T03:19:36.542837+0000","last_became_peered":"2024-06-03T03:19:36.542837+0000","last_unstale":"2024-06-03T03:22:09.897381+0000","last_undegraded":"2024-06-03T03:22:09.897381+0000","last_fullsized":"2024-06-03T03:22:09.897381+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:55:05.451521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"99'9","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.813173+0000","last_change":"2024-06-03T03:19:41.629820+0000","last_active":"2024-06-03T03:22:54.813173+0000","last_peered":"2024-06-03T03:22:54.813173+0000","last_clean":"2024-06-03T03:22:54.813173+0000","last_became_active":"2024-06-03T03:19:41.629439+0000","last_became_peered":"2024-06-03T03:19:41.629439+0000","last_unstale":"2024-06-03T03:22:54.813173+0000","last_undegraded":"2024-06-03T03:22:54.813173+0000","last_fullsized":"2024-06-03T03:22:54.813173+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:05:27.438308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:27.311072+0000","last_change":"2024-06-03T03:19:56.923214+0000","last_active":"2024-06-03T03:22:27.311072+0000","last_peered":"2024-06-03T03:22:27.311072+0000","last_clean":"2024-06-03T03:22:27.311072+0000","last_became_active":"2024-06-03T03:19:56.922751+0000","last_became_peered":"2024-06-03T03:19:56.922751+0000","last_unstale":"2024-06-03T03:22:27.311072+0000","last_undegraded":"2024-06-03T03:22:27.311072+0000","last_fullsized":"2024-06-03T03:22:27.311072+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:05:59.249818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"97'5","reported_seq":51,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:09.897329+0000","last_change":"2024-06-03T03:19:36.538141+0000","last_active":"2024-06-03T03:22:09.897329+0000","last_peered":"2024-06-03T03:22:09.897329+0000","last_clean":"2024-06-03T03:22:09.897329+0000","last_became_active":"2024-06-03T03:19:36.537786+0000","last_became_peered":"2024-06-03T03:19:36.537786+0000","last_unstale":"2024-06-03T03:22:09.897329+0000","last_undegraded":"2024-06-03T03:22:09.897329+0000","last_fullsized":"2024-06-03T03:22:09.897329+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T15:03:32.859097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"99'15","reported_seq":55,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.816490+0000","last_change":"2024-06-03T03:19:41.608471+0000","last_active":"2024-06-03T03:22:54.816490+0000","last_peered":"2024-06-03T03:22:54.816490+0000","last_clean":"2024-06-03T03:22:54.816490+0000","last_became_active":"2024-06-03T03:19:41.608096+0000","last_became_peered":"2024-06-03T03:19:41.608096+0000","last_unstale":"2024-06-03T03:22:54.816490+0000","last_undegraded":"2024-06-03T03:22:54.816490+0000","last_fullsized":"2024-06-03T03:22:54.816490+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:31:10.541803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:29.129303+0000","last_change":"2024-06-03T03:19:56.915540+0000","last_active":"2024-06-03T03:22:29.129303+0000","last_peered":"2024-06-03T03:22:29.129303+0000","last_clean":"2024-06-03T03:22:29.129303+0000","last_became_active":"2024-06-03T03:19:56.915017+0000","last_became_peered":"2024-06-03T03:19:56.915017+0000","last_unstale":"2024-06-03T03:22:29.129303+0000","last_undegraded":"2024-06-03T03:22:29.129303+0000","last_fullsized":"2024-06-03T03:22:29.129303+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:56:23.158344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"99'9","reported_seq":310,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:25:15.590058+0000","last_change":"2024-06-03T03:19:36.542337+0000","last_active":"2024-06-03T03:25:15.590058+0000","last_peered":"2024-06-03T03:25:15.590058+0000","last_clean":"2024-06-03T03:25:15.590058+0000","last_became_active":"2024-06-03T03:19:36.541990+0000","last_became_peered":"2024-06-03T03:19:36.541990+0000","last_unstale":"2024-06-03T03:25:15.590058+0000","last_undegraded":"2024-06-03T03:25:15.590058+0000","last_fullsized":"2024-06-03T03:25:15.590058+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:08:15.656321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1849,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":255,"num_read_kb":6,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.815481+0000","last_change":"2024-06-03T03:19:41.616293+0000","last_active":"2024-06-03T03:22:54.815481+0000","last_peered":"2024-06-03T03:22:54.815481+0000","last_clean":"2024-06-03T03:22:54.815481+0000","last_became_active":"2024-06-03T03:19:41.615946+0000","last_became_peered":"2024-06-03T03:19:41.615946+0000","last_unstale":"2024-06-03T03:22:54.815481+0000","last_undegraded":"2024-06-03T03:22:54.815481+0000","last_fullsized":"2024-06-03T03:22:54.815481+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:18:43.835977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:29.129278+0000","last_change":"2024-06-03T03:20:26.146237+0000","last_active":"2024-06-03T03:22:29.129278+0000","last_peered":"2024-06-03T03:22:29.129278+0000","last_clean":"2024-06-03T03:22:29.129278+0000","last_became_active":"2024-06-03T03:20:26.145731+0000","last_became_peered":"2024-06-03T03:20:26.145731+0000","last_unstale":"2024-06-03T03:22:29.129278+0000","last_undegraded":"2024-06-03T03:22:29.129278+0000","last_fullsized":"2024-06-03T03:22:29.129278+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:15:29.723833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"97'7","reported_seq":53,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:07.058194+0000","last_change":"2024-06-03T03:19:36.546501+0000","last_active":"2024-06-03T03:22:07.058194+0000","last_peered":"2024-06-03T03:22:07.058194+0000","last_clean":"2024-06-03T03:22:07.058194+0000","last_became_active":"2024-06-03T03:19:36.546079+0000","last_became_peered":"2024-06-03T03:19:36.546079+0000","last_unstale":"2024-06-03T03:22:07.058194+0000","last_undegraded":"2024-06-03T03:22:07.058194+0000","last_fullsized":"2024-06-03T03:22:07.058194+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:16:39.473141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"99'8","reported_seq":54,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.829180+0000","last_change":"2024-06-03T03:20:26.149410+0000","last_active":"2024-06-03T03:22:54.829180+0000","last_peered":"2024-06-03T03:22:54.829180+0000","last_clean":"2024-06-03T03:22:54.829180+0000","last_became_active":"2024-06-03T03:20:26.149084+0000","last_became_peered":"2024-06-03T03:20:26.149084+0000","last_unstale":"2024-06-03T03:22:54.829180+0000","last_undegraded":"2024-06-03T03:22:54.829180+0000","last_fullsized":"2024-06-03T03:22:54.829180+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:06:22.316449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:04.314994+0000","last_change":"2024-06-03T03:19:56.926498+0000","last_active":"2024-06-03T03:22:04.314994+0000","last_peered":"2024-06-03T03:22:04.314994+0000","last_clean":"2024-06-03T03:22:04.314994+0000","last_became_active":"2024-06-03T03:19:56.926101+0000","last_became_peered":"2024-06-03T03:19:56.926101+0000","last_unstale":"2024-06-03T03:22:04.314994+0000","last_undegraded":"2024-06-03T03:22:04.314994+0000","last_fullsized":"2024-06-03T03:22:04.314994+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:00:12.024443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"97'3","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:03.279571+0000","last_change":"2024-06-03T03:19:36.553151+0000","last_active":"2024-06-03T03:22:03.279571+0000","last_peered":"2024-06-03T03:22:03.279571+0000","last_clean":"2024-06-03T03:22:03.279571+0000","last_became_active":"2024-06-03T03:19:36.551914+0000","last_became_peered":"2024-06-03T03:19:36.551914+0000","last_unstale":"2024-06-03T03:22:03.279571+0000","last_undegraded":"2024-06-03T03:22:03.279571+0000","last_fullsized":"2024-06-03T03:22:03.279571+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:00:42.053922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"99'8","reported_seq":25,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.824386+0000","last_change":"2024-06-03T03:20:26.147909+0000","last_active":"2024-06-03T03:22:54.824386+0000","last_peered":"2024-06-03T03:22:54.824386+0000","last_clean":"2024-06-03T03:22:54.824386+0000","last_became_active":"2024-06-03T03:20:26.146448+0000","last_became_peered":"2024-06-03T03:20:26.146448+0000","last_unstale":"2024-06-03T03:22:54.824386+0000","last_undegraded":"2024-06-03T03:22:54.824386+0000","last_fullsized":"2024-06-03T03:22:54.824386+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:43:30.878472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:59.843167+0000","last_change":"2024-06-03T03:19:56.941910+0000","last_active":"2024-06-03T03:21:59.843167+0000","last_peered":"2024-06-03T03:21:59.843167+0000","last_clean":"2024-06-03T03:21:59.843167+0000","last_became_active":"2024-06-03T03:19:56.940963+0000","last_became_peered":"2024-06-03T03:19:56.940963+0000","last_unstale":"2024-06-03T03:21:59.843167+0000","last_undegraded":"2024-06-03T03:21:59.843167+0000","last_fullsized":"2024-06-03T03:21:59.843167+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T14:18:32.109231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"97'3","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:02.447068+0000","last_change":"2024-06-03T03:19:36.538928+0000","last_active":"2024-06-03T03:22:02.447068+0000","last_peered":"2024-06-03T03:22:02.447068+0000","last_clean":"2024-06-03T03:22:02.447068+0000","last_became_active":"2024-06-03T03:19:36.538580+0000","last_became_peered":"2024-06-03T03:19:36.538580+0000","last_unstale":"2024-06-03T03:22:02.447068+0000","last_undegraded":"2024-06-03T03:22:02.447068+0000","last_fullsized":"2024-06-03T03:22:02.447068+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:08:05.906257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"99'11","reported_seq":51,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.829295+0000","last_change":"2024-06-03T03:19:41.623051+0000","last_active":"2024-06-03T03:22:54.829295+0000","last_peered":"2024-06-03T03:22:54.829295+0000","last_clean":"2024-06-03T03:22:54.829295+0000","last_became_active":"2024-06-03T03:19:41.622656+0000","last_became_peered":"2024-06-03T03:19:41.622656+0000","last_unstale":"2024-06-03T03:22:54.829295+0000","last_undegraded":"2024-06-03T03:22:54.829295+0000","last_fullsized":"2024-06-03T03:22:54.829295+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:11:34.587723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:16.773697+0000","last_change":"2024-06-03T03:19:56.921215+0000","last_active":"2024-06-03T03:22:16.773697+0000","last_peered":"2024-06-03T03:22:16.773697+0000","last_clean":"2024-06-03T03:22:16.773697+0000","last_became_active":"2024-06-03T03:19:56.920658+0000","last_became_peered":"2024-06-03T03:19:56.920658+0000","last_unstale":"2024-06-03T03:22:16.773697+0000","last_undegraded":"2024-06-03T03:22:16.773697+0000","last_fullsized":"2024-06-03T03:22:16.773697+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T14:04:54.163818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"99'7","reported_seq":551,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:25:16.411576+0000","last_change":"2024-06-03T03:19:36.545245+0000","last_active":"2024-06-03T03:25:16.411576+0000","last_peered":"2024-06-03T03:25:16.411576+0000","last_clean":"2024-06-03T03:25:16.411576+0000","last_became_active":"2024-06-03T03:19:36.544821+0000","last_became_peered":"2024-06-03T03:19:36.544821+0000","last_unstale":"2024-06-03T03:25:16.411576+0000","last_undegraded":"2024-06-03T03:25:16.411576+0000","last_fullsized":"2024-06-03T03:25:16.411576+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:55:24.011925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":498,"num_read_kb":498,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"99'10","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.822488+0000","last_change":"2024-06-03T03:19:41.616384+0000","last_active":"2024-06-03T03:22:54.822488+0000","last_peered":"2024-06-03T03:22:54.822488+0000","last_clean":"2024-06-03T03:22:54.822488+0000","last_became_active":"2024-06-03T03:19:41.616034+0000","last_became_peered":"2024-06-03T03:19:41.616034+0000","last_unstale":"2024-06-03T03:22:54.822488+0000","last_undegraded":"2024-06-03T03:22:54.822488+0000","last_fullsized":"2024-06-03T03:22:54.822488+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:10:06.823375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:20.886218+0000","last_change":"2024-06-03T03:19:56.933045+0000","last_active":"2024-06-03T03:22:20.886218+0000","last_peered":"2024-06-03T03:22:20.886218+0000","last_clean":"2024-06-03T03:22:20.886218+0000","last_became_active":"2024-06-03T03:19:56.928956+0000","last_became_peered":"2024-06-03T03:19:56.928956+0000","last_unstale":"2024-06-03T03:22:20.886218+0000","last_undegraded":"2024-06-03T03:22:20.886218+0000","last_fullsized":"2024-06-03T03:22:20.886218+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:37:44.270511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"97'4","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:04.852741+0000","last_change":"2024-06-03T03:19:36.541559+0000","last_active":"2024-06-03T03:22:04.852741+0000","last_peered":"2024-06-03T03:22:04.852741+0000","last_clean":"2024-06-03T03:22:04.852741+0000","last_became_active":"2024-06-03T03:19:36.541166+0000","last_became_peered":"2024-06-03T03:19:36.541166+0000","last_unstale":"2024-06-03T03:22:04.852741+0000","last_undegraded":"2024-06-03T03:22:04.852741+0000","last_fullsized":"2024-06-03T03:22:04.852741+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T14:49:26.847713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.833224+0000","last_change":"2024-06-03T03:19:41.629789+0000","last_active":"2024-06-03T03:22:54.833224+0000","last_peered":"2024-06-03T03:22:54.833224+0000","last_clean":"2024-06-03T03:22:54.833224+0000","last_became_active":"2024-06-03T03:19:41.628026+0000","last_became_peered":"2024-06-03T03:19:41.628026+0000","last_unstale":"2024-06-03T03:22:54.833224+0000","last_undegraded":"2024-06-03T03:22:54.833224+0000","last_fullsized":"2024-06-03T03:22:54.833224+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:56:26.411907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:17.309900+0000","last_change":"2024-06-03T03:21:26.056114+0000","last_active":"2024-06-03T03:22:17.309900+0000","last_peered":"2024-06-03T03:22:17.309900+0000","last_clean":"2024-06-03T03:22:17.309900+0000","last_became_active":"2024-06-03T03:21:26.055505+0000","last_became_peered":"2024-06-03T03:21:26.055505+0000","last_unstale":"2024-06-03T03:22:17.309900+0000","last_undegraded":"2024-06-03T03:22:17.309900+0000","last_fullsized":"2024-06-03T03:22:17.309900+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:22:55.722566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,10],"acting":[4,1,2,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"97'5","reported_seq":52,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:02.447113+0000","last_change":"2024-06-03T03:19:36.744930+0000","last_active":"2024-06-03T03:22:02.447113+0000","last_peered":"2024-06-03T03:22:02.447113+0000","last_clean":"2024-06-03T03:22:02.447113+0000","last_became_active":"2024-06-03T03:19:36.744567+0000","last_became_peered":"2024-06-03T03:19:36.744567+0000","last_unstale":"2024-06-03T03:22:02.447113+0000","last_undegraded":"2024-06-03T03:22:02.447113+0000","last_fullsized":"2024-06-03T03:22:02.447113+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:35:31.927211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"99'11","reported_seq":51,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.812800+0000","last_change":"2024-06-03T03:19:41.607604+0000","last_active":"2024-06-03T03:22:54.812800+0000","last_peered":"2024-06-03T03:22:54.812800+0000","last_clean":"2024-06-03T03:22:54.812800+0000","last_became_active":"2024-06-03T03:19:41.607148+0000","last_became_peered":"2024-06-03T03:19:41.607148+0000","last_unstale":"2024-06-03T03:22:54.812800+0000","last_undegraded":"2024-06-03T03:22:54.812800+0000","last_fullsized":"2024-06-03T03:22:54.812800+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:44:44.037836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:20.886272+0000","last_change":"2024-06-03T03:19:56.928689+0000","last_active":"2024-06-03T03:22:20.886272+0000","last_peered":"2024-06-03T03:22:20.886272+0000","last_clean":"2024-06-03T03:22:20.886272+0000","last_became_active":"2024-06-03T03:19:56.924955+0000","last_became_peered":"2024-06-03T03:19:56.924955+0000","last_unstale":"2024-06-03T03:22:20.886272+0000","last_undegraded":"2024-06-03T03:22:20.886272+0000","last_fullsized":"2024-06-03T03:22:20.886272+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:11:07.413430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"99'46","reported_seq":604,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:25:16.411088+0000","last_change":"2024-06-03T03:19:36.539688+0000","last_active":"2024-06-03T03:25:16.411088+0000","last_peered":"2024-06-03T03:25:16.411088+0000","last_clean":"2024-06-03T03:25:16.411088+0000","last_became_active":"2024-06-03T03:19:36.539302+0000","last_became_peered":"2024-06-03T03:19:36.539302+0000","last_unstale":"2024-06-03T03:25:16.411088+0000","last_undegraded":"2024-06-03T03:25:16.411088+0000","last_fullsized":"2024-06-03T03:25:16.411088+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:28:52.790227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":74214,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":512,"num_read_kb":329,"num_write":48,"num_write_kb":95,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.815494+0000","last_change":"2024-06-03T03:19:41.608202+0000","last_active":"2024-06-03T03:22:54.815494+0000","last_peered":"2024-06-03T03:22:54.815494+0000","last_clean":"2024-06-03T03:22:54.815494+0000","last_became_active":"2024-06-03T03:19:41.607891+0000","last_became_peered":"2024-06-03T03:19:41.607891+0000","last_unstale":"2024-06-03T03:22:54.815494+0000","last_undegraded":"2024-06-03T03:22:54.815494+0000","last_fullsized":"2024-06-03T03:22:54.815494+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T14:10:44.042892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:22.449231+0000","last_change":"2024-06-03T03:19:56.923547+0000","last_active":"2024-06-03T03:22:22.449231+0000","last_peered":"2024-06-03T03:22:22.449231+0000","last_clean":"2024-06-03T03:22:22.449231+0000","last_became_active":"2024-06-03T03:19:56.923046+0000","last_became_peered":"2024-06-03T03:19:56.923046+0000","last_unstale":"2024-06-03T03:22:22.449231+0000","last_undegraded":"2024-06-03T03:22:22.449231+0000","last_fullsized":"2024-06-03T03:22:22.449231+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:32:19.438793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"97'9","reported_seq":56,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:59.125620+0000","last_change":"2024-06-03T03:19:36.547024+0000","last_active":"2024-06-03T03:21:59.125620+0000","last_peered":"2024-06-03T03:21:59.125620+0000","last_clean":"2024-06-03T03:21:59.125620+0000","last_became_active":"2024-06-03T03:19:36.546663+0000","last_became_peered":"2024-06-03T03:19:36.546663+0000","last_unstale":"2024-06-03T03:21:59.125620+0000","last_undegraded":"2024-06-03T03:21:59.125620+0000","last_fullsized":"2024-06-03T03:21:59.125620+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:58:41.446730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.830570+0000","last_change":"2024-06-03T03:19:41.630119+0000","last_active":"2024-06-03T03:22:54.830570+0000","last_peered":"2024-06-03T03:22:54.830570+0000","last_clean":"2024-06-03T03:22:54.830570+0000","last_became_active":"2024-06-03T03:19:41.629563+0000","last_became_peered":"2024-06-03T03:19:41.629563+0000","last_unstale":"2024-06-03T03:22:54.830570+0000","last_undegraded":"2024-06-03T03:22:54.830570+0000","last_fullsized":"2024-06-03T03:22:54.830570+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:49:33.983471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:19.316793+0000","last_change":"2024-06-03T03:19:56.941451+0000","last_active":"2024-06-03T03:22:19.316793+0000","last_peered":"2024-06-03T03:22:19.316793+0000","last_clean":"2024-06-03T03:22:19.316793+0000","last_became_active":"2024-06-03T03:19:56.941003+0000","last_became_peered":"2024-06-03T03:19:56.941003+0000","last_unstale":"2024-06-03T03:22:19.316793+0000","last_undegraded":"2024-06-03T03:22:19.316793+0000","last_fullsized":"2024-06-03T03:22:19.316793+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:28:58.505495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"97'5","reported_seq":52,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:57.446530+0000","last_change":"2024-06-03T03:19:36.547128+0000","last_active":"2024-06-03T03:21:57.446530+0000","last_peered":"2024-06-03T03:21:57.446530+0000","last_clean":"2024-06-03T03:21:57.446530+0000","last_became_active":"2024-06-03T03:19:36.546576+0000","last_became_peered":"2024-06-03T03:19:36.546576+0000","last_unstale":"2024-06-03T03:21:57.446530+0000","last_undegraded":"2024-06-03T03:21:57.446530+0000","last_fullsized":"2024-06-03T03:21:57.446530+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:23:30.196543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"99'15","reported_seq":55,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.836544+0000","last_change":"2024-06-03T03:19:41.634941+0000","last_active":"2024-06-03T03:22:54.836544+0000","last_peered":"2024-06-03T03:22:54.836544+0000","last_clean":"2024-06-03T03:22:54.836544+0000","last_became_active":"2024-06-03T03:19:41.634561+0000","last_became_peered":"2024-06-03T03:19:41.634561+0000","last_unstale":"2024-06-03T03:22:54.836544+0000","last_undegraded":"2024-06-03T03:22:54.836544+0000","last_fullsized":"2024-06-03T03:22:54.836544+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:00:27.258078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:25.887154+0000","last_change":"2024-06-03T03:19:56.934612+0000","last_active":"2024-06-03T03:22:25.887154+0000","last_peered":"2024-06-03T03:22:25.887154+0000","last_clean":"2024-06-03T03:22:25.887154+0000","last_became_active":"2024-06-03T03:19:56.930212+0000","last_became_peered":"2024-06-03T03:19:56.930212+0000","last_unstale":"2024-06-03T03:22:25.887154+0000","last_undegraded":"2024-06-03T03:22:25.887154+0000","last_fullsized":"2024-06-03T03:22:25.887154+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:42:38.490095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"97'5","reported_seq":301,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:25:15.522909+0000","last_change":"2024-06-03T03:19:36.542052+0000","last_active":"2024-06-03T03:25:15.522909+0000","last_peered":"2024-06-03T03:25:15.522909+0000","last_clean":"2024-06-03T03:25:15.522909+0000","last_became_active":"2024-06-03T03:19:36.540987+0000","last_became_peered":"2024-06-03T03:19:36.540987+0000","last_unstale":"2024-06-03T03:25:15.522909+0000","last_undegraded":"2024-06-03T03:25:15.522909+0000","last_fullsized":"2024-06-03T03:25:15.522909+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T15:18:48.821400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":252,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"99'6","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.818326+0000","last_change":"2024-06-03T03:19:41.607610+0000","last_active":"2024-06-03T03:22:54.818326+0000","last_peered":"2024-06-03T03:22:54.818326+0000","last_clean":"2024-06-03T03:22:54.818326+0000","last_became_active":"2024-06-03T03:19:41.607219+0000","last_became_peered":"2024-06-03T03:19:41.607219+0000","last_unstale":"2024-06-03T03:22:54.818326+0000","last_undegraded":"2024-06-03T03:22:54.818326+0000","last_fullsized":"2024-06-03T03:22:54.818326+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:06:20.707623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:23.282636+0000","last_change":"2024-06-03T03:19:56.936440+0000","last_active":"2024-06-03T03:22:23.282636+0000","last_peered":"2024-06-03T03:22:23.282636+0000","last_clean":"2024-06-03T03:22:23.282636+0000","last_became_active":"2024-06-03T03:19:56.936048+0000","last_became_peered":"2024-06-03T03:19:56.936048+0000","last_unstale":"2024-06-03T03:22:23.282636+0000","last_undegraded":"2024-06-03T03:22:23.282636+0000","last_fullsized":"2024-06-03T03:22:23.282636+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:22:15.758950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"97'17","reported_seq":63,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:59.896170+0000","last_change":"2024-06-03T03:19:36.736105+0000","last_active":"2024-06-03T03:21:59.896170+0000","last_peered":"2024-06-03T03:21:59.896170+0000","last_clean":"2024-06-03T03:21:59.896170+0000","last_became_active":"2024-06-03T03:19:36.735663+0000","last_became_peered":"2024-06-03T03:19:36.735663+0000","last_unstale":"2024-06-03T03:21:59.896170+0000","last_undegraded":"2024-06-03T03:21:59.896170+0000","last_fullsized":"2024-06-03T03:21:59.896170+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:41:46.216923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"99'6","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.814118+0000","last_change":"2024-06-03T03:19:41.609758+0000","last_active":"2024-06-03T03:22:54.814118+0000","last_peered":"2024-06-03T03:22:54.814118+0000","last_clean":"2024-06-03T03:22:54.814118+0000","last_became_active":"2024-06-03T03:19:41.609421+0000","last_became_peered":"2024-06-03T03:19:41.609421+0000","last_unstale":"2024-06-03T03:22:54.814118+0000","last_undegraded":"2024-06-03T03:22:54.814118+0000","last_fullsized":"2024-06-03T03:22:54.814118+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:45:54.919352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:07.447579+0000","last_change":"2024-06-03T03:19:56.925422+0000","last_active":"2024-06-03T03:22:07.447579+0000","last_peered":"2024-06-03T03:22:07.447579+0000","last_clean":"2024-06-03T03:22:07.447579+0000","last_became_active":"2024-06-03T03:19:56.924891+0000","last_became_peered":"2024-06-03T03:19:56.924891+0000","last_unstale":"2024-06-03T03:22:07.447579+0000","last_undegraded":"2024-06-03T03:22:07.447579+0000","last_fullsized":"2024-06-03T03:22:07.447579+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:34:59.986256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"97'2","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:54.895740+0000","last_change":"2024-06-03T03:19:36.540605+0000","last_active":"2024-06-03T03:21:54.895740+0000","last_peered":"2024-06-03T03:21:54.895740+0000","last_clean":"2024-06-03T03:21:54.895740+0000","last_became_active":"2024-06-03T03:19:36.540180+0000","last_became_peered":"2024-06-03T03:19:36.540180+0000","last_unstale":"2024-06-03T03:21:54.895740+0000","last_undegraded":"2024-06-03T03:21:54.895740+0000","last_fullsized":"2024-06-03T03:21:54.895740+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:01:26.556726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"99'5","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.825730+0000","last_change":"2024-06-03T03:20:25.904215+0000","last_active":"2024-06-03T03:22:54.825730+0000","last_peered":"2024-06-03T03:22:54.825730+0000","last_clean":"2024-06-03T03:22:54.825730+0000","last_became_active":"2024-06-03T03:20:25.903894+0000","last_became_peered":"2024-06-03T03:20:25.903894+0000","last_unstale":"2024-06-03T03:22:54.825730+0000","last_undegraded":"2024-06-03T03:22:54.825730+0000","last_fullsized":"2024-06-03T03:22:54.825730+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:02:07.587159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:04.896819+0000","last_change":"2024-06-03T03:19:56.910041+0000","last_active":"2024-06-03T03:22:04.896819+0000","last_peered":"2024-06-03T03:22:04.896819+0000","last_clean":"2024-06-03T03:22:04.896819+0000","last_became_active":"2024-06-03T03:19:56.909591+0000","last_became_peered":"2024-06-03T03:19:56.909591+0000","last_unstale":"2024-06-03T03:22:04.896819+0000","last_undegraded":"2024-06-03T03:22:04.896819+0000","last_fullsized":"2024-06-03T03:22:04.896819+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:42:20.016838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"97'1","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:40.880350+0000","last_change":"2024-06-03T03:19:36.741731+0000","last_active":"2024-06-03T03:21:40.880350+0000","last_peered":"2024-06-03T03:21:40.880350+0000","last_clean":"2024-06-03T03:21:40.880350+0000","last_became_active":"2024-06-03T03:19:36.741329+0000","last_became_peered":"2024-06-03T03:19:36.741329+0000","last_unstale":"2024-06-03T03:21:40.880350+0000","last_undegraded":"2024-06-03T03:21:40.880350+0000","last_fullsized":"2024-06-03T03:21:40.880350+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:02:45.778368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"99'4","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.825952+0000","last_change":"2024-06-03T03:19:41.632225+0000","last_active":"2024-06-03T03:22:54.825952+0000","last_peered":"2024-06-03T03:22:54.825952+0000","last_clean":"2024-06-03T03:22:54.825952+0000","last_became_active":"2024-06-03T03:19:41.631851+0000","last_became_peered":"2024-06-03T03:19:41.631851+0000","last_unstale":"2024-06-03T03:22:54.825952+0000","last_undegraded":"2024-06-03T03:22:54.825952+0000","last_fullsized":"2024-06-03T03:22:54.825952+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:52:23.364233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:03.279638+0000","last_change":"2024-06-03T03:19:56.934200+0000","last_active":"2024-06-03T03:22:03.279638+0000","last_peered":"2024-06-03T03:22:03.279638+0000","last_clean":"2024-06-03T03:22:03.279638+0000","last_became_active":"2024-06-03T03:19:56.932998+0000","last_became_peered":"2024-06-03T03:19:56.932998+0000","last_unstale":"2024-06-03T03:22:03.279638+0000","last_undegraded":"2024-06-03T03:22:03.279638+0000","last_fullsized":"2024-06-03T03:22:03.279638+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:45:28.916023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"99'6","reported_seq":301,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:25:13.519391+0000","last_change":"2024-06-03T03:19:36.546630+0000","last_active":"2024-06-03T03:25:13.519391+0000","last_peered":"2024-06-03T03:25:13.519391+0000","last_clean":"2024-06-03T03:25:13.519391+0000","last_became_active":"2024-06-03T03:19:36.546248+0000","last_became_peered":"2024-06-03T03:19:36.546248+0000","last_unstale":"2024-06-03T03:25:13.519391+0000","last_undegraded":"2024-06-03T03:25:13.519391+0000","last_fullsized":"2024-06-03T03:25:13.519391+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T15:09:50.323113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":249,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"99'7","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.831073+0000","last_change":"2024-06-03T03:19:41.624584+0000","last_active":"2024-06-03T03:22:54.831073+0000","last_peered":"2024-06-03T03:22:54.831073+0000","last_clean":"2024-06-03T03:22:54.831073+0000","last_became_active":"2024-06-03T03:19:41.624216+0000","last_became_peered":"2024-06-03T03:19:41.624216+0000","last_unstale":"2024-06-03T03:22:54.831073+0000","last_undegraded":"2024-06-03T03:22:54.831073+0000","last_fullsized":"2024-06-03T03:22:54.831073+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:22:09.645341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:08.280228+0000","last_change":"2024-06-03T03:19:56.937422+0000","last_active":"2024-06-03T03:22:08.280228+0000","last_peered":"2024-06-03T03:22:08.280228+0000","last_clean":"2024-06-03T03:22:08.280228+0000","last_became_active":"2024-06-03T03:19:56.934075+0000","last_became_peered":"2024-06-03T03:19:56.934075+0000","last_unstale":"2024-06-03T03:22:08.280228+0000","last_undegraded":"2024-06-03T03:22:08.280228+0000","last_fullsized":"2024-06-03T03:22:08.280228+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:25:45.273161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"99'8","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.814169+0000","last_change":"2024-06-03T03:19:41.589595+0000","last_active":"2024-06-03T03:22:54.814169+0000","last_peered":"2024-06-03T03:22:54.814169+0000","last_clean":"2024-06-03T03:22:54.814169+0000","last_became_active":"2024-06-03T03:19:41.589251+0000","last_became_peered":"2024-06-03T03:19:41.589251+0000","last_unstale":"2024-06-03T03:22:54.814169+0000","last_undegraded":"2024-06-03T03:22:54.814169+0000","last_fullsized":"2024-06-03T03:22:54.814169+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:51:40.417386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"97'5","reported_seq":301,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:25:16.524308+0000","last_change":"2024-06-03T03:19:36.737523+0000","last_active":"2024-06-03T03:25:16.524308+0000","last_peered":"2024-06-03T03:25:16.524308+0000","last_clean":"2024-06-03T03:25:16.524308+0000","last_became_active":"2024-06-03T03:19:36.736698+0000","last_became_peered":"2024-06-03T03:19:36.736698+0000","last_unstale":"2024-06-03T03:25:16.524308+0000","last_undegraded":"2024-06-03T03:25:16.524308+0000","last_fullsized":"2024-06-03T03:25:16.524308+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:14:46.330549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":251,"num_read_kb":251,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"97'1","reported_seq":31,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:04.852726+0000","last_change":"2024-06-03T03:19:56.924421+0000","last_active":"2024-06-03T03:22:04.852726+0000","last_peered":"2024-06-03T03:22:04.852726+0000","last_clean":"2024-06-03T03:22:04.852726+0000","last_became_active":"2024-06-03T03:19:56.924043+0000","last_became_peered":"2024-06-03T03:19:56.924043+0000","last_unstale":"2024-06-03T03:22:04.852726+0000","last_undegraded":"2024-06-03T03:22:04.852726+0000","last_fullsized":"2024-06-03T03:22:04.852726+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:00:47.674717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.826839+0000","last_change":"2024-06-03T03:19:41.613800+0000","last_active":"2024-06-03T03:22:54.826839+0000","last_peered":"2024-06-03T03:22:54.826839+0000","last_clean":"2024-06-03T03:22:54.826839+0000","last_became_active":"2024-06-03T03:19:41.613387+0000","last_became_peered":"2024-06-03T03:19:41.613387+0000","last_unstale":"2024-06-03T03:22:54.826839+0000","last_undegraded":"2024-06-03T03:22:54.826839+0000","last_fullsized":"2024-06-03T03:22:54.826839+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:04:04.785023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"97'6","reported_seq":54,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:42.305912+0000","last_change":"2024-06-03T03:19:36.544518+0000","last_active":"2024-06-03T03:21:42.305912+0000","last_peered":"2024-06-03T03:21:42.305912+0000","last_clean":"2024-06-03T03:21:42.305912+0000","last_became_active":"2024-06-03T03:19:36.544056+0000","last_became_peered":"2024-06-03T03:19:36.544056+0000","last_unstale":"2024-06-03T03:21:42.305912+0000","last_undegraded":"2024-06-03T03:21:42.305912+0000","last_fullsized":"2024-06-03T03:21:42.305912+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:33:25.012986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":368,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:39.856348+0000","last_change":"2024-06-03T03:18:05.481351+0000","last_active":"2024-06-03T03:22:39.856348+0000","last_peered":"2024-06-03T03:22:39.856348+0000","last_clean":"2024-06-03T03:22:39.856348+0000","last_became_active":"2024-06-03T03:18:05.480929+0000","last_became_peered":"2024-06-03T03:18:05.480929+0000","last_unstale":"2024-06-03T03:22:39.856348+0000","last_undegraded":"2024-06-03T03:22:39.856348+0000","last_fullsized":"2024-06-03T03:22:39.856348+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:14:09.558597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:14:09.558597+0000","last_clean_scrub_stamp":"2024-06-03T03:14:09.558597+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:54:45.563733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:07.447611+0000","last_change":"2024-06-03T03:19:56.931867+0000","last_active":"2024-06-03T03:22:07.447611+0000","last_peered":"2024-06-03T03:22:07.447611+0000","last_clean":"2024-06-03T03:22:07.447611+0000","last_became_active":"2024-06-03T03:19:56.931538+0000","last_became_peered":"2024-06-03T03:19:56.931538+0000","last_unstale":"2024-06-03T03:22:07.447611+0000","last_undegraded":"2024-06-03T03:22:07.447611+0000","last_fullsized":"2024-06-03T03:22:07.447611+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:55:15.144192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"99'5","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.814110+0000","last_change":"2024-06-03T03:19:41.622713+0000","last_active":"2024-06-03T03:22:54.814110+0000","last_peered":"2024-06-03T03:22:54.814110+0000","last_clean":"2024-06-03T03:22:54.814110+0000","last_became_active":"2024-06-03T03:19:41.622363+0000","last_became_peered":"2024-06-03T03:19:41.622363+0000","last_unstale":"2024-06-03T03:22:54.814110+0000","last_undegraded":"2024-06-03T03:22:54.814110+0000","last_fullsized":"2024-06-03T03:22:54.814110+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:43:57.671797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"97'4","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:39.850108+0000","last_change":"2024-06-03T03:19:36.539663+0000","last_active":"2024-06-03T03:21:39.850108+0000","last_peered":"2024-06-03T03:21:39.850108+0000","last_clean":"2024-06-03T03:21:39.850108+0000","last_became_active":"2024-06-03T03:19:36.539320+0000","last_became_peered":"2024-06-03T03:19:36.539320+0000","last_unstale":"2024-06-03T03:21:39.850108+0000","last_undegraded":"2024-06-03T03:21:39.850108+0000","last_fullsized":"2024-06-03T03:21:39.850108+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:36:50.699231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:04.314937+0000","last_change":"2024-06-03T03:19:56.938231+0000","last_active":"2024-06-03T03:22:04.314937+0000","last_peered":"2024-06-03T03:22:04.314937+0000","last_clean":"2024-06-03T03:22:04.314937+0000","last_became_active":"2024-06-03T03:19:56.937840+0000","last_became_peered":"2024-06-03T03:19:56.937840+0000","last_unstale":"2024-06-03T03:22:04.314937+0000","last_undegraded":"2024-06-03T03:22:04.314937+0000","last_fullsized":"2024-06-03T03:22:04.314937+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:02:33.916980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"97'4","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:44.894507+0000","last_change":"2024-06-03T03:19:36.737519+0000","last_active":"2024-06-03T03:21:44.894507+0000","last_peered":"2024-06-03T03:21:44.894507+0000","last_clean":"2024-06-03T03:21:44.894507+0000","last_became_active":"2024-06-03T03:19:36.737191+0000","last_became_peered":"2024-06-03T03:19:36.737191+0000","last_unstale":"2024-06-03T03:21:44.894507+0000","last_undegraded":"2024-06-03T03:21:44.894507+0000","last_fullsized":"2024-06-03T03:21:44.894507+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:50:21.389739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.817624+0000","last_change":"2024-06-03T03:19:41.633113+0000","last_active":"2024-06-03T03:22:54.817624+0000","last_peered":"2024-06-03T03:22:54.817624+0000","last_clean":"2024-06-03T03:22:54.817624+0000","last_became_active":"2024-06-03T03:19:41.632186+0000","last_became_peered":"2024-06-03T03:19:41.632186+0000","last_unstale":"2024-06-03T03:22:54.817624+0000","last_undegraded":"2024-06-03T03:22:54.817624+0000","last_fullsized":"2024-06-03T03:22:54.817624+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T15:12:26.835061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:56.770202+0000","last_change":"2024-06-03T03:19:56.928603+0000","last_active":"2024-06-03T03:21:56.770202+0000","last_peered":"2024-06-03T03:21:56.770202+0000","last_clean":"2024-06-03T03:21:56.770202+0000","last_became_active":"2024-06-03T03:19:56.928175+0000","last_became_peered":"2024-06-03T03:19:56.928175+0000","last_unstale":"2024-06-03T03:21:56.770202+0000","last_undegraded":"2024-06-03T03:21:56.770202+0000","last_fullsized":"2024-06-03T03:21:56.770202+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:26:03.531091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"99'21","reported_seq":67,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:25:10.676315+0000","last_change":"2024-06-03T03:19:36.550577+0000","last_active":"2024-06-03T03:25:10.676315+0000","last_peered":"2024-06-03T03:25:10.676315+0000","last_clean":"2024-06-03T03:25:10.676315+0000","last_became_active":"2024-06-03T03:19:36.549968+0000","last_became_peered":"2024-06-03T03:19:36.549968+0000","last_unstale":"2024-06-03T03:25:10.676315+0000","last_undegraded":"2024-06-03T03:25:10.676315+0000","last_fullsized":"2024-06-03T03:25:10.676315+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:22:43.892002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":38,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"99'5","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.825483+0000","last_change":"2024-06-03T03:19:41.613883+0000","last_active":"2024-06-03T03:22:54.825483+0000","last_peered":"2024-06-03T03:22:54.825483+0000","last_clean":"2024-06-03T03:22:54.825483+0000","last_became_active":"2024-06-03T03:19:41.613472+0000","last_became_peered":"2024-06-03T03:19:41.613472+0000","last_unstale":"2024-06-03T03:22:54.825483+0000","last_undegraded":"2024-06-03T03:22:54.825483+0000","last_fullsized":"2024-06-03T03:22:54.825483+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T14:38:42.044534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:04.126263+0000","last_change":"2024-06-03T03:19:56.914487+0000","last_active":"2024-06-03T03:22:04.126263+0000","last_peered":"2024-06-03T03:22:04.126263+0000","last_clean":"2024-06-03T03:22:04.126263+0000","last_became_active":"2024-06-03T03:19:56.913904+0000","last_became_peered":"2024-06-03T03:19:56.913904+0000","last_unstale":"2024-06-03T03:22:04.126263+0000","last_undegraded":"2024-06-03T03:22:04.126263+0000","last_fullsized":"2024-06-03T03:22:04.126263+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T15:13:24.084034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"97'7","reported_seq":53,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:47.306495+0000","last_change":"2024-06-03T03:19:36.671355+0000","last_active":"2024-06-03T03:21:47.306495+0000","last_peered":"2024-06-03T03:21:47.306495+0000","last_clean":"2024-06-03T03:21:47.306495+0000","last_became_active":"2024-06-03T03:19:36.670926+0000","last_became_peered":"2024-06-03T03:19:36.670926+0000","last_unstale":"2024-06-03T03:21:47.306495+0000","last_undegraded":"2024-06-03T03:21:47.306495+0000","last_fullsized":"2024-06-03T03:21:47.306495+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:38:50.914238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.835937+0000","last_change":"2024-06-03T03:19:41.629054+0000","last_active":"2024-06-03T03:22:54.835937+0000","last_peered":"2024-06-03T03:22:54.835937+0000","last_clean":"2024-06-03T03:22:54.835937+0000","last_became_active":"2024-06-03T03:19:41.627118+0000","last_became_peered":"2024-06-03T03:19:41.627118+0000","last_unstale":"2024-06-03T03:22:54.835937+0000","last_undegraded":"2024-06-03T03:22:54.835937+0000","last_fullsized":"2024-06-03T03:22:54.835937+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:59:40.645002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:12.309263+0000","last_change":"2024-06-03T03:21:26.058741+0000","last_active":"2024-06-03T03:22:12.309263+0000","last_peered":"2024-06-03T03:22:12.309263+0000","last_clean":"2024-06-03T03:22:12.309263+0000","last_became_active":"2024-06-03T03:21:26.058271+0000","last_became_peered":"2024-06-03T03:21:26.058271+0000","last_unstale":"2024-06-03T03:22:12.309263+0000","last_undegraded":"2024-06-03T03:22:12.309263+0000","last_fullsized":"2024-06-03T03:22:12.309263+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:31:32.359893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"97'2","reported_seq":54,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:50.231392+0000","last_change":"2024-06-03T03:20:26.132994+0000","last_active":"2024-06-03T03:21:50.231392+0000","last_peered":"2024-06-03T03:21:50.231392+0000","last_clean":"2024-06-03T03:21:50.231392+0000","last_became_active":"2024-06-03T03:20:26.132574+0000","last_became_peered":"2024-06-03T03:20:26.132574+0000","last_unstale":"2024-06-03T03:21:50.231392+0000","last_undegraded":"2024-06-03T03:21:50.231392+0000","last_fullsized":"2024-06-03T03:21:50.231392+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T15:12:33.599154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"99'2","reported_seq":41,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.791970+0000","last_change":"2024-06-03T03:19:41.603630+0000","last_active":"2024-06-03T03:22:54.791970+0000","last_peered":"2024-06-03T03:22:54.791970+0000","last_clean":"2024-06-03T03:22:54.791970+0000","last_became_active":"2024-06-03T03:19:41.603301+0000","last_became_peered":"2024-06-03T03:19:41.603301+0000","last_unstale":"2024-06-03T03:22:54.791970+0000","last_undegraded":"2024-06-03T03:22:54.791970+0000","last_fullsized":"2024-06-03T03:22:54.791970+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:20:33.076622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:15.885256+0000","last_change":"2024-06-03T03:19:56.929023+0000","last_active":"2024-06-03T03:22:15.885256+0000","last_peered":"2024-06-03T03:22:15.885256+0000","last_clean":"2024-06-03T03:22:15.885256+0000","last_became_active":"2024-06-03T03:19:56.928454+0000","last_became_peered":"2024-06-03T03:19:56.928454+0000","last_unstale":"2024-06-03T03:22:15.885256+0000","last_undegraded":"2024-06-03T03:22:15.885256+0000","last_fullsized":"2024-06-03T03:22:15.885256+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:17:31.755649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"97'7","reported_seq":52,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:46.768385+0000","last_change":"2024-06-03T03:19:36.551413+0000","last_active":"2024-06-03T03:21:46.768385+0000","last_peered":"2024-06-03T03:21:46.768385+0000","last_clean":"2024-06-03T03:21:46.768385+0000","last_became_active":"2024-06-03T03:19:36.550622+0000","last_became_peered":"2024-06-03T03:19:36.550622+0000","last_unstale":"2024-06-03T03:21:46.768385+0000","last_undegraded":"2024-06-03T03:21:46.768385+0000","last_fullsized":"2024-06-03T03:21:46.768385+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:50:09.472960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.830703+0000","last_change":"2024-06-03T03:19:41.624871+0000","last_active":"2024-06-03T03:22:54.830703+0000","last_peered":"2024-06-03T03:22:54.830703+0000","last_clean":"2024-06-03T03:22:54.830703+0000","last_became_active":"2024-06-03T03:19:41.624638+0000","last_became_peered":"2024-06-03T03:19:41.624638+0000","last_unstale":"2024-06-03T03:22:54.830703+0000","last_undegraded":"2024-06-03T03:22:54.830703+0000","last_fullsized":"2024-06-03T03:22:54.830703+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:21:16.193817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"99'1","reported_seq":31,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:12.309224+0000","last_change":"2024-06-03T03:19:56.924638+0000","last_active":"2024-06-03T03:22:12.309224+0000","last_peered":"2024-06-03T03:22:12.309224+0000","last_clean":"2024-06-03T03:22:12.309224+0000","last_became_active":"2024-06-03T03:19:56.924200+0000","last_became_peered":"2024-06-03T03:19:56.924200+0000","last_unstale":"2024-06-03T03:22:12.309224+0000","last_undegraded":"2024-06-03T03:22:12.309224+0000","last_fullsized":"2024-06-03T03:22:12.309224+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:38:01.138253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"99'5","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:25:11.017683+0000","last_change":"2024-06-03T03:19:36.546974+0000","last_active":"2024-06-03T03:25:11.017683+0000","last_peered":"2024-06-03T03:25:11.017683+0000","last_clean":"2024-06-03T03:25:11.017683+0000","last_became_active":"2024-06-03T03:19:36.546433+0000","last_became_peered":"2024-06-03T03:19:36.546433+0000","last_unstale":"2024-06-03T03:25:11.017683+0000","last_undegraded":"2024-06-03T03:25:11.017683+0000","last_fullsized":"2024-06-03T03:25:11.017683+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:41:46.980327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.834248+0000","last_change":"2024-06-03T03:19:41.622118+0000","last_active":"2024-06-03T03:22:54.834248+0000","last_peered":"2024-06-03T03:22:54.834248+0000","last_clean":"2024-06-03T03:22:54.834248+0000","last_became_active":"2024-06-03T03:19:41.620789+0000","last_became_peered":"2024-06-03T03:19:41.620789+0000","last_unstale":"2024-06-03T03:22:54.834248+0000","last_undegraded":"2024-06-03T03:22:54.834248+0000","last_fullsized":"2024-06-03T03:22:54.834248+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:00:36.232238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:12.309247+0000","last_change":"2024-06-03T03:19:56.921299+0000","last_active":"2024-06-03T03:22:12.309247+0000","last_peered":"2024-06-03T03:22:12.309247+0000","last_clean":"2024-06-03T03:22:12.309247+0000","last_became_active":"2024-06-03T03:19:56.920870+0000","last_became_peered":"2024-06-03T03:19:56.920870+0000","last_unstale":"2024-06-03T03:22:12.309247+0000","last_undegraded":"2024-06-03T03:22:12.309247+0000","last_fullsized":"2024-06-03T03:22:12.309247+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:10:15.062381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"97'4","reported_seq":24,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:44.319531+0000","last_change":"2024-06-03T03:20:25.903279+0000","last_active":"2024-06-03T03:22:44.319531+0000","last_peered":"2024-06-03T03:22:44.319531+0000","last_clean":"2024-06-03T03:22:44.319531+0000","last_became_active":"2024-06-03T03:20:25.900311+0000","last_became_peered":"2024-06-03T03:20:25.900311+0000","last_unstale":"2024-06-03T03:22:44.319531+0000","last_undegraded":"2024-06-03T03:22:44.319531+0000","last_fullsized":"2024-06-03T03:22:44.319531+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:25:49.708581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"99'5","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.836223+0000","last_change":"2024-06-03T03:19:41.628659+0000","last_active":"2024-06-03T03:22:54.836223+0000","last_peered":"2024-06-03T03:22:54.836223+0000","last_clean":"2024-06-03T03:22:54.836223+0000","last_became_active":"2024-06-03T03:19:41.628291+0000","last_became_peered":"2024-06-03T03:19:41.628291+0000","last_unstale":"2024-06-03T03:22:54.836223+0000","last_undegraded":"2024-06-03T03:22:54.836223+0000","last_fullsized":"2024-06-03T03:22:54.836223+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:26:49.579669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:10.233742+0000","last_change":"2024-06-03T03:19:56.922766+0000","last_active":"2024-06-03T03:22:10.233742+0000","last_peered":"2024-06-03T03:22:10.233742+0000","last_clean":"2024-06-03T03:22:10.233742+0000","last_became_active":"2024-06-03T03:19:56.922310+0000","last_became_peered":"2024-06-03T03:19:56.922310+0000","last_unstale":"2024-06-03T03:22:10.233742+0000","last_undegraded":"2024-06-03T03:22:10.233742+0000","last_fullsized":"2024-06-03T03:22:10.233742+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T14:35:53.133519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"99'6","reported_seq":275,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:25:15.410739+0000","last_change":"2024-06-03T03:20:26.133562+0000","last_active":"2024-06-03T03:25:15.410739+0000","last_peered":"2024-06-03T03:25:15.410739+0000","last_clean":"2024-06-03T03:25:15.410739+0000","last_became_active":"2024-06-03T03:20:26.133074+0000","last_became_peered":"2024-06-03T03:20:26.133074+0000","last_unstale":"2024-06-03T03:25:15.410739+0000","last_undegraded":"2024-06-03T03:25:15.410739+0000","last_fullsized":"2024-06-03T03:25:15.410739+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:51:33.717584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1560,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":252,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"99'7","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.818537+0000","last_change":"2024-06-03T03:19:41.611450+0000","last_active":"2024-06-03T03:22:54.818537+0000","last_peered":"2024-06-03T03:22:54.818537+0000","last_clean":"2024-06-03T03:22:54.818537+0000","last_became_active":"2024-06-03T03:19:41.610214+0000","last_became_peered":"2024-06-03T03:19:41.610214+0000","last_unstale":"2024-06-03T03:22:54.818537+0000","last_undegraded":"2024-06-03T03:22:54.818537+0000","last_fullsized":"2024-06-03T03:22:54.818537+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:01:01.521917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:14.845604+0000","last_change":"2024-06-03T03:19:56.935686+0000","last_active":"2024-06-03T03:22:14.845604+0000","last_peered":"2024-06-03T03:22:14.845604+0000","last_clean":"2024-06-03T03:22:14.845604+0000","last_became_active":"2024-06-03T03:19:56.935314+0000","last_became_peered":"2024-06-03T03:19:56.935314+0000","last_unstale":"2024-06-03T03:22:14.845604+0000","last_undegraded":"2024-06-03T03:22:14.845604+0000","last_fullsized":"2024-06-03T03:22:14.845604+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T15:10:16.966321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"97'3","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:52.445989+0000","last_change":"2024-06-03T03:19:36.547756+0000","last_active":"2024-06-03T03:21:52.445989+0000","last_peered":"2024-06-03T03:21:52.445989+0000","last_clean":"2024-06-03T03:21:52.445989+0000","last_became_active":"2024-06-03T03:19:36.547350+0000","last_became_peered":"2024-06-03T03:19:36.547350+0000","last_unstale":"2024-06-03T03:21:52.445989+0000","last_undegraded":"2024-06-03T03:21:52.445989+0000","last_fullsized":"2024-06-03T03:21:52.445989+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:57:39.214380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"99'6","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.828041+0000","last_change":"2024-06-03T03:19:41.621083+0000","last_active":"2024-06-03T03:22:54.828041+0000","last_peered":"2024-06-03T03:22:54.828041+0000","last_clean":"2024-06-03T03:22:54.828041+0000","last_became_active":"2024-06-03T03:19:41.620719+0000","last_became_peered":"2024-06-03T03:19:41.620719+0000","last_unstale":"2024-06-03T03:22:54.828041+0000","last_undegraded":"2024-06-03T03:22:54.828041+0000","last_fullsized":"2024-06-03T03:22:54.828041+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:50:35.007025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:07.447552+0000","last_change":"2024-06-03T03:19:56.915406+0000","last_active":"2024-06-03T03:22:07.447552+0000","last_peered":"2024-06-03T03:22:07.447552+0000","last_clean":"2024-06-03T03:22:07.447552+0000","last_became_active":"2024-06-03T03:19:56.914918+0000","last_became_peered":"2024-06-03T03:19:56.914918+0000","last_unstale":"2024-06-03T03:22:07.447552+0000","last_undegraded":"2024-06-03T03:22:07.447552+0000","last_fullsized":"2024-06-03T03:22:07.447552+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:42:59.324011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"97'5","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:53.277541+0000","last_change":"2024-06-03T03:19:36.549502+0000","last_active":"2024-06-03T03:21:53.277541+0000","last_peered":"2024-06-03T03:21:53.277541+0000","last_clean":"2024-06-03T03:21:53.277541+0000","last_became_active":"2024-06-03T03:19:36.547538+0000","last_became_peered":"2024-06-03T03:19:36.547538+0000","last_unstale":"2024-06-03T03:21:53.277541+0000","last_undegraded":"2024-06-03T03:21:53.277541+0000","last_fullsized":"2024-06-03T03:21:53.277541+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:28:50.558069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"99'9","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.815966+0000","last_change":"2024-06-03T03:19:41.607441+0000","last_active":"2024-06-03T03:22:54.815966+0000","last_peered":"2024-06-03T03:22:54.815966+0000","last_clean":"2024-06-03T03:22:54.815966+0000","last_became_active":"2024-06-03T03:19:41.607086+0000","last_became_peered":"2024-06-03T03:19:41.607086+0000","last_unstale":"2024-06-03T03:22:54.815966+0000","last_undegraded":"2024-06-03T03:22:54.815966+0000","last_fullsized":"2024-06-03T03:22:54.815966+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T14:53:50.091872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.819602+0000","last_change":"2024-06-03T03:19:41.607004+0000","last_active":"2024-06-03T03:22:54.819602+0000","last_peered":"2024-06-03T03:22:54.819602+0000","last_clean":"2024-06-03T03:22:54.819602+0000","last_became_active":"2024-06-03T03:19:41.606684+0000","last_became_peered":"2024-06-03T03:19:41.606684+0000","last_unstale":"2024-06-03T03:22:54.819602+0000","last_undegraded":"2024-06-03T03:22:54.819602+0000","last_fullsized":"2024-06-03T03:22:54.819602+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:48:55.320597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"99'12","reported_seq":51,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.817158+0000","last_change":"2024-06-03T03:19:41.589426+0000","last_active":"2024-06-03T03:22:54.817158+0000","last_peered":"2024-06-03T03:22:54.817158+0000","last_clean":"2024-06-03T03:22:54.817158+0000","last_became_active":"2024-06-03T03:19:41.589091+0000","last_became_peered":"2024-06-03T03:19:41.589091+0000","last_unstale":"2024-06-03T03:22:54.817158+0000","last_undegraded":"2024-06-03T03:22:54.817158+0000","last_fullsized":"2024-06-03T03:22:54.817158+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:40:34.313214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.832557+0000","last_change":"2024-06-03T03:19:41.626149+0000","last_active":"2024-06-03T03:22:54.832557+0000","last_peered":"2024-06-03T03:22:54.832557+0000","last_clean":"2024-06-03T03:22:54.832557+0000","last_became_active":"2024-06-03T03:19:41.625710+0000","last_became_peered":"2024-06-03T03:19:41.625710+0000","last_unstale":"2024-06-03T03:22:54.832557+0000","last_undegraded":"2024-06-03T03:22:54.832557+0000","last_fullsized":"2024-06-03T03:22:54.832557+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:15:15.220975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.838595+0000","last_change":"2024-06-03T03:19:41.636585+0000","last_active":"2024-06-03T03:22:54.838595+0000","last_peered":"2024-06-03T03:22:54.838595+0000","last_clean":"2024-06-03T03:22:54.838595+0000","last_became_active":"2024-06-03T03:19:41.636241+0000","last_became_peered":"2024-06-03T03:19:41.636241+0000","last_unstale":"2024-06-03T03:22:54.838595+0000","last_undegraded":"2024-06-03T03:22:54.838595+0000","last_fullsized":"2024-06-03T03:22:54.838595+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:19:28.548297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"99'4","reported_seq":43,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.834220+0000","last_change":"2024-06-03T03:19:41.629530+0000","last_active":"2024-06-03T03:22:54.834220+0000","last_peered":"2024-06-03T03:22:54.834220+0000","last_clean":"2024-06-03T03:22:54.834220+0000","last_became_active":"2024-06-03T03:19:41.629197+0000","last_became_peered":"2024-06-03T03:19:41.629197+0000","last_unstale":"2024-06-03T03:22:54.834220+0000","last_undegraded":"2024-06-03T03:22:54.834220+0000","last_fullsized":"2024-06-03T03:22:54.834220+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:12:02.807445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.819264+0000","last_change":"2024-06-03T03:19:41.606995+0000","last_active":"2024-06-03T03:22:54.819264+0000","last_peered":"2024-06-03T03:22:54.819264+0000","last_clean":"2024-06-03T03:22:54.819264+0000","last_became_active":"2024-06-03T03:19:41.606651+0000","last_became_peered":"2024-06-03T03:19:41.606651+0000","last_unstale":"2024-06-03T03:22:54.819264+0000","last_undegraded":"2024-06-03T03:22:54.819264+0000","last_fullsized":"2024-06-03T03:22:54.819264+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:07:11.751606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"99'15","reported_seq":54,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.810136+0000","last_change":"2024-06-03T03:19:41.609848+0000","last_active":"2024-06-03T03:22:54.810136+0000","last_peered":"2024-06-03T03:22:54.810136+0000","last_clean":"2024-06-03T03:22:54.810136+0000","last_became_active":"2024-06-03T03:19:41.609544+0000","last_became_peered":"2024-06-03T03:19:41.609544+0000","last_unstale":"2024-06-03T03:22:54.810136+0000","last_undegraded":"2024-06-03T03:22:54.810136+0000","last_fullsized":"2024-06-03T03:22:54.810136+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:12:00.804638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"99'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.810509+0000","last_change":"2024-06-03T03:20:25.912049+0000","last_active":"2024-06-03T03:22:54.810509+0000","last_peered":"2024-06-03T03:22:54.810509+0000","last_clean":"2024-06-03T03:22:54.810509+0000","last_became_active":"2024-06-03T03:20:25.910664+0000","last_became_peered":"2024-06-03T03:20:25.910664+0000","last_unstale":"2024-06-03T03:22:54.810509+0000","last_undegraded":"2024-06-03T03:22:54.810509+0000","last_fullsized":"2024-06-03T03:22:54.810509+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:47:19.282569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"99'6","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.828822+0000","last_change":"2024-06-03T03:19:41.610800+0000","last_active":"2024-06-03T03:22:54.828822+0000","last_peered":"2024-06-03T03:22:54.828822+0000","last_clean":"2024-06-03T03:22:54.828822+0000","last_became_active":"2024-06-03T03:19:41.610457+0000","last_became_peered":"2024-06-03T03:19:41.610457+0000","last_unstale":"2024-06-03T03:22:54.828822+0000","last_undegraded":"2024-06-03T03:22:54.828822+0000","last_fullsized":"2024-06-03T03:22:54.828822+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:31:06.415159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.821379+0000","last_change":"2024-06-03T03:19:41.617020+0000","last_active":"2024-06-03T03:22:54.821379+0000","last_peered":"2024-06-03T03:22:54.821379+0000","last_clean":"2024-06-03T03:22:54.821379+0000","last_became_active":"2024-06-03T03:19:41.616614+0000","last_became_peered":"2024-06-03T03:19:41.616614+0000","last_unstale":"2024-06-03T03:22:54.821379+0000","last_undegraded":"2024-06-03T03:22:54.821379+0000","last_fullsized":"2024-06-03T03:22:54.821379+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:35:32.657685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"99'10","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.828254+0000","last_change":"2024-06-03T03:19:41.622785+0000","last_active":"2024-06-03T03:22:54.828254+0000","last_peered":"2024-06-03T03:22:54.828254+0000","last_clean":"2024-06-03T03:22:54.828254+0000","last_became_active":"2024-06-03T03:19:41.622407+0000","last_became_peered":"2024-06-03T03:19:41.622407+0000","last_unstale":"2024-06-03T03:22:54.828254+0000","last_undegraded":"2024-06-03T03:22:54.828254+0000","last_fullsized":"2024-06-03T03:22:54.828254+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:53:50.804889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"99'11","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.838978+0000","last_change":"2024-06-03T03:19:41.633429+0000","last_active":"2024-06-03T03:22:54.838978+0000","last_peered":"2024-06-03T03:22:54.838978+0000","last_clean":"2024-06-03T03:22:54.838978+0000","last_became_active":"2024-06-03T03:19:41.631028+0000","last_became_peered":"2024-06-03T03:19:41.631028+0000","last_unstale":"2024-06-03T03:22:54.838978+0000","last_undegraded":"2024-06-03T03:22:54.838978+0000","last_fullsized":"2024-06-03T03:22:54.838978+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:37:10.666591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.818463+0000","last_change":"2024-06-03T03:19:41.606204+0000","last_active":"2024-06-03T03:22:54.818463+0000","last_peered":"2024-06-03T03:22:54.818463+0000","last_clean":"2024-06-03T03:22:54.818463+0000","last_became_active":"2024-06-03T03:19:41.605756+0000","last_became_peered":"2024-06-03T03:19:41.605756+0000","last_unstale":"2024-06-03T03:22:54.818463+0000","last_undegraded":"2024-06-03T03:22:54.818463+0000","last_fullsized":"2024-06-03T03:22:54.818463+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:24:32.507426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":245318,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":82643,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":2538,"num_read_kb":1361,"num_write":283,"num_write_kb":174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589824,"data_stored":247929,"data_compressed":5580,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":241,"ondisk_log_size":241,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383764,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1101824,"data_stored":570765,"data_compressed":32353,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645402,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32404,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712748,"statfs":{"total":95995035648,"available":95961853952,"internally_reserved":0,"allocated":1257472,"data_stored":1231405,"data_compressed":69963,"data_compressed_allocated":94208,"data_compressed_original":688128,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808927,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32280,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712872,"statfs":{"total":95995035648,"available":95961980928,"internally_reserved":0,"allocated":1130496,"data_stored":574401,"data_compressed":32353,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070566,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32272,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712880,"statfs":{"total":95995035648,"available":95961989120,"internally_reserved":0,"allocated":1122304,"data_stored":570683,"data_compressed":32353,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332203,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32324,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712828,"statfs":{"total":95995035648,"available":95961935872,"internally_reserved":0,"allocated":1175552,"data_stored":1168406,"data_compressed":68103,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495730,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1101824,"data_stored":566199,"data_compressed":32353,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574989,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28164,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716988,"statfs":{"total":95995035648,"available":95966195712,"internally_reserved":0,"allocated":1175552,"data_stored":641443,"data_compressed":34213,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738513,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1126400,"data_stored":567655,"data_compressed":32353,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411463,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36496,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708656,"statfs":{"total":95995035648,"available":95957663744,"internally_reserved":0,"allocated":1187840,"data_stored":641330,"data_compressed":34213,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182530,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1114112,"data_stored":565517,"data_compressed":32353,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920893,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28108,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717044,"statfs":{"total":95995035648,"available":95966253056,"internally_reserved":0,"allocated":1118208,"data_stored":571443,"data_compressed":32353,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757368,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32332,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712820,"statfs":{"total":95995035648,"available":95961927680,"internally_reserved":0,"allocated":1183744,"data_stored":1163598,"data_compressed":68103,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34106,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34106,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34106,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35750,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35750,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35750,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35750,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":76356,"data_compressed":1860,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":76243,"data_compressed":1860,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":4047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":75950,"data_compressed":1860,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-03T03:25:19.957 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-06-03T03:25:19.957 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-06-03T03:25:20.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:20.201+0000 7fc348600640 1 -- 172.21.15.117:0/3788253357 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc340102470 msgr2=0x7fc340104860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:20.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:20.201+0000 7fc348600640 1 --2- 172.21.15.117:0/3788253357 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc340102470 0x7fc340104860 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc334009900 tx=0x7fc33402a3f0 comp rx=0 tx=0).stop 2024-06-03T03:25:20.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:20.201+0000 7fc348600640 1 -- 172.21.15.117:0/3788253357 shutdown_connections 2024-06-03T03:25:20.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:20.201+0000 7fc348600640 1 --2- 172.21.15.117:0/3788253357 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3401076d0 0x7fc340109b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:20.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:20.201+0000 7fc348600640 1 --2- 172.21.15.117:0/3788253357 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc340104da0 0x7fc340107190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:20.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:20.201+0000 7fc348600640 1 --2- 172.21.15.117:0/3788253357 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc340102470 0x7fc340104860 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:20.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:20.201+0000 7fc348600640 1 -- 172.21.15.117:0/3788253357 >> 172.21.15.117:0/3788253357 conn(0x7fc3400fbe80 msgr2=0x7fc3400fe2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:25:20.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:20.201+0000 7fc348600640 1 -- 172.21.15.117:0/3788253357 shutdown_connections 2024-06-03T03:25:20.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:20.201+0000 7fc348600640 1 -- 172.21.15.117:0/3788253357 wait complete. 2024-06-03T03:25:20.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:20.202+0000 7fc348600640 1 Processor -- start 2024-06-03T03:25:20.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:20.203+0000 7fc348600640 1 -- start start 2024-06-03T03:25:20.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:20.203+0000 7fc348600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc340102470 0x7fc340101de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:20.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:20.203+0000 7fc348600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc340104da0 0x7fc340100420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:20.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:20.203+0000 7fc348600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3401076d0 0x7fc340100960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:20.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:20.203+0000 7fc348600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc34010d690 con 0x7fc3401076d0 2024-06-03T03:25:20.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:20.203+0000 7fc348600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc34010d510 con 0x7fc340102470 2024-06-03T03:25:20.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:20.203+0000 7fc348600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc34010d810 con 0x7fc340104da0 2024-06-03T03:25:20.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:20.203+0000 7fc346600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3401076d0 0x7fc340100960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:20.206 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:20.203+0000 7fc346600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3401076d0 0x7fc340100960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39836/0 (socket says 172.21.15.117:39836) 2024-06-03T03:25:20.206 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:20.203+0000 7fc346600640 1 -- 172.21.15.117:0/655277513 learned_addr learned my addr 172.21.15.117:0/655277513 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:25:20.206 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:20.203+0000 7fc345200640 1 --2- 172.21.15.117:0/655277513 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc340104da0 0x7fc340100420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:20.206 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:20.203+0000 7fc345c00640 1 --2- 172.21.15.117:0/655277513 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc340102470 0x7fc340101de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:20.206 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:20.204+0000 7fc346600640 1 -- 172.21.15.117:0/655277513 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc340104da0 msgr2=0x7fc340100420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:20.206 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:20.204+0000 7fc346600640 1 --2- 172.21.15.117:0/655277513 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc340104da0 0x7fc340100420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:20.206 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:20.204+0000 7fc346600640 1 -- 172.21.15.117:0/655277513 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc340102470 msgr2=0x7fc340101de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:20.206 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:20.204+0000 7fc346600640 1 --2- 172.21.15.117:0/655277513 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc340102470 0x7fc340101de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:20.206 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:20.204+0000 7fc346600640 1 -- 172.21.15.117:0/655277513 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3401011f0 con 0x7fc3401076d0 2024-06-03T03:25:20.206 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:20.204+0000 7fc346600640 1 --2- 172.21.15.117:0/655277513 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3401076d0 0x7fc340100960 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc33800eaa0 tx=0x7fc33800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:25:20.206 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:20.204+0000 7fc33e000640 1 -- 172.21.15.117:0/655277513 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc33800cdb0 con 0x7fc3401076d0 2024-06-03T03:25:20.206 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:20.204+0000 7fc33e000640 1 -- 172.21.15.117:0/655277513 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc338005020 con 0x7fc3401076d0 2024-06-03T03:25:20.206 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:20.204+0000 7fc33e000640 1 -- 172.21.15.117:0/655277513 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3380116b0 con 0x7fc3401076d0 2024-06-03T03:25:20.207 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:20.204+0000 7fc348600640 1 -- 172.21.15.117:0/655277513 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3401a1e80 con 0x7fc3401076d0 2024-06-03T03:25:20.207 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:20.205+0000 7fc348600640 1 -- 172.21.15.117:0/655277513 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3401a22e0 con 0x7fc3401076d0 2024-06-03T03:25:20.207 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:20.206+0000 7fc348600640 1 -- 172.21.15.117:0/655277513 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3400fd970 con 0x7fc3401076d0 2024-06-03T03:25:20.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:20.208+0000 7fc33e000640 1 -- 172.21.15.117:0/655277513 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7fc338005560 con 0x7fc3401076d0 2024-06-03T03:25:20.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:20.209+0000 7fc33e000640 1 --2- 172.21.15.117:0/655277513 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fc3280a2c20 0x7fc3280a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:20.210 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:20.209+0000 7fc33e000640 1 -- 172.21.15.117:0/655277513 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7fc3380cf6f0 con 0x7fc3401076d0 2024-06-03T03:25:20.210 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:20.209+0000 7fc345c00640 1 --2- 172.21.15.117:0/655277513 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fc3280a2c20 0x7fc3280a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:20.210 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:20.210+0000 7fc345c00640 1 --2- 172.21.15.117:0/655277513 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fc3280a2c20 0x7fc3280a50e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc3340314e0 tx=0x7fc334000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:25:20.214 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:20.214+0000 7fc33e000640 1 -- 172.21.15.117:0/655277513 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc338016030 con 0x7fc3401076d0 2024-06-03T03:25:20.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:25:20 smithi151 ceph-mon[37043]: pgmap v506: 129 pgs: 129 active+clean; 657 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-06-03T03:25:20.412 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:20.411+0000 7fc348600640 1 -- 172.21.15.117:0/655277513 --> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc340061de0 con 0x7fc3280a2c20 2024-06-03T03:25:20.441 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:20.440+0000 7fc33e000640 1 -- 172.21.15.117:0/655277513 <== mgr.14152 v2:172.21.15.117:6800/1398160887 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340831 (secure 0 0 0) 0x7fc340061de0 con 0x7fc3280a2c20 2024-06-03T03:25:20.446 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T03:25:20.448 INFO:teuthology.orchestra.run.smithi117.stderr:dumped all 2024-06-03T03:25:20.449 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:25:20 smithi117 ceph-mon[31649]: pgmap v506: 129 pgs: 129 active+clean; 657 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-06-03T03:25:20.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:20.448+0000 7fc348600640 1 -- 172.21.15.117:0/655277513 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fc3280a2c20 msgr2=0x7fc3280a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:20.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:20.448+0000 7fc348600640 1 --2- 172.21.15.117:0/655277513 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fc3280a2c20 0x7fc3280a50e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc3340314e0 tx=0x7fc334000bc0 comp rx=0 tx=0).stop 2024-06-03T03:25:20.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:20.448+0000 7fc348600640 1 -- 172.21.15.117:0/655277513 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3401076d0 msgr2=0x7fc340100960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:20.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:20.448+0000 7fc348600640 1 --2- 172.21.15.117:0/655277513 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3401076d0 0x7fc340100960 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc33800eaa0 tx=0x7fc33800ef60 comp rx=0 tx=0).stop 2024-06-03T03:25:20.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:20.448+0000 7fc348600640 1 -- 172.21.15.117:0/655277513 shutdown_connections 2024-06-03T03:25:20.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:20.448+0000 7fc348600640 1 --2- 172.21.15.117:0/655277513 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3401076d0 0x7fc340100960 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:20.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:20.448+0000 7fc348600640 1 --2- 172.21.15.117:0/655277513 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc340104da0 0x7fc340100420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:20.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:20.448+0000 7fc348600640 1 --2- 172.21.15.117:0/655277513 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fc3280a2c20 0x7fc3280a50e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:20.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:20.448+0000 7fc348600640 1 --2- 172.21.15.117:0/655277513 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc340102470 0x7fc340101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:20.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:20.448+0000 7fc348600640 1 -- 172.21.15.117:0/655277513 >> 172.21.15.117:0/655277513 conn(0x7fc3400fbe80 msgr2=0x7fc340102dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:25:20.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:20.448+0000 7fc348600640 1 -- 172.21.15.117:0/655277513 shutdown_connections 2024-06-03T03:25:20.452 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:20.449+0000 7fc348600640 1 -- 172.21.15.117:0/655277513 wait complete. 2024-06-03T03:25:20.472 INFO:teuthology.orchestra.run.smithi117.stdout:{"pg_ready":true,"pg_map":{"version":506,"stamp":"2024-06-03T03:25:18.957567+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":673245,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":2664,"num_read_kb":1470,"num_write":466,"num_write_kb":3114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":880,"ondisk_log_size":880,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":370944,"kb_used_data":13472,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124570880,"statfs":{"total":1151940427776,"available":1151560581120,"internally_reserved":0,"allocated":13795328,"data_stored":8832845,"data_compressed":501066,"data_compressed_allocated":688128,"data_compressed_original":2875392,"omap_allocated":19142,"internal_metadata":365933882},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6486,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":146,"num_read_kb":80,"num_write":19,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005661"},"pg_stats":[{"pgid":"3.3f","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.832850+0000","last_change":"2024-06-03T03:19:41.626030+0000","last_active":"2024-06-03T03:22:54.832850+0000","last_peered":"2024-06-03T03:22:54.832850+0000","last_clean":"2024-06-03T03:22:54.832850+0000","last_became_active":"2024-06-03T03:19:41.625681+0000","last_became_peered":"2024-06-03T03:19:41.625681+0000","last_unstale":"2024-06-03T03:22:54.832850+0000","last_undegraded":"2024-06-03T03:22:54.832850+0000","last_fullsized":"2024-06-03T03:22:54.832850+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:35:46.396360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"99'4","reported_seq":43,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.831429+0000","last_change":"2024-06-03T03:19:41.619668+0000","last_active":"2024-06-03T03:22:54.831429+0000","last_peered":"2024-06-03T03:22:54.831429+0000","last_clean":"2024-06-03T03:22:54.831429+0000","last_became_active":"2024-06-03T03:19:41.619318+0000","last_became_peered":"2024-06-03T03:19:41.619318+0000","last_unstale":"2024-06-03T03:22:54.831429+0000","last_undegraded":"2024-06-03T03:22:54.831429+0000","last_fullsized":"2024-06-03T03:22:54.831429+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:07:21.656045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.832725+0000","last_change":"2024-06-03T03:19:41.623689+0000","last_active":"2024-06-03T03:22:54.832725+0000","last_peered":"2024-06-03T03:22:54.832725+0000","last_clean":"2024-06-03T03:22:54.832725+0000","last_became_active":"2024-06-03T03:19:41.623027+0000","last_became_peered":"2024-06-03T03:19:41.623027+0000","last_unstale":"2024-06-03T03:22:54.832725+0000","last_undegraded":"2024-06-03T03:22:54.832725+0000","last_fullsized":"2024-06-03T03:22:54.832725+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:18:05.584397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.815076+0000","last_change":"2024-06-03T03:19:41.603802+0000","last_active":"2024-06-03T03:22:54.815076+0000","last_peered":"2024-06-03T03:22:54.815076+0000","last_clean":"2024-06-03T03:22:54.815076+0000","last_became_active":"2024-06-03T03:19:41.603470+0000","last_became_peered":"2024-06-03T03:19:41.603470+0000","last_unstale":"2024-06-03T03:22:54.815076+0000","last_undegraded":"2024-06-03T03:22:54.815076+0000","last_fullsized":"2024-06-03T03:22:54.815076+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:56:10.636108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"99'4","reported_seq":43,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.826310+0000","last_change":"2024-06-03T03:19:41.636436+0000","last_active":"2024-06-03T03:22:54.826310+0000","last_peered":"2024-06-03T03:22:54.826310+0000","last_clean":"2024-06-03T03:22:54.826310+0000","last_became_active":"2024-06-03T03:19:41.636150+0000","last_became_peered":"2024-06-03T03:19:41.636150+0000","last_unstale":"2024-06-03T03:22:54.826310+0000","last_undegraded":"2024-06-03T03:22:54.826310+0000","last_fullsized":"2024-06-03T03:22:54.826310+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:02:10.928251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"99'6","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.809082+0000","last_change":"2024-06-03T03:19:41.622426+0000","last_active":"2024-06-03T03:22:54.809082+0000","last_peered":"2024-06-03T03:22:54.809082+0000","last_clean":"2024-06-03T03:22:54.809082+0000","last_became_active":"2024-06-03T03:19:41.622092+0000","last_became_peered":"2024-06-03T03:19:41.622092+0000","last_unstale":"2024-06-03T03:22:54.809082+0000","last_undegraded":"2024-06-03T03:22:54.809082+0000","last_fullsized":"2024-06-03T03:22:54.809082+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T14:08:52.666771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"99'5","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.825682+0000","last_change":"2024-06-03T03:19:41.608303+0000","last_active":"2024-06-03T03:22:54.825682+0000","last_peered":"2024-06-03T03:22:54.825682+0000","last_clean":"2024-06-03T03:22:54.825682+0000","last_became_active":"2024-06-03T03:19:41.607929+0000","last_became_peered":"2024-06-03T03:19:41.607929+0000","last_unstale":"2024-06-03T03:22:54.825682+0000","last_undegraded":"2024-06-03T03:22:54.825682+0000","last_fullsized":"2024-06-03T03:22:54.825682+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:40:05.643531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.822841+0000","last_change":"2024-06-03T03:19:41.606722+0000","last_active":"2024-06-03T03:22:54.822841+0000","last_peered":"2024-06-03T03:22:54.822841+0000","last_clean":"2024-06-03T03:22:54.822841+0000","last_became_active":"2024-06-03T03:19:41.606131+0000","last_became_peered":"2024-06-03T03:19:41.606131+0000","last_unstale":"2024-06-03T03:22:54.822841+0000","last_undegraded":"2024-06-03T03:22:54.822841+0000","last_fullsized":"2024-06-03T03:22:54.822841+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:09:17.306848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"99'11","reported_seq":51,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.813800+0000","last_change":"2024-06-03T03:19:41.609287+0000","last_active":"2024-06-03T03:22:54.813800+0000","last_peered":"2024-06-03T03:22:54.813800+0000","last_clean":"2024-06-03T03:22:54.813800+0000","last_became_active":"2024-06-03T03:19:41.608521+0000","last_became_peered":"2024-06-03T03:19:41.608521+0000","last_unstale":"2024-06-03T03:22:54.813800+0000","last_undegraded":"2024-06-03T03:22:54.813800+0000","last_fullsized":"2024-06-03T03:22:54.813800+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:35:37.128667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"99'16","reported_seq":56,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.814705+0000","last_change":"2024-06-03T03:19:41.605193+0000","last_active":"2024-06-03T03:22:54.814705+0000","last_peered":"2024-06-03T03:22:54.814705+0000","last_clean":"2024-06-03T03:22:54.814705+0000","last_became_active":"2024-06-03T03:19:41.604787+0000","last_became_peered":"2024-06-03T03:19:41.604787+0000","last_unstale":"2024-06-03T03:22:54.814705+0000","last_undegraded":"2024-06-03T03:22:54.814705+0000","last_fullsized":"2024-06-03T03:22:54.814705+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:02:48.334599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"99'6","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.817666+0000","last_change":"2024-06-03T03:19:41.611522+0000","last_active":"2024-06-03T03:22:54.817666+0000","last_peered":"2024-06-03T03:22:54.817666+0000","last_clean":"2024-06-03T03:22:54.817666+0000","last_became_active":"2024-06-03T03:19:41.610932+0000","last_became_peered":"2024-06-03T03:19:41.610932+0000","last_unstale":"2024-06-03T03:22:54.817666+0000","last_undegraded":"2024-06-03T03:22:54.817666+0000","last_fullsized":"2024-06-03T03:22:54.817666+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:35:44.702791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"99'7","reported_seq":24,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.820810+0000","last_change":"2024-06-03T03:20:25.950663+0000","last_active":"2024-06-03T03:22:54.820810+0000","last_peered":"2024-06-03T03:22:54.820810+0000","last_clean":"2024-06-03T03:22:54.820810+0000","last_became_active":"2024-06-03T03:20:25.949962+0000","last_became_peered":"2024-06-03T03:20:25.949962+0000","last_unstale":"2024-06-03T03:22:54.820810+0000","last_undegraded":"2024-06-03T03:22:54.820810+0000","last_fullsized":"2024-06-03T03:22:54.820810+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:55:33.972567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"99'10","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.839179+0000","last_change":"2024-06-03T03:19:41.633464+0000","last_active":"2024-06-03T03:22:54.839179+0000","last_peered":"2024-06-03T03:22:54.839179+0000","last_clean":"2024-06-03T03:22:54.839179+0000","last_became_active":"2024-06-03T03:19:41.630625+0000","last_became_peered":"2024-06-03T03:19:41.630625+0000","last_unstale":"2024-06-03T03:22:54.839179+0000","last_undegraded":"2024-06-03T03:22:54.839179+0000","last_fullsized":"2024-06-03T03:22:54.839179+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:23:28.799058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.831893+0000","last_change":"2024-06-03T03:19:41.627234+0000","last_active":"2024-06-03T03:22:54.831893+0000","last_peered":"2024-06-03T03:22:54.831893+0000","last_clean":"2024-06-03T03:22:54.831893+0000","last_became_active":"2024-06-03T03:19:41.626643+0000","last_became_peered":"2024-06-03T03:19:41.626643+0000","last_unstale":"2024-06-03T03:22:54.831893+0000","last_undegraded":"2024-06-03T03:22:54.831893+0000","last_fullsized":"2024-06-03T03:22:54.831893+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:37:44.126101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.831893+0000","last_change":"2024-06-03T03:19:41.621833+0000","last_active":"2024-06-03T03:22:54.831893+0000","last_peered":"2024-06-03T03:22:54.831893+0000","last_clean":"2024-06-03T03:22:54.831893+0000","last_became_active":"2024-06-03T03:19:41.620689+0000","last_became_peered":"2024-06-03T03:19:41.620689+0000","last_unstale":"2024-06-03T03:22:54.831893+0000","last_undegraded":"2024-06-03T03:22:54.831893+0000","last_fullsized":"2024-06-03T03:22:54.831893+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:39:31.188372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"99'9","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.830743+0000","last_change":"2024-06-03T03:19:41.624072+0000","last_active":"2024-06-03T03:22:54.830743+0000","last_peered":"2024-06-03T03:22:54.830743+0000","last_clean":"2024-06-03T03:22:54.830743+0000","last_became_active":"2024-06-03T03:19:41.623689+0000","last_became_peered":"2024-06-03T03:19:41.623689+0000","last_unstale":"2024-06-03T03:22:54.830743+0000","last_undegraded":"2024-06-03T03:22:54.830743+0000","last_fullsized":"2024-06-03T03:22:54.830743+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:38:24.075256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.833104+0000","last_change":"2024-06-03T03:19:41.626145+0000","last_active":"2024-06-03T03:22:54.833104+0000","last_peered":"2024-06-03T03:22:54.833104+0000","last_clean":"2024-06-03T03:22:54.833104+0000","last_became_active":"2024-06-03T03:19:41.625826+0000","last_became_peered":"2024-06-03T03:19:41.625826+0000","last_unstale":"2024-06-03T03:22:54.833104+0000","last_undegraded":"2024-06-03T03:22:54.833104+0000","last_fullsized":"2024-06-03T03:22:54.833104+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:04:10.996134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"99'10","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.821280+0000","last_change":"2024-06-03T03:19:41.615754+0000","last_active":"2024-06-03T03:22:54.821280+0000","last_peered":"2024-06-03T03:22:54.821280+0000","last_clean":"2024-06-03T03:22:54.821280+0000","last_became_active":"2024-06-03T03:19:41.614577+0000","last_became_peered":"2024-06-03T03:19:41.614577+0000","last_unstale":"2024-06-03T03:22:54.821280+0000","last_undegraded":"2024-06-03T03:22:54.821280+0000","last_fullsized":"2024-06-03T03:22:54.821280+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:45:50.084332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.826381+0000","last_change":"2024-06-03T03:19:41.633000+0000","last_active":"2024-06-03T03:22:54.826381+0000","last_peered":"2024-06-03T03:22:54.826381+0000","last_clean":"2024-06-03T03:22:54.826381+0000","last_became_active":"2024-06-03T03:19:41.632731+0000","last_became_peered":"2024-06-03T03:19:41.632731+0000","last_unstale":"2024-06-03T03:22:54.826381+0000","last_undegraded":"2024-06-03T03:22:54.826381+0000","last_fullsized":"2024-06-03T03:22:54.826381+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:10:29.990256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:24.128512+0000","last_change":"2024-06-03T03:20:26.133763+0000","last_active":"2024-06-03T03:22:24.128512+0000","last_peered":"2024-06-03T03:22:24.128512+0000","last_clean":"2024-06-03T03:22:24.128512+0000","last_became_active":"2024-06-03T03:20:26.133252+0000","last_became_peered":"2024-06-03T03:20:26.133252+0000","last_unstale":"2024-06-03T03:22:24.128512+0000","last_undegraded":"2024-06-03T03:22:24.128512+0000","last_fullsized":"2024-06-03T03:22:24.128512+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:33:21.461038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"99'14","reported_seq":309,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:25:14.520260+0000","last_change":"2024-06-03T03:19:36.542545+0000","last_active":"2024-06-03T03:25:14.520260+0000","last_peered":"2024-06-03T03:25:14.520260+0000","last_clean":"2024-06-03T03:25:14.520260+0000","last_became_active":"2024-06-03T03:19:36.541682+0000","last_became_peered":"2024-06-03T03:19:36.541682+0000","last_unstale":"2024-06-03T03:25:14.520260+0000","last_undegraded":"2024-06-03T03:25:14.520260+0000","last_fullsized":"2024-06-03T03:25:14.520260+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:06:23.909253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":249,"num_read_kb":249,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"99'5","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.819154+0000","last_change":"2024-06-03T03:19:41.605848+0000","last_active":"2024-06-03T03:22:54.819154+0000","last_peered":"2024-06-03T03:22:54.819154+0000","last_clean":"2024-06-03T03:22:54.819154+0000","last_became_active":"2024-06-03T03:19:41.603131+0000","last_became_peered":"2024-06-03T03:19:41.603131+0000","last_unstale":"2024-06-03T03:22:54.819154+0000","last_undegraded":"2024-06-03T03:22:54.819154+0000","last_fullsized":"2024-06-03T03:22:54.819154+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:07:38.931055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:22.310371+0000","last_change":"2024-06-03T03:19:56.923180+0000","last_active":"2024-06-03T03:22:22.310371+0000","last_peered":"2024-06-03T03:22:22.310371+0000","last_clean":"2024-06-03T03:22:22.310371+0000","last_became_active":"2024-06-03T03:19:56.922726+0000","last_became_peered":"2024-06-03T03:19:56.922726+0000","last_unstale":"2024-06-03T03:22:22.310371+0000","last_undegraded":"2024-06-03T03:22:22.310371+0000","last_fullsized":"2024-06-03T03:22:22.310371+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:18:45.727680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"97'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:36.766695+0000","last_change":"2024-06-03T03:19:36.746805+0000","last_active":"2024-06-03T03:21:36.766695+0000","last_peered":"2024-06-03T03:21:36.766695+0000","last_clean":"2024-06-03T03:21:36.766695+0000","last_became_active":"2024-06-03T03:19:36.745747+0000","last_became_peered":"2024-06-03T03:19:36.745747+0000","last_unstale":"2024-06-03T03:21:36.766695+0000","last_undegraded":"2024-06-03T03:21:36.766695+0000","last_fullsized":"2024-06-03T03:21:36.766695+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:54:28.427708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.832890+0000","last_change":"2024-06-03T03:19:41.630582+0000","last_active":"2024-06-03T03:22:54.832890+0000","last_peered":"2024-06-03T03:22:54.832890+0000","last_clean":"2024-06-03T03:22:54.832890+0000","last_became_active":"2024-06-03T03:19:41.630238+0000","last_became_peered":"2024-06-03T03:19:41.630238+0000","last_unstale":"2024-06-03T03:22:54.832890+0000","last_undegraded":"2024-06-03T03:22:54.832890+0000","last_fullsized":"2024-06-03T03:22:54.832890+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:49:58.039697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:27.311089+0000","last_change":"2024-06-03T03:19:56.925142+0000","last_active":"2024-06-03T03:22:27.311089+0000","last_peered":"2024-06-03T03:22:27.311089+0000","last_clean":"2024-06-03T03:22:27.311089+0000","last_became_active":"2024-06-03T03:19:56.924760+0000","last_became_peered":"2024-06-03T03:19:56.924760+0000","last_unstale":"2024-06-03T03:22:27.311089+0000","last_undegraded":"2024-06-03T03:22:27.311089+0000","last_fullsized":"2024-06-03T03:22:27.311089+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:11:27.537077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"97'12","reported_seq":58,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:09.897381+0000","last_change":"2024-06-03T03:19:36.543344+0000","last_active":"2024-06-03T03:22:09.897381+0000","last_peered":"2024-06-03T03:22:09.897381+0000","last_clean":"2024-06-03T03:22:09.897381+0000","last_became_active":"2024-06-03T03:19:36.542837+0000","last_became_peered":"2024-06-03T03:19:36.542837+0000","last_unstale":"2024-06-03T03:22:09.897381+0000","last_undegraded":"2024-06-03T03:22:09.897381+0000","last_fullsized":"2024-06-03T03:22:09.897381+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:55:05.451521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"99'9","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.813173+0000","last_change":"2024-06-03T03:19:41.629820+0000","last_active":"2024-06-03T03:22:54.813173+0000","last_peered":"2024-06-03T03:22:54.813173+0000","last_clean":"2024-06-03T03:22:54.813173+0000","last_became_active":"2024-06-03T03:19:41.629439+0000","last_became_peered":"2024-06-03T03:19:41.629439+0000","last_unstale":"2024-06-03T03:22:54.813173+0000","last_undegraded":"2024-06-03T03:22:54.813173+0000","last_fullsized":"2024-06-03T03:22:54.813173+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:05:27.438308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:27.311072+0000","last_change":"2024-06-03T03:19:56.923214+0000","last_active":"2024-06-03T03:22:27.311072+0000","last_peered":"2024-06-03T03:22:27.311072+0000","last_clean":"2024-06-03T03:22:27.311072+0000","last_became_active":"2024-06-03T03:19:56.922751+0000","last_became_peered":"2024-06-03T03:19:56.922751+0000","last_unstale":"2024-06-03T03:22:27.311072+0000","last_undegraded":"2024-06-03T03:22:27.311072+0000","last_fullsized":"2024-06-03T03:22:27.311072+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:05:59.249818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"97'5","reported_seq":51,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:09.897329+0000","last_change":"2024-06-03T03:19:36.538141+0000","last_active":"2024-06-03T03:22:09.897329+0000","last_peered":"2024-06-03T03:22:09.897329+0000","last_clean":"2024-06-03T03:22:09.897329+0000","last_became_active":"2024-06-03T03:19:36.537786+0000","last_became_peered":"2024-06-03T03:19:36.537786+0000","last_unstale":"2024-06-03T03:22:09.897329+0000","last_undegraded":"2024-06-03T03:22:09.897329+0000","last_fullsized":"2024-06-03T03:22:09.897329+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T15:03:32.859097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"99'15","reported_seq":55,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.816490+0000","last_change":"2024-06-03T03:19:41.608471+0000","last_active":"2024-06-03T03:22:54.816490+0000","last_peered":"2024-06-03T03:22:54.816490+0000","last_clean":"2024-06-03T03:22:54.816490+0000","last_became_active":"2024-06-03T03:19:41.608096+0000","last_became_peered":"2024-06-03T03:19:41.608096+0000","last_unstale":"2024-06-03T03:22:54.816490+0000","last_undegraded":"2024-06-03T03:22:54.816490+0000","last_fullsized":"2024-06-03T03:22:54.816490+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:31:10.541803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:29.129303+0000","last_change":"2024-06-03T03:19:56.915540+0000","last_active":"2024-06-03T03:22:29.129303+0000","last_peered":"2024-06-03T03:22:29.129303+0000","last_clean":"2024-06-03T03:22:29.129303+0000","last_became_active":"2024-06-03T03:19:56.915017+0000","last_became_peered":"2024-06-03T03:19:56.915017+0000","last_unstale":"2024-06-03T03:22:29.129303+0000","last_undegraded":"2024-06-03T03:22:29.129303+0000","last_fullsized":"2024-06-03T03:22:29.129303+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:56:23.158344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"99'9","reported_seq":310,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:25:15.590058+0000","last_change":"2024-06-03T03:19:36.542337+0000","last_active":"2024-06-03T03:25:15.590058+0000","last_peered":"2024-06-03T03:25:15.590058+0000","last_clean":"2024-06-03T03:25:15.590058+0000","last_became_active":"2024-06-03T03:19:36.541990+0000","last_became_peered":"2024-06-03T03:19:36.541990+0000","last_unstale":"2024-06-03T03:25:15.590058+0000","last_undegraded":"2024-06-03T03:25:15.590058+0000","last_fullsized":"2024-06-03T03:25:15.590058+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:08:15.656321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1849,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":255,"num_read_kb":6,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.815481+0000","last_change":"2024-06-03T03:19:41.616293+0000","last_active":"2024-06-03T03:22:54.815481+0000","last_peered":"2024-06-03T03:22:54.815481+0000","last_clean":"2024-06-03T03:22:54.815481+0000","last_became_active":"2024-06-03T03:19:41.615946+0000","last_became_peered":"2024-06-03T03:19:41.615946+0000","last_unstale":"2024-06-03T03:22:54.815481+0000","last_undegraded":"2024-06-03T03:22:54.815481+0000","last_fullsized":"2024-06-03T03:22:54.815481+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:18:43.835977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:29.129278+0000","last_change":"2024-06-03T03:20:26.146237+0000","last_active":"2024-06-03T03:22:29.129278+0000","last_peered":"2024-06-03T03:22:29.129278+0000","last_clean":"2024-06-03T03:22:29.129278+0000","last_became_active":"2024-06-03T03:20:26.145731+0000","last_became_peered":"2024-06-03T03:20:26.145731+0000","last_unstale":"2024-06-03T03:22:29.129278+0000","last_undegraded":"2024-06-03T03:22:29.129278+0000","last_fullsized":"2024-06-03T03:22:29.129278+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:15:29.723833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"97'7","reported_seq":53,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:07.058194+0000","last_change":"2024-06-03T03:19:36.546501+0000","last_active":"2024-06-03T03:22:07.058194+0000","last_peered":"2024-06-03T03:22:07.058194+0000","last_clean":"2024-06-03T03:22:07.058194+0000","last_became_active":"2024-06-03T03:19:36.546079+0000","last_became_peered":"2024-06-03T03:19:36.546079+0000","last_unstale":"2024-06-03T03:22:07.058194+0000","last_undegraded":"2024-06-03T03:22:07.058194+0000","last_fullsized":"2024-06-03T03:22:07.058194+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:16:39.473141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"99'8","reported_seq":54,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.829180+0000","last_change":"2024-06-03T03:20:26.149410+0000","last_active":"2024-06-03T03:22:54.829180+0000","last_peered":"2024-06-03T03:22:54.829180+0000","last_clean":"2024-06-03T03:22:54.829180+0000","last_became_active":"2024-06-03T03:20:26.149084+0000","last_became_peered":"2024-06-03T03:20:26.149084+0000","last_unstale":"2024-06-03T03:22:54.829180+0000","last_undegraded":"2024-06-03T03:22:54.829180+0000","last_fullsized":"2024-06-03T03:22:54.829180+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:06:22.316449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:04.314994+0000","last_change":"2024-06-03T03:19:56.926498+0000","last_active":"2024-06-03T03:22:04.314994+0000","last_peered":"2024-06-03T03:22:04.314994+0000","last_clean":"2024-06-03T03:22:04.314994+0000","last_became_active":"2024-06-03T03:19:56.926101+0000","last_became_peered":"2024-06-03T03:19:56.926101+0000","last_unstale":"2024-06-03T03:22:04.314994+0000","last_undegraded":"2024-06-03T03:22:04.314994+0000","last_fullsized":"2024-06-03T03:22:04.314994+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:00:12.024443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"97'3","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:03.279571+0000","last_change":"2024-06-03T03:19:36.553151+0000","last_active":"2024-06-03T03:22:03.279571+0000","last_peered":"2024-06-03T03:22:03.279571+0000","last_clean":"2024-06-03T03:22:03.279571+0000","last_became_active":"2024-06-03T03:19:36.551914+0000","last_became_peered":"2024-06-03T03:19:36.551914+0000","last_unstale":"2024-06-03T03:22:03.279571+0000","last_undegraded":"2024-06-03T03:22:03.279571+0000","last_fullsized":"2024-06-03T03:22:03.279571+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:00:42.053922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"99'8","reported_seq":25,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.824386+0000","last_change":"2024-06-03T03:20:26.147909+0000","last_active":"2024-06-03T03:22:54.824386+0000","last_peered":"2024-06-03T03:22:54.824386+0000","last_clean":"2024-06-03T03:22:54.824386+0000","last_became_active":"2024-06-03T03:20:26.146448+0000","last_became_peered":"2024-06-03T03:20:26.146448+0000","last_unstale":"2024-06-03T03:22:54.824386+0000","last_undegraded":"2024-06-03T03:22:54.824386+0000","last_fullsized":"2024-06-03T03:22:54.824386+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:43:30.878472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:59.843167+0000","last_change":"2024-06-03T03:19:56.941910+0000","last_active":"2024-06-03T03:21:59.843167+0000","last_peered":"2024-06-03T03:21:59.843167+0000","last_clean":"2024-06-03T03:21:59.843167+0000","last_became_active":"2024-06-03T03:19:56.940963+0000","last_became_peered":"2024-06-03T03:19:56.940963+0000","last_unstale":"2024-06-03T03:21:59.843167+0000","last_undegraded":"2024-06-03T03:21:59.843167+0000","last_fullsized":"2024-06-03T03:21:59.843167+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T14:18:32.109231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"97'3","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:02.447068+0000","last_change":"2024-06-03T03:19:36.538928+0000","last_active":"2024-06-03T03:22:02.447068+0000","last_peered":"2024-06-03T03:22:02.447068+0000","last_clean":"2024-06-03T03:22:02.447068+0000","last_became_active":"2024-06-03T03:19:36.538580+0000","last_became_peered":"2024-06-03T03:19:36.538580+0000","last_unstale":"2024-06-03T03:22:02.447068+0000","last_undegraded":"2024-06-03T03:22:02.447068+0000","last_fullsized":"2024-06-03T03:22:02.447068+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:08:05.906257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"99'11","reported_seq":51,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.829295+0000","last_change":"2024-06-03T03:19:41.623051+0000","last_active":"2024-06-03T03:22:54.829295+0000","last_peered":"2024-06-03T03:22:54.829295+0000","last_clean":"2024-06-03T03:22:54.829295+0000","last_became_active":"2024-06-03T03:19:41.622656+0000","last_became_peered":"2024-06-03T03:19:41.622656+0000","last_unstale":"2024-06-03T03:22:54.829295+0000","last_undegraded":"2024-06-03T03:22:54.829295+0000","last_fullsized":"2024-06-03T03:22:54.829295+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:11:34.587723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:16.773697+0000","last_change":"2024-06-03T03:19:56.921215+0000","last_active":"2024-06-03T03:22:16.773697+0000","last_peered":"2024-06-03T03:22:16.773697+0000","last_clean":"2024-06-03T03:22:16.773697+0000","last_became_active":"2024-06-03T03:19:56.920658+0000","last_became_peered":"2024-06-03T03:19:56.920658+0000","last_unstale":"2024-06-03T03:22:16.773697+0000","last_undegraded":"2024-06-03T03:22:16.773697+0000","last_fullsized":"2024-06-03T03:22:16.773697+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T14:04:54.163818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"99'7","reported_seq":551,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:25:16.411576+0000","last_change":"2024-06-03T03:19:36.545245+0000","last_active":"2024-06-03T03:25:16.411576+0000","last_peered":"2024-06-03T03:25:16.411576+0000","last_clean":"2024-06-03T03:25:16.411576+0000","last_became_active":"2024-06-03T03:19:36.544821+0000","last_became_peered":"2024-06-03T03:19:36.544821+0000","last_unstale":"2024-06-03T03:25:16.411576+0000","last_undegraded":"2024-06-03T03:25:16.411576+0000","last_fullsized":"2024-06-03T03:25:16.411576+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:55:24.011925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":498,"num_read_kb":498,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"99'10","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.822488+0000","last_change":"2024-06-03T03:19:41.616384+0000","last_active":"2024-06-03T03:22:54.822488+0000","last_peered":"2024-06-03T03:22:54.822488+0000","last_clean":"2024-06-03T03:22:54.822488+0000","last_became_active":"2024-06-03T03:19:41.616034+0000","last_became_peered":"2024-06-03T03:19:41.616034+0000","last_unstale":"2024-06-03T03:22:54.822488+0000","last_undegraded":"2024-06-03T03:22:54.822488+0000","last_fullsized":"2024-06-03T03:22:54.822488+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:10:06.823375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:20.886218+0000","last_change":"2024-06-03T03:19:56.933045+0000","last_active":"2024-06-03T03:22:20.886218+0000","last_peered":"2024-06-03T03:22:20.886218+0000","last_clean":"2024-06-03T03:22:20.886218+0000","last_became_active":"2024-06-03T03:19:56.928956+0000","last_became_peered":"2024-06-03T03:19:56.928956+0000","last_unstale":"2024-06-03T03:22:20.886218+0000","last_undegraded":"2024-06-03T03:22:20.886218+0000","last_fullsized":"2024-06-03T03:22:20.886218+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:37:44.270511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"97'4","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:04.852741+0000","last_change":"2024-06-03T03:19:36.541559+0000","last_active":"2024-06-03T03:22:04.852741+0000","last_peered":"2024-06-03T03:22:04.852741+0000","last_clean":"2024-06-03T03:22:04.852741+0000","last_became_active":"2024-06-03T03:19:36.541166+0000","last_became_peered":"2024-06-03T03:19:36.541166+0000","last_unstale":"2024-06-03T03:22:04.852741+0000","last_undegraded":"2024-06-03T03:22:04.852741+0000","last_fullsized":"2024-06-03T03:22:04.852741+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T14:49:26.847713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.833224+0000","last_change":"2024-06-03T03:19:41.629789+0000","last_active":"2024-06-03T03:22:54.833224+0000","last_peered":"2024-06-03T03:22:54.833224+0000","last_clean":"2024-06-03T03:22:54.833224+0000","last_became_active":"2024-06-03T03:19:41.628026+0000","last_became_peered":"2024-06-03T03:19:41.628026+0000","last_unstale":"2024-06-03T03:22:54.833224+0000","last_undegraded":"2024-06-03T03:22:54.833224+0000","last_fullsized":"2024-06-03T03:22:54.833224+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:56:26.411907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:17.309900+0000","last_change":"2024-06-03T03:21:26.056114+0000","last_active":"2024-06-03T03:22:17.309900+0000","last_peered":"2024-06-03T03:22:17.309900+0000","last_clean":"2024-06-03T03:22:17.309900+0000","last_became_active":"2024-06-03T03:21:26.055505+0000","last_became_peered":"2024-06-03T03:21:26.055505+0000","last_unstale":"2024-06-03T03:22:17.309900+0000","last_undegraded":"2024-06-03T03:22:17.309900+0000","last_fullsized":"2024-06-03T03:22:17.309900+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:22:55.722566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,10],"acting":[4,1,2,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"97'5","reported_seq":52,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:02.447113+0000","last_change":"2024-06-03T03:19:36.744930+0000","last_active":"2024-06-03T03:22:02.447113+0000","last_peered":"2024-06-03T03:22:02.447113+0000","last_clean":"2024-06-03T03:22:02.447113+0000","last_became_active":"2024-06-03T03:19:36.744567+0000","last_became_peered":"2024-06-03T03:19:36.744567+0000","last_unstale":"2024-06-03T03:22:02.447113+0000","last_undegraded":"2024-06-03T03:22:02.447113+0000","last_fullsized":"2024-06-03T03:22:02.447113+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:35:31.927211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"99'11","reported_seq":51,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.812800+0000","last_change":"2024-06-03T03:19:41.607604+0000","last_active":"2024-06-03T03:22:54.812800+0000","last_peered":"2024-06-03T03:22:54.812800+0000","last_clean":"2024-06-03T03:22:54.812800+0000","last_became_active":"2024-06-03T03:19:41.607148+0000","last_became_peered":"2024-06-03T03:19:41.607148+0000","last_unstale":"2024-06-03T03:22:54.812800+0000","last_undegraded":"2024-06-03T03:22:54.812800+0000","last_fullsized":"2024-06-03T03:22:54.812800+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:44:44.037836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:20.886272+0000","last_change":"2024-06-03T03:19:56.928689+0000","last_active":"2024-06-03T03:22:20.886272+0000","last_peered":"2024-06-03T03:22:20.886272+0000","last_clean":"2024-06-03T03:22:20.886272+0000","last_became_active":"2024-06-03T03:19:56.924955+0000","last_became_peered":"2024-06-03T03:19:56.924955+0000","last_unstale":"2024-06-03T03:22:20.886272+0000","last_undegraded":"2024-06-03T03:22:20.886272+0000","last_fullsized":"2024-06-03T03:22:20.886272+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:11:07.413430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"99'46","reported_seq":604,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:25:16.411088+0000","last_change":"2024-06-03T03:19:36.539688+0000","last_active":"2024-06-03T03:25:16.411088+0000","last_peered":"2024-06-03T03:25:16.411088+0000","last_clean":"2024-06-03T03:25:16.411088+0000","last_became_active":"2024-06-03T03:19:36.539302+0000","last_became_peered":"2024-06-03T03:19:36.539302+0000","last_unstale":"2024-06-03T03:25:16.411088+0000","last_undegraded":"2024-06-03T03:25:16.411088+0000","last_fullsized":"2024-06-03T03:25:16.411088+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:28:52.790227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":74214,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":512,"num_read_kb":329,"num_write":48,"num_write_kb":95,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.815494+0000","last_change":"2024-06-03T03:19:41.608202+0000","last_active":"2024-06-03T03:22:54.815494+0000","last_peered":"2024-06-03T03:22:54.815494+0000","last_clean":"2024-06-03T03:22:54.815494+0000","last_became_active":"2024-06-03T03:19:41.607891+0000","last_became_peered":"2024-06-03T03:19:41.607891+0000","last_unstale":"2024-06-03T03:22:54.815494+0000","last_undegraded":"2024-06-03T03:22:54.815494+0000","last_fullsized":"2024-06-03T03:22:54.815494+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T14:10:44.042892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:22.449231+0000","last_change":"2024-06-03T03:19:56.923547+0000","last_active":"2024-06-03T03:22:22.449231+0000","last_peered":"2024-06-03T03:22:22.449231+0000","last_clean":"2024-06-03T03:22:22.449231+0000","last_became_active":"2024-06-03T03:19:56.923046+0000","last_became_peered":"2024-06-03T03:19:56.923046+0000","last_unstale":"2024-06-03T03:22:22.449231+0000","last_undegraded":"2024-06-03T03:22:22.449231+0000","last_fullsized":"2024-06-03T03:22:22.449231+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:32:19.438793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"97'9","reported_seq":56,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:59.125620+0000","last_change":"2024-06-03T03:19:36.547024+0000","last_active":"2024-06-03T03:21:59.125620+0000","last_peered":"2024-06-03T03:21:59.125620+0000","last_clean":"2024-06-03T03:21:59.125620+0000","last_became_active":"2024-06-03T03:19:36.546663+0000","last_became_peered":"2024-06-03T03:19:36.546663+0000","last_unstale":"2024-06-03T03:21:59.125620+0000","last_undegraded":"2024-06-03T03:21:59.125620+0000","last_fullsized":"2024-06-03T03:21:59.125620+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:58:41.446730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.830570+0000","last_change":"2024-06-03T03:19:41.630119+0000","last_active":"2024-06-03T03:22:54.830570+0000","last_peered":"2024-06-03T03:22:54.830570+0000","last_clean":"2024-06-03T03:22:54.830570+0000","last_became_active":"2024-06-03T03:19:41.629563+0000","last_became_peered":"2024-06-03T03:19:41.629563+0000","last_unstale":"2024-06-03T03:22:54.830570+0000","last_undegraded":"2024-06-03T03:22:54.830570+0000","last_fullsized":"2024-06-03T03:22:54.830570+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:49:33.983471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:19.316793+0000","last_change":"2024-06-03T03:19:56.941451+0000","last_active":"2024-06-03T03:22:19.316793+0000","last_peered":"2024-06-03T03:22:19.316793+0000","last_clean":"2024-06-03T03:22:19.316793+0000","last_became_active":"2024-06-03T03:19:56.941003+0000","last_became_peered":"2024-06-03T03:19:56.941003+0000","last_unstale":"2024-06-03T03:22:19.316793+0000","last_undegraded":"2024-06-03T03:22:19.316793+0000","last_fullsized":"2024-06-03T03:22:19.316793+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:28:58.505495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"97'5","reported_seq":52,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:57.446530+0000","last_change":"2024-06-03T03:19:36.547128+0000","last_active":"2024-06-03T03:21:57.446530+0000","last_peered":"2024-06-03T03:21:57.446530+0000","last_clean":"2024-06-03T03:21:57.446530+0000","last_became_active":"2024-06-03T03:19:36.546576+0000","last_became_peered":"2024-06-03T03:19:36.546576+0000","last_unstale":"2024-06-03T03:21:57.446530+0000","last_undegraded":"2024-06-03T03:21:57.446530+0000","last_fullsized":"2024-06-03T03:21:57.446530+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:23:30.196543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"99'15","reported_seq":55,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.836544+0000","last_change":"2024-06-03T03:19:41.634941+0000","last_active":"2024-06-03T03:22:54.836544+0000","last_peered":"2024-06-03T03:22:54.836544+0000","last_clean":"2024-06-03T03:22:54.836544+0000","last_became_active":"2024-06-03T03:19:41.634561+0000","last_became_peered":"2024-06-03T03:19:41.634561+0000","last_unstale":"2024-06-03T03:22:54.836544+0000","last_undegraded":"2024-06-03T03:22:54.836544+0000","last_fullsized":"2024-06-03T03:22:54.836544+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:00:27.258078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:25.887154+0000","last_change":"2024-06-03T03:19:56.934612+0000","last_active":"2024-06-03T03:22:25.887154+0000","last_peered":"2024-06-03T03:22:25.887154+0000","last_clean":"2024-06-03T03:22:25.887154+0000","last_became_active":"2024-06-03T03:19:56.930212+0000","last_became_peered":"2024-06-03T03:19:56.930212+0000","last_unstale":"2024-06-03T03:22:25.887154+0000","last_undegraded":"2024-06-03T03:22:25.887154+0000","last_fullsized":"2024-06-03T03:22:25.887154+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:42:38.490095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"97'5","reported_seq":301,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:25:15.522909+0000","last_change":"2024-06-03T03:19:36.542052+0000","last_active":"2024-06-03T03:25:15.522909+0000","last_peered":"2024-06-03T03:25:15.522909+0000","last_clean":"2024-06-03T03:25:15.522909+0000","last_became_active":"2024-06-03T03:19:36.540987+0000","last_became_peered":"2024-06-03T03:19:36.540987+0000","last_unstale":"2024-06-03T03:25:15.522909+0000","last_undegraded":"2024-06-03T03:25:15.522909+0000","last_fullsized":"2024-06-03T03:25:15.522909+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T15:18:48.821400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":252,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"99'6","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.818326+0000","last_change":"2024-06-03T03:19:41.607610+0000","last_active":"2024-06-03T03:22:54.818326+0000","last_peered":"2024-06-03T03:22:54.818326+0000","last_clean":"2024-06-03T03:22:54.818326+0000","last_became_active":"2024-06-03T03:19:41.607219+0000","last_became_peered":"2024-06-03T03:19:41.607219+0000","last_unstale":"2024-06-03T03:22:54.818326+0000","last_undegraded":"2024-06-03T03:22:54.818326+0000","last_fullsized":"2024-06-03T03:22:54.818326+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:06:20.707623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:23.282636+0000","last_change":"2024-06-03T03:19:56.936440+0000","last_active":"2024-06-03T03:22:23.282636+0000","last_peered":"2024-06-03T03:22:23.282636+0000","last_clean":"2024-06-03T03:22:23.282636+0000","last_became_active":"2024-06-03T03:19:56.936048+0000","last_became_peered":"2024-06-03T03:19:56.936048+0000","last_unstale":"2024-06-03T03:22:23.282636+0000","last_undegraded":"2024-06-03T03:22:23.282636+0000","last_fullsized":"2024-06-03T03:22:23.282636+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:22:15.758950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"97'17","reported_seq":63,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:59.896170+0000","last_change":"2024-06-03T03:19:36.736105+0000","last_active":"2024-06-03T03:21:59.896170+0000","last_peered":"2024-06-03T03:21:59.896170+0000","last_clean":"2024-06-03T03:21:59.896170+0000","last_became_active":"2024-06-03T03:19:36.735663+0000","last_became_peered":"2024-06-03T03:19:36.735663+0000","last_unstale":"2024-06-03T03:21:59.896170+0000","last_undegraded":"2024-06-03T03:21:59.896170+0000","last_fullsized":"2024-06-03T03:21:59.896170+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:41:46.216923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"99'6","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.814118+0000","last_change":"2024-06-03T03:19:41.609758+0000","last_active":"2024-06-03T03:22:54.814118+0000","last_peered":"2024-06-03T03:22:54.814118+0000","last_clean":"2024-06-03T03:22:54.814118+0000","last_became_active":"2024-06-03T03:19:41.609421+0000","last_became_peered":"2024-06-03T03:19:41.609421+0000","last_unstale":"2024-06-03T03:22:54.814118+0000","last_undegraded":"2024-06-03T03:22:54.814118+0000","last_fullsized":"2024-06-03T03:22:54.814118+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:45:54.919352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:07.447579+0000","last_change":"2024-06-03T03:19:56.925422+0000","last_active":"2024-06-03T03:22:07.447579+0000","last_peered":"2024-06-03T03:22:07.447579+0000","last_clean":"2024-06-03T03:22:07.447579+0000","last_became_active":"2024-06-03T03:19:56.924891+0000","last_became_peered":"2024-06-03T03:19:56.924891+0000","last_unstale":"2024-06-03T03:22:07.447579+0000","last_undegraded":"2024-06-03T03:22:07.447579+0000","last_fullsized":"2024-06-03T03:22:07.447579+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:34:59.986256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"97'2","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:54.895740+0000","last_change":"2024-06-03T03:19:36.540605+0000","last_active":"2024-06-03T03:21:54.895740+0000","last_peered":"2024-06-03T03:21:54.895740+0000","last_clean":"2024-06-03T03:21:54.895740+0000","last_became_active":"2024-06-03T03:19:36.540180+0000","last_became_peered":"2024-06-03T03:19:36.540180+0000","last_unstale":"2024-06-03T03:21:54.895740+0000","last_undegraded":"2024-06-03T03:21:54.895740+0000","last_fullsized":"2024-06-03T03:21:54.895740+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:01:26.556726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"99'5","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.825730+0000","last_change":"2024-06-03T03:20:25.904215+0000","last_active":"2024-06-03T03:22:54.825730+0000","last_peered":"2024-06-03T03:22:54.825730+0000","last_clean":"2024-06-03T03:22:54.825730+0000","last_became_active":"2024-06-03T03:20:25.903894+0000","last_became_peered":"2024-06-03T03:20:25.903894+0000","last_unstale":"2024-06-03T03:22:54.825730+0000","last_undegraded":"2024-06-03T03:22:54.825730+0000","last_fullsized":"2024-06-03T03:22:54.825730+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:02:07.587159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:04.896819+0000","last_change":"2024-06-03T03:19:56.910041+0000","last_active":"2024-06-03T03:22:04.896819+0000","last_peered":"2024-06-03T03:22:04.896819+0000","last_clean":"2024-06-03T03:22:04.896819+0000","last_became_active":"2024-06-03T03:19:56.909591+0000","last_became_peered":"2024-06-03T03:19:56.909591+0000","last_unstale":"2024-06-03T03:22:04.896819+0000","last_undegraded":"2024-06-03T03:22:04.896819+0000","last_fullsized":"2024-06-03T03:22:04.896819+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:42:20.016838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"97'1","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:40.880350+0000","last_change":"2024-06-03T03:19:36.741731+0000","last_active":"2024-06-03T03:21:40.880350+0000","last_peered":"2024-06-03T03:21:40.880350+0000","last_clean":"2024-06-03T03:21:40.880350+0000","last_became_active":"2024-06-03T03:19:36.741329+0000","last_became_peered":"2024-06-03T03:19:36.741329+0000","last_unstale":"2024-06-03T03:21:40.880350+0000","last_undegraded":"2024-06-03T03:21:40.880350+0000","last_fullsized":"2024-06-03T03:21:40.880350+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:02:45.778368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"99'4","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.825952+0000","last_change":"2024-06-03T03:19:41.632225+0000","last_active":"2024-06-03T03:22:54.825952+0000","last_peered":"2024-06-03T03:22:54.825952+0000","last_clean":"2024-06-03T03:22:54.825952+0000","last_became_active":"2024-06-03T03:19:41.631851+0000","last_became_peered":"2024-06-03T03:19:41.631851+0000","last_unstale":"2024-06-03T03:22:54.825952+0000","last_undegraded":"2024-06-03T03:22:54.825952+0000","last_fullsized":"2024-06-03T03:22:54.825952+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:52:23.364233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:03.279638+0000","last_change":"2024-06-03T03:19:56.934200+0000","last_active":"2024-06-03T03:22:03.279638+0000","last_peered":"2024-06-03T03:22:03.279638+0000","last_clean":"2024-06-03T03:22:03.279638+0000","last_became_active":"2024-06-03T03:19:56.932998+0000","last_became_peered":"2024-06-03T03:19:56.932998+0000","last_unstale":"2024-06-03T03:22:03.279638+0000","last_undegraded":"2024-06-03T03:22:03.279638+0000","last_fullsized":"2024-06-03T03:22:03.279638+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:45:28.916023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"99'6","reported_seq":301,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:25:13.519391+0000","last_change":"2024-06-03T03:19:36.546630+0000","last_active":"2024-06-03T03:25:13.519391+0000","last_peered":"2024-06-03T03:25:13.519391+0000","last_clean":"2024-06-03T03:25:13.519391+0000","last_became_active":"2024-06-03T03:19:36.546248+0000","last_became_peered":"2024-06-03T03:19:36.546248+0000","last_unstale":"2024-06-03T03:25:13.519391+0000","last_undegraded":"2024-06-03T03:25:13.519391+0000","last_fullsized":"2024-06-03T03:25:13.519391+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T15:09:50.323113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":249,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"99'7","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.831073+0000","last_change":"2024-06-03T03:19:41.624584+0000","last_active":"2024-06-03T03:22:54.831073+0000","last_peered":"2024-06-03T03:22:54.831073+0000","last_clean":"2024-06-03T03:22:54.831073+0000","last_became_active":"2024-06-03T03:19:41.624216+0000","last_became_peered":"2024-06-03T03:19:41.624216+0000","last_unstale":"2024-06-03T03:22:54.831073+0000","last_undegraded":"2024-06-03T03:22:54.831073+0000","last_fullsized":"2024-06-03T03:22:54.831073+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:22:09.645341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:08.280228+0000","last_change":"2024-06-03T03:19:56.937422+0000","last_active":"2024-06-03T03:22:08.280228+0000","last_peered":"2024-06-03T03:22:08.280228+0000","last_clean":"2024-06-03T03:22:08.280228+0000","last_became_active":"2024-06-03T03:19:56.934075+0000","last_became_peered":"2024-06-03T03:19:56.934075+0000","last_unstale":"2024-06-03T03:22:08.280228+0000","last_undegraded":"2024-06-03T03:22:08.280228+0000","last_fullsized":"2024-06-03T03:22:08.280228+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:25:45.273161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"99'8","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.814169+0000","last_change":"2024-06-03T03:19:41.589595+0000","last_active":"2024-06-03T03:22:54.814169+0000","last_peered":"2024-06-03T03:22:54.814169+0000","last_clean":"2024-06-03T03:22:54.814169+0000","last_became_active":"2024-06-03T03:19:41.589251+0000","last_became_peered":"2024-06-03T03:19:41.589251+0000","last_unstale":"2024-06-03T03:22:54.814169+0000","last_undegraded":"2024-06-03T03:22:54.814169+0000","last_fullsized":"2024-06-03T03:22:54.814169+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:51:40.417386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"97'5","reported_seq":301,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:25:16.524308+0000","last_change":"2024-06-03T03:19:36.737523+0000","last_active":"2024-06-03T03:25:16.524308+0000","last_peered":"2024-06-03T03:25:16.524308+0000","last_clean":"2024-06-03T03:25:16.524308+0000","last_became_active":"2024-06-03T03:19:36.736698+0000","last_became_peered":"2024-06-03T03:19:36.736698+0000","last_unstale":"2024-06-03T03:25:16.524308+0000","last_undegraded":"2024-06-03T03:25:16.524308+0000","last_fullsized":"2024-06-03T03:25:16.524308+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:14:46.330549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":251,"num_read_kb":251,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"97'1","reported_seq":31,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:04.852726+0000","last_change":"2024-06-03T03:19:56.924421+0000","last_active":"2024-06-03T03:22:04.852726+0000","last_peered":"2024-06-03T03:22:04.852726+0000","last_clean":"2024-06-03T03:22:04.852726+0000","last_became_active":"2024-06-03T03:19:56.924043+0000","last_became_peered":"2024-06-03T03:19:56.924043+0000","last_unstale":"2024-06-03T03:22:04.852726+0000","last_undegraded":"2024-06-03T03:22:04.852726+0000","last_fullsized":"2024-06-03T03:22:04.852726+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:00:47.674717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.826839+0000","last_change":"2024-06-03T03:19:41.613800+0000","last_active":"2024-06-03T03:22:54.826839+0000","last_peered":"2024-06-03T03:22:54.826839+0000","last_clean":"2024-06-03T03:22:54.826839+0000","last_became_active":"2024-06-03T03:19:41.613387+0000","last_became_peered":"2024-06-03T03:19:41.613387+0000","last_unstale":"2024-06-03T03:22:54.826839+0000","last_undegraded":"2024-06-03T03:22:54.826839+0000","last_fullsized":"2024-06-03T03:22:54.826839+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:04:04.785023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"97'6","reported_seq":54,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:42.305912+0000","last_change":"2024-06-03T03:19:36.544518+0000","last_active":"2024-06-03T03:21:42.305912+0000","last_peered":"2024-06-03T03:21:42.305912+0000","last_clean":"2024-06-03T03:21:42.305912+0000","last_became_active":"2024-06-03T03:19:36.544056+0000","last_became_peered":"2024-06-03T03:19:36.544056+0000","last_unstale":"2024-06-03T03:21:42.305912+0000","last_undegraded":"2024-06-03T03:21:42.305912+0000","last_fullsized":"2024-06-03T03:21:42.305912+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:33:25.012986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":368,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:39.856348+0000","last_change":"2024-06-03T03:18:05.481351+0000","last_active":"2024-06-03T03:22:39.856348+0000","last_peered":"2024-06-03T03:22:39.856348+0000","last_clean":"2024-06-03T03:22:39.856348+0000","last_became_active":"2024-06-03T03:18:05.480929+0000","last_became_peered":"2024-06-03T03:18:05.480929+0000","last_unstale":"2024-06-03T03:22:39.856348+0000","last_undegraded":"2024-06-03T03:22:39.856348+0000","last_fullsized":"2024-06-03T03:22:39.856348+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:14:09.558597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:14:09.558597+0000","last_clean_scrub_stamp":"2024-06-03T03:14:09.558597+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:54:45.563733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:07.447611+0000","last_change":"2024-06-03T03:19:56.931867+0000","last_active":"2024-06-03T03:22:07.447611+0000","last_peered":"2024-06-03T03:22:07.447611+0000","last_clean":"2024-06-03T03:22:07.447611+0000","last_became_active":"2024-06-03T03:19:56.931538+0000","last_became_peered":"2024-06-03T03:19:56.931538+0000","last_unstale":"2024-06-03T03:22:07.447611+0000","last_undegraded":"2024-06-03T03:22:07.447611+0000","last_fullsized":"2024-06-03T03:22:07.447611+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:55:15.144192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"99'5","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.814110+0000","last_change":"2024-06-03T03:19:41.622713+0000","last_active":"2024-06-03T03:22:54.814110+0000","last_peered":"2024-06-03T03:22:54.814110+0000","last_clean":"2024-06-03T03:22:54.814110+0000","last_became_active":"2024-06-03T03:19:41.622363+0000","last_became_peered":"2024-06-03T03:19:41.622363+0000","last_unstale":"2024-06-03T03:22:54.814110+0000","last_undegraded":"2024-06-03T03:22:54.814110+0000","last_fullsized":"2024-06-03T03:22:54.814110+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:43:57.671797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"97'4","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:39.850108+0000","last_change":"2024-06-03T03:19:36.539663+0000","last_active":"2024-06-03T03:21:39.850108+0000","last_peered":"2024-06-03T03:21:39.850108+0000","last_clean":"2024-06-03T03:21:39.850108+0000","last_became_active":"2024-06-03T03:19:36.539320+0000","last_became_peered":"2024-06-03T03:19:36.539320+0000","last_unstale":"2024-06-03T03:21:39.850108+0000","last_undegraded":"2024-06-03T03:21:39.850108+0000","last_fullsized":"2024-06-03T03:21:39.850108+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:36:50.699231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:04.314937+0000","last_change":"2024-06-03T03:19:56.938231+0000","last_active":"2024-06-03T03:22:04.314937+0000","last_peered":"2024-06-03T03:22:04.314937+0000","last_clean":"2024-06-03T03:22:04.314937+0000","last_became_active":"2024-06-03T03:19:56.937840+0000","last_became_peered":"2024-06-03T03:19:56.937840+0000","last_unstale":"2024-06-03T03:22:04.314937+0000","last_undegraded":"2024-06-03T03:22:04.314937+0000","last_fullsized":"2024-06-03T03:22:04.314937+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:02:33.916980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"97'4","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:44.894507+0000","last_change":"2024-06-03T03:19:36.737519+0000","last_active":"2024-06-03T03:21:44.894507+0000","last_peered":"2024-06-03T03:21:44.894507+0000","last_clean":"2024-06-03T03:21:44.894507+0000","last_became_active":"2024-06-03T03:19:36.737191+0000","last_became_peered":"2024-06-03T03:19:36.737191+0000","last_unstale":"2024-06-03T03:21:44.894507+0000","last_undegraded":"2024-06-03T03:21:44.894507+0000","last_fullsized":"2024-06-03T03:21:44.894507+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:50:21.389739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.817624+0000","last_change":"2024-06-03T03:19:41.633113+0000","last_active":"2024-06-03T03:22:54.817624+0000","last_peered":"2024-06-03T03:22:54.817624+0000","last_clean":"2024-06-03T03:22:54.817624+0000","last_became_active":"2024-06-03T03:19:41.632186+0000","last_became_peered":"2024-06-03T03:19:41.632186+0000","last_unstale":"2024-06-03T03:22:54.817624+0000","last_undegraded":"2024-06-03T03:22:54.817624+0000","last_fullsized":"2024-06-03T03:22:54.817624+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T15:12:26.835061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:56.770202+0000","last_change":"2024-06-03T03:19:56.928603+0000","last_active":"2024-06-03T03:21:56.770202+0000","last_peered":"2024-06-03T03:21:56.770202+0000","last_clean":"2024-06-03T03:21:56.770202+0000","last_became_active":"2024-06-03T03:19:56.928175+0000","last_became_peered":"2024-06-03T03:19:56.928175+0000","last_unstale":"2024-06-03T03:21:56.770202+0000","last_undegraded":"2024-06-03T03:21:56.770202+0000","last_fullsized":"2024-06-03T03:21:56.770202+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:26:03.531091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"99'21","reported_seq":67,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:25:10.676315+0000","last_change":"2024-06-03T03:19:36.550577+0000","last_active":"2024-06-03T03:25:10.676315+0000","last_peered":"2024-06-03T03:25:10.676315+0000","last_clean":"2024-06-03T03:25:10.676315+0000","last_became_active":"2024-06-03T03:19:36.549968+0000","last_became_peered":"2024-06-03T03:19:36.549968+0000","last_unstale":"2024-06-03T03:25:10.676315+0000","last_undegraded":"2024-06-03T03:25:10.676315+0000","last_fullsized":"2024-06-03T03:25:10.676315+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:22:43.892002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":38,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"99'5","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.825483+0000","last_change":"2024-06-03T03:19:41.613883+0000","last_active":"2024-06-03T03:22:54.825483+0000","last_peered":"2024-06-03T03:22:54.825483+0000","last_clean":"2024-06-03T03:22:54.825483+0000","last_became_active":"2024-06-03T03:19:41.613472+0000","last_became_peered":"2024-06-03T03:19:41.613472+0000","last_unstale":"2024-06-03T03:22:54.825483+0000","last_undegraded":"2024-06-03T03:22:54.825483+0000","last_fullsized":"2024-06-03T03:22:54.825483+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T14:38:42.044534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:04.126263+0000","last_change":"2024-06-03T03:19:56.914487+0000","last_active":"2024-06-03T03:22:04.126263+0000","last_peered":"2024-06-03T03:22:04.126263+0000","last_clean":"2024-06-03T03:22:04.126263+0000","last_became_active":"2024-06-03T03:19:56.913904+0000","last_became_peered":"2024-06-03T03:19:56.913904+0000","last_unstale":"2024-06-03T03:22:04.126263+0000","last_undegraded":"2024-06-03T03:22:04.126263+0000","last_fullsized":"2024-06-03T03:22:04.126263+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T15:13:24.084034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"97'7","reported_seq":53,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:47.306495+0000","last_change":"2024-06-03T03:19:36.671355+0000","last_active":"2024-06-03T03:21:47.306495+0000","last_peered":"2024-06-03T03:21:47.306495+0000","last_clean":"2024-06-03T03:21:47.306495+0000","last_became_active":"2024-06-03T03:19:36.670926+0000","last_became_peered":"2024-06-03T03:19:36.670926+0000","last_unstale":"2024-06-03T03:21:47.306495+0000","last_undegraded":"2024-06-03T03:21:47.306495+0000","last_fullsized":"2024-06-03T03:21:47.306495+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:38:50.914238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.835937+0000","last_change":"2024-06-03T03:19:41.629054+0000","last_active":"2024-06-03T03:22:54.835937+0000","last_peered":"2024-06-03T03:22:54.835937+0000","last_clean":"2024-06-03T03:22:54.835937+0000","last_became_active":"2024-06-03T03:19:41.627118+0000","last_became_peered":"2024-06-03T03:19:41.627118+0000","last_unstale":"2024-06-03T03:22:54.835937+0000","last_undegraded":"2024-06-03T03:22:54.835937+0000","last_fullsized":"2024-06-03T03:22:54.835937+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:59:40.645002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:12.309263+0000","last_change":"2024-06-03T03:21:26.058741+0000","last_active":"2024-06-03T03:22:12.309263+0000","last_peered":"2024-06-03T03:22:12.309263+0000","last_clean":"2024-06-03T03:22:12.309263+0000","last_became_active":"2024-06-03T03:21:26.058271+0000","last_became_peered":"2024-06-03T03:21:26.058271+0000","last_unstale":"2024-06-03T03:22:12.309263+0000","last_undegraded":"2024-06-03T03:22:12.309263+0000","last_fullsized":"2024-06-03T03:22:12.309263+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:31:32.359893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"97'2","reported_seq":54,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:50.231392+0000","last_change":"2024-06-03T03:20:26.132994+0000","last_active":"2024-06-03T03:21:50.231392+0000","last_peered":"2024-06-03T03:21:50.231392+0000","last_clean":"2024-06-03T03:21:50.231392+0000","last_became_active":"2024-06-03T03:20:26.132574+0000","last_became_peered":"2024-06-03T03:20:26.132574+0000","last_unstale":"2024-06-03T03:21:50.231392+0000","last_undegraded":"2024-06-03T03:21:50.231392+0000","last_fullsized":"2024-06-03T03:21:50.231392+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T15:12:33.599154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"99'2","reported_seq":41,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.791970+0000","last_change":"2024-06-03T03:19:41.603630+0000","last_active":"2024-06-03T03:22:54.791970+0000","last_peered":"2024-06-03T03:22:54.791970+0000","last_clean":"2024-06-03T03:22:54.791970+0000","last_became_active":"2024-06-03T03:19:41.603301+0000","last_became_peered":"2024-06-03T03:19:41.603301+0000","last_unstale":"2024-06-03T03:22:54.791970+0000","last_undegraded":"2024-06-03T03:22:54.791970+0000","last_fullsized":"2024-06-03T03:22:54.791970+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:20:33.076622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:15.885256+0000","last_change":"2024-06-03T03:19:56.929023+0000","last_active":"2024-06-03T03:22:15.885256+0000","last_peered":"2024-06-03T03:22:15.885256+0000","last_clean":"2024-06-03T03:22:15.885256+0000","last_became_active":"2024-06-03T03:19:56.928454+0000","last_became_peered":"2024-06-03T03:19:56.928454+0000","last_unstale":"2024-06-03T03:22:15.885256+0000","last_undegraded":"2024-06-03T03:22:15.885256+0000","last_fullsized":"2024-06-03T03:22:15.885256+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:17:31.755649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"97'7","reported_seq":52,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:46.768385+0000","last_change":"2024-06-03T03:19:36.551413+0000","last_active":"2024-06-03T03:21:46.768385+0000","last_peered":"2024-06-03T03:21:46.768385+0000","last_clean":"2024-06-03T03:21:46.768385+0000","last_became_active":"2024-06-03T03:19:36.550622+0000","last_became_peered":"2024-06-03T03:19:36.550622+0000","last_unstale":"2024-06-03T03:21:46.768385+0000","last_undegraded":"2024-06-03T03:21:46.768385+0000","last_fullsized":"2024-06-03T03:21:46.768385+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:50:09.472960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.830703+0000","last_change":"2024-06-03T03:19:41.624871+0000","last_active":"2024-06-03T03:22:54.830703+0000","last_peered":"2024-06-03T03:22:54.830703+0000","last_clean":"2024-06-03T03:22:54.830703+0000","last_became_active":"2024-06-03T03:19:41.624638+0000","last_became_peered":"2024-06-03T03:19:41.624638+0000","last_unstale":"2024-06-03T03:22:54.830703+0000","last_undegraded":"2024-06-03T03:22:54.830703+0000","last_fullsized":"2024-06-03T03:22:54.830703+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:21:16.193817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"99'1","reported_seq":31,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:12.309224+0000","last_change":"2024-06-03T03:19:56.924638+0000","last_active":"2024-06-03T03:22:12.309224+0000","last_peered":"2024-06-03T03:22:12.309224+0000","last_clean":"2024-06-03T03:22:12.309224+0000","last_became_active":"2024-06-03T03:19:56.924200+0000","last_became_peered":"2024-06-03T03:19:56.924200+0000","last_unstale":"2024-06-03T03:22:12.309224+0000","last_undegraded":"2024-06-03T03:22:12.309224+0000","last_fullsized":"2024-06-03T03:22:12.309224+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:38:01.138253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"99'5","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:25:11.017683+0000","last_change":"2024-06-03T03:19:36.546974+0000","last_active":"2024-06-03T03:25:11.017683+0000","last_peered":"2024-06-03T03:25:11.017683+0000","last_clean":"2024-06-03T03:25:11.017683+0000","last_became_active":"2024-06-03T03:19:36.546433+0000","last_became_peered":"2024-06-03T03:19:36.546433+0000","last_unstale":"2024-06-03T03:25:11.017683+0000","last_undegraded":"2024-06-03T03:25:11.017683+0000","last_fullsized":"2024-06-03T03:25:11.017683+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:41:46.980327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.834248+0000","last_change":"2024-06-03T03:19:41.622118+0000","last_active":"2024-06-03T03:22:54.834248+0000","last_peered":"2024-06-03T03:22:54.834248+0000","last_clean":"2024-06-03T03:22:54.834248+0000","last_became_active":"2024-06-03T03:19:41.620789+0000","last_became_peered":"2024-06-03T03:19:41.620789+0000","last_unstale":"2024-06-03T03:22:54.834248+0000","last_undegraded":"2024-06-03T03:22:54.834248+0000","last_fullsized":"2024-06-03T03:22:54.834248+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:00:36.232238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:12.309247+0000","last_change":"2024-06-03T03:19:56.921299+0000","last_active":"2024-06-03T03:22:12.309247+0000","last_peered":"2024-06-03T03:22:12.309247+0000","last_clean":"2024-06-03T03:22:12.309247+0000","last_became_active":"2024-06-03T03:19:56.920870+0000","last_became_peered":"2024-06-03T03:19:56.920870+0000","last_unstale":"2024-06-03T03:22:12.309247+0000","last_undegraded":"2024-06-03T03:22:12.309247+0000","last_fullsized":"2024-06-03T03:22:12.309247+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:10:15.062381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"97'4","reported_seq":24,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:44.319531+0000","last_change":"2024-06-03T03:20:25.903279+0000","last_active":"2024-06-03T03:22:44.319531+0000","last_peered":"2024-06-03T03:22:44.319531+0000","last_clean":"2024-06-03T03:22:44.319531+0000","last_became_active":"2024-06-03T03:20:25.900311+0000","last_became_peered":"2024-06-03T03:20:25.900311+0000","last_unstale":"2024-06-03T03:22:44.319531+0000","last_undegraded":"2024-06-03T03:22:44.319531+0000","last_fullsized":"2024-06-03T03:22:44.319531+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:25:49.708581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"99'5","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.836223+0000","last_change":"2024-06-03T03:19:41.628659+0000","last_active":"2024-06-03T03:22:54.836223+0000","last_peered":"2024-06-03T03:22:54.836223+0000","last_clean":"2024-06-03T03:22:54.836223+0000","last_became_active":"2024-06-03T03:19:41.628291+0000","last_became_peered":"2024-06-03T03:19:41.628291+0000","last_unstale":"2024-06-03T03:22:54.836223+0000","last_undegraded":"2024-06-03T03:22:54.836223+0000","last_fullsized":"2024-06-03T03:22:54.836223+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:26:49.579669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:10.233742+0000","last_change":"2024-06-03T03:19:56.922766+0000","last_active":"2024-06-03T03:22:10.233742+0000","last_peered":"2024-06-03T03:22:10.233742+0000","last_clean":"2024-06-03T03:22:10.233742+0000","last_became_active":"2024-06-03T03:19:56.922310+0000","last_became_peered":"2024-06-03T03:19:56.922310+0000","last_unstale":"2024-06-03T03:22:10.233742+0000","last_undegraded":"2024-06-03T03:22:10.233742+0000","last_fullsized":"2024-06-03T03:22:10.233742+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T14:35:53.133519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"99'6","reported_seq":275,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:25:15.410739+0000","last_change":"2024-06-03T03:20:26.133562+0000","last_active":"2024-06-03T03:25:15.410739+0000","last_peered":"2024-06-03T03:25:15.410739+0000","last_clean":"2024-06-03T03:25:15.410739+0000","last_became_active":"2024-06-03T03:20:26.133074+0000","last_became_peered":"2024-06-03T03:20:26.133074+0000","last_unstale":"2024-06-03T03:25:15.410739+0000","last_undegraded":"2024-06-03T03:25:15.410739+0000","last_fullsized":"2024-06-03T03:25:15.410739+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:51:33.717584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1560,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":252,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"99'7","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.818537+0000","last_change":"2024-06-03T03:19:41.611450+0000","last_active":"2024-06-03T03:22:54.818537+0000","last_peered":"2024-06-03T03:22:54.818537+0000","last_clean":"2024-06-03T03:22:54.818537+0000","last_became_active":"2024-06-03T03:19:41.610214+0000","last_became_peered":"2024-06-03T03:19:41.610214+0000","last_unstale":"2024-06-03T03:22:54.818537+0000","last_undegraded":"2024-06-03T03:22:54.818537+0000","last_fullsized":"2024-06-03T03:22:54.818537+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:01:01.521917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:14.845604+0000","last_change":"2024-06-03T03:19:56.935686+0000","last_active":"2024-06-03T03:22:14.845604+0000","last_peered":"2024-06-03T03:22:14.845604+0000","last_clean":"2024-06-03T03:22:14.845604+0000","last_became_active":"2024-06-03T03:19:56.935314+0000","last_became_peered":"2024-06-03T03:19:56.935314+0000","last_unstale":"2024-06-03T03:22:14.845604+0000","last_undegraded":"2024-06-03T03:22:14.845604+0000","last_fullsized":"2024-06-03T03:22:14.845604+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T15:10:16.966321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"97'3","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:52.445989+0000","last_change":"2024-06-03T03:19:36.547756+0000","last_active":"2024-06-03T03:21:52.445989+0000","last_peered":"2024-06-03T03:21:52.445989+0000","last_clean":"2024-06-03T03:21:52.445989+0000","last_became_active":"2024-06-03T03:19:36.547350+0000","last_became_peered":"2024-06-03T03:19:36.547350+0000","last_unstale":"2024-06-03T03:21:52.445989+0000","last_undegraded":"2024-06-03T03:21:52.445989+0000","last_fullsized":"2024-06-03T03:21:52.445989+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:57:39.214380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"99'6","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.828041+0000","last_change":"2024-06-03T03:19:41.621083+0000","last_active":"2024-06-03T03:22:54.828041+0000","last_peered":"2024-06-03T03:22:54.828041+0000","last_clean":"2024-06-03T03:22:54.828041+0000","last_became_active":"2024-06-03T03:19:41.620719+0000","last_became_peered":"2024-06-03T03:19:41.620719+0000","last_unstale":"2024-06-03T03:22:54.828041+0000","last_undegraded":"2024-06-03T03:22:54.828041+0000","last_fullsized":"2024-06-03T03:22:54.828041+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:50:35.007025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:07.447552+0000","last_change":"2024-06-03T03:19:56.915406+0000","last_active":"2024-06-03T03:22:07.447552+0000","last_peered":"2024-06-03T03:22:07.447552+0000","last_clean":"2024-06-03T03:22:07.447552+0000","last_became_active":"2024-06-03T03:19:56.914918+0000","last_became_peered":"2024-06-03T03:19:56.914918+0000","last_unstale":"2024-06-03T03:22:07.447552+0000","last_undegraded":"2024-06-03T03:22:07.447552+0000","last_fullsized":"2024-06-03T03:22:07.447552+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:55.874205+0000","last_clean_scrub_stamp":"2024-06-03T03:19:55.874205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:42:59.324011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"97'5","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:21:53.277541+0000","last_change":"2024-06-03T03:19:36.549502+0000","last_active":"2024-06-03T03:21:53.277541+0000","last_peered":"2024-06-03T03:21:53.277541+0000","last_clean":"2024-06-03T03:21:53.277541+0000","last_became_active":"2024-06-03T03:19:36.547538+0000","last_became_peered":"2024-06-03T03:19:36.547538+0000","last_unstale":"2024-06-03T03:21:53.277541+0000","last_undegraded":"2024-06-03T03:21:53.277541+0000","last_fullsized":"2024-06-03T03:21:53.277541+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:35.503645+0000","last_clean_scrub_stamp":"2024-06-03T03:19:35.503645+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:28:50.558069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"99'9","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.815966+0000","last_change":"2024-06-03T03:19:41.607441+0000","last_active":"2024-06-03T03:22:54.815966+0000","last_peered":"2024-06-03T03:22:54.815966+0000","last_clean":"2024-06-03T03:22:54.815966+0000","last_became_active":"2024-06-03T03:19:41.607086+0000","last_became_peered":"2024-06-03T03:19:41.607086+0000","last_unstale":"2024-06-03T03:22:54.815966+0000","last_undegraded":"2024-06-03T03:22:54.815966+0000","last_fullsized":"2024-06-03T03:22:54.815966+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T14:53:50.091872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.819602+0000","last_change":"2024-06-03T03:19:41.607004+0000","last_active":"2024-06-03T03:22:54.819602+0000","last_peered":"2024-06-03T03:22:54.819602+0000","last_clean":"2024-06-03T03:22:54.819602+0000","last_became_active":"2024-06-03T03:19:41.606684+0000","last_became_peered":"2024-06-03T03:19:41.606684+0000","last_unstale":"2024-06-03T03:22:54.819602+0000","last_undegraded":"2024-06-03T03:22:54.819602+0000","last_fullsized":"2024-06-03T03:22:54.819602+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:48:55.320597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"99'12","reported_seq":51,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.817158+0000","last_change":"2024-06-03T03:19:41.589426+0000","last_active":"2024-06-03T03:22:54.817158+0000","last_peered":"2024-06-03T03:22:54.817158+0000","last_clean":"2024-06-03T03:22:54.817158+0000","last_became_active":"2024-06-03T03:19:41.589091+0000","last_became_peered":"2024-06-03T03:19:41.589091+0000","last_unstale":"2024-06-03T03:22:54.817158+0000","last_undegraded":"2024-06-03T03:22:54.817158+0000","last_fullsized":"2024-06-03T03:22:54.817158+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:40:34.313214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.832557+0000","last_change":"2024-06-03T03:19:41.626149+0000","last_active":"2024-06-03T03:22:54.832557+0000","last_peered":"2024-06-03T03:22:54.832557+0000","last_clean":"2024-06-03T03:22:54.832557+0000","last_became_active":"2024-06-03T03:19:41.625710+0000","last_became_peered":"2024-06-03T03:19:41.625710+0000","last_unstale":"2024-06-03T03:22:54.832557+0000","last_undegraded":"2024-06-03T03:22:54.832557+0000","last_fullsized":"2024-06-03T03:22:54.832557+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:15:15.220975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.838595+0000","last_change":"2024-06-03T03:19:41.636585+0000","last_active":"2024-06-03T03:22:54.838595+0000","last_peered":"2024-06-03T03:22:54.838595+0000","last_clean":"2024-06-03T03:22:54.838595+0000","last_became_active":"2024-06-03T03:19:41.636241+0000","last_became_peered":"2024-06-03T03:19:41.636241+0000","last_unstale":"2024-06-03T03:22:54.838595+0000","last_undegraded":"2024-06-03T03:22:54.838595+0000","last_fullsized":"2024-06-03T03:22:54.838595+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:19:28.548297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"99'4","reported_seq":43,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.834220+0000","last_change":"2024-06-03T03:19:41.629530+0000","last_active":"2024-06-03T03:22:54.834220+0000","last_peered":"2024-06-03T03:22:54.834220+0000","last_clean":"2024-06-03T03:22:54.834220+0000","last_became_active":"2024-06-03T03:19:41.629197+0000","last_became_peered":"2024-06-03T03:19:41.629197+0000","last_unstale":"2024-06-03T03:22:54.834220+0000","last_undegraded":"2024-06-03T03:22:54.834220+0000","last_fullsized":"2024-06-03T03:22:54.834220+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:12:02.807445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.819264+0000","last_change":"2024-06-03T03:19:41.606995+0000","last_active":"2024-06-03T03:22:54.819264+0000","last_peered":"2024-06-03T03:22:54.819264+0000","last_clean":"2024-06-03T03:22:54.819264+0000","last_became_active":"2024-06-03T03:19:41.606651+0000","last_became_peered":"2024-06-03T03:19:41.606651+0000","last_unstale":"2024-06-03T03:22:54.819264+0000","last_undegraded":"2024-06-03T03:22:54.819264+0000","last_fullsized":"2024-06-03T03:22:54.819264+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:07:11.751606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"99'15","reported_seq":54,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.810136+0000","last_change":"2024-06-03T03:19:41.609848+0000","last_active":"2024-06-03T03:22:54.810136+0000","last_peered":"2024-06-03T03:22:54.810136+0000","last_clean":"2024-06-03T03:22:54.810136+0000","last_became_active":"2024-06-03T03:19:41.609544+0000","last_became_peered":"2024-06-03T03:19:41.609544+0000","last_unstale":"2024-06-03T03:22:54.810136+0000","last_undegraded":"2024-06-03T03:22:54.810136+0000","last_fullsized":"2024-06-03T03:22:54.810136+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:12:00.804638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"99'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.810509+0000","last_change":"2024-06-03T03:20:25.912049+0000","last_active":"2024-06-03T03:22:54.810509+0000","last_peered":"2024-06-03T03:22:54.810509+0000","last_clean":"2024-06-03T03:22:54.810509+0000","last_became_active":"2024-06-03T03:20:25.910664+0000","last_became_peered":"2024-06-03T03:20:25.910664+0000","last_unstale":"2024-06-03T03:22:54.810509+0000","last_undegraded":"2024-06-03T03:22:54.810509+0000","last_fullsized":"2024-06-03T03:22:54.810509+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:47:19.282569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"99'6","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.828822+0000","last_change":"2024-06-03T03:19:41.610800+0000","last_active":"2024-06-03T03:22:54.828822+0000","last_peered":"2024-06-03T03:22:54.828822+0000","last_clean":"2024-06-03T03:22:54.828822+0000","last_became_active":"2024-06-03T03:19:41.610457+0000","last_became_peered":"2024-06-03T03:19:41.610457+0000","last_unstale":"2024-06-03T03:22:54.828822+0000","last_undegraded":"2024-06-03T03:22:54.828822+0000","last_fullsized":"2024-06-03T03:22:54.828822+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:31:06.415159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.821379+0000","last_change":"2024-06-03T03:19:41.617020+0000","last_active":"2024-06-03T03:22:54.821379+0000","last_peered":"2024-06-03T03:22:54.821379+0000","last_clean":"2024-06-03T03:22:54.821379+0000","last_became_active":"2024-06-03T03:19:41.616614+0000","last_became_peered":"2024-06-03T03:19:41.616614+0000","last_unstale":"2024-06-03T03:22:54.821379+0000","last_undegraded":"2024-06-03T03:22:54.821379+0000","last_fullsized":"2024-06-03T03:22:54.821379+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:35:32.657685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"99'10","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.828254+0000","last_change":"2024-06-03T03:19:41.622785+0000","last_active":"2024-06-03T03:22:54.828254+0000","last_peered":"2024-06-03T03:22:54.828254+0000","last_clean":"2024-06-03T03:22:54.828254+0000","last_became_active":"2024-06-03T03:19:41.622407+0000","last_became_peered":"2024-06-03T03:19:41.622407+0000","last_unstale":"2024-06-03T03:22:54.828254+0000","last_undegraded":"2024-06-03T03:22:54.828254+0000","last_fullsized":"2024-06-03T03:22:54.828254+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:53:50.804889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"99'11","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.838978+0000","last_change":"2024-06-03T03:19:41.633429+0000","last_active":"2024-06-03T03:22:54.838978+0000","last_peered":"2024-06-03T03:22:54.838978+0000","last_clean":"2024-06-03T03:22:54.838978+0000","last_became_active":"2024-06-03T03:19:41.631028+0000","last_became_peered":"2024-06-03T03:19:41.631028+0000","last_unstale":"2024-06-03T03:22:54.838978+0000","last_undegraded":"2024-06-03T03:22:54.838978+0000","last_fullsized":"2024-06-03T03:22:54.838978+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:37:10.666591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-03T03:22:54.818463+0000","last_change":"2024-06-03T03:19:41.606204+0000","last_active":"2024-06-03T03:22:54.818463+0000","last_peered":"2024-06-03T03:22:54.818463+0000","last_clean":"2024-06-03T03:22:54.818463+0000","last_became_active":"2024-06-03T03:19:41.605756+0000","last_became_peered":"2024-06-03T03:19:41.605756+0000","last_unstale":"2024-06-03T03:22:54.818463+0000","last_undegraded":"2024-06-03T03:22:54.818463+0000","last_fullsized":"2024-06-03T03:22:54.818463+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T03:19:40.551357+0000","last_clean_scrub_stamp":"2024-06-03T03:19:40.551357+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:24:32.507426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":245318,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":82643,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":2538,"num_read_kb":1361,"num_write":283,"num_write_kb":174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589824,"data_stored":247929,"data_compressed":5580,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":241,"ondisk_log_size":241,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383764,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1101824,"data_stored":570765,"data_compressed":32353,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645402,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32404,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712748,"statfs":{"total":95995035648,"available":95961853952,"internally_reserved":0,"allocated":1257472,"data_stored":1231405,"data_compressed":69963,"data_compressed_allocated":94208,"data_compressed_original":688128,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808927,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32280,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712872,"statfs":{"total":95995035648,"available":95961980928,"internally_reserved":0,"allocated":1130496,"data_stored":574401,"data_compressed":32353,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070566,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32272,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712880,"statfs":{"total":95995035648,"available":95961989120,"internally_reserved":0,"allocated":1122304,"data_stored":570683,"data_compressed":32353,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332203,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32324,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712828,"statfs":{"total":95995035648,"available":95961935872,"internally_reserved":0,"allocated":1175552,"data_stored":1168406,"data_compressed":68103,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495730,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1101824,"data_stored":566199,"data_compressed":32353,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574989,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28164,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716988,"statfs":{"total":95995035648,"available":95966195712,"internally_reserved":0,"allocated":1175552,"data_stored":641443,"data_compressed":34213,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738513,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1126400,"data_stored":567655,"data_compressed":32353,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411463,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36496,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708656,"statfs":{"total":95995035648,"available":95957663744,"internally_reserved":0,"allocated":1187840,"data_stored":641330,"data_compressed":34213,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182530,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1114112,"data_stored":565517,"data_compressed":32353,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920893,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28108,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717044,"statfs":{"total":95995035648,"available":95966253056,"internally_reserved":0,"allocated":1118208,"data_stored":571443,"data_compressed":32353,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757368,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32332,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712820,"statfs":{"total":95995035648,"available":95961927680,"internally_reserved":0,"allocated":1183744,"data_stored":1163598,"data_compressed":68103,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34106,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34106,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34106,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35750,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35750,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35750,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35750,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":76356,"data_compressed":1860,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":76243,"data_compressed":1860,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":4047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":75950,"data_compressed":1860,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-03T03:25:20.477 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-03T03:25:20.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:25:20 smithi183 ceph-mon[37326]: pgmap v506: 129 pgs: 129 active+clean; 657 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-06-03T03:25:20.965 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:25:21.355 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:25:21 smithi117 ceph-mon[31649]: from='client.25241 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-03T03:25:21.356 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:25:21 smithi117 ceph-mon[31649]: from='client.15135 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-03T03:25:21.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:25:21 smithi151 ceph-mon[37043]: from='client.25241 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-03T03:25:21.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:25:21 smithi151 ceph-mon[37043]: from='client.15135 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-03T03:25:21.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:25:21 smithi183 ceph-mon[37326]: from='client.25241 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-03T03:25:21.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:25:21 smithi183 ceph-mon[37326]: from='client.15135 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-03T03:25:22.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:25:22 smithi151 ceph-mon[37043]: from='client.15141 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-03T03:25:22.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:25:22 smithi151 ceph-mon[37043]: pgmap v507: 129 pgs: 129 active+clean; 657 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-06-03T03:25:22.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:25:22 smithi117 ceph-mon[31649]: from='client.15141 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-03T03:25:22.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:25:22 smithi117 ceph-mon[31649]: pgmap v507: 129 pgs: 129 active+clean; 657 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-06-03T03:25:22.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:25:22 smithi183 ceph-mon[37326]: from='client.15141 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-03T03:25:22.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:25:22 smithi183 ceph-mon[37326]: pgmap v507: 129 pgs: 129 active+clean; 657 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-06-03T03:25:23.464 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:23.463+0000 7fefd0200700 1 -- 172.21.15.117:0/4222453901 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefc8105070 msgr2=0x7fefc8111580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:23.464 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:23.463+0000 7fefd0200700 1 --2- 172.21.15.117:0/4222453901 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefc8105070 0x7fefc8111580 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fefc000b0a0 tx=0x7fefc000b3b0 comp rx=0 tx=0).stop 2024-06-03T03:25:23.464 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:23.463+0000 7fefd0200700 1 -- 172.21.15.117:0/4222453901 shutdown_connections 2024-06-03T03:25:23.464 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:23.463+0000 7fefd0200700 1 --2- 172.21.15.117:0/4222453901 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefc8105070 0x7fefc8111580 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:23.464 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:23.463+0000 7fefd0200700 1 --2- 172.21.15.117:0/4222453901 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fefc810a6b0 0x7fefc810aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:23.464 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:23.463+0000 7fefd0200700 1 --2- 172.21.15.117:0/4222453901 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fefc81046b0 0x7fefc8104b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:23.464 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:23.463+0000 7fefd0200700 1 -- 172.21.15.117:0/4222453901 >> 172.21.15.117:0/4222453901 conn(0x7fefc81001a0 msgr2=0x7fefc81025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:25:23.465 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:23.463+0000 7fefd0200700 1 -- 172.21.15.117:0/4222453901 shutdown_connections 2024-06-03T03:25:23.465 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:23.463+0000 7fefd0200700 1 -- 172.21.15.117:0/4222453901 wait complete. 2024-06-03T03:25:23.466 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:23.464+0000 7fefd0200700 1 Processor -- start 2024-06-03T03:25:23.466 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:23.464+0000 7fefd0200700 1 -- start start 2024-06-03T03:25:23.466 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:23.464+0000 7fefd0200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fefc81046b0 0x7fefc819ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:23.466 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:23.465+0000 7fefd0200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefc8105070 0x7fefc819b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:23.466 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:23.465+0000 7fefd0200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fefc810a6b0 0x7fefc819f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:23.466 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:23.465+0000 7fefd0200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fefc8114a90 con 0x7fefc8105070 2024-06-03T03:25:23.466 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:23.465+0000 7fefd0200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fefc8114910 con 0x7fefc810a6b0 2024-06-03T03:25:23.466 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:23.465+0000 7fefd0200700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fefc8114c10 con 0x7fefc81046b0 2024-06-03T03:25:23.466 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:23.465+0000 7fefcce00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefc8105070 0x7fefc819b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:23.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:23.465+0000 7fefcd800700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fefc81046b0 0x7fefc819ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:23.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:23.465+0000 7fefcd800700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fefc81046b0 0x7fefc819ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:47142/0 (socket says 172.21.15.117:47142) 2024-06-03T03:25:23.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:23.465+0000 7fefcd800700 1 -- 172.21.15.117:0/2210926067 learned_addr learned my addr 172.21.15.117:0/2210926067 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:25:23.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:23.465+0000 7fefce200700 1 --2- 172.21.15.117:0/2210926067 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fefc810a6b0 0x7fefc819f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:23.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:23.465+0000 7fefcce00700 1 -- 172.21.15.117:0/2210926067 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fefc81046b0 msgr2=0x7fefc819ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:23.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:23.465+0000 7fefcce00700 1 --2- 172.21.15.117:0/2210926067 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fefc81046b0 0x7fefc819ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:23.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:23.465+0000 7fefcce00700 1 -- 172.21.15.117:0/2210926067 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fefc810a6b0 msgr2=0x7fefc819f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:23.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:23.465+0000 7fefcce00700 1 --2- 172.21.15.117:0/2210926067 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fefc810a6b0 0x7fefc819f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:23.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:23.465+0000 7fefcce00700 1 -- 172.21.15.117:0/2210926067 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefc819fe90 con 0x7fefc8105070 2024-06-03T03:25:23.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:23.466+0000 7fefcd800700 1 --2- 172.21.15.117:0/2210926067 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fefc81046b0 0x7fefc819ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:25:23.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:23.466+0000 7fefce200700 1 --2- 172.21.15.117:0/2210926067 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fefc810a6b0 0x7fefc819f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:25:23.468 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:23.466+0000 7fefcce00700 1 --2- 172.21.15.117:0/2210926067 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefc8105070 0x7fefc819b3c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fefb400e9f0 tx=0x7fefb400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:25:23.468 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:23.466+0000 7fefc6000700 1 -- 172.21.15.117:0/2210926067 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb400c9c0 con 0x7fefc8105070 2024-06-03T03:25:23.468 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:23.466+0000 7fefc6000700 1 -- 172.21.15.117:0/2210926067 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefb400cb60 con 0x7fefc8105070 2024-06-03T03:25:23.468 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:23.466+0000 7fefc6000700 1 -- 172.21.15.117:0/2210926067 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb4018680 con 0x7fefc8105070 2024-06-03T03:25:23.468 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:23.466+0000 7fefd0200700 1 -- 172.21.15.117:0/2210926067 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefc81a0180 con 0x7fefc8105070 2024-06-03T03:25:23.468 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:23.466+0000 7fefd0200700 1 -- 172.21.15.117:0/2210926067 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefc81a7a00 con 0x7fefc8105070 2024-06-03T03:25:23.469 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:23.467+0000 7fefd0200700 1 -- 172.21.15.117:0/2210926067 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefc8069180 con 0x7fefc8105070 2024-06-03T03:25:23.470 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:23.470+0000 7fefc6000700 1 -- 172.21.15.117:0/2210926067 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7fefb4010430 con 0x7fefc8105070 2024-06-03T03:25:23.471 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:23.471+0000 7fefc6000700 1 --2- 172.21.15.117:0/2210926067 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fefb00a2d30 0x7fefb00a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:23.472 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:23.471+0000 7fefc6000700 1 -- 172.21.15.117:0/2210926067 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7fefb40cf830 con 0x7fefc8105070 2024-06-03T03:25:23.472 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:23.472+0000 7fefcd800700 1 --2- 172.21.15.117:0/2210926067 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fefb00a2d30 0x7fefb00a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:23.472 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:23.472+0000 7fefcd800700 1 --2- 172.21.15.117:0/2210926067 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fefb00a2d30 0x7fefb00a51f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fefbc011550 tx=0x7fefbc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:25:23.474 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:23.474+0000 7fefc6000700 1 -- 172.21.15.117:0/2210926067 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fefb409c240 con 0x7fefc8105070 2024-06-03T03:25:23.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:23.743+0000 7fefd0200700 1 -- 172.21.15.117:0/2210926067 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fefc819c1e0 con 0x7fefc8105070 2024-06-03T03:25:23.747 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:23.745+0000 7fefc6000700 1 -- 172.21.15.117:0/2210926067 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11955 (secure 0 0 0) 0x7fefb409bb40 con 0x7fefc8105070 2024-06-03T03:25:23.747 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T03:25:23.747 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3353001119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3353001119},{"type":"v1","addr":"172.21.15.117:6837","nonce":3353001119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14388,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6839/405782304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":405782304},{"type":"v1","addr":"172.21.15.117:6839","nonce":405782304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-03T03:19:44.629822+0000","modified":"2024-06-03T03:25:11.576498+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24517,"mds_1":24512,"mds_2":24503,"mds_3":24499,"mds_4":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24431":{"gid":24431,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6833/3355662416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3355662416},{"type":"v1","addr":"172.21.15.151:6833","nonce":3355662416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/2429725085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2429725085},{"type":"v1","addr":"172.21.15.183:6833","nonce":2429725085}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/31153579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":31153579},{"type":"v1","addr":"172.21.15.151:6835","nonce":31153579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"j","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/2884460088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2884460088},{"type":"v1","addr":"172.21.15.117:6835","nonce":2884460088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/2392407848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":2392407848},{"type":"v1","addr":"172.21.15.151:6837","nonce":2392407848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":32,"addr":"172.21.15.183:6835/1009004320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1009004320},{"type":"v1","addr":"172.21.15.183:6835","nonce":1009004320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.183:6837/2697518526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":2697518526},{"type":"v1","addr":"172.21.15.183:6837","nonce":2697518526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"a","rank":2,"incarnation":22,"state":"up:active","state_seq":27,"addr":"172.21.15.117:6841/1212242967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6840","nonce":1212242967},{"type":"v1","addr":"172.21.15.117:6841","nonce":1212242967}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.151:6839/3623444616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":3623444616},{"type":"v1","addr":"172.21.15.151:6839","nonce":3623444616}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":10,"addr":"172.21.15.183:6839/1439615083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1439615083},{"type":"v1","addr":"172.21.15.183:6839","nonce":1439615083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24517,"qdb_cluster":[24499,24487,24512,24503,24517]},"id":1}]} 2024-06-03T03:25:23.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:23.748+0000 7fefd0200700 1 -- 172.21.15.117:0/2210926067 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fefb00a2d30 msgr2=0x7fefb00a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:23.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:23.748+0000 7fefd0200700 1 --2- 172.21.15.117:0/2210926067 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fefb00a2d30 0x7fefb00a51f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fefbc011550 tx=0x7fefbc009260 comp rx=0 tx=0).stop 2024-06-03T03:25:23.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:23.748+0000 7fefd0200700 1 -- 172.21.15.117:0/2210926067 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefc8105070 msgr2=0x7fefc819b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:23.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:23.748+0000 7fefd0200700 1 --2- 172.21.15.117:0/2210926067 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefc8105070 0x7fefc819b3c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fefb400e9f0 tx=0x7fefb400ed00 comp rx=0 tx=0).stop 2024-06-03T03:25:23.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:23.749+0000 7fefd0200700 1 -- 172.21.15.117:0/2210926067 shutdown_connections 2024-06-03T03:25:23.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:23.749+0000 7fefd0200700 1 --2- 172.21.15.117:0/2210926067 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fefb00a2d30 0x7fefb00a51f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:23.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:23.749+0000 7fefd0200700 1 --2- 172.21.15.117:0/2210926067 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fefc810a6b0 0x7fefc819f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:23.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:23.749+0000 7fefd0200700 1 --2- 172.21.15.117:0/2210926067 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fefc81046b0 0x7fefc819ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:23.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:23.749+0000 7fefd0200700 1 --2- 172.21.15.117:0/2210926067 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefc8105070 0x7fefc819b3c0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:23.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:23.749+0000 7fefd0200700 1 -- 172.21.15.117:0/2210926067 >> 172.21.15.117:0/2210926067 conn(0x7fefc81001a0 msgr2=0x7fefc8100720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:25:23.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:23.749+0000 7fefd0200700 1 -- 172.21.15.117:0/2210926067 shutdown_connections 2024-06-03T03:25:23.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:23.749+0000 7fefd0200700 1 -- 172.21.15.117:0/2210926067 wait complete. 2024-06-03T03:25:23.751 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 31 2024-06-03T03:25:24.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:25:24 smithi151 ceph-mon[37043]: pgmap v508: 129 pgs: 129 active+clean; 660 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-06-03T03:25:24.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:25:24 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/2210926067' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T03:25:24.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:25:24 smithi117 ceph-mon[31649]: pgmap v508: 129 pgs: 129 active+clean; 660 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-06-03T03:25:24.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:25:24 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/2210926067' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T03:25:24.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:25:24 smithi183 ceph-mon[37326]: pgmap v508: 129 pgs: 129 active+clean; 660 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-06-03T03:25:24.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:25:24 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/2210926067' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T03:25:24.736 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-03T03:25:25.205 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:25:26.354 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:25:26 smithi117 ceph-mon[31649]: pgmap v509: 129 pgs: 129 active+clean; 660 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-06-03T03:25:26.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:25:26 smithi151 ceph-mon[37043]: pgmap v509: 129 pgs: 129 active+clean; 660 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-06-03T03:25:26.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:25:26 smithi183 ceph-mon[37326]: pgmap v509: 129 pgs: 129 active+clean; 660 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-06-03T03:25:27.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:27.801+0000 7f2d9d000700 1 -- 172.21.15.117:0/4126365446 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d981030f0 msgr2=0x7f2d98107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:27.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:27.801+0000 7f2d9d000700 1 --2- 172.21.15.117:0/4126365446 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d981030f0 0x7f2d98107110 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2d840099c0 tx=0x7f2d84009cd0 comp rx=0 tx=0).stop 2024-06-03T03:25:27.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:27.802+0000 7f2d9d000700 1 -- 172.21.15.117:0/4126365446 shutdown_connections 2024-06-03T03:25:27.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:27.802+0000 7f2d9d000700 1 --2- 172.21.15.117:0/4126365446 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2d98107840 0x7f2d9810f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:27.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:27.802+0000 7f2d9d000700 1 --2- 172.21.15.117:0/4126365446 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d98102740 0x7f2d98102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:27.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:27.802+0000 7f2d9d000700 1 --2- 172.21.15.117:0/4126365446 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d981030f0 0x7f2d98107110 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:27.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:27.802+0000 7f2d9d000700 1 -- 172.21.15.117:0/4126365446 >> 172.21.15.117:0/4126365446 conn(0x7f2d980795f0 msgr2=0x7f2d98079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:25:27.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:27.802+0000 7f2d9d000700 1 -- 172.21.15.117:0/4126365446 shutdown_connections 2024-06-03T03:25:27.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:27.802+0000 7f2d9d000700 1 -- 172.21.15.117:0/4126365446 wait complete. 2024-06-03T03:25:27.804 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:27.803+0000 7f2d9d000700 1 Processor -- start 2024-06-03T03:25:27.805 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:27.803+0000 7f2d9d000700 1 -- start start 2024-06-03T03:25:27.805 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:27.804+0000 7f2d9d000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d98102740 0x7f2d9819add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:27.805 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:27.804+0000 7f2d9d000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d981030f0 0x7f2d9819b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:27.805 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:27.804+0000 7f2d9d000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2d98107840 0x7f2d9819f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:27.805 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:27.804+0000 7f2d9d000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2d981127c0 con 0x7f2d98102740 2024-06-03T03:25:27.805 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:27.804+0000 7f2d9d000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2d98112640 con 0x7f2d981030f0 2024-06-03T03:25:27.805 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:27.804+0000 7f2d9d000700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2d98112940 con 0x7f2d98107840 2024-06-03T03:25:27.805 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:27.804+0000 7f2d95e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d98102740 0x7f2d9819add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:27.805 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:27.804+0000 7f2d95e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d98102740 0x7f2d9819add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:49848/0 (socket says 172.21.15.117:49848) 2024-06-03T03:25:27.805 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:27.804+0000 7f2d95e00700 1 -- 172.21.15.117:0/4201155840 learned_addr learned my addr 172.21.15.117:0/4201155840 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:25:27.805 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:27.804+0000 7f2d95400700 1 --2- 172.21.15.117:0/4201155840 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d981030f0 0x7f2d9819b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:27.805 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:27.804+0000 7f2d96800700 1 --2- 172.21.15.117:0/4201155840 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2d98107840 0x7f2d9819f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:27.806 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:27.805+0000 7f2d95e00700 1 -- 172.21.15.117:0/4201155840 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2d98107840 msgr2=0x7f2d9819f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:27.806 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:27.805+0000 7f2d95e00700 1 --2- 172.21.15.117:0/4201155840 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2d98107840 0x7f2d9819f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:27.806 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:27.805+0000 7f2d95e00700 1 -- 172.21.15.117:0/4201155840 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d981030f0 msgr2=0x7f2d9819b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:27.806 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:27.805+0000 7f2d95e00700 1 --2- 172.21.15.117:0/4201155840 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d981030f0 0x7f2d9819b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:27.807 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:27.805+0000 7f2d95e00700 1 -- 172.21.15.117:0/4201155840 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d9819fde0 con 0x7f2d98102740 2024-06-03T03:25:27.807 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:27.805+0000 7f2d95400700 1 --2- 172.21.15.117:0/4201155840 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d981030f0 0x7f2d9819b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:25:27.807 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:27.805+0000 7f2d95e00700 1 --2- 172.21.15.117:0/4201155840 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d98102740 0x7f2d9819add0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2d8800eb30 tx=0x7f2d8800ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:25:27.807 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:27.805+0000 7f2d8ea00700 1 -- 172.21.15.117:0/4201155840 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d8800cbd0 con 0x7f2d98102740 2024-06-03T03:25:27.807 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:27.806+0000 7f2d9d000700 1 -- 172.21.15.117:0/4201155840 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d981a00d0 con 0x7f2d98102740 2024-06-03T03:25:27.807 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:27.806+0000 7f2d8ea00700 1 -- 172.21.15.117:0/4201155840 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d88004500 con 0x7f2d98102740 2024-06-03T03:25:27.807 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:27.806+0000 7f2d8ea00700 1 -- 172.21.15.117:0/4201155840 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d88010430 con 0x7f2d98102740 2024-06-03T03:25:27.807 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:27.806+0000 7f2d9d000700 1 -- 172.21.15.117:0/4201155840 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d981a7950 con 0x7f2d98102740 2024-06-03T03:25:27.807 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:27.807+0000 7f2d9d000700 1 -- 172.21.15.117:0/4201155840 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d98069180 con 0x7f2d98102740 2024-06-03T03:25:27.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:27.809+0000 7f2d8ea00700 1 -- 172.21.15.117:0/4201155840 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f2d88003f00 con 0x7f2d98102740 2024-06-03T03:25:27.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:27.810+0000 7f2d8ea00700 1 --2- 172.21.15.117:0/4201155840 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f2d7c0a2cb0 0x7f2d7c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:27.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:27.810+0000 7f2d8ea00700 1 -- 172.21.15.117:0/4201155840 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f2d880cf920 con 0x7f2d98102740 2024-06-03T03:25:27.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:27.810+0000 7f2d95400700 1 --2- 172.21.15.117:0/4201155840 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f2d7c0a2cb0 0x7f2d7c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:27.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:27.811+0000 7f2d95400700 1 --2- 172.21.15.117:0/4201155840 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f2d7c0a2cb0 0x7f2d7c0a5170 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2d9810f770 tx=0x7f2d84005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:25:27.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:27.815+0000 7f2d8ea00700 1 -- 172.21.15.117:0/4201155840 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2d8809c330 con 0x7f2d98102740 2024-06-03T03:25:28.041 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:28.040+0000 7f2d9d000700 1 -- 172.21.15.117:0/4201155840 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2d9819c130 con 0x7f2d98102740 2024-06-03T03:25:28.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:28.050+0000 7f2d8ea00700 1 -- 172.21.15.117:0/4201155840 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+24807 (secure 0 0 0) 0x7f2d8809bc30 con 0x7f2d98102740 2024-06-03T03:25:28.052 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T03:25:28.052 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":99,"fsid":"a0e8473a-2156-11ef-bc9b-c7b262605968","created":"2024-06-03T03:10:08.446859+0000","modified":"2024-06-03T03:21:25.966405+0000","last_up_change":"2024-06-03T03:18:28.736904+0000","last_in_change":"2024-06-03T03:18:08.485711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-03T03:14:09.370986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-03T03:19:35.044683+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-03T03:19:40.097569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-03T03:19:55.394480+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8fefa089-9401-4a1c-b9fb-fe30c63e6a1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":4286644492},{"type":"v1","addr":"172.21.15.117:6803","nonce":4286644492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":4286644492},{"type":"v1","addr":"172.21.15.117:6805","nonce":4286644492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":4286644492},{"type":"v1","addr":"172.21.15.117:6809","nonce":4286644492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":4286644492},{"type":"v1","addr":"172.21.15.117:6807","nonce":4286644492}]},"public_addr":"172.21.15.117:6803/4286644492","cluster_addr":"172.21.15.117:6805/4286644492","heartbeat_back_addr":"172.21.15.117:6809/4286644492","heartbeat_front_addr":"172.21.15.117:6807/4286644492","state":["exists","up"]},{"osd":1,"uuid":"5b12b371-f155-4b67-a470-25cbe59a2999","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":262491158},{"type":"v1","addr":"172.21.15.151:6801","nonce":262491158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":262491158},{"type":"v1","addr":"172.21.15.151:6803","nonce":262491158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":262491158},{"type":"v1","addr":"172.21.15.151:6807","nonce":262491158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":262491158},{"type":"v1","addr":"172.21.15.151:6805","nonce":262491158}]},"public_addr":"172.21.15.151:6801/262491158","cluster_addr":"172.21.15.151:6803/262491158","heartbeat_back_addr":"172.21.15.151:6807/262491158","heartbeat_front_addr":"172.21.15.151:6805/262491158","state":["exists","up"]},{"osd":2,"uuid":"dbc17818-c54c-4ecf-b656-03578b95d8c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":3542453623},{"type":"v1","addr":"172.21.15.183:6801","nonce":3542453623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":3542453623},{"type":"v1","addr":"172.21.15.183:6803","nonce":3542453623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":3542453623},{"type":"v1","addr":"172.21.15.183:6807","nonce":3542453623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":3542453623},{"type":"v1","addr":"172.21.15.183:6805","nonce":3542453623}]},"public_addr":"172.21.15.183:6801/3542453623","cluster_addr":"172.21.15.183:6803/3542453623","heartbeat_back_addr":"172.21.15.183:6807/3542453623","heartbeat_front_addr":"172.21.15.183:6805/3542453623","state":["exists","up"]},{"osd":3,"uuid":"ca08d2ac-b1d6-4a49-9ae8-e3ec146899a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":589012315},{"type":"v1","addr":"172.21.15.117:6811","nonce":589012315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":589012315},{"type":"v1","addr":"172.21.15.117:6813","nonce":589012315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":589012315},{"type":"v1","addr":"172.21.15.117:6817","nonce":589012315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":589012315},{"type":"v1","addr":"172.21.15.117:6815","nonce":589012315}]},"public_addr":"172.21.15.117:6811/589012315","cluster_addr":"172.21.15.117:6813/589012315","heartbeat_back_addr":"172.21.15.117:6817/589012315","heartbeat_front_addr":"172.21.15.117:6815/589012315","state":["exists","up"]},{"osd":4,"uuid":"810d6c7a-023f-4930-82af-bc2bbfd647da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":1239048647},{"type":"v1","addr":"172.21.15.151:6809","nonce":1239048647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":1239048647},{"type":"v1","addr":"172.21.15.151:6811","nonce":1239048647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":1239048647},{"type":"v1","addr":"172.21.15.151:6815","nonce":1239048647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":1239048647},{"type":"v1","addr":"172.21.15.151:6813","nonce":1239048647}]},"public_addr":"172.21.15.151:6809/1239048647","cluster_addr":"172.21.15.151:6811/1239048647","heartbeat_back_addr":"172.21.15.151:6815/1239048647","heartbeat_front_addr":"172.21.15.151:6813/1239048647","state":["exists","up"]},{"osd":5,"uuid":"8ab6a5f0-34bb-48d8-9471-a7dc836e486e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":2985874686},{"type":"v1","addr":"172.21.15.183:6809","nonce":2985874686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":2985874686},{"type":"v1","addr":"172.21.15.183:6811","nonce":2985874686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":2985874686},{"type":"v1","addr":"172.21.15.183:6815","nonce":2985874686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":2985874686},{"type":"v1","addr":"172.21.15.183:6813","nonce":2985874686}]},"public_addr":"172.21.15.183:6809/2985874686","cluster_addr":"172.21.15.183:6811/2985874686","heartbeat_back_addr":"172.21.15.183:6815/2985874686","heartbeat_front_addr":"172.21.15.183:6813/2985874686","state":["exists","up"]},{"osd":6,"uuid":"78d15739-e9e0-4f19-ae5f-ee8527a5b527","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4250583193},{"type":"v1","addr":"172.21.15.117:6819","nonce":4250583193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":4250583193},{"type":"v1","addr":"172.21.15.117:6821","nonce":4250583193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":4250583193},{"type":"v1","addr":"172.21.15.117:6825","nonce":4250583193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4250583193},{"type":"v1","addr":"172.21.15.117:6823","nonce":4250583193}]},"public_addr":"172.21.15.117:6819/4250583193","cluster_addr":"172.21.15.117:6821/4250583193","heartbeat_back_addr":"172.21.15.117:6825/4250583193","heartbeat_front_addr":"172.21.15.117:6823/4250583193","state":["exists","up"]},{"osd":7,"uuid":"3724fe1f-f4d7-443b-aea8-ff3ab02bfdc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":386094645},{"type":"v1","addr":"172.21.15.151:6817","nonce":386094645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":386094645},{"type":"v1","addr":"172.21.15.151:6819","nonce":386094645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":386094645},{"type":"v1","addr":"172.21.15.151:6823","nonce":386094645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":386094645},{"type":"v1","addr":"172.21.15.151:6821","nonce":386094645}]},"public_addr":"172.21.15.151:6817/386094645","cluster_addr":"172.21.15.151:6819/386094645","heartbeat_back_addr":"172.21.15.151:6823/386094645","heartbeat_front_addr":"172.21.15.151:6821/386094645","state":["exists","up"]},{"osd":8,"uuid":"006ac092-f015-448e-a0c0-f7b0c4597a18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1829748222},{"type":"v1","addr":"172.21.15.183:6817","nonce":1829748222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1829748222},{"type":"v1","addr":"172.21.15.183:6819","nonce":1829748222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1829748222},{"type":"v1","addr":"172.21.15.183:6823","nonce":1829748222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1829748222},{"type":"v1","addr":"172.21.15.183:6821","nonce":1829748222}]},"public_addr":"172.21.15.183:6817/1829748222","cluster_addr":"172.21.15.183:6819/1829748222","heartbeat_back_addr":"172.21.15.183:6823/1829748222","heartbeat_front_addr":"172.21.15.183:6821/1829748222","state":["exists","up"]},{"osd":9,"uuid":"00a970e6-2502-459e-975c-badac93c9bd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2538315141},{"type":"v1","addr":"172.21.15.117:6827","nonce":2538315141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2538315141},{"type":"v1","addr":"172.21.15.117:6829","nonce":2538315141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2538315141},{"type":"v1","addr":"172.21.15.117:6833","nonce":2538315141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2538315141},{"type":"v1","addr":"172.21.15.117:6831","nonce":2538315141}]},"public_addr":"172.21.15.117:6827/2538315141","cluster_addr":"172.21.15.117:6829/2538315141","heartbeat_back_addr":"172.21.15.117:6833/2538315141","heartbeat_front_addr":"172.21.15.117:6831/2538315141","state":["exists","up"]},{"osd":10,"uuid":"ba2333cd-32a3-41ef-9ea5-287c707f512b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":3801045133},{"type":"v1","addr":"172.21.15.151:6825","nonce":3801045133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3801045133},{"type":"v1","addr":"172.21.15.151:6827","nonce":3801045133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3801045133},{"type":"v1","addr":"172.21.15.151:6831","nonce":3801045133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3801045133},{"type":"v1","addr":"172.21.15.151:6829","nonce":3801045133}]},"public_addr":"172.21.15.151:6825/3801045133","cluster_addr":"172.21.15.151:6827/3801045133","heartbeat_back_addr":"172.21.15.151:6831/3801045133","heartbeat_front_addr":"172.21.15.151:6829/3801045133","state":["exists","up"]},{"osd":11,"uuid":"1556d654-e6fa-42c1-b399-aa53f6837a1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3513899696},{"type":"v1","addr":"172.21.15.183:6825","nonce":3513899696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3513899696},{"type":"v1","addr":"172.21.15.183:6827","nonce":3513899696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3513899696},{"type":"v1","addr":"172.21.15.183:6831","nonce":3513899696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3513899696},{"type":"v1","addr":"172.21.15.183:6829","nonce":3513899696}]},"public_addr":"172.21.15.183:6825/3513899696","cluster_addr":"172.21.15.183:6827/3513899696","heartbeat_back_addr":"172.21.15.183:6831/3513899696","heartbeat_front_addr":"172.21.15.183:6829/3513899696","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:13:17.174041+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:13:42.237001+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:14:06.273315+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:14:39.912564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:15:06.063202+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:15:30.424818+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:16:05.623554+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:16:32.926393+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:16:58.165899+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:17:32.897477+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:18:00.720866+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:18:26.474441+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.117:0/186309008":"2024-06-04T03:11:24.655516+0000","172.21.15.117:6801/2142313590":"2024-06-04T03:11:24.655516+0000","172.21.15.117:6800/2142313590":"2024-06-04T03:11:24.655516+0000","172.21.15.117:0/2399725156":"2024-06-04T03:11:24.655516+0000","172.21.15.117:0/1453232061":"2024-06-04T03:10:50.552899+0000","172.21.15.117:0/4281674974":"2024-06-04T03:10:50.552899+0000","172.21.15.117:6800/2424554985":"2024-06-04T03:10:50.552899+0000","172.21.15.117:0/3946365225":"2024-06-04T03:11:24.655516+0000","172.21.15.117:0/1146162354":"2024-06-04T03:10:50.552899+0000","172.21.15.117:6801/2424554985":"2024-06-04T03:10:50.552899+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-03T03:25:28.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:28.053+0000 7f2d9d000700 1 -- 172.21.15.117:0/4201155840 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f2d7c0a2cb0 msgr2=0x7f2d7c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:28.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:28.053+0000 7f2d9d000700 1 --2- 172.21.15.117:0/4201155840 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f2d7c0a2cb0 0x7f2d7c0a5170 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2d9810f770 tx=0x7f2d84005fb0 comp rx=0 tx=0).stop 2024-06-03T03:25:28.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:28.053+0000 7f2d9d000700 1 -- 172.21.15.117:0/4201155840 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d98102740 msgr2=0x7f2d9819add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:28.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:28.053+0000 7f2d9d000700 1 --2- 172.21.15.117:0/4201155840 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d98102740 0x7f2d9819add0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2d8800eb30 tx=0x7f2d8800ee40 comp rx=0 tx=0).stop 2024-06-03T03:25:28.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:28.054+0000 7f2d9d000700 1 -- 172.21.15.117:0/4201155840 shutdown_connections 2024-06-03T03:25:28.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:28.054+0000 7f2d9d000700 1 --2- 172.21.15.117:0/4201155840 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f2d7c0a2cb0 0x7f2d7c0a5170 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:28.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:28.054+0000 7f2d9d000700 1 --2- 172.21.15.117:0/4201155840 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2d98107840 0x7f2d9819f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:28.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:28.054+0000 7f2d9d000700 1 --2- 172.21.15.117:0/4201155840 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d98102740 0x7f2d9819add0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:28.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:28.054+0000 7f2d9d000700 1 --2- 172.21.15.117:0/4201155840 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d981030f0 0x7f2d9819b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:28.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:28.054+0000 7f2d9d000700 1 -- 172.21.15.117:0/4201155840 >> 172.21.15.117:0/4201155840 conn(0x7f2d980795f0 msgr2=0x7f2d98101c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:25:28.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:28.054+0000 7f2d9d000700 1 -- 172.21.15.117:0/4201155840 shutdown_connections 2024-06-03T03:25:28.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:28.054+0000 7f2d9d000700 1 -- 172.21.15.117:0/4201155840 wait complete. 2024-06-03T03:25:28.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:25:28 smithi151 ceph-mon[37043]: pgmap v510: 129 pgs: 129 active+clean; 660 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-06-03T03:25:28.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:25:28 smithi117 ceph-mon[31649]: pgmap v510: 129 pgs: 129 active+clean; 660 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-06-03T03:25:28.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:25:28 smithi183 ceph-mon[37326]: pgmap v510: 129 pgs: 129 active+clean; 660 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-06-03T03:25:29.099 DEBUG:tasks.fwd_scrub:Yielding 2024-06-03T03:25:29.100 INFO:teuthology.run_tasks:Running task background_exec... 2024-06-03T03:25:29.111 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi117.front.sepia.ceph.com 2024-06-03T03:25:29.111 DEBUG:teuthology.orchestra.run.smithi117:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-06-03T03:25:29.111 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-06-03T03:25:29.111 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-03T03:25:29.111 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-03T03:25:29.119 INFO:teuthology.run_tasks:Running task workunit... 2024-06-03T03:25:29.144 INFO:tasks.workunit:Pulling workunits from ref dfcc4b3c2c55532c4c04fa47551c9df1dffdf746 2024-06-03T03:25:29.144 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-06-03T03:25:29.144 INFO:tasks.workunit:timeout=3h 2024-06-03T03:25:29.145 INFO:tasks.workunit:cleanup=True 2024-06-03T03:25:29.145 DEBUG:teuthology.orchestra.run.smithi117:> stat -- /home/ubuntu/cephtest/mnt.0 2024-06-03T03:25:29.191 INFO:teuthology.orchestra.run.smithi117.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-06-03T03:25:29.192 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-06-03T03:25:29.192 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 77h/119d Inode: 1099511628283 Links: 2 2024-06-03T03:25:29.192 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-03T03:25:29.192 INFO:teuthology.orchestra.run.smithi117.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-06-03T03:25:29.192 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2024-06-03 03:21:29.251234825 +0000 2024-06-03T03:25:29.192 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-06-03 03:21:29.251234825 +0000 2024-06-03T03:25:29.192 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-06-03 03:25:10.525451236 +0000 2024-06-03T03:25:29.192 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: 2024-06-03 03:21:29.251234825 +0000 2024-06-03T03:25:29.192 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-06-03T03:25:29.192 DEBUG:teuthology.orchestra.run.smithi117:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-06-03T03:25:29.286 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:25:29 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/4201155840' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T03:25:29.305 DEBUG:teuthology.orchestra.run.smithi117:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout dfcc4b3c2c55532c4c04fa47551c9df1dffdf746 2024-06-03T03:25:29.354 INFO:tasks.workunit.client.0.smithi117.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-06-03T03:25:29.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:25:29 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/4201155840' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T03:25:29.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:25:29 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/4201155840' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T03:25:29.536 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:25:30.212 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:25:30 smithi117 ceph-mon[31649]: pgmap v511: 129 pgs: 129 active+clean; 660 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1023 B/s wr, 12 op/s 2024-06-03T03:25:30.307 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:25:30 smithi151 ceph-mon[37043]: pgmap v511: 129 pgs: 129 active+clean; 660 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1023 B/s wr, 12 op/s 2024-06-03T03:25:30.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:25:30 smithi183 ceph-mon[37326]: pgmap v511: 129 pgs: 129 active+clean; 660 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1023 B/s wr, 12 op/s 2024-06-03T03:25:31.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:25:31 smithi151 ceph-mon[37043]: pgmap v512: 129 pgs: 129 active+clean; 660 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 341 B/s wr, 9 op/s 2024-06-03T03:25:31.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:25:31 smithi117 ceph-mon[31649]: pgmap v512: 129 pgs: 129 active+clean; 660 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 341 B/s wr, 9 op/s 2024-06-03T03:25:31.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:25:31 smithi183 ceph-mon[37326]: pgmap v512: 129 pgs: 129 active+clean; 660 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 341 B/s wr, 9 op/s 2024-06-03T03:25:32.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.057+0000 7f3141a00700 1 -- 172.21.15.117:0/251020754 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f313c10a670 msgr2=0x7f313c10aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:32.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.057+0000 7f3141a00700 1 --2- 172.21.15.117:0/251020754 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f313c10a670 0x7f313c10aa50 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f312c0099c0 tx=0x7f312c009cd0 comp rx=0 tx=0).stop 2024-06-03T03:25:32.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.057+0000 7f3141a00700 1 -- 172.21.15.117:0/251020754 shutdown_connections 2024-06-03T03:25:32.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.057+0000 7f3141a00700 1 --2- 172.21.15.117:0/251020754 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f313c104f80 0x7f313c111440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:32.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.057+0000 7f3141a00700 1 --2- 172.21.15.117:0/251020754 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f313c10a670 0x7f313c10aa50 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:32.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.057+0000 7f3141a00700 1 --2- 172.21.15.117:0/251020754 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f313c1045c0 0x7f313c104a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:32.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.057+0000 7f3141a00700 1 -- 172.21.15.117:0/251020754 >> 172.21.15.117:0/251020754 conn(0x7f313c1000d0 msgr2=0x7f313c1024f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:25:32.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.057+0000 7f3141a00700 1 -- 172.21.15.117:0/251020754 shutdown_connections 2024-06-03T03:25:32.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.057+0000 7f3141a00700 1 -- 172.21.15.117:0/251020754 wait complete. 2024-06-03T03:25:32.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.059+0000 7f3141a00700 1 Processor -- start 2024-06-03T03:25:32.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.059+0000 7f3141a00700 1 -- start start 2024-06-03T03:25:32.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.059+0000 7f3141a00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f313c1045c0 0x7f313c1111d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:32.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.059+0000 7f3141a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f313c104f80 0x7f313c10e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:32.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.059+0000 7f3141a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f313c10a670 0x7f313c10e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:32.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.059+0000 7f3141a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f313c1148c0 con 0x7f313c10a670 2024-06-03T03:25:32.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.059+0000 7f3141a00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f313c114740 con 0x7f313c104f80 2024-06-03T03:25:32.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.059+0000 7f3141a00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f313c114a40 con 0x7f313c1045c0 2024-06-03T03:25:32.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.060+0000 7f313be00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f313c1045c0 0x7f313c1111d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:32.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.060+0000 7f313b400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f313c104f80 0x7f313c10e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:32.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.060+0000 7f313be00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f313c1045c0 0x7f313c1111d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:44770/0 (socket says 172.21.15.117:44770) 2024-06-03T03:25:32.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.060+0000 7f313be00700 1 -- 172.21.15.117:0/2036254423 learned_addr learned my addr 172.21.15.117:0/2036254423 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:25:32.062 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.060+0000 7f3141000700 1 --2- 172.21.15.117:0/2036254423 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f313c10a670 0x7f313c10e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:32.062 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.060+0000 7f313be00700 1 -- 172.21.15.117:0/2036254423 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f313c104f80 msgr2=0x7f313c10e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:32.062 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.060+0000 7f313be00700 1 --2- 172.21.15.117:0/2036254423 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f313c104f80 0x7f313c10e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:32.062 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.060+0000 7f313be00700 1 -- 172.21.15.117:0/2036254423 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f313c10a670 msgr2=0x7f313c10e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:32.062 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.060+0000 7f313be00700 1 --2- 172.21.15.117:0/2036254423 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f313c10a670 0x7f313c10e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:32.062 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.060+0000 7f313be00700 1 -- 172.21.15.117:0/2036254423 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f313c10f080 con 0x7f313c1045c0 2024-06-03T03:25:32.062 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.061+0000 7f313be00700 1 --2- 172.21.15.117:0/2036254423 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f313c1045c0 0x7f313c1111d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f312c006010 tx=0x7f312c003a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:25:32.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.061+0000 7f3138c00700 1 -- 172.21.15.117:0/2036254423 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f312c003680 con 0x7f313c1045c0 2024-06-03T03:25:32.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.061+0000 7f3138c00700 1 -- 172.21.15.117:0/2036254423 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f312c003e50 con 0x7f313c1045c0 2024-06-03T03:25:32.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.061+0000 7f3141a00700 1 -- 172.21.15.117:0/2036254423 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f313c1ab5e0 con 0x7f313c1045c0 2024-06-03T03:25:32.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.061+0000 7f3138c00700 1 -- 172.21.15.117:0/2036254423 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f312c00fe10 con 0x7f313c1045c0 2024-06-03T03:25:32.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.061+0000 7f3141a00700 1 -- 172.21.15.117:0/2036254423 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f313c1abae0 con 0x7f313c1045c0 2024-06-03T03:25:32.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.064+0000 7f3138c00700 1 -- 172.21.15.117:0/2036254423 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f312c00f460 con 0x7f313c1045c0 2024-06-03T03:25:32.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.066+0000 7f314d915b80 1 Processor -- start 2024-06-03T03:25:32.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.066+0000 7f3138c00700 1 --2- 172.21.15.117:0/2036254423 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f31200a2ed0 0x7f31200a5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:32.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.066+0000 7f314d915b80 1 -- start start 2024-06-03T03:25:32.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.066+0000 7f3138c00700 1 -- 172.21.15.117:0/2036254423 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f312c0d0190 con 0x7f313c1045c0 2024-06-03T03:25:32.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.066+0000 7f3141000700 1 --2- 172.21.15.117:0/2036254423 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f31200a2ed0 0x7f31200a5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:32.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.066+0000 7f314d915b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ac63baec0 0x555ac64404a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:32.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.066+0000 7f314d915b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555ac64409e0 0x555ac6440e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:32.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.066+0000 7f314d915b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555ac64495f0 0x555ac644b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:32.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.066+0000 7f314d915b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555ac634da90 con 0x555ac63baec0 2024-06-03T03:25:32.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.066+0000 7f314d915b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x555ac6390b80 con 0x555ac64409e0 2024-06-03T03:25:32.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.066+0000 7f314d915b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555ac634b970 con 0x555ac64495f0 2024-06-03T03:25:32.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.067+0000 7f313be00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ac63baec0 0x555ac64404a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:32.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.067+0000 7f3141000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555ac64495f0 0x555ac644b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:32.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.067+0000 7f3141000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555ac64495f0 0x555ac644b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:44784/0 (socket says 172.21.15.117:44784) 2024-06-03T03:25:32.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.067+0000 7f3141000700 1 -- 172.21.15.117:0/16432392 learned_addr learned my addr 172.21.15.117:0/16432392 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:25:32.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.067+0000 7f313be00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ac63baec0 0x555ac64404a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:49876/0 (socket says 172.21.15.117:49876) 2024-06-03T03:25:32.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.067+0000 7f313b400700 1 --2- 172.21.15.117:0/16432392 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555ac64409e0 0x555ac6440e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:32.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.067+0000 7f3141000700 1 -- 172.21.15.117:0/16432392 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555ac64409e0 msgr2=0x555ac6440e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:25:32.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.067+0000 7f3141000700 1 --2- 172.21.15.117:0/16432392 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555ac64409e0 0x555ac6440e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:32.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.067+0000 7f3141000700 1 -- 172.21.15.117:0/16432392 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ac63baec0 msgr2=0x555ac64404a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:25:32.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.067+0000 7f3141000700 1 --2- 172.21.15.117:0/16432392 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ac63baec0 0x555ac64404a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:32.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.067+0000 7f3141000700 1 -- 172.21.15.117:0/16432392 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ac6339010 con 0x555ac64495f0 2024-06-03T03:25:32.074 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.067+0000 7f3141000700 1 --2- 172.21.15.117:0/2036254423 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f31200a2ed0 0x7f31200a5390 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f313400f930 tx=0x7f313400fcf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:25:32.074 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.067+0000 7f313be00700 1 --2- 172.21.15.117:0/16432392 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ac63baec0 0x555ac64404a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:25:32.074 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.068+0000 7f3141000700 1 --2- 172.21.15.117:0/16432392 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555ac64495f0 0x555ac644b9e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3134018640 tx=0x7f313401d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=540050af71627dfe server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:25:32.074 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.068+0000 7f3131600700 1 -- 172.21.15.117:0/16432392 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3134004bb0 con 0x555ac64495f0 2024-06-03T03:25:32.074 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.068+0000 7f3131600700 1 -- 172.21.15.117:0/16432392 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3134004d50 con 0x555ac64495f0 2024-06-03T03:25:32.074 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.068+0000 7f3131600700 1 -- 172.21.15.117:0/16432392 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31340044c0 con 0x555ac64495f0 2024-06-03T03:25:32.074 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.069+0000 7f314d915b80 1 -- 172.21.15.117:0/16432392 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555ac64495f0 msgr2=0x555ac644b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:32.074 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.069+0000 7f314d915b80 1 --2- 172.21.15.117:0/16432392 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555ac64495f0 0x555ac644b9e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3134018640 tx=0x7f313401d000 comp rx=0 tx=0).stop 2024-06-03T03:25:32.074 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.069+0000 7f314d915b80 1 -- 172.21.15.117:0/16432392 shutdown_connections 2024-06-03T03:25:32.074 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.069+0000 7f314d915b80 1 --2- 172.21.15.117:0/16432392 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555ac64495f0 0x555ac644b9e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:32.074 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.069+0000 7f314d915b80 1 --2- 172.21.15.117:0/16432392 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ac63baec0 0x555ac64404a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:32.074 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.069+0000 7f314d915b80 1 --2- 172.21.15.117:0/16432392 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555ac64409e0 0x555ac6440e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:32.074 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.069+0000 7f314d915b80 1 -- 172.21.15.117:0/16432392 >> 172.21.15.117:0/16432392 conn(0x555ac636b1f0 msgr2=0x555ac643a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:25:32.075 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.069+0000 7f314d915b80 1 -- 172.21.15.117:0/16432392 shutdown_connections 2024-06-03T03:25:32.075 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.069+0000 7f314d915b80 1 -- 172.21.15.117:0/16432392 wait complete. 2024-06-03T03:25:32.075 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.069+0000 7f314d915b80 1 -- start start 2024-06-03T03:25:32.075 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.070+0000 7f314d915b80 1 Processor -- start 2024-06-03T03:25:32.075 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.070+0000 7f314d915b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555ac63baec0 0x555ac6448de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:32.075 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.070+0000 7f314d915b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555ac64409e0 0x555ac64460e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:32.075 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.070+0000 7f314d915b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ac64495f0 0x555ac6446620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:32.075 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.070+0000 7f314d915b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555ac632cfe0 con 0x555ac64495f0 2024-06-03T03:25:32.075 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.070+0000 7f314d915b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x555ac638cb40 con 0x555ac63baec0 2024-06-03T03:25:32.075 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.070+0000 7f314d915b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555ac644c1a0 con 0x555ac64409e0 2024-06-03T03:25:32.075 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.070+0000 7f314d915b80 10 client.0 _ll_register_callbacks cb 0x555ac632a7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:25:32.075 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.071+0000 7f313b400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555ac64409e0 0x555ac64460e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:32.075 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.071+0000 7f313be00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555ac63baec0 0x555ac6448de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:32.075 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.071+0000 7f313b400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555ac64409e0 0x555ac64460e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:44788/0 (socket says 172.21.15.117:44788) 2024-06-03T03:25:32.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.071+0000 7f313b400700 1 -- 172.21.15.117:0/2257321409 learned_addr learned my addr 172.21.15.117:0/2257321409 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:25:32.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.071+0000 7f3141000700 1 --2- 172.21.15.117:0/2257321409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ac64495f0 0x555ac6446620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:32.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.071+0000 7f313b400700 1 -- 172.21.15.117:0/2257321409 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555ac63baec0 msgr2=0x555ac6448de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:25:32.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.071+0000 7f313b400700 1 --2- 172.21.15.117:0/2257321409 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555ac63baec0 0x555ac6448de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:32.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.071+0000 7f313b400700 1 -- 172.21.15.117:0/2257321409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ac64495f0 msgr2=0x555ac6446620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:25:32.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.071+0000 7f313b400700 1 --2- 172.21.15.117:0/2257321409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ac64495f0 0x555ac6446620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:32.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.071+0000 7f313b400700 1 -- 172.21.15.117:0/2257321409 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ac62df950 con 0x555ac64409e0 2024-06-03T03:25:32.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.071+0000 7f313be00700 1 --2- 172.21.15.117:0/2257321409 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555ac63baec0 0x555ac6448de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:25:32.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.071+0000 7f313b400700 1 --2- 172.21.15.117:0/2257321409 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555ac64409e0 0x555ac64460e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3128012770 tx=0x7f3128012a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54046a160b914923 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:25:32.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.071+0000 7f3126000700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-06-03T03:25:32.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.072+0000 7f3126000700 1 -- 172.21.15.117:0/2257321409 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f312800f540 con 0x555ac64409e0 2024-06-03T03:25:32.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.072+0000 7f314d915b80 1 -- 172.21.15.117:0/2257321409 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3134004d50 con 0x555ac64409e0 2024-06-03T03:25:32.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.072+0000 7f3126000700 1 -- 172.21.15.117:0/2257321409 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f312800f6e0 con 0x555ac64409e0 2024-06-03T03:25:32.077 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.072+0000 7f3126000700 1 -- 172.21.15.117:0/2257321409 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f312800f8e0 con 0x555ac64409e0 2024-06-03T03:25:32.077 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.072+0000 7f314d915b80 10 client.25309 fetch_fsmap learned FSMap version 31 2024-06-03T03:25:32.077 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.072+0000 7f314d915b80 1 -- 172.21.15.117:0/2257321409 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x555ac6456a00 con 0x555ac64409e0 2024-06-03T03:25:32.077 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.072+0000 7f3126000700 1 -- 172.21.15.117:0/2257321409 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3134004d50 con 0x555ac64409e0 2024-06-03T03:25:32.077 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.073+0000 7f3126000700 1 -- 172.21.15.117:0/2257321409 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f3128010b80 con 0x555ac64409e0 2024-06-03T03:25:32.077 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.073+0000 7f314d915b80 10 client.25309 fetch_fsmap finished waiting for FSMap version 31 2024-06-03T03:25:32.077 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.073+0000 7f314d915b80 10 client.25309 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-03T03:25:32.077 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.073+0000 7f314d915b80 20 client.25309 populate_metadata read hostname 'smithi117' 2024-06-03T03:25:32.077 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.073+0000 7f314d915b80 1 --2- 172.21.15.117:0/2257321409 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x555ac6459040 0x555ac645b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:32.077 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.073+0000 7f314d915b80 4 client.25309 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T03:25:32.077 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.073+0000 7f314d915b80 1 -- 172.21.15.117:0/2257321409 --> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ac639e080 con 0x555ac6459040 2024-06-03T03:25:32.078 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.073+0000 7f313be00700 1 --2- 172.21.15.117:0/2257321409 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x555ac6459040 0x555ac645b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:32.078 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.074+0000 7f313be00700 1 --2- 172.21.15.117:0/2257321409 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x555ac6459040 0x555ac645b430 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39a15f563c6fd2a server_cookie=c915ad6eb8070c31 in_seq=0 out_seq=0 2024-06-03T03:25:32.079 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.074+0000 7f3126000700 10 client.25309 ms_handle_connect on v2:172.21.15.183:6838/1439615083 2024-06-03T03:25:32.079 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.074+0000 7f3126000700 1 -- 172.21.15.117:0/2257321409 <== mds.0 v2:172.21.15.183:6838/1439615083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x555ac639e080 con 0x555ac6459040 2024-06-03T03:25:32.079 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.074+0000 7f3126000700 10 client.25309 handle_command_reply: tid=0 2024-06-03T03:25:32.079 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.075+0000 7f314d915b80 1 client.25309 shutdown 2024-06-03T03:25:32.079 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.075+0000 7f314d915b80 1 -- 172.21.15.117:0/2257321409 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555ac64409e0 msgr2=0x555ac64460e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:32.079 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.075+0000 7f314d915b80 1 --2- 172.21.15.117:0/2257321409 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555ac64409e0 0x555ac64460e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3128012770 tx=0x7f3128012a80 comp rx=0 tx=0).stop 2024-06-03T03:25:32.079 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.075+0000 7f314d915b80 1 -- 172.21.15.117:0/2257321409 shutdown_connections 2024-06-03T03:25:32.079 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.075+0000 7f314d915b80 1 --2- 172.21.15.117:0/2257321409 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x555ac6459040 0x555ac645b430 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:32.079 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.075+0000 7f314d915b80 1 --2- 172.21.15.117:0/2257321409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ac64495f0 0x555ac6446620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:32.079 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.075+0000 7f3126000700 0 client.25309 ms_handle_reset on v2:172.21.15.183:6838/1439615083 2024-06-03T03:25:32.079 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.075+0000 7f314d915b80 1 --2- 172.21.15.117:0/2257321409 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555ac63baec0 0x555ac6448de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:32.079 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.075+0000 7f314d915b80 1 --2- 172.21.15.117:0/2257321409 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555ac64409e0 0x555ac64460e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:32.079 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.075+0000 7f314d915b80 1 -- 172.21.15.117:0/2257321409 >> 172.21.15.117:0/2257321409 conn(0x555ac636b1f0 msgr2=0x555ac643a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:25:32.079 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.075+0000 7f314d915b80 1 -- 172.21.15.117:0/2257321409 shutdown_connections 2024-06-03T03:25:32.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.075+0000 7f314d915b80 1 -- 172.21.15.117:0/2257321409 wait complete. 2024-06-03T03:25:32.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.075+0000 7f314d915b80 20 client.25309 trim_cache size 0 max 16384 2024-06-03T03:25:32.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.098+0000 7f314d915b80 1 Processor -- start 2024-06-03T03:25:32.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.098+0000 7f314d915b80 1 -- start start 2024-06-03T03:25:32.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.099+0000 7f314d915b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555ac63baec0 0x555ac6455310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:32.100 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.099+0000 7f314d915b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555ac64409e0 0x555ac6448920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:32.100 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.099+0000 7f314d915b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ac64495f0 0x555ac6460d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:32.100 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.099+0000 7f314d915b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f313c1b4620 con 0x555ac64495f0 2024-06-03T03:25:32.100 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.099+0000 7f314d915b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f313c1b6160 con 0x555ac64409e0 2024-06-03T03:25:32.100 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.099+0000 7f314d915b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f313c1b4cf0 con 0x555ac63baec0 2024-06-03T03:25:32.100 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.099+0000 7f3141000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ac64495f0 0x555ac6460d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:32.100 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.099+0000 7f313be00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555ac63baec0 0x555ac6455310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:32.100 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.099+0000 7f313b400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555ac64409e0 0x555ac6448920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:32.100 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.099+0000 7f313be00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555ac63baec0 0x555ac6455310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:44798/0 (socket says 172.21.15.117:44798) 2024-06-03T03:25:32.100 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.099+0000 7f3141000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ac64495f0 0x555ac6460d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:49894/0 (socket says 172.21.15.117:49894) 2024-06-03T03:25:32.100 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.099+0000 7f313be00700 1 -- 172.21.15.117:0/2217849769 learned_addr learned my addr 172.21.15.117:0/2217849769 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:25:32.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.099+0000 7f313be00700 1 -- 172.21.15.117:0/2217849769 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555ac64409e0 msgr2=0x555ac6448920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:25:32.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.099+0000 7f313be00700 1 --2- 172.21.15.117:0/2217849769 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555ac64409e0 0x555ac6448920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:32.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.099+0000 7f313be00700 1 -- 172.21.15.117:0/2217849769 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ac64495f0 msgr2=0x555ac6460d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:25:32.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.100+0000 7f313be00700 1 --2- 172.21.15.117:0/2217849769 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ac64495f0 0x555ac6460d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:32.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.100+0000 7f313be00700 1 -- 172.21.15.117:0/2217849769 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ac6444650 con 0x555ac63baec0 2024-06-03T03:25:32.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.100+0000 7f313b400700 1 --2- 172.21.15.117:0/2217849769 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555ac64409e0 0x555ac6448920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:25:32.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.100+0000 7f313be00700 1 --2- 172.21.15.117:0/2217849769 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555ac63baec0 0x555ac6455310 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f312c09c4c0 tx=0x7f312c087040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a59f1cf5cbd2b68a server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:25:32.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.100+0000 7f3130c00700 1 -- 172.21.15.117:0/2217849769 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f312c09c530 con 0x555ac63baec0 2024-06-03T03:25:32.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.100+0000 7f3130c00700 1 -- 172.21.15.117:0/2217849769 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f312c0a08e0 con 0x555ac63baec0 2024-06-03T03:25:32.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.100+0000 7f3130c00700 1 -- 172.21.15.117:0/2217849769 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f312c0b5270 con 0x555ac63baec0 2024-06-03T03:25:32.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.101+0000 7f314d915b80 1 -- 172.21.15.117:0/2217849769 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555ac63baec0 msgr2=0x555ac6455310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:32.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.101+0000 7f314d915b80 1 --2- 172.21.15.117:0/2217849769 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555ac63baec0 0x555ac6455310 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f312c09c4c0 tx=0x7f312c087040 comp rx=0 tx=0).stop 2024-06-03T03:25:32.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.101+0000 7f314d915b80 1 -- 172.21.15.117:0/2217849769 shutdown_connections 2024-06-03T03:25:32.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.101+0000 7f314d915b80 1 --2- 172.21.15.117:0/2217849769 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ac64495f0 0x555ac6460d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:32.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.101+0000 7f314d915b80 1 --2- 172.21.15.117:0/2217849769 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555ac63baec0 0x555ac6455310 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:32.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.101+0000 7f314d915b80 1 --2- 172.21.15.117:0/2217849769 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555ac64409e0 0x555ac6448920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:32.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.101+0000 7f314d915b80 1 -- 172.21.15.117:0/2217849769 >> 172.21.15.117:0/2217849769 conn(0x555ac636b1f0 msgr2=0x555ac6441650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:25:32.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.101+0000 7f314d915b80 1 -- 172.21.15.117:0/2217849769 shutdown_connections 2024-06-03T03:25:32.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.101+0000 7f314d915b80 1 -- 172.21.15.117:0/2217849769 wait complete. 2024-06-03T03:25:32.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.101+0000 7f314d915b80 1 -- start start 2024-06-03T03:25:32.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.102+0000 7f314d915b80 1 Processor -- start 2024-06-03T03:25:32.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.102+0000 7f314d915b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555ac63baec0 0x555ac644d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:32.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.102+0000 7f314d915b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ac64409e0 0x555ac644d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:32.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.102+0000 7f314d915b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555ac64495f0 0x555ac644de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:32.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.102+0000 7f314d915b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f313c1b5a90 con 0x555ac64409e0 2024-06-03T03:25:32.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.102+0000 7f314d915b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f313c1b53c0 con 0x555ac64495f0 2024-06-03T03:25:32.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.102+0000 7f314d915b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f313c1b25c0 con 0x555ac63baec0 2024-06-03T03:25:32.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.102+0000 7f314d915b80 10 client.0 _ll_register_callbacks cb 0x555ac632a7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:25:32.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.102+0000 7f313be00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555ac63baec0 0x555ac644d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:32.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.103+0000 7f313be00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555ac63baec0 0x555ac644d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:44810/0 (socket says 172.21.15.117:44810) 2024-06-03T03:25:32.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.103+0000 7f313be00700 1 -- 172.21.15.117:0/2187446290 learned_addr learned my addr 172.21.15.117:0/2187446290 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:25:32.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.103+0000 7f3141000700 1 --2- 172.21.15.117:0/2187446290 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555ac64495f0 0x555ac644de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:32.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.103+0000 7f313b400700 1 --2- 172.21.15.117:0/2187446290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ac64409e0 0x555ac644d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:32.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.103+0000 7f313be00700 1 -- 172.21.15.117:0/2187446290 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555ac64495f0 msgr2=0x555ac644de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:25:32.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.103+0000 7f313be00700 1 --2- 172.21.15.117:0/2187446290 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555ac64495f0 0x555ac644de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:32.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.103+0000 7f313be00700 1 -- 172.21.15.117:0/2187446290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ac64409e0 msgr2=0x555ac644d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:25:32.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.103+0000 7f313be00700 1 --2- 172.21.15.117:0/2187446290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ac64409e0 0x555ac644d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:32.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.103+0000 7f313be00700 1 -- 172.21.15.117:0/2187446290 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ac64c2ed0 con 0x555ac63baec0 2024-06-03T03:25:32.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.103+0000 7f313be00700 1 --2- 172.21.15.117:0/2187446290 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555ac63baec0 0x555ac644d3b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f312c011040 tx=0x7f312c0095f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f17c6cc679ae0a1 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:25:32.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.103+0000 7f3126000700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-06-03T03:25:32.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.103+0000 7f3126000700 1 -- 172.21.15.117:0/2187446290 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f312c0a5110 con 0x555ac63baec0 2024-06-03T03:25:32.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.104+0000 7f3126000700 1 -- 172.21.15.117:0/2187446290 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f312c09a830 con 0x555ac63baec0 2024-06-03T03:25:32.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.104+0000 7f314d915b80 1 -- 172.21.15.117:0/2187446290 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f312c0a08e0 con 0x555ac63baec0 2024-06-03T03:25:32.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.104+0000 7f3126000700 1 -- 172.21.15.117:0/2187446290 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f312c020370 con 0x555ac63baec0 2024-06-03T03:25:32.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.104+0000 7f314d915b80 10 client.25315 fetch_fsmap learned FSMap version 31 2024-06-03T03:25:32.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.104+0000 7f314d915b80 1 -- 172.21.15.117:0/2187446290 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x555ac64c41d0 con 0x555ac63baec0 2024-06-03T03:25:32.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.104+0000 7f3126000700 1 -- 172.21.15.117:0/2187446290 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f312c0a08e0 con 0x555ac63baec0 2024-06-03T03:25:32.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.104+0000 7f3126000700 1 -- 172.21.15.117:0/2187446290 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f312c0a4c80 con 0x555ac63baec0 2024-06-03T03:25:32.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.104+0000 7f314d915b80 10 client.25315 fetch_fsmap finished waiting for FSMap version 31 2024-06-03T03:25:32.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.104+0000 7f314d915b80 10 client.25315 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-03T03:25:32.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.104+0000 7f314d915b80 20 client.25315 populate_metadata read hostname 'smithi117' 2024-06-03T03:25:32.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.104+0000 7f314d915b80 1 --2- 172.21.15.117:0/2187446290 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x555ac64c47d0 0x555ac64c6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:32.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.104+0000 7f314d915b80 4 client.25315 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-03T03:25:32.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.104+0000 7f314d915b80 1 -- 172.21.15.117:0/2187446290 --> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x555ac6398600 con 0x555ac64c47d0 2024-06-03T03:25:32.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.105+0000 7f313b400700 1 --2- 172.21.15.117:0/2187446290 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x555ac64c47d0 0x555ac64c6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:32.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.105+0000 7f313b400700 1 --2- 172.21.15.117:0/2187446290 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x555ac64c47d0 0x555ac64c6bc0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17e5c3c7fb9a49fa server_cookie=296aef86317b7c84 in_seq=0 out_seq=0 2024-06-03T03:25:32.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.105+0000 7f3126000700 10 client.25315 ms_handle_connect on v2:172.21.15.183:6838/1439615083 2024-06-03T03:25:32.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.106+0000 7f3126000700 1 -- 172.21.15.117:0/2187446290 <== mds.0 v2:172.21.15.183:6838/1439615083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x555ac6398600 con 0x555ac64c47d0 2024-06-03T03:25:32.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.106+0000 7f3126000700 10 client.25315 handle_command_reply: tid=0 2024-06-03T03:25:32.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.106+0000 7f314d915b80 1 client.25315 shutdown 2024-06-03T03:25:32.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.106+0000 7f314d915b80 1 -- 172.21.15.117:0/2187446290 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555ac63baec0 msgr2=0x555ac644d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:32.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.106+0000 7f314d915b80 1 --2- 172.21.15.117:0/2187446290 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555ac63baec0 0x555ac644d3b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f312c011040 tx=0x7f312c0095f0 comp rx=0 tx=0).stop 2024-06-03T03:25:32.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.107+0000 7f314d915b80 1 -- 172.21.15.117:0/2187446290 shutdown_connections 2024-06-03T03:25:32.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.107+0000 7f314d915b80 1 --2- 172.21.15.117:0/2187446290 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x555ac64c47d0 0x555ac64c6bc0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:32.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.107+0000 7f314d915b80 1 --2- 172.21.15.117:0/2187446290 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555ac64495f0 0x555ac644de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:32.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.107+0000 7f3126000700 0 client.25315 ms_handle_reset on v2:172.21.15.183:6838/1439615083 2024-06-03T03:25:32.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.107+0000 7f314d915b80 1 --2- 172.21.15.117:0/2187446290 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555ac63baec0 0x555ac644d3b0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:32.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.107+0000 7f314d915b80 1 --2- 172.21.15.117:0/2187446290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ac64409e0 0x555ac644d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:32.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.107+0000 7f314d915b80 1 -- 172.21.15.117:0/2187446290 >> 172.21.15.117:0/2187446290 conn(0x555ac636b1f0 msgr2=0x555ac6440dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:25:32.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.107+0000 7f314d915b80 1 -- 172.21.15.117:0/2187446290 shutdown_connections 2024-06-03T03:25:32.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.107+0000 7f314d915b80 1 -- 172.21.15.117:0/2187446290 wait complete. 2024-06-03T03:25:32.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.107+0000 7f314d915b80 20 client.25315 trim_cache size 0 max 16384 2024-06-03T03:25:32.110 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-06-03T03:25:32.110 INFO:teuthology.orchestra.run.smithi117.stdout: "return_code": 0, 2024-06-03T03:25:32.110 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_tag": "5abaee35-60ff-4f7e-a25a-04edc9482b99", 2024-06-03T03:25:32.110 INFO:teuthology.orchestra.run.smithi117.stdout: "mode": "asynchronous" 2024-06-03T03:25:32.110 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-06-03T03:25:32.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.108+0000 7f3141a00700 1 -- 172.21.15.117:0/2036254423 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f31200a2ed0 msgr2=0x7f31200a5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:32.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.108+0000 7f3141a00700 1 --2- 172.21.15.117:0/2036254423 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f31200a2ed0 0x7f31200a5390 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f313400f930 tx=0x7f313400fcf0 comp rx=0 tx=0).stop 2024-06-03T03:25:32.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.108+0000 7f3141a00700 1 -- 172.21.15.117:0/2036254423 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f313c1045c0 msgr2=0x7f313c1111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:32.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.108+0000 7f3141a00700 1 --2- 172.21.15.117:0/2036254423 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f313c1045c0 0x7f313c1111d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f312c006010 tx=0x7f312c003a90 comp rx=0 tx=0).stop 2024-06-03T03:25:32.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.108+0000 7f3141a00700 1 -- 172.21.15.117:0/2036254423 shutdown_connections 2024-06-03T03:25:32.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.108+0000 7f3141a00700 1 --2- 172.21.15.117:0/2036254423 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f31200a2ed0 0x7f31200a5390 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:32.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.108+0000 7f3141a00700 1 --2- 172.21.15.117:0/2036254423 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f313c10a670 0x7f313c10e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:32.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.108+0000 7f3141a00700 1 --2- 172.21.15.117:0/2036254423 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f313c1045c0 0x7f313c1111d0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:32.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.108+0000 7f3141a00700 1 --2- 172.21.15.117:0/2036254423 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f313c104f80 0x7f313c10e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:32.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.108+0000 7f3141a00700 1 -- 172.21.15.117:0/2036254423 >> 172.21.15.117:0/2036254423 conn(0x7f313c1000d0 msgr2=0x7f313c1016d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:25:32.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.108+0000 7f3141a00700 1 -- 172.21.15.117:0/2036254423 shutdown_connections 2024-06-03T03:25:32.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:32.108+0000 7f3141a00700 1 -- 172.21.15.117:0/2036254423 wait complete. 2024-06-03T03:25:32.812 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:25:32 smithi117 ceph-mon[31649]: scrub queued for path: / 2024-06-03T03:25:32.812 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:25:32 smithi117 ceph-mon[31649]: scrub summary: idle+waiting paths [/] 2024-06-03T03:25:32.812 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:25:32 smithi117 ceph-mon[31649]: scrub summary: active paths [/] 2024-06-03T03:25:32.812 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:25:32 smithi117 ceph-mon[31649]: scrub summary: idle+waiting paths [/] 2024-06-03T03:25:32.906 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:25:32 smithi151 ceph-mon[37043]: scrub queued for path: / 2024-06-03T03:25:32.906 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:25:32 smithi151 ceph-mon[37043]: scrub summary: idle+waiting paths [/] 2024-06-03T03:25:32.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:25:32 smithi151 ceph-mon[37043]: scrub summary: active paths [/] 2024-06-03T03:25:32.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:25:32 smithi151 ceph-mon[37043]: scrub summary: idle+waiting paths [/] 2024-06-03T03:25:32.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:25:32 smithi183 ceph-mon[37326]: scrub queued for path: / 2024-06-03T03:25:32.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:25:32 smithi183 ceph-mon[37326]: scrub summary: idle+waiting paths [/] 2024-06-03T03:25:32.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:25:32 smithi183 ceph-mon[37326]: scrub summary: active paths [/] 2024-06-03T03:25:32.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:25:32 smithi183 ceph-mon[37326]: scrub summary: idle+waiting paths [/] 2024-06-03T03:25:33.194 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-03T03:25:33.652 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:25:33.866 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:25:33 smithi117 ceph-mon[31649]: pgmap v513: 129 pgs: 129 active+clean; 665 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 938 B/s wr, 11 op/s 2024-06-03T03:25:33.909 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:25:33 smithi151 ceph-mon[37043]: pgmap v513: 129 pgs: 129 active+clean; 665 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 938 B/s wr, 11 op/s 2024-06-03T03:25:33.981 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:25:33 smithi183 ceph-mon[37326]: pgmap v513: 129 pgs: 129 active+clean; 665 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 938 B/s wr, 11 op/s 2024-06-03T03:25:36.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.073+0000 7fdcd4e00700 1 -- 172.21.15.117:0/3787689681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdcd010ebe0 msgr2=0x7fdcd0111050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:36.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.073+0000 7fdcd4e00700 1 --2- 172.21.15.117:0/3787689681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdcd010ebe0 0x7fdcd0111050 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fdcc800b0a0 tx=0x7fdcc800b3b0 comp rx=0 tx=0).stop 2024-06-03T03:25:36.076 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:25:36 smithi117 ceph-mon[31649]: pgmap v514: 129 pgs: 129 active+clean; 665 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 767 B/s wr, 9 op/s 2024-06-03T03:25:36.077 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.074+0000 7fdcd4e00700 1 -- 172.21.15.117:0/3787689681 shutdown_connections 2024-06-03T03:25:36.077 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.074+0000 7fdcd4e00700 1 --2- 172.21.15.117:0/3787689681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdcd010ebe0 0x7fdcd0111050 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:36.077 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.074+0000 7fdcd4e00700 1 --2- 172.21.15.117:0/3787689681 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdcd00748c0 0x7fdcd0074ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:36.077 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.074+0000 7fdcd4e00700 1 --2- 172.21.15.117:0/3787689681 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdcd00751e0 0x7fdcd010e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:36.077 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.074+0000 7fdcd4e00700 1 -- 172.21.15.117:0/3787689681 >> 172.21.15.117:0/3787689681 conn(0x7fdcd00fdeb0 msgr2=0x7fdcd01002d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:25:36.077 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.074+0000 7fdcd4e00700 1 -- 172.21.15.117:0/3787689681 shutdown_connections 2024-06-03T03:25:36.077 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.074+0000 7fdcd4e00700 1 -- 172.21.15.117:0/3787689681 wait complete. 2024-06-03T03:25:36.078 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.075+0000 7fdcd4e00700 1 Processor -- start 2024-06-03T03:25:36.078 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.075+0000 7fdcd4e00700 1 -- start start 2024-06-03T03:25:36.078 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.075+0000 7fdcd4e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdcd00748c0 0x7fdcd019ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:36.078 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.076+0000 7fdcd4e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdcd00751e0 0x7fdcd019b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:36.078 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.076+0000 7fdcd4e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdcd010ebe0 0x7fdcd019f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:36.078 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.076+0000 7fdcd4e00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd0114920 con 0x7fdcd00748c0 2024-06-03T03:25:36.078 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.076+0000 7fdcd4e00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd01147a0 con 0x7fdcd010ebe0 2024-06-03T03:25:36.078 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.076+0000 7fdcd4e00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd0114aa0 con 0x7fdcd00751e0 2024-06-03T03:25:36.079 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.076+0000 7fdccf400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdcd00748c0 0x7fdcd019ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:36.079 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.076+0000 7fdccea00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdcd00751e0 0x7fdcd019b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:36.079 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.076+0000 7fdccf400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdcd00748c0 0x7fdcd019ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:49926/0 (socket says 172.21.15.117:49926) 2024-06-03T03:25:36.079 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.076+0000 7fdccf400700 1 -- 172.21.15.117:0/1020374889 learned_addr learned my addr 172.21.15.117:0/1020374889 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:25:36.079 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.076+0000 7fdccfe00700 1 --2- 172.21.15.117:0/1020374889 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdcd010ebe0 0x7fdcd019f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:36.079 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.076+0000 7fdccea00700 1 -- 172.21.15.117:0/1020374889 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdcd010ebe0 msgr2=0x7fdcd019f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:36.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.076+0000 7fdccea00700 1 --2- 172.21.15.117:0/1020374889 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdcd010ebe0 0x7fdcd019f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:36.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.076+0000 7fdccea00700 1 -- 172.21.15.117:0/1020374889 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdcd00748c0 msgr2=0x7fdcd019ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:36.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.076+0000 7fdccea00700 1 --2- 172.21.15.117:0/1020374889 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdcd00748c0 0x7fdcd019ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:36.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.076+0000 7fdccea00700 1 -- 172.21.15.117:0/1020374889 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcd019fd80 con 0x7fdcd00751e0 2024-06-03T03:25:36.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.077+0000 7fdccea00700 1 --2- 172.21.15.117:0/1020374889 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdcd00751e0 0x7fdcd019b2b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fdcb800d750 tx=0x7fdcb800da60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:25:36.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.077+0000 7fdcc7e00700 1 -- 172.21.15.117:0/1020374889 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb8014030 con 0x7fdcd00751e0 2024-06-03T03:25:36.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.077+0000 7fdcc7e00700 1 -- 172.21.15.117:0/1020374889 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcb8004500 con 0x7fdcd00751e0 2024-06-03T03:25:36.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.077+0000 7fdcd4e00700 1 -- 172.21.15.117:0/1020374889 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcd01a0070 con 0x7fdcd00751e0 2024-06-03T03:25:36.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.077+0000 7fdcc7e00700 1 -- 172.21.15.117:0/1020374889 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb80054e0 con 0x7fdcd00751e0 2024-06-03T03:25:36.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.077+0000 7fdcd4e00700 1 -- 172.21.15.117:0/1020374889 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcd01a78f0 con 0x7fdcd00751e0 2024-06-03T03:25:36.082 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.080+0000 7fdcc7e00700 1 -- 172.21.15.117:0/1020374889 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7fdcb8004750 con 0x7fdcd00751e0 2024-06-03T03:25:36.083 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.082+0000 7fdcc7e00700 1 --2- 172.21.15.117:0/1020374889 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fdcb40a2cb0 0x7fdcb40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:36.084 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.082+0000 7fdce0d89b80 1 Processor -- start 2024-06-03T03:25:36.084 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.082+0000 7fdcc7e00700 1 -- 172.21.15.117:0/1020374889 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7fdcb80cedd0 con 0x7fdcd00751e0 2024-06-03T03:25:36.084 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.082+0000 7fdccf400700 1 --2- 172.21.15.117:0/1020374889 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fdcb40a2cb0 0x7fdcb40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:36.084 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.082+0000 7fdce0d89b80 1 -- start start 2024-06-03T03:25:36.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.082+0000 7fdce0d89b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eb2b7e4d60 0x55eb2b7e5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:36.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.083+0000 7fdce0d89b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb2b7e5680 0x55eb2b7efff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:36.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.083+0000 7fdce0d89b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb2b7f0530 0x55eb2b7f2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:36.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.083+0000 7fdce0d89b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55eb2b6f4a10 con 0x55eb2b7e4d60 2024-06-03T03:25:36.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.083+0000 7fdce0d89b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55eb2b737b00 con 0x55eb2b7f0530 2024-06-03T03:25:36.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.083+0000 7fdce0d89b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55eb2b6f28f0 con 0x55eb2b7e5680 2024-06-03T03:25:36.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.083+0000 7fdccf400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eb2b7e4d60 0x55eb2b7e5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:36.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.083+0000 7fdccf400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eb2b7e4d60 0x55eb2b7e5140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:49932/0 (socket says 172.21.15.117:49932) 2024-06-03T03:25:36.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.083+0000 7fdccf400700 1 -- 172.21.15.117:0/1184734620 learned_addr learned my addr 172.21.15.117:0/1184734620 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:25:36.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.083+0000 7fdccea00700 1 --2- 172.21.15.117:0/1184734620 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb2b7e5680 0x55eb2b7efff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:36.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.083+0000 7fdccfe00700 1 --2- 172.21.15.117:0/1184734620 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb2b7f0530 0x55eb2b7f2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:36.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.083+0000 7fdccf400700 1 --2- 172.21.15.117:0/1020374889 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fdcb40a2cb0 0x7fdcb40a5170 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0009d60 tx=0x7fdcc0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:25:36.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.083+0000 7fdccea00700 1 -- 172.21.15.117:0/1184734620 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb2b7f0530 msgr2=0x55eb2b7f2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:25:36.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.083+0000 7fdccea00700 1 --2- 172.21.15.117:0/1184734620 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb2b7f0530 0x55eb2b7f2920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:36.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.083+0000 7fdccea00700 1 -- 172.21.15.117:0/1184734620 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eb2b7e4d60 msgr2=0x55eb2b7e5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:25:36.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.083+0000 7fdccea00700 1 --2- 172.21.15.117:0/1184734620 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eb2b7e4d60 0x55eb2b7e5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:36.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.083+0000 7fdccea00700 1 -- 172.21.15.117:0/1184734620 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb2b6dff90 con 0x55eb2b7e5680 2024-06-03T03:25:36.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.083+0000 7fdccf400700 1 --2- 172.21.15.117:0/1184734620 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eb2b7e4d60 0x55eb2b7e5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:25:36.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.084+0000 7fdccea00700 1 --2- 172.21.15.117:0/1184734620 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb2b7e5680 0x55eb2b7efff0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fdcb80cc390 tx=0x7fdcb809b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4432a387a052e838 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:25:36.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.084+0000 7fdcc4c00700 1 -- 172.21.15.117:0/1184734620 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb801d040 con 0x55eb2b7e5680 2024-06-03T03:25:36.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.084+0000 7fdcc4c00700 1 -- 172.21.15.117:0/1184734620 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcb809f490 con 0x55eb2b7e5680 2024-06-03T03:25:36.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.084+0000 7fdcc4c00700 1 -- 172.21.15.117:0/1184734620 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb809f120 con 0x55eb2b7e5680 2024-06-03T03:25:36.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.085+0000 7fdce0d89b80 1 -- 172.21.15.117:0/1184734620 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb2b7e5680 msgr2=0x55eb2b7efff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:36.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.085+0000 7fdce0d89b80 1 --2- 172.21.15.117:0/1184734620 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb2b7e5680 0x55eb2b7efff0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fdcb80cc390 tx=0x7fdcb809b860 comp rx=0 tx=0).stop 2024-06-03T03:25:36.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.085+0000 7fdce0d89b80 1 -- 172.21.15.117:0/1184734620 shutdown_connections 2024-06-03T03:25:36.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.085+0000 7fdce0d89b80 1 --2- 172.21.15.117:0/1184734620 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb2b7f0530 0x55eb2b7f2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:36.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.085+0000 7fdce0d89b80 1 --2- 172.21.15.117:0/1184734620 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eb2b7e4d60 0x55eb2b7e5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:36.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.085+0000 7fdce0d89b80 1 --2- 172.21.15.117:0/1184734620 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb2b7e5680 0x55eb2b7efff0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:36.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.085+0000 7fdce0d89b80 1 -- 172.21.15.117:0/1184734620 >> 172.21.15.117:0/1184734620 conn(0x55eb2b712170 msgr2=0x55eb2b7e1280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:25:36.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.085+0000 7fdce0d89b80 1 -- 172.21.15.117:0/1184734620 shutdown_connections 2024-06-03T03:25:36.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.085+0000 7fdce0d89b80 1 -- 172.21.15.117:0/1184734620 wait complete. 2024-06-03T03:25:36.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.085+0000 7fdce0d89b80 1 -- start start 2024-06-03T03:25:36.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.086+0000 7fdce0d89b80 1 Processor -- start 2024-06-03T03:25:36.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.086+0000 7fdce0d89b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eb2b7e4d60 0x55eb2b7efd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:36.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.086+0000 7fdce0d89b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb2b7e5680 0x55eb2b7ed020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:36.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.086+0000 7fdce0d89b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb2b7f0530 0x55eb2b7ed560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:36.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.086+0000 7fdce0d89b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55eb2b6d3f60 con 0x55eb2b7e4d60 2024-06-03T03:25:36.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.086+0000 7fdce0d89b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55eb2b733ac0 con 0x55eb2b7f0530 2024-06-03T03:25:36.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.086+0000 7fdce0d89b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55eb2b7f2fc0 con 0x55eb2b7e5680 2024-06-03T03:25:36.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.086+0000 7fdce0d89b80 10 client.0 _ll_register_callbacks cb 0x55eb2b6cd380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:25:36.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.087+0000 7fdccea00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb2b7e5680 0x55eb2b7ed020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:36.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.087+0000 7fdccf400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eb2b7e4d60 0x55eb2b7efd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:36.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.087+0000 7fdccea00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb2b7e5680 0x55eb2b7ed020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:44864/0 (socket says 172.21.15.117:44864) 2024-06-03T03:25:36.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.087+0000 7fdccea00700 1 -- 172.21.15.117:0/1561388824 learned_addr learned my addr 172.21.15.117:0/1561388824 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:25:36.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.087+0000 7fdccfe00700 1 --2- 172.21.15.117:0/1561388824 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb2b7f0530 0x55eb2b7ed560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:36.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.087+0000 7fdccea00700 1 -- 172.21.15.117:0/1561388824 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb2b7f0530 msgr2=0x55eb2b7ed560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:25:36.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.087+0000 7fdccea00700 1 --2- 172.21.15.117:0/1561388824 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb2b7f0530 0x55eb2b7ed560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:36.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.087+0000 7fdccea00700 1 -- 172.21.15.117:0/1561388824 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eb2b7e4d60 msgr2=0x55eb2b7efd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:25:36.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.087+0000 7fdccea00700 1 --2- 172.21.15.117:0/1561388824 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eb2b7e4d60 0x55eb2b7efd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:36.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.087+0000 7fdccea00700 1 -- 172.21.15.117:0/1561388824 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb2b6868d0 con 0x55eb2b7e5680 2024-06-03T03:25:36.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.087+0000 7fdccfe00700 1 --2- 172.21.15.117:0/1561388824 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb2b7f0530 0x55eb2b7ed560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T03:25:36.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.087+0000 7fdccea00700 1 --2- 172.21.15.117:0/1561388824 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb2b7e5680 0x55eb2b7ed020 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fdcb800d720 tx=0x7fdcb8004f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14fa9521cabfd900 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:25:36.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.087+0000 7fdcbd600700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-06-03T03:25:36.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.088+0000 7fdcbd600700 1 -- 172.21.15.117:0/1561388824 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb8065060 con 0x55eb2b7e5680 2024-06-03T03:25:36.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.088+0000 7fdce0d89b80 1 -- 172.21.15.117:0/1561388824 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdcb809f490 con 0x55eb2b7e5680 2024-06-03T03:25:36.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.088+0000 7fdcbd600700 1 -- 172.21.15.117:0/1561388824 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7fdcb80bb0d0 con 0x55eb2b7e5680 2024-06-03T03:25:36.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.088+0000 7fdcbd600700 1 -- 172.21.15.117:0/1561388824 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fdcb8005680 con 0x55eb2b7e5680 2024-06-03T03:25:36.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.088+0000 7fdce0d89b80 10 client.25327 fetch_fsmap learned FSMap version 31 2024-06-03T03:25:36.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.088+0000 7fdce0d89b80 1 -- 172.21.15.117:0/1561388824 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55eb2b7fd730 con 0x55eb2b7e5680 2024-06-03T03:25:36.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.088+0000 7fdcbd600700 1 -- 172.21.15.117:0/1561388824 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcb809f490 con 0x55eb2b7e5680 2024-06-03T03:25:36.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.088+0000 7fdcbd600700 1 -- 172.21.15.117:0/1561388824 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fdcb80a38c0 con 0x55eb2b7e5680 2024-06-03T03:25:36.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.088+0000 7fdce0d89b80 10 client.25327 fetch_fsmap finished waiting for FSMap version 31 2024-06-03T03:25:36.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.088+0000 7fdce0d89b80 10 client.25327 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-03T03:25:36.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.088+0000 7fdce0d89b80 20 client.25327 populate_metadata read hostname 'smithi117' 2024-06-03T03:25:36.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.089+0000 7fdce0d89b80 1 --2- 172.21.15.117:0/1561388824 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55eb2b7ffc00 0x55eb2b801ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:36.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.089+0000 7fdce0d89b80 4 client.25327 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T03:25:36.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.089+0000 7fdce0d89b80 1 -- 172.21.15.117:0/1561388824 --> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb2b745000 con 0x55eb2b7ffc00 2024-06-03T03:25:36.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.089+0000 7fdccf400700 1 --2- 172.21.15.117:0/1561388824 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55eb2b7ffc00 0x55eb2b801ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:36.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.089+0000 7fdccf400700 1 --2- 172.21.15.117:0/1561388824 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55eb2b7ffc00 0x55eb2b801ff0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf085f2c0121d7dc server_cookie=ed605fdd6eedd427 in_seq=0 out_seq=0 2024-06-03T03:25:36.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.089+0000 7fdcbd600700 10 client.25327 ms_handle_connect on v2:172.21.15.183:6838/1439615083 2024-06-03T03:25:36.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.090+0000 7fdcbd600700 1 -- 172.21.15.117:0/1561388824 <== mds.0 v2:172.21.15.183:6838/1439615083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55eb2b745000 con 0x55eb2b7ffc00 2024-06-03T03:25:36.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.090+0000 7fdcbd600700 10 client.25327 handle_command_reply: tid=0 2024-06-03T03:25:36.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.090+0000 7fdce0d89b80 1 client.25327 shutdown 2024-06-03T03:25:36.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.090+0000 7fdce0d89b80 1 -- 172.21.15.117:0/1561388824 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb2b7e5680 msgr2=0x55eb2b7ed020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:36.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.090+0000 7fdce0d89b80 1 --2- 172.21.15.117:0/1561388824 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb2b7e5680 0x55eb2b7ed020 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fdcb800d720 tx=0x7fdcb8004f00 comp rx=0 tx=0).stop 2024-06-03T03:25:36.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.090+0000 7fdce0d89b80 1 -- 172.21.15.117:0/1561388824 shutdown_connections 2024-06-03T03:25:36.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.090+0000 7fdce0d89b80 1 --2- 172.21.15.117:0/1561388824 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55eb2b7ffc00 0x55eb2b801ff0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:36.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.090+0000 7fdce0d89b80 1 --2- 172.21.15.117:0/1561388824 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb2b7f0530 0x55eb2b7ed560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:36.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.090+0000 7fdce0d89b80 1 --2- 172.21.15.117:0/1561388824 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eb2b7e4d60 0x55eb2b7efd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:36.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.090+0000 7fdce0d89b80 1 --2- 172.21.15.117:0/1561388824 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb2b7e5680 0x55eb2b7ed020 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:36.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.090+0000 7fdcbd600700 0 client.25327 ms_handle_reset on v2:172.21.15.183:6838/1439615083 2024-06-03T03:25:36.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.091+0000 7fdce0d89b80 1 -- 172.21.15.117:0/1561388824 >> 172.21.15.117:0/1561388824 conn(0x55eb2b712170 msgr2=0x55eb2b7e0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:25:36.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.091+0000 7fdce0d89b80 1 -- 172.21.15.117:0/1561388824 shutdown_connections 2024-06-03T03:25:36.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.091+0000 7fdce0d89b80 1 -- 172.21.15.117:0/1561388824 wait complete. 2024-06-03T03:25:36.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.091+0000 7fdce0d89b80 20 client.25327 trim_cache size 0 max 16384 2024-06-03T03:25:36.115 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.113+0000 7fdce0d89b80 1 Processor -- start 2024-06-03T03:25:36.115 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.113+0000 7fdce0d89b80 1 -- start start 2024-06-03T03:25:36.115 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.114+0000 7fdce0d89b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eb2b7e4d60 0x55eb2b7ecfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:36.116 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.114+0000 7fdce0d89b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb2b7e5680 0x55eb2b7ed520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:36.116 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.114+0000 7fdce0d89b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb2b7f0530 0x55eb2b805020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:36.116 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.114+0000 7fdce0d89b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd01b04b0 con 0x55eb2b7e4d60 2024-06-03T03:25:36.116 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.114+0000 7fdce0d89b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd01b1ff0 con 0x55eb2b7f0530 2024-06-03T03:25:36.116 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.114+0000 7fdce0d89b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd01b0b80 con 0x55eb2b7e5680 2024-06-03T03:25:36.116 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.114+0000 7fdccea00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb2b7e5680 0x55eb2b7ed520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:36.116 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.114+0000 7fdccf400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eb2b7e4d60 0x55eb2b7ecfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:36.116 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.114+0000 7fdccfe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb2b7f0530 0x55eb2b805020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:36.116 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.114+0000 7fdccea00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb2b7e5680 0x55eb2b7ed520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:44866/0 (socket says 172.21.15.117:44866) 2024-06-03T03:25:36.117 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.114+0000 7fdccea00700 1 -- 172.21.15.117:0/2353994696 learned_addr learned my addr 172.21.15.117:0/2353994696 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:25:36.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.115+0000 7fdccfe00700 1 -- 172.21.15.117:0/2353994696 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb2b7e5680 msgr2=0x55eb2b7ed520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:25:36.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.115+0000 7fdccfe00700 1 --2- 172.21.15.117:0/2353994696 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb2b7e5680 0x55eb2b7ed520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:36.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.115+0000 7fdccfe00700 1 -- 172.21.15.117:0/2353994696 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eb2b7e4d60 msgr2=0x55eb2b7ecfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:25:36.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.115+0000 7fdccfe00700 1 --2- 172.21.15.117:0/2353994696 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eb2b7e4d60 0x55eb2b7ecfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:36.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.115+0000 7fdccfe00700 1 -- 172.21.15.117:0/2353994696 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb2b7e6520 con 0x55eb2b7f0530 2024-06-03T03:25:36.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.115+0000 7fdccea00700 1 --2- 172.21.15.117:0/2353994696 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb2b7e5680 0x55eb2b7ed520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:25:36.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.115+0000 7fdccf400700 1 --2- 172.21.15.117:0/2353994696 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eb2b7e4d60 0x55eb2b7ecfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T03:25:36.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.115+0000 7fdccfe00700 1 --2- 172.21.15.117:0/2353994696 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb2b7f0530 0x55eb2b805020 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fdcc8009fd0 tx=0x7fdcc8009520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97177216ac2b262d server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:25:36.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.115+0000 7fdcbfe00700 1 -- 172.21.15.117:0/2353994696 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc800bcb0 con 0x55eb2b7f0530 2024-06-03T03:25:36.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.115+0000 7fdcbfe00700 1 -- 172.21.15.117:0/2353994696 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcc800be50 con 0x55eb2b7f0530 2024-06-03T03:25:36.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.115+0000 7fdcbfe00700 1 -- 172.21.15.117:0/2353994696 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc8026470 con 0x55eb2b7f0530 2024-06-03T03:25:36.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.116+0000 7fdce0d89b80 1 -- 172.21.15.117:0/2353994696 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb2b7f0530 msgr2=0x55eb2b805020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:36.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.116+0000 7fdce0d89b80 1 --2- 172.21.15.117:0/2353994696 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb2b7f0530 0x55eb2b805020 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fdcc8009fd0 tx=0x7fdcc8009520 comp rx=0 tx=0).stop 2024-06-03T03:25:36.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.116+0000 7fdce0d89b80 1 -- 172.21.15.117:0/2353994696 shutdown_connections 2024-06-03T03:25:36.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.116+0000 7fdce0d89b80 1 --2- 172.21.15.117:0/2353994696 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb2b7f0530 0x55eb2b805020 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:36.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.116+0000 7fdce0d89b80 1 --2- 172.21.15.117:0/2353994696 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eb2b7e4d60 0x55eb2b7ecfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:36.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.116+0000 7fdce0d89b80 1 --2- 172.21.15.117:0/2353994696 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb2b7e5680 0x55eb2b7ed520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:36.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.116+0000 7fdce0d89b80 1 -- 172.21.15.117:0/2353994696 >> 172.21.15.117:0/2353994696 conn(0x55eb2b712170 msgr2=0x55eb2b7ec8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:25:36.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.116+0000 7fdce0d89b80 1 -- 172.21.15.117:0/2353994696 shutdown_connections 2024-06-03T03:25:36.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.116+0000 7fdce0d89b80 1 -- 172.21.15.117:0/2353994696 wait complete. 2024-06-03T03:25:36.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.117+0000 7fdce0d89b80 1 -- start start 2024-06-03T03:25:36.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.117+0000 7fdce0d89b80 1 Processor -- start 2024-06-03T03:25:36.122 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.117+0000 7fdce0d89b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eb2b7e4d60 0x55eb2b85e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:36.122 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.117+0000 7fdce0d89b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb2b7e5680 0x55eb2b85ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:36.122 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.117+0000 7fdce0d89b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb2b7f0530 0x55eb2b869550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:36.122 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.117+0000 7fdce0d89b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd01b1920 con 0x55eb2b7e4d60 2024-06-03T03:25:36.122 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.117+0000 7fdce0d89b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd01b1250 con 0x55eb2b7e5680 2024-06-03T03:25:36.122 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.118+0000 7fdce0d89b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd01ae450 con 0x55eb2b7f0530 2024-06-03T03:25:36.122 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.118+0000 7fdce0d89b80 10 client.0 _ll_register_callbacks cb 0x55eb2b6cd380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:25:36.122 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.118+0000 7fdccf400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eb2b7e4d60 0x55eb2b85e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:36.122 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.118+0000 7fdccf400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eb2b7e4d60 0x55eb2b85e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:49946/0 (socket says 172.21.15.117:49946) 2024-06-03T03:25:36.122 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.118+0000 7fdccf400700 1 -- 172.21.15.117:0/344512372 learned_addr learned my addr 172.21.15.117:0/344512372 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:25:36.122 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.118+0000 7fdccea00700 1 --2- 172.21.15.117:0/344512372 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb2b7e5680 0x55eb2b85ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:36.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.118+0000 7fdccfe00700 1 --2- 172.21.15.117:0/344512372 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb2b7f0530 0x55eb2b869550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:36.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.118+0000 7fdccea00700 1 -- 172.21.15.117:0/344512372 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb2b7f0530 msgr2=0x55eb2b869550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:25:36.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.118+0000 7fdccea00700 1 --2- 172.21.15.117:0/344512372 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb2b7f0530 0x55eb2b869550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:36.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.118+0000 7fdccea00700 1 -- 172.21.15.117:0/344512372 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eb2b7e4d60 msgr2=0x55eb2b85e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:25:36.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.118+0000 7fdccea00700 1 --2- 172.21.15.117:0/344512372 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eb2b7e4d60 0x55eb2b85e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:36.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.118+0000 7fdccea00700 1 -- 172.21.15.117:0/344512372 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb2b869b40 con 0x55eb2b7e5680 2024-06-03T03:25:36.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.118+0000 7fdccf400700 1 --2- 172.21.15.117:0/344512372 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eb2b7e4d60 0x55eb2b85e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:25:36.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.119+0000 7fdccea00700 1 --2- 172.21.15.117:0/344512372 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb2b7e5680 0x55eb2b85ebf0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fdcb809ade0 tx=0x7fdcb8097400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3f6152971fa26e0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:25:36.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.119+0000 7fdcbd600700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-06-03T03:25:36.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.119+0000 7fdcbd600700 1 -- 172.21.15.117:0/344512372 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb801d040 con 0x55eb2b7e5680 2024-06-03T03:25:36.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.119+0000 7fdcbd600700 1 -- 172.21.15.117:0/344512372 <== mon.2 v2:172.21.15.151:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7fdcb80bb0d0 con 0x55eb2b7e5680 2024-06-03T03:25:36.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.119+0000 7fdce0d89b80 1 -- 172.21.15.117:0/344512372 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdcc800be50 con 0x55eb2b7e5680 2024-06-03T03:25:36.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.119+0000 7fdcbd600700 1 -- 172.21.15.117:0/344512372 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fdcb8065040 con 0x55eb2b7e5680 2024-06-03T03:25:36.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.119+0000 7fdce0d89b80 10 client.25298 fetch_fsmap learned FSMap version 31 2024-06-03T03:25:36.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.119+0000 7fdce0d89b80 1 -- 172.21.15.117:0/344512372 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55eb2b86ae50 con 0x55eb2b7e5680 2024-06-03T03:25:36.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.120+0000 7fdcbd600700 1 -- 172.21.15.117:0/344512372 <== mon.2 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcc800be50 con 0x55eb2b7e5680 2024-06-03T03:25:36.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.120+0000 7fdcbd600700 1 -- 172.21.15.117:0/344512372 <== mon.2 v2:172.21.15.151:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fdcb80cc5e0 con 0x55eb2b7e5680 2024-06-03T03:25:36.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.120+0000 7fdce0d89b80 10 client.25298 fetch_fsmap finished waiting for FSMap version 31 2024-06-03T03:25:36.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.120+0000 7fdce0d89b80 10 client.25298 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-03T03:25:36.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.120+0000 7fdce0d89b80 20 client.25298 populate_metadata read hostname 'smithi117' 2024-06-03T03:25:36.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.120+0000 7fdce0d89b80 1 --2- 172.21.15.117:0/344512372 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55eb2b86b2b0 0x55eb2b86d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:36.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.120+0000 7fdce0d89b80 4 client.25298 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-03T03:25:36.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.120+0000 7fdce0d89b80 1 -- 172.21.15.117:0/344512372 --> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55eb2b73f580 con 0x55eb2b86b2b0 2024-06-03T03:25:36.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.120+0000 7fdccf400700 1 --2- 172.21.15.117:0/344512372 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55eb2b86b2b0 0x55eb2b86d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:36.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.121+0000 7fdccf400700 1 --2- 172.21.15.117:0/344512372 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55eb2b86b2b0 0x55eb2b86d6a0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=848d4d379f5e0c0d server_cookie=15bfb3aae25ac864 in_seq=0 out_seq=0 2024-06-03T03:25:36.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.121+0000 7fdcbd600700 10 client.25298 ms_handle_connect on v2:172.21.15.183:6838/1439615083 2024-06-03T03:25:36.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.121+0000 7fdcbd600700 1 -- 172.21.15.117:0/344512372 <== mds.0 v2:172.21.15.183:6838/1439615083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55eb2b73f580 con 0x55eb2b86b2b0 2024-06-03T03:25:36.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.121+0000 7fdcbd600700 10 client.25298 handle_command_reply: tid=0 2024-06-03T03:25:36.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.121+0000 7fdce0d89b80 1 client.25298 shutdown 2024-06-03T03:25:36.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.121+0000 7fdce0d89b80 1 -- 172.21.15.117:0/344512372 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb2b7e5680 msgr2=0x55eb2b85ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:36.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.121+0000 7fdce0d89b80 1 --2- 172.21.15.117:0/344512372 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb2b7e5680 0x55eb2b85ebf0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fdcb809ade0 tx=0x7fdcb8097400 comp rx=0 tx=0).stop 2024-06-03T03:25:36.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.121+0000 7fdce0d89b80 1 -- 172.21.15.117:0/344512372 shutdown_connections 2024-06-03T03:25:36.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.121+0000 7fdce0d89b80 1 --2- 172.21.15.117:0/344512372 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55eb2b86b2b0 0x55eb2b86d6a0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:36.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.122+0000 7fdce0d89b80 1 --2- 172.21.15.117:0/344512372 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eb2b7f0530 0x55eb2b869550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:36.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.122+0000 7fdce0d89b80 1 --2- 172.21.15.117:0/344512372 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eb2b7e4d60 0x55eb2b85e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:36.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.122+0000 7fdce0d89b80 1 --2- 172.21.15.117:0/344512372 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb2b7e5680 0x55eb2b85ebf0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:36.127 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.122+0000 7fdcbd600700 0 client.25298 ms_handle_reset on v2:172.21.15.183:6838/1439615083 2024-06-03T03:25:36.127 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.122+0000 7fdce0d89b80 1 -- 172.21.15.117:0/344512372 >> 172.21.15.117:0/344512372 conn(0x55eb2b712170 msgr2=0x55eb2b7ed160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:25:36.127 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.122+0000 7fdce0d89b80 1 -- 172.21.15.117:0/344512372 shutdown_connections 2024-06-03T03:25:36.127 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.122+0000 7fdce0d89b80 1 -- 172.21.15.117:0/344512372 wait complete. 2024-06-03T03:25:36.127 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.122+0000 7fdce0d89b80 20 client.25298 trim_cache size 0 max 16384 2024-06-03T03:25:36.127 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-06-03T03:25:36.127 INFO:teuthology.orchestra.run.smithi117.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-03T03:25:36.127 INFO:teuthology.orchestra.run.smithi117.stdout: "scrubs": { 2024-06-03T03:25:36.127 INFO:teuthology.orchestra.run.smithi117.stdout: "5abaee35-60ff-4f7e-a25a-04edc9482b99": { 2024-06-03T03:25:36.127 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-06-03T03:25:36.128 INFO:teuthology.orchestra.run.smithi117.stdout: "tag": "5abaee35-60ff-4f7e-a25a-04edc9482b99", 2024-06-03T03:25:36.128 INFO:teuthology.orchestra.run.smithi117.stdout: "options": "recursive,force" 2024-06-03T03:25:36.128 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:25:36.128 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:25:36.128 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-06-03T03:25:36.128 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.123+0000 7fdcd4e00700 1 -- 172.21.15.117:0/1020374889 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fdcb40a2cb0 msgr2=0x7fdcb40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:36.128 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.123+0000 7fdcd4e00700 1 --2- 172.21.15.117:0/1020374889 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fdcb40a2cb0 0x7fdcb40a5170 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0009d60 tx=0x7fdcc0009260 comp rx=0 tx=0).stop 2024-06-03T03:25:36.128 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.123+0000 7fdcd4e00700 1 -- 172.21.15.117:0/1020374889 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdcd00751e0 msgr2=0x7fdcd019b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:36.129 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.123+0000 7fdcd4e00700 1 --2- 172.21.15.117:0/1020374889 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdcd00751e0 0x7fdcd019b2b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fdcb800d750 tx=0x7fdcb800da60 comp rx=0 tx=0).stop 2024-06-03T03:25:36.129 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.123+0000 7fdcd4e00700 1 -- 172.21.15.117:0/1020374889 shutdown_connections 2024-06-03T03:25:36.129 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.123+0000 7fdcd4e00700 1 --2- 172.21.15.117:0/1020374889 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fdcb40a2cb0 0x7fdcb40a5170 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:36.129 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.123+0000 7fdcd4e00700 1 --2- 172.21.15.117:0/1020374889 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdcd010ebe0 0x7fdcd019f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:36.129 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.123+0000 7fdcd4e00700 1 --2- 172.21.15.117:0/1020374889 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdcd00748c0 0x7fdcd019ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:36.129 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.123+0000 7fdcd4e00700 1 --2- 172.21.15.117:0/1020374889 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdcd00751e0 0x7fdcd019b2b0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:36.129 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.123+0000 7fdcd4e00700 1 -- 172.21.15.117:0/1020374889 >> 172.21.15.117:0/1020374889 conn(0x7fdcd00fdeb0 msgr2=0x7fdcd00feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:25:36.129 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.123+0000 7fdcd4e00700 1 -- 172.21.15.117:0/1020374889 shutdown_connections 2024-06-03T03:25:36.129 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:25:36.123+0000 7fdcd4e00700 1 -- 172.21.15.117:0/1020374889 wait complete. 2024-06-03T03:25:36.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:25:36 smithi151 ceph-mon[37043]: pgmap v514: 129 pgs: 129 active+clean; 665 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 767 B/s wr, 9 op/s 2024-06-03T03:25:36.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:25:36 smithi183 ceph-mon[37326]: pgmap v514: 129 pgs: 129 active+clean; 665 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 767 B/s wr, 9 op/s 2024-06-03T03:25:37.000 INFO:tasks.cephfs.filesystem:scrub status for tag:5abaee35-60ff-4f7e-a25a-04edc9482b99 - {'path': '/', 'tag': '5abaee35-60ff-4f7e-a25a-04edc9482b99', 'options': 'recursive,force'} 2024-06-03T03:25:38.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:25:38 smithi151 ceph-mon[37043]: pgmap v515: 129 pgs: 129 active+clean; 665 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 9 op/s 2024-06-03T03:25:38.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:25:38 smithi117 ceph-mon[31649]: pgmap v515: 129 pgs: 129 active+clean; 665 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 9 op/s 2024-06-03T03:25:38.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:25:38 smithi183 ceph-mon[37326]: pgmap v515: 129 pgs: 129 active+clean; 665 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 9 op/s 2024-06-03T03:25:40.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:25:40 smithi151 ceph-mon[37043]: pgmap v516: 129 pgs: 129 active+clean; 665 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 767 B/s wr, 12 op/s 2024-06-03T03:25:40.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:25:40 smithi117 ceph-mon[31649]: pgmap v516: 129 pgs: 129 active+clean; 665 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 767 B/s wr, 12 op/s 2024-06-03T03:25:40.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:25:40 smithi183 ceph-mon[37326]: pgmap v516: 129 pgs: 129 active+clean; 665 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 767 B/s wr, 12 op/s 2024-06-03T03:25:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.360+0000 7f9e3ee00640 1 -- 172.21.15.117:0/2440316591 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e3810e0b0 msgr2=0x7f9e38110570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.360+0000 7f9e3ee00640 1 --2- 172.21.15.117:0/2440316591 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e3810e0b0 0x7f9e38110570 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9e30009ca0 tx=0x7f9e3002a8c0 comp rx=0 tx=0).stop 2024-06-03T03:25:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.360+0000 7f9e3ee00640 1 -- 172.21.15.117:0/2440316591 shutdown_connections 2024-06-03T03:25:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.360+0000 7f9e3ee00640 1 --2- 172.21.15.117:0/2440316591 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e3810e0b0 0x7f9e38110570 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.360+0000 7f9e3ee00640 1 --2- 172.21.15.117:0/2440316591 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e3810b5d0 0x7f9e3810da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:42.364 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.360+0000 7f9e3ee00640 1 --2- 172.21.15.117:0/2440316591 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9e38108c10 0x7f9e3810b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:42.364 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.360+0000 7f9e3ee00640 1 -- 172.21.15.117:0/2440316591 >> 172.21.15.117:0/2440316591 conn(0x7f9e3806ebd0 msgr2=0x7f9e38071010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:25:42.364 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.360+0000 7f9e3ee00640 1 -- 172.21.15.117:0/2440316591 shutdown_connections 2024-06-03T03:25:42.364 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.360+0000 7f9e3ee00640 1 -- 172.21.15.117:0/2440316591 wait complete. 2024-06-03T03:25:42.364 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.361+0000 7f9e3ee00640 1 Processor -- start 2024-06-03T03:25:42.367 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.361+0000 7f9e3ee00640 1 -- start start 2024-06-03T03:25:42.368 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.362+0000 7f9e3ee00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e38108c10 0x7f9e381a0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:42.368 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.362+0000 7f9e3ee00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e3810b5d0 0x7f9e381a0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:42.368 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.362+0000 7f9e3ee00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9e3810e0b0 0x7f9e381a79e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:42.368 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.362+0000 7f9e3ee00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9e38114de0 con 0x7f9e3810b5d0 2024-06-03T03:25:42.368 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.362+0000 7f9e3ee00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9e38114c60 con 0x7f9e3810e0b0 2024-06-03T03:25:42.368 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.362+0000 7f9e3ee00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9e38114f60 con 0x7f9e38108c10 2024-06-03T03:25:42.368 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.362+0000 7f9e2f400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e3810b5d0 0x7f9e381a0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:42.369 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.362+0000 7f9e3ce00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9e3810e0b0 0x7f9e381a79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:42.369 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.362+0000 7f9e2f400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e3810b5d0 0x7f9e381a0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:57632/0 (socket says 172.21.15.117:57632) 2024-06-03T03:25:42.369 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.362+0000 7f9e2f400640 1 -- 172.21.15.117:0/1050612317 learned_addr learned my addr 172.21.15.117:0/1050612317 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:25:42.369 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.362+0000 7f9e37e00640 1 --2- 172.21.15.117:0/1050612317 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e38108c10 0x7f9e381a0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:42.369 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.362+0000 7f9e2f400640 1 -- 172.21.15.117:0/1050612317 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e38108c10 msgr2=0x7f9e381a0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:42.369 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.362+0000 7f9e2f400640 1 --2- 172.21.15.117:0/1050612317 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e38108c10 0x7f9e381a0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:42.369 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.362+0000 7f9e2f400640 1 -- 172.21.15.117:0/1050612317 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9e3810e0b0 msgr2=0x7f9e381a79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:42.370 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.362+0000 7f9e2f400640 1 --2- 172.21.15.117:0/1050612317 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9e3810e0b0 0x7f9e381a79e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:42.370 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.363+0000 7f9e2f400640 1 -- 172.21.15.117:0/1050612317 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e381a8050 con 0x7f9e3810b5d0 2024-06-03T03:25:42.370 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.363+0000 7f9e3ce00640 1 --2- 172.21.15.117:0/1050612317 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9e3810e0b0 0x7f9e381a79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:25:42.370 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.363+0000 7f9e2f400640 1 --2- 172.21.15.117:0/1050612317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e3810b5d0 0x7f9e381a0970 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9e20005fd0 tx=0x7f9e2000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:25:42.370 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.363+0000 7f9e35600640 1 -- 172.21.15.117:0/1050612317 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e20017070 con 0x7f9e3810b5d0 2024-06-03T03:25:42.370 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.363+0000 7f9e35600640 1 -- 172.21.15.117:0/1050612317 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e20009c30 con 0x7f9e3810b5d0 2024-06-03T03:25:42.370 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.363+0000 7f9e35600640 1 -- 172.21.15.117:0/1050612317 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e20010460 con 0x7f9e3810b5d0 2024-06-03T03:25:42.370 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.363+0000 7f9e3ee00640 1 -- 172.21.15.117:0/1050612317 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e381a8340 con 0x7f9e3810b5d0 2024-06-03T03:25:42.371 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.363+0000 7f9e3ee00640 1 -- 172.21.15.117:0/1050612317 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e381a88e0 con 0x7f9e3810b5d0 2024-06-03T03:25:42.380 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.367+0000 7f9e41b84740 1 Processor -- start 2024-06-03T03:25:42.381 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.367+0000 7f9e35600640 1 -- 172.21.15.117:0/1050612317 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f9e20003e70 con 0x7f9e3810b5d0 2024-06-03T03:25:42.381 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.367+0000 7f9e41b84740 1 -- start start 2024-06-03T03:25:42.381 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.367+0000 7f9e41b84740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559d78106fe0 0x559d78107400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:42.381 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.367+0000 7f9e41b84740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d781079d0 0x559d7810c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:42.381 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.367+0000 7f9e41b84740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d7810cb40 0x559d7810efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:42.381 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.367+0000 7f9e41b84740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559d78066420 con 0x559d781079d0 2024-06-03T03:25:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.367+0000 7f9e41b84740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559d7800dc50 con 0x559d78106fe0 2024-06-03T03:25:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.367+0000 7f9e41b84740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559d780623e0 con 0x559d7810cb40 2024-06-03T03:25:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.368+0000 7f9e37e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d781079d0 0x559d7810c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.368+0000 7f9e3ce00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559d78106fe0 0x559d78107400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.368+0000 7f9e2f400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d7810cb40 0x559d7810efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.368+0000 7f9e3ce00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559d78106fe0 0x559d78107400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:41480/0 (socket says 172.21.15.117:41480) 2024-06-03T03:25:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.368+0000 7f9e37e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d781079d0 0x559d7810c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:57646/0 (socket says 172.21.15.117:57646) 2024-06-03T03:25:42.383 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.368+0000 7f9e3ce00640 1 -- 172.21.15.117:0/4030054896 learned_addr learned my addr 172.21.15.117:0/4030054896 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:25:42.383 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.368+0000 7f9e3ce00640 1 -- 172.21.15.117:0/4030054896 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d7810cb40 msgr2=0x559d7810efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:25:42.383 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.368+0000 7f9e3ce00640 1 --2- 172.21.15.117:0/4030054896 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d7810cb40 0x559d7810efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:42.383 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.368+0000 7f9e3ce00640 1 -- 172.21.15.117:0/4030054896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d781079d0 msgr2=0x559d7810c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:25:42.383 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.368+0000 7f9e3ce00640 1 --2- 172.21.15.117:0/4030054896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d781079d0 0x559d7810c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:42.383 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.368+0000 7f9e3ce00640 1 -- 172.21.15.117:0/4030054896 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d77d7a210 con 0x559d78106fe0 2024-06-03T03:25:42.384 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.368+0000 7f9e3ce00640 1 --2- 172.21.15.117:0/4030054896 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559d78106fe0 0x559d78107400 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9e30009ca0 tx=0x7f9e30005fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d1ce63414298352 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:25:42.384 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.369+0000 7f9e35600640 1 --2- 172.21.15.117:0/1050612317 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f9e1c0a2d50 0x7f9e1c0a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:42.384 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.369+0000 7f9e2d600640 1 -- 172.21.15.117:0/4030054896 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e30031650 con 0x559d78106fe0 2024-06-03T03:25:42.384 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.369+0000 7f9e2d600640 1 -- 172.21.15.117:0/4030054896 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e30036040 con 0x559d78106fe0 2024-06-03T03:25:42.384 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.369+0000 7f9e2d600640 1 -- 172.21.15.117:0/4030054896 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e30004790 con 0x559d78106fe0 2024-06-03T03:25:42.384 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.369+0000 7f9e35600640 1 -- 172.21.15.117:0/1050612317 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f9e200cf0d0 con 0x7f9e3810b5d0 2024-06-03T03:25:42.384 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.369+0000 7f9e37e00640 1 --2- 172.21.15.117:0/1050612317 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f9e1c0a2d50 0x7f9e1c0a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:42.384 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.369+0000 7f9e41b84740 1 -- 172.21.15.117:0/4030054896 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559d78106fe0 msgr2=0x559d78107400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:42.385 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.369+0000 7f9e41b84740 1 --2- 172.21.15.117:0/4030054896 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559d78106fe0 0x559d78107400 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9e30009ca0 tx=0x7f9e30005fa0 comp rx=0 tx=0).stop 2024-06-03T03:25:42.385 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.370+0000 7f9e41b84740 1 -- 172.21.15.117:0/4030054896 shutdown_connections 2024-06-03T03:25:42.385 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.370+0000 7f9e41b84740 1 --2- 172.21.15.117:0/4030054896 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d7810cb40 0x559d7810efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:42.385 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.370+0000 7f9e41b84740 1 --2- 172.21.15.117:0/4030054896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d781079d0 0x559d7810c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:42.385 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.370+0000 7f9e41b84740 1 --2- 172.21.15.117:0/4030054896 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559d78106fe0 0x559d78107400 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:42.385 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.370+0000 7f9e41b84740 1 -- 172.21.15.117:0/4030054896 >> 172.21.15.117:0/4030054896 conn(0x559d780caf80 msgr2=0x559d780fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:25:42.385 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.370+0000 7f9e41b84740 1 -- 172.21.15.117:0/4030054896 shutdown_connections 2024-06-03T03:25:42.385 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.370+0000 7f9e37e00640 1 --2- 172.21.15.117:0/1050612317 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f9e1c0a2d50 0x7f9e1c0a5210 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x559d78107dd0 tx=0x7f9e28005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:25:42.386 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.370+0000 7f9e41b84740 1 -- 172.21.15.117:0/4030054896 wait complete. 2024-06-03T03:25:42.386 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.370+0000 7f9e41b84740 1 -- start start 2024-06-03T03:25:42.386 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.370+0000 7f9e41b84740 1 Processor -- start 2024-06-03T03:25:42.386 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.371+0000 7f9e41b84740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d78106fe0 0x559d78111190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:42.386 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.371+0000 7f9e41b84740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559d781079d0 0x559d781116d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:42.386 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.371+0000 7f9e41b84740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d7810cb40 0x559d78118710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:42.386 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.371+0000 7f9e41b84740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559d7810f910 con 0x559d7810cb40 2024-06-03T03:25:42.386 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.371+0000 7f9e41b84740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559d780210b0 con 0x559d781079d0 2024-06-03T03:25:42.387 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.371+0000 7f9e41b84740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559d7810fa90 con 0x559d78106fe0 2024-06-03T03:25:42.387 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.371+0000 7f9e41b84740 10 client.0 _ll_register_callbacks cb 0x559d7800cf40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:25:42.387 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.371+0000 7f9e2f400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559d781079d0 0x559d781116d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:42.387 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.371+0000 7f9e37e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d78106fe0 0x559d78111190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:42.387 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.371+0000 7f9e3ce00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d7810cb40 0x559d78118710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:42.387 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.371+0000 7f9e37e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d78106fe0 0x559d78111190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:49336/0 (socket says 172.21.15.117:49336) 2024-06-03T03:25:42.387 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.371+0000 7f9e37e00640 1 -- 172.21.15.117:0/141875589 learned_addr learned my addr 172.21.15.117:0/141875589 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:25:42.388 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.372+0000 7f9e2f400640 1 -- 172.21.15.117:0/141875589 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d78106fe0 msgr2=0x559d78111190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:25:42.388 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.372+0000 7f9e2f400640 1 --2- 172.21.15.117:0/141875589 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d78106fe0 0x559d78111190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:42.388 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.372+0000 7f9e2f400640 1 -- 172.21.15.117:0/141875589 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d7810cb40 msgr2=0x559d78118710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:25:42.388 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.372+0000 7f9e2f400640 1 --2- 172.21.15.117:0/141875589 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d7810cb40 0x559d78118710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:42.388 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.372+0000 7f9e2f400640 1 -- 172.21.15.117:0/141875589 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d77e46620 con 0x559d781079d0 2024-06-03T03:25:42.388 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.372+0000 7f9e37e00640 1 --2- 172.21.15.117:0/141875589 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d78106fe0 0x559d78111190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:25:42.388 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.372+0000 7f9e3ce00640 1 --2- 172.21.15.117:0/141875589 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d7810cb40 0x559d78118710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:25:42.389 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.372+0000 7f9e2f400640 1 --2- 172.21.15.117:0/141875589 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559d781079d0 0x559d781116d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9e200cc6f0 tx=0x7f9e2000bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8047d643576a877e server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:25:42.389 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.372+0000 7f9e26000640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-06-03T03:25:42.389 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.372+0000 7f9e26000640 1 -- 172.21.15.117:0/141875589 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e2009ada0 con 0x559d781079d0 2024-06-03T03:25:42.389 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.372+0000 7f9e41b84740 1 -- 172.21.15.117:0/141875589 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e30036040 con 0x559d781079d0 2024-06-03T03:25:42.389 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.372+0000 7f9e26000640 1 -- 172.21.15.117:0/141875589 <== mon.2 v2:172.21.15.151:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f9e20097ab0 con 0x559d781079d0 2024-06-03T03:25:42.389 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.372+0000 7f9e26000640 1 -- 172.21.15.117:0/141875589 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f9e2009bb40 con 0x559d781079d0 2024-06-03T03:25:42.389 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.372+0000 7f9e41b84740 10 client.25310 fetch_fsmap learned FSMap version 31 2024-06-03T03:25:42.389 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.372+0000 7f9e41b84740 1 -- 172.21.15.117:0/141875589 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x559d7811bb40 con 0x559d781079d0 2024-06-03T03:25:42.390 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.373+0000 7f9e26000640 1 -- 172.21.15.117:0/141875589 <== mon.2 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e30036040 con 0x559d781079d0 2024-06-03T03:25:42.390 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.373+0000 7f9e26000640 1 -- 172.21.15.117:0/141875589 <== mon.2 v2:172.21.15.151:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f9e2009a890 con 0x559d781079d0 2024-06-03T03:25:42.390 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.373+0000 7f9e41b84740 10 client.25310 fetch_fsmap finished waiting for FSMap version 31 2024-06-03T03:25:42.390 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.373+0000 7f9e41b84740 10 client.25310 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-03T03:25:42.390 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.373+0000 7f9e41b84740 20 client.25310 populate_metadata read hostname 'smithi117' 2024-06-03T03:25:42.390 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.373+0000 7f9e41b84740 1 --2- 172.21.15.117:0/141875589 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x559d7811d200 0x559d7811f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:42.390 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.373+0000 7f9e41b84740 4 client.25310 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T03:25:42.391 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.373+0000 7f9e41b84740 1 -- 172.21.15.117:0/141875589 --> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d78073b50 con 0x559d7811d200 2024-06-03T03:25:42.391 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.373+0000 7f9e37e00640 1 --2- 172.21.15.117:0/141875589 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x559d7811d200 0x559d7811f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:42.391 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.374+0000 7f9e37e00640 1 --2- 172.21.15.117:0/141875589 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x559d7811d200 0x559d7811f5f0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28e8158f961f573d server_cookie=4c3e028f453d3f23 in_seq=0 out_seq=0 2024-06-03T03:25:42.391 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.374+0000 7f9e26000640 10 client.25310 ms_handle_connect on v2:172.21.15.183:6838/1439615083 2024-06-03T03:25:42.391 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.375+0000 7f9e26000640 1 -- 172.21.15.117:0/141875589 <== mds.0 v2:172.21.15.183:6838/1439615083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559d78073b50 con 0x559d7811d200 2024-06-03T03:25:42.391 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.375+0000 7f9e26000640 10 client.25310 handle_command_reply: tid=0 2024-06-03T03:25:42.391 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.375+0000 7f9e41b84740 1 client.25310 shutdown 2024-06-03T03:25:42.392 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.375+0000 7f9e41b84740 1 -- 172.21.15.117:0/141875589 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559d781079d0 msgr2=0x559d781116d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:42.392 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.375+0000 7f9e41b84740 1 --2- 172.21.15.117:0/141875589 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559d781079d0 0x559d781116d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9e200cc6f0 tx=0x7f9e2000bf60 comp rx=0 tx=0).stop 2024-06-03T03:25:42.392 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.375+0000 7f9e41b84740 1 -- 172.21.15.117:0/141875589 shutdown_connections 2024-06-03T03:25:42.392 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.375+0000 7f9e41b84740 1 --2- 172.21.15.117:0/141875589 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x559d7811d200 0x559d7811f5f0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:42.392 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.375+0000 7f9e41b84740 1 --2- 172.21.15.117:0/141875589 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d7810cb40 0x559d78118710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:42.392 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.375+0000 7f9e26000640 0 client.25310 ms_handle_reset on v2:172.21.15.183:6838/1439615083 2024-06-03T03:25:42.392 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.375+0000 7f9e41b84740 1 --2- 172.21.15.117:0/141875589 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559d781079d0 0x559d781116d0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:42.392 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.375+0000 7f9e41b84740 1 --2- 172.21.15.117:0/141875589 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d78106fe0 0x559d78111190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:42.393 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.375+0000 7f9e41b84740 1 -- 172.21.15.117:0/141875589 >> 172.21.15.117:0/141875589 conn(0x559d780caf80 msgr2=0x559d780fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:25:42.393 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.375+0000 7f9e41b84740 1 -- 172.21.15.117:0/141875589 shutdown_connections 2024-06-03T03:25:42.393 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.375+0000 7f9e41b84740 1 -- 172.21.15.117:0/141875589 wait complete. 2024-06-03T03:25:42.393 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.375+0000 7f9e41b84740 20 client.25310 trim_cache size 0 max 16384 2024-06-03T03:25:42.395 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.392+0000 7f9e41b84740 1 Processor -- start 2024-06-03T03:25:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.392+0000 7f9e41b84740 1 -- start start 2024-06-03T03:25:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.392+0000 7f9e41b84740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d78106fe0 0x559d780fe840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.392+0000 7f9e41b84740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559d781079d0 0x559d781186d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.392+0000 7f9e41b84740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d7810cb40 0x559d780f96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.392+0000 7f9e41b84740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9e381b0b00 con 0x559d7810cb40 2024-06-03T03:25:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.392+0000 7f9e41b84740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9e381b2640 con 0x559d781079d0 2024-06-03T03:25:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.392+0000 7f9e41b84740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9e381b11d0 con 0x559d78106fe0 2024-06-03T03:25:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.393+0000 7f9e3ce00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d7810cb40 0x559d780f96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.393+0000 7f9e37e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d78106fe0 0x559d780fe840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.393+0000 7f9e3ce00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d7810cb40 0x559d780f96a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:57668/0 (socket says 172.21.15.117:57668) 2024-06-03T03:25:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.393+0000 7f9e3ce00640 1 -- 172.21.15.117:0/2968573575 learned_addr learned my addr 172.21.15.117:0/2968573575 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:25:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.393+0000 7f9e2f400640 1 --2- 172.21.15.117:0/2968573575 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559d781079d0 0x559d781186d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.393+0000 7f9e3ce00640 1 -- 172.21.15.117:0/2968573575 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d78106fe0 msgr2=0x559d780fe840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:25:42.397 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.393+0000 7f9e3ce00640 1 --2- 172.21.15.117:0/2968573575 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d78106fe0 0x559d780fe840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:42.397 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.393+0000 7f9e3ce00640 1 -- 172.21.15.117:0/2968573575 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559d781079d0 msgr2=0x559d781186d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:25:42.397 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.393+0000 7f9e3ce00640 1 --2- 172.21.15.117:0/2968573575 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559d781079d0 0x559d781186d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:42.397 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.393+0000 7f9e3ce00640 1 -- 172.21.15.117:0/2968573575 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d7810bfc0 con 0x559d7810cb40 2024-06-03T03:25:42.397 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.393+0000 7f9e2f400640 1 --2- 172.21.15.117:0/2968573575 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559d781079d0 0x559d781186d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:25:42.397 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.394+0000 7f9e3ce00640 1 --2- 172.21.15.117:0/2968573575 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d7810cb40 0x559d780f96a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9e30009ca0 tx=0x7f9e3002a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d03c9e40b684263b server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:25:42.397 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.394+0000 7f9e2cc00640 1 -- 172.21.15.117:0/2968573575 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e30053070 con 0x559d7810cb40 2024-06-03T03:25:42.397 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.394+0000 7f9e2cc00640 1 -- 172.21.15.117:0/2968573575 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e30036040 con 0x559d7810cb40 2024-06-03T03:25:42.397 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.394+0000 7f9e2cc00640 1 -- 172.21.15.117:0/2968573575 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e3002fbe0 con 0x559d7810cb40 2024-06-03T03:25:42.400 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.395+0000 7f9e41b84740 1 -- 172.21.15.117:0/2968573575 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d7810cb40 msgr2=0x559d780f96a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:42.400 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.395+0000 7f9e41b84740 1 --2- 172.21.15.117:0/2968573575 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d7810cb40 0x559d780f96a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9e30009ca0 tx=0x7f9e3002a8c0 comp rx=0 tx=0).stop 2024-06-03T03:25:42.400 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.395+0000 7f9e41b84740 1 -- 172.21.15.117:0/2968573575 shutdown_connections 2024-06-03T03:25:42.401 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.395+0000 7f9e41b84740 1 --2- 172.21.15.117:0/2968573575 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d7810cb40 0x559d780f96a0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:42.401 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.395+0000 7f9e41b84740 1 --2- 172.21.15.117:0/2968573575 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559d781079d0 0x559d781186d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:42.401 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.395+0000 7f9e41b84740 1 --2- 172.21.15.117:0/2968573575 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d78106fe0 0x559d780fe840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:42.401 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.395+0000 7f9e41b84740 1 -- 172.21.15.117:0/2968573575 >> 172.21.15.117:0/2968573575 conn(0x559d780caf80 msgr2=0x559d780fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:25:42.401 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.395+0000 7f9e41b84740 1 -- 172.21.15.117:0/2968573575 shutdown_connections 2024-06-03T03:25:42.401 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.395+0000 7f9e41b84740 1 -- 172.21.15.117:0/2968573575 wait complete. 2024-06-03T03:25:42.401 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.395+0000 7f9e41b84740 1 -- start start 2024-06-03T03:25:42.401 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.395+0000 7f9e41b84740 1 Processor -- start 2024-06-03T03:25:42.401 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.396+0000 7f9e41b84740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d78106fe0 0x559d781155a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:42.401 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.396+0000 7f9e41b84740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d781079d0 0x559d78115ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:42.401 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.396+0000 7f9e41b84740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559d7810cb40 0x559d78185e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:42.401 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.396+0000 7f9e41b84740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9e381b1f70 con 0x559d781079d0 2024-06-03T03:25:42.401 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.396+0000 7f9e41b84740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9e381b18a0 con 0x559d7810cb40 2024-06-03T03:25:42.402 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.396+0000 7f9e41b84740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9e381aeae0 con 0x559d78106fe0 2024-06-03T03:25:42.402 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.396+0000 7f9e41b84740 10 client.0 _ll_register_callbacks cb 0x7f9e2000df10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:25:42.402 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.396+0000 7f9e37e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d78106fe0 0x559d781155a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:42.402 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.396+0000 7f9e3ce00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559d7810cb40 0x559d78185e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:42.402 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.396+0000 7f9e3ce00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559d7810cb40 0x559d78185e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:41506/0 (socket says 172.21.15.117:41506) 2024-06-03T03:25:42.402 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.396+0000 7f9e37e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d78106fe0 0x559d781155a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:49356/0 (socket says 172.21.15.117:49356) 2024-06-03T03:25:42.402 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.396+0000 7f9e3ce00640 1 -- 172.21.15.117:0/662067889 learned_addr learned my addr 172.21.15.117:0/662067889 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:25:42.403 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.396+0000 7f9e2f400640 1 --2- 172.21.15.117:0/662067889 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d781079d0 0x559d78115ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:42.403 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.397+0000 7f9e3ce00640 1 -- 172.21.15.117:0/662067889 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d78106fe0 msgr2=0x559d781155a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:25:42.403 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.397+0000 7f9e3ce00640 1 --2- 172.21.15.117:0/662067889 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d78106fe0 0x559d781155a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:42.403 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.397+0000 7f9e3ce00640 1 -- 172.21.15.117:0/662067889 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d781079d0 msgr2=0x559d78115ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:25:42.403 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.397+0000 7f9e3ce00640 1 --2- 172.21.15.117:0/662067889 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d781079d0 0x559d78115ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:42.403 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.397+0000 7f9e3ce00640 1 -- 172.21.15.117:0/662067889 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d78186390 con 0x559d7810cb40 2024-06-03T03:25:42.403 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.397+0000 7f9e37e00640 1 --2- 172.21.15.117:0/662067889 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d78106fe0 0x559d781155a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:25:42.403 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.397+0000 7f9e3ce00640 1 --2- 172.21.15.117:0/662067889 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559d7810cb40 0x559d78185e50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9e30005e30 tx=0x7f9e30005fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5c3eaefcbdf8e4 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:25:42.403 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.397+0000 7f9e26000640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-06-03T03:25:42.403 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.397+0000 7f9e26000640 1 -- 172.21.15.117:0/662067889 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e3005d070 con 0x559d7810cb40 2024-06-03T03:25:42.403 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.397+0000 7f9e26000640 1 -- 172.21.15.117:0/662067889 <== mon.2 v2:172.21.15.151:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f9e3002fd70 con 0x559d7810cb40 2024-06-03T03:25:42.403 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.397+0000 7f9e41b84740 1 -- 172.21.15.117:0/662067889 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e30036040 con 0x559d7810cb40 2024-06-03T03:25:42.403 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.398+0000 7f9e26000640 1 -- 172.21.15.117:0/662067889 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f9e30069050 con 0x559d7810cb40 2024-06-03T03:25:42.404 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.398+0000 7f9e41b84740 10 client.25316 fetch_fsmap learned FSMap version 31 2024-06-03T03:25:42.404 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.398+0000 7f9e41b84740 1 -- 172.21.15.117:0/662067889 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x559d78187540 con 0x559d7810cb40 2024-06-03T03:25:42.404 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.398+0000 7f9e26000640 1 -- 172.21.15.117:0/662067889 <== mon.2 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e30036040 con 0x559d7810cb40 2024-06-03T03:25:42.404 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.398+0000 7f9e26000640 1 -- 172.21.15.117:0/662067889 <== mon.2 v2:172.21.15.151:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f9e30044070 con 0x559d7810cb40 2024-06-03T03:25:42.404 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.398+0000 7f9e41b84740 10 client.25316 fetch_fsmap finished waiting for FSMap version 31 2024-06-03T03:25:42.404 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.398+0000 7f9e41b84740 10 client.25316 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-03T03:25:42.404 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.398+0000 7f9e41b84740 20 client.25316 populate_metadata read hostname 'smithi117' 2024-06-03T03:25:42.404 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.398+0000 7f9e41b84740 1 --2- 172.21.15.117:0/662067889 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x559d78187be0 0x559d7818a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:42.404 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.398+0000 7f9e41b84740 4 client.25316 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-03T03:25:42.404 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.398+0000 7f9e41b84740 1 -- 172.21.15.117:0/662067889 --> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559d780f9d80 con 0x559d78187be0 2024-06-03T03:25:42.404 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.398+0000 7f9e37e00640 1 --2- 172.21.15.117:0/662067889 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x559d78187be0 0x559d7818a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:42.404 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.399+0000 7f9e37e00640 1 --2- 172.21.15.117:0/662067889 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x559d78187be0 0x559d7818a000 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efa6527ae24c9083 server_cookie=798c826f095e4602 in_seq=0 out_seq=0 2024-06-03T03:25:42.404 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.399+0000 7f9e26000640 10 client.25316 ms_handle_connect on v2:172.21.15.183:6838/1439615083 2024-06-03T03:25:42.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:25:42 smithi151 ceph-mon[37043]: pgmap v517: 129 pgs: 129 active+clean; 665 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 9 op/s 2024-06-03T03:25:42.414 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.412+0000 7f9e26000640 1 -- 172.21.15.117:0/662067889 <== mds.0 v2:172.21.15.183:6838/1439615083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x559d780f9d80 con 0x559d78187be0 2024-06-03T03:25:42.414 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.412+0000 7f9e26000640 10 client.25316 handle_command_reply: tid=0 2024-06-03T03:25:42.414 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.412+0000 7f9e41b84740 1 client.25316 shutdown 2024-06-03T03:25:42.414 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.412+0000 7f9e41b84740 1 -- 172.21.15.117:0/662067889 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559d7810cb40 msgr2=0x559d78185e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:42.414 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.412+0000 7f9e41b84740 1 --2- 172.21.15.117:0/662067889 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559d7810cb40 0x559d78185e50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9e30005e30 tx=0x7f9e30005fa0 comp rx=0 tx=0).stop 2024-06-03T03:25:42.415 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.412+0000 7f9e41b84740 1 -- 172.21.15.117:0/662067889 shutdown_connections 2024-06-03T03:25:42.415 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.412+0000 7f9e41b84740 1 --2- 172.21.15.117:0/662067889 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x559d78187be0 0x559d7818a000 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:42.415 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.412+0000 7f9e41b84740 1 --2- 172.21.15.117:0/662067889 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559d7810cb40 0x559d78185e50 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:42.415 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.412+0000 7f9e41b84740 1 --2- 172.21.15.117:0/662067889 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d781079d0 0x559d78115ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:42.415 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.412+0000 7f9e41b84740 1 --2- 172.21.15.117:0/662067889 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d78106fe0 0x559d781155a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:42.415 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.412+0000 7f9e26000640 0 client.25316 ms_handle_reset on v2:172.21.15.183:6838/1439615083 2024-06-03T03:25:42.415 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.412+0000 7f9e41b84740 1 -- 172.21.15.117:0/662067889 >> 172.21.15.117:0/662067889 conn(0x559d780caf80 msgr2=0x559d78101540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:25:42.415 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.412+0000 7f9e41b84740 1 -- 172.21.15.117:0/662067889 shutdown_connections 2024-06-03T03:25:42.415 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.412+0000 7f9e41b84740 1 -- 172.21.15.117:0/662067889 wait complete. 2024-06-03T03:25:42.415 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.412+0000 7f9e41b84740 20 client.25316 trim_cache size 0 max 16384 2024-06-03T03:25:42.416 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:{ 2024-06-03T03:25:42.416 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "return_code": 0 2024-06-03T03:25:42.416 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:} 2024-06-03T03:25:42.416 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.413+0000 7f9e3ee00640 1 -- 172.21.15.117:0/1050612317 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f9e1c0a2d50 msgr2=0x7f9e1c0a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.413+0000 7f9e3ee00640 1 --2- 172.21.15.117:0/1050612317 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f9e1c0a2d50 0x7f9e1c0a5210 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x559d78107dd0 tx=0x7f9e28005f90 comp rx=0 tx=0).stop 2024-06-03T03:25:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.413+0000 7f9e3ee00640 1 -- 172.21.15.117:0/1050612317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e3810b5d0 msgr2=0x7f9e381a0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.413+0000 7f9e3ee00640 1 --2- 172.21.15.117:0/1050612317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e3810b5d0 0x7f9e381a0970 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9e20005fd0 tx=0x7f9e2000dde0 comp rx=0 tx=0).stop 2024-06-03T03:25:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.413+0000 7f9e3ee00640 1 -- 172.21.15.117:0/1050612317 shutdown_connections 2024-06-03T03:25:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.413+0000 7f9e3ee00640 1 --2- 172.21.15.117:0/1050612317 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f9e1c0a2d50 0x7f9e1c0a5210 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.413+0000 7f9e3ee00640 1 --2- 172.21.15.117:0/1050612317 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9e3810e0b0 0x7f9e381a79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.413+0000 7f9e3ee00640 1 --2- 172.21.15.117:0/1050612317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e3810b5d0 0x7f9e381a0970 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.413+0000 7f9e3ee00640 1 --2- 172.21.15.117:0/1050612317 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e38108c10 0x7f9e381a0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.413+0000 7f9e3ee00640 1 -- 172.21.15.117:0/1050612317 >> 172.21.15.117:0/1050612317 conn(0x7f9e3806ebd0 msgr2=0x7f9e38111f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:25:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.414+0000 7f9e3ee00640 1 -- 172.21.15.117:0/1050612317 shutdown_connections 2024-06-03T03:25:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:42.414+0000 7f9e3ee00640 1 -- 172.21.15.117:0/1050612317 wait complete. 2024-06-03T03:25:42.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:25:42 smithi117 ceph-mon[31649]: pgmap v517: 129 pgs: 129 active+clean; 665 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 9 op/s 2024-06-03T03:25:42.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:25:42 smithi183 ceph-mon[37326]: pgmap v517: 129 pgs: 129 active+clean; 665 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 9 op/s 2024-06-03T03:25:43.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:25:43 smithi151 ceph-mon[37043]: scrub summary: idle 2024-06-03T03:25:43.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:25:43 smithi117 ceph-mon[31649]: scrub summary: idle 2024-06-03T03:25:43.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:25:43 smithi183 ceph-mon[37326]: scrub summary: idle 2024-06-03T03:25:44.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:25:44 smithi151 ceph-mon[37043]: pgmap v518: 129 pgs: 129 active+clean; 665 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 852 B/s wr, 12 op/s 2024-06-03T03:25:44.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:25:44 smithi117 ceph-mon[31649]: pgmap v518: 129 pgs: 129 active+clean; 665 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 852 B/s wr, 12 op/s 2024-06-03T03:25:44.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:25:44 smithi183 ceph-mon[37326]: pgmap v518: 129 pgs: 129 active+clean; 665 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 852 B/s wr, 12 op/s 2024-06-03T03:25:45.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:25:45 smithi151 ceph-mon[37043]: pgmap v519: 129 pgs: 129 active+clean; 665 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 11 op/s 2024-06-03T03:25:45.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:25:45 smithi117 ceph-mon[31649]: pgmap v519: 129 pgs: 129 active+clean; 665 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 11 op/s 2024-06-03T03:25:45.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:25:45 smithi183 ceph-mon[37326]: pgmap v519: 129 pgs: 129 active+clean; 665 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 11 op/s 2024-06-03T03:25:48.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:25:48 smithi151 ceph-mon[37043]: pgmap v520: 129 pgs: 129 active+clean; 665 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 852 B/s wr, 12 op/s 2024-06-03T03:25:48.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:25:48 smithi117 ceph-mon[31649]: pgmap v520: 129 pgs: 129 active+clean; 665 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 852 B/s wr, 12 op/s 2024-06-03T03:25:48.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:25:48 smithi183 ceph-mon[37326]: pgmap v520: 129 pgs: 129 active+clean; 665 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 852 B/s wr, 12 op/s 2024-06-03T03:25:50.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:25:50 smithi151 ceph-mon[37043]: pgmap v521: 129 pgs: 129 active+clean; 666 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.4 KiB/s wr, 16 op/s 2024-06-03T03:25:50.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:25:50 smithi117 ceph-mon[31649]: pgmap v521: 129 pgs: 129 active+clean; 666 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.4 KiB/s wr, 16 op/s 2024-06-03T03:25:50.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:25:50 smithi183 ceph-mon[37326]: pgmap v521: 129 pgs: 129 active+clean; 666 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.4 KiB/s wr, 16 op/s 2024-06-03T03:25:52.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:25:52 smithi151 ceph-mon[37043]: pgmap v522: 129 pgs: 129 active+clean; 666 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.4 KiB/s wr, 13 op/s 2024-06-03T03:25:52.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:25:52 smithi117 ceph-mon[31649]: pgmap v522: 129 pgs: 129 active+clean; 666 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.4 KiB/s wr, 13 op/s 2024-06-03T03:25:52.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:25:52 smithi183 ceph-mon[37326]: pgmap v522: 129 pgs: 129 active+clean; 666 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.4 KiB/s wr, 13 op/s 2024-06-03T03:25:53.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:25:53 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:25:53.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:25:53 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:25:53.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:25:53 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:25:54.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:25:54 smithi151 ceph-mon[37043]: pgmap v523: 129 pgs: 129 active+clean; 666 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.4 KiB/s wr, 15 op/s 2024-06-03T03:25:54.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:25:54 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:25:54.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:25:54 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:25:54.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:25:54 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:25:54.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:25:54 smithi117 ceph-mon[31649]: pgmap v523: 129 pgs: 129 active+clean; 666 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.4 KiB/s wr, 15 op/s 2024-06-03T03:25:54.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:25:54 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:25:54.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:25:54 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:25:54.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:25:54 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:25:54.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:25:54 smithi183 ceph-mon[37326]: pgmap v523: 129 pgs: 129 active+clean; 666 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.4 KiB/s wr, 15 op/s 2024-06-03T03:25:54.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:25:54 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:25:54.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:25:54 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:25:54.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:25:54 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:25:55.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:25:55 smithi151 ceph-mon[37043]: pgmap v524: 129 pgs: 129 active+clean; 666 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-06-03T03:25:55.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:25:55 smithi117 ceph-mon[31649]: pgmap v524: 129 pgs: 129 active+clean; 666 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-06-03T03:25:55.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:25:55 smithi183 ceph-mon[37326]: pgmap v524: 129 pgs: 129 active+clean; 666 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-06-03T03:25:55.557 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.556+0000 7f6379200640 1 -- 172.21.15.117:0/4205828101 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63741057d0 msgr2=0x7f637410c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:55.557 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.556+0000 7f6379200640 1 --2- 172.21.15.117:0/4205828101 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63741057d0 0x7f637410c060 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f636c009ca0 tx=0x7f636c02a8c0 comp rx=0 tx=0).stop 2024-06-03T03:25:55.557 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.556+0000 7f6379200640 1 -- 172.21.15.117:0/4205828101 shutdown_connections 2024-06-03T03:25:55.557 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.556+0000 7f6379200640 1 --2- 172.21.15.117:0/4205828101 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63741057d0 0x7f637410c060 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:55.557 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.556+0000 7f6379200640 1 --2- 172.21.15.117:0/4205828101 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6374104e10 0x7f6374105290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:55.558 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.556+0000 7f6379200640 1 --2- 172.21.15.117:0/4205828101 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6374103c20 0x7f6374104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:55.558 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.556+0000 7f6379200640 1 -- 172.21.15.117:0/4205828101 >> 172.21.15.117:0/4205828101 conn(0x7f63740ff190 msgr2=0x7f63741015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:25:55.558 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.556+0000 7f6379200640 1 -- 172.21.15.117:0/4205828101 shutdown_connections 2024-06-03T03:25:55.558 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.556+0000 7f6379200640 1 -- 172.21.15.117:0/4205828101 wait complete. 2024-06-03T03:25:55.560 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.557+0000 7f6379200640 1 Processor -- start 2024-06-03T03:25:55.560 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.558+0000 7f6379200640 1 -- start start 2024-06-03T03:25:55.560 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.558+0000 7f6379200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6374103c20 0x7f6374199e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:55.560 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.558+0000 7f6379200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6374104e10 0x7f637419a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:55.560 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.558+0000 7f6379200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63741057d0 0x7f63741a13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:55.560 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.558+0000 7f6379200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f637410f8f0 con 0x7f63741057d0 2024-06-03T03:25:55.560 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.558+0000 7f6379200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f637410f770 con 0x7f6374104e10 2024-06-03T03:25:55.560 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.558+0000 7f6379200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f637410fa70 con 0x7f6374103c20 2024-06-03T03:25:55.561 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.558+0000 7f6371e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6374103c20 0x7f6374199e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:55.561 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.558+0000 7f6371e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6374103c20 0x7f6374199e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:46268/0 (socket says 172.21.15.117:46268) 2024-06-03T03:25:55.561 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.558+0000 7f6371e00640 1 -- 172.21.15.117:0/3025227368 learned_addr learned my addr 172.21.15.117:0/3025227368 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:25:55.561 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.558+0000 7f6372800640 1 --2- 172.21.15.117:0/3025227368 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63741057d0 0x7f63741a13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:55.561 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.558+0000 7f6369400640 1 --2- 172.21.15.117:0/3025227368 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6374104e10 0x7f637419a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:55.561 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.559+0000 7f6371e00640 1 -- 172.21.15.117:0/3025227368 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6374104e10 msgr2=0x7f637419a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:55.561 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.559+0000 7f6371e00640 1 --2- 172.21.15.117:0/3025227368 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6374104e10 0x7f637419a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:55.561 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.559+0000 7f6371e00640 1 -- 172.21.15.117:0/3025227368 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63741057d0 msgr2=0x7f63741a13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:55.561 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.559+0000 7f6371e00640 1 --2- 172.21.15.117:0/3025227368 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63741057d0 0x7f63741a13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:55.561 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.559+0000 7f6371e00640 1 -- 172.21.15.117:0/3025227368 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63741a1a30 con 0x7f6374103c20 2024-06-03T03:25:55.561 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.559+0000 7f6371e00640 1 --2- 172.21.15.117:0/3025227368 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6374103c20 0x7f6374199e10 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f636400e9c0 tx=0x7f636400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:25:55.562 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.559+0000 7f636b400640 1 -- 172.21.15.117:0/3025227368 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f636400cd80 con 0x7f6374103c20 2024-06-03T03:25:55.562 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.559+0000 7f636b400640 1 -- 172.21.15.117:0/3025227368 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63640035f0 con 0x7f6374103c20 2024-06-03T03:25:55.562 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.559+0000 7f6379200640 1 -- 172.21.15.117:0/3025227368 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63741a1d20 con 0x7f6374103c20 2024-06-03T03:25:55.562 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.559+0000 7f636b400640 1 -- 172.21.15.117:0/3025227368 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6364010690 con 0x7f6374103c20 2024-06-03T03:25:55.562 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.559+0000 7f6379200640 1 -- 172.21.15.117:0/3025227368 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63741a21c0 con 0x7f6374103c20 2024-06-03T03:25:55.563 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.563+0000 7f637c16a740 1 Processor -- start 2024-06-03T03:25:55.563 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.563+0000 7f636b400640 1 -- 172.21.15.117:0/3025227368 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f63640037a0 con 0x7f6374103c20 2024-06-03T03:25:55.563 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.563+0000 7f637c16a740 1 -- start start 2024-06-03T03:25:55.566 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.563+0000 7f637c16a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573496fdbd0 0x5573496fdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:55.566 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.563+0000 7f637c16a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5573496fe5c0 0x5573497030c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:55.566 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.563+0000 7f637c16a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557349703730 0x557349705bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:55.566 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.563+0000 7f637c16a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557349661420 con 0x5573496fdbd0 2024-06-03T03:25:55.567 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.563+0000 7f637c16a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x557349608c50 con 0x5573496fe5c0 2024-06-03T03:25:55.567 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.563+0000 7f637c16a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55734965d3e0 con 0x557349703730 2024-06-03T03:25:55.567 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.564+0000 7f6372800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573496fdbd0 0x5573496fdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:55.567 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.564+0000 7f6372800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573496fdbd0 0x5573496fdff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:45012/0 (socket says 172.21.15.117:45012) 2024-06-03T03:25:55.567 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.564+0000 7f6372800640 1 -- 172.21.15.117:0/374027985 learned_addr learned my addr 172.21.15.117:0/374027985 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:25:55.567 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.564+0000 7f6371e00640 1 --2- 172.21.15.117:0/374027985 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5573496fe5c0 0x5573497030c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:55.567 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.564+0000 7f6369400640 1 --2- 172.21.15.117:0/374027985 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557349703730 0x557349705bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:55.567 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.564+0000 7f6372800640 1 -- 172.21.15.117:0/374027985 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557349703730 msgr2=0x557349705bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:25:55.567 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.564+0000 7f6372800640 1 --2- 172.21.15.117:0/374027985 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557349703730 0x557349705bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:55.567 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.564+0000 7f6372800640 1 -- 172.21.15.117:0/374027985 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5573496fe5c0 msgr2=0x5573497030c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:25:55.567 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.564+0000 7f6372800640 1 --2- 172.21.15.117:0/374027985 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5573496fe5c0 0x5573497030c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:55.567 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.564+0000 7f6372800640 1 -- 172.21.15.117:0/374027985 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557349375210 con 0x5573496fdbd0 2024-06-03T03:25:55.568 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.564+0000 7f6372800640 1 --2- 172.21.15.117:0/374027985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573496fdbd0 0x5573496fdff0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f636c009ca0 tx=0x7f636c0093c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a383ec6d4620d36c server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:25:55.568 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.565+0000 7f636b400640 1 --2- 172.21.15.117:0/3025227368 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f63580a2cd0 0x7f63580a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:55.568 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.565+0000 7f6363400640 1 -- 172.21.15.117:0/374027985 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f636c031650 con 0x5573496fdbd0 2024-06-03T03:25:55.568 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.565+0000 7f6363400640 1 -- 172.21.15.117:0/374027985 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f636c036030 con 0x5573496fdbd0 2024-06-03T03:25:55.568 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.565+0000 7f6363400640 1 -- 172.21.15.117:0/374027985 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f636c004700 con 0x5573496fdbd0 2024-06-03T03:25:55.568 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.565+0000 7f636b400640 1 -- 172.21.15.117:0/3025227368 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f63640cf910 con 0x7f6374103c20 2024-06-03T03:25:55.568 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.565+0000 7f6371e00640 1 --2- 172.21.15.117:0/3025227368 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f63580a2cd0 0x7f63580a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:55.568 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.565+0000 7f637c16a740 1 -- 172.21.15.117:0/374027985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573496fdbd0 msgr2=0x5573496fdff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:55.568 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.565+0000 7f637c16a740 1 --2- 172.21.15.117:0/374027985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573496fdbd0 0x5573496fdff0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f636c009ca0 tx=0x7f636c0093c0 comp rx=0 tx=0).stop 2024-06-03T03:25:55.568 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.566+0000 7f6371e00640 1 --2- 172.21.15.117:0/3025227368 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f63580a2cd0 0x7f63580a5190 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x5573496fe9c0 tx=0x7f63640d5530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:25:55.568 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.566+0000 7f637c16a740 1 -- 172.21.15.117:0/374027985 shutdown_connections 2024-06-03T03:25:55.568 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.566+0000 7f637c16a740 1 --2- 172.21.15.117:0/374027985 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557349703730 0x557349705bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:55.568 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.566+0000 7f637c16a740 1 --2- 172.21.15.117:0/374027985 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5573496fe5c0 0x5573497030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:55.569 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.566+0000 7f637c16a740 1 --2- 172.21.15.117:0/374027985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573496fdbd0 0x5573496fdff0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:55.569 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.566+0000 7f637c16a740 1 -- 172.21.15.117:0/374027985 >> 172.21.15.117:0/374027985 conn(0x5573496c5f80 msgr2=0x5573496f9650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:25:55.569 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.566+0000 7f637c16a740 1 -- 172.21.15.117:0/374027985 shutdown_connections 2024-06-03T03:25:55.569 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.566+0000 7f637c16a740 1 -- 172.21.15.117:0/374027985 wait complete. 2024-06-03T03:25:55.573 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.566+0000 7f637c16a740 1 -- start start 2024-06-03T03:25:55.573 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.566+0000 7f637c16a740 1 Processor -- start 2024-06-03T03:25:55.573 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.567+0000 7f637c16a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573496fdbd0 0x557349707cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:55.573 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.567+0000 7f637c16a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5573496fe5c0 0x557349708230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:55.573 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.567+0000 7f637c16a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557349703730 0x55734970f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:55.573 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.567+0000 7f637c16a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557349706500 con 0x5573496fdbd0 2024-06-03T03:25:55.573 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.567+0000 7f637c16a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55734961c0b0 con 0x5573496fe5c0 2024-06-03T03:25:55.573 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.567+0000 7f637c16a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557349706680 con 0x557349703730 2024-06-03T03:25:55.573 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.567+0000 7f637c16a740 10 client.0 _ll_register_callbacks cb 0x557349607f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:25:55.573 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.567+0000 7f6372800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573496fdbd0 0x557349707cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:55.573 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.567+0000 7f6372800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573496fdbd0 0x557349707cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:45016/0 (socket says 172.21.15.117:45016) 2024-06-03T03:25:55.574 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.567+0000 7f6372800640 1 -- 172.21.15.117:0/3477751167 learned_addr learned my addr 172.21.15.117:0/3477751167 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:25:55.574 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.567+0000 7f6369400640 1 --2- 172.21.15.117:0/3477751167 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5573496fe5c0 0x557349708230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:55.574 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.567+0000 7f6372800640 1 --2- 172.21.15.117:0/3477751167 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557349703730 0x55734970f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:55.574 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.568+0000 7f6372800640 1 -- 172.21.15.117:0/3477751167 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557349703730 msgr2=0x55734970f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:25:55.574 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.568+0000 7f6372800640 1 --2- 172.21.15.117:0/3477751167 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557349703730 0x55734970f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:55.574 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.568+0000 7f6372800640 1 -- 172.21.15.117:0/3477751167 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5573496fe5c0 msgr2=0x557349708230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:25:55.574 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.568+0000 7f6372800640 1 --2- 172.21.15.117:0/3477751167 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5573496fe5c0 0x557349708230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:55.574 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.568+0000 7f6372800640 1 -- 172.21.15.117:0/3477751167 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557349441620 con 0x5573496fdbd0 2024-06-03T03:25:55.574 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.568+0000 7f6369400640 1 --2- 172.21.15.117:0/3477751167 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5573496fe5c0 0x557349708230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:25:55.575 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.568+0000 7f6372800640 1 --2- 172.21.15.117:0/3477751167 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573496fdbd0 0x557349707cf0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f636c005c40 tx=0x7f636c005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c64acabd1760ad85 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:25:55.575 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.568+0000 7f6357e00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-06-03T03:25:55.575 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.568+0000 7f6357e00640 1 -- 172.21.15.117:0/3477751167 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f636c058060 con 0x5573496fdbd0 2024-06-03T03:25:55.575 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.568+0000 7f6357e00640 1 -- 172.21.15.117:0/3477751167 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f636c003b70 con 0x5573496fdbd0 2024-06-03T03:25:55.575 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.568+0000 7f637c16a740 1 -- 172.21.15.117:0/3477751167 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f636c036030 con 0x5573496fdbd0 2024-06-03T03:25:55.575 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.568+0000 7f6357e00640 1 -- 172.21.15.117:0/3477751167 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f636c031890 con 0x5573496fdbd0 2024-06-03T03:25:55.575 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.568+0000 7f637c16a740 10 client.15219 fetch_fsmap learned FSMap version 31 2024-06-03T03:25:55.575 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.568+0000 7f637c16a740 1 -- 172.21.15.117:0/3477751167 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x557349706500 con 0x5573496fdbd0 2024-06-03T03:25:55.575 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.569+0000 7f6357e00640 1 -- 172.21.15.117:0/3477751167 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f636c036030 con 0x5573496fdbd0 2024-06-03T03:25:55.575 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.569+0000 7f6357e00640 1 -- 172.21.15.117:0/3477751167 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f636c043070 con 0x5573496fdbd0 2024-06-03T03:25:55.575 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.569+0000 7f637c16a740 10 client.15219 fetch_fsmap finished waiting for FSMap version 31 2024-06-03T03:25:55.576 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.569+0000 7f637c16a740 10 client.15219 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-03T03:25:55.576 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.569+0000 7f637c16a740 20 client.15219 populate_metadata read hostname 'smithi117' 2024-06-03T03:25:55.576 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.569+0000 7f637c16a740 1 --2- 172.21.15.117:0/3477751167 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x5573497139e0 0x557349715dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:55.576 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.569+0000 7f637c16a740 4 client.15219 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T03:25:55.576 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.569+0000 7f637c16a740 1 -- 172.21.15.117:0/3477751167 --> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55734966eb50 con 0x5573497139e0 2024-06-03T03:25:55.576 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.569+0000 7f6369400640 1 --2- 172.21.15.117:0/3477751167 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x5573497139e0 0x557349715dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:55.576 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.570+0000 7f6369400640 1 --2- 172.21.15.117:0/3477751167 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x5573497139e0 0x557349715dd0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed961751fb55798b server_cookie=c914bb75bbeeaf36 in_seq=0 out_seq=0 2024-06-03T03:25:55.576 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.570+0000 7f6357e00640 10 client.15219 ms_handle_connect on v2:172.21.15.183:6838/1439615083 2024-06-03T03:25:55.576 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.571+0000 7f6357e00640 1 -- 172.21.15.117:0/3477751167 <== mds.0 v2:172.21.15.183:6838/1439615083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55734966eb50 con 0x5573497139e0 2024-06-03T03:25:55.576 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.571+0000 7f6357e00640 10 client.15219 handle_command_reply: tid=0 2024-06-03T03:25:55.576 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.571+0000 7f637c16a740 1 client.15219 shutdown 2024-06-03T03:25:55.576 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.571+0000 7f637c16a740 1 -- 172.21.15.117:0/3477751167 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573496fdbd0 msgr2=0x557349707cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:55.577 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.571+0000 7f637c16a740 1 --2- 172.21.15.117:0/3477751167 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573496fdbd0 0x557349707cf0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f636c005c40 tx=0x7f636c005bd0 comp rx=0 tx=0).stop 2024-06-03T03:25:55.577 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.571+0000 7f637c16a740 1 -- 172.21.15.117:0/3477751167 shutdown_connections 2024-06-03T03:25:55.577 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.571+0000 7f637c16a740 1 --2- 172.21.15.117:0/3477751167 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x5573497139e0 0x557349715dd0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:55.577 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.571+0000 7f637c16a740 1 --2- 172.21.15.117:0/3477751167 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557349703730 0x55734970f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:55.577 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.571+0000 7f637c16a740 1 --2- 172.21.15.117:0/3477751167 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5573496fe5c0 0x557349708230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:55.577 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.571+0000 7f637c16a740 1 --2- 172.21.15.117:0/3477751167 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573496fdbd0 0x557349707cf0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:55.577 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.571+0000 7f637c16a740 1 -- 172.21.15.117:0/3477751167 >> 172.21.15.117:0/3477751167 conn(0x5573496c5f80 msgr2=0x5573496f8810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:25:55.577 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.571+0000 7f6357e00640 0 client.15219 ms_handle_reset on v2:172.21.15.183:6838/1439615083 2024-06-03T03:25:55.577 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.571+0000 7f637c16a740 1 -- 172.21.15.117:0/3477751167 shutdown_connections 2024-06-03T03:25:55.577 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.571+0000 7f637c16a740 1 -- 172.21.15.117:0/3477751167 wait complete. 2024-06-03T03:25:55.577 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.572+0000 7f637c16a740 20 client.15219 trim_cache size 0 max 16384 2024-06-03T03:25:55.593 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.588+0000 7f637c16a740 1 Processor -- start 2024-06-03T03:25:55.593 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.588+0000 7f637c16a740 1 -- start start 2024-06-03T03:25:55.593 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.589+0000 7f637c16a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573496fdbd0 0x557349708010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:55.594 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.589+0000 7f637c16a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5573496fe5c0 0x55734970f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:55.594 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.589+0000 7f637c16a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557349703730 0x5573496f8f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:55.594 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.589+0000 7f637c16a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f63741ac630 con 0x5573496fdbd0 2024-06-03T03:25:55.594 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.589+0000 7f637c16a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f63741ae170 con 0x5573496fe5c0 2024-06-03T03:25:55.594 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.589+0000 7f637c16a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f63741acd00 con 0x557349703730 2024-06-03T03:25:55.594 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.589+0000 7f6372800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573496fdbd0 0x557349708010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:55.594 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.589+0000 7f6369400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5573496fe5c0 0x55734970f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:55.594 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.589+0000 7f6372800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573496fdbd0 0x557349708010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:45030/0 (socket says 172.21.15.117:45030) 2024-06-03T03:25:55.594 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.589+0000 7f6372800640 1 -- 172.21.15.117:0/1596716948 learned_addr learned my addr 172.21.15.117:0/1596716948 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:25:55.594 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.589+0000 7f6372800640 1 --2- 172.21.15.117:0/1596716948 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557349703730 0x5573496f8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:55.594 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.589+0000 7f6369400640 1 -- 172.21.15.117:0/1596716948 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557349703730 msgr2=0x5573496f8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:25:55.595 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.590+0000 7f6369400640 1 --2- 172.21.15.117:0/1596716948 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557349703730 0x5573496f8f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:55.595 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.590+0000 7f6369400640 1 -- 172.21.15.117:0/1596716948 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573496fdbd0 msgr2=0x557349708010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:25:55.595 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.590+0000 7f6369400640 1 --2- 172.21.15.117:0/1596716948 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573496fdbd0 0x557349708010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:55.595 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.590+0000 7f6369400640 1 -- 172.21.15.117:0/1596716948 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573496fd520 con 0x5573496fe5c0 2024-06-03T03:25:55.595 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.590+0000 7f6372800640 1 --2- 172.21.15.117:0/1596716948 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557349703730 0x5573496f8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T03:25:55.595 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.590+0000 7f6369400640 1 --2- 172.21.15.117:0/1596716948 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5573496fe5c0 0x55734970f230 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x557349716630 tx=0x7f635c005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf354d918c1cf387 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:25:55.595 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.590+0000 7f6362a00640 1 -- 172.21.15.117:0/1596716948 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f635c004020 con 0x5573496fe5c0 2024-06-03T03:25:55.595 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.590+0000 7f6362a00640 1 -- 172.21.15.117:0/1596716948 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f635c0041c0 con 0x5573496fe5c0 2024-06-03T03:25:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.590+0000 7f6362a00640 1 -- 172.21.15.117:0/1596716948 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f635c01e830 con 0x5573496fe5c0 2024-06-03T03:25:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.591+0000 7f637c16a740 1 -- 172.21.15.117:0/1596716948 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5573496fe5c0 msgr2=0x55734970f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.591+0000 7f637c16a740 1 --2- 172.21.15.117:0/1596716948 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5573496fe5c0 0x55734970f230 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x557349716630 tx=0x7f635c005f90 comp rx=0 tx=0).stop 2024-06-03T03:25:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.591+0000 7f637c16a740 1 -- 172.21.15.117:0/1596716948 shutdown_connections 2024-06-03T03:25:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.591+0000 7f637c16a740 1 --2- 172.21.15.117:0/1596716948 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557349703730 0x5573496f8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.591+0000 7f637c16a740 1 --2- 172.21.15.117:0/1596716948 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5573496fe5c0 0x55734970f230 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.591+0000 7f637c16a740 1 --2- 172.21.15.117:0/1596716948 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573496fdbd0 0x557349708010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.591+0000 7f637c16a740 1 -- 172.21.15.117:0/1596716948 >> 172.21.15.117:0/1596716948 conn(0x5573496c5f80 msgr2=0x557349707900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:25:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.591+0000 7f637c16a740 1 -- 172.21.15.117:0/1596716948 shutdown_connections 2024-06-03T03:25:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.591+0000 7f637c16a740 1 -- 172.21.15.117:0/1596716948 wait complete. 2024-06-03T03:25:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.591+0000 7f637c16a740 1 -- start start 2024-06-03T03:25:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.592+0000 7f637c16a740 1 Processor -- start 2024-06-03T03:25:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.592+0000 7f637c16a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5573496fdbd0 0x55734970c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:55.597 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.592+0000 7f637c16a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573496fe5c0 0x55734970c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:55.597 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.592+0000 7f637c16a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557349703730 0x55734970d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:55.597 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.592+0000 7f637c16a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f63741adaa0 con 0x5573496fe5c0 2024-06-03T03:25:55.597 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.592+0000 7f637c16a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f63741ad3d0 con 0x5573496fdbd0 2024-06-03T03:25:55.597 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.592+0000 7f637c16a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f63741aa610 con 0x557349703730 2024-06-03T03:25:55.597 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.592+0000 7f637c16a740 10 client.0 _ll_register_callbacks cb 0x7f63640d5400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:25:55.597 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.592+0000 7f6369400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573496fe5c0 0x55734970c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:55.597 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.592+0000 7f6372800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5573496fdbd0 0x55734970c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:55.597 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.592+0000 7f6369400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573496fe5c0 0x55734970c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:45036/0 (socket says 172.21.15.117:45036) 2024-06-03T03:25:55.597 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.592+0000 7f6369400640 1 -- 172.21.15.117:0/793142174 learned_addr learned my addr 172.21.15.117:0/793142174 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:25:55.597 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.592+0000 7f6372800640 1 --2- 172.21.15.117:0/793142174 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557349703730 0x55734970d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:55.597 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.593+0000 7f6369400640 1 -- 172.21.15.117:0/793142174 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557349703730 msgr2=0x55734970d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:25:55.597 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.593+0000 7f6369400640 1 --2- 172.21.15.117:0/793142174 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557349703730 0x55734970d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:55.598 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.593+0000 7f6369400640 1 -- 172.21.15.117:0/793142174 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5573496fdbd0 msgr2=0x55734970c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:25:55.598 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.593+0000 7f6369400640 1 --2- 172.21.15.117:0/793142174 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5573496fdbd0 0x55734970c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:55.598 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.593+0000 7f6369400640 1 -- 172.21.15.117:0/793142174 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55734970ded0 con 0x5573496fe5c0 2024-06-03T03:25:55.598 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.593+0000 7f6369400640 1 --2- 172.21.15.117:0/793142174 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573496fe5c0 0x55734970c8d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f635c00a010 tx=0x7f635c004020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29183f79497df16d server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:25:55.598 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.593+0000 7f6357e00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-06-03T03:25:55.600 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.593+0000 7f6357e00640 1 -- 172.21.15.117:0/793142174 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f635c004370 con 0x5573496fe5c0 2024-06-03T03:25:55.600 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.593+0000 7f637c16a740 1 -- 172.21.15.117:0/793142174 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f635c0041c0 con 0x5573496fe5c0 2024-06-03T03:25:55.600 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.593+0000 7f6357e00640 1 -- 172.21.15.117:0/793142174 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f635c005070 con 0x5573496fe5c0 2024-06-03T03:25:55.600 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.594+0000 7f6357e00640 1 -- 172.21.15.117:0/793142174 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f635c0052b0 con 0x5573496fe5c0 2024-06-03T03:25:55.601 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.594+0000 7f637c16a740 10 client.15225 fetch_fsmap learned FSMap version 31 2024-06-03T03:25:55.601 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.594+0000 7f637c16a740 1 -- 172.21.15.117:0/793142174 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55734970f1f0 con 0x5573496fe5c0 2024-06-03T03:25:55.601 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.594+0000 7f6357e00640 1 -- 172.21.15.117:0/793142174 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f635c0041c0 con 0x5573496fe5c0 2024-06-03T03:25:55.601 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.594+0000 7f6357e00640 1 -- 172.21.15.117:0/793142174 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f635c025030 con 0x5573496fe5c0 2024-06-03T03:25:55.601 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.594+0000 7f637c16a740 10 client.15225 fetch_fsmap finished waiting for FSMap version 31 2024-06-03T03:25:55.601 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.594+0000 7f637c16a740 10 client.15225 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-03T03:25:55.601 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.594+0000 7f637c16a740 20 client.15225 populate_metadata read hostname 'smithi117' 2024-06-03T03:25:55.601 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.594+0000 7f637c16a740 1 --2- 172.21.15.117:0/793142174 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x557349787280 0x5573497896a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:25:55.601 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.594+0000 7f637c16a740 4 client.15225 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-03T03:25:55.601 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.594+0000 7f637c16a740 1 -- 172.21.15.117:0/793142174 --> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557349668ea0 con 0x557349787280 2024-06-03T03:25:55.601 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.595+0000 7f6372800640 1 --2- 172.21.15.117:0/793142174 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x557349787280 0x5573497896a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:25:55.601 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.595+0000 7f6372800640 1 --2- 172.21.15.117:0/793142174 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x557349787280 0x5573497896a0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=622def5462c0d73b server_cookie=861f638255009eb2 in_seq=0 out_seq=0 2024-06-03T03:25:55.601 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.595+0000 7f6357e00640 10 client.15225 ms_handle_connect on v2:172.21.15.183:6838/1439615083 2024-06-03T03:25:55.602 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.599+0000 7f6357e00640 1 -- 172.21.15.117:0/793142174 <== mds.0 v2:172.21.15.183:6838/1439615083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x557349668ea0 con 0x557349787280 2024-06-03T03:25:55.602 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.599+0000 7f6357e00640 10 client.15225 handle_command_reply: tid=0 2024-06-03T03:25:55.602 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.599+0000 7f637c16a740 1 client.15225 shutdown 2024-06-03T03:25:55.602 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.599+0000 7f637c16a740 1 -- 172.21.15.117:0/793142174 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573496fe5c0 msgr2=0x55734970c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:55.602 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.599+0000 7f637c16a740 1 --2- 172.21.15.117:0/793142174 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573496fe5c0 0x55734970c8d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f635c00a010 tx=0x7f635c004020 comp rx=0 tx=0).stop 2024-06-03T03:25:55.602 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.599+0000 7f637c16a740 1 -- 172.21.15.117:0/793142174 shutdown_connections 2024-06-03T03:25:55.602 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.599+0000 7f637c16a740 1 --2- 172.21.15.117:0/793142174 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x557349787280 0x5573497896a0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:55.602 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.599+0000 7f637c16a740 1 --2- 172.21.15.117:0/793142174 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557349703730 0x55734970d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:55.602 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.599+0000 7f6357e00640 0 client.15225 ms_handle_reset on v2:172.21.15.183:6838/1439615083 2024-06-03T03:25:55.602 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.599+0000 7f637c16a740 1 --2- 172.21.15.117:0/793142174 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573496fe5c0 0x55734970c8d0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:55.602 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.599+0000 7f637c16a740 1 --2- 172.21.15.117:0/793142174 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5573496fdbd0 0x55734970c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:55.602 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.599+0000 7f637c16a740 1 -- 172.21.15.117:0/793142174 >> 172.21.15.117:0/793142174 conn(0x5573496c5f80 msgr2=0x5573496f7f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:25:55.603 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.599+0000 7f637c16a740 1 -- 172.21.15.117:0/793142174 shutdown_connections 2024-06-03T03:25:55.603 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.599+0000 7f637c16a740 1 -- 172.21.15.117:0/793142174 wait complete. 2024-06-03T03:25:55.603 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.599+0000 7f637c16a740 20 client.15225 trim_cache size 0 max 16384 2024-06-03T03:25:55.603 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:{ 2024-06-03T03:25:55.603 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "return_code": 0 2024-06-03T03:25:55.603 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:} 2024-06-03T03:25:55.604 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.600+0000 7f6379200640 1 -- 172.21.15.117:0/3025227368 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f63580a2cd0 msgr2=0x7f63580a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:55.604 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.600+0000 7f6379200640 1 --2- 172.21.15.117:0/3025227368 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f63580a2cd0 0x7f63580a5190 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x5573496fe9c0 tx=0x7f63640d5530 comp rx=0 tx=0).stop 2024-06-03T03:25:55.604 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.600+0000 7f6379200640 1 -- 172.21.15.117:0/3025227368 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6374103c20 msgr2=0x7f6374199e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:25:55.604 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.600+0000 7f6379200640 1 --2- 172.21.15.117:0/3025227368 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6374103c20 0x7f6374199e10 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f636400e9c0 tx=0x7f636400ee80 comp rx=0 tx=0).stop 2024-06-03T03:25:55.604 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.600+0000 7f6379200640 1 -- 172.21.15.117:0/3025227368 shutdown_connections 2024-06-03T03:25:55.604 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.600+0000 7f6379200640 1 --2- 172.21.15.117:0/3025227368 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63741057d0 0x7f63741a13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:55.604 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.600+0000 7f6379200640 1 --2- 172.21.15.117:0/3025227368 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f63580a2cd0 0x7f63580a5190 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:55.604 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.600+0000 7f6379200640 1 --2- 172.21.15.117:0/3025227368 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6374104e10 0x7f637419a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:55.604 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.600+0000 7f6379200640 1 --2- 172.21.15.117:0/3025227368 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6374103c20 0x7f6374199e10 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:25:55.605 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.600+0000 7f6379200640 1 -- 172.21.15.117:0/3025227368 >> 172.21.15.117:0/3025227368 conn(0x7f63740ff190 msgr2=0x7f6374108040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:25:55.605 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.600+0000 7f6379200640 1 -- 172.21.15.117:0/3025227368 shutdown_connections 2024-06-03T03:25:55.605 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:25:55.600+0000 7f6379200640 1 -- 172.21.15.117:0/3025227368 wait complete. 2024-06-03T03:25:58.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:25:58 smithi151 ceph-mon[37043]: pgmap v525: 129 pgs: 129 active+clean; 666 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 852 B/s wr, 11 op/s 2024-06-03T03:25:58.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:25:58 smithi117 ceph-mon[31649]: pgmap v525: 129 pgs: 129 active+clean; 666 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 852 B/s wr, 11 op/s 2024-06-03T03:25:58.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:25:58 smithi183 ceph-mon[37326]: pgmap v525: 129 pgs: 129 active+clean; 666 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 852 B/s wr, 11 op/s 2024-06-03T03:26:00.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:26:00 smithi151 ceph-mon[37043]: pgmap v526: 129 pgs: 129 active+clean; 667 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 852 B/s wr, 13 op/s 2024-06-03T03:26:00.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:26:00 smithi117 ceph-mon[31649]: pgmap v526: 129 pgs: 129 active+clean; 667 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 852 B/s wr, 13 op/s 2024-06-03T03:26:00.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:26:00 smithi183 ceph-mon[37326]: pgmap v526: 129 pgs: 129 active+clean; 667 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 852 B/s wr, 13 op/s 2024-06-03T03:26:02.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:26:02 smithi151 ceph-mon[37043]: pgmap v527: 129 pgs: 129 active+clean; 667 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-06-03T03:26:02.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:26:02 smithi117 ceph-mon[31649]: pgmap v527: 129 pgs: 129 active+clean; 667 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-06-03T03:26:02.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:26:02 smithi183 ceph-mon[37326]: pgmap v527: 129 pgs: 129 active+clean; 667 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-06-03T03:26:04.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:26:04 smithi151 ceph-mon[37043]: pgmap v528: 129 pgs: 129 active+clean; 667 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-06-03T03:26:04.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:26:04 smithi117 ceph-mon[31649]: pgmap v528: 129 pgs: 129 active+clean; 667 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-06-03T03:26:04.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:26:04 smithi183 ceph-mon[37326]: pgmap v528: 129 pgs: 129 active+clean; 667 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-06-03T03:26:05.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:26:05 smithi151 ceph-mon[37043]: pgmap v529: 129 pgs: 129 active+clean; 667 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-06-03T03:26:05.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:26:05 smithi117 ceph-mon[31649]: pgmap v529: 129 pgs: 129 active+clean; 667 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-06-03T03:26:05.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:26:05 smithi183 ceph-mon[37326]: pgmap v529: 129 pgs: 129 active+clean; 667 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-06-03T03:26:07.001 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-03T03:26:07.433 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:26:08.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:26:08 smithi117 ceph-mon[31649]: pgmap v530: 129 pgs: 129 active+clean; 667 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-06-03T03:26:08.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:26:08 smithi151 ceph-mon[37043]: pgmap v530: 129 pgs: 129 active+clean; 667 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-06-03T03:26:08.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:26:08 smithi183 ceph-mon[37326]: pgmap v530: 129 pgs: 129 active+clean; 667 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-06-03T03:26:08.749 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.748+0000 7eff33200640 1 -- 172.21.15.117:0/4014293797 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7eff34102370 msgr2=0x7eff34104760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:08.749 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.748+0000 7eff33200640 1 --2- 172.21.15.117:0/4014293797 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7eff34102370 0x7eff34104760 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7eff24009900 tx=0x7eff2402a3f0 comp rx=0 tx=0).stop 2024-06-03T03:26:08.749 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.748+0000 7eff33200640 1 -- 172.21.15.117:0/4014293797 shutdown_connections 2024-06-03T03:26:08.749 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.748+0000 7eff33200640 1 --2- 172.21.15.117:0/4014293797 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff341075d0 0x7eff341099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:08.749 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.748+0000 7eff33200640 1 --2- 172.21.15.117:0/4014293797 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff34104ca0 0x7eff34107090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:08.749 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.748+0000 7eff33200640 1 --2- 172.21.15.117:0/4014293797 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7eff34102370 0x7eff34104760 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:08.749 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.748+0000 7eff33200640 1 -- 172.21.15.117:0/4014293797 >> 172.21.15.117:0/4014293797 conn(0x7eff340fbce0 msgr2=0x7eff340fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:26:08.749 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.748+0000 7eff33200640 1 -- 172.21.15.117:0/4014293797 shutdown_connections 2024-06-03T03:26:08.749 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.748+0000 7eff33200640 1 -- 172.21.15.117:0/4014293797 wait complete. 2024-06-03T03:26:08.750 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.749+0000 7eff33200640 1 Processor -- start 2024-06-03T03:26:08.751 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.749+0000 7eff33200640 1 -- start start 2024-06-03T03:26:08.751 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.750+0000 7eff33200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7eff34102370 0x7eff34199db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:08.751 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.750+0000 7eff33200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff34104ca0 0x7eff3419a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:08.751 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.750+0000 7eff33200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff341075d0 0x7eff341a1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:08.751 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.750+0000 7eff33200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7eff3410d540 con 0x7eff34102370 2024-06-03T03:26:08.751 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.750+0000 7eff33200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7eff3410d3c0 con 0x7eff34104ca0 2024-06-03T03:26:08.751 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.750+0000 7eff33200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7eff3410d6c0 con 0x7eff341075d0 2024-06-03T03:26:08.751 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.750+0000 7eff31e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7eff34102370 0x7eff34199db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:08.751 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.750+0000 7eff32800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff341075d0 0x7eff341a1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:08.751 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.750+0000 7eff31e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7eff34102370 0x7eff34199db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:57680/0 (socket says 172.21.15.117:57680) 2024-06-03T03:26:08.751 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.750+0000 7eff31400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff34104ca0 0x7eff3419a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:08.751 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.750+0000 7eff31e00640 1 -- 172.21.15.117:0/3437890135 learned_addr learned my addr 172.21.15.117:0/3437890135 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:26:08.752 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.751+0000 7eff32800640 1 -- 172.21.15.117:0/3437890135 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff34104ca0 msgr2=0x7eff3419a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:08.752 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.751+0000 7eff32800640 1 --2- 172.21.15.117:0/3437890135 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff34104ca0 0x7eff3419a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:08.753 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.751+0000 7eff32800640 1 -- 172.21.15.117:0/3437890135 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7eff34102370 msgr2=0x7eff34199db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:08.753 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.751+0000 7eff32800640 1 --2- 172.21.15.117:0/3437890135 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7eff34102370 0x7eff34199db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:08.753 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.751+0000 7eff32800640 1 -- 172.21.15.117:0/3437890135 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff341a19c0 con 0x7eff341075d0 2024-06-03T03:26:08.753 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.751+0000 7eff31400640 1 --2- 172.21.15.117:0/3437890135 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff34104ca0 0x7eff3419a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:26:08.753 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.751+0000 7eff32800640 1 --2- 172.21.15.117:0/3437890135 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff341075d0 0x7eff341a1320 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7eff2c00da10 tx=0x7eff2c00ded0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:26:08.753 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.751+0000 7eff2aa00640 1 -- 172.21.15.117:0/3437890135 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff2c00bd70 con 0x7eff341075d0 2024-06-03T03:26:08.753 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.751+0000 7eff33200640 1 -- 172.21.15.117:0/3437890135 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff341a1cb0 con 0x7eff341075d0 2024-06-03T03:26:08.753 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.751+0000 7eff2aa00640 1 -- 172.21.15.117:0/3437890135 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff2c003e50 con 0x7eff341075d0 2024-06-03T03:26:08.753 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.751+0000 7eff2aa00640 1 -- 172.21.15.117:0/3437890135 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff2c0106a0 con 0x7eff341075d0 2024-06-03T03:26:08.753 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.751+0000 7eff33200640 1 -- 172.21.15.117:0/3437890135 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff341a21b0 con 0x7eff341075d0 2024-06-03T03:26:08.755 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.755+0000 7eff3b9ca740 1 Processor -- start 2024-06-03T03:26:08.755 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.755+0000 7eff2aa00640 1 -- 172.21.15.117:0/3437890135 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7eff2c003970 con 0x7eff341075d0 2024-06-03T03:26:08.755 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.755+0000 7eff3b9ca740 1 -- start start 2024-06-03T03:26:08.756 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.756+0000 7eff3b9ca740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572126345a0 0x5572126349c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:08.756 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.756+0000 7eff3b9ca740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557212634f90 0x557212639a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:08.756 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.756+0000 7eff3b9ca740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55721263a100 0x55721263c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:08.756 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.756+0000 7eff3b9ca740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55721258f420 con 0x55721263a100 2024-06-03T03:26:08.756 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.756+0000 7eff3b9ca740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x557212536c50 con 0x557212634f90 2024-06-03T03:26:08.756 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.756+0000 7eff3b9ca740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55721258b3e0 con 0x5572126345a0 2024-06-03T03:26:08.757 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.756+0000 7eff31400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572126345a0 0x5572126349c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:08.757 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.756+0000 7eff31e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557212634f90 0x557212639a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:08.757 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.756+0000 7eff31400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572126345a0 0x5572126349c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:43848/0 (socket says 172.21.15.117:43848) 2024-06-03T03:26:08.757 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.756+0000 7eff31400640 1 -- 172.21.15.117:0/2373657156 learned_addr learned my addr 172.21.15.117:0/2373657156 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:26:08.757 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.756+0000 7eff31400640 1 --2- 172.21.15.117:0/2373657156 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55721263a100 0x55721263c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:08.758 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.757+0000 7eff31e00640 1 -- 172.21.15.117:0/2373657156 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572126345a0 msgr2=0x5572126349c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:08.758 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.757+0000 7eff31e00640 1 --2- 172.21.15.117:0/2373657156 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572126345a0 0x5572126349c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:08.758 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.757+0000 7eff31e00640 1 -- 172.21.15.117:0/2373657156 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55721263a100 msgr2=0x55721263c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:08.758 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.757+0000 7eff31e00640 1 --2- 172.21.15.117:0/2373657156 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55721263a100 0x55721263c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:08.758 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.757+0000 7eff31e00640 1 -- 172.21.15.117:0/2373657156 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572122a3210 con 0x557212634f90 2024-06-03T03:26:08.759 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.757+0000 7eff2aa00640 1 --2- 172.21.15.117:0/3437890135 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7eff180a2c10 0x7eff180a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:08.759 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.757+0000 7eff31e00640 1 --2- 172.21.15.117:0/2373657156 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557212634f90 0x557212639a90 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7eff3419b280 tx=0x7eff24004170 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2891b7294ad999c3 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:26:08.759 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.757+0000 7eff2aa00640 1 -- 172.21.15.117:0/3437890135 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7eff2c0cfa30 con 0x7eff341075d0 2024-06-03T03:26:08.759 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.758+0000 7eff31400640 1 --2- 172.21.15.117:0/3437890135 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7eff180a2c10 0x7eff180a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:08.759 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.758+0000 7eff1f000640 1 -- 172.21.15.117:0/2373657156 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff24040070 con 0x557212634f90 2024-06-03T03:26:08.759 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.758+0000 7eff1f000640 1 -- 172.21.15.117:0/2373657156 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff24005500 con 0x557212634f90 2024-06-03T03:26:08.759 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.758+0000 7eff1f000640 1 -- 172.21.15.117:0/2373657156 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff24035040 con 0x557212634f90 2024-06-03T03:26:08.760 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.759+0000 7eff3b9ca740 1 -- 172.21.15.117:0/2373657156 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557212634f90 msgr2=0x557212639a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:08.760 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.759+0000 7eff31400640 1 --2- 172.21.15.117:0/3437890135 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7eff180a2c10 0x7eff180a50d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x5572126357f0 tx=0x7eff20005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:26:08.760 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.759+0000 7eff3b9ca740 1 --2- 172.21.15.117:0/2373657156 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557212634f90 0x557212639a90 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7eff3419b280 tx=0x7eff24004170 comp rx=0 tx=0).stop 2024-06-03T03:26:08.760 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.759+0000 7eff3b9ca740 1 -- 172.21.15.117:0/2373657156 shutdown_connections 2024-06-03T03:26:08.760 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.759+0000 7eff3b9ca740 1 --2- 172.21.15.117:0/2373657156 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55721263a100 0x55721263c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:08.760 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.759+0000 7eff3b9ca740 1 --2- 172.21.15.117:0/2373657156 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557212634f90 0x557212639a90 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:08.760 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.759+0000 7eff3b9ca740 1 --2- 172.21.15.117:0/2373657156 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572126345a0 0x5572126349c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:08.760 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.759+0000 7eff3b9ca740 1 -- 172.21.15.117:0/2373657156 >> 172.21.15.117:0/2373657156 conn(0x5572125f3f80 msgr2=0x557212627810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:26:08.761 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.759+0000 7eff3b9ca740 1 -- 172.21.15.117:0/2373657156 shutdown_connections 2024-06-03T03:26:08.761 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.759+0000 7eff3b9ca740 1 -- 172.21.15.117:0/2373657156 wait complete. 2024-06-03T03:26:08.761 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.759+0000 7eff3b9ca740 1 -- start start 2024-06-03T03:26:08.761 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.760+0000 7eff3b9ca740 1 Processor -- start 2024-06-03T03:26:08.764 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.760+0000 7eff3b9ca740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572126345a0 0x55721263e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:08.764 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.760+0000 7eff3b9ca740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557212634f90 0x55721263ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:08.764 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.760+0000 7eff3b9ca740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55721263a100 0x557212645e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:08.764 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.760+0000 7eff3b9ca740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55721263d050 con 0x55721263a100 2024-06-03T03:26:08.764 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.760+0000 7eff3b9ca740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55721254a0b0 con 0x557212634f90 2024-06-03T03:26:08.764 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.760+0000 7eff3b9ca740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55721263d1d0 con 0x5572126345a0 2024-06-03T03:26:08.764 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.760+0000 7eff3b9ca740 10 client.0 _ll_register_callbacks cb 0x557212535f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:26:08.764 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.761+0000 7eff31e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572126345a0 0x55721263e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:08.764 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.761+0000 7eff31e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572126345a0 0x55721263e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:43850/0 (socket says 172.21.15.117:43850) 2024-06-03T03:26:08.764 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.761+0000 7eff31e00640 1 -- 172.21.15.117:0/3401341425 learned_addr learned my addr 172.21.15.117:0/3401341425 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:26:08.764 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.761+0000 7eff32800640 1 --2- 172.21.15.117:0/3401341425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55721263a100 0x557212645e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:08.764 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.761+0000 7eff31400640 1 --2- 172.21.15.117:0/3401341425 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557212634f90 0x55721263ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:08.765 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.761+0000 7eff31e00640 1 -- 172.21.15.117:0/3401341425 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557212634f90 msgr2=0x55721263ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:08.765 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.761+0000 7eff31e00640 1 --2- 172.21.15.117:0/3401341425 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557212634f90 0x55721263ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:08.765 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.761+0000 7eff31e00640 1 -- 172.21.15.117:0/3401341425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55721263a100 msgr2=0x557212645e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:08.765 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.761+0000 7eff31e00640 1 --2- 172.21.15.117:0/3401341425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55721263a100 0x557212645e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:08.765 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.761+0000 7eff31e00640 1 -- 172.21.15.117:0/3401341425 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55721236f620 con 0x5572126345a0 2024-06-03T03:26:08.765 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.761+0000 7eff31400640 1 --2- 172.21.15.117:0/3401341425 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557212634f90 0x55721263ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T03:26:08.765 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.761+0000 7eff31e00640 1 --2- 172.21.15.117:0/3401341425 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572126345a0 0x55721263e8d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7eff2402a430 tx=0x7eff240058c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b020b493a5de6efd server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:26:08.765 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.761+0000 7eff13400640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-06-03T03:26:08.765 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.762+0000 7eff13400640 1 -- 172.21.15.117:0/3401341425 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff24035070 con 0x5572126345a0 2024-06-03T03:26:08.765 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.762+0000 7eff3b9ca740 1 -- 172.21.15.117:0/3401341425 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff24005500 con 0x5572126345a0 2024-06-03T03:26:08.765 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.762+0000 7eff13400640 1 -- 172.21.15.117:0/3401341425 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7eff24004e50 con 0x5572126345a0 2024-06-03T03:26:08.765 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.762+0000 7eff13400640 1 -- 172.21.15.117:0/3401341425 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7eff24058050 con 0x5572126345a0 2024-06-03T03:26:08.765 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.762+0000 7eff3b9ca740 10 client.25378 fetch_fsmap learned FSMap version 31 2024-06-03T03:26:08.765 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.762+0000 7eff3b9ca740 1 -- 172.21.15.117:0/3401341425 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55721263d0a0 con 0x5572126345a0 2024-06-03T03:26:08.766 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.762+0000 7eff13400640 1 -- 172.21.15.117:0/3401341425 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff24005500 con 0x5572126345a0 2024-06-03T03:26:08.766 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.763+0000 7eff13400640 1 -- 172.21.15.117:0/3401341425 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7eff240035f0 con 0x5572126345a0 2024-06-03T03:26:08.766 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.763+0000 7eff3b9ca740 10 client.25378 fetch_fsmap finished waiting for FSMap version 31 2024-06-03T03:26:08.766 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.763+0000 7eff3b9ca740 10 client.25378 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-03T03:26:08.766 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.763+0000 7eff3b9ca740 20 client.25378 populate_metadata read hostname 'smithi117' 2024-06-03T03:26:08.766 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.763+0000 7eff3b9ca740 1 --2- 172.21.15.117:0/3401341425 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55721264a470 0x55721264c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:08.766 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.763+0000 7eff3b9ca740 4 client.25378 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T03:26:08.766 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.763+0000 7eff3b9ca740 1 -- 172.21.15.117:0/3401341425 --> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55721259cb50 con 0x55721264a470 2024-06-03T03:26:08.766 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.763+0000 7eff31400640 1 --2- 172.21.15.117:0/3401341425 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55721264a470 0x55721264c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:08.767 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.764+0000 7eff13400640 10 client.25378 ms_handle_connect on v2:172.21.15.183:6838/1439615083 2024-06-03T03:26:08.767 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.764+0000 7eff31400640 1 --2- 172.21.15.117:0/3401341425 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55721264a470 0x55721264c860 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16d442a63776d0b3 server_cookie=56173d74c72c01ba in_seq=0 out_seq=0 2024-06-03T03:26:08.767 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.765+0000 7eff13400640 1 -- 172.21.15.117:0/3401341425 <== mds.0 v2:172.21.15.183:6838/1439615083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55721259cb50 con 0x55721264a470 2024-06-03T03:26:08.767 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.765+0000 7eff13400640 10 client.25378 handle_command_reply: tid=0 2024-06-03T03:26:08.768 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.765+0000 7eff3b9ca740 1 client.25378 shutdown 2024-06-03T03:26:08.768 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.765+0000 7eff3b9ca740 1 -- 172.21.15.117:0/3401341425 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572126345a0 msgr2=0x55721263e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:08.768 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.765+0000 7eff3b9ca740 1 --2- 172.21.15.117:0/3401341425 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572126345a0 0x55721263e8d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7eff2402a430 tx=0x7eff240058c0 comp rx=0 tx=0).stop 2024-06-03T03:26:08.768 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.765+0000 7eff3b9ca740 1 -- 172.21.15.117:0/3401341425 shutdown_connections 2024-06-03T03:26:08.768 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.765+0000 7eff3b9ca740 1 --2- 172.21.15.117:0/3401341425 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55721264a470 0x55721264c860 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:08.768 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.765+0000 7eff3b9ca740 1 --2- 172.21.15.117:0/3401341425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55721263a100 0x557212645e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:08.768 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.765+0000 7eff3b9ca740 1 --2- 172.21.15.117:0/3401341425 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557212634f90 0x55721263ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:08.768 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.765+0000 7eff3b9ca740 1 --2- 172.21.15.117:0/3401341425 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572126345a0 0x55721263e8d0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:08.768 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.765+0000 7eff13400640 0 client.25378 ms_handle_reset on v2:172.21.15.183:6838/1439615083 2024-06-03T03:26:08.768 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.765+0000 7eff3b9ca740 1 -- 172.21.15.117:0/3401341425 >> 172.21.15.117:0/3401341425 conn(0x5572125f3f80 msgr2=0x557212626970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:26:08.768 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.765+0000 7eff3b9ca740 1 -- 172.21.15.117:0/3401341425 shutdown_connections 2024-06-03T03:26:08.768 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.765+0000 7eff3b9ca740 1 -- 172.21.15.117:0/3401341425 wait complete. 2024-06-03T03:26:08.768 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.765+0000 7eff3b9ca740 20 client.25378 trim_cache size 0 max 16384 2024-06-03T03:26:08.783 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.782+0000 7eff3b9ca740 1 Processor -- start 2024-06-03T03:26:08.783 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.782+0000 7eff3b9ca740 1 -- start start 2024-06-03T03:26:08.783 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.782+0000 7eff3b9ca740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5572126345a0 0x55721262ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:08.783 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.782+0000 7eff3b9ca740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557212634f90 0x557212645de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:08.783 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.782+0000 7eff3b9ca740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55721263a100 0x55721263def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:08.783 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.782+0000 7eff3b9ca740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7eff341a81c0 con 0x5572126345a0 2024-06-03T03:26:08.783 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.782+0000 7eff3b9ca740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7eff341a9d00 con 0x55721263a100 2024-06-03T03:26:08.783 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.782+0000 7eff3b9ca740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7eff341a8890 con 0x557212634f90 2024-06-03T03:26:08.784 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.782+0000 7eff31e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5572126345a0 0x55721262ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:08.784 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.783+0000 7eff31e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5572126345a0 0x55721262ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:57706/0 (socket says 172.21.15.117:57706) 2024-06-03T03:26:08.784 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.783+0000 7eff31e00640 1 -- 172.21.15.117:0/2813240435 learned_addr learned my addr 172.21.15.117:0/2813240435 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:26:08.784 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.783+0000 7eff31400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557212634f90 0x557212645de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:08.784 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.783+0000 7eff32800640 1 --2- 172.21.15.117:0/2813240435 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55721263a100 0x55721263def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:08.785 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.783+0000 7eff31400640 1 -- 172.21.15.117:0/2813240435 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55721263a100 msgr2=0x55721263def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:08.785 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.783+0000 7eff31400640 1 --2- 172.21.15.117:0/2813240435 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55721263a100 0x55721263def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:08.785 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.783+0000 7eff31400640 1 -- 172.21.15.117:0/2813240435 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5572126345a0 msgr2=0x55721262ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:08.785 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.783+0000 7eff31400640 1 --2- 172.21.15.117:0/2813240435 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5572126345a0 0x55721262ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:08.786 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.783+0000 7eff31400640 1 -- 172.21.15.117:0/2813240435 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557212622fd0 con 0x557212634f90 2024-06-03T03:26:08.786 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.783+0000 7eff31e00640 1 --2- 172.21.15.117:0/2813240435 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5572126345a0 0x55721262ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:26:08.786 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.783+0000 7eff31400640 1 --2- 172.21.15.117:0/2813240435 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557212634f90 0x557212645de0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x55721264d0a0 tx=0x7eff200046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98353ee73e970d17 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:26:08.786 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.784+0000 7eff1e600640 1 -- 172.21.15.117:0/2813240435 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff20032070 con 0x557212634f90 2024-06-03T03:26:08.786 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.784+0000 7eff1e600640 1 -- 172.21.15.117:0/2813240435 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff20005500 con 0x557212634f90 2024-06-03T03:26:08.786 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.784+0000 7eff1e600640 1 -- 172.21.15.117:0/2813240435 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff2002d620 con 0x557212634f90 2024-06-03T03:26:08.786 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.784+0000 7eff3b9ca740 1 -- 172.21.15.117:0/2813240435 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557212634f90 msgr2=0x557212645de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:08.786 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.784+0000 7eff3b9ca740 1 --2- 172.21.15.117:0/2813240435 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557212634f90 0x557212645de0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x55721264d0a0 tx=0x7eff200046c0 comp rx=0 tx=0).stop 2024-06-03T03:26:08.786 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.785+0000 7eff3b9ca740 1 -- 172.21.15.117:0/2813240435 shutdown_connections 2024-06-03T03:26:08.786 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.785+0000 7eff3b9ca740 1 --2- 172.21.15.117:0/2813240435 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55721263a100 0x55721263def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:08.786 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.785+0000 7eff3b9ca740 1 --2- 172.21.15.117:0/2813240435 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557212634f90 0x557212645de0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:08.786 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.785+0000 7eff3b9ca740 1 --2- 172.21.15.117:0/2813240435 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5572126345a0 0x55721262ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:08.786 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.785+0000 7eff3b9ca740 1 -- 172.21.15.117:0/2813240435 >> 172.21.15.117:0/2813240435 conn(0x5572125f3f80 msgr2=0x55721262a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:26:08.787 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.785+0000 7eff3b9ca740 1 -- 172.21.15.117:0/2813240435 shutdown_connections 2024-06-03T03:26:08.787 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.785+0000 7eff3b9ca740 1 -- 172.21.15.117:0/2813240435 wait complete. 2024-06-03T03:26:08.787 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.785+0000 7eff3b9ca740 1 -- start start 2024-06-03T03:26:08.789 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.785+0000 7eff3b9ca740 1 Processor -- start 2024-06-03T03:26:08.789 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.786+0000 7eff3b9ca740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572126345a0 0x557212645c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:08.789 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.786+0000 7eff3b9ca740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557212634f90 0x557212642750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:08.789 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.786+0000 7eff3b9ca740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55721263a100 0x557212642c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:08.789 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.786+0000 7eff3b9ca740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7eff341a9630 con 0x55721263a100 2024-06-03T03:26:08.789 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.786+0000 7eff3b9ca740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7eff341a8f60 con 0x5572126345a0 2024-06-03T03:26:08.789 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.786+0000 7eff3b9ca740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7eff341a61c0 con 0x557212634f90 2024-06-03T03:26:08.789 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.786+0000 7eff3b9ca740 10 client.0 _ll_register_callbacks cb 0x7eff200116d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:26:08.789 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.786+0000 7eff31e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572126345a0 0x557212645c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:08.790 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.786+0000 7eff32800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55721263a100 0x557212642c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:08.790 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.786+0000 7eff31e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572126345a0 0x557212645c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:46792/0 (socket says 172.21.15.117:46792) 2024-06-03T03:26:08.790 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.786+0000 7eff31e00640 1 -- 172.21.15.117:0/4222362167 learned_addr learned my addr 172.21.15.117:0/4222362167 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:26:08.790 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.786+0000 7eff31400640 1 --2- 172.21.15.117:0/4222362167 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557212634f90 0x557212642750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:08.790 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.787+0000 7eff31e00640 1 -- 172.21.15.117:0/4222362167 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557212634f90 msgr2=0x557212642750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:08.790 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.787+0000 7eff31e00640 1 --2- 172.21.15.117:0/4222362167 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557212634f90 0x557212642750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:08.790 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.787+0000 7eff31e00640 1 -- 172.21.15.117:0/4222362167 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55721263a100 msgr2=0x557212642c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:08.790 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.787+0000 7eff31e00640 1 --2- 172.21.15.117:0/4222362167 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55721263a100 0x557212642c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:08.790 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.787+0000 7eff31e00640 1 -- 172.21.15.117:0/4222362167 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572126b30c0 con 0x5572126345a0 2024-06-03T03:26:08.790 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.787+0000 7eff32800640 1 --2- 172.21.15.117:0/4222362167 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55721263a100 0x557212642c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:26:08.790 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.787+0000 7eff31400640 1 --2- 172.21.15.117:0/4222362167 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557212634f90 0x557212642750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:26:08.790 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.787+0000 7eff31e00640 1 --2- 172.21.15.117:0/4222362167 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572126345a0 0x557212645c40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7eff24003bb0 tx=0x7eff24003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6fa10b07ad36d54 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:26:08.790 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.787+0000 7eff13400640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-06-03T03:26:08.791 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.787+0000 7eff13400640 1 -- 172.21.15.117:0/4222362167 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff24035040 con 0x5572126345a0 2024-06-03T03:26:08.791 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.788+0000 7eff13400640 1 -- 172.21.15.117:0/4222362167 <== mon.2 v2:172.21.15.151:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7eff240316b0 con 0x5572126345a0 2024-06-03T03:26:08.791 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.788+0000 7eff3b9ca740 1 -- 172.21.15.117:0/4222362167 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff20005500 con 0x5572126345a0 2024-06-03T03:26:08.791 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.788+0000 7eff13400640 1 -- 172.21.15.117:0/4222362167 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7eff24069050 con 0x5572126345a0 2024-06-03T03:26:08.791 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.788+0000 7eff3b9ca740 10 client.25352 fetch_fsmap learned FSMap version 31 2024-06-03T03:26:08.791 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.788+0000 7eff3b9ca740 1 -- 172.21.15.117:0/4222362167 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5572126b42e0 con 0x5572126345a0 2024-06-03T03:26:08.791 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.788+0000 7eff13400640 1 -- 172.21.15.117:0/4222362167 <== mon.2 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff20005500 con 0x5572126345a0 2024-06-03T03:26:08.791 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.788+0000 7eff13400640 1 -- 172.21.15.117:0/4222362167 <== mon.2 v2:172.21.15.151:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7eff24064030 con 0x5572126345a0 2024-06-03T03:26:08.791 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.789+0000 7eff3b9ca740 10 client.25352 fetch_fsmap finished waiting for FSMap version 31 2024-06-03T03:26:08.791 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.789+0000 7eff3b9ca740 10 client.25352 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-03T03:26:08.791 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.789+0000 7eff3b9ca740 20 client.25352 populate_metadata read hostname 'smithi117' 2024-06-03T03:26:08.791 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.789+0000 7eff3b9ca740 1 --2- 172.21.15.117:0/4222362167 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x5572126b4c30 0x5572126b7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:08.791 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.789+0000 7eff3b9ca740 4 client.25352 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-03T03:26:08.791 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.789+0000 7eff3b9ca740 1 -- 172.21.15.117:0/4222362167 --> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557212596ea0 con 0x5572126b4c30 2024-06-03T03:26:08.792 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.789+0000 7eff31400640 1 --2- 172.21.15.117:0/4222362167 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x5572126b4c30 0x5572126b7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:08.792 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.790+0000 7eff13400640 10 client.25352 ms_handle_connect on v2:172.21.15.183:6838/1439615083 2024-06-03T03:26:08.792 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.790+0000 7eff31400640 1 --2- 172.21.15.117:0/4222362167 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x5572126b4c30 0x5572126b7050 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6d0227cd9d3be70 server_cookie=e81d056c722dd717 in_seq=0 out_seq=0 2024-06-03T03:26:08.797 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.797+0000 7eff13400640 1 -- 172.21.15.117:0/4222362167 <== mds.0 v2:172.21.15.183:6838/1439615083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x557212596ea0 con 0x5572126b4c30 2024-06-03T03:26:08.798 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.797+0000 7eff13400640 10 client.25352 handle_command_reply: tid=0 2024-06-03T03:26:08.798 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.797+0000 7eff3b9ca740 1 client.25352 shutdown 2024-06-03T03:26:08.798 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.797+0000 7eff3b9ca740 1 -- 172.21.15.117:0/4222362167 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572126345a0 msgr2=0x557212645c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:08.798 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.797+0000 7eff3b9ca740 1 --2- 172.21.15.117:0/4222362167 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572126345a0 0x557212645c40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7eff24003bb0 tx=0x7eff24003820 comp rx=0 tx=0).stop 2024-06-03T03:26:08.799 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.797+0000 7eff3b9ca740 1 -- 172.21.15.117:0/4222362167 shutdown_connections 2024-06-03T03:26:08.799 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.797+0000 7eff3b9ca740 1 --2- 172.21.15.117:0/4222362167 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x5572126b4c30 0x5572126b7050 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:08.799 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.798+0000 7eff3b9ca740 1 --2- 172.21.15.117:0/4222362167 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55721263a100 0x557212642c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:08.799 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.798+0000 7eff3b9ca740 1 --2- 172.21.15.117:0/4222362167 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557212634f90 0x557212642750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:08.799 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.798+0000 7eff3b9ca740 1 --2- 172.21.15.117:0/4222362167 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572126345a0 0x557212645c40 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:08.799 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.798+0000 7eff13400640 0 client.25352 ms_handle_reset on v2:172.21.15.183:6838/1439615083 2024-06-03T03:26:08.799 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.798+0000 7eff3b9ca740 1 -- 172.21.15.117:0/4222362167 >> 172.21.15.117:0/4222362167 conn(0x5572125f3f80 msgr2=0x55721262af60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:26:08.799 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.798+0000 7eff3b9ca740 1 -- 172.21.15.117:0/4222362167 shutdown_connections 2024-06-03T03:26:08.799 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.798+0000 7eff3b9ca740 1 -- 172.21.15.117:0/4222362167 wait complete. 2024-06-03T03:26:08.799 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.798+0000 7eff3b9ca740 20 client.25352 trim_cache size 0 max 16384 2024-06-03T03:26:08.800 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:{ 2024-06-03T03:26:08.800 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "return_code": 0 2024-06-03T03:26:08.800 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:} 2024-06-03T03:26:08.800 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.799+0000 7eff33200640 1 -- 172.21.15.117:0/3437890135 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7eff180a2c10 msgr2=0x7eff180a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:08.800 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.799+0000 7eff33200640 1 --2- 172.21.15.117:0/3437890135 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7eff180a2c10 0x7eff180a50d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x5572126357f0 tx=0x7eff20005f90 comp rx=0 tx=0).stop 2024-06-03T03:26:08.801 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.799+0000 7eff33200640 1 -- 172.21.15.117:0/3437890135 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff341075d0 msgr2=0x7eff341a1320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:08.801 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.799+0000 7eff33200640 1 --2- 172.21.15.117:0/3437890135 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff341075d0 0x7eff341a1320 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7eff2c00da10 tx=0x7eff2c00ded0 comp rx=0 tx=0).stop 2024-06-03T03:26:08.801 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.799+0000 7eff33200640 1 -- 172.21.15.117:0/3437890135 shutdown_connections 2024-06-03T03:26:08.801 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.799+0000 7eff33200640 1 --2- 172.21.15.117:0/3437890135 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff341075d0 0x7eff341a1320 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:08.801 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.799+0000 7eff33200640 1 --2- 172.21.15.117:0/3437890135 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7eff180a2c10 0x7eff180a50d0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:08.801 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.799+0000 7eff33200640 1 --2- 172.21.15.117:0/3437890135 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff34104ca0 0x7eff3419a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:08.801 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.799+0000 7eff33200640 1 --2- 172.21.15.117:0/3437890135 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7eff34102370 0x7eff34199db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:08.801 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.799+0000 7eff33200640 1 -- 172.21.15.117:0/3437890135 >> 172.21.15.117:0/3437890135 conn(0x7eff340fbce0 msgr2=0x7eff34101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:26:08.801 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.799+0000 7eff33200640 1 -- 172.21.15.117:0/3437890135 shutdown_connections 2024-06-03T03:26:08.801 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:08.799+0000 7eff33200640 1 -- 172.21.15.117:0/3437890135 wait complete. 2024-06-03T03:26:10.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.048+0000 7fb07a800700 1 -- 172.21.15.117:0/2231887476 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb074103fc0 msgr2=0x7fb074104440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:10.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.048+0000 7fb07a800700 1 --2- 172.21.15.117:0/2231887476 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb074103fc0 0x7fb074104440 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb0600099c0 tx=0x7fb060009cd0 comp rx=0 tx=0).stop 2024-06-03T03:26:10.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.048+0000 7fb07a800700 1 -- 172.21.15.117:0/2231887476 shutdown_connections 2024-06-03T03:26:10.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.048+0000 7fb07a800700 1 --2- 172.21.15.117:0/2231887476 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb074104980 0x7fb07410f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:10.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.048+0000 7fb07a800700 1 --2- 172.21.15.117:0/2231887476 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0741026d0 0x7fb074102ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:10.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.048+0000 7fb07a800700 1 --2- 172.21.15.117:0/2231887476 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb074103fc0 0x7fb074104440 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:10.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.049+0000 7fb07a800700 1 -- 172.21.15.117:0/2231887476 >> 172.21.15.117:0/2231887476 conn(0x7fb0740fde80 msgr2=0x7fb0741002a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:26:10.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.049+0000 7fb07a800700 1 -- 172.21.15.117:0/2231887476 shutdown_connections 2024-06-03T03:26:10.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.049+0000 7fb07a800700 1 -- 172.21.15.117:0/2231887476 wait complete. 2024-06-03T03:26:10.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.050+0000 7fb07a800700 1 Processor -- start 2024-06-03T03:26:10.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.050+0000 7fb07a800700 1 -- start start 2024-06-03T03:26:10.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.050+0000 7fb07a800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0741026d0 0x7fb07419c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:10.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.050+0000 7fb07a800700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb074103fc0 0x7fb07419cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:10.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.051+0000 7fb07a800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb074104980 0x7fb074196820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:10.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.051+0000 7fb07a800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb074110640 con 0x7fb0741026d0 2024-06-03T03:26:10.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.051+0000 7fb07a800700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb0741104c0 con 0x7fb074104980 2024-06-03T03:26:10.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.051+0000 7fb07a800700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb0741107c0 con 0x7fb074103fc0 2024-06-03T03:26:10.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.051+0000 7fb079400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0741026d0 0x7fb07419c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:10.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.051+0000 7fb079400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0741026d0 0x7fb07419c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:57740/0 (socket says 172.21.15.117:57740) 2024-06-03T03:26:10.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.051+0000 7fb079400700 1 -- 172.21.15.117:0/250830900 learned_addr learned my addr 172.21.15.117:0/250830900 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:26:10.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.051+0000 7fb078a00700 1 --2- 172.21.15.117:0/250830900 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb074103fc0 0x7fb07419cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:10.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.051+0000 7fb079e00700 1 --2- 172.21.15.117:0/250830900 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb074104980 0x7fb074196820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:10.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.052+0000 7fb079400700 1 -- 172.21.15.117:0/250830900 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb074103fc0 msgr2=0x7fb07419cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:10.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.052+0000 7fb079400700 1 --2- 172.21.15.117:0/250830900 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb074103fc0 0x7fb07419cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:10.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.052+0000 7fb079400700 1 -- 172.21.15.117:0/250830900 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb074104980 msgr2=0x7fb074196820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:10.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.052+0000 7fb079400700 1 --2- 172.21.15.117:0/250830900 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb074104980 0x7fb074196820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:10.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.052+0000 7fb079400700 1 -- 172.21.15.117:0/250830900 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb074197000 con 0x7fb0741026d0 2024-06-03T03:26:10.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.052+0000 7fb079e00700 1 --2- 172.21.15.117:0/250830900 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb074104980 0x7fb074196820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T03:26:10.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.052+0000 7fb079400700 1 --2- 172.21.15.117:0/250830900 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0741026d0 0x7fb07419c650 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb06800b590 tx=0x7fb06800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:26:10.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.052+0000 7fb072000700 1 -- 172.21.15.117:0/250830900 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb068023070 con 0x7fb0741026d0 2024-06-03T03:26:10.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.052+0000 7fb072000700 1 -- 172.21.15.117:0/250830900 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb06800c9c0 con 0x7fb0741026d0 2024-06-03T03:26:10.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.052+0000 7fb072000700 1 -- 172.21.15.117:0/250830900 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb06801ea70 con 0x7fb0741026d0 2024-06-03T03:26:10.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.053+0000 7fb07a800700 1 -- 172.21.15.117:0/250830900 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0741972f0 con 0x7fb0741026d0 2024-06-03T03:26:10.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.053+0000 7fb07a800700 1 -- 172.21.15.117:0/250830900 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0741a34b0 con 0x7fb0741026d0 2024-06-03T03:26:10.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.058+0000 7fb086877b80 1 Processor -- start 2024-06-03T03:26:10.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.058+0000 7fb086877b80 1 -- start start 2024-06-03T03:26:10.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.058+0000 7fb072000700 1 -- 172.21.15.117:0/250830900 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7fb06800cb70 con 0x7fb0741026d0 2024-06-03T03:26:10.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.058+0000 7fb086877b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ea2732d60 0x559ea2733140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:10.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.058+0000 7fb086877b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ea2733680 0x559ea273dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:10.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.059+0000 7fb086877b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ea273e530 0x559ea2740920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:10.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.059+0000 7fb086877b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559ea2642a10 con 0x559ea2732d60 2024-06-03T03:26:10.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.059+0000 7fb086877b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559ea2685b00 con 0x559ea2733680 2024-06-03T03:26:10.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.059+0000 7fb086877b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559ea26408f0 con 0x559ea273e530 2024-06-03T03:26:10.060 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.059+0000 7fb079400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ea2733680 0x559ea273dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:10.060 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.059+0000 7fb079400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ea2733680 0x559ea273dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:46802/0 (socket says 172.21.15.117:46802) 2024-06-03T03:26:10.060 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.059+0000 7fb079400700 1 -- 172.21.15.117:0/2382327255 learned_addr learned my addr 172.21.15.117:0/2382327255 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:26:10.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.059+0000 7fb079e00700 1 --2- 172.21.15.117:0/2382327255 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ea2732d60 0x559ea2733140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:10.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.059+0000 7fb078a00700 1 --2- 172.21.15.117:0/2382327255 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ea273e530 0x559ea2740920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:10.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.060+0000 7fb079400700 1 -- 172.21.15.117:0/2382327255 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ea273e530 msgr2=0x559ea2740920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:10.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.060+0000 7fb079400700 1 --2- 172.21.15.117:0/2382327255 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ea273e530 0x559ea2740920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:10.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.060+0000 7fb079400700 1 -- 172.21.15.117:0/2382327255 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ea2732d60 msgr2=0x559ea2733140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:10.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.060+0000 7fb079400700 1 --2- 172.21.15.117:0/2382327255 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ea2732d60 0x559ea2733140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:10.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.060+0000 7fb079400700 1 -- 172.21.15.117:0/2382327255 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ea262df90 con 0x559ea2733680 2024-06-03T03:26:10.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.060+0000 7fb079400700 1 --2- 172.21.15.117:0/2382327255 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ea2733680 0x559ea273dff0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb0680d5330 tx=0x7fb0680dbf50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4539e4002269359b server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:26:10.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.060+0000 7fb072000700 1 --2- 172.21.15.117:0/250830900 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fb05c0a2cb0 0x7fb05c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:10.062 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.060+0000 7fb066a00700 1 -- 172.21.15.117:0/2382327255 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0680eb070 con 0x559ea2733680 2024-06-03T03:26:10.062 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.061+0000 7fb072000700 1 -- 172.21.15.117:0/250830900 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7fb0680d7dd0 con 0x7fb0741026d0 2024-06-03T03:26:10.062 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.061+0000 7fb066a00700 1 -- 172.21.15.117:0/2382327255 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0680a4840 con 0x559ea2733680 2024-06-03T03:26:10.062 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.061+0000 7fb079e00700 1 --2- 172.21.15.117:0/250830900 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fb05c0a2cb0 0x7fb05c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:10.062 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.061+0000 7fb066a00700 1 -- 172.21.15.117:0/2382327255 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0680a3c80 con 0x559ea2733680 2024-06-03T03:26:10.063 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.062+0000 7fb086877b80 1 -- 172.21.15.117:0/2382327255 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ea2733680 msgr2=0x559ea273dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:10.063 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.062+0000 7fb079e00700 1 --2- 172.21.15.117:0/250830900 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fb05c0a2cb0 0x7fb05c0a5170 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb06c009fd0 tx=0x7fb06c009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:26:10.063 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.062+0000 7fb086877b80 1 --2- 172.21.15.117:0/2382327255 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ea2733680 0x559ea273dff0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb0680d5330 tx=0x7fb0680dbf50 comp rx=0 tx=0).stop 2024-06-03T03:26:10.063 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.062+0000 7fb086877b80 1 -- 172.21.15.117:0/2382327255 shutdown_connections 2024-06-03T03:26:10.063 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.062+0000 7fb086877b80 1 --2- 172.21.15.117:0/2382327255 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ea273e530 0x559ea2740920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:10.063 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.062+0000 7fb086877b80 1 --2- 172.21.15.117:0/2382327255 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ea2732d60 0x559ea2733140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:10.063 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.062+0000 7fb086877b80 1 --2- 172.21.15.117:0/2382327255 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ea2733680 0x559ea273dff0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:10.063 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.062+0000 7fb086877b80 1 -- 172.21.15.117:0/2382327255 >> 172.21.15.117:0/2382327255 conn(0x559ea2660170 msgr2=0x559ea272f280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:26:10.063 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.062+0000 7fb086877b80 1 -- 172.21.15.117:0/2382327255 shutdown_connections 2024-06-03T03:26:10.063 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.062+0000 7fb086877b80 1 -- 172.21.15.117:0/2382327255 wait complete. 2024-06-03T03:26:10.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.063+0000 7fb086877b80 1 -- start start 2024-06-03T03:26:10.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.063+0000 7fb086877b80 1 Processor -- start 2024-06-03T03:26:10.065 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.064+0000 7fb086877b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ea2732d60 0x559ea273dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:10.065 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.064+0000 7fb086877b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ea2733680 0x559ea273af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:10.065 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.064+0000 7fb086877b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ea273e530 0x559ea273b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:10.065 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.064+0000 7fb086877b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559ea2621f60 con 0x559ea2733680 2024-06-03T03:26:10.065 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.064+0000 7fb086877b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559ea2681ac0 con 0x559ea273e530 2024-06-03T03:26:10.065 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.064+0000 7fb086877b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559ea2740e60 con 0x559ea2732d60 2024-06-03T03:26:10.065 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.064+0000 7fb086877b80 10 client.0 _ll_register_callbacks cb 0x559ea261b380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:26:10.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.064+0000 7fb079e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ea273e530 0x559ea273b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:10.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.064+0000 7fb078a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ea2733680 0x559ea273af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:10.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.064+0000 7fb079e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ea273e530 0x559ea273b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:46804/0 (socket says 172.21.15.117:46804) 2024-06-03T03:26:10.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.064+0000 7fb079e00700 1 -- 172.21.15.117:0/4286223834 learned_addr learned my addr 172.21.15.117:0/4286223834 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:26:10.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.064+0000 7fb079400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ea2732d60 0x559ea273dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:10.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.065+0000 7fb079e00700 1 -- 172.21.15.117:0/4286223834 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ea2732d60 msgr2=0x559ea273dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:10.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.065+0000 7fb079e00700 1 --2- 172.21.15.117:0/4286223834 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ea2732d60 0x559ea273dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:10.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.065+0000 7fb079e00700 1 -- 172.21.15.117:0/4286223834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ea2733680 msgr2=0x559ea273af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:10.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.065+0000 7fb079e00700 1 --2- 172.21.15.117:0/4286223834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ea2733680 0x559ea273af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:10.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.065+0000 7fb079e00700 1 -- 172.21.15.117:0/4286223834 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ea25d48d0 con 0x559ea273e530 2024-06-03T03:26:10.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.065+0000 7fb079e00700 1 --2- 172.21.15.117:0/4286223834 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ea273e530 0x559ea273b450 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb06c018040 tx=0x7fb06c00b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d45c288f0b6850a7 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:26:10.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.065+0000 7fb05b400700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-06-03T03:26:10.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.066+0000 7fb05b400700 1 -- 172.21.15.117:0/4286223834 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb06c026030 con 0x559ea273e530 2024-06-03T03:26:10.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.066+0000 7fb05b400700 1 -- 172.21.15.117:0/4286223834 <== mon.2 v2:172.21.15.151:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7fb06c015a30 con 0x559ea273e530 2024-06-03T03:26:10.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.066+0000 7fb086877b80 1 -- 172.21.15.117:0/4286223834 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0680a4840 con 0x559ea273e530 2024-06-03T03:26:10.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.066+0000 7fb05b400700 1 -- 172.21.15.117:0/4286223834 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fb06c015c30 con 0x559ea273e530 2024-06-03T03:26:10.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.066+0000 7fb086877b80 10 client.25364 fetch_fsmap learned FSMap version 31 2024-06-03T03:26:10.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.066+0000 7fb086877b80 1 -- 172.21.15.117:0/4286223834 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x559ea274b5a0 con 0x559ea273e530 2024-06-03T03:26:10.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.067+0000 7fb05b400700 1 -- 172.21.15.117:0/4286223834 <== mon.2 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0680a4840 con 0x559ea273e530 2024-06-03T03:26:10.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.067+0000 7fb05b400700 1 -- 172.21.15.117:0/4286223834 <== mon.2 v2:172.21.15.151:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fb06c02a3a0 con 0x559ea273e530 2024-06-03T03:26:10.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.067+0000 7fb086877b80 10 client.25364 fetch_fsmap finished waiting for FSMap version 31 2024-06-03T03:26:10.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.067+0000 7fb086877b80 10 client.25364 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-03T03:26:10.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.067+0000 7fb086877b80 20 client.25364 populate_metadata read hostname 'smithi117' 2024-06-03T03:26:10.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.067+0000 7fb086877b80 1 --2- 172.21.15.117:0/4286223834 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x559ea274dc30 0x559ea2750020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:10.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.067+0000 7fb086877b80 4 client.25364 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T03:26:10.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.067+0000 7fb086877b80 1 -- 172.21.15.117:0/4286223834 --> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559ea2693000 con 0x559ea274dc30 2024-06-03T03:26:10.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.068+0000 7fb079400700 1 --2- 172.21.15.117:0/4286223834 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x559ea274dc30 0x559ea2750020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:10.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.068+0000 7fb079400700 1 --2- 172.21.15.117:0/4286223834 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x559ea274dc30 0x559ea2750020 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5005ef2a2b1d061 server_cookie=94230ca2419b31f8 in_seq=0 out_seq=0 2024-06-03T03:26:10.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.068+0000 7fb05b400700 10 client.25364 ms_handle_connect on v2:172.21.15.183:6838/1439615083 2024-06-03T03:26:10.070 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.069+0000 7fb05b400700 1 -- 172.21.15.117:0/4286223834 <== mds.0 v2:172.21.15.183:6838/1439615083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559ea2693000 con 0x559ea274dc30 2024-06-03T03:26:10.070 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.069+0000 7fb05b400700 10 client.25364 handle_command_reply: tid=0 2024-06-03T03:26:10.070 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.069+0000 7fb086877b80 1 client.25364 shutdown 2024-06-03T03:26:10.070 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.069+0000 7fb086877b80 1 -- 172.21.15.117:0/4286223834 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ea273e530 msgr2=0x559ea273b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:10.070 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.069+0000 7fb086877b80 1 --2- 172.21.15.117:0/4286223834 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ea273e530 0x559ea273b450 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb06c018040 tx=0x7fb06c00b040 comp rx=0 tx=0).stop 2024-06-03T03:26:10.071 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.069+0000 7fb086877b80 1 -- 172.21.15.117:0/4286223834 shutdown_connections 2024-06-03T03:26:10.071 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.069+0000 7fb086877b80 1 --2- 172.21.15.117:0/4286223834 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x559ea274dc30 0x559ea2750020 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:10.071 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.069+0000 7fb086877b80 1 --2- 172.21.15.117:0/4286223834 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ea273e530 0x559ea273b450 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:10.071 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.069+0000 7fb086877b80 1 --2- 172.21.15.117:0/4286223834 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ea2732d60 0x559ea273dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:10.071 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.069+0000 7fb086877b80 1 --2- 172.21.15.117:0/4286223834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ea2733680 0x559ea273af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:10.071 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.069+0000 7fb05b400700 0 client.25364 ms_handle_reset on v2:172.21.15.183:6838/1439615083 2024-06-03T03:26:10.071 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.070+0000 7fb086877b80 1 -- 172.21.15.117:0/4286223834 >> 172.21.15.117:0/4286223834 conn(0x559ea2660170 msgr2=0x559ea272ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:26:10.071 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.070+0000 7fb086877b80 1 -- 172.21.15.117:0/4286223834 shutdown_connections 2024-06-03T03:26:10.072 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.070+0000 7fb086877b80 1 -- 172.21.15.117:0/4286223834 wait complete. 2024-06-03T03:26:10.072 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.070+0000 7fb086877b80 20 client.25364 trim_cache size 0 max 16384 2024-06-03T03:26:10.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.094+0000 7fb086877b80 1 Processor -- start 2024-06-03T03:26:10.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.094+0000 7fb086877b80 1 -- start start 2024-06-03T03:26:10.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.094+0000 7fb086877b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ea2732d60 0x559ea27365f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:10.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.094+0000 7fb086877b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ea2733680 0x559ea27558c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:10.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.094+0000 7fb086877b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ea273e530 0x559ea2755e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:10.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.094+0000 7fb086877b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb0741abf50 con 0x559ea2732d60 2024-06-03T03:26:10.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.094+0000 7fb086877b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb0741ada90 con 0x559ea273e530 2024-06-03T03:26:10.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.094+0000 7fb086877b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb0741ac620 con 0x559ea2733680 2024-06-03T03:26:10.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.095+0000 7fb079400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ea2732d60 0x559ea27365f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:10.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.095+0000 7fb079400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ea2732d60 0x559ea27365f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:57756/0 (socket says 172.21.15.117:57756) 2024-06-03T03:26:10.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.095+0000 7fb079400700 1 -- 172.21.15.117:0/2195451459 learned_addr learned my addr 172.21.15.117:0/2195451459 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:26:10.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.095+0000 7fb079e00700 1 --2- 172.21.15.117:0/2195451459 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ea273e530 0x559ea2755e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:10.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.095+0000 7fb078a00700 1 --2- 172.21.15.117:0/2195451459 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ea2733680 0x559ea27558c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:10.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.095+0000 7fb079400700 1 -- 172.21.15.117:0/2195451459 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ea2733680 msgr2=0x559ea27558c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:10.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.095+0000 7fb079400700 1 --2- 172.21.15.117:0/2195451459 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ea2733680 0x559ea27558c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:10.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.095+0000 7fb079400700 1 -- 172.21.15.117:0/2195451459 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ea273e530 msgr2=0x559ea2755e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:10.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.095+0000 7fb079400700 1 --2- 172.21.15.117:0/2195451459 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ea273e530 0x559ea2755e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:10.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.095+0000 7fb079400700 1 -- 172.21.15.117:0/2195451459 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ea2750650 con 0x559ea2732d60 2024-06-03T03:26:10.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.096+0000 7fb079400700 1 --2- 172.21.15.117:0/2195451459 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ea2732d60 0x559ea27365f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb0680a3e40 tx=0x7fb0680c4040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ef752dd42e7171 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:26:10.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.096+0000 7fb066000700 1 -- 172.21.15.117:0/2195451459 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0680a3aa0 con 0x559ea2732d60 2024-06-03T03:26:10.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.096+0000 7fb066000700 1 -- 172.21.15.117:0/2195451459 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0680a4840 con 0x559ea2732d60 2024-06-03T03:26:10.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.096+0000 7fb066000700 1 -- 172.21.15.117:0/2195451459 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb06800bdd0 con 0x559ea2732d60 2024-06-03T03:26:10.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.097+0000 7fb086877b80 1 -- 172.21.15.117:0/2195451459 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ea2732d60 msgr2=0x559ea27365f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:10.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.097+0000 7fb086877b80 1 --2- 172.21.15.117:0/2195451459 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ea2732d60 0x559ea27365f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb0680a3e40 tx=0x7fb0680c4040 comp rx=0 tx=0).stop 2024-06-03T03:26:10.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.097+0000 7fb086877b80 1 -- 172.21.15.117:0/2195451459 shutdown_connections 2024-06-03T03:26:10.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.097+0000 7fb086877b80 1 --2- 172.21.15.117:0/2195451459 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ea273e530 0x559ea2755e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:10.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.097+0000 7fb086877b80 1 --2- 172.21.15.117:0/2195451459 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ea2732d60 0x559ea27365f0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:10.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.097+0000 7fb086877b80 1 --2- 172.21.15.117:0/2195451459 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ea2733680 0x559ea27558c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:10.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.097+0000 7fb086877b80 1 -- 172.21.15.117:0/2195451459 >> 172.21.15.117:0/2195451459 conn(0x559ea2660170 msgr2=0x559ea2735ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:26:10.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.097+0000 7fb086877b80 1 -- 172.21.15.117:0/2195451459 shutdown_connections 2024-06-03T03:26:10.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.097+0000 7fb086877b80 1 -- 172.21.15.117:0/2195451459 wait complete. 2024-06-03T03:26:10.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.098+0000 7fb086877b80 1 -- start start 2024-06-03T03:26:10.100 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.098+0000 7fb086877b80 1 Processor -- start 2024-06-03T03:26:10.100 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.099+0000 7fb086877b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ea2733680 0x559ea2746630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:10.100 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.099+0000 7fb086877b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ea273e530 0x559ea27410f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:10.100 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.099+0000 7fb086877b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ea2741630 0x559ea2741ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:10.100 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.099+0000 7fb086877b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb0741ad3c0 con 0x559ea273e530 2024-06-03T03:26:10.100 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.099+0000 7fb086877b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb0741accf0 con 0x559ea2733680 2024-06-03T03:26:10.100 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.099+0000 7fb086877b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb0741a9ef0 con 0x559ea2741630 2024-06-03T03:26:10.100 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.099+0000 7fb086877b80 10 client.0 _ll_register_callbacks cb 0x559ea261b380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:26:10.100 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.099+0000 7fb078a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ea273e530 0x559ea27410f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:10.100 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.099+0000 7fb079400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ea2733680 0x559ea2746630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:10.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.099+0000 7fb078a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ea273e530 0x559ea27410f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:57766/0 (socket says 172.21.15.117:57766) 2024-06-03T03:26:10.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.099+0000 7fb078a00700 1 -- 172.21.15.117:0/3189437482 learned_addr learned my addr 172.21.15.117:0/3189437482 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:26:10.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.100+0000 7fb078a00700 1 -- 172.21.15.117:0/3189437482 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ea2741630 msgr2=0x559ea2741ab0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-03T03:26:10.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.100+0000 7fb078a00700 1 --2- 172.21.15.117:0/3189437482 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ea2741630 0x559ea2741ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:10.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.100+0000 7fb078a00700 1 -- 172.21.15.117:0/3189437482 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ea2733680 msgr2=0x559ea2746630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:10.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.100+0000 7fb078a00700 1 --2- 172.21.15.117:0/3189437482 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ea2733680 0x559ea2746630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:10.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.100+0000 7fb078a00700 1 -- 172.21.15.117:0/3189437482 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ea2746dd0 con 0x559ea273e530 2024-06-03T03:26:10.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.100+0000 7fb078a00700 1 --2- 172.21.15.117:0/3189437482 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ea273e530 0x559ea27410f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb060009990 tx=0x7fb060009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=571659019c052dc server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:26:10.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.100+0000 7fb05b400700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-06-03T03:26:10.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.101+0000 7fb05b400700 1 -- 172.21.15.117:0/3189437482 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0600054c0 con 0x559ea273e530 2024-06-03T03:26:10.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.101+0000 7fb05b400700 1 -- 172.21.15.117:0/3189437482 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7fb060005660 con 0x559ea273e530 2024-06-03T03:26:10.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.101+0000 7fb086877b80 1 -- 172.21.15.117:0/3189437482 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0680a4840 con 0x559ea273e530 2024-06-03T03:26:10.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.101+0000 7fb05b400700 1 -- 172.21.15.117:0/3189437482 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fb060032050 con 0x559ea273e530 2024-06-03T03:26:10.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.101+0000 7fb086877b80 10 client.15261 fetch_fsmap learned FSMap version 31 2024-06-03T03:26:10.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.101+0000 7fb086877b80 1 -- 172.21.15.117:0/3189437482 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x559ea27b99b0 con 0x559ea273e530 2024-06-03T03:26:10.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.102+0000 7fb05b400700 1 -- 172.21.15.117:0/3189437482 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0680a4840 con 0x559ea273e530 2024-06-03T03:26:10.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.102+0000 7fb05b400700 1 -- 172.21.15.117:0/3189437482 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fb06000f8b0 con 0x559ea273e530 2024-06-03T03:26:10.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.102+0000 7fb086877b80 10 client.15261 fetch_fsmap finished waiting for FSMap version 31 2024-06-03T03:26:10.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.102+0000 7fb086877b80 10 client.15261 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-03T03:26:10.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.102+0000 7fb086877b80 20 client.15261 populate_metadata read hostname 'smithi117' 2024-06-03T03:26:10.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.102+0000 7fb086877b80 1 --2- 172.21.15.117:0/3189437482 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x559ea27ba140 0x559ea27bc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:10.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.102+0000 7fb086877b80 4 client.15261 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-03T03:26:10.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.102+0000 7fb086877b80 1 -- 172.21.15.117:0/3189437482 --> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559ea268d580 con 0x559ea27ba140 2024-06-03T03:26:10.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.102+0000 7fb079400700 1 --2- 172.21.15.117:0/3189437482 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x559ea27ba140 0x559ea27bc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:10.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.103+0000 7fb079400700 1 --2- 172.21.15.117:0/3189437482 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x559ea27ba140 0x559ea27bc560 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5ad8656afe9cfa3 server_cookie=d8ce1c52a41617df in_seq=0 out_seq=0 2024-06-03T03:26:10.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.103+0000 7fb05b400700 10 client.15261 ms_handle_connect on v2:172.21.15.183:6838/1439615083 2024-06-03T03:26:10.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.103+0000 7fb05b400700 1 -- 172.21.15.117:0/3189437482 <== mds.0 v2:172.21.15.183:6838/1439615083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559ea268d580 con 0x559ea27ba140 2024-06-03T03:26:10.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.103+0000 7fb05b400700 10 client.15261 handle_command_reply: tid=0 2024-06-03T03:26:10.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.103+0000 7fb086877b80 1 client.15261 shutdown 2024-06-03T03:26:10.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.103+0000 7fb086877b80 1 -- 172.21.15.117:0/3189437482 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ea273e530 msgr2=0x559ea27410f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:10.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.103+0000 7fb086877b80 1 --2- 172.21.15.117:0/3189437482 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ea273e530 0x559ea27410f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb060009990 tx=0x7fb060009ee0 comp rx=0 tx=0).stop 2024-06-03T03:26:10.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.103+0000 7fb086877b80 1 -- 172.21.15.117:0/3189437482 shutdown_connections 2024-06-03T03:26:10.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.103+0000 7fb086877b80 1 --2- 172.21.15.117:0/3189437482 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x559ea27ba140 0x559ea27bc560 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:10.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.103+0000 7fb086877b80 1 --2- 172.21.15.117:0/3189437482 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ea2741630 0x559ea2741ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:10.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.103+0000 7fb086877b80 1 --2- 172.21.15.117:0/3189437482 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ea2733680 0x559ea2746630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:10.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.103+0000 7fb086877b80 1 --2- 172.21.15.117:0/3189437482 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ea273e530 0x559ea27410f0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:10.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.103+0000 7fb086877b80 1 -- 172.21.15.117:0/3189437482 >> 172.21.15.117:0/3189437482 conn(0x559ea2660170 msgr2=0x559ea273c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:26:10.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.104+0000 7fb05b400700 0 client.15261 ms_handle_reset on v2:172.21.15.183:6838/1439615083 2024-06-03T03:26:10.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.104+0000 7fb086877b80 1 -- 172.21.15.117:0/3189437482 shutdown_connections 2024-06-03T03:26:10.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.104+0000 7fb086877b80 1 -- 172.21.15.117:0/3189437482 wait complete. 2024-06-03T03:26:10.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.104+0000 7fb086877b80 20 client.15261 trim_cache size 0 max 16384 2024-06-03T03:26:10.107 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-06-03T03:26:10.107 INFO:teuthology.orchestra.run.smithi117.stdout: "status": "no active scrubs running", 2024-06-03T03:26:10.107 INFO:teuthology.orchestra.run.smithi117.stdout: "scrubs": {} 2024-06-03T03:26:10.107 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-06-03T03:26:10.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.105+0000 7fb07a800700 1 -- 172.21.15.117:0/250830900 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fb05c0a2cb0 msgr2=0x7fb05c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:10.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.105+0000 7fb07a800700 1 --2- 172.21.15.117:0/250830900 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fb05c0a2cb0 0x7fb05c0a5170 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb06c009fd0 tx=0x7fb06c009040 comp rx=0 tx=0).stop 2024-06-03T03:26:10.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.105+0000 7fb07a800700 1 -- 172.21.15.117:0/250830900 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0741026d0 msgr2=0x7fb07419c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:10.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.105+0000 7fb07a800700 1 --2- 172.21.15.117:0/250830900 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0741026d0 0x7fb07419c650 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb06800b590 tx=0x7fb06800b8a0 comp rx=0 tx=0).stop 2024-06-03T03:26:10.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.105+0000 7fb07a800700 1 -- 172.21.15.117:0/250830900 shutdown_connections 2024-06-03T03:26:10.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.105+0000 7fb07a800700 1 --2- 172.21.15.117:0/250830900 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fb05c0a2cb0 0x7fb05c0a5170 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:10.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.105+0000 7fb07a800700 1 --2- 172.21.15.117:0/250830900 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb074104980 0x7fb074196820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:10.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.105+0000 7fb07a800700 1 --2- 172.21.15.117:0/250830900 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0741026d0 0x7fb07419c650 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:10.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.105+0000 7fb07a800700 1 --2- 172.21.15.117:0/250830900 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb074103fc0 0x7fb07419cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:10.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.105+0000 7fb07a800700 1 -- 172.21.15.117:0/250830900 >> 172.21.15.117:0/250830900 conn(0x7fb0740fde80 msgr2=0x7fb0740fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:26:10.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.105+0000 7fb07a800700 1 -- 172.21.15.117:0/250830900 shutdown_connections 2024-06-03T03:26:10.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:10.106+0000 7fb07a800700 1 -- 172.21.15.117:0/250830900 wait complete. 2024-06-03T03:26:10.127 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:26:10 smithi117 ceph-mon[31649]: pgmap v531: 129 pgs: 129 active+clean; 677 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-06-03T03:26:10.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:26:10 smithi151 ceph-mon[37043]: pgmap v531: 129 pgs: 129 active+clean; 677 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-06-03T03:26:10.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:26:10 smithi183 ceph-mon[37326]: pgmap v531: 129 pgs: 129 active+clean; 677 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-06-03T03:26:11.151 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-03T03:26:11.152 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-03T03:26:11.606 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:26:12.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:26:12 smithi151 ceph-mon[37043]: pgmap v532: 129 pgs: 129 active+clean; 677 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-06-03T03:26:12.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:26:12 smithi117 ceph-mon[31649]: pgmap v532: 129 pgs: 129 active+clean; 677 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-06-03T03:26:12.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:26:12 smithi183 ceph-mon[37326]: pgmap v532: 129 pgs: 129 active+clean; 677 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-06-03T03:26:14.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:26:14 smithi151 ceph-mon[37043]: pgmap v533: 129 pgs: 129 active+clean; 678 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-06-03T03:26:14.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:26:14 smithi117 ceph-mon[31649]: pgmap v533: 129 pgs: 129 active+clean; 678 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-06-03T03:26:14.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:26:14 smithi183 ceph-mon[37326]: pgmap v533: 129 pgs: 129 active+clean; 678 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-06-03T03:26:15.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:26:15 smithi151 ceph-mon[37043]: pgmap v534: 129 pgs: 129 active+clean; 678 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-06-03T03:26:15.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:26:15 smithi117 ceph-mon[31649]: pgmap v534: 129 pgs: 129 active+clean; 678 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-06-03T03:26:15.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:26:15 smithi183 ceph-mon[37326]: pgmap v534: 129 pgs: 129 active+clean; 678 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-06-03T03:26:16.831 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:16.829+0000 7f5b32800700 1 -- 172.21.15.117:0/3859497682 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b2c10ec60 msgr2=0x7f5b2c111070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:16.831 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:16.829+0000 7f5b32800700 1 --2- 172.21.15.117:0/3859497682 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b2c10ec60 0x7f5b2c111070 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f5b2400b0a0 tx=0x7f5b2400b3b0 comp rx=0 tx=0).stop 2024-06-03T03:26:16.831 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:16.829+0000 7f5b32800700 1 -- 172.21.15.117:0/3859497682 shutdown_connections 2024-06-03T03:26:16.831 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:16.829+0000 7f5b32800700 1 --2- 172.21.15.117:0/3859497682 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b2c10ec60 0x7f5b2c111070 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:16.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:16.829+0000 7f5b32800700 1 --2- 172.21.15.117:0/3859497682 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5b2c074860 0x7f5b2c074c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:16.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:16.829+0000 7f5b32800700 1 --2- 172.21.15.117:0/3859497682 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b2c075180 0x7f5b2c10e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:16.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:16.829+0000 7f5b32800700 1 -- 172.21.15.117:0/3859497682 >> 172.21.15.117:0/3859497682 conn(0x7f5b2c0fdf10 msgr2=0x7f5b2c100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:26:16.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:16.829+0000 7f5b32800700 1 -- 172.21.15.117:0/3859497682 shutdown_connections 2024-06-03T03:26:16.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:16.829+0000 7f5b32800700 1 -- 172.21.15.117:0/3859497682 wait complete. 2024-06-03T03:26:16.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:16.830+0000 7f5b32800700 1 Processor -- start 2024-06-03T03:26:16.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:16.830+0000 7f5b32800700 1 -- start start 2024-06-03T03:26:16.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:16.831+0000 7f5b32800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b2c074860 0x7f5b2c19ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:16.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:16.831+0000 7f5b32800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5b2c075180 0x7f5b2c19b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:16.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:16.831+0000 7f5b32800700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b2c10ec60 0x7f5b2c19f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:16.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:16.831+0000 7f5b32800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5b2c114940 con 0x7f5b2c074860 2024-06-03T03:26:16.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:16.831+0000 7f5b32800700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5b2c1147c0 con 0x7f5b2c075180 2024-06-03T03:26:16.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:16.831+0000 7f5b32800700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5b2c114ac0 con 0x7f5b2c10ec60 2024-06-03T03:26:16.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:16.831+0000 7f5b31400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b2c074860 0x7f5b2c19ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:16.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:16.831+0000 7f5b31400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b2c074860 0x7f5b2c19ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:57796/0 (socket says 172.21.15.117:57796) 2024-06-03T03:26:16.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:16.831+0000 7f5b31400700 1 -- 172.21.15.117:0/1517716549 learned_addr learned my addr 172.21.15.117:0/1517716549 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:26:16.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:16.831+0000 7f5b30a00700 1 --2- 172.21.15.117:0/1517716549 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5b2c075180 0x7f5b2c19b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:16.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:16.831+0000 7f5b31e00700 1 --2- 172.21.15.117:0/1517716549 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b2c10ec60 0x7f5b2c19f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:16.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:16.832+0000 7f5b30a00700 1 -- 172.21.15.117:0/1517716549 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b2c10ec60 msgr2=0x7f5b2c19f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:16.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:16.832+0000 7f5b30a00700 1 --2- 172.21.15.117:0/1517716549 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b2c10ec60 0x7f5b2c19f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:16.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:16.832+0000 7f5b30a00700 1 -- 172.21.15.117:0/1517716549 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b2c074860 msgr2=0x7f5b2c19ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:16.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:16.832+0000 7f5b30a00700 1 --2- 172.21.15.117:0/1517716549 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b2c074860 0x7f5b2c19ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:16.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:16.832+0000 7f5b30a00700 1 -- 172.21.15.117:0/1517716549 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b2c19fd40 con 0x7f5b2c075180 2024-06-03T03:26:16.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:16.832+0000 7f5b31400700 1 --2- 172.21.15.117:0/1517716549 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b2c074860 0x7f5b2c19ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:26:16.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:16.832+0000 7f5b30a00700 1 --2- 172.21.15.117:0/1517716549 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5b2c075180 0x7f5b2c19b270 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5b1800e9f0 tx=0x7f5b1800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:26:16.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:16.832+0000 7f5b2a000700 1 -- 172.21.15.117:0/1517716549 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b18004220 con 0x7f5b2c075180 2024-06-03T03:26:16.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:16.832+0000 7f5b32800700 1 -- 172.21.15.117:0/1517716549 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b2c1a0030 con 0x7f5b2c075180 2024-06-03T03:26:16.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:16.832+0000 7f5b2a000700 1 -- 172.21.15.117:0/1517716549 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b18004d10 con 0x7f5b2c075180 2024-06-03T03:26:16.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:16.832+0000 7f5b2a000700 1 -- 172.21.15.117:0/1517716549 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b18005020 con 0x7f5b2c075180 2024-06-03T03:26:16.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:16.832+0000 7f5b32800700 1 -- 172.21.15.117:0/1517716549 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b2c1a78b0 con 0x7f5b2c075180 2024-06-03T03:26:16.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:16.833+0000 7f5b32800700 1 -- 172.21.15.117:0/1517716549 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b2c069180 con 0x7f5b2c075180 2024-06-03T03:26:16.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:16.836+0000 7f5b2a000700 1 -- 172.21.15.117:0/1517716549 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f5b18004750 con 0x7f5b2c075180 2024-06-03T03:26:16.838 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:16.837+0000 7f5b2a000700 1 --2- 172.21.15.117:0/1517716549 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f5b140a2cb0 0x7f5b140a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:16.838 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:16.837+0000 7f5b2a000700 1 -- 172.21.15.117:0/1517716549 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f5b180cf4f0 con 0x7f5b2c075180 2024-06-03T03:26:16.838 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:16.837+0000 7f5b31400700 1 --2- 172.21.15.117:0/1517716549 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f5b140a2cb0 0x7f5b140a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:16.838 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:16.838+0000 7f5b31400700 1 --2- 172.21.15.117:0/1517716549 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f5b140a2cb0 0x7f5b140a5170 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f5b20011550 tx=0x7f5b20009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:26:16.840 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:16.839+0000 7f5b2a000700 1 -- 172.21.15.117:0/1517716549 <== mon.2 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5b1809bf90 con 0x7f5b2c075180 2024-06-03T03:26:17.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:17.147+0000 7f5b32800700 1 -- 172.21.15.117:0/1517716549 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5b2c19c090 con 0x7f5b2c075180 2024-06-03T03:26:17.149 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:17.148+0000 7f5b2a000700 1 -- 172.21.15.117:0/1517716549 <== mon.2 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11955 (secure 0 0 0) 0x7f5b2c19c090 con 0x7f5b2c075180 2024-06-03T03:26:17.149 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T03:26:17.150 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3353001119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3353001119},{"type":"v1","addr":"172.21.15.117:6837","nonce":3353001119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14388,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6839/405782304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":405782304},{"type":"v1","addr":"172.21.15.117:6839","nonce":405782304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-03T03:19:44.629822+0000","modified":"2024-06-03T03:25:11.576498+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24517,"mds_1":24512,"mds_2":24503,"mds_3":24499,"mds_4":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24431":{"gid":24431,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6833/3355662416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3355662416},{"type":"v1","addr":"172.21.15.151:6833","nonce":3355662416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/2429725085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2429725085},{"type":"v1","addr":"172.21.15.183:6833","nonce":2429725085}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/31153579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":31153579},{"type":"v1","addr":"172.21.15.151:6835","nonce":31153579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"j","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/2884460088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2884460088},{"type":"v1","addr":"172.21.15.117:6835","nonce":2884460088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/2392407848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":2392407848},{"type":"v1","addr":"172.21.15.151:6837","nonce":2392407848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":32,"addr":"172.21.15.183:6835/1009004320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1009004320},{"type":"v1","addr":"172.21.15.183:6835","nonce":1009004320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.183:6837/2697518526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":2697518526},{"type":"v1","addr":"172.21.15.183:6837","nonce":2697518526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"a","rank":2,"incarnation":22,"state":"up:active","state_seq":27,"addr":"172.21.15.117:6841/1212242967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6840","nonce":1212242967},{"type":"v1","addr":"172.21.15.117:6841","nonce":1212242967}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.151:6839/3623444616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":3623444616},{"type":"v1","addr":"172.21.15.151:6839","nonce":3623444616}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":10,"addr":"172.21.15.183:6839/1439615083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1439615083},{"type":"v1","addr":"172.21.15.183:6839","nonce":1439615083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24517,"qdb_cluster":[24499,24487,24512,24503,24517]},"id":1}]} 2024-06-03T03:26:17.152 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:17.151+0000 7f5b32800700 1 -- 172.21.15.117:0/1517716549 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f5b140a2cb0 msgr2=0x7f5b140a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:17.152 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:17.151+0000 7f5b32800700 1 --2- 172.21.15.117:0/1517716549 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f5b140a2cb0 0x7f5b140a5170 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f5b20011550 tx=0x7f5b20009260 comp rx=0 tx=0).stop 2024-06-03T03:26:17.152 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:17.151+0000 7f5b32800700 1 -- 172.21.15.117:0/1517716549 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5b2c075180 msgr2=0x7f5b2c19b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:17.152 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:17.151+0000 7f5b32800700 1 --2- 172.21.15.117:0/1517716549 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5b2c075180 0x7f5b2c19b270 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5b1800e9f0 tx=0x7f5b1800ed00 comp rx=0 tx=0).stop 2024-06-03T03:26:17.152 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:17.151+0000 7f5b32800700 1 -- 172.21.15.117:0/1517716549 shutdown_connections 2024-06-03T03:26:17.152 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:17.151+0000 7f5b32800700 1 --2- 172.21.15.117:0/1517716549 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f5b140a2cb0 0x7f5b140a5170 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:17.153 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:17.151+0000 7f5b32800700 1 --2- 172.21.15.117:0/1517716549 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b2c10ec60 0x7f5b2c19f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:17.153 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:17.151+0000 7f5b32800700 1 --2- 172.21.15.117:0/1517716549 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b2c074860 0x7f5b2c19ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:17.153 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:17.151+0000 7f5b32800700 1 --2- 172.21.15.117:0/1517716549 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5b2c075180 0x7f5b2c19b270 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:17.153 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:17.151+0000 7f5b32800700 1 -- 172.21.15.117:0/1517716549 >> 172.21.15.117:0/1517716549 conn(0x7f5b2c0fdf10 msgr2=0x7f5b2c110e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:26:17.153 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:17.152+0000 7f5b32800700 1 -- 172.21.15.117:0/1517716549 shutdown_connections 2024-06-03T03:26:17.153 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:17.152+0000 7f5b32800700 1 -- 172.21.15.117:0/1517716549 wait complete. 2024-06-03T03:26:17.153 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 31 2024-06-03T03:26:18.090 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-06-03T03:26:18.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:26:18 smithi151 ceph-mon[37043]: pgmap v535: 129 pgs: 129 active+clean; 678 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-06-03T03:26:18.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:26:18 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/1517716549' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T03:26:18.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:26:18 smithi117 ceph-mon[31649]: pgmap v535: 129 pgs: 129 active+clean; 678 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-06-03T03:26:18.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:26:18 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/1517716549' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T03:26:18.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:26:18 smithi183 ceph-mon[37326]: pgmap v535: 129 pgs: 129 active+clean; 678 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-06-03T03:26:18.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:26:18 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/1517716549' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T03:26:18.525 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:26:20.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:26:20 smithi151 ceph-mon[37043]: pgmap v536: 129 pgs: 129 active+clean; 678 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-06-03T03:26:20.409 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:26:20 smithi117 ceph-mon[31649]: pgmap v536: 129 pgs: 129 active+clean; 678 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-06-03T03:26:20.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:26:20 smithi183 ceph-mon[37326]: pgmap v536: 129 pgs: 129 active+clean; 678 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-06-03T03:26:20.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.934+0000 7f176dc00700 1 -- 172.21.15.117:0/669124919 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1768074fe0 msgr2=0x7f176810e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:20.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.934+0000 7f176dc00700 1 --2- 172.21.15.117:0/669124919 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1768074fe0 0x7f176810e950 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f17540098f0 tx=0x7f1754009c00 comp rx=0 tx=0).stop 2024-06-03T03:26:20.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.935+0000 7f176dc00700 1 -- 172.21.15.117:0/669124919 shutdown_connections 2024-06-03T03:26:20.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.935+0000 7f176dc00700 1 --2- 172.21.15.117:0/669124919 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f176810ee90 0x7f1768111280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:20.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.935+0000 7f176dc00700 1 --2- 172.21.15.117:0/669124919 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f17680746c0 0x7f1768074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:20.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.935+0000 7f176dc00700 1 --2- 172.21.15.117:0/669124919 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1768074fe0 0x7f176810e950 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:20.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.935+0000 7f176dc00700 1 -- 172.21.15.117:0/669124919 >> 172.21.15.117:0/669124919 conn(0x7f1768079640 msgr2=0x7f1768079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:26:20.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.935+0000 7f176dc00700 1 -- 172.21.15.117:0/669124919 shutdown_connections 2024-06-03T03:26:20.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.935+0000 7f176dc00700 1 -- 172.21.15.117:0/669124919 wait complete. 2024-06-03T03:26:20.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.937+0000 7f176dc00700 1 Processor -- start 2024-06-03T03:26:20.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.938+0000 7f176dc00700 1 -- start start 2024-06-03T03:26:20.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.938+0000 7f176dc00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f17680746c0 0x7f176819afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:20.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.938+0000 7f176dc00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1768074fe0 0x7f176819b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:20.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.938+0000 7f176dc00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f176810ee90 0x7f176819f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:20.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.938+0000 7f176dc00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1768114a70 con 0x7f176810ee90 2024-06-03T03:26:20.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.938+0000 7f176dc00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f17681148f0 con 0x7f17680746c0 2024-06-03T03:26:20.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.938+0000 7f176dc00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1768114bf0 con 0x7f1768074fe0 2024-06-03T03:26:20.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.939+0000 7f1766a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f17680746c0 0x7f176819afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:20.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.939+0000 7f1766a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f17680746c0 0x7f176819afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:54410/0 (socket says 172.21.15.117:54410) 2024-06-03T03:26:20.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.939+0000 7f1766a00700 1 -- 172.21.15.117:0/3121726733 learned_addr learned my addr 172.21.15.117:0/3121726733 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:26:20.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.939+0000 7f1766000700 1 --2- 172.21.15.117:0/3121726733 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1768074fe0 0x7f176819b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:20.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.939+0000 7f1767400700 1 --2- 172.21.15.117:0/3121726733 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f176810ee90 0x7f176819f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:20.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.939+0000 7f1766a00700 1 -- 172.21.15.117:0/3121726733 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1768074fe0 msgr2=0x7f176819b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:20.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.939+0000 7f1766a00700 1 --2- 172.21.15.117:0/3121726733 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1768074fe0 0x7f176819b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:20.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.939+0000 7f1766a00700 1 -- 172.21.15.117:0/3121726733 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f176810ee90 msgr2=0x7f176819f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:20.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.939+0000 7f1766a00700 1 --2- 172.21.15.117:0/3121726733 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f176810ee90 0x7f176819f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:20.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.939+0000 7f1766a00700 1 -- 172.21.15.117:0/3121726733 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f176819ff80 con 0x7f17680746c0 2024-06-03T03:26:20.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.939+0000 7f1767400700 1 --2- 172.21.15.117:0/3121726733 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f176810ee90 0x7f176819f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T03:26:20.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.939+0000 7f1766000700 1 --2- 172.21.15.117:0/3121726733 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1768074fe0 0x7f176819b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T03:26:20.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.940+0000 7f1766a00700 1 --2- 172.21.15.117:0/3121726733 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f17680746c0 0x7f176819afc0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f175800e9a0 tx=0x7f175800ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:26:20.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.940+0000 7f175f400700 1 -- 172.21.15.117:0/3121726733 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f175800cbd0 con 0x7f17680746c0 2024-06-03T03:26:20.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.940+0000 7f176dc00700 1 -- 172.21.15.117:0/3121726733 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17681a0270 con 0x7f17680746c0 2024-06-03T03:26:20.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.940+0000 7f175f400700 1 -- 172.21.15.117:0/3121726733 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f175800cd70 con 0x7f17680746c0 2024-06-03T03:26:20.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.940+0000 7f176dc00700 1 -- 172.21.15.117:0/3121726733 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17681a7af0 con 0x7f17680746c0 2024-06-03T03:26:20.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.941+0000 7f175f400700 1 -- 172.21.15.117:0/3121726733 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17580187c0 con 0x7f17680746c0 2024-06-03T03:26:20.947 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.946+0000 7f175f400700 1 -- 172.21.15.117:0/3121726733 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f1758010430 con 0x7f17680746c0 2024-06-03T03:26:20.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.948+0000 7f177860cb80 1 Processor -- start 2024-06-03T03:26:20.949 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.948+0000 7f177860cb80 1 -- start start 2024-06-03T03:26:20.950 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.948+0000 7f175f400700 1 --2- 172.21.15.117:0/3121726733 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f174c0a2d30 0x7f174c0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:20.950 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.949+0000 7f177860cb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55777e865e40 0x55777e8e9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:20.950 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.949+0000 7f175f400700 1 -- 172.21.15.117:0/3121726733 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f17580cf900 con 0x7f17680746c0 2024-06-03T03:26:20.951 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.949+0000 7f177860cb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55777e8e95b0 0x55777e8e9a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:20.951 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.949+0000 7f177860cb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55777e8fcb10 0x55777e8fef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:20.951 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.949+0000 7f1767400700 1 --2- 172.21.15.117:0/3121726733 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f174c0a2d30 0x7f174c0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:20.952 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.949+0000 7f177860cb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55777e7f8a10 con 0x55777e865e40 2024-06-03T03:26:20.952 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.949+0000 7f1766a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55777e865e40 0x55777e8e9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:20.952 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.950+0000 7f1767400700 1 --2- 172.21.15.117:0/3121726733 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f174c0a2d30 0x7f174c0a51f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1760009fd0 tx=0x7f176000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:26:20.952 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.950+0000 7f1767400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55777e8fcb10 0x55777e8fef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:20.952 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.950+0000 7f1766a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55777e865e40 0x55777e8e9070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39014/0 (socket says 172.21.15.117:39014) 2024-06-03T03:26:20.952 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.950+0000 7f1766a00700 1 -- 172.21.15.117:0/189453395 learned_addr learned my addr 172.21.15.117:0/189453395 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:26:20.952 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.950+0000 7f1766000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55777e8e95b0 0x55777e8e9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:20.952 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.951+0000 7f177860cb80 1 -- 172.21.15.117:0/189453395 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55777e83bb00 con 0x55777e8fcb10 2024-06-03T03:26:20.952 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.951+0000 7f177860cb80 1 -- 172.21.15.117:0/189453395 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55777e7f68f0 con 0x55777e8e95b0 2024-06-03T03:26:20.952 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.951+0000 7f1767400700 1 -- 172.21.15.117:0/189453395 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55777e8e95b0 msgr2=0x55777e8e9a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:20.952 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.951+0000 7f1767400700 1 --2- 172.21.15.117:0/189453395 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55777e8e95b0 0x55777e8e9a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:20.953 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.951+0000 7f1767400700 1 -- 172.21.15.117:0/189453395 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55777e865e40 msgr2=0x55777e8e9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:20.953 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.951+0000 7f1767400700 1 --2- 172.21.15.117:0/189453395 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55777e865e40 0x55777e8e9070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:20.953 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.951+0000 7f1767400700 1 -- 172.21.15.117:0/189453395 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55777e7e3f90 con 0x55777e8fcb10 2024-06-03T03:26:20.953 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.951+0000 7f1766000700 1 --2- 172.21.15.117:0/189453395 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55777e8e95b0 0x55777e8e9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T03:26:20.953 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.951+0000 7f1766a00700 1 --2- 172.21.15.117:0/189453395 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55777e865e40 0x55777e8e9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:26:20.953 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.952+0000 7f1767400700 1 --2- 172.21.15.117:0/189453395 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55777e8fcb10 0x55777e8fef00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1760009e60 tx=0x7f1760013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54dc448a3ec00d2e server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:26:20.953 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.952+0000 7f1753e00700 1 -- 172.21.15.117:0/189453395 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1760026070 con 0x55777e8fcb10 2024-06-03T03:26:20.953 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.952+0000 7f1753e00700 1 -- 172.21.15.117:0/189453395 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f176001d040 con 0x55777e8fcb10 2024-06-03T03:26:20.954 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.952+0000 7f1753e00700 1 -- 172.21.15.117:0/189453395 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f176001c050 con 0x55777e8fcb10 2024-06-03T03:26:20.954 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.953+0000 7f177860cb80 1 -- 172.21.15.117:0/189453395 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55777e8fcb10 msgr2=0x55777e8fef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:20.954 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.953+0000 7f177860cb80 1 --2- 172.21.15.117:0/189453395 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55777e8fcb10 0x55777e8fef00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1760009e60 tx=0x7f1760013000 comp rx=0 tx=0).stop 2024-06-03T03:26:20.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.954+0000 7f177860cb80 1 -- 172.21.15.117:0/189453395 shutdown_connections 2024-06-03T03:26:20.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.954+0000 7f177860cb80 1 --2- 172.21.15.117:0/189453395 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55777e8fcb10 0x55777e8fef00 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:20.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.954+0000 7f177860cb80 1 --2- 172.21.15.117:0/189453395 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55777e865e40 0x55777e8e9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:20.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.954+0000 7f177860cb80 1 --2- 172.21.15.117:0/189453395 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55777e8e95b0 0x55777e8e9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:20.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.954+0000 7f177860cb80 1 -- 172.21.15.117:0/189453395 >> 172.21.15.117:0/189453395 conn(0x55777e816170 msgr2=0x55777e8e54b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:26:20.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.955+0000 7f177860cb80 1 -- 172.21.15.117:0/189453395 shutdown_connections 2024-06-03T03:26:20.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.955+0000 7f177860cb80 1 -- 172.21.15.117:0/189453395 wait complete. 2024-06-03T03:26:20.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.955+0000 7f177860cb80 1 -- start start 2024-06-03T03:26:20.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.956+0000 7f177860cb80 1 Processor -- start 2024-06-03T03:26:20.961 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.957+0000 7f177860cb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55777e865e40 0x55777e8f6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:20.961 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.957+0000 7f177860cb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55777e8e95b0 0x55777e8f65d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:20.961 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.957+0000 7f177860cb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55777e8fcb10 0x55777e8fbdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:20.961 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.957+0000 7f1766a00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55777e865e40 0x55777e8f6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:20.961 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.957+0000 7f1766a00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55777e865e40 0x55777e8f6090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:35322/0 (socket says 172.21.15.117:35322) 2024-06-03T03:26:20.961 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.957+0000 7f1766a00700 1 -- 172.21.15.117:0/126072879 learned_addr learned my addr 172.21.15.117:0/126072879 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:26:20.961 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.957+0000 7f177860cb80 1 -- 172.21.15.117:0/126072879 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55777e7d7f60 con 0x55777e8e95b0 2024-06-03T03:26:20.961 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.957+0000 7f177860cb80 1 -- 172.21.15.117:0/126072879 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55777e837ac0 con 0x55777e8fcb10 2024-06-03T03:26:20.961 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.957+0000 7f177860cb80 1 -- 172.21.15.117:0/126072879 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55777e8ec630 con 0x55777e865e40 2024-06-03T03:26:20.961 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.958+0000 7f177860cb80 10 client.0 _ll_register_callbacks cb 0x55777e7d5720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:26:20.961 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.958+0000 7f1766000700 1 --2- 172.21.15.117:0/126072879 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55777e8e95b0 0x55777e8f65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:20.961 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.958+0000 7f1766a00700 1 -- 172.21.15.117:0/126072879 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55777e8fcb10 msgr2=0x55777e8fbdc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-03T03:26:20.961 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.958+0000 7f1766a00700 1 --2- 172.21.15.117:0/126072879 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55777e8fcb10 0x55777e8fbdc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:20.962 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.958+0000 7f1766a00700 1 -- 172.21.15.117:0/126072879 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55777e8e95b0 msgr2=0x55777e8f65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:20.962 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.958+0000 7f1766a00700 1 --2- 172.21.15.117:0/126072879 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55777e8e95b0 0x55777e8f65d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:20.962 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.958+0000 7f1766a00700 1 -- 172.21.15.117:0/126072879 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55777e78a8d0 con 0x55777e865e40 2024-06-03T03:26:20.962 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.958+0000 7f1766000700 1 --2- 172.21.15.117:0/126072879 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55777e8e95b0 0x55777e8f65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T03:26:20.962 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.958+0000 7f1766a00700 1 --2- 172.21.15.117:0/126072879 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55777e865e40 0x55777e8f6090 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1758000c00 tx=0x7f175809c440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9579dac6902f35bf server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:26:20.962 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.958+0000 7f1750c00700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-06-03T03:26:20.962 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.958+0000 7f1750c00700 1 -- 172.21.15.117:0/126072879 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17580a4880 con 0x55777e865e40 2024-06-03T03:26:20.962 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.959+0000 7f1750c00700 1 -- 172.21.15.117:0/126072879 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f175809be70 con 0x55777e865e40 2024-06-03T03:26:20.962 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.959+0000 7f177860cb80 1 -- 172.21.15.117:0/126072879 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f176001d040 con 0x55777e865e40 2024-06-03T03:26:20.962 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.959+0000 7f1750c00700 1 -- 172.21.15.117:0/126072879 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f175809c070 con 0x55777e865e40 2024-06-03T03:26:20.964 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.959+0000 7f177860cb80 10 client.25417 fetch_fsmap learned FSMap version 31 2024-06-03T03:26:20.964 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.959+0000 7f177860cb80 1 -- 172.21.15.117:0/126072879 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55777e8f2240 con 0x55777e865e40 2024-06-03T03:26:20.964 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.959+0000 7f1750c00700 1 -- 172.21.15.117:0/126072879 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f176001d040 con 0x55777e865e40 2024-06-03T03:26:20.964 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.959+0000 7f1750c00700 1 -- 172.21.15.117:0/126072879 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f175809aad0 con 0x55777e865e40 2024-06-03T03:26:20.964 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.960+0000 7f177860cb80 10 client.25417 fetch_fsmap finished waiting for FSMap version 31 2024-06-03T03:26:20.964 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.960+0000 7f177860cb80 10 client.25417 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-06-03T03:26:20.964 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.960+0000 7f177860cb80 20 client.25417 populate_metadata read hostname 'smithi117' 2024-06-03T03:26:20.964 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.960+0000 7f177860cb80 1 --2- 172.21.15.117:0/126072879 >> [v2:172.21.15.183:6834/1009004320,v1:172.21.15.183:6835/1009004320] conn(0x55777e914dc0 0x55777e9171b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:20.964 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.960+0000 7f177860cb80 4 client.25417 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T03:26:20.964 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.960+0000 7f177860cb80 1 -- 172.21.15.117:0/126072879 --> [v2:172.21.15.183:6834/1009004320,v1:172.21.15.183:6835/1009004320] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55777e849000 con 0x55777e914dc0 2024-06-03T03:26:20.965 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.960+0000 7f1766000700 1 --2- 172.21.15.117:0/126072879 >> [v2:172.21.15.183:6834/1009004320,v1:172.21.15.183:6835/1009004320] conn(0x55777e914dc0 0x55777e9171b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:20.965 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.961+0000 7f1766000700 1 --2- 172.21.15.117:0/126072879 >> [v2:172.21.15.183:6834/1009004320,v1:172.21.15.183:6835/1009004320] conn(0x55777e914dc0 0x55777e9171b0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=73c83a85edd53bbe server_cookie=ceca0b27aa1fd485 in_seq=0 out_seq=0 2024-06-03T03:26:20.965 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.961+0000 7f1750c00700 10 client.25417 ms_handle_connect on v2:172.21.15.183:6834/1009004320 2024-06-03T03:26:20.965 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.962+0000 7f1750c00700 1 -- 172.21.15.117:0/126072879 <== mds.4 v2:172.21.15.183:6834/1009004320 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55777e849000 con 0x55777e914dc0 2024-06-03T03:26:20.965 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.962+0000 7f1750c00700 10 client.25417 handle_command_reply: tid=0 2024-06-03T03:26:20.965 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.962+0000 7f177860cb80 1 client.25417 shutdown 2024-06-03T03:26:20.965 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.962+0000 7f177860cb80 1 -- 172.21.15.117:0/126072879 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55777e865e40 msgr2=0x55777e8f6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:20.965 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.962+0000 7f177860cb80 1 --2- 172.21.15.117:0/126072879 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55777e865e40 0x55777e8f6090 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1758000c00 tx=0x7f175809c440 comp rx=0 tx=0).stop 2024-06-03T03:26:20.965 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.962+0000 7f177860cb80 1 -- 172.21.15.117:0/126072879 shutdown_connections 2024-06-03T03:26:20.965 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.962+0000 7f177860cb80 1 --2- 172.21.15.117:0/126072879 >> [v2:172.21.15.183:6834/1009004320,v1:172.21.15.183:6835/1009004320] conn(0x55777e914dc0 0x55777e9171b0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:20.965 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.962+0000 7f177860cb80 1 --2- 172.21.15.117:0/126072879 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55777e8fcb10 0x55777e8fbdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:20.965 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.962+0000 7f1750c00700 0 client.25417 ms_handle_reset on v2:172.21.15.183:6834/1009004320 2024-06-03T03:26:20.965 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.962+0000 7f177860cb80 1 --2- 172.21.15.117:0/126072879 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55777e865e40 0x55777e8f6090 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:20.966 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.962+0000 7f177860cb80 1 --2- 172.21.15.117:0/126072879 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55777e8e95b0 0x55777e8f65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:20.966 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.962+0000 7f177860cb80 1 -- 172.21.15.117:0/126072879 >> 172.21.15.117:0/126072879 conn(0x55777e816170 msgr2=0x55777e8e4740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:26:20.966 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.963+0000 7f177860cb80 1 -- 172.21.15.117:0/126072879 shutdown_connections 2024-06-03T03:26:20.966 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.963+0000 7f177860cb80 1 -- 172.21.15.117:0/126072879 wait complete. 2024-06-03T03:26:20.966 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.964+0000 7f177860cb80 20 client.25417 trim_cache size 0 max 16384 2024-06-03T03:26:20.982 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.981+0000 7f177860cb80 1 Processor -- start 2024-06-03T03:26:20.982 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.981+0000 7f177860cb80 1 -- start start 2024-06-03T03:26:20.983 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.982+0000 7f177860cb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55777e8e95b0 0x55777e8fbd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:20.983 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.982+0000 7f177860cb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55777e8fcb10 0x55777e8fdf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:20.983 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.982+0000 7f177860cb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55777e914dc0 0x55777e91ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:20.983 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.982+0000 7f177860cb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f17681ac240 con 0x55777e914dc0 2024-06-03T03:26:20.983 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.982+0000 7f177860cb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f17681add80 con 0x55777e8e95b0 2024-06-03T03:26:20.983 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.982+0000 7f177860cb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f17681ac910 con 0x55777e8fcb10 2024-06-03T03:26:20.983 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.982+0000 7f1766a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55777e8e95b0 0x55777e8fbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:20.983 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.982+0000 7f1766a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55777e8e95b0 0x55777e8fbd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:54438/0 (socket says 172.21.15.117:54438) 2024-06-03T03:26:20.983 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.982+0000 7f1766a00700 1 -- 172.21.15.117:0/1671549590 learned_addr learned my addr 172.21.15.117:0/1671549590 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:26:20.983 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.982+0000 7f1766000700 1 --2- 172.21.15.117:0/1671549590 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55777e8fcb10 0x55777e8fdf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:20.983 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.982+0000 7f1767400700 1 --2- 172.21.15.117:0/1671549590 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55777e914dc0 0x55777e91ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:20.984 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.983+0000 7f1766a00700 1 -- 172.21.15.117:0/1671549590 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55777e8fcb10 msgr2=0x55777e8fdf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:20.984 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.983+0000 7f1766a00700 1 --2- 172.21.15.117:0/1671549590 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55777e8fcb10 0x55777e8fdf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:20.984 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.983+0000 7f1766a00700 1 -- 172.21.15.117:0/1671549590 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55777e914dc0 msgr2=0x55777e91ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:20.984 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.983+0000 7f1766a00700 1 --2- 172.21.15.117:0/1671549590 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55777e914dc0 0x55777e91ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:20.984 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.983+0000 7f1766a00700 1 -- 172.21.15.117:0/1671549590 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55777e8eabd0 con 0x55777e8e95b0 2024-06-03T03:26:20.984 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.983+0000 7f1766a00700 1 --2- 172.21.15.117:0/1671549590 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55777e8e95b0 0x55777e8fbd30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f17580985e0 tx=0x7f175801f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4f99adaea674912 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:26:20.984 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.983+0000 7f1753400700 1 -- 172.21.15.117:0/1671549590 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f175809c070 con 0x55777e8e95b0 2024-06-03T03:26:20.985 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.983+0000 7f1753400700 1 -- 172.21.15.117:0/1671549590 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f175809be70 con 0x55777e8e95b0 2024-06-03T03:26:20.985 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.983+0000 7f1753400700 1 -- 172.21.15.117:0/1671549590 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17580a4650 con 0x55777e8e95b0 2024-06-03T03:26:20.985 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.984+0000 7f177860cb80 1 -- 172.21.15.117:0/1671549590 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55777e8e95b0 msgr2=0x55777e8fbd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:20.985 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.984+0000 7f177860cb80 1 --2- 172.21.15.117:0/1671549590 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55777e8e95b0 0x55777e8fbd30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f17580985e0 tx=0x7f175801f000 comp rx=0 tx=0).stop 2024-06-03T03:26:20.985 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.984+0000 7f177860cb80 1 -- 172.21.15.117:0/1671549590 shutdown_connections 2024-06-03T03:26:20.985 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.984+0000 7f177860cb80 1 --2- 172.21.15.117:0/1671549590 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55777e914dc0 0x55777e91ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:20.986 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.984+0000 7f177860cb80 1 --2- 172.21.15.117:0/1671549590 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55777e8e95b0 0x55777e8fbd30 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:20.986 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.984+0000 7f177860cb80 1 --2- 172.21.15.117:0/1671549590 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55777e8fcb10 0x55777e8fdf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:20.986 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.984+0000 7f177860cb80 1 -- 172.21.15.117:0/1671549590 >> 172.21.15.117:0/1671549590 conn(0x55777e816170 msgr2=0x55777e8efcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:26:20.986 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.984+0000 7f177860cb80 1 -- 172.21.15.117:0/1671549590 shutdown_connections 2024-06-03T03:26:20.986 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.985+0000 7f177860cb80 1 -- 172.21.15.117:0/1671549590 wait complete. 2024-06-03T03:26:20.986 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.985+0000 7f177860cb80 1 -- start start 2024-06-03T03:26:20.986 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.985+0000 7f177860cb80 1 Processor -- start 2024-06-03T03:26:20.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.986+0000 7f177860cb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55777e8e95b0 0x55777e928990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:20.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.986+0000 7f177860cb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55777e8fcb10 0x55777e928ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:20.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.986+0000 7f177860cb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55777e914dc0 0x55777e8ee6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:20.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.986+0000 7f177860cb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f17681ad6b0 con 0x55777e8e95b0 2024-06-03T03:26:20.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.986+0000 7f177860cb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f17681acfe0 con 0x55777e914dc0 2024-06-03T03:26:20.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.986+0000 7f177860cb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f17681aa210 con 0x55777e8fcb10 2024-06-03T03:26:20.988 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.986+0000 7f177860cb80 10 client.0 _ll_register_callbacks cb 0x55777e7d5720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:26:20.988 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.986+0000 7f1766a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55777e8e95b0 0x55777e928990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:20.988 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.986+0000 7f1766a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55777e8e95b0 0x55777e928990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39054/0 (socket says 172.21.15.117:39054) 2024-06-03T03:26:20.988 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.986+0000 7f1766a00700 1 -- 172.21.15.117:0/2015308565 learned_addr learned my addr 172.21.15.117:0/2015308565 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:26:20.988 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.987+0000 7f1766000700 1 --2- 172.21.15.117:0/2015308565 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55777e8fcb10 0x55777e928ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:20.988 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.987+0000 7f1767400700 1 --2- 172.21.15.117:0/2015308565 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55777e914dc0 0x55777e8ee6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:20.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.987+0000 7f1766a00700 1 -- 172.21.15.117:0/2015308565 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55777e8fcb10 msgr2=0x55777e928ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:20.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.987+0000 7f1766a00700 1 --2- 172.21.15.117:0/2015308565 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55777e8fcb10 0x55777e928ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:20.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.987+0000 7f1766a00700 1 -- 172.21.15.117:0/2015308565 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55777e914dc0 msgr2=0x55777e8ee6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:20.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.987+0000 7f1766a00700 1 --2- 172.21.15.117:0/2015308565 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55777e914dc0 0x55777e8ee6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:20.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.987+0000 7f1766a00700 1 -- 172.21.15.117:0/2015308565 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55777e8eee40 con 0x55777e8e95b0 2024-06-03T03:26:20.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.987+0000 7f1766a00700 1 --2- 172.21.15.117:0/2015308565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55777e8e95b0 0x55777e928990 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f17580a1330 tx=0x7f175809ba70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c38e662a6574e29 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:26:20.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.988+0000 7f1750c00700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-06-03T03:26:20.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.988+0000 7f1750c00700 1 -- 172.21.15.117:0/2015308565 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1758080060 con 0x55777e8e95b0 2024-06-03T03:26:20.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.988+0000 7f1750c00700 1 -- 172.21.15.117:0/2015308565 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f175809b060 con 0x55777e8e95b0 2024-06-03T03:26:20.990 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.988+0000 7f177860cb80 1 -- 172.21.15.117:0/2015308565 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f175809be70 con 0x55777e8e95b0 2024-06-03T03:26:20.990 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.988+0000 7f1750c00700 1 -- 172.21.15.117:0/2015308565 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f175809b260 con 0x55777e8e95b0 2024-06-03T03:26:20.990 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.988+0000 7f177860cb80 10 client.15285 fetch_fsmap learned FSMap version 31 2024-06-03T03:26:20.990 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.988+0000 7f177860cb80 1 -- 172.21.15.117:0/2015308565 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55777e8eee40 con 0x55777e8e95b0 2024-06-03T03:26:20.991 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.989+0000 7f1750c00700 1 -- 172.21.15.117:0/2015308565 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f175809be70 con 0x55777e8e95b0 2024-06-03T03:26:20.991 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.989+0000 7f1750c00700 1 -- 172.21.15.117:0/2015308565 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f175807aa80 con 0x55777e8e95b0 2024-06-03T03:26:20.991 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.990+0000 7f177860cb80 10 client.15285 fetch_fsmap finished waiting for FSMap version 31 2024-06-03T03:26:20.991 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.990+0000 7f177860cb80 10 client.15285 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-06-03T03:26:20.991 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.990+0000 7f177860cb80 20 client.15285 populate_metadata read hostname 'smithi117' 2024-06-03T03:26:20.991 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.990+0000 7f177860cb80 1 --2- 172.21.15.117:0/2015308565 >> [v2:172.21.15.183:6834/1009004320,v1:172.21.15.183:6835/1009004320] conn(0x55777e97f8d0 0x55777e981cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:20.991 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.990+0000 7f177860cb80 4 client.15285 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T03:26:20.991 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.990+0000 7f177860cb80 1 -- 172.21.15.117:0/2015308565 --> [v2:172.21.15.183:6834/1009004320,v1:172.21.15.183:6835/1009004320] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55777e843580 con 0x55777e97f8d0 2024-06-03T03:26:20.991 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.990+0000 7f1766000700 1 --2- 172.21.15.117:0/2015308565 >> [v2:172.21.15.183:6834/1009004320,v1:172.21.15.183:6835/1009004320] conn(0x55777e97f8d0 0x55777e981cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:20.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.991+0000 7f1766000700 1 --2- 172.21.15.117:0/2015308565 >> [v2:172.21.15.183:6834/1009004320,v1:172.21.15.183:6835/1009004320] conn(0x55777e97f8d0 0x55777e981cc0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c95e61668122fa2c server_cookie=d1d5232eb8a05eaa in_seq=0 out_seq=0 2024-06-03T03:26:20.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.991+0000 7f1750c00700 10 client.15285 ms_handle_connect on v2:172.21.15.183:6834/1009004320 2024-06-03T03:26:20.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.991+0000 7f1750c00700 1 -- 172.21.15.117:0/2015308565 <== mds.4 v2:172.21.15.183:6834/1009004320 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55777e843580 con 0x55777e97f8d0 2024-06-03T03:26:20.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.991+0000 7f1750c00700 10 client.15285 handle_command_reply: tid=0 2024-06-03T03:26:20.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.991+0000 7f177860cb80 1 client.15285 shutdown 2024-06-03T03:26:20.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.992+0000 7f177860cb80 1 -- 172.21.15.117:0/2015308565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55777e8e95b0 msgr2=0x55777e928990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:20.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.992+0000 7f177860cb80 1 --2- 172.21.15.117:0/2015308565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55777e8e95b0 0x55777e928990 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f17580a1330 tx=0x7f175809ba70 comp rx=0 tx=0).stop 2024-06-03T03:26:20.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.992+0000 7f177860cb80 1 -- 172.21.15.117:0/2015308565 shutdown_connections 2024-06-03T03:26:20.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.992+0000 7f177860cb80 1 --2- 172.21.15.117:0/2015308565 >> [v2:172.21.15.183:6834/1009004320,v1:172.21.15.183:6835/1009004320] conn(0x55777e97f8d0 0x55777e981cc0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:20.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.992+0000 7f177860cb80 1 --2- 172.21.15.117:0/2015308565 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55777e914dc0 0x55777e8ee6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:20.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.992+0000 7f177860cb80 1 --2- 172.21.15.117:0/2015308565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55777e8e95b0 0x55777e928990 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:20.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.992+0000 7f177860cb80 1 --2- 172.21.15.117:0/2015308565 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55777e8fcb10 0x55777e928ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:20.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.992+0000 7f177860cb80 1 -- 172.21.15.117:0/2015308565 >> 172.21.15.117:0/2015308565 conn(0x55777e816170 msgr2=0x55777e8e3eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:26:20.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.992+0000 7f1750c00700 0 client.15285 ms_handle_reset on v2:172.21.15.183:6834/1009004320 2024-06-03T03:26:20.994 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.993+0000 7f177860cb80 1 -- 172.21.15.117:0/2015308565 shutdown_connections 2024-06-03T03:26:20.994 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.993+0000 7f177860cb80 1 -- 172.21.15.117:0/2015308565 wait complete. 2024-06-03T03:26:20.994 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.993+0000 7f177860cb80 20 client.15285 trim_cache size 0 max 16384 2024-06-03T03:26:20.994 INFO:teuthology.orchestra.run.smithi117.stdout:[] 2024-06-03T03:26:20.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.994+0000 7f176dc00700 1 -- 172.21.15.117:0/3121726733 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f174c0a2d30 msgr2=0x7f174c0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:20.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.994+0000 7f176dc00700 1 --2- 172.21.15.117:0/3121726733 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f174c0a2d30 0x7f174c0a51f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1760009fd0 tx=0x7f176000b040 comp rx=0 tx=0).stop 2024-06-03T03:26:20.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.994+0000 7f176dc00700 1 -- 172.21.15.117:0/3121726733 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f17680746c0 msgr2=0x7f176819afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:20.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.994+0000 7f176dc00700 1 --2- 172.21.15.117:0/3121726733 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f17680746c0 0x7f176819afc0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f175800e9a0 tx=0x7f175800ed60 comp rx=0 tx=0).stop 2024-06-03T03:26:20.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.994+0000 7f176dc00700 1 -- 172.21.15.117:0/3121726733 shutdown_connections 2024-06-03T03:26:20.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.994+0000 7f176dc00700 1 --2- 172.21.15.117:0/3121726733 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f174c0a2d30 0x7f174c0a51f0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:20.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.994+0000 7f176dc00700 1 --2- 172.21.15.117:0/3121726733 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f176810ee90 0x7f176819f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:20.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.994+0000 7f176dc00700 1 --2- 172.21.15.117:0/3121726733 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f17680746c0 0x7f176819afc0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:20.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.994+0000 7f176dc00700 1 --2- 172.21.15.117:0/3121726733 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1768074fe0 0x7f176819b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:20.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.994+0000 7f176dc00700 1 -- 172.21.15.117:0/3121726733 >> 172.21.15.117:0/3121726733 conn(0x7f1768079640 msgr2=0x7f17680ff740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:26:20.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.995+0000 7f176dc00700 1 -- 172.21.15.117:0/3121726733 shutdown_connections 2024-06-03T03:26:20.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:20.995+0000 7f176dc00700 1 -- 172.21.15.117:0/3121726733 wait complete. 2024-06-03T03:26:21.979 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.978+0000 7f0cdb400640 1 -- 172.21.15.117:0/3646511135 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cd4104e20 msgr2=0x7f0cd41052a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:21.979 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.978+0000 7f0cdb400640 1 --2- 172.21.15.117:0/3646511135 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cd4104e20 0x7f0cd41052a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0cc0009900 tx=0x7f0cc002a410 comp rx=0 tx=0).stop 2024-06-03T03:26:21.979 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.978+0000 7f0cdb400640 1 -- 172.21.15.117:0/3646511135 shutdown_connections 2024-06-03T03:26:21.979 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.978+0000 7f0cdb400640 1 --2- 172.21.15.117:0/3646511135 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0cd41057e0 0x7f0cd410c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:21.979 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.978+0000 7f0cdb400640 1 --2- 172.21.15.117:0/3646511135 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cd4104e20 0x7f0cd41052a0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:21.979 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.978+0000 7f0cdb400640 1 --2- 172.21.15.117:0/3646511135 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0cd4103c30 0x7f0cd4104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:21.979 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.978+0000 7f0cdb400640 1 -- 172.21.15.117:0/3646511135 >> 172.21.15.117:0/3646511135 conn(0x7f0cd40ff1c0 msgr2=0x7f0cd41015e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:26:21.979 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.978+0000 7f0cdb400640 1 -- 172.21.15.117:0/3646511135 shutdown_connections 2024-06-03T03:26:21.980 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.979+0000 7f0cdb400640 1 -- 172.21.15.117:0/3646511135 wait complete. 2024-06-03T03:26:21.980 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.980+0000 7f0cdb400640 1 Processor -- start 2024-06-03T03:26:21.980 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.980+0000 7f0cdb400640 1 -- start start 2024-06-03T03:26:21.981 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.980+0000 7f0cdb400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0cd4103c30 0x7f0cd4199ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:21.981 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.980+0000 7f0cdb400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0cd4104e20 0x7f0cd419a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:21.981 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.980+0000 7f0cdb400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cd41057e0 0x7f0cd41a14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:21.981 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.980+0000 7f0cdb400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd410f930 con 0x7f0cd41057e0 2024-06-03T03:26:21.981 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.980+0000 7f0cdb400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd410f7b0 con 0x7f0cd4103c30 2024-06-03T03:26:21.981 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.980+0000 7f0cdb400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd410fab0 con 0x7f0cd4104e20 2024-06-03T03:26:21.982 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.981+0000 7f0cd9400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cd41057e0 0x7f0cd41a14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:21.982 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.981+0000 7f0cd8a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0cd4103c30 0x7f0cd4199ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:21.982 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.981+0000 7f0cd8a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0cd4103c30 0x7f0cd4199ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:54468/0 (socket says 172.21.15.117:54468) 2024-06-03T03:26:21.982 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.981+0000 7f0cd8a00640 1 -- 172.21.15.117:0/1536248551 learned_addr learned my addr 172.21.15.117:0/1536248551 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:26:21.982 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.981+0000 7f0cd3e00640 1 --2- 172.21.15.117:0/1536248551 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0cd4104e20 0x7f0cd419a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:21.982 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.981+0000 7f0cd8a00640 1 -- 172.21.15.117:0/1536248551 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0cd4104e20 msgr2=0x7f0cd419a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:21.983 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.981+0000 7f0cd8a00640 1 --2- 172.21.15.117:0/1536248551 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0cd4104e20 0x7f0cd419a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:21.983 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.981+0000 7f0cd8a00640 1 -- 172.21.15.117:0/1536248551 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cd41057e0 msgr2=0x7f0cd41a14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:21.983 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.981+0000 7f0cd8a00640 1 --2- 172.21.15.117:0/1536248551 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cd41057e0 0x7f0cd41a14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:21.983 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.981+0000 7f0cd8a00640 1 -- 172.21.15.117:0/1536248551 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cd41a1b10 con 0x7f0cd4103c30 2024-06-03T03:26:21.983 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.983+0000 7f0cd8a00640 1 --2- 172.21.15.117:0/1536248551 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0cd4103c30 0x7f0cd4199ef0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0cc800d950 tx=0x7f0cc800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:26:21.984 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.983+0000 7f0cd1600640 1 -- 172.21.15.117:0/1536248551 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc8017020 con 0x7f0cd4103c30 2024-06-03T03:26:21.984 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.983+0000 7f0cd1600640 1 -- 172.21.15.117:0/1536248551 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cc8009c30 con 0x7f0cd4103c30 2024-06-03T03:26:21.984 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.983+0000 7f0cd1600640 1 -- 172.21.15.117:0/1536248551 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc800f740 con 0x7f0cd4103c30 2024-06-03T03:26:21.985 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.985+0000 7f0cdb400640 1 -- 172.21.15.117:0/1536248551 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cd41a1e00 con 0x7f0cd4103c30 2024-06-03T03:26:21.985 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.985+0000 7f0cdb400640 1 -- 172.21.15.117:0/1536248551 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cd41a21b0 con 0x7f0cd4103c30 2024-06-03T03:26:21.990 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.989+0000 7f0cde163740 1 Processor -- start 2024-06-03T03:26:21.990 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.989+0000 7f0cde163740 1 -- start start 2024-06-03T03:26:21.990 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.990+0000 7f0cd1600640 1 -- 172.21.15.117:0/1536248551 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f0cc801a020 con 0x7f0cd4103c30 2024-06-03T03:26:21.991 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.990+0000 7f0cde163740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f22c043bd0 0x55f22c043ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:21.991 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.991+0000 7f0cde163740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f22c0445c0 0x55f22c0490c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:21.991 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.991+0000 7f0cde163740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f22c049730 0x55f22c04bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:21.992 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.991+0000 7f0cde163740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f22bfa7420 con 0x55f22c043bd0 2024-06-03T03:26:21.992 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.991+0000 7f0cde163740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f22bf4ec50 con 0x55f22c0445c0 2024-06-03T03:26:21.992 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.991+0000 7f0cd8a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f22c0445c0 0x55f22c0490c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:21.992 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.991+0000 7f0cde163740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f22bfa33e0 con 0x55f22c049730 2024-06-03T03:26:21.992 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.991+0000 7f0cd8a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f22c0445c0 0x55f22c0490c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:54484/0 (socket says 172.21.15.117:54484) 2024-06-03T03:26:21.992 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.991+0000 7f0cd8a00640 1 -- 172.21.15.117:0/3900473096 learned_addr learned my addr 172.21.15.117:0/3900473096 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:26:21.993 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-06-03T03:26:21.995 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.991+0000 7f0cd1600640 1 --2- 172.21.15.117:0/1536248551 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f0cbc0a2d50 0x7f0cbc0a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:21.995 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.991+0000 7f0cd9400640 1 --2- 172.21.15.117:0/3900473096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f22c043bd0 0x55f22c043ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:21.995 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.992+0000 7f0cd3e00640 1 --2- 172.21.15.117:0/3900473096 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f22c049730 0x55f22c04bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:21.995 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.992+0000 7f0cd8a00640 1 -- 172.21.15.117:0/3900473096 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f22c049730 msgr2=0x55f22c04bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:21.995 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.992+0000 7f0cd8a00640 1 --2- 172.21.15.117:0/3900473096 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f22c049730 0x55f22c04bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:21.995 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.992+0000 7f0cd8a00640 1 -- 172.21.15.117:0/3900473096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f22c043bd0 msgr2=0x55f22c043ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:21.995 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.992+0000 7f0cd8a00640 1 --2- 172.21.15.117:0/3900473096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f22c043bd0 0x55f22c043ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:21.996 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.992+0000 7f0cd8a00640 1 -- 172.21.15.117:0/3900473096 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f22bcbb210 con 0x55f22c0445c0 2024-06-03T03:26:21.996 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.992+0000 7f0cd9400640 1 --2- 172.21.15.117:0/3900473096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f22c043bd0 0x55f22c043ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T03:26:21.996 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.992+0000 7f0cd1600640 1 -- 172.21.15.117:0/1536248551 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f0cc80ce670 con 0x7f0cd4103c30 2024-06-03T03:26:21.996 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.992+0000 7f0cd8a00640 1 --2- 172.21.15.117:0/3900473096 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f22c0445c0 0x55f22c0490c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0cc8055d90 tx=0x7f0cc801f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba26efdf3c03813e server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:26:21.996 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.992+0000 7f0cd9400640 1 --2- 172.21.15.117:0/1536248551 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f0cbc0a2d50 0x7f0cbc0a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:21.996 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.992+0000 7f0cd3e00640 1 --2- 172.21.15.117:0/3900473096 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f22c049730 0x55f22c04bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T03:26:21.996 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.992+0000 7f0cc5c00640 1 -- 172.21.15.117:0/3900473096 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc809a570 con 0x55f22c0445c0 2024-06-03T03:26:21.996 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.992+0000 7f0cc5c00640 1 -- 172.21.15.117:0/3900473096 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cc8009de0 con 0x55f22c0445c0 2024-06-03T03:26:21.996 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.992+0000 7f0cc5c00640 1 -- 172.21.15.117:0/3900473096 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc8052ab0 con 0x55f22c0445c0 2024-06-03T03:26:21.996 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.994+0000 7f0cd9400640 1 --2- 172.21.15.117:0/1536248551 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f0cbc0a2d50 0x7f0cbc0a5210 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x55f22c044e20 tx=0x7f0ccc00d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:26:21.996 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.994+0000 7f0cde163740 1 -- 172.21.15.117:0/3900473096 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f22c0445c0 msgr2=0x55f22c0490c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:21.996 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.994+0000 7f0cde163740 1 --2- 172.21.15.117:0/3900473096 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f22c0445c0 0x55f22c0490c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0cc8055d90 tx=0x7f0cc801f000 comp rx=0 tx=0).stop 2024-06-03T03:26:21.996 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.994+0000 7f0cde163740 1 -- 172.21.15.117:0/3900473096 shutdown_connections 2024-06-03T03:26:21.997 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.994+0000 7f0cde163740 1 --2- 172.21.15.117:0/3900473096 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f22c049730 0x55f22c04bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:21.997 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.994+0000 7f0cde163740 1 --2- 172.21.15.117:0/3900473096 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f22c0445c0 0x55f22c0490c0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:21.997 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.994+0000 7f0cde163740 1 --2- 172.21.15.117:0/3900473096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f22c043bd0 0x55f22c043ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:21.997 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.994+0000 7f0cde163740 1 -- 172.21.15.117:0/3900473096 >> 172.21.15.117:0/3900473096 conn(0x55f22c00bf80 msgr2=0x55f22c03f650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:26:21.997 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.994+0000 7f0cde163740 1 -- 172.21.15.117:0/3900473096 shutdown_connections 2024-06-03T03:26:21.997 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.994+0000 7f0cde163740 1 -- 172.21.15.117:0/3900473096 wait complete. 2024-06-03T03:26:21.997 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.995+0000 7f0cde163740 1 -- start start 2024-06-03T03:26:22.000 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.995+0000 7f0cde163740 1 Processor -- start 2024-06-03T03:26:22.000 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.995+0000 7f0cde163740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f22c043bd0 0x55f22c04ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:22.001 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.995+0000 7f0cde163740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f22c0445c0 0x55f22c04e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:22.001 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.995+0000 7f0cde163740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f22c049730 0x55f22c055370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:22.001 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.995+0000 7f0cde163740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f22c04c680 con 0x55f22c043bd0 2024-06-03T03:26:22.001 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.996+0000 7f0cde163740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f22bf620b0 con 0x55f22c0445c0 2024-06-03T03:26:22.001 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.996+0000 7f0cde163740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f22c04c800 con 0x55f22c049730 2024-06-03T03:26:22.001 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.996+0000 7f0cde163740 10 client.0 _ll_register_callbacks cb 0x55f22bf4df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:26:22.001 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.996+0000 7f0cd8a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f22c043bd0 0x55f22c04ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:22.001 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.996+0000 7f0cd3e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f22c0445c0 0x55f22c04e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:22.001 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.996+0000 7f0cd8a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f22c043bd0 0x55f22c04ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39086/0 (socket says 172.21.15.117:39086) 2024-06-03T03:26:22.001 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.996+0000 7f0cd8a00640 1 -- 172.21.15.117:0/320668275 learned_addr learned my addr 172.21.15.117:0/320668275 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:26:22.001 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.996+0000 7f0cd9400640 1 --2- 172.21.15.117:0/320668275 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f22c049730 0x55f22c055370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:22.001 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.996+0000 7f0cd8a00640 1 -- 172.21.15.117:0/320668275 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f22c049730 msgr2=0x55f22c055370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.996+0000 7f0cd8a00640 1 --2- 172.21.15.117:0/320668275 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f22c049730 0x55f22c055370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.996+0000 7f0cd8a00640 1 -- 172.21.15.117:0/320668275 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f22c0445c0 msgr2=0x55f22c04e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.996+0000 7f0cd8a00640 1 --2- 172.21.15.117:0/320668275 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f22c0445c0 0x55f22c04e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.996+0000 7f0cd8a00640 1 -- 172.21.15.117:0/320668275 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f22bd87620 con 0x55f22c043bd0 2024-06-03T03:26:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.996+0000 7f0cd3e00640 1 --2- 172.21.15.117:0/320668275 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f22c0445c0 0x55f22c04e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:26:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.997+0000 7f0cd8a00640 1 --2- 172.21.15.117:0/320668275 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f22c043bd0 0x55f22c04ddf0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0cc80ce9a0 tx=0x7f0cc809aaa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40caa1346b117e16 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:26:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.997+0000 7f0cba000640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-06-03T03:26:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.997+0000 7f0cba000640 1 -- 172.21.15.117:0/320668275 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc80140f0 con 0x55f22c043bd0 2024-06-03T03:26:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.997+0000 7f0cba000640 1 -- 172.21.15.117:0/320668275 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f0cc80cbe80 con 0x55f22c043bd0 2024-06-03T03:26:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.997+0000 7f0cde163740 1 -- 172.21.15.117:0/320668275 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cc8009de0 con 0x55f22c043bd0 2024-06-03T03:26:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.997+0000 7f0cba000640 1 -- 172.21.15.117:0/320668275 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f0cc809a390 con 0x55f22c043bd0 2024-06-03T03:26:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.998+0000 7f0cde163740 10 client.15294 fetch_fsmap learned FSMap version 31 2024-06-03T03:26:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.998+0000 7f0cde163740 1 -- 172.21.15.117:0/320668275 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55f22c058650 con 0x55f22c043bd0 2024-06-03T03:26:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.998+0000 7f0cba000640 1 -- 172.21.15.117:0/320668275 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cc8009de0 con 0x55f22c043bd0 2024-06-03T03:26:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.998+0000 7f0cba000640 1 -- 172.21.15.117:0/320668275 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f0cc80970a0 con 0x55f22c043bd0 2024-06-03T03:26:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.998+0000 7f0cde163740 10 client.15294 fetch_fsmap finished waiting for FSMap version 31 2024-06-03T03:26:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.998+0000 7f0cde163740 10 client.15294 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-03T03:26:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.998+0000 7f0cde163740 20 client.15294 populate_metadata read hostname 'smithi117' 2024-06-03T03:26:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.998+0000 7f0cde163740 1 --2- 172.21.15.117:0/320668275 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55f22c0599a0 0x55f22c05bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.998+0000 7f0cde163740 4 client.15294 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T03:26:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.998+0000 7f0cde163740 1 -- 172.21.15.117:0/320668275 --> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f22bfb4b50 con 0x55f22c0599a0 2024-06-03T03:26:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.999+0000 7f0cd3e00640 1 --2- 172.21.15.117:0/320668275 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55f22c0599a0 0x55f22c05bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.999+0000 7f0cd3e00640 1 --2- 172.21.15.117:0/320668275 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55f22c0599a0 0x55f22c05bd90 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f819c80ce8f9f09 server_cookie=a87ad7470fae4988 in_seq=0 out_seq=0 2024-06-03T03:26:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:21.999+0000 7f0cba000640 10 client.15294 ms_handle_connect on v2:172.21.15.183:6838/1439615083 2024-06-03T03:26:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.000+0000 7f0cba000640 1 -- 172.21.15.117:0/320668275 <== mds.0 v2:172.21.15.183:6838/1439615083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f22bfb4b50 con 0x55f22c0599a0 2024-06-03T03:26:22.004 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.000+0000 7f0cba000640 10 client.15294 handle_command_reply: tid=0 2024-06-03T03:26:22.005 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.000+0000 7f0cde163740 1 client.15294 shutdown 2024-06-03T03:26:22.005 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.001+0000 7f0cde163740 1 -- 172.21.15.117:0/320668275 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f22c043bd0 msgr2=0x55f22c04ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:22.005 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.001+0000 7f0cde163740 1 --2- 172.21.15.117:0/320668275 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f22c043bd0 0x55f22c04ddf0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0cc80ce9a0 tx=0x7f0cc809aaa0 comp rx=0 tx=0).stop 2024-06-03T03:26:22.005 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.001+0000 7f0cde163740 1 -- 172.21.15.117:0/320668275 shutdown_connections 2024-06-03T03:26:22.005 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.001+0000 7f0cde163740 1 --2- 172.21.15.117:0/320668275 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55f22c0599a0 0x55f22c05bd90 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:22.005 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.001+0000 7f0cde163740 1 --2- 172.21.15.117:0/320668275 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f22c049730 0x55f22c055370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:22.005 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.001+0000 7f0cde163740 1 --2- 172.21.15.117:0/320668275 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f22c0445c0 0x55f22c04e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:22.005 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.001+0000 7f0cde163740 1 --2- 172.21.15.117:0/320668275 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f22c043bd0 0x55f22c04ddf0 secure :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0cc80ce9a0 tx=0x7f0cc809aaa0 comp rx=0 tx=0).stop 2024-06-03T03:26:22.005 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.001+0000 7f0cde163740 1 -- 172.21.15.117:0/320668275 >> 172.21.15.117:0/320668275 conn(0x55f22c00bf80 msgr2=0x55f22c03e810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:26:22.005 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.001+0000 7f0cba000640 0 client.15294 ms_handle_reset on v2:172.21.15.183:6838/1439615083 2024-06-03T03:26:22.005 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.002+0000 7f0cde163740 1 -- 172.21.15.117:0/320668275 shutdown_connections 2024-06-03T03:26:22.005 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.002+0000 7f0cde163740 1 -- 172.21.15.117:0/320668275 wait complete. 2024-06-03T03:26:22.006 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.002+0000 7f0cde163740 20 client.15294 trim_cache size 0 max 16384 2024-06-03T03:26:22.023 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.023+0000 7f0cde163740 1 Processor -- start 2024-06-03T03:26:22.023 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.023+0000 7f0cde163740 1 -- start start 2024-06-03T03:26:22.024 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.023+0000 7f0cde163740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f22c0445c0 0x55f22c03f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:22.024 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.023+0000 7f0cde163740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f22c049730 0x55f22c055330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:22.024 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.023+0000 7f0cde163740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f22c04da00 0x55f22c04de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:22.025 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.023+0000 7f0cde163740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd41ac820 con 0x55f22c04da00 2024-06-03T03:26:22.025 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.023+0000 7f0cde163740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd41ae360 con 0x55f22c049730 2024-06-03T03:26:22.025 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.023+0000 7f0cde163740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd41acef0 con 0x55f22c0445c0 2024-06-03T03:26:22.025 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.024+0000 7f0cd9400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f22c04da00 0x55f22c04de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:22.025 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.024+0000 7f0cd9400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f22c04da00 0x55f22c04de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39092/0 (socket says 172.21.15.117:39092) 2024-06-03T03:26:22.025 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.024+0000 7f0cd9400640 1 -- 172.21.15.117:0/4148812349 learned_addr learned my addr 172.21.15.117:0/4148812349 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:26:22.025 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.024+0000 7f0cd3e00640 1 --2- 172.21.15.117:0/4148812349 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f22c049730 0x55f22c055330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:22.025 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.024+0000 7f0cd8a00640 1 --2- 172.21.15.117:0/4148812349 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f22c0445c0 0x55f22c03f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:22.026 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.024+0000 7f0cd9400640 1 -- 172.21.15.117:0/4148812349 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f22c0445c0 msgr2=0x55f22c03f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:22.026 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.024+0000 7f0cd9400640 1 --2- 172.21.15.117:0/4148812349 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f22c0445c0 0x55f22c03f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:22.026 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.024+0000 7f0cd9400640 1 -- 172.21.15.117:0/4148812349 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f22c049730 msgr2=0x55f22c055330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:22.026 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.024+0000 7f0cd9400640 1 --2- 172.21.15.117:0/4148812349 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f22c049730 0x55f22c055330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:22.026 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.024+0000 7f0cd9400640 1 -- 172.21.15.117:0/4148812349 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f22c040000 con 0x55f22c04da00 2024-06-03T03:26:22.026 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.025+0000 7f0cd8a00640 1 --2- 172.21.15.117:0/4148812349 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f22c0445c0 0x55f22c03f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T03:26:22.027 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.025+0000 7f0cd9400640 1 --2- 172.21.15.117:0/4148812349 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f22c04da00 0x55f22c04de80 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc0157d0 tx=0x7f0ccc012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cff8cade6dbde5a2 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:26:22.027 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.025+0000 7f0cd3e00640 1 --2- 172.21.15.117:0/4148812349 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f22c049730 0x55f22c055330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T03:26:22.027 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.025+0000 7f0cc5200640 1 -- 172.21.15.117:0/4148812349 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ccc014b50 con 0x55f22c04da00 2024-06-03T03:26:22.027 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.025+0000 7f0cc5200640 1 -- 172.21.15.117:0/4148812349 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ccc027070 con 0x55f22c04da00 2024-06-03T03:26:22.027 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.025+0000 7f0cc5200640 1 -- 172.21.15.117:0/4148812349 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ccc013570 con 0x55f22c04da00 2024-06-03T03:26:22.028 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.026+0000 7f0cde163740 1 -- 172.21.15.117:0/4148812349 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f22c04da00 msgr2=0x55f22c04de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:22.028 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.026+0000 7f0cde163740 1 --2- 172.21.15.117:0/4148812349 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f22c04da00 0x55f22c04de80 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc0157d0 tx=0x7f0ccc012000 comp rx=0 tx=0).stop 2024-06-03T03:26:22.028 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.026+0000 7f0cde163740 1 -- 172.21.15.117:0/4148812349 shutdown_connections 2024-06-03T03:26:22.028 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.026+0000 7f0cde163740 1 --2- 172.21.15.117:0/4148812349 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f22c04da00 0x55f22c04de80 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:22.029 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.026+0000 7f0cde163740 1 --2- 172.21.15.117:0/4148812349 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f22c049730 0x55f22c055330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:22.029 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.026+0000 7f0cde163740 1 --2- 172.21.15.117:0/4148812349 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f22c0445c0 0x55f22c03f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:22.029 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.026+0000 7f0cde163740 1 -- 172.21.15.117:0/4148812349 >> 172.21.15.117:0/4148812349 conn(0x55f22c00bf80 msgr2=0x55f22c03f0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:26:22.029 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.027+0000 7f0cde163740 1 -- 172.21.15.117:0/4148812349 shutdown_connections 2024-06-03T03:26:22.029 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.027+0000 7f0cde163740 1 -- 172.21.15.117:0/4148812349 wait complete. 2024-06-03T03:26:22.030 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.027+0000 7f0cde163740 1 -- start start 2024-06-03T03:26:22.030 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.027+0000 7f0cde163740 1 Processor -- start 2024-06-03T03:26:22.030 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.028+0000 7f0cde163740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f22c0445c0 0x55f22c052040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:22.031 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.028+0000 7f0cde163740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f22c049730 0x55f22c052580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:22.032 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.028+0000 7f0cde163740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f22c04da00 0x55f22c0c2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:22.032 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.028+0000 7f0cde163740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd41adc90 con 0x55f22c049730 2024-06-03T03:26:22.032 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.028+0000 7f0cde163740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd41ad5c0 con 0x55f22c04da00 2024-06-03T03:26:22.032 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.028+0000 7f0cde163740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd41aa800 con 0x55f22c0445c0 2024-06-03T03:26:22.032 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.028+0000 7f0cde163740 10 client.0 _ll_register_callbacks cb 0x7f0cc8096b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:26:22.032 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.028+0000 7f0cd9400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f22c04da00 0x55f22c0c2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:22.032 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.028+0000 7f0cd8a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f22c0445c0 0x55f22c052040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:22.032 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.028+0000 7f0cd9400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f22c04da00 0x55f22c0c2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:54520/0 (socket says 172.21.15.117:54520) 2024-06-03T03:26:22.032 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.028+0000 7f0cd9400640 1 -- 172.21.15.117:0/2917996084 learned_addr learned my addr 172.21.15.117:0/2917996084 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:26:22.032 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.028+0000 7f0cd3e00640 1 --2- 172.21.15.117:0/2917996084 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f22c049730 0x55f22c052580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:22.032 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.029+0000 7f0cd9400640 1 -- 172.21.15.117:0/2917996084 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f22c0445c0 msgr2=0x55f22c052040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:22.032 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.029+0000 7f0cd9400640 1 --2- 172.21.15.117:0/2917996084 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f22c0445c0 0x55f22c052040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:22.033 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.029+0000 7f0cd9400640 1 -- 172.21.15.117:0/2917996084 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f22c049730 msgr2=0x55f22c052580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:22.033 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.029+0000 7f0cd9400640 1 --2- 172.21.15.117:0/2917996084 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f22c049730 0x55f22c052580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:22.033 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.029+0000 7f0cd9400640 1 -- 172.21.15.117:0/2917996084 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f22c055650 con 0x55f22c04da00 2024-06-03T03:26:22.033 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.029+0000 7f0cd3e00640 1 --2- 172.21.15.117:0/2917996084 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f22c049730 0x55f22c052580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T03:26:22.033 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.029+0000 7f0cd9400640 1 --2- 172.21.15.117:0/2917996084 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f22c04da00 0x55f22c0c2580 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc0070b0 tx=0x7f0ccc022660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e802cc38ff53f399 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:26:22.033 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.029+0000 7f0cba000640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-06-03T03:26:22.033 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.029+0000 7f0cba000640 1 -- 172.21.15.117:0/2917996084 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ccc047430 con 0x55f22c04da00 2024-06-03T03:26:22.033 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.030+0000 7f0cde163740 1 -- 172.21.15.117:0/2917996084 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ccc027070 con 0x55f22c04da00 2024-06-03T03:26:22.033 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.030+0000 7f0cba000640 1 -- 172.21.15.117:0/2917996084 <== mon.2 v2:172.21.15.151:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f0ccc0475d0 con 0x55f22c04da00 2024-06-03T03:26:22.033 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.030+0000 7f0cba000640 1 -- 172.21.15.117:0/2917996084 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f0ccc050050 con 0x55f22c04da00 2024-06-03T03:26:22.033 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.030+0000 7f0cde163740 10 client.25406 fetch_fsmap learned FSMap version 31 2024-06-03T03:26:22.033 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.030+0000 7f0cde163740 1 -- 172.21.15.117:0/2917996084 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55f22c0c3a80 con 0x55f22c04da00 2024-06-03T03:26:22.033 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.030+0000 7f0cba000640 1 -- 172.21.15.117:0/2917996084 <== mon.2 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ccc027070 con 0x55f22c04da00 2024-06-03T03:26:22.034 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.031+0000 7f0cba000640 1 -- 172.21.15.117:0/2917996084 <== mon.2 v2:172.21.15.151:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f0ccc021070 con 0x55f22c04da00 2024-06-03T03:26:22.034 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.031+0000 7f0cde163740 10 client.25406 fetch_fsmap finished waiting for FSMap version 31 2024-06-03T03:26:22.035 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.031+0000 7f0cde163740 10 client.25406 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-03T03:26:22.035 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.032+0000 7f0cde163740 20 client.25406 populate_metadata read hostname 'smithi117' 2024-06-03T03:26:22.035 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.032+0000 7f0cde163740 1 --2- 172.21.15.117:0/2917996084 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55f22c0c4380 0x55f22c0c67a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:22.035 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.032+0000 7f0cde163740 4 client.25406 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-03T03:26:22.035 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.032+0000 7f0cde163740 1 -- 172.21.15.117:0/2917996084 --> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f22bfaeea0 con 0x55f22c0c4380 2024-06-03T03:26:22.035 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.032+0000 7f0cd8a00640 1 --2- 172.21.15.117:0/2917996084 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55f22c0c4380 0x55f22c0c67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:22.035 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.033+0000 7f0cd8a00640 1 --2- 172.21.15.117:0/2917996084 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55f22c0c4380 0x55f22c0c67a0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6909b9aed6d5abed server_cookie=deb1c727d8507e7b in_seq=0 out_seq=0 2024-06-03T03:26:22.035 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.033+0000 7f0cba000640 10 client.25406 ms_handle_connect on v2:172.21.15.183:6838/1439615083 2024-06-03T03:26:22.037 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.036+0000 7f0cba000640 1 -- 172.21.15.117:0/2917996084 <== mds.0 v2:172.21.15.183:6838/1439615083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f22bfaeea0 con 0x55f22c0c4380 2024-06-03T03:26:22.037 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.036+0000 7f0cba000640 10 client.25406 handle_command_reply: tid=0 2024-06-03T03:26:22.037 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.036+0000 7f0cde163740 1 client.25406 shutdown 2024-06-03T03:26:22.038 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.036+0000 7f0cde163740 1 -- 172.21.15.117:0/2917996084 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f22c04da00 msgr2=0x55f22c0c2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:22.038 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.036+0000 7f0cde163740 1 --2- 172.21.15.117:0/2917996084 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f22c04da00 0x55f22c0c2580 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc0070b0 tx=0x7f0ccc022660 comp rx=0 tx=0).stop 2024-06-03T03:26:22.038 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.037+0000 7f0cde163740 1 -- 172.21.15.117:0/2917996084 shutdown_connections 2024-06-03T03:26:22.038 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.037+0000 7f0cde163740 1 --2- 172.21.15.117:0/2917996084 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55f22c0c4380 0x55f22c0c67a0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:22.038 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.037+0000 7f0cde163740 1 --2- 172.21.15.117:0/2917996084 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f22c04da00 0x55f22c0c2580 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:22.038 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.037+0000 7f0cde163740 1 --2- 172.21.15.117:0/2917996084 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f22c049730 0x55f22c052580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:22.038 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.037+0000 7f0cde163740 1 --2- 172.21.15.117:0/2917996084 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f22c0445c0 0x55f22c052040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:22.038 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.037+0000 7f0cde163740 1 -- 172.21.15.117:0/2917996084 >> 172.21.15.117:0/2917996084 conn(0x55f22c00bf80 msgr2=0x55f22c03f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:26:22.038 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.037+0000 7f0cba000640 0 client.25406 ms_handle_reset on v2:172.21.15.183:6838/1439615083 2024-06-03T03:26:22.038 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.037+0000 7f0cde163740 1 -- 172.21.15.117:0/2917996084 shutdown_connections 2024-06-03T03:26:22.038 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.037+0000 7f0cde163740 1 -- 172.21.15.117:0/2917996084 wait complete. 2024-06-03T03:26:22.038 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.037+0000 7f0cde163740 20 client.25406 trim_cache size 0 max 16384 2024-06-03T03:26:22.039 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:{ 2024-06-03T03:26:22.039 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "return_code": 0 2024-06-03T03:26:22.039 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:} 2024-06-03T03:26:22.040 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.038+0000 7f0cdb400640 1 -- 172.21.15.117:0/1536248551 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f0cbc0a2d50 msgr2=0x7f0cbc0a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:22.040 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.038+0000 7f0cdb400640 1 --2- 172.21.15.117:0/1536248551 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f0cbc0a2d50 0x7f0cbc0a5210 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x55f22c044e20 tx=0x7f0ccc00d040 comp rx=0 tx=0).stop 2024-06-03T03:26:22.040 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.038+0000 7f0cdb400640 1 -- 172.21.15.117:0/1536248551 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0cd4103c30 msgr2=0x7f0cd4199ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:22.040 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.038+0000 7f0cdb400640 1 --2- 172.21.15.117:0/1536248551 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0cd4103c30 0x7f0cd4199ef0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0cc800d950 tx=0x7f0cc800de10 comp rx=0 tx=0).stop 2024-06-03T03:26:22.040 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.038+0000 7f0cdb400640 1 -- 172.21.15.117:0/1536248551 shutdown_connections 2024-06-03T03:26:22.040 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.038+0000 7f0cdb400640 1 --2- 172.21.15.117:0/1536248551 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cd41057e0 0x7f0cd41a14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:22.040 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.038+0000 7f0cdb400640 1 --2- 172.21.15.117:0/1536248551 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0cd4104e20 0x7f0cd419a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:22.040 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.038+0000 7f0cdb400640 1 --2- 172.21.15.117:0/1536248551 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f0cbc0a2d50 0x7f0cbc0a5210 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:22.040 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.038+0000 7f0cdb400640 1 --2- 172.21.15.117:0/1536248551 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0cd4103c30 0x7f0cd4199ef0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:22.040 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.038+0000 7f0cdb400640 1 -- 172.21.15.117:0/1536248551 >> 172.21.15.117:0/1536248551 conn(0x7f0cd40ff1c0 msgr2=0x7f0cd41080c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:26:22.041 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.038+0000 7f0cdb400640 1 -- 172.21.15.117:0/1536248551 shutdown_connections 2024-06-03T03:26:22.041 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:22.038+0000 7f0cdb400640 1 -- 172.21.15.117:0/1536248551 wait complete. 2024-06-03T03:26:22.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:26:22 smithi117 ceph-mon[31649]: pgmap v537: 129 pgs: 129 active+clean; 678 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 852 B/s wr, 11 op/s 2024-06-03T03:26:22.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:26:22 smithi151 ceph-mon[37043]: pgmap v537: 129 pgs: 129 active+clean; 678 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 852 B/s wr, 11 op/s 2024-06-03T03:26:22.450 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:26:22.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:26:22 smithi183 ceph-mon[37326]: pgmap v537: 129 pgs: 129 active+clean; 678 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 852 B/s wr, 11 op/s 2024-06-03T03:26:24.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:26:24 smithi151 ceph-mon[37043]: pgmap v538: 129 pgs: 129 active+clean; 678 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 682 B/s wr, 12 op/s 2024-06-03T03:26:24.432 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:26:24 smithi117 ceph-mon[31649]: pgmap v538: 129 pgs: 129 active+clean; 678 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 682 B/s wr, 12 op/s 2024-06-03T03:26:24.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:26:24 smithi183 ceph-mon[37326]: pgmap v538: 129 pgs: 129 active+clean; 678 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 682 B/s wr, 12 op/s 2024-06-03T03:26:25.043 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.040+0000 7fb441400700 1 -- 172.21.15.117:0/1053670238 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb43c108500 msgr2=0x7fb43c1088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:25.043 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.041+0000 7fb441400700 1 --2- 172.21.15.117:0/1053670238 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb43c108500 0x7fb43c1088e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb42c009990 tx=0x7fb42c009ca0 comp rx=0 tx=0).stop 2024-06-03T03:26:25.043 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.041+0000 7fb441400700 1 -- 172.21.15.117:0/1053670238 shutdown_connections 2024-06-03T03:26:25.043 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.041+0000 7fb441400700 1 --2- 172.21.15.117:0/1053670238 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb43c06ab30 0x7fb43c10f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:25.043 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.041+0000 7fb441400700 1 --2- 172.21.15.117:0/1053670238 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb43c108500 0x7fb43c1088e0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:25.043 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.041+0000 7fb441400700 1 --2- 172.21.15.117:0/1053670238 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb43c06a170 0x7fb43c06a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:25.043 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.041+0000 7fb441400700 1 -- 172.21.15.117:0/1053670238 >> 172.21.15.117:0/1053670238 conn(0x7fb43c078340 msgr2=0x7fb43c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:26:25.043 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.041+0000 7fb441400700 1 -- 172.21.15.117:0/1053670238 shutdown_connections 2024-06-03T03:26:25.044 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.041+0000 7fb441400700 1 -- 172.21.15.117:0/1053670238 wait complete. 2024-06-03T03:26:25.044 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.042+0000 7fb441400700 1 Processor -- start 2024-06-03T03:26:25.044 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.042+0000 7fb441400700 1 -- start start 2024-06-03T03:26:25.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.043+0000 7fb441400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb43c06a170 0x7fb43c19e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:25.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.043+0000 7fb441400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb43c06ab30 0x7fb43c19ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:25.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.043+0000 7fb441400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb43c108500 0x7fb43c198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:25.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.043+0000 7fb441400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb43c1127d0 con 0x7fb43c108500 2024-06-03T03:26:25.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.043+0000 7fb441400700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb43c112650 con 0x7fb43c06a170 2024-06-03T03:26:25.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.043+0000 7fb441400700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb43c112950 con 0x7fb43c06ab30 2024-06-03T03:26:25.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.043+0000 7fb439e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb43c06ab30 0x7fb43c19ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:25.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.043+0000 7fb43b200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb43c108500 0x7fb43c198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:25.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.043+0000 7fb439e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb43c06ab30 0x7fb43c19ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:35436/0 (socket says 172.21.15.117:35436) 2024-06-03T03:26:25.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.043+0000 7fb439e00700 1 -- 172.21.15.117:0/1870156428 learned_addr learned my addr 172.21.15.117:0/1870156428 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:26:25.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.043+0000 7fb43b200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb43c108500 0x7fb43c198af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39124/0 (socket says 172.21.15.117:39124) 2024-06-03T03:26:25.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.044+0000 7fb43a800700 1 --2- 172.21.15.117:0/1870156428 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb43c06a170 0x7fb43c19e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:25.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.044+0000 7fb439e00700 1 -- 172.21.15.117:0/1870156428 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb43c06a170 msgr2=0x7fb43c19e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:25.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.044+0000 7fb439e00700 1 --2- 172.21.15.117:0/1870156428 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb43c06a170 0x7fb43c19e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:25.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.044+0000 7fb439e00700 1 -- 172.21.15.117:0/1870156428 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb43c108500 msgr2=0x7fb43c198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:25.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.044+0000 7fb439e00700 1 --2- 172.21.15.117:0/1870156428 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb43c108500 0x7fb43c198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:25.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.044+0000 7fb439e00700 1 -- 172.21.15.117:0/1870156428 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb43c199250 con 0x7fb43c06ab30 2024-06-03T03:26:25.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.044+0000 7fb439e00700 1 --2- 172.21.15.117:0/1870156428 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb43c06ab30 0x7fb43c19ee70 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb42800e9a0 tx=0x7fb42800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:26:25.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.045+0000 7fb433400700 1 -- 172.21.15.117:0/1870156428 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb42800cbd0 con 0x7fb43c06ab30 2024-06-03T03:26:25.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.045+0000 7fb433400700 1 -- 172.21.15.117:0/1870156428 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb42800cd70 con 0x7fb43c06ab30 2024-06-03T03:26:25.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.045+0000 7fb433400700 1 -- 172.21.15.117:0/1870156428 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb428018750 con 0x7fb43c06ab30 2024-06-03T03:26:25.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.045+0000 7fb441400700 1 -- 172.21.15.117:0/1870156428 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb43c199540 con 0x7fb43c06ab30 2024-06-03T03:26:25.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.045+0000 7fb441400700 1 -- 172.21.15.117:0/1870156428 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb43c1a5800 con 0x7fb43c06ab30 2024-06-03T03:26:25.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.051+0000 7fb433400700 1 -- 172.21.15.117:0/1870156428 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7fb428010430 con 0x7fb43c06ab30 2024-06-03T03:26:25.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.051+0000 7fb44bdf0b80 1 Processor -- start 2024-06-03T03:26:25.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.051+0000 7fb44bdf0b80 1 -- start start 2024-06-03T03:26:25.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.052+0000 7fb44bdf0b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e330e0e40 0x555e33164070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:25.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.052+0000 7fb44bdf0b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e331645b0 0x555e33164a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:25.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.052+0000 7fb44bdf0b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e33173710 0x555e33175b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:25.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.052+0000 7fb44bdf0b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555e33073a10 con 0x555e33173710 2024-06-03T03:26:25.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.052+0000 7fb44bdf0b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x555e330b6b00 con 0x555e331645b0 2024-06-03T03:26:25.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.052+0000 7fb44bdf0b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555e330718f0 con 0x555e330e0e40 2024-06-03T03:26:25.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.052+0000 7fb43a800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e331645b0 0x555e33164a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:25.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.053+0000 7fb43a800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e331645b0 0x555e33164a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:54556/0 (socket says 172.21.15.117:54556) 2024-06-03T03:26:25.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.053+0000 7fb43a800700 1 -- 172.21.15.117:0/892926988 learned_addr learned my addr 172.21.15.117:0/892926988 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:26:25.056 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.053+0000 7fb439e00700 1 --2- 172.21.15.117:0/892926988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e33173710 0x555e33175b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:25.056 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.053+0000 7fb43b200700 1 --2- 172.21.15.117:0/892926988 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e330e0e40 0x555e33164070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:25.056 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.053+0000 7fb43a800700 1 -- 172.21.15.117:0/892926988 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e330e0e40 msgr2=0x555e33164070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:25.056 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.053+0000 7fb43a800700 1 --2- 172.21.15.117:0/892926988 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e330e0e40 0x555e33164070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:25.056 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.053+0000 7fb43a800700 1 -- 172.21.15.117:0/892926988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e33173710 msgr2=0x555e33175b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:25.056 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.053+0000 7fb433400700 1 --2- 172.21.15.117:0/1870156428 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fb4200a2d30 0x7fb4200a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:25.056 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.054+0000 7fb43a800700 1 --2- 172.21.15.117:0/892926988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e33173710 0x555e33175b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:25.056 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.054+0000 7fb43a800700 1 -- 172.21.15.117:0/892926988 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e3305ef90 con 0x555e331645b0 2024-06-03T03:26:25.056 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.054+0000 7fb433400700 1 -- 172.21.15.117:0/1870156428 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7fb4280cf7c0 con 0x7fb43c06ab30 2024-06-03T03:26:25.056 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.054+0000 7fb439e00700 1 --2- 172.21.15.117:0/892926988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e33173710 0x555e33175b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T03:26:25.057 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.054+0000 7fb43a800700 1 --2- 172.21.15.117:0/892926988 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e331645b0 0x555e33164a30 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb42c005830 tx=0x7fb42c005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38af685ea51d85e8 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:26:25.057 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.055+0000 7fb43b200700 1 --2- 172.21.15.117:0/1870156428 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fb4200a2d30 0x7fb4200a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:25.057 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.055+0000 7fb43b200700 1 --2- 172.21.15.117:0/1870156428 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fb4200a2d30 0x7fb4200a51f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x555e331613b0 tx=0x7fb43400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:26:25.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.057+0000 7fb427e00700 1 -- 172.21.15.117:0/892926988 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb42c00bd10 con 0x555e331645b0 2024-06-03T03:26:25.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.057+0000 7fb427e00700 1 -- 172.21.15.117:0/892926988 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb42c003680 con 0x555e331645b0 2024-06-03T03:26:25.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.057+0000 7fb427e00700 1 -- 172.21.15.117:0/892926988 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb42c00f6c0 con 0x555e331645b0 2024-06-03T03:26:25.060 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.058+0000 7fb44bdf0b80 1 -- 172.21.15.117:0/892926988 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e331645b0 msgr2=0x555e33164a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:25.060 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.058+0000 7fb44bdf0b80 1 --2- 172.21.15.117:0/892926988 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e331645b0 0x555e33164a30 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb42c005830 tx=0x7fb42c005c00 comp rx=0 tx=0).stop 2024-06-03T03:26:25.060 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.058+0000 7fb44bdf0b80 1 -- 172.21.15.117:0/892926988 shutdown_connections 2024-06-03T03:26:25.060 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.058+0000 7fb44bdf0b80 1 --2- 172.21.15.117:0/892926988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e33173710 0x555e33175b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:25.060 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.058+0000 7fb44bdf0b80 1 --2- 172.21.15.117:0/892926988 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e330e0e40 0x555e33164070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:25.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.058+0000 7fb44bdf0b80 1 --2- 172.21.15.117:0/892926988 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e331645b0 0x555e33164a30 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:25.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.058+0000 7fb44bdf0b80 1 -- 172.21.15.117:0/892926988 >> 172.21.15.117:0/892926988 conn(0x555e33091170 msgr2=0x555e331604b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:26:25.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.059+0000 7fb44bdf0b80 1 -- 172.21.15.117:0/892926988 shutdown_connections 2024-06-03T03:26:25.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.059+0000 7fb44bdf0b80 1 -- 172.21.15.117:0/892926988 wait complete. 2024-06-03T03:26:25.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.060+0000 7fb44bdf0b80 1 -- start start 2024-06-03T03:26:25.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.060+0000 7fb44bdf0b80 1 Processor -- start 2024-06-03T03:26:25.062 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.060+0000 7fb44bdf0b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e330e0e40 0x555e33173070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:25.062 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.060+0000 7fb44bdf0b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e331645b0 0x555e3316c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:25.062 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.061+0000 7fb44bdf0b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e33173710 0x555e3316c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:25.063 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.061+0000 7fb44bdf0b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555e33052f60 con 0x555e33173710 2024-06-03T03:26:25.063 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.061+0000 7fb44bdf0b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x555e330b2ac0 con 0x555e330e0e40 2024-06-03T03:26:25.063 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.061+0000 7fb44bdf0b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555e33167570 con 0x555e331645b0 2024-06-03T03:26:25.063 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.061+0000 7fb44bdf0b80 10 client.0 _ll_register_callbacks cb 0x555e33050720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:26:25.063 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.061+0000 7fb43a800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e330e0e40 0x555e33173070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:25.063 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.061+0000 7fb439e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e331645b0 0x555e3316c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:25.063 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.061+0000 7fb43a800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e330e0e40 0x555e33173070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:54568/0 (socket says 172.21.15.117:54568) 2024-06-03T03:26:25.063 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.061+0000 7fb43a800700 1 -- 172.21.15.117:0/1998767475 learned_addr learned my addr 172.21.15.117:0/1998767475 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:26:25.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.061+0000 7fb43b200700 1 --2- 172.21.15.117:0/1998767475 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e33173710 0x555e3316c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:25.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.061+0000 7fb43a800700 1 -- 172.21.15.117:0/1998767475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e331645b0 msgr2=0x555e3316c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:25.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.061+0000 7fb43a800700 1 --2- 172.21.15.117:0/1998767475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e331645b0 0x555e3316c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:25.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.061+0000 7fb43a800700 1 -- 172.21.15.117:0/1998767475 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e33173710 msgr2=0x555e3316c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:25.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.061+0000 7fb43a800700 1 --2- 172.21.15.117:0/1998767475 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e33173710 0x555e3316c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:25.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.061+0000 7fb43a800700 1 -- 172.21.15.117:0/1998767475 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e330058d0 con 0x555e330e0e40 2024-06-03T03:26:25.065 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.061+0000 7fb439e00700 1 --2- 172.21.15.117:0/1998767475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e331645b0 0x555e3316c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:26:25.065 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.062+0000 7fb43a800700 1 --2- 172.21.15.117:0/1998767475 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e330e0e40 0x555e33173070 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb42c006010 tx=0x7fb42c00b760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2694b2f03e8d343f server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:26:25.065 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.062+0000 7fb424c00700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-06-03T03:26:25.065 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.062+0000 7fb424c00700 1 -- 172.21.15.117:0/1998767475 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb42c00be50 con 0x555e330e0e40 2024-06-03T03:26:25.065 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.062+0000 7fb44bdf0b80 1 -- 172.21.15.117:0/1998767475 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb42c003680 con 0x555e330e0e40 2024-06-03T03:26:25.065 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.062+0000 7fb424c00700 1 -- 172.21.15.117:0/1998767475 <== mon.2 v2:172.21.15.151:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7fb42c003a40 con 0x555e330e0e40 2024-06-03T03:26:25.065 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.062+0000 7fb424c00700 1 -- 172.21.15.117:0/1998767475 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fb42c038050 con 0x555e330e0e40 2024-06-03T03:26:25.065 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.062+0000 7fb44bdf0b80 10 client.25418 fetch_fsmap learned FSMap version 31 2024-06-03T03:26:25.065 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.062+0000 7fb44bdf0b80 1 -- 172.21.15.117:0/1998767475 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x555e33184dd0 con 0x555e330e0e40 2024-06-03T03:26:25.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.063+0000 7fb424c00700 1 -- 172.21.15.117:0/1998767475 <== mon.2 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb42c003680 con 0x555e330e0e40 2024-06-03T03:26:25.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.063+0000 7fb424c00700 1 -- 172.21.15.117:0/1998767475 <== mon.2 v2:172.21.15.151:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fb42c00fb60 con 0x555e330e0e40 2024-06-03T03:26:25.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.063+0000 7fb44bdf0b80 10 client.25418 fetch_fsmap finished waiting for FSMap version 31 2024-06-03T03:26:25.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.063+0000 7fb44bdf0b80 10 client.25418 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-06-03T03:26:25.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.063+0000 7fb44bdf0b80 20 client.25418 populate_metadata read hostname 'smithi117' 2024-06-03T03:26:25.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.063+0000 7fb44bdf0b80 1 --2- 172.21.15.117:0/1998767475 >> [v2:172.21.15.183:6836/2697518526,v1:172.21.15.183:6837/2697518526] conn(0x555e331872a0 0x555e33189690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:25.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.063+0000 7fb44bdf0b80 4 client.25418 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T03:26:25.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.063+0000 7fb44bdf0b80 1 -- 172.21.15.117:0/1998767475 --> [v2:172.21.15.183:6836/2697518526,v1:172.21.15.183:6837/2697518526] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555e330c4000 con 0x555e331872a0 2024-06-03T03:26:25.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.064+0000 7fb439e00700 1 --2- 172.21.15.117:0/1998767475 >> [v2:172.21.15.183:6836/2697518526,v1:172.21.15.183:6837/2697518526] conn(0x555e331872a0 0x555e33189690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:25.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.064+0000 7fb424c00700 10 client.25418 ms_handle_connect on v2:172.21.15.183:6836/2697518526 2024-06-03T03:26:25.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.064+0000 7fb439e00700 1 --2- 172.21.15.117:0/1998767475 >> [v2:172.21.15.183:6836/2697518526,v1:172.21.15.183:6837/2697518526] conn(0x555e331872a0 0x555e33189690 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a62ce3cd818eb98c server_cookie=8763b19ea6b7c9bc in_seq=0 out_seq=0 2024-06-03T03:26:25.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.065+0000 7fb424c00700 1 -- 172.21.15.117:0/1998767475 <== mds.3 v2:172.21.15.183:6836/2697518526 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x555e330c4000 con 0x555e331872a0 2024-06-03T03:26:25.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.065+0000 7fb424c00700 10 client.25418 handle_command_reply: tid=0 2024-06-03T03:26:25.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.065+0000 7fb44bdf0b80 1 client.25418 shutdown 2024-06-03T03:26:25.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.066+0000 7fb44bdf0b80 1 -- 172.21.15.117:0/1998767475 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e330e0e40 msgr2=0x555e33173070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:25.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.066+0000 7fb44bdf0b80 1 --2- 172.21.15.117:0/1998767475 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e330e0e40 0x555e33173070 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb42c006010 tx=0x7fb42c00b760 comp rx=0 tx=0).stop 2024-06-03T03:26:25.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.066+0000 7fb44bdf0b80 1 -- 172.21.15.117:0/1998767475 shutdown_connections 2024-06-03T03:26:25.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.066+0000 7fb44bdf0b80 1 --2- 172.21.15.117:0/1998767475 >> [v2:172.21.15.183:6836/2697518526,v1:172.21.15.183:6837/2697518526] conn(0x555e331872a0 0x555e33189690 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:25.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.066+0000 7fb44bdf0b80 1 --2- 172.21.15.117:0/1998767475 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e33173710 0x555e3316c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:25.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.066+0000 7fb44bdf0b80 1 --2- 172.21.15.117:0/1998767475 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e330e0e40 0x555e33173070 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:25.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.066+0000 7fb44bdf0b80 1 --2- 172.21.15.117:0/1998767475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e331645b0 0x555e3316c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:25.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.066+0000 7fb424c00700 0 client.25418 ms_handle_reset on v2:172.21.15.183:6836/2697518526 2024-06-03T03:26:25.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.066+0000 7fb44bdf0b80 1 -- 172.21.15.117:0/1998767475 >> 172.21.15.117:0/1998767475 conn(0x555e33091170 msgr2=0x555e3315f740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:26:25.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.066+0000 7fb44bdf0b80 1 -- 172.21.15.117:0/1998767475 shutdown_connections 2024-06-03T03:26:25.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.066+0000 7fb44bdf0b80 1 -- 172.21.15.117:0/1998767475 wait complete. 2024-06-03T03:26:25.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.066+0000 7fb44bdf0b80 20 client.25418 trim_cache size 0 max 16384 2024-06-03T03:26:25.083 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.082+0000 7fb44bdf0b80 1 Processor -- start 2024-06-03T03:26:25.084 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.082+0000 7fb44bdf0b80 1 -- start start 2024-06-03T03:26:25.084 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.082+0000 7fb44bdf0b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e330e0e40 0x555e33160620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:25.084 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.082+0000 7fb44bdf0b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e331645b0 0x555e33172cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:25.084 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.082+0000 7fb44bdf0b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e33173710 0x555e33174b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:25.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.082+0000 7fb44bdf0b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb43c1ac100 con 0x555e331645b0 2024-06-03T03:26:25.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.082+0000 7fb44bdf0b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb43c1adc40 con 0x555e33173710 2024-06-03T03:26:25.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.082+0000 7fb44bdf0b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb43c1ac7d0 con 0x555e330e0e40 2024-06-03T03:26:25.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.083+0000 7fb439e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e331645b0 0x555e33172cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:25.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.083+0000 7fb439e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e331645b0 0x555e33172cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39152/0 (socket says 172.21.15.117:39152) 2024-06-03T03:26:25.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.083+0000 7fb439e00700 1 -- 172.21.15.117:0/3069006202 learned_addr learned my addr 172.21.15.117:0/3069006202 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:26:25.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.083+0000 7fb43a800700 1 --2- 172.21.15.117:0/3069006202 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e330e0e40 0x555e33160620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:25.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.083+0000 7fb43b200700 1 --2- 172.21.15.117:0/3069006202 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e33173710 0x555e33174b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:25.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.083+0000 7fb439e00700 1 -- 172.21.15.117:0/3069006202 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e330e0e40 msgr2=0x555e33160620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:25.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.083+0000 7fb439e00700 1 --2- 172.21.15.117:0/3069006202 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e330e0e40 0x555e33160620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:25.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.083+0000 7fb439e00700 1 -- 172.21.15.117:0/3069006202 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e33173710 msgr2=0x555e33174b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:25.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.083+0000 7fb439e00700 1 --2- 172.21.15.117:0/3069006202 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e33173710 0x555e33174b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:25.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.083+0000 7fb439e00700 1 -- 172.21.15.117:0/3069006202 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e3316a8a0 con 0x555e331645b0 2024-06-03T03:26:25.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.084+0000 7fb439e00700 1 --2- 172.21.15.117:0/3069006202 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e331645b0 0x555e33172cb0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb42800f010 tx=0x7fb4280866b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f1876e801ec7f34 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:26:25.087 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.084+0000 7fb427400700 1 -- 172.21.15.117:0/3069006202 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb42809ce60 con 0x555e331645b0 2024-06-03T03:26:25.087 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.084+0000 7fb427400700 1 -- 172.21.15.117:0/3069006202 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb42809bb30 con 0x555e331645b0 2024-06-03T03:26:25.087 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.084+0000 7fb427400700 1 -- 172.21.15.117:0/3069006202 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb428086320 con 0x555e331645b0 2024-06-03T03:26:25.087 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.085+0000 7fb44bdf0b80 1 -- 172.21.15.117:0/3069006202 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e331645b0 msgr2=0x555e33172cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:25.087 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.085+0000 7fb44bdf0b80 1 --2- 172.21.15.117:0/3069006202 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e331645b0 0x555e33172cb0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb42800f010 tx=0x7fb4280866b0 comp rx=0 tx=0).stop 2024-06-03T03:26:25.087 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.085+0000 7fb44bdf0b80 1 -- 172.21.15.117:0/3069006202 shutdown_connections 2024-06-03T03:26:25.087 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.085+0000 7fb44bdf0b80 1 --2- 172.21.15.117:0/3069006202 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e33173710 0x555e33174b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:25.087 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.085+0000 7fb44bdf0b80 1 --2- 172.21.15.117:0/3069006202 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e330e0e40 0x555e33160620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:25.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.085+0000 7fb44bdf0b80 1 --2- 172.21.15.117:0/3069006202 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e331645b0 0x555e33172cb0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:25.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.085+0000 7fb44bdf0b80 1 -- 172.21.15.117:0/3069006202 >> 172.21.15.117:0/3069006202 conn(0x555e33091170 msgr2=0x555e3315f740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:26:25.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.085+0000 7fb44bdf0b80 1 -- 172.21.15.117:0/3069006202 shutdown_connections 2024-06-03T03:26:25.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.085+0000 7fb44bdf0b80 1 -- 172.21.15.117:0/3069006202 wait complete. 2024-06-03T03:26:25.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.086+0000 7fb44bdf0b80 1 -- start start 2024-06-03T03:26:25.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.086+0000 7fb44bdf0b80 1 Processor -- start 2024-06-03T03:26:25.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.087+0000 7fb44bdf0b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e330e0e40 0x555e3317e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:25.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.087+0000 7fb44bdf0b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e33173710 0x555e3317ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:25.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.087+0000 7fb44bdf0b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e3317f2d0 0x555e331efbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:25.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.087+0000 7fb44bdf0b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb43c1ad570 con 0x555e33173710 2024-06-03T03:26:25.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.087+0000 7fb44bdf0b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb43c1acea0 con 0x555e3317f2d0 2024-06-03T03:26:25.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.087+0000 7fb44bdf0b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb43c1aa0c0 con 0x555e330e0e40 2024-06-03T03:26:25.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.087+0000 7fb44bdf0b80 10 client.0 _ll_register_callbacks cb 0x555e33050720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:26:25.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.087+0000 7fb43a800700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e330e0e40 0x555e3317e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:25.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.087+0000 7fb43a800700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e330e0e40 0x555e3317e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:35488/0 (socket says 172.21.15.117:35488) 2024-06-03T03:26:25.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.087+0000 7fb43a800700 1 -- 172.21.15.117:0/3322443457 learned_addr learned my addr 172.21.15.117:0/3322443457 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:26:25.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.087+0000 7fb43b200700 1 --2- 172.21.15.117:0/3322443457 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e3317f2d0 0x555e331efbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:25.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.088+0000 7fb439e00700 1 --2- 172.21.15.117:0/3322443457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e33173710 0x555e3317ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:25.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.088+0000 7fb43a800700 1 -- 172.21.15.117:0/3322443457 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e3317f2d0 msgr2=0x555e331efbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:25.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.088+0000 7fb43a800700 1 --2- 172.21.15.117:0/3322443457 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e3317f2d0 0x555e331efbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:25.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.088+0000 7fb43a800700 1 -- 172.21.15.117:0/3322443457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e33173710 msgr2=0x555e3317ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:25.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.088+0000 7fb43a800700 1 --2- 172.21.15.117:0/3322443457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e33173710 0x555e3317ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:25.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.088+0000 7fb43a800700 1 -- 172.21.15.117:0/3322443457 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e331f0100 con 0x555e330e0e40 2024-06-03T03:26:25.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.088+0000 7fb439e00700 1 --2- 172.21.15.117:0/3322443457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e33173710 0x555e3317ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T03:26:25.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.089+0000 7fb43a800700 1 --2- 172.21.15.117:0/3322443457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e330e0e40 0x555e3317e850 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb42c006010 tx=0x7fb42c005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e399f55c1312bc31 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:26:25.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.089+0000 7fb424c00700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-06-03T03:26:25.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.089+0000 7fb424c00700 1 -- 172.21.15.117:0/3322443457 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb42c041060 con 0x555e330e0e40 2024-06-03T03:26:25.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.089+0000 7fb44bdf0b80 1 -- 172.21.15.117:0/3322443457 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb42809bb30 con 0x555e330e0e40 2024-06-03T03:26:25.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.090+0000 7fb424c00700 1 -- 172.21.15.117:0/3322443457 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7fb42c031070 con 0x555e330e0e40 2024-06-03T03:26:25.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.090+0000 7fb424c00700 1 -- 172.21.15.117:0/3322443457 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fb42c048050 con 0x555e330e0e40 2024-06-03T03:26:25.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.090+0000 7fb44bdf0b80 10 client.25450 fetch_fsmap learned FSMap version 31 2024-06-03T03:26:25.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.090+0000 7fb44bdf0b80 1 -- 172.21.15.117:0/3322443457 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x555e331f1340 con 0x555e330e0e40 2024-06-03T03:26:25.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.091+0000 7fb424c00700 1 -- 172.21.15.117:0/3322443457 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb42809bb30 con 0x555e330e0e40 2024-06-03T03:26:25.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.091+0000 7fb424c00700 1 -- 172.21.15.117:0/3322443457 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fb42c0229a0 con 0x555e330e0e40 2024-06-03T03:26:25.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.091+0000 7fb44bdf0b80 10 client.25450 fetch_fsmap finished waiting for FSMap version 31 2024-06-03T03:26:25.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.091+0000 7fb44bdf0b80 10 client.25450 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-06-03T03:26:25.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.091+0000 7fb44bdf0b80 20 client.25450 populate_metadata read hostname 'smithi117' 2024-06-03T03:26:25.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.091+0000 7fb44bdf0b80 1 --2- 172.21.15.117:0/3322443457 >> [v2:172.21.15.183:6836/2697518526,v1:172.21.15.183:6837/2697518526] conn(0x555e331f1a50 0x555e331f3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:25.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.091+0000 7fb44bdf0b80 4 client.25450 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T03:26:25.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.091+0000 7fb44bdf0b80 1 -- 172.21.15.117:0/3322443457 --> [v2:172.21.15.183:6836/2697518526,v1:172.21.15.183:6837/2697518526] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555e330be580 con 0x555e331f1a50 2024-06-03T03:26:25.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.092+0000 7fb439e00700 1 --2- 172.21.15.117:0/3322443457 >> [v2:172.21.15.183:6836/2697518526,v1:172.21.15.183:6837/2697518526] conn(0x555e331f1a50 0x555e331f3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:25.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.093+0000 7fb439e00700 1 --2- 172.21.15.117:0/3322443457 >> [v2:172.21.15.183:6836/2697518526,v1:172.21.15.183:6837/2697518526] conn(0x555e331f1a50 0x555e331f3e70 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=94c2d2e93240d822 server_cookie=d4be8ee002355776 in_seq=0 out_seq=0 2024-06-03T03:26:25.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.093+0000 7fb424c00700 10 client.25450 ms_handle_connect on v2:172.21.15.183:6836/2697518526 2024-06-03T03:26:25.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.093+0000 7fb424c00700 1 -- 172.21.15.117:0/3322443457 <== mds.3 v2:172.21.15.183:6836/2697518526 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555e330be580 con 0x555e331f1a50 2024-06-03T03:26:25.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.093+0000 7fb424c00700 10 client.25450 handle_command_reply: tid=0 2024-06-03T03:26:25.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.093+0000 7fb44bdf0b80 1 client.25450 shutdown 2024-06-03T03:26:25.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.094+0000 7fb44bdf0b80 1 -- 172.21.15.117:0/3322443457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e330e0e40 msgr2=0x555e3317e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:25.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.094+0000 7fb44bdf0b80 1 --2- 172.21.15.117:0/3322443457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e330e0e40 0x555e3317e850 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb42c006010 tx=0x7fb42c005e80 comp rx=0 tx=0).stop 2024-06-03T03:26:25.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.094+0000 7fb44bdf0b80 1 -- 172.21.15.117:0/3322443457 shutdown_connections 2024-06-03T03:26:25.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.094+0000 7fb44bdf0b80 1 --2- 172.21.15.117:0/3322443457 >> [v2:172.21.15.183:6836/2697518526,v1:172.21.15.183:6837/2697518526] conn(0x555e331f1a50 0x555e331f3e70 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:25.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.094+0000 7fb44bdf0b80 1 --2- 172.21.15.117:0/3322443457 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e3317f2d0 0x555e331efbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:25.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.094+0000 7fb44bdf0b80 1 --2- 172.21.15.117:0/3322443457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e330e0e40 0x555e3317e850 secure :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb42c006010 tx=0x7fb42c005e80 comp rx=0 tx=0).stop 2024-06-03T03:26:25.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.094+0000 7fb44bdf0b80 1 --2- 172.21.15.117:0/3322443457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e33173710 0x555e3317ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:25.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.094+0000 7fb44bdf0b80 1 -- 172.21.15.117:0/3322443457 >> 172.21.15.117:0/3322443457 conn(0x555e33091170 msgr2=0x555e33166060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:26:25.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.094+0000 7fb424c00700 0 client.25450 ms_handle_reset on v2:172.21.15.183:6836/2697518526 2024-06-03T03:26:25.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.095+0000 7fb44bdf0b80 1 -- 172.21.15.117:0/3322443457 shutdown_connections 2024-06-03T03:26:25.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.095+0000 7fb44bdf0b80 1 -- 172.21.15.117:0/3322443457 wait complete. 2024-06-03T03:26:25.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.095+0000 7fb44bdf0b80 20 client.25450 trim_cache size 0 max 16384 2024-06-03T03:26:25.097 INFO:teuthology.orchestra.run.smithi117.stdout:[] 2024-06-03T03:26:25.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.096+0000 7fb441400700 1 -- 172.21.15.117:0/1870156428 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fb4200a2d30 msgr2=0x7fb4200a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:25.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.096+0000 7fb441400700 1 --2- 172.21.15.117:0/1870156428 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fb4200a2d30 0x7fb4200a51f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x555e331613b0 tx=0x7fb43400b040 comp rx=0 tx=0).stop 2024-06-03T03:26:25.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.096+0000 7fb441400700 1 -- 172.21.15.117:0/1870156428 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb43c06ab30 msgr2=0x7fb43c19ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:25.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.096+0000 7fb441400700 1 --2- 172.21.15.117:0/1870156428 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb43c06ab30 0x7fb43c19ee70 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb42800e9a0 tx=0x7fb42800ecb0 comp rx=0 tx=0).stop 2024-06-03T03:26:25.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.096+0000 7fb441400700 1 -- 172.21.15.117:0/1870156428 shutdown_connections 2024-06-03T03:26:25.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.096+0000 7fb441400700 1 --2- 172.21.15.117:0/1870156428 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fb4200a2d30 0x7fb4200a51f0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:25.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.096+0000 7fb441400700 1 --2- 172.21.15.117:0/1870156428 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb43c108500 0x7fb43c198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:25.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.096+0000 7fb441400700 1 --2- 172.21.15.117:0/1870156428 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb43c06a170 0x7fb43c19e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:25.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.096+0000 7fb441400700 1 --2- 172.21.15.117:0/1870156428 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb43c06ab30 0x7fb43c19ee70 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:25.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.096+0000 7fb441400700 1 -- 172.21.15.117:0/1870156428 >> 172.21.15.117:0/1870156428 conn(0x7fb43c078340 msgr2=0x7fb43c100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:26:25.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.096+0000 7fb441400700 1 -- 172.21.15.117:0/1870156428 shutdown_connections 2024-06-03T03:26:25.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:25.096+0000 7fb441400700 1 -- 172.21.15.117:0/1870156428 wait complete. 2024-06-03T03:26:25.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:26:25 smithi151 ceph-mon[37043]: pgmap v539: 129 pgs: 129 active+clean; 679 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 341 B/s wr, 9 op/s 2024-06-03T03:26:25.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:26:25 smithi117 ceph-mon[31649]: pgmap v539: 129 pgs: 129 active+clean; 679 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 341 B/s wr, 9 op/s 2024-06-03T03:26:25.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:26:25 smithi183 ceph-mon[37326]: pgmap v539: 129 pgs: 129 active+clean; 679 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 341 B/s wr, 9 op/s 2024-06-03T03:26:26.137 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-03T03:26:26.643 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:26:27.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:26:27 smithi151 ceph-mon[37043]: mgrmap e20: x(active, since 15m), standbys: z, y 2024-06-03T03:26:27.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:26:27 smithi151 ceph-mon[37043]: pgmap v540: 129 pgs: 129 active+clean; 679 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 511 B/s wr, 9 op/s 2024-06-03T03:26:27.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:26:27 smithi117 ceph-mon[31649]: mgrmap e20: x(active, since 15m), standbys: z, y 2024-06-03T03:26:27.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:26:27 smithi117 ceph-mon[31649]: pgmap v540: 129 pgs: 129 active+clean; 679 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 511 B/s wr, 9 op/s 2024-06-03T03:26:27.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:26:27 smithi183 ceph-mon[37326]: mgrmap e20: x(active, since 15m), standbys: z, y 2024-06-03T03:26:27.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:26:27 smithi183 ceph-mon[37326]: pgmap v540: 129 pgs: 129 active+clean; 679 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 511 B/s wr, 9 op/s 2024-06-03T03:26:29.032 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.031+0000 7fc8fb000700 1 -- 172.21.15.117:0/174481871 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8f4103fc0 msgr2=0x7fc8f4104440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:29.033 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.031+0000 7fc8fb000700 1 --2- 172.21.15.117:0/174481871 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8f4103fc0 0x7fc8f4104440 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc8e00099c0 tx=0x7fc8e0009cd0 comp rx=0 tx=0).stop 2024-06-03T03:26:29.033 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.031+0000 7fc8fb000700 1 -- 172.21.15.117:0/174481871 shutdown_connections 2024-06-03T03:26:29.033 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.031+0000 7fc8fb000700 1 --2- 172.21.15.117:0/174481871 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc8f4104980 0x7fc8f410f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:29.033 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.031+0000 7fc8fb000700 1 --2- 172.21.15.117:0/174481871 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8f41026d0 0x7fc8f4102ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:29.034 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.031+0000 7fc8fb000700 1 --2- 172.21.15.117:0/174481871 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8f4103fc0 0x7fc8f4104440 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:29.034 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.031+0000 7fc8fb000700 1 -- 172.21.15.117:0/174481871 >> 172.21.15.117:0/174481871 conn(0x7fc8f40fde80 msgr2=0x7fc8f41002a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:26:29.034 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.031+0000 7fc8fb000700 1 -- 172.21.15.117:0/174481871 shutdown_connections 2024-06-03T03:26:29.034 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.032+0000 7fc8fb000700 1 -- 172.21.15.117:0/174481871 wait complete. 2024-06-03T03:26:29.034 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.033+0000 7fc8fb000700 1 Processor -- start 2024-06-03T03:26:29.034 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.033+0000 7fc8fb000700 1 -- start start 2024-06-03T03:26:29.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.033+0000 7fc8fb000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8f41026d0 0x7fc8f419b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:29.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.033+0000 7fc8fb000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8f4103fc0 0x7fc8f419bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:29.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.033+0000 7fc8fb000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc8f4104980 0x7fc8f410e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:29.036 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.033+0000 7fc8fb000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f4112780 con 0x7fc8f41026d0 2024-06-03T03:26:29.036 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.033+0000 7fc8fb000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f4112600 con 0x7fc8f4103fc0 2024-06-03T03:26:29.036 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.033+0000 7fc8fb000700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f4112900 con 0x7fc8f4104980 2024-06-03T03:26:29.036 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.034+0000 7fc8f9200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8f4103fc0 0x7fc8f419bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:29.036 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.034+0000 7fc8f9200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8f4103fc0 0x7fc8f419bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:39626/0 (socket says 172.21.15.117:39626) 2024-06-03T03:26:29.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.034+0000 7fc8f9200700 1 -- 172.21.15.117:0/1713893829 learned_addr learned my addr 172.21.15.117:0/1713893829 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:26:29.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.034+0000 7fc8f9c00700 1 --2- 172.21.15.117:0/1713893829 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8f41026d0 0x7fc8f419b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:29.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.034+0000 7fc8fa600700 1 --2- 172.21.15.117:0/1713893829 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc8f4104980 0x7fc8f410e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:29.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.034+0000 7fc8f9200700 1 -- 172.21.15.117:0/1713893829 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc8f4104980 msgr2=0x7fc8f410e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:29.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.034+0000 7fc8f9200700 1 --2- 172.21.15.117:0/1713893829 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc8f4104980 0x7fc8f410e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:29.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.034+0000 7fc8f9200700 1 -- 172.21.15.117:0/1713893829 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8f41026d0 msgr2=0x7fc8f419b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:29.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.034+0000 7fc8f9200700 1 --2- 172.21.15.117:0/1713893829 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8f41026d0 0x7fc8f419b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:29.038 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.034+0000 7fc8f9200700 1 -- 172.21.15.117:0/1713893829 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8f410edd0 con 0x7fc8f4103fc0 2024-06-03T03:26:29.038 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.035+0000 7fc8f9200700 1 --2- 172.21.15.117:0/1713893829 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8f4103fc0 0x7fc8f419bb20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc8e00059d0 tx=0x7fc8e000fab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:26:29.038 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.035+0000 7fc8f2000700 1 -- 172.21.15.117:0/1713893829 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e0027070 con 0x7fc8f4103fc0 2024-06-03T03:26:29.038 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.035+0000 7fc8f2000700 1 -- 172.21.15.117:0/1713893829 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8e0003680 con 0x7fc8f4103fc0 2024-06-03T03:26:29.038 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.035+0000 7fc8f2000700 1 -- 172.21.15.117:0/1713893829 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e0017650 con 0x7fc8f4103fc0 2024-06-03T03:26:29.038 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.035+0000 7fc8fb000700 1 -- 172.21.15.117:0/1713893829 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8f410f060 con 0x7fc8f4103fc0 2024-06-03T03:26:29.038 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.035+0000 7fc8fb000700 1 -- 172.21.15.117:0/1713893829 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8f419f4e0 con 0x7fc8f4103fc0 2024-06-03T03:26:29.042 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.041+0000 7fc8f2000700 1 -- 172.21.15.117:0/1713893829 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fc8e002a070 con 0x7fc8f4103fc0 2024-06-03T03:26:29.044 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.042+0000 7fc906f64b80 1 Processor -- start 2024-06-03T03:26:29.044 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.043+0000 7fc8f2000700 1 --2- 172.21.15.117:0/1713893829 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fc8dc0a2ca0 0x7fc8dc0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:29.044 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.043+0000 7fc8f2000700 1 -- 172.21.15.117:0/1713893829 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7fc8e00d00f0 con 0x7fc8f4103fc0 2024-06-03T03:26:29.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.043+0000 7fc8fa600700 1 --2- 172.21.15.117:0/1713893829 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fc8dc0a2ca0 0x7fc8dc0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:29.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.044+0000 7fc906f64b80 1 -- start start 2024-06-03T03:26:29.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.044+0000 7fc906f64b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561bccc2fd60 0x561bccc30140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:29.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.044+0000 7fc906f64b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561bccc30680 0x561bccc3aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:29.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.044+0000 7fc906f64b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561bccc3b530 0x561bccc3d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:29.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.044+0000 7fc906f64b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561bccb3fa10 con 0x561bccc2fd60 2024-06-03T03:26:29.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.044+0000 7fc906f64b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x561bccb82b00 con 0x561bccc3b530 2024-06-03T03:26:29.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.044+0000 7fc906f64b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561bccb3d8f0 con 0x561bccc30680 2024-06-03T03:26:29.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.044+0000 7fc8fa600700 1 --2- 172.21.15.117:0/1713893829 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fc8dc0a2ca0 0x7fc8dc0a5160 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc8f419c230 tx=0x7fc8ec009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:26:29.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.045+0000 7fc8f9c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561bccc2fd60 0x561bccc30140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:29.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.045+0000 7fc8f9c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561bccc2fd60 0x561bccc30140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:55144/0 (socket says 172.21.15.117:55144) 2024-06-03T03:26:29.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.045+0000 7fc8fa600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561bccc3b530 0x561bccc3d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:29.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.045+0000 7fc8f9c00700 1 -- 172.21.15.117:0/3824689603 learned_addr learned my addr 172.21.15.117:0/3824689603 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:26:29.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.045+0000 7fc8fa600700 1 -- 172.21.15.117:0/3824689603 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561bccc30680 msgr2=0x561bccc3aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:29.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.045+0000 7fc8fa600700 1 --2- 172.21.15.117:0/3824689603 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561bccc30680 0x561bccc3aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:29.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.045+0000 7fc8fa600700 1 -- 172.21.15.117:0/3824689603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561bccc2fd60 msgr2=0x561bccc30140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:29.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.045+0000 7fc8fa600700 1 --2- 172.21.15.117:0/3824689603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561bccc2fd60 0x561bccc30140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:29.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.045+0000 7fc8fa600700 1 -- 172.21.15.117:0/3824689603 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561bccb2af90 con 0x561bccc3b530 2024-06-03T03:26:29.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.045+0000 7fc8fa600700 1 --2- 172.21.15.117:0/3824689603 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561bccc3b530 0x561bccc3d920 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec007df0 tx=0x7fc8ec012000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c371763b0e4fb705 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:26:29.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.046+0000 7fc8e6a00700 1 -- 172.21.15.117:0/3824689603 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8ec026070 con 0x561bccc3b530 2024-06-03T03:26:29.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.046+0000 7fc8e6a00700 1 -- 172.21.15.117:0/3824689603 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8ec016e50 con 0x561bccc3b530 2024-06-03T03:26:29.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.046+0000 7fc8e6a00700 1 -- 172.21.15.117:0/3824689603 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8ec014a70 con 0x561bccc3b530 2024-06-03T03:26:29.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.047+0000 7fc906f64b80 1 -- 172.21.15.117:0/3824689603 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561bccc3b530 msgr2=0x561bccc3d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:29.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.047+0000 7fc906f64b80 1 --2- 172.21.15.117:0/3824689603 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561bccc3b530 0x561bccc3d920 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec007df0 tx=0x7fc8ec012000 comp rx=0 tx=0).stop 2024-06-03T03:26:29.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.051+0000 7fc906f64b80 1 -- 172.21.15.117:0/3824689603 shutdown_connections 2024-06-03T03:26:29.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.051+0000 7fc906f64b80 1 --2- 172.21.15.117:0/3824689603 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561bccc3b530 0x561bccc3d920 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:29.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.051+0000 7fc906f64b80 1 --2- 172.21.15.117:0/3824689603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561bccc2fd60 0x561bccc30140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:29.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.051+0000 7fc906f64b80 1 --2- 172.21.15.117:0/3824689603 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561bccc30680 0x561bccc3aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:29.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.051+0000 7fc906f64b80 1 -- 172.21.15.117:0/3824689603 >> 172.21.15.117:0/3824689603 conn(0x561bccb5d170 msgr2=0x561bccc2c280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:26:29.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.052+0000 7fc906f64b80 1 -- 172.21.15.117:0/3824689603 shutdown_connections 2024-06-03T03:26:29.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.052+0000 7fc906f64b80 1 -- 172.21.15.117:0/3824689603 wait complete. 2024-06-03T03:26:29.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.052+0000 7fc906f64b80 1 -- start start 2024-06-03T03:26:29.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.053+0000 7fc906f64b80 1 Processor -- start 2024-06-03T03:26:29.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.053+0000 7fc906f64b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561bccc2fd60 0x561bccc3adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:29.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.053+0000 7fc906f64b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561bccc30680 0x561bccc380b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:29.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.053+0000 7fc906f64b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561bccc3b530 0x561bccc385f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:29.056 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.053+0000 7fc906f64b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561bccb1ef60 con 0x561bccc2fd60 2024-06-03T03:26:29.056 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.053+0000 7fc906f64b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x561bccb7eac0 con 0x561bccc3b530 2024-06-03T03:26:29.056 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.053+0000 7fc906f64b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561bccc3dfc0 con 0x561bccc30680 2024-06-03T03:26:29.056 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.053+0000 7fc906f64b80 10 client.0 _ll_register_callbacks cb 0x561bccb18380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:26:29.056 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.053+0000 7fc8f9c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561bccc2fd60 0x561bccc3adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:29.056 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.053+0000 7fc8f9c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561bccc2fd60 0x561bccc3adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:55156/0 (socket says 172.21.15.117:55156) 2024-06-03T03:26:29.056 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.053+0000 7fc8f9c00700 1 -- 172.21.15.117:0/2560518227 learned_addr learned my addr 172.21.15.117:0/2560518227 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:26:29.056 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.053+0000 7fc8f9200700 1 --2- 172.21.15.117:0/2560518227 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561bccc30680 0x561bccc380b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:29.056 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.054+0000 7fc8f9c00700 1 -- 172.21.15.117:0/2560518227 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561bccc30680 msgr2=0x561bccc380b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:29.056 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.054+0000 7fc8f9c00700 1 --2- 172.21.15.117:0/2560518227 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561bccc30680 0x561bccc380b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:29.056 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.054+0000 7fc8f9c00700 1 -- 172.21.15.117:0/2560518227 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561bccc3b530 msgr2=0x561bccc385f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:29.056 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.054+0000 7fc8f9c00700 1 --2- 172.21.15.117:0/2560518227 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561bccc3b530 0x561bccc385f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:29.056 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.054+0000 7fc8f9c00700 1 -- 172.21.15.117:0/2560518227 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561bccad18d0 con 0x561bccc2fd60 2024-06-03T03:26:29.057 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.054+0000 7fc8f9c00700 1 --2- 172.21.15.117:0/2560518227 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561bccc2fd60 0x561bccc3adb0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc8e8013750 tx=0x7fc8e8013a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b76a642ef607cbea server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:26:29.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.054+0000 7fc8db400700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-06-03T03:26:29.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.055+0000 7fc8db400700 1 -- 172.21.15.117:0/2560518227 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e8011990 con 0x561bccc2fd60 2024-06-03T03:26:29.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.055+0000 7fc8db400700 1 -- 172.21.15.117:0/2560518227 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7fc8e8011b30 con 0x561bccc2fd60 2024-06-03T03:26:29.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.055+0000 7fc906f64b80 1 -- 172.21.15.117:0/2560518227 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8ec016e50 con 0x561bccc2fd60 2024-06-03T03:26:29.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.055+0000 7fc8db400700 1 -- 172.21.15.117:0/2560518227 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc8e8028050 con 0x561bccc2fd60 2024-06-03T03:26:29.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.055+0000 7fc906f64b80 10 client.15324 fetch_fsmap learned FSMap version 31 2024-06-03T03:26:29.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.055+0000 7fc906f64b80 1 -- 172.21.15.117:0/2560518227 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x561bccc59870 con 0x561bccc2fd60 2024-06-03T03:26:29.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.056+0000 7fc8db400700 1 -- 172.21.15.117:0/2560518227 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8ec016e50 con 0x561bccc2fd60 2024-06-03T03:26:29.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.057+0000 7fc8db400700 1 -- 172.21.15.117:0/2560518227 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fc8e800f430 con 0x561bccc2fd60 2024-06-03T03:26:29.060 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.057+0000 7fc906f64b80 10 client.15324 fetch_fsmap finished waiting for FSMap version 31 2024-06-03T03:26:29.060 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.057+0000 7fc906f64b80 10 client.15324 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.a 2024-06-03T03:26:29.060 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.057+0000 7fc906f64b80 20 client.15324 populate_metadata read hostname 'smithi117' 2024-06-03T03:26:29.060 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.057+0000 7fc906f64b80 1 --2- 172.21.15.117:0/2560518227 >> [v2:172.21.15.117:6840/1212242967,v1:172.21.15.117:6841/1212242967] conn(0x561bccc5bed0 0x561bccc5e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:29.060 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.057+0000 7fc906f64b80 4 client.15324 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T03:26:29.060 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.057+0000 7fc906f64b80 1 -- 172.21.15.117:0/2560518227 --> [v2:172.21.15.117:6840/1212242967,v1:172.21.15.117:6841/1212242967] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561bccb90000 con 0x561bccc5bed0 2024-06-03T03:26:29.060 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.058+0000 7fc8f9200700 1 --2- 172.21.15.117:0/2560518227 >> [v2:172.21.15.117:6840/1212242967,v1:172.21.15.117:6841/1212242967] conn(0x561bccc5bed0 0x561bccc5e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:29.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.059+0000 7fc8f9200700 1 --2- 172.21.15.117:0/2560518227 >> [v2:172.21.15.117:6840/1212242967,v1:172.21.15.117:6841/1212242967] conn(0x561bccc5bed0 0x561bccc5e360 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1c5553815a65c6a6 server_cookie=45ce50c840fb0003 in_seq=0 out_seq=0 2024-06-03T03:26:29.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.059+0000 7fc8db400700 10 client.15324 ms_handle_connect on v2:172.21.15.117:6840/1212242967 2024-06-03T03:26:29.062 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.060+0000 7fc8db400700 1 -- 172.21.15.117:0/2560518227 <== mds.2 v2:172.21.15.117:6840/1212242967 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x561bccb90000 con 0x561bccc5bed0 2024-06-03T03:26:29.062 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.060+0000 7fc8db400700 10 client.15324 handle_command_reply: tid=0 2024-06-03T03:26:29.062 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.060+0000 7fc906f64b80 1 client.15324 shutdown 2024-06-03T03:26:29.062 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.061+0000 7fc906f64b80 1 -- 172.21.15.117:0/2560518227 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561bccc2fd60 msgr2=0x561bccc3adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:29.062 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.061+0000 7fc906f64b80 1 --2- 172.21.15.117:0/2560518227 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561bccc2fd60 0x561bccc3adb0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc8e8013750 tx=0x7fc8e8013a60 comp rx=0 tx=0).stop 2024-06-03T03:26:29.063 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.061+0000 7fc906f64b80 1 -- 172.21.15.117:0/2560518227 shutdown_connections 2024-06-03T03:26:29.063 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.061+0000 7fc906f64b80 1 --2- 172.21.15.117:0/2560518227 >> [v2:172.21.15.117:6840/1212242967,v1:172.21.15.117:6841/1212242967] conn(0x561bccc5bed0 0x561bccc5e360 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:29.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.061+0000 7fc906f64b80 1 --2- 172.21.15.117:0/2560518227 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561bccc3b530 0x561bccc385f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:29.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.061+0000 7fc8db400700 0 client.15324 ms_handle_reset on v2:172.21.15.117:6840/1212242967 2024-06-03T03:26:29.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.061+0000 7fc906f64b80 1 --2- 172.21.15.117:0/2560518227 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561bccc2fd60 0x561bccc3adb0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:29.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.061+0000 7fc906f64b80 1 --2- 172.21.15.117:0/2560518227 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561bccc30680 0x561bccc380b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:29.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.061+0000 7fc906f64b80 1 -- 172.21.15.117:0/2560518227 >> 172.21.15.117:0/2560518227 conn(0x561bccb5d170 msgr2=0x561bccc2be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:26:29.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.062+0000 7fc906f64b80 1 -- 172.21.15.117:0/2560518227 shutdown_connections 2024-06-03T03:26:29.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.062+0000 7fc906f64b80 1 -- 172.21.15.117:0/2560518227 wait complete. 2024-06-03T03:26:29.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.062+0000 7fc906f64b80 20 client.15324 trim_cache size 0 max 16384 2024-06-03T03:26:29.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.092+0000 7fc906f64b80 1 Processor -- start 2024-06-03T03:26:29.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.092+0000 7fc906f64b80 1 -- start start 2024-06-03T03:26:29.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.093+0000 7fc906f64b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561bccc2fd60 0x561bccc2c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:29.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.093+0000 7fc906f64b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561bccc30680 0x561bccc63cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:29.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.093+0000 7fc906f64b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561bccc3b530 0x561bccc64200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:29.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.093+0000 7fc906f64b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f4073980 con 0x561bccc2fd60 2024-06-03T03:26:29.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.093+0000 7fc906f64b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f41b6290 con 0x561bccc3b530 2024-06-03T03:26:29.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.093+0000 7fc906f64b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f4074050 con 0x561bccc30680 2024-06-03T03:26:29.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.093+0000 7fc8f9200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561bccc30680 0x561bccc63cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:29.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.094+0000 7fc8fa600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561bccc3b530 0x561bccc64200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:29.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.094+0000 7fc8fa600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561bccc3b530 0x561bccc64200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:39650/0 (socket says 172.21.15.117:39650) 2024-06-03T03:26:29.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.094+0000 7fc8f9200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561bccc30680 0x561bccc63cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:46602/0 (socket says 172.21.15.117:46602) 2024-06-03T03:26:29.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.094+0000 7fc8f9200700 1 -- 172.21.15.117:0/783741970 learned_addr learned my addr 172.21.15.117:0/783741970 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:26:29.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.094+0000 7fc8f9c00700 1 --2- 172.21.15.117:0/783741970 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561bccc2fd60 0x561bccc2c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:29.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.094+0000 7fc8f9200700 1 -- 172.21.15.117:0/783741970 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561bccc3b530 msgr2=0x561bccc64200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:29.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.094+0000 7fc8f9200700 1 --2- 172.21.15.117:0/783741970 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561bccc3b530 0x561bccc64200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:29.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.094+0000 7fc8f9200700 1 -- 172.21.15.117:0/783741970 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561bccc2fd60 msgr2=0x561bccc2c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:29.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.094+0000 7fc8f9200700 1 --2- 172.21.15.117:0/783741970 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561bccc2fd60 0x561bccc2c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:29.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.094+0000 7fc8f9200700 1 -- 172.21.15.117:0/783741970 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561bccc589e0 con 0x561bccc30680 2024-06-03T03:26:29.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.094+0000 7fc8f9c00700 1 --2- 172.21.15.117:0/783741970 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561bccc2fd60 0x561bccc2c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:26:29.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.095+0000 7fc8f9200700 1 --2- 172.21.15.117:0/783741970 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561bccc30680 0x561bccc63cc0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc8e009c5e0 tx=0x7fc8e0022c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf1e73ee7423f362 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:26:29.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.095+0000 7fc8e6000700 1 -- 172.21.15.117:0/783741970 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e0009e60 con 0x561bccc30680 2024-06-03T03:26:29.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.095+0000 7fc8e6000700 1 -- 172.21.15.117:0/783741970 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x561bccb90000 con 0x561bccc30680 2024-06-03T03:26:29.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.095+0000 7fc8e6000700 1 -- 172.21.15.117:0/783741970 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e009a710 con 0x561bccc30680 2024-06-03T03:26:29.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.096+0000 7fc906f64b80 1 -- 172.21.15.117:0/783741970 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561bccc30680 msgr2=0x561bccc63cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:29.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.096+0000 7fc906f64b80 1 --2- 172.21.15.117:0/783741970 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561bccc30680 0x561bccc63cc0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc8e009c5e0 tx=0x7fc8e0022c10 comp rx=0 tx=0).stop 2024-06-03T03:26:29.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.097+0000 7fc906f64b80 1 -- 172.21.15.117:0/783741970 shutdown_connections 2024-06-03T03:26:29.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.097+0000 7fc906f64b80 1 --2- 172.21.15.117:0/783741970 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561bccc3b530 0x561bccc64200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:29.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.097+0000 7fc906f64b80 1 --2- 172.21.15.117:0/783741970 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561bccc2fd60 0x561bccc2c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:29.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.097+0000 7fc906f64b80 1 --2- 172.21.15.117:0/783741970 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561bccc30680 0x561bccc63cc0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:29.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.097+0000 7fc906f64b80 1 -- 172.21.15.117:0/783741970 >> 172.21.15.117:0/783741970 conn(0x561bccb5d170 msgr2=0x561bccc369a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:26:29.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.097+0000 7fc906f64b80 1 -- 172.21.15.117:0/783741970 shutdown_connections 2024-06-03T03:26:29.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.097+0000 7fc906f64b80 1 -- 172.21.15.117:0/783741970 wait complete. 2024-06-03T03:26:29.100 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.098+0000 7fc906f64b80 1 -- start start 2024-06-03T03:26:29.100 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.098+0000 7fc906f64b80 1 Processor -- start 2024-06-03T03:26:29.100 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.099+0000 7fc906f64b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561bccc2fd60 0x561bccc3e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:29.100 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.099+0000 7fc906f64b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561bccc30680 0x561bccc3ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:29.100 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.099+0000 7fc906f64b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561bccc3b530 0x561bcccc4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:29.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.099+0000 7fc906f64b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f41b5bc0 con 0x561bccc30680 2024-06-03T03:26:29.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.099+0000 7fc906f64b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f41b5510 con 0x561bccc3b530 2024-06-03T03:26:29.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.099+0000 7fc906f64b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f4071920 con 0x561bccc2fd60 2024-06-03T03:26:29.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.099+0000 7fc906f64b80 10 client.0 _ll_register_callbacks cb 0x561bccb18380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:26:29.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.099+0000 7fc8f9c00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561bccc2fd60 0x561bccc3e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:29.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.099+0000 7fc8f9c00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561bccc2fd60 0x561bccc3e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:46616/0 (socket says 172.21.15.117:46616) 2024-06-03T03:26:29.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.099+0000 7fc8f9c00700 1 -- 172.21.15.117:0/533455175 learned_addr learned my addr 172.21.15.117:0/533455175 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:26:29.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.099+0000 7fc8f9200700 1 --2- 172.21.15.117:0/533455175 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561bccc30680 0x561bccc3ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:29.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.100+0000 7fc8fa600700 1 --2- 172.21.15.117:0/533455175 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561bccc3b530 0x561bcccc4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:29.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.100+0000 7fc8f9200700 1 -- 172.21.15.117:0/533455175 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561bccc2fd60 msgr2=0x561bccc3e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:29.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.100+0000 7fc8f9200700 1 --2- 172.21.15.117:0/533455175 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561bccc2fd60 0x561bccc3e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:29.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.100+0000 7fc8f9200700 1 -- 172.21.15.117:0/533455175 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561bccc3b530 msgr2=0x561bcccc4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:29.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.100+0000 7fc8f9200700 1 --2- 172.21.15.117:0/533455175 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561bccc3b530 0x561bcccc4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:29.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.100+0000 7fc8f9200700 1 -- 172.21.15.117:0/533455175 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561bcccc51c0 con 0x561bccc30680 2024-06-03T03:26:29.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.101+0000 7fc8f9200700 1 --2- 172.21.15.117:0/533455175 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561bccc30680 0x561bccc3ed10 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc8e009c5e0 tx=0x7fc8e00a5230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=596b20a3aaadc46 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:26:29.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.101+0000 7fc8db400700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-06-03T03:26:29.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.101+0000 7fc8db400700 1 -- 172.21.15.117:0/533455175 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e00ba060 con 0x561bccc30680 2024-06-03T03:26:29.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.101+0000 7fc8db400700 1 -- 172.21.15.117:0/533455175 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7fc8e009a950 con 0x561bccc30680 2024-06-03T03:26:29.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.101+0000 7fc906f64b80 1 -- 172.21.15.117:0/533455175 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561bccb90000 con 0x561bccc30680 2024-06-03T03:26:29.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.101+0000 7fc8db400700 1 -- 172.21.15.117:0/533455175 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc8e00b4070 con 0x561bccc30680 2024-06-03T03:26:29.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.101+0000 7fc906f64b80 10 client.15330 fetch_fsmap learned FSMap version 31 2024-06-03T03:26:29.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.101+0000 7fc906f64b80 1 -- 172.21.15.117:0/533455175 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x561bcccc6470 con 0x561bccc30680 2024-06-03T03:26:29.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.102+0000 7fc8db400700 1 -- 172.21.15.117:0/533455175 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x561bccb90000 con 0x561bccc30680 2024-06-03T03:26:29.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.102+0000 7fc8db400700 1 -- 172.21.15.117:0/533455175 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fc8e009c8b0 con 0x561bccc30680 2024-06-03T03:26:29.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.103+0000 7fc906f64b80 10 client.15330 fetch_fsmap finished waiting for FSMap version 31 2024-06-03T03:26:29.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.103+0000 7fc906f64b80 10 client.15330 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.a 2024-06-03T03:26:29.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.103+0000 7fc906f64b80 20 client.15330 populate_metadata read hostname 'smithi117' 2024-06-03T03:26:29.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.103+0000 7fc906f64b80 1 --2- 172.21.15.117:0/533455175 >> [v2:172.21.15.117:6840/1212242967,v1:172.21.15.117:6841/1212242967] conn(0x561bcccc6a70 0x561bcccc8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:29.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.103+0000 7fc906f64b80 4 client.15330 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T03:26:29.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.103+0000 7fc906f64b80 1 -- 172.21.15.117:0/533455175 --> [v2:172.21.15.117:6840/1212242967,v1:172.21.15.117:6841/1212242967] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561bccb8a580 con 0x561bcccc6a70 2024-06-03T03:26:29.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.103+0000 7fc8f9c00700 1 --2- 172.21.15.117:0/533455175 >> [v2:172.21.15.117:6840/1212242967,v1:172.21.15.117:6841/1212242967] conn(0x561bcccc6a70 0x561bcccc8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:29.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.104+0000 7fc8f9c00700 1 --2- 172.21.15.117:0/533455175 >> [v2:172.21.15.117:6840/1212242967,v1:172.21.15.117:6841/1212242967] conn(0x561bcccc6a70 0x561bcccc8e60 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b862e18fe604adf3 server_cookie=ff4f1b0547b5781c in_seq=0 out_seq=0 2024-06-03T03:26:29.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.104+0000 7fc8db400700 10 client.15330 ms_handle_connect on v2:172.21.15.117:6840/1212242967 2024-06-03T03:26:29.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.104+0000 7fc8db400700 1 -- 172.21.15.117:0/533455175 <== mds.2 v2:172.21.15.117:6840/1212242967 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561bccb8a580 con 0x561bcccc6a70 2024-06-03T03:26:29.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.104+0000 7fc8db400700 10 client.15330 handle_command_reply: tid=0 2024-06-03T03:26:29.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.105+0000 7fc906f64b80 1 client.15330 shutdown 2024-06-03T03:26:29.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.105+0000 7fc906f64b80 1 -- 172.21.15.117:0/533455175 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561bccc30680 msgr2=0x561bccc3ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:29.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.105+0000 7fc906f64b80 1 --2- 172.21.15.117:0/533455175 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561bccc30680 0x561bccc3ed10 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc8e009c5e0 tx=0x7fc8e00a5230 comp rx=0 tx=0).stop 2024-06-03T03:26:29.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.105+0000 7fc906f64b80 1 -- 172.21.15.117:0/533455175 shutdown_connections 2024-06-03T03:26:29.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.105+0000 7fc906f64b80 1 --2- 172.21.15.117:0/533455175 >> [v2:172.21.15.117:6840/1212242967,v1:172.21.15.117:6841/1212242967] conn(0x561bcccc6a70 0x561bcccc8e60 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:29.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.106+0000 7fc906f64b80 1 --2- 172.21.15.117:0/533455175 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561bccc3b530 0x561bcccc4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:29.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.106+0000 7fc906f64b80 1 --2- 172.21.15.117:0/533455175 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561bccc2fd60 0x561bccc3e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:29.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.106+0000 7fc906f64b80 1 --2- 172.21.15.117:0/533455175 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561bccc30680 0x561bccc3ed10 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:29.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.106+0000 7fc906f64b80 1 -- 172.21.15.117:0/533455175 >> 172.21.15.117:0/533455175 conn(0x561bccb5d170 msgr2=0x561bccc32f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:26:29.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.106+0000 7fc8db400700 0 client.15330 ms_handle_reset on v2:172.21.15.117:6840/1212242967 2024-06-03T03:26:29.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.106+0000 7fc906f64b80 1 -- 172.21.15.117:0/533455175 shutdown_connections 2024-06-03T03:26:29.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.106+0000 7fc906f64b80 1 -- 172.21.15.117:0/533455175 wait complete. 2024-06-03T03:26:29.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.107+0000 7fc906f64b80 20 client.15330 trim_cache size 0 max 16384 2024-06-03T03:26:29.109 INFO:teuthology.orchestra.run.smithi117.stdout:[] 2024-06-03T03:26:29.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.108+0000 7fc8fb000700 1 -- 172.21.15.117:0/1713893829 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fc8dc0a2ca0 msgr2=0x7fc8dc0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:29.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.108+0000 7fc8fb000700 1 --2- 172.21.15.117:0/1713893829 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fc8dc0a2ca0 0x7fc8dc0a5160 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc8f419c230 tx=0x7fc8ec009040 comp rx=0 tx=0).stop 2024-06-03T03:26:29.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.108+0000 7fc8fb000700 1 -- 172.21.15.117:0/1713893829 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8f4103fc0 msgr2=0x7fc8f419bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:29.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.108+0000 7fc8fb000700 1 --2- 172.21.15.117:0/1713893829 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8f4103fc0 0x7fc8f419bb20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc8e00059d0 tx=0x7fc8e000fab0 comp rx=0 tx=0).stop 2024-06-03T03:26:29.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.108+0000 7fc8fb000700 1 -- 172.21.15.117:0/1713893829 shutdown_connections 2024-06-03T03:26:29.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.108+0000 7fc8fb000700 1 --2- 172.21.15.117:0/1713893829 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fc8dc0a2ca0 0x7fc8dc0a5160 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:29.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.108+0000 7fc8fb000700 1 --2- 172.21.15.117:0/1713893829 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc8f4104980 0x7fc8f410e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:29.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.108+0000 7fc8fb000700 1 --2- 172.21.15.117:0/1713893829 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8f41026d0 0x7fc8f419b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:29.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.108+0000 7fc8fb000700 1 --2- 172.21.15.117:0/1713893829 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8f4103fc0 0x7fc8f419bb20 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:29.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.108+0000 7fc8fb000700 1 -- 172.21.15.117:0/1713893829 >> 172.21.15.117:0/1713893829 conn(0x7fc8f40fde80 msgr2=0x7fc8f40746c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:26:29.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.109+0000 7fc8fb000700 1 -- 172.21.15.117:0/1713893829 shutdown_connections 2024-06-03T03:26:29.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:29.109+0000 7fc8fb000700 1 -- 172.21.15.117:0/1713893829 wait complete. 2024-06-03T03:26:30.105 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-03T03:26:30.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:26:30 smithi151 ceph-mon[37043]: pgmap v541: 129 pgs: 129 active+clean; 679 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 938 B/s wr, 13 op/s 2024-06-03T03:26:30.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:26:30 smithi117 ceph-mon[31649]: pgmap v541: 129 pgs: 129 active+clean; 679 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 938 B/s wr, 13 op/s 2024-06-03T03:26:30.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:26:30 smithi183 ceph-mon[37326]: pgmap v541: 129 pgs: 129 active+clean; 679 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 938 B/s wr, 13 op/s 2024-06-03T03:26:30.527 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:26:31.063 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:26:31 smithi117 ceph-mon[31649]: pgmap v542: 129 pgs: 129 active+clean; 679 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 10 op/s 2024-06-03T03:26:31.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:26:31 smithi151 ceph-mon[37043]: pgmap v542: 129 pgs: 129 active+clean; 679 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 10 op/s 2024-06-03T03:26:31.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:26:31 smithi183 ceph-mon[37326]: pgmap v542: 129 pgs: 129 active+clean; 679 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 10 op/s 2024-06-03T03:26:33.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.066+0000 7f11ebc00700 1 -- 172.21.15.117:0/1713587600 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f11e406a6e0 msgr2=0x7f11e406ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:33.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.066+0000 7f11ebc00700 1 --2- 172.21.15.117:0/1713587600 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f11e406a6e0 0x7f11e406ab40 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f11d0009990 tx=0x7f11d0009ca0 comp rx=0 tx=0).stop 2024-06-03T03:26:33.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.066+0000 7f11ebc00700 1 -- 172.21.15.117:0/1713587600 shutdown_connections 2024-06-03T03:26:33.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.066+0000 7f11ebc00700 1 --2- 172.21.15.117:0/1713587600 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f11e4104ed0 0x7f11e4105360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:33.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.066+0000 7f11ebc00700 1 --2- 172.21.15.117:0/1713587600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f11e41073c0 0x7f11e406a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:33.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.066+0000 7f11ebc00700 1 --2- 172.21.15.117:0/1713587600 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f11e406a6e0 0x7f11e406ab40 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:33.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.066+0000 7f11ebc00700 1 -- 172.21.15.117:0/1713587600 >> 172.21.15.117:0/1713587600 conn(0x7f11e4078340 msgr2=0x7f11e4078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:26:33.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.066+0000 7f11ebc00700 1 -- 172.21.15.117:0/1713587600 shutdown_connections 2024-06-03T03:26:33.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.066+0000 7f11ebc00700 1 -- 172.21.15.117:0/1713587600 wait complete. 2024-06-03T03:26:33.070 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.068+0000 7f11ebc00700 1 Processor -- start 2024-06-03T03:26:33.070 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.068+0000 7f11ebc00700 1 -- start start 2024-06-03T03:26:33.070 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.069+0000 7f11ebc00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f11e406a6e0 0x7f11e4115790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:33.071 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.069+0000 7f11ebc00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f11e4104ed0 0x7f11e410e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:33.071 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.069+0000 7f11ebc00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f11e41073c0 0x7f11e410ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:33.071 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.069+0000 7f11ebc00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f11e4118dc0 con 0x7f11e406a6e0 2024-06-03T03:26:33.071 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.069+0000 7f11ebc00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f11e4118c40 con 0x7f11e41073c0 2024-06-03T03:26:33.071 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.069+0000 7f11ebc00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f11e4118f40 con 0x7f11e4104ed0 2024-06-03T03:26:33.072 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.070+0000 7f11e9200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f11e406a6e0 0x7f11e4115790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:33.072 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.070+0000 7f11e9c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f11e41073c0 0x7f11e410ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:33.072 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.070+0000 7f11e9c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f11e41073c0 0x7f11e410ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:39692/0 (socket says 172.21.15.117:39692) 2024-06-03T03:26:33.072 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.070+0000 7f11e9c00700 1 -- 172.21.15.117:0/586980065 learned_addr learned my addr 172.21.15.117:0/586980065 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:26:33.072 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.070+0000 7f11e3e00700 1 --2- 172.21.15.117:0/586980065 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f11e4104ed0 0x7f11e410e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:33.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.070+0000 7f11e9c00700 1 -- 172.21.15.117:0/586980065 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f11e4104ed0 msgr2=0x7f11e410e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:33.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.070+0000 7f11e9c00700 1 --2- 172.21.15.117:0/586980065 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f11e4104ed0 0x7f11e410e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:33.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.070+0000 7f11e9c00700 1 -- 172.21.15.117:0/586980065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f11e406a6e0 msgr2=0x7f11e4115790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:33.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.070+0000 7f11e9c00700 1 --2- 172.21.15.117:0/586980065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f11e406a6e0 0x7f11e4115790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:33.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.070+0000 7f11e9c00700 1 -- 172.21.15.117:0/586980065 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11e410f350 con 0x7f11e41073c0 2024-06-03T03:26:33.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.071+0000 7f11e9200700 1 --2- 172.21.15.117:0/586980065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f11e406a6e0 0x7f11e4115790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T03:26:33.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.071+0000 7f11e9c00700 1 --2- 172.21.15.117:0/586980065 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f11e41073c0 0x7f11e410ed60 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f11dc00ef30 tx=0x7f11dc00c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:26:33.074 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.071+0000 7f11e1600700 1 -- 172.21.15.117:0/586980065 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11dc0092e0 con 0x7f11e41073c0 2024-06-03T03:26:33.074 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.071+0000 7f11e1600700 1 -- 172.21.15.117:0/586980065 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11dc004bb0 con 0x7f11e41073c0 2024-06-03T03:26:33.074 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.071+0000 7f11ebc00700 1 -- 172.21.15.117:0/586980065 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11e410f5e0 con 0x7f11e41073c0 2024-06-03T03:26:33.074 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.071+0000 7f11e1600700 1 -- 172.21.15.117:0/586980065 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11dc004280 con 0x7f11e41073c0 2024-06-03T03:26:33.074 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.072+0000 7f11ebc00700 1 -- 172.21.15.117:0/586980065 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11e4113550 con 0x7f11e41073c0 2024-06-03T03:26:33.078 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.076+0000 7f11f65b4b80 1 Processor -- start 2024-06-03T03:26:33.078 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.077+0000 7f11e1600700 1 -- 172.21.15.117:0/586980065 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f11dc01b030 con 0x7f11e41073c0 2024-06-03T03:26:33.079 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.077+0000 7f11f65b4b80 1 -- start start 2024-06-03T03:26:33.079 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.078+0000 7f11f65b4b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a339d32b90 0x55a339d32f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:33.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.078+0000 7f11f65b4b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a339d334b0 0x55a339d2cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:33.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.078+0000 7f11f65b4b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a339d2d240 0x55a339d2d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:33.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.078+0000 7f11f65b4b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a339c3ba10 con 0x55a339d2d240 2024-06-03T03:26:33.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.078+0000 7f11f65b4b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a339c7eb00 con 0x55a339d32b90 2024-06-03T03:26:33.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.078+0000 7f11f65b4b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a339c398f0 con 0x55a339d334b0 2024-06-03T03:26:33.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.078+0000 7f11e9200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a339d334b0 0x55a339d2cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:33.081 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.078+0000 7f11e9200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a339d334b0 0x55a339d2cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:46640/0 (socket says 172.21.15.117:46640) 2024-06-03T03:26:33.081 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.078+0000 7f11e9200700 1 -- 172.21.15.117:0/2266327858 learned_addr learned my addr 172.21.15.117:0/2266327858 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:26:33.081 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.079+0000 7f11e3e00700 1 --2- 172.21.15.117:0/2266327858 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a339d32b90 0x55a339d32f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:33.081 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.079+0000 7f11e9200700 1 -- 172.21.15.117:0/2266327858 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a339d32b90 msgr2=0x55a339d32f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:33.081 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.079+0000 7f11e9200700 1 --2- 172.21.15.117:0/2266327858 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a339d32b90 0x55a339d32f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:33.081 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.079+0000 7f11e9200700 1 -- 172.21.15.117:0/2266327858 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a339d2d240 msgr2=0x55a339d2d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:33.081 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.079+0000 7f11e9200700 1 --2- 172.21.15.117:0/2266327858 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a339d2d240 0x55a339d2d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:33.081 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.079+0000 7f11e9200700 1 -- 172.21.15.117:0/2266327858 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a339c26f90 con 0x55a339d334b0 2024-06-03T03:26:33.082 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.079+0000 7f11e9200700 1 --2- 172.21.15.117:0/2266327858 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a339d334b0 0x55a339d2cd00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f11d8009da0 tx=0x7f11d800f640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1e230770a32a13c server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:26:33.082 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.079+0000 7f11e1600700 1 --2- 172.21.15.117:0/586980065 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f11cc0a2ca0 0x7f11cc0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:33.082 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.080+0000 7f11e1600700 1 -- 172.21.15.117:0/586980065 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f11dc0cef80 con 0x7f11e41073c0 2024-06-03T03:26:33.082 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.080+0000 7f11e3e00700 1 --2- 172.21.15.117:0/586980065 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f11cc0a2ca0 0x7f11cc0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:33.082 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.080+0000 7f11d6000700 1 -- 172.21.15.117:0/2266327858 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d800fe50 con 0x55a339d334b0 2024-06-03T03:26:33.082 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.081+0000 7f11d6000700 1 -- 172.21.15.117:0/2266327858 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11d8011920 con 0x55a339d334b0 2024-06-03T03:26:33.083 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.081+0000 7f11f65b4b80 1 -- 172.21.15.117:0/2266327858 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a339d334b0 msgr2=0x55a339d2cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:33.083 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.081+0000 7f11e3e00700 1 --2- 172.21.15.117:0/586980065 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f11cc0a2ca0 0x7f11cc0a5160 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f11d0005b40 tx=0x7f11d0005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:26:33.084 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.081+0000 7f11f65b4b80 1 --2- 172.21.15.117:0/2266327858 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a339d334b0 0x55a339d2cd00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f11d8009da0 tx=0x7f11d800f640 comp rx=0 tx=0).stop 2024-06-03T03:26:33.084 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.082+0000 7f11f65b4b80 1 -- 172.21.15.117:0/2266327858 shutdown_connections 2024-06-03T03:26:33.084 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.082+0000 7f11f65b4b80 1 --2- 172.21.15.117:0/2266327858 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a339d2d240 0x55a339d2d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:33.084 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.082+0000 7f11f65b4b80 1 --2- 172.21.15.117:0/2266327858 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a339d32b90 0x55a339d32f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:33.084 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.082+0000 7f11f65b4b80 1 --2- 172.21.15.117:0/2266327858 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a339d334b0 0x55a339d2cd00 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:33.084 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.082+0000 7f11f65b4b80 1 -- 172.21.15.117:0/2266327858 >> 172.21.15.117:0/2266327858 conn(0x55a339c59170 msgr2=0x55a339d27ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:26:33.084 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.082+0000 7f11f65b4b80 1 -- 172.21.15.117:0/2266327858 shutdown_connections 2024-06-03T03:26:33.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.082+0000 7f11f65b4b80 1 -- 172.21.15.117:0/2266327858 wait complete. 2024-06-03T03:26:33.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.083+0000 7f11f65b4b80 1 -- start start 2024-06-03T03:26:33.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.083+0000 7f11f65b4b80 1 Processor -- start 2024-06-03T03:26:33.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.084+0000 7f11f65b4b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a339d2d240 0x55a339d391d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:33.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.084+0000 7f11f65b4b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a339d32b90 0x55a339d364d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:33.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.084+0000 7f11f65b4b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a339d334b0 0x55a339d36a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:33.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.084+0000 7f11f65b4b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a339c1af60 con 0x55a339d32b90 2024-06-03T03:26:33.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.084+0000 7f11f65b4b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a339c7aac0 con 0x55a339d2d240 2024-06-03T03:26:33.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.084+0000 7f11f65b4b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a339d39f30 con 0x55a339d334b0 2024-06-03T03:26:33.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.084+0000 7f11f65b4b80 10 client.0 _ll_register_callbacks cb 0x55a339c14380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:26:33.087 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.085+0000 7f11e9200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a339d2d240 0x55a339d391d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:33.087 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.085+0000 7f11e9200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a339d2d240 0x55a339d391d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:39716/0 (socket says 172.21.15.117:39716) 2024-06-03T03:26:33.087 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.085+0000 7f11e9200700 1 -- 172.21.15.117:0/3887026104 learned_addr learned my addr 172.21.15.117:0/3887026104 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:26:33.087 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.085+0000 7f11e3e00700 1 --2- 172.21.15.117:0/3887026104 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a339d32b90 0x55a339d364d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:33.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.085+0000 7f11e9200700 1 -- 172.21.15.117:0/3887026104 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a339d334b0 msgr2=0x55a339d36a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:33.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.085+0000 7f11e9200700 1 --2- 172.21.15.117:0/3887026104 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a339d334b0 0x55a339d36a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:33.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.085+0000 7f11e9200700 1 -- 172.21.15.117:0/3887026104 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a339d32b90 msgr2=0x55a339d364d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:33.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.086+0000 7f11e9200700 1 --2- 172.21.15.117:0/3887026104 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a339d32b90 0x55a339d364d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:33.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.086+0000 7f11e9200700 1 -- 172.21.15.117:0/3887026104 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a339bcd8d0 con 0x55a339d2d240 2024-06-03T03:26:33.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.086+0000 7f11e3e00700 1 --2- 172.21.15.117:0/3887026104 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a339d32b90 0x55a339d364d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T03:26:33.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.086+0000 7f11e9200700 1 --2- 172.21.15.117:0/3887026104 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a339d2d240 0x55a339d391d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f11d8010600 tx=0x7f11d80106e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c2e41e9111b519f server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:26:33.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.086+0000 7f11caa00700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-06-03T03:26:33.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.086+0000 7f11caa00700 1 -- 172.21.15.117:0/3887026104 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d800fe60 con 0x55a339d2d240 2024-06-03T03:26:33.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.086+0000 7f11caa00700 1 -- 172.21.15.117:0/3887026104 <== mon.2 v2:172.21.15.151:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f11d8010c30 con 0x55a339d2d240 2024-06-03T03:26:33.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.087+0000 7f11f65b4b80 1 -- 172.21.15.117:0/3887026104 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a339c8c000 con 0x55a339d2d240 2024-06-03T03:26:33.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.087+0000 7f11caa00700 1 -- 172.21.15.117:0/3887026104 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f11d8010e30 con 0x55a339d2d240 2024-06-03T03:26:33.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.087+0000 7f11f65b4b80 10 client.25448 fetch_fsmap learned FSMap version 31 2024-06-03T03:26:33.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.087+0000 7f11f65b4b80 1 -- 172.21.15.117:0/3887026104 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a339d44770 con 0x55a339d2d240 2024-06-03T03:26:33.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.087+0000 7f11caa00700 1 -- 172.21.15.117:0/3887026104 <== mon.2 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11d8003680 con 0x55a339d2d240 2024-06-03T03:26:33.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.088+0000 7f11caa00700 1 -- 172.21.15.117:0/3887026104 <== mon.2 v2:172.21.15.151:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f11d80114d0 con 0x55a339d2d240 2024-06-03T03:26:33.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.088+0000 7f11f65b4b80 10 client.25448 fetch_fsmap finished waiting for FSMap version 31 2024-06-03T03:26:33.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.088+0000 7f11f65b4b80 10 client.25448 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-06-03T03:26:33.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.088+0000 7f11f65b4b80 20 client.25448 populate_metadata read hostname 'smithi117' 2024-06-03T03:26:33.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.088+0000 7f11f65b4b80 1 --2- 172.21.15.117:0/3887026104 >> [v2:172.21.15.151:6838/3623444616,v1:172.21.15.151:6839/3623444616] conn(0x55a339d46c40 0x55a339d49030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:33.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.088+0000 7f11f65b4b80 4 client.25448 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T03:26:33.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.088+0000 7f11f65b4b80 1 -- 172.21.15.117:0/3887026104 --> [v2:172.21.15.151:6838/3623444616,v1:172.21.15.151:6839/3623444616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a339c8c000 con 0x55a339d46c40 2024-06-03T03:26:33.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.088+0000 7f11e3e00700 1 --2- 172.21.15.117:0/3887026104 >> [v2:172.21.15.151:6838/3623444616,v1:172.21.15.151:6839/3623444616] conn(0x55a339d46c40 0x55a339d49030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:33.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.089+0000 7f11e3e00700 1 --2- 172.21.15.117:0/3887026104 >> [v2:172.21.15.151:6838/3623444616,v1:172.21.15.151:6839/3623444616] conn(0x55a339d46c40 0x55a339d49030 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=afb6a0c43f118615 server_cookie=a3106a86894452d5 in_seq=0 out_seq=0 2024-06-03T03:26:33.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.089+0000 7f11caa00700 10 client.25448 ms_handle_connect on v2:172.21.15.151:6838/3623444616 2024-06-03T03:26:33.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.090+0000 7f11caa00700 1 -- 172.21.15.117:0/3887026104 <== mds.1 v2:172.21.15.151:6838/3623444616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a339c8c000 con 0x55a339d46c40 2024-06-03T03:26:33.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.090+0000 7f11caa00700 10 client.25448 handle_command_reply: tid=0 2024-06-03T03:26:33.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.090+0000 7f11f65b4b80 1 client.25448 shutdown 2024-06-03T03:26:33.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.091+0000 7f11f65b4b80 1 -- 172.21.15.117:0/3887026104 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a339d2d240 msgr2=0x55a339d391d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:33.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.091+0000 7f11f65b4b80 1 --2- 172.21.15.117:0/3887026104 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a339d2d240 0x55a339d391d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f11d8010600 tx=0x7f11d80106e0 comp rx=0 tx=0).stop 2024-06-03T03:26:33.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.091+0000 7f11f65b4b80 1 -- 172.21.15.117:0/3887026104 shutdown_connections 2024-06-03T03:26:33.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.091+0000 7f11f65b4b80 1 --2- 172.21.15.117:0/3887026104 >> [v2:172.21.15.151:6838/3623444616,v1:172.21.15.151:6839/3623444616] conn(0x55a339d46c40 0x55a339d49030 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:33.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.091+0000 7f11f65b4b80 1 --2- 172.21.15.117:0/3887026104 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a339d334b0 0x55a339d36a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:33.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.091+0000 7f11f65b4b80 1 --2- 172.21.15.117:0/3887026104 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a339d2d240 0x55a339d391d0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:33.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.091+0000 7f11caa00700 0 client.25448 ms_handle_reset on v2:172.21.15.151:6838/3623444616 2024-06-03T03:26:33.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.091+0000 7f11f65b4b80 1 --2- 172.21.15.117:0/3887026104 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a339d32b90 0x55a339d364d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:33.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.091+0000 7f11f65b4b80 1 -- 172.21.15.117:0/3887026104 >> 172.21.15.117:0/3887026104 conn(0x55a339c59170 msgr2=0x55a339d2bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:26:33.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.091+0000 7f11f65b4b80 1 -- 172.21.15.117:0/3887026104 shutdown_connections 2024-06-03T03:26:33.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.091+0000 7f11f65b4b80 1 -- 172.21.15.117:0/3887026104 wait complete. 2024-06-03T03:26:33.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.091+0000 7f11f65b4b80 20 client.25448 trim_cache size 0 max 16384 2024-06-03T03:26:33.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.122+0000 7f11f65b4b80 1 Processor -- start 2024-06-03T03:26:33.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.122+0000 7f11f65b4b80 1 -- start start 2024-06-03T03:26:33.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.123+0000 7f11f65b4b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a339d2d240 0x55a339d22130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:33.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.123+0000 7f11f65b4b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a339d32b90 0x55a339d28100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:33.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.123+0000 7f11f65b4b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a339d334b0 0x55a339d34040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:33.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.123+0000 7f11f65b4b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f11e41bd2a0 con 0x55a339d334b0 2024-06-03T03:26:33.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.123+0000 7f11f65b4b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f11e41bede0 con 0x55a339d2d240 2024-06-03T03:26:33.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.123+0000 7f11f65b4b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f11e41bd970 con 0x55a339d32b90 2024-06-03T03:26:33.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.123+0000 7f11e9200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a339d2d240 0x55a339d22130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:33.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.123+0000 7f11e9200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a339d2d240 0x55a339d22130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:39724/0 (socket says 172.21.15.117:39724) 2024-06-03T03:26:33.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.123+0000 7f11e9200700 1 -- 172.21.15.117:0/3063619004 learned_addr learned my addr 172.21.15.117:0/3063619004 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:26:33.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.123+0000 7f11e3e00700 1 --2- 172.21.15.117:0/3063619004 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a339d32b90 0x55a339d28100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:33.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.123+0000 7f11e9c00700 1 --2- 172.21.15.117:0/3063619004 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a339d334b0 0x55a339d34040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:33.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.124+0000 7f11e9200700 1 -- 172.21.15.117:0/3063619004 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a339d32b90 msgr2=0x55a339d28100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:33.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.124+0000 7f11e9200700 1 --2- 172.21.15.117:0/3063619004 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a339d32b90 0x55a339d28100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:33.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.124+0000 7f11e9200700 1 -- 172.21.15.117:0/3063619004 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a339d334b0 msgr2=0x55a339d34040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:33.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.124+0000 7f11e9200700 1 --2- 172.21.15.117:0/3063619004 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a339d334b0 0x55a339d34040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:33.127 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.124+0000 7f11e9200700 1 -- 172.21.15.117:0/3063619004 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a339d362a0 con 0x55a339d2d240 2024-06-03T03:26:33.127 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.124+0000 7f11e9200700 1 --2- 172.21.15.117:0/3063619004 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a339d2d240 0x55a339d22130 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f11d8001eb0 tx=0x7f11d8010fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=deff3d30921b3865 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:26:33.128 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.124+0000 7f11d5600700 1 -- 172.21.15.117:0/3063619004 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d800fe60 con 0x55a339d2d240 2024-06-03T03:26:33.128 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.125+0000 7f11d5600700 1 -- 172.21.15.117:0/3063619004 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11d800fa40 con 0x55a339d2d240 2024-06-03T03:26:33.128 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.125+0000 7f11d5600700 1 -- 172.21.15.117:0/3063619004 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d8003680 con 0x55a339d2d240 2024-06-03T03:26:33.128 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.125+0000 7f11f65b4b80 1 -- 172.21.15.117:0/3063619004 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a339d2d240 msgr2=0x55a339d22130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:33.128 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.125+0000 7f11f65b4b80 1 --2- 172.21.15.117:0/3063619004 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a339d2d240 0x55a339d22130 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f11d8001eb0 tx=0x7f11d8010fd0 comp rx=0 tx=0).stop 2024-06-03T03:26:33.128 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.126+0000 7f11f65b4b80 1 -- 172.21.15.117:0/3063619004 shutdown_connections 2024-06-03T03:26:33.128 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.126+0000 7f11f65b4b80 1 --2- 172.21.15.117:0/3063619004 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a339d334b0 0x55a339d34040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:33.128 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.126+0000 7f11f65b4b80 1 --2- 172.21.15.117:0/3063619004 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a339d2d240 0x55a339d22130 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:33.128 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.126+0000 7f11f65b4b80 1 --2- 172.21.15.117:0/3063619004 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a339d32b90 0x55a339d28100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:33.128 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.126+0000 7f11f65b4b80 1 -- 172.21.15.117:0/3063619004 >> 172.21.15.117:0/3063619004 conn(0x55a339c59170 msgr2=0x55a339d33930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:26:33.129 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.126+0000 7f11f65b4b80 1 -- 172.21.15.117:0/3063619004 shutdown_connections 2024-06-03T03:26:33.129 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.126+0000 7f11f65b4b80 1 -- 172.21.15.117:0/3063619004 wait complete. 2024-06-03T03:26:33.129 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.126+0000 7f11f65b4b80 1 -- start start 2024-06-03T03:26:33.129 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.127+0000 7f11f65b4b80 1 Processor -- start 2024-06-03T03:26:33.130 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.127+0000 7f11f65b4b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a339d2d240 0x55a339d3f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:33.130 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.127+0000 7f11f65b4b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a339d32b90 0x55a339d3f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:33.130 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.127+0000 7f11f65b4b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a339d334b0 0x55a339d4d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:33.130 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.127+0000 7f11f65b4b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f11e41be710 con 0x55a339d2d240 2024-06-03T03:26:33.130 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.127+0000 7f11f65b4b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f11e41be040 con 0x55a339d32b90 2024-06-03T03:26:33.130 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.127+0000 7f11f65b4b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f11e41bb240 con 0x55a339d334b0 2024-06-03T03:26:33.130 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.127+0000 7f11f65b4b80 10 client.0 _ll_register_callbacks cb 0x55a339c14380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:26:33.130 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.128+0000 7f11e9200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a339d2d240 0x55a339d3f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:33.130 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.128+0000 7f11e9200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a339d2d240 0x55a339d3f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:55228/0 (socket says 172.21.15.117:55228) 2024-06-03T03:26:33.131 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.128+0000 7f11e9200700 1 -- 172.21.15.117:0/667887898 learned_addr learned my addr 172.21.15.117:0/667887898 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:26:33.131 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.128+0000 7f11e9c00700 1 --2- 172.21.15.117:0/667887898 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a339d334b0 0x55a339d4d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:33.131 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.128+0000 7f11e9200700 1 -- 172.21.15.117:0/667887898 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a339d334b0 msgr2=0x55a339d4d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:33.131 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.128+0000 7f11e9200700 1 --2- 172.21.15.117:0/667887898 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a339d334b0 0x55a339d4d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:33.131 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.128+0000 7f11e9200700 1 -- 172.21.15.117:0/667887898 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a339d32b90 msgr2=0x55a339d3f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:33.131 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.128+0000 7f11e9200700 1 --2- 172.21.15.117:0/667887898 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a339d32b90 0x55a339d3f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:33.131 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.128+0000 7f11e9200700 1 -- 172.21.15.117:0/667887898 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a339d4e220 con 0x55a339d2d240 2024-06-03T03:26:33.131 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.129+0000 7f11e9200700 1 --2- 172.21.15.117:0/667887898 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a339d2d240 0x55a339d3f450 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f11d8014040 tx=0x7f11d800f640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70c14aa90115cbc1 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:26:33.131 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.129+0000 7f11caa00700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-06-03T03:26:33.132 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.129+0000 7f11caa00700 1 -- 172.21.15.117:0/667887898 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d8010b70 con 0x55a339d2d240 2024-06-03T03:26:33.132 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.129+0000 7f11caa00700 1 -- 172.21.15.117:0/667887898 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f11d8005320 con 0x55a339d2d240 2024-06-03T03:26:33.132 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.129+0000 7f11f65b4b80 1 -- 172.21.15.117:0/667887898 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11d800fa40 con 0x55a339d2d240 2024-06-03T03:26:33.133 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.129+0000 7f11caa00700 1 -- 172.21.15.117:0/667887898 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f11d803f050 con 0x55a339d2d240 2024-06-03T03:26:33.133 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.129+0000 7f11f65b4b80 10 client.15342 fetch_fsmap learned FSMap version 31 2024-06-03T03:26:33.133 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.129+0000 7f11f65b4b80 1 -- 172.21.15.117:0/667887898 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a339db1040 con 0x55a339d2d240 2024-06-03T03:26:33.133 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.131+0000 7f11caa00700 1 -- 172.21.15.117:0/667887898 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11d800fa40 con 0x55a339d2d240 2024-06-03T03:26:33.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.131+0000 7f11caa00700 1 -- 172.21.15.117:0/667887898 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f11d8025420 con 0x55a339d2d240 2024-06-03T03:26:33.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.131+0000 7f11f65b4b80 10 client.15342 fetch_fsmap finished waiting for FSMap version 31 2024-06-03T03:26:33.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.131+0000 7f11f65b4b80 10 client.15342 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-06-03T03:26:33.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.131+0000 7f11f65b4b80 20 client.15342 populate_metadata read hostname 'smithi117' 2024-06-03T03:26:33.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.131+0000 7f11f65b4b80 1 --2- 172.21.15.117:0/667887898 >> [v2:172.21.15.151:6838/3623444616,v1:172.21.15.151:6839/3623444616] conn(0x55a339db15e0 0x55a339db39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:33.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.131+0000 7f11f65b4b80 4 client.15342 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T03:26:33.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.131+0000 7f11f65b4b80 1 -- 172.21.15.117:0/667887898 --> [v2:172.21.15.151:6838/3623444616,v1:172.21.15.151:6839/3623444616] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a339c86580 con 0x55a339db15e0 2024-06-03T03:26:33.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.132+0000 7f11e3e00700 1 --2- 172.21.15.117:0/667887898 >> [v2:172.21.15.151:6838/3623444616,v1:172.21.15.151:6839/3623444616] conn(0x55a339db15e0 0x55a339db39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:33.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.132+0000 7f11e3e00700 1 --2- 172.21.15.117:0/667887898 >> [v2:172.21.15.151:6838/3623444616,v1:172.21.15.151:6839/3623444616] conn(0x55a339db15e0 0x55a339db39f0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1991c67f7cb6fb3c server_cookie=14b72af937f06f86 in_seq=0 out_seq=0 2024-06-03T03:26:33.138 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.132+0000 7f11caa00700 10 client.15342 ms_handle_connect on v2:172.21.15.151:6838/3623444616 2024-06-03T03:26:33.138 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.133+0000 7f11caa00700 1 -- 172.21.15.117:0/667887898 <== mds.1 v2:172.21.15.151:6838/3623444616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a339c86580 con 0x55a339db15e0 2024-06-03T03:26:33.138 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.133+0000 7f11caa00700 10 client.15342 handle_command_reply: tid=0 2024-06-03T03:26:33.138 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.133+0000 7f11f65b4b80 1 client.15342 shutdown 2024-06-03T03:26:33.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.133+0000 7f11f65b4b80 1 -- 172.21.15.117:0/667887898 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a339d2d240 msgr2=0x55a339d3f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:33.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.133+0000 7f11f65b4b80 1 --2- 172.21.15.117:0/667887898 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a339d2d240 0x55a339d3f450 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f11d8014040 tx=0x7f11d800f640 comp rx=0 tx=0).stop 2024-06-03T03:26:33.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.133+0000 7f11f65b4b80 1 -- 172.21.15.117:0/667887898 shutdown_connections 2024-06-03T03:26:33.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.133+0000 7f11f65b4b80 1 --2- 172.21.15.117:0/667887898 >> [v2:172.21.15.151:6838/3623444616,v1:172.21.15.151:6839/3623444616] conn(0x55a339db15e0 0x55a339db39f0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:33.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.133+0000 7f11f65b4b80 1 --2- 172.21.15.117:0/667887898 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a339d334b0 0x55a339d4d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:33.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.133+0000 7f11f65b4b80 1 --2- 172.21.15.117:0/667887898 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a339d2d240 0x55a339d3f450 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:33.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.133+0000 7f11f65b4b80 1 --2- 172.21.15.117:0/667887898 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a339d32b90 0x55a339d3f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:33.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.133+0000 7f11f65b4b80 1 -- 172.21.15.117:0/667887898 >> 172.21.15.117:0/667887898 conn(0x55a339c59170 msgr2=0x55a339d34c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:26:33.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.134+0000 7f11caa00700 0 client.15342 ms_handle_reset on v2:172.21.15.151:6838/3623444616 2024-06-03T03:26:33.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.134+0000 7f11f65b4b80 1 -- 172.21.15.117:0/667887898 shutdown_connections 2024-06-03T03:26:33.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.134+0000 7f11f65b4b80 1 -- 172.21.15.117:0/667887898 wait complete. 2024-06-03T03:26:33.141 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.134+0000 7f11f65b4b80 20 client.15342 trim_cache size 0 max 16384 2024-06-03T03:26:33.141 INFO:teuthology.orchestra.run.smithi117.stdout:[] 2024-06-03T03:26:33.142 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.135+0000 7f11ebc00700 1 -- 172.21.15.117:0/586980065 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f11cc0a2ca0 msgr2=0x7f11cc0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:33.142 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.135+0000 7f11ebc00700 1 --2- 172.21.15.117:0/586980065 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f11cc0a2ca0 0x7f11cc0a5160 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f11d0005b40 tx=0x7f11d0005a90 comp rx=0 tx=0).stop 2024-06-03T03:26:33.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.135+0000 7f11ebc00700 1 -- 172.21.15.117:0/586980065 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f11e41073c0 msgr2=0x7f11e410ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:33.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.135+0000 7f11ebc00700 1 --2- 172.21.15.117:0/586980065 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f11e41073c0 0x7f11e410ed60 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f11dc00ef30 tx=0x7f11dc00c450 comp rx=0 tx=0).stop 2024-06-03T03:26:33.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.135+0000 7f11ebc00700 1 -- 172.21.15.117:0/586980065 shutdown_connections 2024-06-03T03:26:33.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.135+0000 7f11ebc00700 1 --2- 172.21.15.117:0/586980065 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f11cc0a2ca0 0x7f11cc0a5160 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:33.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.135+0000 7f11ebc00700 1 --2- 172.21.15.117:0/586980065 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f11e41073c0 0x7f11e410ed60 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:33.144 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.135+0000 7f11ebc00700 1 --2- 172.21.15.117:0/586980065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f11e406a6e0 0x7f11e4115790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:33.144 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.135+0000 7f11ebc00700 1 --2- 172.21.15.117:0/586980065 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f11e4104ed0 0x7f11e410e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:33.144 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.135+0000 7f11ebc00700 1 -- 172.21.15.117:0/586980065 >> 172.21.15.117:0/586980065 conn(0x7f11e4078340 msgr2=0x7f11e4103eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:26:33.144 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.136+0000 7f11ebc00700 1 -- 172.21.15.117:0/586980065 shutdown_connections 2024-06-03T03:26:33.144 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:33.136+0000 7f11ebc00700 1 -- 172.21.15.117:0/586980065 wait complete. 2024-06-03T03:26:34.212 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-03T03:26:34.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:26:34 smithi151 ceph-mon[37043]: pgmap v543: 129 pgs: 129 active+clean; 679 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 938 B/s wr, 12 op/s 2024-06-03T03:26:34.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:26:34 smithi117 ceph-mon[31649]: pgmap v543: 129 pgs: 129 active+clean; 679 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 938 B/s wr, 12 op/s 2024-06-03T03:26:34.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:26:34 smithi183 ceph-mon[37326]: pgmap v543: 129 pgs: 129 active+clean; 679 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 938 B/s wr, 12 op/s 2024-06-03T03:26:34.647 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:26:35.190 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.188+0000 7f8bcc800640 1 -- 172.21.15.117:0/2247605562 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8bc4076360 msgr2=0x7f8bc4076780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:35.190 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.188+0000 7f8bcc800640 1 --2- 172.21.15.117:0/2247605562 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8bc4076360 0x7f8bc4076780 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc009ca0 tx=0x7f8bbc02a8e0 comp rx=0 tx=0).stop 2024-06-03T03:26:35.190 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.188+0000 7f8bcc800640 1 -- 172.21.15.117:0/2247605562 shutdown_connections 2024-06-03T03:26:35.190 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.188+0000 7f8bcc800640 1 --2- 172.21.15.117:0/2247605562 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8bc4077f40 0x7f8bc40787e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:35.190 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.188+0000 7f8bcc800640 1 --2- 172.21.15.117:0/2247605562 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8bc40775a0 0x7f8bc4077a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:35.190 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.188+0000 7f8bcc800640 1 --2- 172.21.15.117:0/2247605562 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8bc4076360 0x7f8bc4076780 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:35.190 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.188+0000 7f8bcc800640 1 -- 172.21.15.117:0/2247605562 >> 172.21.15.117:0/2247605562 conn(0x7f8bc40ff1c0 msgr2=0x7f8bc41015e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:26:35.190 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.188+0000 7f8bcc800640 1 -- 172.21.15.117:0/2247605562 shutdown_connections 2024-06-03T03:26:35.190 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.188+0000 7f8bcc800640 1 -- 172.21.15.117:0/2247605562 wait complete. 2024-06-03T03:26:35.191 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.189+0000 7f8bcc800640 1 Processor -- start 2024-06-03T03:26:35.191 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.190+0000 7f8bcc800640 1 -- start start 2024-06-03T03:26:35.191 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.190+0000 7f8bcc800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8bc4076360 0x7f8bc419e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:35.191 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.190+0000 7f8bcc800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8bc40775a0 0x7f8bc419e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:35.192 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.190+0000 7f8bcc800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8bc4077f40 0x7f8bc41a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:35.192 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.190+0000 7f8bcc800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc4113d00 con 0x7f8bc4076360 2024-06-03T03:26:35.192 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.190+0000 7f8bcc800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc4113b80 con 0x7f8bc40775a0 2024-06-03T03:26:35.192 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.190+0000 7f8bcc800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc4113e80 con 0x7f8bc4077f40 2024-06-03T03:26:35.193 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.190+0000 7f8bc9e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8bc4076360 0x7f8bc419e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:35.193 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.190+0000 7f8bc9e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8bc4076360 0x7f8bc419e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:55258/0 (socket says 172.21.15.117:55258) 2024-06-03T03:26:35.193 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.190+0000 7f8bc9e00640 1 -- 172.21.15.117:0/614836408 learned_addr learned my addr 172.21.15.117:0/614836408 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:26:35.193 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.190+0000 7f8bc9400640 1 --2- 172.21.15.117:0/614836408 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8bc40775a0 0x7f8bc419e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:35.193 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.190+0000 7f8bca800640 1 --2- 172.21.15.117:0/614836408 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8bc4077f40 0x7f8bc41a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:35.193 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.191+0000 7f8bca800640 1 -- 172.21.15.117:0/614836408 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8bc40775a0 msgr2=0x7f8bc419e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:35.193 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.191+0000 7f8bca800640 1 --2- 172.21.15.117:0/614836408 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8bc40775a0 0x7f8bc419e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:35.193 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.191+0000 7f8bca800640 1 -- 172.21.15.117:0/614836408 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8bc4076360 msgr2=0x7f8bc419e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:35.194 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.191+0000 7f8bca800640 1 --2- 172.21.15.117:0/614836408 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8bc4076360 0x7f8bc419e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:35.194 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.191+0000 7f8bca800640 1 -- 172.21.15.117:0/614836408 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bc41a5e80 con 0x7f8bc4077f40 2024-06-03T03:26:35.194 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.191+0000 7f8bc9e00640 1 --2- 172.21.15.117:0/614836408 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8bc4076360 0x7f8bc419e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:26:35.194 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.191+0000 7f8bc9400640 1 --2- 172.21.15.117:0/614836408 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8bc40775a0 0x7f8bc419e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:26:35.194 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.191+0000 7f8bca800640 1 --2- 172.21.15.117:0/614836408 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8bc4077f40 0x7f8bc41a5810 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8bc000e970 tx=0x7f8bc000ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:26:35.194 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.192+0000 7f8bbaa00640 1 -- 172.21.15.117:0/614836408 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc000ccc0 con 0x7f8bc4077f40 2024-06-03T03:26:35.194 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.192+0000 7f8bbaa00640 1 -- 172.21.15.117:0/614836408 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bc0005020 con 0x7f8bc4077f40 2024-06-03T03:26:35.195 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.192+0000 7f8bbaa00640 1 -- 172.21.15.117:0/614836408 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc0004a00 con 0x7f8bc4077f40 2024-06-03T03:26:35.195 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.192+0000 7f8bcc800640 1 -- 172.21.15.117:0/614836408 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bc41a6170 con 0x7f8bc4077f40 2024-06-03T03:26:35.195 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.192+0000 7f8bcc800640 1 -- 172.21.15.117:0/614836408 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bc41a6670 con 0x7f8bc4077f40 2024-06-03T03:26:35.198 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.197+0000 7f8bbaa00640 1 -- 172.21.15.117:0/614836408 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f8bc00051d0 con 0x7f8bc4077f40 2024-06-03T03:26:35.198 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.197+0000 7f8bcf570740 1 Processor -- start 2024-06-03T03:26:35.198 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.197+0000 7f8bcf570740 1 -- start start 2024-06-03T03:26:35.199 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.197+0000 7f8bcf570740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e05bf78bd0 0x55e05bf78ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:35.199 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.198+0000 7f8bcf570740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e05bf795c0 0x55e05bf7e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:35.200 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.198+0000 7f8bcf570740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e05bf7e730 0x55e05bf80bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:35.200 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.198+0000 7f8bcf570740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e05bedc420 con 0x55e05bf795c0 2024-06-03T03:26:35.200 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.198+0000 7f8bcf570740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e05be83c50 con 0x55e05bf78bd0 2024-06-03T03:26:35.200 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.198+0000 7f8bcf570740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e05bed83e0 con 0x55e05bf7e730 2024-06-03T03:26:35.200 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.198+0000 7f8bc9400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e05bf78bd0 0x55e05bf78ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:35.200 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.198+0000 7f8bc9400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e05bf78bd0 0x55e05bf78ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:39774/0 (socket says 172.21.15.117:39774) 2024-06-03T03:26:35.200 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.198+0000 7f8bc9400640 1 -- 172.21.15.117:0/1906850586 learned_addr learned my addr 172.21.15.117:0/1906850586 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:26:35.201 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.198+0000 7f8bc9e00640 1 --2- 172.21.15.117:0/1906850586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e05bf795c0 0x55e05bf7e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:35.201 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.198+0000 7f8bc9400640 1 --2- 172.21.15.117:0/1906850586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e05bf7e730 0x55e05bf80bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:35.201 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.199+0000 7f8bbaa00640 1 --2- 172.21.15.117:0/614836408 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f8bb00a2c30 0x7f8bb00a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:35.201 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.199+0000 7f8bbaa00640 1 -- 172.21.15.117:0/614836408 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f8bc00cf370 con 0x7f8bc4077f40 2024-06-03T03:26:35.201 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.199+0000 7f8bc9e00640 1 -- 172.21.15.117:0/1906850586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e05bf7e730 msgr2=0x55e05bf80bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:35.201 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.199+0000 7f8bc9e00640 1 --2- 172.21.15.117:0/1906850586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e05bf7e730 0x55e05bf80bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:35.201 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.199+0000 7f8bc9e00640 1 -- 172.21.15.117:0/1906850586 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e05bf78bd0 msgr2=0x55e05bf78ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:35.201 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.199+0000 7f8bc9e00640 1 --2- 172.21.15.117:0/1906850586 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e05bf78bd0 0x55e05bf78ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:35.201 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.199+0000 7f8bc9e00640 1 -- 172.21.15.117:0/1906850586 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e05bbf0210 con 0x55e05bf795c0 2024-06-03T03:26:35.202 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.199+0000 7f8bc9400640 1 --2- 172.21.15.117:0/1906850586 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e05bf78bd0 0x55e05bf78ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:26:35.202 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.200+0000 7f8bc9e00640 1 --2- 172.21.15.117:0/1906850586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e05bf795c0 0x55e05bf7e0c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc006010 tx=0x7f8bbc005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fb7cc10f88f7b2a server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:26:35.202 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.200+0000 7f8baf000640 1 -- 172.21.15.117:0/1906850586 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc031d60 con 0x55e05bf795c0 2024-06-03T03:26:35.202 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.200+0000 7f8baf000640 1 -- 172.21.15.117:0/1906850586 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bbc036030 con 0x55e05bf795c0 2024-06-03T03:26:35.203 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.200+0000 7f8baf000640 1 -- 172.21.15.117:0/1906850586 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc0305e0 con 0x55e05bf795c0 2024-06-03T03:26:35.203 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.201+0000 7f8bcf570740 1 -- 172.21.15.117:0/1906850586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e05bf795c0 msgr2=0x55e05bf7e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:35.203 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.201+0000 7f8bcf570740 1 --2- 172.21.15.117:0/1906850586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e05bf795c0 0x55e05bf7e0c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc006010 tx=0x7f8bbc005ef0 comp rx=0 tx=0).stop 2024-06-03T03:26:35.204 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.203+0000 7f8bca800640 1 --2- 172.21.15.117:0/614836408 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f8bb00a2c30 0x7f8bb00a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:35.205 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.203+0000 7f8bca800640 1 --2- 172.21.15.117:0/614836408 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f8bb00a2c30 0x7f8bb00a50f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f8bc0098210 tx=0x7f8bc0005750 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:26:35.205 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.204+0000 7f8bcf570740 1 -- 172.21.15.117:0/1906850586 shutdown_connections 2024-06-03T03:26:35.206 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.204+0000 7f8bcf570740 1 --2- 172.21.15.117:0/1906850586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e05bf7e730 0x55e05bf80bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:35.206 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.204+0000 7f8bcf570740 1 --2- 172.21.15.117:0/1906850586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e05bf795c0 0x55e05bf7e0c0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:35.206 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.204+0000 7f8bcf570740 1 --2- 172.21.15.117:0/1906850586 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e05bf78bd0 0x55e05bf78ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:35.206 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.204+0000 7f8bcf570740 1 -- 172.21.15.117:0/1906850586 >> 172.21.15.117:0/1906850586 conn(0x55e05bf40f80 msgr2=0x55e05bf74650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:26:35.206 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.204+0000 7f8bcf570740 1 -- 172.21.15.117:0/1906850586 shutdown_connections 2024-06-03T03:26:35.207 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.204+0000 7f8bcf570740 1 -- 172.21.15.117:0/1906850586 wait complete. 2024-06-03T03:26:35.207 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.204+0000 7f8bcf570740 1 -- start start 2024-06-03T03:26:35.207 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.205+0000 7f8bcf570740 1 Processor -- start 2024-06-03T03:26:35.207 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.205+0000 7f8bcf570740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e05bf78bd0 0x55e05bf82f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:35.207 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.205+0000 7f8bcf570740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e05bf795c0 0x55e05bf83440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:35.207 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.205+0000 7f8bcf570740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e05bf7e730 0x55e05bf8a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:35.208 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.205+0000 7f8bcf570740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e05bf81680 con 0x55e05bf78bd0 2024-06-03T03:26:35.208 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.205+0000 7f8bcf570740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e05be970b0 con 0x55e05bf795c0 2024-06-03T03:26:35.208 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.205+0000 7f8bcf570740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e05bf81800 con 0x55e05bf7e730 2024-06-03T03:26:35.208 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.205+0000 7f8bcf570740 10 client.0 _ll_register_callbacks cb 0x55e05be82f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:26:35.208 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.206+0000 7f8bc9400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e05bf78bd0 0x55e05bf82f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:35.208 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.206+0000 7f8bc9400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e05bf78bd0 0x55e05bf82f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:55282/0 (socket says 172.21.15.117:55282) 2024-06-03T03:26:35.208 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.206+0000 7f8bc9400640 1 -- 172.21.15.117:0/3509613865 learned_addr learned my addr 172.21.15.117:0/3509613865 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:26:35.208 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.206+0000 7f8bc9e00640 1 --2- 172.21.15.117:0/3509613865 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e05bf795c0 0x55e05bf83440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:35.208 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.206+0000 7f8bc9400640 1 --2- 172.21.15.117:0/3509613865 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e05bf7e730 0x55e05bf8a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:35.209 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.206+0000 7f8bc9400640 1 -- 172.21.15.117:0/3509613865 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e05bf7e730 msgr2=0x55e05bf8a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:35.209 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.206+0000 7f8bc9400640 1 --2- 172.21.15.117:0/3509613865 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e05bf7e730 0x55e05bf8a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:35.210 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.206+0000 7f8bc9400640 1 -- 172.21.15.117:0/3509613865 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e05bf795c0 msgr2=0x55e05bf83440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:35.210 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.206+0000 7f8bc9400640 1 --2- 172.21.15.117:0/3509613865 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e05bf795c0 0x55e05bf83440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:35.210 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.206+0000 7f8bc9400640 1 -- 172.21.15.117:0/3509613865 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e05bcbc620 con 0x55e05bf78bd0 2024-06-03T03:26:35.210 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.207+0000 7f8bc9400640 1 --2- 172.21.15.117:0/3509613865 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e05bf78bd0 0x55e05bf82f00 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8bb4004240 tx=0x7f8bb4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77ecc35652677dc3 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:26:35.210 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.207+0000 7f8ba3400640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-06-03T03:26:35.210 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.207+0000 7f8ba3400640 1 -- 172.21.15.117:0/3509613865 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb4011950 con 0x55e05bf78bd0 2024-06-03T03:26:35.210 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.207+0000 7f8ba3400640 1 -- 172.21.15.117:0/3509613865 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f8bb4009040 con 0x55e05bf78bd0 2024-06-03T03:26:35.210 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.208+0000 7f8bcf570740 1 -- 172.21.15.117:0/3509613865 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bbc036030 con 0x55e05bf78bd0 2024-06-03T03:26:35.210 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.208+0000 7f8ba3400640 1 -- 172.21.15.117:0/3509613865 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8bb402a050 con 0x55e05bf78bd0 2024-06-03T03:26:35.210 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.208+0000 7f8bcf570740 10 client.15354 fetch_fsmap learned FSMap version 31 2024-06-03T03:26:35.210 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.208+0000 7f8bcf570740 1 -- 172.21.15.117:0/3509613865 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55e05bf8d860 con 0x55e05bf78bd0 2024-06-03T03:26:35.211 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.208+0000 7f8ba3400640 1 -- 172.21.15.117:0/3509613865 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bbc036030 con 0x55e05bf78bd0 2024-06-03T03:26:35.211 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.209+0000 7f8ba3400640 1 -- 172.21.15.117:0/3509613865 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f8bb40047a0 con 0x55e05bf78bd0 2024-06-03T03:26:35.212 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.209+0000 7f8bcf570740 10 client.15354 fetch_fsmap finished waiting for FSMap version 31 2024-06-03T03:26:35.212 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.209+0000 7f8bcf570740 10 client.15354 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-03T03:26:35.212 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.209+0000 7f8bcf570740 20 client.15354 populate_metadata read hostname 'smithi117' 2024-06-03T03:26:35.212 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.209+0000 7f8bcf570740 1 --2- 172.21.15.117:0/3509613865 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55e05bf8ef80 0x55e05bf91390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:35.212 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.209+0000 7f8bcf570740 4 client.15354 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T03:26:35.212 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.209+0000 7f8bcf570740 1 -- 172.21.15.117:0/3509613865 --> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e05bee9b50 con 0x55e05bf8ef80 2024-06-03T03:26:35.212 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.210+0000 7f8bc9e00640 1 --2- 172.21.15.117:0/3509613865 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55e05bf8ef80 0x55e05bf91390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:35.212 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.210+0000 7f8bc9e00640 1 --2- 172.21.15.117:0/3509613865 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55e05bf8ef80 0x55e05bf91390 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfaeb0bb0cf27ea2 server_cookie=59d625020cd81b53 in_seq=0 out_seq=0 2024-06-03T03:26:35.212 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.210+0000 7f8ba3400640 10 client.15354 ms_handle_connect on v2:172.21.15.183:6838/1439615083 2024-06-03T03:26:35.214 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.211+0000 7f8ba3400640 1 -- 172.21.15.117:0/3509613865 <== mds.0 v2:172.21.15.183:6838/1439615083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e05bee9b50 con 0x55e05bf8ef80 2024-06-03T03:26:35.214 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.211+0000 7f8ba3400640 10 client.15354 handle_command_reply: tid=0 2024-06-03T03:26:35.214 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.211+0000 7f8bcf570740 1 client.15354 shutdown 2024-06-03T03:26:35.215 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.211+0000 7f8bcf570740 1 -- 172.21.15.117:0/3509613865 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e05bf78bd0 msgr2=0x55e05bf82f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:35.215 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.211+0000 7f8bcf570740 1 --2- 172.21.15.117:0/3509613865 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e05bf78bd0 0x55e05bf82f00 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8bb4004240 tx=0x7f8bb4004730 comp rx=0 tx=0).stop 2024-06-03T03:26:35.215 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.212+0000 7f8bcf570740 1 -- 172.21.15.117:0/3509613865 shutdown_connections 2024-06-03T03:26:35.215 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.212+0000 7f8bcf570740 1 --2- 172.21.15.117:0/3509613865 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55e05bf8ef80 0x55e05bf91390 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:35.215 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.212+0000 7f8bcf570740 1 --2- 172.21.15.117:0/3509613865 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e05bf7e730 0x55e05bf8a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:35.215 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.212+0000 7f8bcf570740 1 --2- 172.21.15.117:0/3509613865 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e05bf795c0 0x55e05bf83440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:35.215 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.212+0000 7f8bcf570740 1 --2- 172.21.15.117:0/3509613865 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e05bf78bd0 0x55e05bf82f00 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:35.215 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.212+0000 7f8bcf570740 1 -- 172.21.15.117:0/3509613865 >> 172.21.15.117:0/3509613865 conn(0x55e05bf40f80 msgr2=0x55e05bf73810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:26:35.216 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.212+0000 7f8ba3400640 0 client.15354 ms_handle_reset on v2:172.21.15.183:6838/1439615083 2024-06-03T03:26:35.216 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.212+0000 7f8bcf570740 1 -- 172.21.15.117:0/3509613865 shutdown_connections 2024-06-03T03:26:35.216 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.212+0000 7f8bcf570740 1 -- 172.21.15.117:0/3509613865 wait complete. 2024-06-03T03:26:35.216 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.212+0000 7f8bcf570740 20 client.15354 trim_cache size 0 max 16384 2024-06-03T03:26:35.244 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.243+0000 7f8bcf570740 1 Processor -- start 2024-06-03T03:26:35.244 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.243+0000 7f8bcf570740 1 -- start start 2024-06-03T03:26:35.245 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.243+0000 7f8bcf570740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e05bf78bd0 0x55e05bf747e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:35.245 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.243+0000 7f8bcf570740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e05bf795c0 0x55e05bf8a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:35.245 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.243+0000 7f8bcf570740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e05bf7e730 0x55e05bf82b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:35.246 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.243+0000 7f8bcf570740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc41b0b90 con 0x55e05bf78bd0 2024-06-03T03:26:35.246 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.243+0000 7f8bcf570740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc41b26d0 con 0x55e05bf795c0 2024-06-03T03:26:35.246 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.243+0000 7f8bcf570740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc41b1260 con 0x55e05bf7e730 2024-06-03T03:26:35.246 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.244+0000 7f8bc9400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e05bf78bd0 0x55e05bf747e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:35.246 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.244+0000 7f8bc9400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e05bf78bd0 0x55e05bf747e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:55296/0 (socket says 172.21.15.117:55296) 2024-06-03T03:26:35.246 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.244+0000 7f8bc9400640 1 -- 172.21.15.117:0/3575573890 learned_addr learned my addr 172.21.15.117:0/3575573890 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:26:35.246 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.244+0000 7f8bc9e00640 1 --2- 172.21.15.117:0/3575573890 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e05bf795c0 0x55e05bf8a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:35.246 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.244+0000 7f8bc9400640 1 --2- 172.21.15.117:0/3575573890 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e05bf7e730 0x55e05bf82b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:35.247 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.244+0000 7f8bc9400640 1 -- 172.21.15.117:0/3575573890 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e05bf7e730 msgr2=0x55e05bf82b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:35.247 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.244+0000 7f8bc9400640 1 --2- 172.21.15.117:0/3575573890 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e05bf7e730 0x55e05bf82b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:35.247 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.244+0000 7f8bc9400640 1 -- 172.21.15.117:0/3575573890 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e05bf795c0 msgr2=0x55e05bf8a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:35.247 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.244+0000 7f8bc9400640 1 --2- 172.21.15.117:0/3575573890 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e05bf795c0 0x55e05bf8a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:35.248 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.244+0000 7f8bc9400640 1 -- 172.21.15.117:0/3575573890 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e05bf75300 con 0x55e05bf78bd0 2024-06-03T03:26:35.248 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.245+0000 7f8bc9400640 1 --2- 172.21.15.117:0/3575573890 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e05bf78bd0 0x55e05bf747e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8bb4000c00 tx=0x7f8bb40114c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5ce9d7a68524ea7 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:26:35.248 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.245+0000 7f8bae600640 1 -- 172.21.15.117:0/3575573890 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb4004540 con 0x55e05bf78bd0 2024-06-03T03:26:35.248 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.245+0000 7f8bae600640 1 -- 172.21.15.117:0/3575573890 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bb4011e30 con 0x55e05bf78bd0 2024-06-03T03:26:35.248 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.245+0000 7f8bae600640 1 -- 172.21.15.117:0/3575573890 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb4005400 con 0x55e05bf78bd0 2024-06-03T03:26:35.248 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.246+0000 7f8bcf570740 1 -- 172.21.15.117:0/3575573890 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e05bf78bd0 msgr2=0x55e05bf747e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:35.248 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.246+0000 7f8bcf570740 1 --2- 172.21.15.117:0/3575573890 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e05bf78bd0 0x55e05bf747e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8bb4000c00 tx=0x7f8bb40114c0 comp rx=0 tx=0).stop 2024-06-03T03:26:35.248 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.246+0000 7f8bcf570740 1 -- 172.21.15.117:0/3575573890 shutdown_connections 2024-06-03T03:26:35.248 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.246+0000 7f8bcf570740 1 --2- 172.21.15.117:0/3575573890 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e05bf7e730 0x55e05bf82b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:35.248 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.246+0000 7f8bcf570740 1 --2- 172.21.15.117:0/3575573890 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e05bf795c0 0x55e05bf8a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:35.248 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.246+0000 7f8bcf570740 1 --2- 172.21.15.117:0/3575573890 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e05bf78bd0 0x55e05bf747e0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:35.248 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.246+0000 7f8bcf570740 1 -- 172.21.15.117:0/3575573890 >> 172.21.15.117:0/3575573890 conn(0x55e05bf40f80 msgr2=0x55e05bf740d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:26:35.248 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.246+0000 7f8bcf570740 1 -- 172.21.15.117:0/3575573890 shutdown_connections 2024-06-03T03:26:35.249 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.246+0000 7f8bcf570740 1 -- 172.21.15.117:0/3575573890 wait complete. 2024-06-03T03:26:35.250 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.247+0000 7f8bcf570740 1 -- start start 2024-06-03T03:26:35.250 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.247+0000 7f8bcf570740 1 Processor -- start 2024-06-03T03:26:35.250 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.247+0000 7f8bcf570740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e05bf78bd0 0x55e05bf87680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:35.250 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.247+0000 7f8bcf570740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e05bf795c0 0x55e05bf87bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:35.250 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.247+0000 7f8bcf570740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e05bf7e730 0x55e05bf88bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:35.250 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.247+0000 7f8bcf570740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc41b2000 con 0x55e05bf7e730 2024-06-03T03:26:35.250 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.247+0000 7f8bcf570740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc41b1930 con 0x55e05bf795c0 2024-06-03T03:26:35.250 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.247+0000 7f8bcf570740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc41aeb70 con 0x55e05bf78bd0 2024-06-03T03:26:35.251 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.248+0000 7f8bcf570740 10 client.0 _ll_register_callbacks cb 0x7f8bb4004060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:26:35.251 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.248+0000 7f8bc9400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e05bf7e730 0x55e05bf88bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:35.251 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.248+0000 7f8bc9400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e05bf7e730 0x55e05bf88bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:55312/0 (socket says 172.21.15.117:55312) 2024-06-03T03:26:35.251 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.248+0000 7f8bc9400640 1 -- 172.21.15.117:0/302270614 learned_addr learned my addr 172.21.15.117:0/302270614 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:26:35.251 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.248+0000 7f8bc9e00640 1 --2- 172.21.15.117:0/302270614 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e05bf795c0 0x55e05bf87bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:35.251 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.248+0000 7f8bc9400640 1 --2- 172.21.15.117:0/302270614 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e05bf78bd0 0x55e05bf87680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:35.251 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.248+0000 7f8bc9e00640 1 -- 172.21.15.117:0/302270614 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e05bf78bd0 msgr2=0x55e05bf87680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:35.251 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.248+0000 7f8bc9e00640 1 --2- 172.21.15.117:0/302270614 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e05bf78bd0 0x55e05bf87680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:35.251 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.248+0000 7f8bc9e00640 1 -- 172.21.15.117:0/302270614 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e05bf7e730 msgr2=0x55e05bf88bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:35.251 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.248+0000 7f8bc9e00640 1 --2- 172.21.15.117:0/302270614 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e05bf7e730 0x55e05bf88bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:35.251 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.248+0000 7f8bc9e00640 1 -- 172.21.15.117:0/302270614 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e05bf891c0 con 0x55e05bf795c0 2024-06-03T03:26:35.251 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.248+0000 7f8bc9400640 1 --2- 172.21.15.117:0/302270614 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e05bf7e730 0x55e05bf88bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:26:35.251 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.249+0000 7f8bc9e00640 1 --2- 172.21.15.117:0/302270614 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e05bf795c0 0x55e05bf87bc0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x55e05bf75500 tx=0x7f8bbc009820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6d770d77908a3b7 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:26:35.252 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.249+0000 7f8ba3400640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-06-03T03:26:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.249+0000 7f8ba3400640 1 -- 172.21.15.117:0/302270614 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc049040 con 0x55e05bf795c0 2024-06-03T03:26:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.249+0000 7f8ba3400640 1 -- 172.21.15.117:0/302270614 <== mon.2 v2:172.21.15.151:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f8bbc031ce0 con 0x55e05bf795c0 2024-06-03T03:26:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.249+0000 7f8bcf570740 1 -- 172.21.15.117:0/302270614 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bb4011e30 con 0x55e05bf795c0 2024-06-03T03:26:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.249+0000 7f8ba3400640 1 -- 172.21.15.117:0/302270614 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8bbc063050 con 0x55e05bf795c0 2024-06-03T03:26:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.250+0000 7f8bcf570740 10 client.25469 fetch_fsmap learned FSMap version 31 2024-06-03T03:26:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.250+0000 7f8bcf570740 1 -- 172.21.15.117:0/302270614 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55e05bf8a540 con 0x55e05bf795c0 2024-06-03T03:26:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.250+0000 7f8ba3400640 1 -- 172.21.15.117:0/302270614 <== mon.2 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bb4011e30 con 0x55e05bf795c0 2024-06-03T03:26:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.250+0000 7f8ba3400640 1 -- 172.21.15.117:0/302270614 <== mon.2 v2:172.21.15.151:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f8bbc042070 con 0x55e05bf795c0 2024-06-03T03:26:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.250+0000 7f8bcf570740 10 client.25469 fetch_fsmap finished waiting for FSMap version 31 2024-06-03T03:26:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.250+0000 7f8bcf570740 10 client.25469 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-03T03:26:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.250+0000 7f8bcf570740 20 client.25469 populate_metadata read hostname 'smithi117' 2024-06-03T03:26:35.254 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.250+0000 7f8bcf570740 1 --2- 172.21.15.117:0/302270614 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55e05bff9da0 0x55e05bffc190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:35.254 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.250+0000 7f8bcf570740 4 client.25469 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-03T03:26:35.254 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.250+0000 7f8bcf570740 1 -- 172.21.15.117:0/302270614 --> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e05bee3ea0 con 0x55e05bff9da0 2024-06-03T03:26:35.254 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.251+0000 7f8bc9400640 1 --2- 172.21.15.117:0/302270614 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55e05bff9da0 0x55e05bffc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:35.254 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.251+0000 7f8bc9400640 1 --2- 172.21.15.117:0/302270614 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55e05bff9da0 0x55e05bffc190 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83f6ef0201b16f7c server_cookie=4121959e03466314 in_seq=0 out_seq=0 2024-06-03T03:26:35.254 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.251+0000 7f8ba3400640 10 client.25469 ms_handle_connect on v2:172.21.15.183:6838/1439615083 2024-06-03T03:26:35.259 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.257+0000 7f8ba3400640 1 -- 172.21.15.117:0/302270614 <== mds.0 v2:172.21.15.183:6838/1439615083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55e05bee3ea0 con 0x55e05bff9da0 2024-06-03T03:26:35.259 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.257+0000 7f8ba3400640 10 client.25469 handle_command_reply: tid=0 2024-06-03T03:26:35.259 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.257+0000 7f8bcf570740 1 client.25469 shutdown 2024-06-03T03:26:35.260 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.257+0000 7f8bcf570740 1 -- 172.21.15.117:0/302270614 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e05bf795c0 msgr2=0x55e05bf87bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:35.260 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.257+0000 7f8bcf570740 1 --2- 172.21.15.117:0/302270614 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e05bf795c0 0x55e05bf87bc0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x55e05bf75500 tx=0x7f8bbc009820 comp rx=0 tx=0).stop 2024-06-03T03:26:35.260 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.257+0000 7f8bcf570740 1 -- 172.21.15.117:0/302270614 shutdown_connections 2024-06-03T03:26:35.260 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.257+0000 7f8bcf570740 1 --2- 172.21.15.117:0/302270614 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55e05bff9da0 0x55e05bffc190 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:35.260 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.257+0000 7f8bcf570740 1 --2- 172.21.15.117:0/302270614 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e05bf7e730 0x55e05bf88bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:35.260 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.257+0000 7f8bcf570740 1 --2- 172.21.15.117:0/302270614 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e05bf795c0 0x55e05bf87bc0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:35.260 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.257+0000 7f8bcf570740 1 --2- 172.21.15.117:0/302270614 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e05bf78bd0 0x55e05bf87680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:35.260 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.257+0000 7f8bcf570740 1 -- 172.21.15.117:0/302270614 >> 172.21.15.117:0/302270614 conn(0x55e05bf40f80 msgr2=0x55e05bf77620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:26:35.260 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.257+0000 7f8ba3400640 0 client.25469 ms_handle_reset on v2:172.21.15.183:6838/1439615083 2024-06-03T03:26:35.261 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.258+0000 7f8bcf570740 1 -- 172.21.15.117:0/302270614 shutdown_connections 2024-06-03T03:26:35.261 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.258+0000 7f8bcf570740 1 -- 172.21.15.117:0/302270614 wait complete. 2024-06-03T03:26:35.261 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.258+0000 7f8bcf570740 20 client.25469 trim_cache size 0 max 16384 2024-06-03T03:26:35.261 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:{ 2024-06-03T03:26:35.261 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "return_code": 0 2024-06-03T03:26:35.261 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:} 2024-06-03T03:26:35.261 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.259+0000 7f8bcc800640 1 -- 172.21.15.117:0/614836408 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f8bb00a2c30 msgr2=0x7f8bb00a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:35.262 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.259+0000 7f8bcc800640 1 --2- 172.21.15.117:0/614836408 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f8bb00a2c30 0x7f8bb00a50f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f8bc0098210 tx=0x7f8bc0005750 comp rx=0 tx=0).stop 2024-06-03T03:26:35.262 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.259+0000 7f8bcc800640 1 -- 172.21.15.117:0/614836408 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8bc4077f40 msgr2=0x7f8bc41a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:35.262 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.259+0000 7f8bcc800640 1 --2- 172.21.15.117:0/614836408 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8bc4077f40 0x7f8bc41a5810 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8bc000e970 tx=0x7f8bc000ee30 comp rx=0 tx=0).stop 2024-06-03T03:26:35.262 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.260+0000 7f8bcc800640 1 -- 172.21.15.117:0/614836408 shutdown_connections 2024-06-03T03:26:35.262 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.260+0000 7f8bcc800640 1 --2- 172.21.15.117:0/614836408 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8bc4077f40 0x7f8bc41a5810 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:35.262 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.260+0000 7f8bcc800640 1 --2- 172.21.15.117:0/614836408 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f8bb00a2c30 0x7f8bb00a50f0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:35.262 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.260+0000 7f8bcc800640 1 --2- 172.21.15.117:0/614836408 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8bc40775a0 0x7f8bc419e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:35.262 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.260+0000 7f8bcc800640 1 --2- 172.21.15.117:0/614836408 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8bc4076360 0x7f8bc419e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:35.262 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.260+0000 7f8bcc800640 1 -- 172.21.15.117:0/614836408 >> 172.21.15.117:0/614836408 conn(0x7f8bc40ff1c0 msgr2=0x7f8bc4100cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:26:35.263 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.262+0000 7f8bcc800640 1 -- 172.21.15.117:0/614836408 shutdown_connections 2024-06-03T03:26:35.263 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:35.262+0000 7f8bcc800640 1 -- 172.21.15.117:0/614836408 wait complete. 2024-06-03T03:26:36.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:26:36 smithi151 ceph-mon[37043]: pgmap v544: 129 pgs: 129 active+clean; 679 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 682 B/s wr, 10 op/s 2024-06-03T03:26:36.418 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:26:36 smithi117 ceph-mon[31649]: pgmap v544: 129 pgs: 129 active+clean; 679 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 682 B/s wr, 10 op/s 2024-06-03T03:26:36.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:26:36 smithi183 ceph-mon[37326]: pgmap v544: 129 pgs: 129 active+clean; 679 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 682 B/s wr, 10 op/s 2024-06-03T03:26:37.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.003+0000 7f2a56800700 1 -- 172.21.15.117:0/939169586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a50108500 msgr2=0x7f2a501088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:37.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.003+0000 7f2a56800700 1 --2- 172.21.15.117:0/939169586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a50108500 0x7f2a501088e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2a400098f0 tx=0x7f2a40009c00 comp rx=0 tx=0).stop 2024-06-03T03:26:37.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.003+0000 7f2a56800700 1 -- 172.21.15.117:0/939169586 shutdown_connections 2024-06-03T03:26:37.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.003+0000 7f2a56800700 1 --2- 172.21.15.117:0/939169586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a5006ab30 0x7f2a5010f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:37.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.003+0000 7f2a56800700 1 --2- 172.21.15.117:0/939169586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a50108500 0x7f2a501088e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:37.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.003+0000 7f2a56800700 1 --2- 172.21.15.117:0/939169586 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2a5006a170 0x7f2a5006a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:37.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.003+0000 7f2a56800700 1 -- 172.21.15.117:0/939169586 >> 172.21.15.117:0/939169586 conn(0x7f2a50078340 msgr2=0x7f2a50078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:26:37.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.003+0000 7f2a56800700 1 -- 172.21.15.117:0/939169586 shutdown_connections 2024-06-03T03:26:37.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.003+0000 7f2a56800700 1 -- 172.21.15.117:0/939169586 wait complete. 2024-06-03T03:26:37.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.004+0000 7f2a56800700 1 Processor -- start 2024-06-03T03:26:37.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.004+0000 7f2a56800700 1 -- start start 2024-06-03T03:26:37.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.005+0000 7f2a56800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a5006a170 0x7f2a5019ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:37.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.005+0000 7f2a56800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2a5006ab30 0x7f2a5019b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:37.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.005+0000 7f2a56800700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a50108500 0x7f2a5019f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:37.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.005+0000 7f2a56800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2a50112760 con 0x7f2a5006a170 2024-06-03T03:26:37.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.005+0000 7f2a56800700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2a501125e0 con 0x7f2a5006ab30 2024-06-03T03:26:37.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.005+0000 7f2a56800700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2a501128e0 con 0x7f2a50108500 2024-06-03T03:26:37.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.005+0000 7f2a4f400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a5006a170 0x7f2a5019ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:37.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.005+0000 7f2a4f400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a5006a170 0x7f2a5019ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56960/0 (socket says 172.21.15.117:56960) 2024-06-03T03:26:37.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.005+0000 7f2a4f400700 1 -- 172.21.15.117:0/3870703510 learned_addr learned my addr 172.21.15.117:0/3870703510 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:26:37.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.005+0000 7f2a4fe00700 1 --2- 172.21.15.117:0/3870703510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a50108500 0x7f2a5019f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:37.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.006+0000 7f2a4ea00700 1 --2- 172.21.15.117:0/3870703510 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2a5006ab30 0x7f2a5019b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:37.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.006+0000 7f2a4f400700 1 -- 172.21.15.117:0/3870703510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a50108500 msgr2=0x7f2a5019f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:37.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.006+0000 7f2a4f400700 1 --2- 172.21.15.117:0/3870703510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a50108500 0x7f2a5019f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:37.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.006+0000 7f2a4f400700 1 -- 172.21.15.117:0/3870703510 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2a5006ab30 msgr2=0x7f2a5019b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:37.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.006+0000 7f2a4f400700 1 --2- 172.21.15.117:0/3870703510 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2a5006ab30 0x7f2a5019b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:37.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.006+0000 7f2a4f400700 1 -- 172.21.15.117:0/3870703510 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a5019fe80 con 0x7f2a5006a170 2024-06-03T03:26:37.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.006+0000 7f2a4fe00700 1 --2- 172.21.15.117:0/3870703510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a50108500 0x7f2a5019f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:26:37.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.007+0000 7f2a4f400700 1 --2- 172.21.15.117:0/3870703510 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a5006a170 0x7f2a5019ae70 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2a40009230 tx=0x7f2a4000fa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:26:37.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.007+0000 7f2a47e00700 1 -- 172.21.15.117:0/3870703510 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a40005180 con 0x7f2a5006a170 2024-06-03T03:26:37.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.007+0000 7f2a47e00700 1 -- 172.21.15.117:0/3870703510 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a4000b560 con 0x7f2a5006a170 2024-06-03T03:26:37.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.007+0000 7f2a47e00700 1 -- 172.21.15.117:0/3870703510 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a40003a70 con 0x7f2a5006a170 2024-06-03T03:26:37.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.008+0000 7f2a56800700 1 -- 172.21.15.117:0/3870703510 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a501a0110 con 0x7f2a5006a170 2024-06-03T03:26:37.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.008+0000 7f2a56800700 1 -- 172.21.15.117:0/3870703510 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a501a0510 con 0x7f2a5006a170 2024-06-03T03:26:37.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.013+0000 7f2a47e00700 1 -- 172.21.15.117:0/3870703510 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f2a40005320 con 0x7f2a5006a170 2024-06-03T03:26:37.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.015+0000 7f2a47e00700 1 --2- 172.21.15.117:0/3870703510 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f2a340a2ca0 0x7f2a340a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:37.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.015+0000 7f2a61119b80 1 Processor -- start 2024-06-03T03:26:37.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.015+0000 7f2a47e00700 1 -- 172.21.15.117:0/3870703510 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f2a400d04d0 con 0x7f2a5006a170 2024-06-03T03:26:37.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.015+0000 7f2a61119b80 1 -- start start 2024-06-03T03:26:37.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.015+0000 7f2a4ea00700 1 --2- 172.21.15.117:0/3870703510 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f2a340a2ca0 0x7f2a340a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:37.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.016+0000 7f2a61119b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a580651e40 0x55a5806d5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:37.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.016+0000 7f2a61119b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a5806d55b0 0x55a5806d5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:37.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.016+0000 7f2a61119b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a5806e0300 0x55a5806e26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:37.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.016+0000 7f2a61119b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a5805e4a10 con 0x55a5806d55b0 2024-06-03T03:26:37.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.016+0000 7f2a61119b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a580627b00 con 0x55a5806e0300 2024-06-03T03:26:37.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.016+0000 7f2a61119b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a5805e28f0 con 0x55a580651e40 2024-06-03T03:26:37.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.017+0000 7f2a4ea00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a5806d55b0 0x55a5806d5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:37.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.017+0000 7f2a4fe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a5806e0300 0x55a5806e26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:37.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.017+0000 7f2a4fe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a5806e0300 0x55a5806e26f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:39190/0 (socket says 172.21.15.117:39190) 2024-06-03T03:26:37.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.017+0000 7f2a4fe00700 1 -- 172.21.15.117:0/1679411280 learned_addr learned my addr 172.21.15.117:0/1679411280 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:26:37.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.017+0000 7f2a4f400700 1 --2- 172.21.15.117:0/1679411280 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a580651e40 0x55a5806d5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:37.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.017+0000 7f2a4fe00700 1 -- 172.21.15.117:0/1679411280 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a580651e40 msgr2=0x55a5806d5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:37.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.017+0000 7f2a4fe00700 1 --2- 172.21.15.117:0/1679411280 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a580651e40 0x55a5806d5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:37.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.017+0000 7f2a4fe00700 1 -- 172.21.15.117:0/1679411280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a5806d55b0 msgr2=0x55a5806d5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:37.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.017+0000 7f2a4fe00700 1 --2- 172.21.15.117:0/1679411280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a5806d55b0 0x55a5806d5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:37.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.018+0000 7f2a4fe00700 1 -- 172.21.15.117:0/1679411280 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5805cff90 con 0x55a5806e0300 2024-06-03T03:26:37.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.018+0000 7f2a4ea00700 1 --2- 172.21.15.117:0/1679411280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a5806d55b0 0x55a5806d5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T03:26:37.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.018+0000 7f2a4ea00700 1 --2- 172.21.15.117:0/3870703510 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f2a340a2ca0 0x7f2a340a5160 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c009680 tx=0x7f2a3c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:26:37.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.018+0000 7f2a4fe00700 1 --2- 172.21.15.117:0/1679411280 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a5806e0300 0x55a5806e26f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2a4800e980 tx=0x7f2a480099e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebd7b8a50fdb92bf server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:26:37.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.018+0000 7f2a44c00700 1 -- 172.21.15.117:0/1679411280 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a48019070 con 0x55a5806e0300 2024-06-03T03:26:37.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.018+0000 7f2a44c00700 1 -- 172.21.15.117:0/1679411280 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a48007aa0 con 0x55a5806e0300 2024-06-03T03:26:37.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.018+0000 7f2a44c00700 1 -- 172.21.15.117:0/1679411280 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a48015b30 con 0x55a5806e0300 2024-06-03T03:26:37.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.019+0000 7f2a61119b80 1 -- 172.21.15.117:0/1679411280 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a5806e0300 msgr2=0x55a5806e26f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:37.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.019+0000 7f2a61119b80 1 --2- 172.21.15.117:0/1679411280 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a5806e0300 0x55a5806e26f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2a4800e980 tx=0x7f2a480099e0 comp rx=0 tx=0).stop 2024-06-03T03:26:37.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.020+0000 7f2a61119b80 1 -- 172.21.15.117:0/1679411280 shutdown_connections 2024-06-03T03:26:37.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.020+0000 7f2a61119b80 1 --2- 172.21.15.117:0/1679411280 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a5806e0300 0x55a5806e26f0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:37.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.020+0000 7f2a61119b80 1 --2- 172.21.15.117:0/1679411280 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a580651e40 0x55a5806d5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:37.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.020+0000 7f2a61119b80 1 --2- 172.21.15.117:0/1679411280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a5806d55b0 0x55a5806d5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:37.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.020+0000 7f2a61119b80 1 -- 172.21.15.117:0/1679411280 >> 172.21.15.117:0/1679411280 conn(0x55a580602170 msgr2=0x55a5806d14b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:26:37.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.020+0000 7f2a61119b80 1 -- 172.21.15.117:0/1679411280 shutdown_connections 2024-06-03T03:26:37.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.020+0000 7f2a61119b80 1 -- 172.21.15.117:0/1679411280 wait complete. 2024-06-03T03:26:37.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.020+0000 7f2a61119b80 1 -- start start 2024-06-03T03:26:37.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.020+0000 7f2a61119b80 1 Processor -- start 2024-06-03T03:26:37.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.021+0000 7f2a61119b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a580651e40 0x55a5806df9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:37.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.021+0000 7f2a61119b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a5806d55b0 0x55a5806dccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:37.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.021+0000 7f2a61119b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a5806e0300 0x55a5806dd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:37.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.022+0000 7f2a61119b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a5805c3f60 con 0x55a5806e0300 2024-06-03T03:26:37.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.022+0000 7f2a61119b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a580623ac0 con 0x55a5806d55b0 2024-06-03T03:26:37.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.022+0000 7f2a61119b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a5806eb7d0 con 0x55a580651e40 2024-06-03T03:26:37.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.022+0000 7f2a4fe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a5806e0300 0x55a5806dd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:37.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.022+0000 7f2a4fe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a5806e0300 0x55a5806dd200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56982/0 (socket says 172.21.15.117:56982) 2024-06-03T03:26:37.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.022+0000 7f2a4fe00700 1 -- 172.21.15.117:0/2273971560 learned_addr learned my addr 172.21.15.117:0/2273971560 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:26:37.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.022+0000 7f2a4ea00700 1 --2- 172.21.15.117:0/2273971560 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a5806d55b0 0x55a5806dccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:37.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.022+0000 7f2a4f400700 1 --2- 172.21.15.117:0/2273971560 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a580651e40 0x55a5806df9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:37.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.022+0000 7f2a61119b80 10 client.0 _ll_register_callbacks cb 0x55a5805c1720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:26:37.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.023+0000 7f2a4fe00700 1 -- 172.21.15.117:0/2273971560 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a580651e40 msgr2=0x55a5806df9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:37.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.023+0000 7f2a4fe00700 1 --2- 172.21.15.117:0/2273971560 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a580651e40 0x55a5806df9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:37.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.023+0000 7f2a4fe00700 1 -- 172.21.15.117:0/2273971560 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a5806d55b0 msgr2=0x55a5806dccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:37.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.023+0000 7f2a4fe00700 1 --2- 172.21.15.117:0/2273971560 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a5806d55b0 0x55a5806dccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:37.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.023+0000 7f2a4fe00700 1 -- 172.21.15.117:0/2273971560 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5805768d0 con 0x55a5806e0300 2024-06-03T03:26:37.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.023+0000 7f2a4ea00700 1 --2- 172.21.15.117:0/2273971560 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a5806d55b0 0x55a5806dccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T03:26:37.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.023+0000 7f2a4fe00700 1 --2- 172.21.15.117:0/2273971560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a5806e0300 0x55a5806dd200 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2a48015580 tx=0x7f2a48006260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=616c47cf120d5b03 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:26:37.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.023+0000 7f2a39600700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-06-03T03:26:37.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.024+0000 7f2a39600700 1 -- 172.21.15.117:0/2273971560 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a48019040 con 0x55a5806e0300 2024-06-03T03:26:37.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.024+0000 7f2a39600700 1 -- 172.21.15.117:0/2273971560 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f2a480157b0 con 0x55a5806e0300 2024-06-03T03:26:37.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.024+0000 7f2a61119b80 1 -- 172.21.15.117:0/2273971560 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a48007aa0 con 0x55a5806e0300 2024-06-03T03:26:37.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.024+0000 7f2a39600700 1 -- 172.21.15.117:0/2273971560 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f2a48031050 con 0x55a5806e0300 2024-06-03T03:26:37.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.024+0000 7f2a61119b80 10 client.15372 fetch_fsmap learned FSMap version 31 2024-06-03T03:26:37.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.024+0000 7f2a61119b80 1 -- 172.21.15.117:0/2273971560 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a5806f5e50 con 0x55a5806e0300 2024-06-03T03:26:37.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.024+0000 7f2a39600700 1 -- 172.21.15.117:0/2273971560 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a48007aa0 con 0x55a5806e0300 2024-06-03T03:26:37.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.025+0000 7f2a39600700 1 -- 172.21.15.117:0/2273971560 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f2a480159b0 con 0x55a5806e0300 2024-06-03T03:26:37.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.025+0000 7f2a61119b80 10 client.15372 fetch_fsmap finished waiting for FSMap version 31 2024-06-03T03:26:37.030 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.025+0000 7f2a61119b80 10 client.15372 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-03T03:26:37.030 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.025+0000 7f2a61119b80 20 client.15372 populate_metadata read hostname 'smithi117' 2024-06-03T03:26:37.030 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.025+0000 7f2a61119b80 1 --2- 172.21.15.117:0/2273971560 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55a5806f8490 0x55a5806fa880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:37.030 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.025+0000 7f2a61119b80 4 client.15372 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T03:26:37.030 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.025+0000 7f2a61119b80 1 -- 172.21.15.117:0/2273971560 --> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a580635000 con 0x55a5806f8490 2024-06-03T03:26:37.030 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.026+0000 7f2a4f400700 1 --2- 172.21.15.117:0/2273971560 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55a5806f8490 0x55a5806fa880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:37.032 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.027+0000 7f2a39600700 10 client.15372 ms_handle_connect on v2:172.21.15.183:6838/1439615083 2024-06-03T03:26:37.032 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.027+0000 7f2a4f400700 1 --2- 172.21.15.117:0/2273971560 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55a5806f8490 0x55a5806fa880 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e12b6c644f2d691 server_cookie=2c95facea2b2dc39 in_seq=0 out_seq=0 2024-06-03T03:26:37.032 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.028+0000 7f2a39600700 1 -- 172.21.15.117:0/2273971560 <== mds.0 v2:172.21.15.183:6838/1439615083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a580635000 con 0x55a5806f8490 2024-06-03T03:26:37.032 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.028+0000 7f2a39600700 10 client.15372 handle_command_reply: tid=0 2024-06-03T03:26:37.032 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.028+0000 7f2a61119b80 1 client.15372 shutdown 2024-06-03T03:26:37.032 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.028+0000 7f2a61119b80 1 -- 172.21.15.117:0/2273971560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a5806e0300 msgr2=0x55a5806dd200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:37.032 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.028+0000 7f2a61119b80 1 --2- 172.21.15.117:0/2273971560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a5806e0300 0x55a5806dd200 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2a48015580 tx=0x7f2a48006260 comp rx=0 tx=0).stop 2024-06-03T03:26:37.032 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.028+0000 7f2a61119b80 1 -- 172.21.15.117:0/2273971560 shutdown_connections 2024-06-03T03:26:37.032 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.028+0000 7f2a61119b80 1 --2- 172.21.15.117:0/2273971560 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55a5806f8490 0x55a5806fa880 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:37.032 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.028+0000 7f2a61119b80 1 --2- 172.21.15.117:0/2273971560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a5806e0300 0x55a5806dd200 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:37.033 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.028+0000 7f2a61119b80 1 --2- 172.21.15.117:0/2273971560 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a580651e40 0x55a5806df9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:37.033 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.028+0000 7f2a39600700 0 client.15372 ms_handle_reset on v2:172.21.15.183:6838/1439615083 2024-06-03T03:26:37.033 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.028+0000 7f2a61119b80 1 --2- 172.21.15.117:0/2273971560 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a5806d55b0 0x55a5806dccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:37.033 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.028+0000 7f2a61119b80 1 -- 172.21.15.117:0/2273971560 >> 172.21.15.117:0/2273971560 conn(0x55a580602170 msgr2=0x55a5806d0740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:26:37.033 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.029+0000 7f2a61119b80 1 -- 172.21.15.117:0/2273971560 shutdown_connections 2024-06-03T03:26:37.033 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.029+0000 7f2a61119b80 1 -- 172.21.15.117:0/2273971560 wait complete. 2024-06-03T03:26:37.033 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.029+0000 7f2a61119b80 20 client.15372 trim_cache size 0 max 16384 2024-06-03T03:26:37.062 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.060+0000 7f2a61119b80 1 Processor -- start 2024-06-03T03:26:37.062 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.060+0000 7f2a61119b80 1 -- start start 2024-06-03T03:26:37.062 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.061+0000 7f2a61119b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a580651e40 0x55a5806e16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:37.062 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.061+0000 7f2a61119b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a5806d55b0 0x55a5806d0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:37.062 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.061+0000 7f2a61119b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a5806e0300 0x55a5806d14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:37.062 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.061+0000 7f2a61119b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2a501ac0a0 con 0x55a580651e40 2024-06-03T03:26:37.063 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.061+0000 7f2a61119b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2a501adbe0 con 0x55a5806e0300 2024-06-03T03:26:37.063 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.061+0000 7f2a61119b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2a501ac770 con 0x55a5806d55b0 2024-06-03T03:26:37.063 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.061+0000 7f2a4fe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a5806e0300 0x55a5806d14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:37.063 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.061+0000 7f2a4f400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a580651e40 0x55a5806e16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:37.063 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.061+0000 7f2a4fe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a5806e0300 0x55a5806d14f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:39200/0 (socket says 172.21.15.117:39200) 2024-06-03T03:26:37.063 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.061+0000 7f2a4fe00700 1 -- 172.21.15.117:0/3747212274 learned_addr learned my addr 172.21.15.117:0/3747212274 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:26:37.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.061+0000 7f2a4ea00700 1 --2- 172.21.15.117:0/3747212274 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a5806d55b0 0x55a5806d0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:37.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.062+0000 7f2a4f400700 1 -- 172.21.15.117:0/3747212274 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a5806d55b0 msgr2=0x55a5806d0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:37.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.062+0000 7f2a4f400700 1 --2- 172.21.15.117:0/3747212274 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a5806d55b0 0x55a5806d0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:37.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.062+0000 7f2a4f400700 1 -- 172.21.15.117:0/3747212274 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a5806e0300 msgr2=0x55a5806d14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:37.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.062+0000 7f2a4f400700 1 --2- 172.21.15.117:0/3747212274 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a5806e0300 0x55a5806d14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:37.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.062+0000 7f2a4f400700 1 -- 172.21.15.117:0/3747212274 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5806d6220 con 0x55a580651e40 2024-06-03T03:26:37.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.062+0000 7f2a4ea00700 1 --2- 172.21.15.117:0/3747212274 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a5806d55b0 0x55a5806d0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T03:26:37.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.063+0000 7f2a4f400700 1 --2- 172.21.15.117:0/3747212274 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a580651e40 0x55a5806e16f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2a400cdbe0 tx=0x7f2a40009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b19f60d666a284e4 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:26:37.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.063+0000 7f2a3be00700 1 -- 172.21.15.117:0/3747212274 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a400b0e60 con 0x55a580651e40 2024-06-03T03:26:37.065 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.063+0000 7f2a3be00700 1 -- 172.21.15.117:0/3747212274 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a400a0d80 con 0x55a580651e40 2024-06-03T03:26:37.065 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.063+0000 7f2a3be00700 1 -- 172.21.15.117:0/3747212274 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a400d0180 con 0x55a580651e40 2024-06-03T03:26:37.065 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.064+0000 7f2a61119b80 1 -- 172.21.15.117:0/3747212274 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a580651e40 msgr2=0x55a5806e16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:37.065 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.064+0000 7f2a61119b80 1 --2- 172.21.15.117:0/3747212274 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a580651e40 0x55a5806e16f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2a400cdbe0 tx=0x7f2a40009260 comp rx=0 tx=0).stop 2024-06-03T03:26:37.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.065+0000 7f2a61119b80 1 -- 172.21.15.117:0/3747212274 shutdown_connections 2024-06-03T03:26:37.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.065+0000 7f2a61119b80 1 --2- 172.21.15.117:0/3747212274 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a5806e0300 0x55a5806d14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:37.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.065+0000 7f2a61119b80 1 --2- 172.21.15.117:0/3747212274 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a580651e40 0x55a5806e16f0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:37.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.065+0000 7f2a61119b80 1 --2- 172.21.15.117:0/3747212274 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a5806d55b0 0x55a5806d0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:37.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.065+0000 7f2a61119b80 1 -- 172.21.15.117:0/3747212274 >> 172.21.15.117:0/3747212274 conn(0x55a580602170 msgr2=0x55a5806e2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:26:37.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.065+0000 7f2a61119b80 1 -- 172.21.15.117:0/3747212274 shutdown_connections 2024-06-03T03:26:37.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.065+0000 7f2a61119b80 1 -- 172.21.15.117:0/3747212274 wait complete. 2024-06-03T03:26:37.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.066+0000 7f2a61119b80 1 -- start start 2024-06-03T03:26:37.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.066+0000 7f2a61119b80 1 Processor -- start 2024-06-03T03:26:37.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.066+0000 7f2a61119b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a580651e40 0x55a5806e1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:37.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.066+0000 7f2a61119b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a5806d55b0 0x55a5806e1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:37.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.066+0000 7f2a61119b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a5806e0300 0x55a5806efa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:37.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.066+0000 7f2a61119b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2a501ad510 con 0x55a5806d55b0 2024-06-03T03:26:37.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.066+0000 7f2a61119b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2a501ace40 con 0x55a5806e0300 2024-06-03T03:26:37.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.066+0000 7f2a61119b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2a501aa070 con 0x55a580651e40 2024-06-03T03:26:37.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.067+0000 7f2a61119b80 10 client.0 _ll_register_callbacks cb 0x55a5805c1720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:26:37.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.067+0000 7f2a4f400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a580651e40 0x55a5806e1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:37.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.067+0000 7f2a4f400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a580651e40 0x55a5806e1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:35028/0 (socket says 172.21.15.117:35028) 2024-06-03T03:26:37.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.067+0000 7f2a4f400700 1 -- 172.21.15.117:0/2304035019 learned_addr learned my addr 172.21.15.117:0/2304035019 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:26:37.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.067+0000 7f2a4f400700 1 -- 172.21.15.117:0/2304035019 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a5806e0300 msgr2=0x55a5806efa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:37.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.067+0000 7f2a4ea00700 1 --2- 172.21.15.117:0/2304035019 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a5806d55b0 0x55a5806e1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:37.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.067+0000 7f2a4f400700 1 --2- 172.21.15.117:0/2304035019 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a5806e0300 0x55a5806efa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:37.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.067+0000 7f2a4f400700 1 -- 172.21.15.117:0/2304035019 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a5806d55b0 msgr2=0x55a5806e1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:37.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.067+0000 7f2a4f400700 1 --2- 172.21.15.117:0/2304035019 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a5806d55b0 0x55a5806e1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:37.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.068+0000 7f2a4f400700 1 -- 172.21.15.117:0/2304035019 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5806f0020 con 0x55a580651e40 2024-06-03T03:26:37.070 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.068+0000 7f2a4f400700 1 --2- 172.21.15.117:0/2304035019 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a580651e40 0x55a5806e1310 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2a400993a0 tx=0x7f2a40009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f011bf2e7a0c4e1 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:26:37.070 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.068+0000 7f2a39600700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-06-03T03:26:37.070 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.068+0000 7f2a39600700 1 -- 172.21.15.117:0/2304035019 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a400b4070 con 0x55a580651e40 2024-06-03T03:26:37.070 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.068+0000 7f2a61119b80 1 -- 172.21.15.117:0/2304035019 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a400a0d80 con 0x55a580651e40 2024-06-03T03:26:37.070 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.068+0000 7f2a39600700 1 -- 172.21.15.117:0/2304035019 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f2a4000baa0 con 0x55a580651e40 2024-06-03T03:26:37.070 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.069+0000 7f2a39600700 1 -- 172.21.15.117:0/2304035019 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f2a400b0e60 con 0x55a580651e40 2024-06-03T03:26:37.070 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.069+0000 7f2a61119b80 10 client.25516 fetch_fsmap learned FSMap version 31 2024-06-03T03:26:37.070 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.069+0000 7f2a61119b80 1 -- 172.21.15.117:0/2304035019 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a5806f1300 con 0x55a580651e40 2024-06-03T03:26:37.070 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.069+0000 7f2a39600700 1 -- 172.21.15.117:0/2304035019 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a4000b710 con 0x55a580651e40 2024-06-03T03:26:37.071 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.070+0000 7f2a39600700 1 -- 172.21.15.117:0/2304035019 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f2a4009cc30 con 0x55a580651e40 2024-06-03T03:26:37.071 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.070+0000 7f2a61119b80 10 client.25516 fetch_fsmap finished waiting for FSMap version 31 2024-06-03T03:26:37.071 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.070+0000 7f2a61119b80 10 client.25516 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-03T03:26:37.071 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.070+0000 7f2a61119b80 20 client.25516 populate_metadata read hostname 'smithi117' 2024-06-03T03:26:37.071 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.070+0000 7f2a61119b80 1 --2- 172.21.15.117:0/2304035019 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55a580762f40 0x55a580765360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:37.071 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.070+0000 7f2a61119b80 4 client.25516 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T03:26:37.071 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.070+0000 7f2a61119b80 1 -- 172.21.15.117:0/2304035019 --> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f2a400a0d80 con 0x55a580762f40 2024-06-03T03:26:37.072 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.070+0000 7f2a4ea00700 1 --2- 172.21.15.117:0/2304035019 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55a580762f40 0x55a580765360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:37.072 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.071+0000 7f2a4ea00700 1 --2- 172.21.15.117:0/2304035019 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55a580762f40 0x55a580765360 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9f61c6df1b3f1dc server_cookie=8ba9fa9c5a62e18b in_seq=0 out_seq=0 2024-06-03T03:26:37.072 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.071+0000 7f2a39600700 10 client.25516 ms_handle_connect on v2:172.21.15.183:6838/1439615083 2024-06-03T03:26:37.072 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.071+0000 7f2a39600700 1 -- 172.21.15.117:0/2304035019 <== mds.0 v2:172.21.15.183:6838/1439615083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f2a400a0d80 con 0x55a580762f40 2024-06-03T03:26:37.072 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.071+0000 7f2a39600700 10 client.25516 handle_command_reply: tid=0 2024-06-03T03:26:37.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.071+0000 7f2a61119b80 1 client.25516 shutdown 2024-06-03T03:26:37.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.071+0000 7f2a61119b80 1 -- 172.21.15.117:0/2304035019 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a580651e40 msgr2=0x55a5806e1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:37.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.071+0000 7f2a61119b80 1 --2- 172.21.15.117:0/2304035019 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a580651e40 0x55a5806e1310 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2a400993a0 tx=0x7f2a40009c00 comp rx=0 tx=0).stop 2024-06-03T03:26:37.074 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.073+0000 7f2a61119b80 1 -- 172.21.15.117:0/2304035019 shutdown_connections 2024-06-03T03:26:37.075 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.073+0000 7f2a61119b80 1 --2- 172.21.15.117:0/2304035019 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55a580762f40 0x55a580765360 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:37.075 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.073+0000 7f2a61119b80 1 --2- 172.21.15.117:0/2304035019 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a5806e0300 0x55a5806efa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:37.075 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.073+0000 7f2a61119b80 1 --2- 172.21.15.117:0/2304035019 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a580651e40 0x55a5806e1310 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:37.075 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.073+0000 7f2a61119b80 1 --2- 172.21.15.117:0/2304035019 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a5806d55b0 0x55a5806e1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:37.075 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.073+0000 7f2a61119b80 1 -- 172.21.15.117:0/2304035019 >> 172.21.15.117:0/2304035019 conn(0x55a580602170 msgr2=0x55a5806db5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:26:37.075 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.073+0000 7f2a39600700 0 client.25516 ms_handle_reset on v2:172.21.15.183:6838/1439615083 2024-06-03T03:26:37.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.073+0000 7f2a61119b80 1 -- 172.21.15.117:0/2304035019 shutdown_connections 2024-06-03T03:26:37.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.073+0000 7f2a61119b80 1 -- 172.21.15.117:0/2304035019 wait complete. 2024-06-03T03:26:37.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.073+0000 7f2a61119b80 20 client.25516 trim_cache size 0 max 16384 2024-06-03T03:26:37.076 INFO:teuthology.orchestra.run.smithi117.stdout:[] 2024-06-03T03:26:37.078 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.074+0000 7f2a56800700 1 -- 172.21.15.117:0/3870703510 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f2a340a2ca0 msgr2=0x7f2a340a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:37.078 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.074+0000 7f2a56800700 1 --2- 172.21.15.117:0/3870703510 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f2a340a2ca0 0x7f2a340a5160 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c009680 tx=0x7f2a3c009260 comp rx=0 tx=0).stop 2024-06-03T03:26:37.078 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.075+0000 7f2a56800700 1 -- 172.21.15.117:0/3870703510 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a5006a170 msgr2=0x7f2a5019ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:37.078 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.075+0000 7f2a56800700 1 --2- 172.21.15.117:0/3870703510 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a5006a170 0x7f2a5019ae70 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2a40009230 tx=0x7f2a4000fa70 comp rx=0 tx=0).stop 2024-06-03T03:26:37.078 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.075+0000 7f2a56800700 1 -- 172.21.15.117:0/3870703510 shutdown_connections 2024-06-03T03:26:37.079 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.075+0000 7f2a56800700 1 --2- 172.21.15.117:0/3870703510 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f2a340a2ca0 0x7f2a340a5160 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:37.079 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.075+0000 7f2a56800700 1 --2- 172.21.15.117:0/3870703510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a50108500 0x7f2a5019f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:37.079 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.075+0000 7f2a56800700 1 --2- 172.21.15.117:0/3870703510 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a5006a170 0x7f2a5019ae70 secure :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2a40009230 tx=0x7f2a4000fa70 comp rx=0 tx=0).stop 2024-06-03T03:26:37.079 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.075+0000 7f2a56800700 1 --2- 172.21.15.117:0/3870703510 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2a5006ab30 0x7f2a5019b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:37.079 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.075+0000 7f2a56800700 1 -- 172.21.15.117:0/3870703510 >> 172.21.15.117:0/3870703510 conn(0x7f2a50078340 msgr2=0x7f2a5010d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:26:37.079 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.076+0000 7f2a56800700 1 -- 172.21.15.117:0/3870703510 shutdown_connections 2024-06-03T03:26:37.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:37.076+0000 7f2a56800700 1 -- 172.21.15.117:0/3870703510 wait complete. 2024-06-03T03:26:38.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:26:38 smithi151 ceph-mon[37043]: pgmap v545: 129 pgs: 129 active+clean; 679 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 597 B/s wr, 10 op/s 2024-06-03T03:26:38.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:26:38 smithi117 ceph-mon[31649]: pgmap v545: 129 pgs: 129 active+clean; 679 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 597 B/s wr, 10 op/s 2024-06-03T03:26:38.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:26:38 smithi183 ceph-mon[37326]: pgmap v545: 129 pgs: 129 active+clean; 679 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 597 B/s wr, 10 op/s 2024-06-03T03:26:39.108 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-03T03:26:39.108 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-03T03:26:39.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:26:39 smithi151 ceph-mon[37043]: pgmap v546: 129 pgs: 129 active+clean; 680 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 682 B/s wr, 13 op/s 2024-06-03T03:26:39.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:26:39 smithi117 ceph-mon[31649]: pgmap v546: 129 pgs: 129 active+clean; 680 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 682 B/s wr, 13 op/s 2024-06-03T03:26:39.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:26:39 smithi183 ceph-mon[37326]: pgmap v546: 129 pgs: 129 active+clean; 680 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 682 B/s wr, 13 op/s 2024-06-03T03:26:39.556 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:26:42.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.031+0000 7f10af200700 1 -- 172.21.15.117:0/3735931807 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10a806a170 msgr2=0x7f10a806a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:42.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.031+0000 7f10af200700 1 --2- 172.21.15.117:0/3735931807 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10a806a170 0x7f10a806a5f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1094009990 tx=0x7f1094009ca0 comp rx=0 tx=0).stop 2024-06-03T03:26:42.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.031+0000 7f10af200700 1 -- 172.21.15.117:0/3735931807 shutdown_connections 2024-06-03T03:26:42.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.031+0000 7f10af200700 1 --2- 172.21.15.117:0/3735931807 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10a806ab30 0x7f10a810f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:42.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.031+0000 7f10af200700 1 --2- 172.21.15.117:0/3735931807 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10a8108500 0x7f10a81088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:42.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.031+0000 7f10af200700 1 --2- 172.21.15.117:0/3735931807 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10a806a170 0x7f10a806a5f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:42.036 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.031+0000 7f10af200700 1 -- 172.21.15.117:0/3735931807 >> 172.21.15.117:0/3735931807 conn(0x7f10a8078340 msgr2=0x7f10a8078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:26:42.036 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.032+0000 7f10af200700 1 -- 172.21.15.117:0/3735931807 shutdown_connections 2024-06-03T03:26:42.036 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.033+0000 7f10af200700 1 -- 172.21.15.117:0/3735931807 wait complete. 2024-06-03T03:26:42.036 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.034+0000 7f10af200700 1 Processor -- start 2024-06-03T03:26:42.036 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.034+0000 7f10af200700 1 -- start start 2024-06-03T03:26:42.038 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.035+0000 7f10af200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10a806a170 0x7f10a819ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:42.038 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.035+0000 7f10af200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10a806ab30 0x7f10a819b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:42.038 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.035+0000 7f10af200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10a8108500 0x7f10a819f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:42.038 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.035+0000 7f10af200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f10a81127b0 con 0x7f10a8108500 2024-06-03T03:26:42.038 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.035+0000 7f10af200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f10a8112630 con 0x7f10a806a170 2024-06-03T03:26:42.038 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.035+0000 7f10af200700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f10a8112930 con 0x7f10a806ab30 2024-06-03T03:26:42.038 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.035+0000 7f10a7e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10a806a170 0x7f10a819ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:42.038 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.035+0000 7f10a7e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10a806a170 0x7f10a819ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:39224/0 (socket says 172.21.15.117:39224) 2024-06-03T03:26:42.038 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.035+0000 7f10a7e00700 1 -- 172.21.15.117:0/3039224409 learned_addr learned my addr 172.21.15.117:0/3039224409 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:26:42.038 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.035+0000 7f10ad200700 1 --2- 172.21.15.117:0/3039224409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10a8108500 0x7f10a819f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:42.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.035+0000 7f10a7400700 1 --2- 172.21.15.117:0/3039224409 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10a806ab30 0x7f10a819b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:42.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.036+0000 7f10a7400700 1 -- 172.21.15.117:0/3039224409 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10a806a170 msgr2=0x7f10a819ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:42.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.036+0000 7f10a7400700 1 --2- 172.21.15.117:0/3039224409 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10a806a170 0x7f10a819ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:42.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.036+0000 7f10a7400700 1 -- 172.21.15.117:0/3039224409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10a8108500 msgr2=0x7f10a819f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:42.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.036+0000 7f10a7400700 1 --2- 172.21.15.117:0/3039224409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10a8108500 0x7f10a819f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:42.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.036+0000 7f10a7400700 1 -- 172.21.15.117:0/3039224409 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10a819fe50 con 0x7f10a806ab30 2024-06-03T03:26:42.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.036+0000 7f10ad200700 1 --2- 172.21.15.117:0/3039224409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10a8108500 0x7f10a819f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:26:42.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.036+0000 7f10a7400700 1 --2- 172.21.15.117:0/3039224409 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10a806ab30 0x7f10a819b380 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1094000c00 tx=0x7f109400bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:26:42.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.037+0000 7f10a4c00700 1 -- 172.21.15.117:0/3039224409 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1094026070 con 0x7f10a806ab30 2024-06-03T03:26:42.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.037+0000 7f10af200700 1 -- 172.21.15.117:0/3039224409 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10a81a00e0 con 0x7f10a806ab30 2024-06-03T03:26:42.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.037+0000 7f10a4c00700 1 -- 172.21.15.117:0/3039224409 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1094003d70 con 0x7f10a806ab30 2024-06-03T03:26:42.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.037+0000 7f10a4c00700 1 -- 172.21.15.117:0/3039224409 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f109400fdc0 con 0x7f10a806ab30 2024-06-03T03:26:42.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.037+0000 7f10af200700 1 -- 172.21.15.117:0/3039224409 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10a81a79c0 con 0x7f10a806ab30 2024-06-03T03:26:42.042 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.042+0000 7f10b9aceb80 1 Processor -- start 2024-06-03T03:26:42.042 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.042+0000 7f10b9aceb80 1 -- start start 2024-06-03T03:26:42.044 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.042+0000 7f10a4c00700 1 -- 172.21.15.117:0/3039224409 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f1094029070 con 0x7f10a806ab30 2024-06-03T03:26:42.044 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.042+0000 7f10b9aceb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c44f3fc10 0x561c44f3fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:42.044 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.042+0000 7f10b9aceb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c44f40530 0x561c44f39d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:42.044 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.042+0000 7f10b9aceb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561c44f3a2c0 0x561c44f3a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:42.044 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.042+0000 7f10b9aceb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561c44e48a90 con 0x561c44f3fc10 2024-06-03T03:26:42.044 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.042+0000 7f10b9aceb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x561c44e8bb80 con 0x561c44f3a2c0 2024-06-03T03:26:42.044 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.042+0000 7f10b9aceb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561c44e46970 con 0x561c44f40530 2024-06-03T03:26:42.044 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.043+0000 7f10a7e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c44f40530 0x561c44f39d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:42.044 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.043+0000 7f10a7e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c44f40530 0x561c44f39d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:35048/0 (socket says 172.21.15.117:35048) 2024-06-03T03:26:42.044 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.043+0000 7f10a7e00700 1 -- 172.21.15.117:0/114764316 learned_addr learned my addr 172.21.15.117:0/114764316 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:26:42.044 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.043+0000 7f10a7400700 1 --2- 172.21.15.117:0/114764316 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561c44f3a2c0 0x561c44f3a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:42.044 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.043+0000 7f10ad200700 1 --2- 172.21.15.117:0/114764316 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c44f3fc10 0x561c44f3fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:42.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.044+0000 7f10a4c00700 1 --2- 172.21.15.117:0/3039224409 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f108c0a2c20 0x7f108c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:42.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.044+0000 7f10a7e00700 1 -- 172.21.15.117:0/114764316 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561c44f3a2c0 msgr2=0x561c44f3a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:42.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.044+0000 7f10a7e00700 1 --2- 172.21.15.117:0/114764316 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561c44f3a2c0 0x561c44f3a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:42.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.044+0000 7f10a7e00700 1 -- 172.21.15.117:0/114764316 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c44f3fc10 msgr2=0x561c44f3fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:42.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.044+0000 7f10a4c00700 1 -- 172.21.15.117:0/3039224409 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f10940cfff0 con 0x7f10a806ab30 2024-06-03T03:26:42.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.044+0000 7f10a7e00700 1 --2- 172.21.15.117:0/114764316 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c44f3fc10 0x561c44f3fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:42.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.044+0000 7f10a7e00700 1 -- 172.21.15.117:0/114764316 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c44e34010 con 0x561c44f40530 2024-06-03T03:26:42.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.044+0000 7f10ad200700 1 --2- 172.21.15.117:0/114764316 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c44f3fc10 0x561c44f3fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:26:42.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.045+0000 7f10ad200700 1 --2- 172.21.15.117:0/3039224409 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f108c0a2c20 0x7f108c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:42.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.045+0000 7f10a7e00700 1 --2- 172.21.15.117:0/114764316 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c44f40530 0x561c44f39d80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1098009e20 tx=0x7f1098011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ffb6b5c069d5e394 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:26:42.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.045+0000 7f109d600700 1 -- 172.21.15.117:0/114764316 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f109801e070 con 0x561c44f40530 2024-06-03T03:26:42.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.045+0000 7f109d600700 1 -- 172.21.15.117:0/114764316 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1098019b10 con 0x561c44f40530 2024-06-03T03:26:42.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.045+0000 7f109d600700 1 -- 172.21.15.117:0/114764316 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f109800fb80 con 0x561c44f40530 2024-06-03T03:26:42.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.046+0000 7f10b9aceb80 1 -- 172.21.15.117:0/114764316 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c44f40530 msgr2=0x561c44f39d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:42.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.046+0000 7f10b9aceb80 1 --2- 172.21.15.117:0/114764316 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c44f40530 0x561c44f39d80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1098009e20 tx=0x7f1098011720 comp rx=0 tx=0).stop 2024-06-03T03:26:42.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.046+0000 7f10b9aceb80 1 -- 172.21.15.117:0/114764316 shutdown_connections 2024-06-03T03:26:42.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.046+0000 7f10b9aceb80 1 --2- 172.21.15.117:0/114764316 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561c44f3a2c0 0x561c44f3a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:42.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.047+0000 7f10b9aceb80 1 --2- 172.21.15.117:0/114764316 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c44f3fc10 0x561c44f3fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:42.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.047+0000 7f10b9aceb80 1 --2- 172.21.15.117:0/114764316 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c44f40530 0x561c44f39d80 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:42.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.047+0000 7f10b9aceb80 1 -- 172.21.15.117:0/114764316 >> 172.21.15.117:0/114764316 conn(0x561c44e661f0 msgr2=0x561c44f35070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:26:42.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.047+0000 7f10b9aceb80 1 -- 172.21.15.117:0/114764316 shutdown_connections 2024-06-03T03:26:42.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.047+0000 7f10ad200700 1 --2- 172.21.15.117:0/3039224409 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f108c0a2c20 0x7f108c0a50e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f10a0009270 tx=0x7f10a0004ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:26:42.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.048+0000 7f10b9aceb80 1 -- 172.21.15.117:0/114764316 wait complete. 2024-06-03T03:26:42.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.048+0000 7f10b9aceb80 1 -- start start 2024-06-03T03:26:42.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.049+0000 7f10b9aceb80 1 Processor -- start 2024-06-03T03:26:42.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.049+0000 7f10b9aceb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c44f3a2c0 0x561c44f46290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:42.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.049+0000 7f10b9aceb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561c44f3fc10 0x561c44f43590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:42.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.049+0000 7f10b9aceb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c44f40530 0x561c44f43ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:42.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.049+0000 7f10b9aceb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561c44e27fe0 con 0x561c44f3a2c0 2024-06-03T03:26:42.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.049+0000 7f10b9aceb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x561c44e87b40 con 0x561c44f3fc10 2024-06-03T03:26:42.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.049+0000 7f10b9aceb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561c44f4f720 con 0x561c44f40530 2024-06-03T03:26:42.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.050+0000 7f10b9aceb80 10 client.0 _ll_register_callbacks cb 0x561c44e21400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:26:42.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.050+0000 7f10a7e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c44f3a2c0 0x561c44f46290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:42.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.050+0000 7f10ad200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c44f40530 0x561c44f43ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:42.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.050+0000 7f10ad200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c44f40530 0x561c44f43ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:35058/0 (socket says 172.21.15.117:35058) 2024-06-03T03:26:42.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.050+0000 7f10ad200700 1 -- 172.21.15.117:0/2602894683 learned_addr learned my addr 172.21.15.117:0/2602894683 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:26:42.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.051+0000 7f10a7400700 1 --2- 172.21.15.117:0/2602894683 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561c44f3fc10 0x561c44f43590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:42.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.051+0000 7f10a7e00700 1 -- 172.21.15.117:0/2602894683 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c44f40530 msgr2=0x561c44f43ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:42.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.051+0000 7f10a7e00700 1 --2- 172.21.15.117:0/2602894683 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c44f40530 0x561c44f43ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:42.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.051+0000 7f10a7e00700 1 -- 172.21.15.117:0/2602894683 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561c44f3fc10 msgr2=0x561c44f43590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:42.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.051+0000 7f10a7e00700 1 --2- 172.21.15.117:0/2602894683 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561c44f3fc10 0x561c44f43590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:42.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.051+0000 7f10a7e00700 1 -- 172.21.15.117:0/2602894683 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c44dda950 con 0x561c44f3a2c0 2024-06-03T03:26:42.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.051+0000 7f10ad200700 1 --2- 172.21.15.117:0/2602894683 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c44f40530 0x561c44f43ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:26:42.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.052+0000 7f10a7e00700 1 --2- 172.21.15.117:0/2602894683 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c44f3a2c0 0x561c44f46290 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1098009a20 tx=0x7f1098004250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=802a4812af1c9bdb server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:26:42.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.052+0000 7f1092000700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-06-03T03:26:42.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.053+0000 7f1092000700 1 -- 172.21.15.117:0/2602894683 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f109801e050 con 0x561c44f3a2c0 2024-06-03T03:26:42.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.053+0000 7f1092000700 1 -- 172.21.15.117:0/2602894683 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f1098019630 con 0x561c44f3a2c0 2024-06-03T03:26:42.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.053+0000 7f10b9aceb80 1 -- 172.21.15.117:0/2602894683 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1098019b10 con 0x561c44f3a2c0 2024-06-03T03:26:42.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.054+0000 7f1092000700 1 -- 172.21.15.117:0/2602894683 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f1098005ce0 con 0x561c44f3a2c0 2024-06-03T03:26:42.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.054+0000 7f10b9aceb80 10 client.15387 fetch_fsmap learned FSMap version 31 2024-06-03T03:26:42.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.054+0000 7f10b9aceb80 1 -- 172.21.15.117:0/2602894683 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x561c44f59ea0 con 0x561c44f3a2c0 2024-06-03T03:26:42.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.054+0000 7f1092000700 1 -- 172.21.15.117:0/2602894683 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1098019b10 con 0x561c44f3a2c0 2024-06-03T03:26:42.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.054+0000 7f1092000700 1 -- 172.21.15.117:0/2602894683 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f1098003680 con 0x561c44f3a2c0 2024-06-03T03:26:42.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.055+0000 7f10b9aceb80 10 client.15387 fetch_fsmap finished waiting for FSMap version 31 2024-06-03T03:26:42.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.055+0000 7f10b9aceb80 10 client.15387 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-03T03:26:42.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.055+0000 7f10b9aceb80 20 client.15387 populate_metadata read hostname 'smithi117' 2024-06-03T03:26:42.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.055+0000 7f10b9aceb80 1 --2- 172.21.15.117:0/2602894683 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x561c44f5c480 0x561c44f5e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:42.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.055+0000 7f10b9aceb80 4 client.15387 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T03:26:42.056 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.055+0000 7f10b9aceb80 1 -- 172.21.15.117:0/2602894683 --> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c44e99080 con 0x561c44f5c480 2024-06-03T03:26:42.056 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.055+0000 7f10a7400700 1 --2- 172.21.15.117:0/2602894683 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x561c44f5c480 0x561c44f5e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:42.056 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.056+0000 7f1092000700 10 client.15387 ms_handle_connect on v2:172.21.15.183:6838/1439615083 2024-06-03T03:26:42.056 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.056+0000 7f10a7400700 1 --2- 172.21.15.117:0/2602894683 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x561c44f5c480 0x561c44f5e870 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c04eaf2fda89e69 server_cookie=6960d733041f0214 in_seq=0 out_seq=0 2024-06-03T03:26:42.057 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.056+0000 7f1092000700 1 -- 172.21.15.117:0/2602894683 <== mds.0 v2:172.21.15.183:6838/1439615083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x561c44e99080 con 0x561c44f5c480 2024-06-03T03:26:42.057 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.056+0000 7f1092000700 10 client.15387 handle_command_reply: tid=0 2024-06-03T03:26:42.057 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.057+0000 7f10b9aceb80 1 client.15387 shutdown 2024-06-03T03:26:42.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.057+0000 7f10b9aceb80 1 -- 172.21.15.117:0/2602894683 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c44f3a2c0 msgr2=0x561c44f46290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:42.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.057+0000 7f10b9aceb80 1 --2- 172.21.15.117:0/2602894683 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c44f3a2c0 0x561c44f46290 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1098009a20 tx=0x7f1098004250 comp rx=0 tx=0).stop 2024-06-03T03:26:42.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.057+0000 7f10b9aceb80 1 -- 172.21.15.117:0/2602894683 shutdown_connections 2024-06-03T03:26:42.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.057+0000 7f10b9aceb80 1 --2- 172.21.15.117:0/2602894683 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x561c44f5c480 0x561c44f5e870 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:42.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.057+0000 7f10b9aceb80 1 --2- 172.21.15.117:0/2602894683 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c44f40530 0x561c44f43ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:42.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.057+0000 7f10b9aceb80 1 --2- 172.21.15.117:0/2602894683 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c44f3a2c0 0x561c44f46290 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:42.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.058+0000 7f10b9aceb80 1 --2- 172.21.15.117:0/2602894683 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561c44f3fc10 0x561c44f43590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:42.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.057+0000 7f1092000700 0 client.15387 ms_handle_reset on v2:172.21.15.183:6838/1439615083 2024-06-03T03:26:42.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.058+0000 7f10b9aceb80 1 -- 172.21.15.117:0/2602894683 >> 172.21.15.117:0/2602894683 conn(0x561c44e661f0 msgr2=0x561c44f2e810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:26:42.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.058+0000 7f10b9aceb80 1 -- 172.21.15.117:0/2602894683 shutdown_connections 2024-06-03T03:26:42.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.058+0000 7f10b9aceb80 1 -- 172.21.15.117:0/2602894683 wait complete. 2024-06-03T03:26:42.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.058+0000 7f10b9aceb80 20 client.15387 trim_cache size 0 max 16384 2024-06-03T03:26:42.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.098+0000 7f10b9aceb80 1 Processor -- start 2024-06-03T03:26:42.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.098+0000 7f10b9aceb80 1 -- start start 2024-06-03T03:26:42.100 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.099+0000 7f10b9aceb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561c44f3a2c0 0x561c44f39d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:42.100 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.099+0000 7f10b9aceb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c44f3fc10 0x561c44f2f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:42.100 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.099+0000 7f10b9aceb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c44f40530 0x561c44f45dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:42.100 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.099+0000 7f10b9aceb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f10a81abf80 con 0x561c44f3fc10 2024-06-03T03:26:42.100 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.099+0000 7f10b9aceb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f10a81adac0 con 0x561c44f3a2c0 2024-06-03T03:26:42.100 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.099+0000 7f10b9aceb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f10a81ac650 con 0x561c44f40530 2024-06-03T03:26:42.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.099+0000 7f10a7e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561c44f3a2c0 0x561c44f39d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:42.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.099+0000 7f10a7e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561c44f3a2c0 0x561c44f39d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:39252/0 (socket says 172.21.15.117:39252) 2024-06-03T03:26:42.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.099+0000 7f10a7e00700 1 -- 172.21.15.117:0/2215989794 learned_addr learned my addr 172.21.15.117:0/2215989794 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:26:42.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.099+0000 7f10ad200700 1 --2- 172.21.15.117:0/2215989794 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c44f40530 0x561c44f45dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:42.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.099+0000 7f10a7400700 1 --2- 172.21.15.117:0/2215989794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c44f3fc10 0x561c44f2f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:42.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.100+0000 7f10ad200700 1 -- 172.21.15.117:0/2215989794 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561c44f3a2c0 msgr2=0x561c44f39d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:42.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.100+0000 7f10ad200700 1 --2- 172.21.15.117:0/2215989794 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561c44f3a2c0 0x561c44f39d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:42.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.100+0000 7f10ad200700 1 -- 172.21.15.117:0/2215989794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c44f3fc10 msgr2=0x561c44f2f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:42.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.100+0000 7f10ad200700 1 --2- 172.21.15.117:0/2215989794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c44f3fc10 0x561c44f2f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:42.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.100+0000 7f10ad200700 1 -- 172.21.15.117:0/2215989794 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c44f43e50 con 0x561c44f40530 2024-06-03T03:26:42.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.100+0000 7f10ad200700 1 --2- 172.21.15.117:0/2215989794 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c44f40530 0x561c44f45dd0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f10a0003fd0 tx=0x7f10a0017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83f056e01ff00fa1 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:26:42.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.101+0000 7f109cc00700 1 -- 172.21.15.117:0/2215989794 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a0029070 con 0x561c44f40530 2024-06-03T03:26:42.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.101+0000 7f109cc00700 1 -- 172.21.15.117:0/2215989794 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10a000ede0 con 0x561c44f40530 2024-06-03T03:26:42.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.101+0000 7f109cc00700 1 -- 172.21.15.117:0/2215989794 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a0025400 con 0x561c44f40530 2024-06-03T03:26:42.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.102+0000 7f10b9aceb80 1 -- 172.21.15.117:0/2215989794 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c44f40530 msgr2=0x561c44f45dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:42.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.102+0000 7f10b9aceb80 1 --2- 172.21.15.117:0/2215989794 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c44f40530 0x561c44f45dd0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f10a0003fd0 tx=0x7f10a0017000 comp rx=0 tx=0).stop 2024-06-03T03:26:42.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.102+0000 7f10b9aceb80 1 -- 172.21.15.117:0/2215989794 shutdown_connections 2024-06-03T03:26:42.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.102+0000 7f10b9aceb80 1 --2- 172.21.15.117:0/2215989794 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c44f40530 0x561c44f45dd0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:42.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.102+0000 7f10b9aceb80 1 --2- 172.21.15.117:0/2215989794 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561c44f3a2c0 0x561c44f39d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:42.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.102+0000 7f10b9aceb80 1 --2- 172.21.15.117:0/2215989794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c44f3fc10 0x561c44f2f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:42.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.102+0000 7f10b9aceb80 1 -- 172.21.15.117:0/2215989794 >> 172.21.15.117:0/2215989794 conn(0x561c44e661f0 msgr2=0x561c44f3e660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:26:42.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.102+0000 7f10b9aceb80 1 -- 172.21.15.117:0/2215989794 shutdown_connections 2024-06-03T03:26:42.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.102+0000 7f10b9aceb80 1 -- 172.21.15.117:0/2215989794 wait complete. 2024-06-03T03:26:42.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.102+0000 7f10b9aceb80 1 -- start start 2024-06-03T03:26:42.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.103+0000 7f10b9aceb80 1 Processor -- start 2024-06-03T03:26:42.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.103+0000 7f10b9aceb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561c44f3a2c0 0x561c44f63d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:42.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.103+0000 7f10b9aceb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c44f3fc10 0x561c44f64280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:42.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.103+0000 7f10b9aceb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c44f40530 0x561c44f647c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:42.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.103+0000 7f10b9aceb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f10a81ad3f0 con 0x561c44f40530 2024-06-03T03:26:42.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.104+0000 7f10b9aceb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f10a81acd20 con 0x561c44f3a2c0 2024-06-03T03:26:42.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.104+0000 7f10b9aceb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f10a81a9fc0 con 0x561c44f3fc10 2024-06-03T03:26:42.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.104+0000 7f10b9aceb80 10 client.0 _ll_register_callbacks cb 0x561c44e21400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:26:42.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.104+0000 7f10a7e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561c44f3a2c0 0x561c44f63d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:42.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.104+0000 7f10a7400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c44f3fc10 0x561c44f64280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:42.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.104+0000 7f10a7e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561c44f3a2c0 0x561c44f63d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:39258/0 (socket says 172.21.15.117:39258) 2024-06-03T03:26:42.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.104+0000 7f10a7e00700 1 -- 172.21.15.117:0/1498515812 learned_addr learned my addr 172.21.15.117:0/1498515812 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:26:42.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.104+0000 7f10ad200700 1 --2- 172.21.15.117:0/1498515812 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c44f40530 0x561c44f647c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:42.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.104+0000 7f10a7400700 1 -- 172.21.15.117:0/1498515812 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561c44f3a2c0 msgr2=0x561c44f63d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:42.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.104+0000 7f10a7400700 1 --2- 172.21.15.117:0/1498515812 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561c44f3a2c0 0x561c44f63d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:42.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.105+0000 7f10a7400700 1 -- 172.21.15.117:0/1498515812 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c44f40530 msgr2=0x561c44f647c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:42.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.105+0000 7f10a7400700 1 --2- 172.21.15.117:0/1498515812 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c44f40530 0x561c44f647c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:42.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.105+0000 7f10a7400700 1 -- 172.21.15.117:0/1498515812 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c44fc6110 con 0x561c44f3fc10 2024-06-03T03:26:42.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.105+0000 7f10ad200700 1 --2- 172.21.15.117:0/1498515812 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c44f40530 0x561c44f647c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:26:42.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.105+0000 7f10a7400700 1 --2- 172.21.15.117:0/1498515812 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c44f3fc10 0x561c44f64280 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f10940cd7b0 tx=0x7f10940d3ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63cf7810046a9340 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:26:42.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.105+0000 7f1092000700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-06-03T03:26:42.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.105+0000 7f1092000700 1 -- 172.21.15.117:0/1498515812 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f109409c320 con 0x561c44f3fc10 2024-06-03T03:26:42.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.106+0000 7f10b9aceb80 1 -- 172.21.15.117:0/1498515812 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10a000ede0 con 0x561c44f3fc10 2024-06-03T03:26:42.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.107+0000 7f1092000700 1 -- 172.21.15.117:0/1498515812 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f10940d2110 con 0x561c44f3fc10 2024-06-03T03:26:42.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.107+0000 7f1092000700 1 -- 172.21.15.117:0/1498515812 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f109409beb0 con 0x561c44f3fc10 2024-06-03T03:26:42.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.107+0000 7f10b9aceb80 10 client.25534 fetch_fsmap learned FSMap version 31 2024-06-03T03:26:42.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.107+0000 7f10b9aceb80 1 -- 172.21.15.117:0/1498515812 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x561c44fc7410 con 0x561c44f3fc10 2024-06-03T03:26:42.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.108+0000 7f1092000700 1 -- 172.21.15.117:0/1498515812 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10a000ede0 con 0x561c44f3fc10 2024-06-03T03:26:42.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.108+0000 7f1092000700 1 -- 172.21.15.117:0/1498515812 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f10940d3c00 con 0x561c44f3fc10 2024-06-03T03:26:42.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.108+0000 7f10b9aceb80 10 client.25534 fetch_fsmap finished waiting for FSMap version 31 2024-06-03T03:26:42.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.108+0000 7f10b9aceb80 10 client.25534 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-03T03:26:42.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.108+0000 7f10b9aceb80 20 client.25534 populate_metadata read hostname 'smithi117' 2024-06-03T03:26:42.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.108+0000 7f10b9aceb80 1 --2- 172.21.15.117:0/1498515812 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x561c44fc7a80 0x561c44fc9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:42.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.108+0000 7f10b9aceb80 4 client.25534 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-03T03:26:42.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.108+0000 7f10b9aceb80 1 -- 172.21.15.117:0/1498515812 --> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561c44e93600 con 0x561c44fc7a80 2024-06-03T03:26:42.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.109+0000 7f10a7e00700 1 --2- 172.21.15.117:0/1498515812 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x561c44fc7a80 0x561c44fc9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:42.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.109+0000 7f10a7e00700 1 --2- 172.21.15.117:0/1498515812 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x561c44fc7a80 0x561c44fc9ea0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4cf2acb4ec87e0d9 server_cookie=da8cb67bae11a483 in_seq=0 out_seq=0 2024-06-03T03:26:42.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.109+0000 7f1092000700 10 client.25534 ms_handle_connect on v2:172.21.15.183:6838/1439615083 2024-06-03T03:26:42.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.110+0000 7f1092000700 1 -- 172.21.15.117:0/1498515812 <== mds.0 v2:172.21.15.183:6838/1439615083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561c44e93600 con 0x561c44fc7a80 2024-06-03T03:26:42.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.110+0000 7f1092000700 10 client.25534 handle_command_reply: tid=0 2024-06-03T03:26:42.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.110+0000 7f10b9aceb80 1 client.25534 shutdown 2024-06-03T03:26:42.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.110+0000 7f10b9aceb80 1 -- 172.21.15.117:0/1498515812 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c44f3fc10 msgr2=0x561c44f64280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:42.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.110+0000 7f10b9aceb80 1 --2- 172.21.15.117:0/1498515812 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c44f3fc10 0x561c44f64280 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f10940cd7b0 tx=0x7f10940d3ad0 comp rx=0 tx=0).stop 2024-06-03T03:26:42.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.111+0000 7f10b9aceb80 1 -- 172.21.15.117:0/1498515812 shutdown_connections 2024-06-03T03:26:42.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.111+0000 7f10b9aceb80 1 --2- 172.21.15.117:0/1498515812 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x561c44fc7a80 0x561c44fc9ea0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:42.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.111+0000 7f10b9aceb80 1 --2- 172.21.15.117:0/1498515812 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c44f40530 0x561c44f647c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:42.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.111+0000 7f10b9aceb80 1 --2- 172.21.15.117:0/1498515812 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561c44f3a2c0 0x561c44f63d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:42.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.111+0000 7f10b9aceb80 1 --2- 172.21.15.117:0/1498515812 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c44f3fc10 0x561c44f64280 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:42.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.111+0000 7f10b9aceb80 1 -- 172.21.15.117:0/1498515812 >> 172.21.15.117:0/1498515812 conn(0x561c44e661f0 msgr2=0x561c44f2e1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:26:42.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.111+0000 7f1092000700 0 client.25534 ms_handle_reset on v2:172.21.15.183:6838/1439615083 2024-06-03T03:26:42.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.111+0000 7f10b9aceb80 1 -- 172.21.15.117:0/1498515812 shutdown_connections 2024-06-03T03:26:42.113 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.111+0000 7f10b9aceb80 1 -- 172.21.15.117:0/1498515812 wait complete. 2024-06-03T03:26:42.113 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.112+0000 7f10b9aceb80 20 client.25534 trim_cache size 0 max 16384 2024-06-03T03:26:42.113 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-06-03T03:26:42.113 INFO:teuthology.orchestra.run.smithi117.stdout: "return_code": 0, 2024-06-03T03:26:42.113 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_tag": "0cea2843-74a0-47ae-9512-7d33166b2a61", 2024-06-03T03:26:42.113 INFO:teuthology.orchestra.run.smithi117.stdout: "mode": "asynchronous" 2024-06-03T03:26:42.113 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-06-03T03:26:42.114 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.113+0000 7f10af200700 1 -- 172.21.15.117:0/3039224409 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f108c0a2c20 msgr2=0x7f108c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:42.114 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.113+0000 7f10af200700 1 --2- 172.21.15.117:0/3039224409 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f108c0a2c20 0x7f108c0a50e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f10a0009270 tx=0x7f10a0004ad0 comp rx=0 tx=0).stop 2024-06-03T03:26:42.114 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.113+0000 7f10af200700 1 -- 172.21.15.117:0/3039224409 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10a806ab30 msgr2=0x7f10a819b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:42.114 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.113+0000 7f10af200700 1 --2- 172.21.15.117:0/3039224409 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10a806ab30 0x7f10a819b380 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1094000c00 tx=0x7f109400bab0 comp rx=0 tx=0).stop 2024-06-03T03:26:42.114 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.114+0000 7f10af200700 1 -- 172.21.15.117:0/3039224409 shutdown_connections 2024-06-03T03:26:42.115 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.114+0000 7f10af200700 1 --2- 172.21.15.117:0/3039224409 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f108c0a2c20 0x7f108c0a50e0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:42.115 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.114+0000 7f10af200700 1 --2- 172.21.15.117:0/3039224409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10a8108500 0x7f10a819f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:42.115 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.114+0000 7f10af200700 1 --2- 172.21.15.117:0/3039224409 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10a806a170 0x7f10a819ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:42.115 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.114+0000 7f10af200700 1 --2- 172.21.15.117:0/3039224409 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10a806ab30 0x7f10a819b380 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:42.115 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.114+0000 7f10af200700 1 -- 172.21.15.117:0/3039224409 >> 172.21.15.117:0/3039224409 conn(0x7f10a8078340 msgr2=0x7f10a810d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:26:42.115 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.114+0000 7f10af200700 1 -- 172.21.15.117:0/3039224409 shutdown_connections 2024-06-03T03:26:42.115 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:42.114+0000 7f10af200700 1 -- 172.21.15.117:0/3039224409 wait complete. 2024-06-03T03:26:42.138 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:26:42 smithi117 ceph-mon[31649]: pgmap v547: 129 pgs: 129 active+clean; 680 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 341 B/s wr, 10 op/s 2024-06-03T03:26:42.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:26:42 smithi151 ceph-mon[37043]: pgmap v547: 129 pgs: 129 active+clean; 680 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 341 B/s wr, 10 op/s 2024-06-03T03:26:42.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:26:42 smithi183 ceph-mon[37326]: pgmap v547: 129 pgs: 129 active+clean; 680 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 341 B/s wr, 10 op/s 2024-06-03T03:26:43.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:26:43 smithi117 ceph-mon[31649]: scrub queued for path: / 2024-06-03T03:26:43.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:26:43 smithi117 ceph-mon[31649]: scrub summary: idle+waiting paths [/] 2024-06-03T03:26:43.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:26:43 smithi117 ceph-mon[31649]: scrub summary: active paths [/] 2024-06-03T03:26:43.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:26:43 smithi117 ceph-mon[31649]: scrub summary: idle+waiting paths [/] 2024-06-03T03:26:43.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:26:43 smithi117 ceph-mon[31649]: scrub summary: idle 2024-06-03T03:26:43.321 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-03T03:26:43.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:26:43 smithi151 ceph-mon[37043]: scrub queued for path: / 2024-06-03T03:26:43.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:26:43 smithi151 ceph-mon[37043]: scrub summary: idle+waiting paths [/] 2024-06-03T03:26:43.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:26:43 smithi151 ceph-mon[37043]: scrub summary: active paths [/] 2024-06-03T03:26:43.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:26:43 smithi151 ceph-mon[37043]: scrub summary: idle+waiting paths [/] 2024-06-03T03:26:43.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:26:43 smithi151 ceph-mon[37043]: scrub summary: idle 2024-06-03T03:26:43.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:26:43 smithi183 ceph-mon[37326]: scrub queued for path: / 2024-06-03T03:26:43.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:26:43 smithi183 ceph-mon[37326]: scrub summary: idle+waiting paths [/] 2024-06-03T03:26:43.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:26:43 smithi183 ceph-mon[37326]: scrub summary: active paths [/] 2024-06-03T03:26:43.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:26:43 smithi183 ceph-mon[37326]: scrub summary: idle+waiting paths [/] 2024-06-03T03:26:43.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:26:43 smithi183 ceph-mon[37326]: scrub summary: idle 2024-06-03T03:26:43.770 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:26:44.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:26:44 smithi117 ceph-mon[31649]: pgmap v548: 129 pgs: 129 active+clean; 680 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 341 B/s wr, 12 op/s 2024-06-03T03:26:44.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:26:44 smithi151 ceph-mon[37043]: pgmap v548: 129 pgs: 129 active+clean; 680 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 341 B/s wr, 12 op/s 2024-06-03T03:26:44.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:26:44 smithi183 ceph-mon[37326]: pgmap v548: 129 pgs: 129 active+clean; 680 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 341 B/s wr, 12 op/s 2024-06-03T03:26:45.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:26:45 smithi117 ceph-mon[31649]: pgmap v549: 129 pgs: 129 active+clean; 680 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 341 B/s wr, 10 op/s 2024-06-03T03:26:45.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:26:45 smithi151 ceph-mon[37043]: pgmap v549: 129 pgs: 129 active+clean; 680 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 341 B/s wr, 10 op/s 2024-06-03T03:26:45.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:26:45 smithi183 ceph-mon[37326]: pgmap v549: 129 pgs: 129 active+clean; 680 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 341 B/s wr, 10 op/s 2024-06-03T03:26:46.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.365+0000 7fb882000700 1 -- 172.21.15.117:0/3021701244 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb87c06ab30 msgr2=0x7fb87c10f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:46.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.365+0000 7fb882000700 1 --2- 172.21.15.117:0/3021701244 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb87c06ab30 0x7fb87c10f320 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb874009f90 tx=0x7fb87400b330 comp rx=0 tx=0).stop 2024-06-03T03:26:46.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.365+0000 7fb882000700 1 -- 172.21.15.117:0/3021701244 shutdown_connections 2024-06-03T03:26:46.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.365+0000 7fb882000700 1 --2- 172.21.15.117:0/3021701244 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb87c06ab30 0x7fb87c10f320 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:46.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.365+0000 7fb882000700 1 --2- 172.21.15.117:0/3021701244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb87c108500 0x7fb87c1088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:46.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.365+0000 7fb882000700 1 --2- 172.21.15.117:0/3021701244 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb87c06a170 0x7fb87c06a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:46.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.365+0000 7fb882000700 1 -- 172.21.15.117:0/3021701244 >> 172.21.15.117:0/3021701244 conn(0x7fb87c078340 msgr2=0x7fb87c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:26:46.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.365+0000 7fb882000700 1 -- 172.21.15.117:0/3021701244 shutdown_connections 2024-06-03T03:26:46.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.365+0000 7fb882000700 1 -- 172.21.15.117:0/3021701244 wait complete. 2024-06-03T03:26:46.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.366+0000 7fb882000700 1 Processor -- start 2024-06-03T03:26:46.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.367+0000 7fb882000700 1 -- start start 2024-06-03T03:26:46.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.367+0000 7fb882000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb87c06a170 0x7fb87c19d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:46.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.367+0000 7fb882000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb87c06ab30 0x7fb87c19dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:46.369 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.367+0000 7fb882000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb87c108500 0x7fb87c1969d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:46.369 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.367+0000 7fb882000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb87c1105d0 con 0x7fb87c108500 2024-06-03T03:26:46.369 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.367+0000 7fb882000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb87c110450 con 0x7fb87c06ab30 2024-06-03T03:26:46.369 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.367+0000 7fb882000700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb87c110750 con 0x7fb87c06a170 2024-06-03T03:26:46.369 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.367+0000 7fb87be00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb87c108500 0x7fb87c1969d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:46.369 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.368+0000 7fb87be00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb87c108500 0x7fb87c1969d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:57086/0 (socket says 172.21.15.117:57086) 2024-06-03T03:26:46.369 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.368+0000 7fb87be00700 1 -- 172.21.15.117:0/1681343865 learned_addr learned my addr 172.21.15.117:0/1681343865 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:26:46.369 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.368+0000 7fb87b400700 1 --2- 172.21.15.117:0/1681343865 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb87c06a170 0x7fb87c19d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:46.369 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.369+0000 7fb87aa00700 1 --2- 172.21.15.117:0/1681343865 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb87c06ab30 0x7fb87c19dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:46.370 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.369+0000 7fb87be00700 1 -- 172.21.15.117:0/1681343865 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb87c06a170 msgr2=0x7fb87c19d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:46.370 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.369+0000 7fb87be00700 1 --2- 172.21.15.117:0/1681343865 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb87c06a170 0x7fb87c19d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:46.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.369+0000 7fb87be00700 1 -- 172.21.15.117:0/1681343865 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb87c06ab30 msgr2=0x7fb87c19dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:46.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.369+0000 7fb87be00700 1 --2- 172.21.15.117:0/1681343865 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb87c06ab30 0x7fb87c19dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:46.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.369+0000 7fb87be00700 1 -- 172.21.15.117:0/1681343865 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb87c197240 con 0x7fb87c108500 2024-06-03T03:26:46.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.370+0000 7fb87be00700 1 --2- 172.21.15.117:0/1681343865 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb87c108500 0x7fb87c1969d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb87400cd30 tx=0x7fb87400cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:26:46.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.371+0000 7fb873e00700 1 -- 172.21.15.117:0/1681343865 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb874021070 con 0x7fb87c108500 2024-06-03T03:26:46.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.371+0000 7fb873e00700 1 -- 172.21.15.117:0/1681343865 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb874007ac0 con 0x7fb87c108500 2024-06-03T03:26:46.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.371+0000 7fb873e00700 1 -- 172.21.15.117:0/1681343865 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb87401de60 con 0x7fb87c108500 2024-06-03T03:26:46.373 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.372+0000 7fb882000700 1 -- 172.21.15.117:0/1681343865 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb87c1974d0 con 0x7fb87c108500 2024-06-03T03:26:46.373 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.372+0000 7fb882000700 1 -- 172.21.15.117:0/1681343865 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb87c19b440 con 0x7fb87c108500 2024-06-03T03:26:46.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.378+0000 7fb873e00700 1 -- 172.21.15.117:0/1681343865 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fb874024070 con 0x7fb87c108500 2024-06-03T03:26:46.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.378+0000 7fb88c95fb80 1 Processor -- start 2024-06-03T03:26:46.384 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.379+0000 7fb88c95fb80 1 -- start start 2024-06-03T03:26:46.384 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.380+0000 7fb88c95fb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5615bdc48e40 0x5615bdccc070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:46.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.380+0000 7fb88c95fb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615bdccc5b0 0x5615bdccca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:46.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.380+0000 7fb88c95fb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615bdcd7300 0x5615bdcd96f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:46.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.380+0000 7fb88c95fb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5615bdbdba10 con 0x5615bdccc5b0 2024-06-03T03:26:46.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.380+0000 7fb88c95fb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5615bdc1eb00 con 0x5615bdc48e40 2024-06-03T03:26:46.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.380+0000 7fb88c95fb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5615bdbd98f0 con 0x5615bdcd7300 2024-06-03T03:26:46.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.380+0000 7fb87b400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615bdccc5b0 0x5615bdccca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:46.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.380+0000 7fb87b400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615bdccc5b0 0x5615bdccca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:57098/0 (socket says 172.21.15.117:57098) 2024-06-03T03:26:46.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.380+0000 7fb87b400700 1 -- 172.21.15.117:0/980520627 learned_addr learned my addr 172.21.15.117:0/980520627 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:26:46.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.380+0000 7fb87aa00700 1 --2- 172.21.15.117:0/980520627 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5615bdc48e40 0x5615bdccc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:46.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.381+0000 7fb87b400700 1 -- 172.21.15.117:0/980520627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615bdcd7300 msgr2=0x5615bdcd96f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-03T03:26:46.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.381+0000 7fb87b400700 1 --2- 172.21.15.117:0/980520627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615bdcd7300 0x5615bdcd96f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:46.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.381+0000 7fb87b400700 1 -- 172.21.15.117:0/980520627 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5615bdc48e40 msgr2=0x5615bdccc070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:46.386 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.381+0000 7fb87b400700 1 --2- 172.21.15.117:0/980520627 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5615bdc48e40 0x5615bdccc070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:46.386 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.381+0000 7fb87b400700 1 -- 172.21.15.117:0/980520627 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615bdbc6f90 con 0x5615bdccc5b0 2024-06-03T03:26:46.386 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.381+0000 7fb87aa00700 1 --2- 172.21.15.117:0/980520627 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5615bdc48e40 0x5615bdccc070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T03:26:46.386 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.381+0000 7fb873e00700 1 --2- 172.21.15.117:0/1681343865 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fb8600a2d20 0x7fb8600a51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:46.386 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.381+0000 7fb873e00700 1 -- 172.21.15.117:0/1681343865 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7fb8740d00e0 con 0x7fb87c108500 2024-06-03T03:26:46.386 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.382+0000 7fb87b400700 1 --2- 172.21.15.117:0/980520627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615bdccc5b0 0x5615bdccca30 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb86c009e70 tx=0x7fb86c011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6c910e8b9c9eda1 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:26:46.387 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.382+0000 7fb870c00700 1 -- 172.21.15.117:0/980520627 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb86c004310 con 0x5615bdccc5b0 2024-06-03T03:26:46.387 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.382+0000 7fb870c00700 1 -- 172.21.15.117:0/980520627 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb86c0044b0 con 0x5615bdccc5b0 2024-06-03T03:26:46.387 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.382+0000 7fb870c00700 1 -- 172.21.15.117:0/980520627 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb86c004310 con 0x5615bdccc5b0 2024-06-03T03:26:46.387 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.383+0000 7fb88c95fb80 1 -- 172.21.15.117:0/980520627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615bdccc5b0 msgr2=0x5615bdccca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:46.387 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.383+0000 7fb88c95fb80 1 --2- 172.21.15.117:0/980520627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615bdccc5b0 0x5615bdccca30 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb86c009e70 tx=0x7fb86c011720 comp rx=0 tx=0).stop 2024-06-03T03:26:46.387 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.384+0000 7fb88c95fb80 1 -- 172.21.15.117:0/980520627 shutdown_connections 2024-06-03T03:26:46.387 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.384+0000 7fb88c95fb80 1 --2- 172.21.15.117:0/980520627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615bdcd7300 0x5615bdcd96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:46.387 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.384+0000 7fb88c95fb80 1 --2- 172.21.15.117:0/980520627 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5615bdc48e40 0x5615bdccc070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:46.387 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.384+0000 7fb88c95fb80 1 --2- 172.21.15.117:0/980520627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615bdccc5b0 0x5615bdccca30 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:46.387 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.384+0000 7fb88c95fb80 1 -- 172.21.15.117:0/980520627 >> 172.21.15.117:0/980520627 conn(0x5615bdbf9170 msgr2=0x5615bdcc84b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:26:46.387 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.384+0000 7fb88c95fb80 1 -- 172.21.15.117:0/980520627 shutdown_connections 2024-06-03T03:26:46.387 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.384+0000 7fb88c95fb80 1 -- 172.21.15.117:0/980520627 wait complete. 2024-06-03T03:26:46.388 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.385+0000 7fb88c95fb80 1 -- start start 2024-06-03T03:26:46.388 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.385+0000 7fb88c95fb80 1 Processor -- start 2024-06-03T03:26:46.388 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.386+0000 7fb87aa00700 1 --2- 172.21.15.117:0/1681343865 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fb8600a2d20 0x7fb8600a51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:46.389 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.387+0000 7fb87aa00700 1 --2- 172.21.15.117:0/1681343865 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fb8600a2d20 0x7fb8600a51e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x5615bdcc93b0 tx=0x7fb86800f040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:26:46.389 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.387+0000 7fb88c95fb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615bdc48e40 0x5615bdcd69a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:46.389 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.387+0000 7fb87b400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615bdc48e40 0x5615bdcd69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:46.389 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.387+0000 7fb88c95fb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5615bdccc5b0 0x5615bdcd3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:46.389 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.388+0000 7fb88c95fb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615bdcd7300 0x5615bdcd4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:46.389 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.388+0000 7fb88c95fb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5615bdbbaf60 con 0x5615bdc48e40 2024-06-03T03:26:46.389 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.388+0000 7fb88c95fb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5615bdc1aac0 con 0x5615bdccc5b0 2024-06-03T03:26:46.390 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.388+0000 7fb88c95fb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5615bdcd9fc0 con 0x5615bdcd7300 2024-06-03T03:26:46.390 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.388+0000 7fb87b400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615bdc48e40 0x5615bdcd69a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:57102/0 (socket says 172.21.15.117:57102) 2024-06-03T03:26:46.390 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.388+0000 7fb87b400700 1 -- 172.21.15.117:0/18472301 learned_addr learned my addr 172.21.15.117:0/18472301 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:26:46.390 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.388+0000 7fb87b400700 1 -- 172.21.15.117:0/18472301 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615bdcd7300 msgr2=0x5615bdcd4270 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-03T03:26:46.390 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.388+0000 7fb87b400700 1 --2- 172.21.15.117:0/18472301 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615bdcd7300 0x5615bdcd4270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:46.390 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.388+0000 7fb87aa00700 1 --2- 172.21.15.117:0/18472301 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5615bdccc5b0 0x5615bdcd3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:46.390 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.388+0000 7fb87b400700 1 -- 172.21.15.117:0/18472301 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5615bdccc5b0 msgr2=0x5615bdcd3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:46.390 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.388+0000 7fb87b400700 1 --2- 172.21.15.117:0/18472301 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5615bdccc5b0 0x5615bdcd3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:46.390 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.388+0000 7fb87b400700 1 -- 172.21.15.117:0/18472301 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615bdb6d8d0 con 0x5615bdc48e40 2024-06-03T03:26:46.390 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.388+0000 7fb87aa00700 1 --2- 172.21.15.117:0/18472301 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5615bdccc5b0 0x5615bdcd3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T03:26:46.391 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.389+0000 7fb88c95fb80 10 client.0 _ll_register_callbacks cb 0x5615bdbb8720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:26:46.391 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.389+0000 7fb87b400700 1 --2- 172.21.15.117:0/18472301 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615bdc48e40 0x5615bdcd69a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb86c014040 tx=0x7fb86c009820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f720863e141c578c server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:26:46.391 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.389+0000 7fb865600700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-06-03T03:26:46.391 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.390+0000 7fb865600700 1 -- 172.21.15.117:0/18472301 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb86c003d20 con 0x5615bdc48e40 2024-06-03T03:26:46.391 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.390+0000 7fb865600700 1 -- 172.21.15.117:0/18472301 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7fb86c005050 con 0x5615bdc48e40 2024-06-03T03:26:46.391 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.390+0000 7fb88c95fb80 1 -- 172.21.15.117:0/18472301 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb86c0044b0 con 0x5615bdc48e40 2024-06-03T03:26:46.391 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.390+0000 7fb865600700 1 -- 172.21.15.117:0/18472301 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fb86c005290 con 0x5615bdc48e40 2024-06-03T03:26:46.391 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.390+0000 7fb88c95fb80 10 client.15405 fetch_fsmap learned FSMap version 31 2024-06-03T03:26:46.391 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.390+0000 7fb88c95fb80 1 -- 172.21.15.117:0/18472301 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({fsmap=31}) -- 0x5615bdce4290 con 0x5615bdc48e40 2024-06-03T03:26:46.392 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.391+0000 7fb865600700 1 -- 172.21.15.117:0/18472301 <== mon.0 v2:172.21.15.117:3300/0 4 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fb86c018070 con 0x5615bdc48e40 2024-06-03T03:26:46.392 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.391+0000 7fb88c95fb80 10 client.15405 fetch_fsmap finished waiting for FSMap version 31 2024-06-03T03:26:46.392 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.391+0000 7fb88c95fb80 10 client.15405 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-03T03:26:46.392 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.391+0000 7fb88c95fb80 20 client.15405 populate_metadata read hostname 'smithi117' 2024-06-03T03:26:46.392 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.391+0000 7fb88c95fb80 1 --2- 172.21.15.117:0/18472301 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x5615bdce6760 0x5615bdce8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:46.392 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.391+0000 7fb88c95fb80 4 client.15405 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T03:26:46.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.391+0000 7fb88c95fb80 1 -- 172.21.15.117:0/18472301 --> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5615bdc2c000 con 0x5615bdce6760 2024-06-03T03:26:46.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.392+0000 7fb87aa00700 1 --2- 172.21.15.117:0/18472301 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x5615bdce6760 0x5615bdce8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:46.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.392+0000 7fb87aa00700 1 --2- 172.21.15.117:0/18472301 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x5615bdce6760 0x5615bdce8b50 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77042726073d854f server_cookie=873f8483192dad24 in_seq=0 out_seq=0 2024-06-03T03:26:46.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.392+0000 7fb865600700 10 client.15405 ms_handle_connect on v2:172.21.15.183:6838/1439615083 2024-06-03T03:26:46.394 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.393+0000 7fb865600700 1 -- 172.21.15.117:0/18472301 <== mds.0 v2:172.21.15.183:6838/1439615083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5615bdc2c000 con 0x5615bdce6760 2024-06-03T03:26:46.394 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.393+0000 7fb865600700 10 client.15405 handle_command_reply: tid=0 2024-06-03T03:26:46.394 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.393+0000 7fb88c95fb80 1 client.15405 shutdown 2024-06-03T03:26:46.394 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.394+0000 7fb88c95fb80 1 -- 172.21.15.117:0/18472301 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615bdc48e40 msgr2=0x5615bdcd69a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:46.395 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.394+0000 7fb88c95fb80 1 --2- 172.21.15.117:0/18472301 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615bdc48e40 0x5615bdcd69a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb86c014040 tx=0x7fb86c009820 comp rx=0 tx=0).stop 2024-06-03T03:26:46.395 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.394+0000 7fb88c95fb80 1 -- 172.21.15.117:0/18472301 shutdown_connections 2024-06-03T03:26:46.395 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.394+0000 7fb88c95fb80 1 --2- 172.21.15.117:0/18472301 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x5615bdce6760 0x5615bdce8b50 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:46.395 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.394+0000 7fb88c95fb80 1 --2- 172.21.15.117:0/18472301 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615bdcd7300 0x5615bdcd4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:46.395 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.394+0000 7fb88c95fb80 1 --2- 172.21.15.117:0/18472301 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615bdc48e40 0x5615bdcd69a0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:46.395 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.394+0000 7fb88c95fb80 1 --2- 172.21.15.117:0/18472301 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5615bdccc5b0 0x5615bdcd3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:46.395 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.394+0000 7fb88c95fb80 1 -- 172.21.15.117:0/18472301 >> 172.21.15.117:0/18472301 conn(0x5615bdbf9170 msgr2=0x5615bdcc7740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:26:46.395 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.394+0000 7fb865600700 0 client.15405 ms_handle_reset on v2:172.21.15.183:6838/1439615083 2024-06-03T03:26:46.395 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.394+0000 7fb88c95fb80 1 -- 172.21.15.117:0/18472301 shutdown_connections 2024-06-03T03:26:46.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.396+0000 7fb88c95fb80 1 -- 172.21.15.117:0/18472301 wait complete. 2024-06-03T03:26:46.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.396+0000 7fb88c95fb80 20 client.15405 trim_cache size 0 max 16384 2024-06-03T03:26:46.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.434+0000 7fb88c95fb80 1 Processor -- start 2024-06-03T03:26:46.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.434+0000 7fb88c95fb80 1 -- start start 2024-06-03T03:26:46.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.435+0000 7fb88c95fb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5615bdc48e40 0x5615bdcd6550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:46.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.435+0000 7fb88c95fb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615bdccc5b0 0x5615bdcc7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:46.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.435+0000 7fb88c95fb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615bdcd7300 0x5615bdcc8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:46.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.435+0000 7fb88c95fb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb87c073cb0 con 0x5615bdcd7300 2024-06-03T03:26:46.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.435+0000 7fb88c95fb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb87c1c73d0 con 0x5615bdc48e40 2024-06-03T03:26:46.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.435+0000 7fb88c95fb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb87c074380 con 0x5615bdccc5b0 2024-06-03T03:26:46.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.435+0000 7fb87b400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5615bdc48e40 0x5615bdcd6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:46.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.435+0000 7fb87b400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5615bdc48e40 0x5615bdcd6550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:39314/0 (socket says 172.21.15.117:39314) 2024-06-03T03:26:46.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.435+0000 7fb87b400700 1 -- 172.21.15.117:0/939691587 learned_addr learned my addr 172.21.15.117:0/939691587 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:26:46.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.435+0000 7fb87be00700 1 --2- 172.21.15.117:0/939691587 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615bdcd7300 0x5615bdcc8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:46.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.435+0000 7fb87aa00700 1 --2- 172.21.15.117:0/939691587 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615bdccc5b0 0x5615bdcc7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:46.438 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.436+0000 7fb87be00700 1 -- 172.21.15.117:0/939691587 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615bdccc5b0 msgr2=0x5615bdcc7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:46.438 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.436+0000 7fb87be00700 1 --2- 172.21.15.117:0/939691587 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615bdccc5b0 0x5615bdcc7ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:46.438 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.436+0000 7fb87be00700 1 -- 172.21.15.117:0/939691587 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5615bdc48e40 msgr2=0x5615bdcd6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:46.438 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.436+0000 7fb87be00700 1 --2- 172.21.15.117:0/939691587 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5615bdc48e40 0x5615bdcd6550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:46.438 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.436+0000 7fb87be00700 1 -- 172.21.15.117:0/939691587 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615bdccdbd0 con 0x5615bdcd7300 2024-06-03T03:26:46.438 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.436+0000 7fb87be00700 1 --2- 172.21.15.117:0/939691587 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615bdcd7300 0x5615bdcc8400 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb87400cd00 tx=0x7fb874006160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cce01a2066299189 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:26:46.438 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.437+0000 7fb867e00700 1 -- 172.21.15.117:0/939691587 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb874009c10 con 0x5615bdcd7300 2024-06-03T03:26:46.438 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.437+0000 7fb867e00700 1 -- 172.21.15.117:0/939691587 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb87409cce0 con 0x5615bdcd7300 2024-06-03T03:26:46.438 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.437+0000 7fb867e00700 1 -- 172.21.15.117:0/939691587 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb874007420 con 0x5615bdcd7300 2024-06-03T03:26:46.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.438+0000 7fb88c95fb80 1 -- 172.21.15.117:0/939691587 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615bdcd7300 msgr2=0x5615bdcc8400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:46.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.438+0000 7fb88c95fb80 1 --2- 172.21.15.117:0/939691587 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615bdcd7300 0x5615bdcc8400 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb87400cd00 tx=0x7fb874006160 comp rx=0 tx=0).stop 2024-06-03T03:26:46.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.438+0000 7fb88c95fb80 1 -- 172.21.15.117:0/939691587 shutdown_connections 2024-06-03T03:26:46.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.438+0000 7fb88c95fb80 1 --2- 172.21.15.117:0/939691587 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615bdcd7300 0x5615bdcc8400 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:46.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.438+0000 7fb88c95fb80 1 --2- 172.21.15.117:0/939691587 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5615bdc48e40 0x5615bdcd6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:46.440 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.438+0000 7fb88c95fb80 1 --2- 172.21.15.117:0/939691587 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615bdccc5b0 0x5615bdcc7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:46.440 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.438+0000 7fb88c95fb80 1 -- 172.21.15.117:0/939691587 >> 172.21.15.117:0/939691587 conn(0x5615bdbf9170 msgr2=0x5615bdce2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:26:46.440 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.438+0000 7fb88c95fb80 1 -- 172.21.15.117:0/939691587 shutdown_connections 2024-06-03T03:26:46.440 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.439+0000 7fb88c95fb80 1 -- 172.21.15.117:0/939691587 wait complete. 2024-06-03T03:26:46.440 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.439+0000 7fb88c95fb80 1 -- start start 2024-06-03T03:26:46.440 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.439+0000 7fb88c95fb80 1 Processor -- start 2024-06-03T03:26:46.441 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.440+0000 7fb88c95fb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5615bdc48e40 0x5615bdcdb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:46.442 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.440+0000 7fb88c95fb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615bdccc5b0 0x5615bdcdb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:46.442 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.440+0000 7fb88c95fb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615bdcd7300 0x5615bdcdbd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:46.442 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.440+0000 7fb88c95fb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb87c1c6d00 con 0x5615bdccc5b0 2024-06-03T03:26:46.442 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.440+0000 7fb88c95fb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb87c1c6630 con 0x5615bdc48e40 2024-06-03T03:26:46.442 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.440+0000 7fb88c95fb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb87c071c50 con 0x5615bdcd7300 2024-06-03T03:26:46.442 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.440+0000 7fb88c95fb80 10 client.0 _ll_register_callbacks cb 0x5615bdbb8720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:26:46.442 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.441+0000 7fb87b400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5615bdc48e40 0x5615bdcdb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:46.442 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.441+0000 7fb87b400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5615bdc48e40 0x5615bdcdb280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:39320/0 (socket says 172.21.15.117:39320) 2024-06-03T03:26:46.442 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.441+0000 7fb87b400700 1 -- 172.21.15.117:0/3347173668 learned_addr learned my addr 172.21.15.117:0/3347173668 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:26:46.442 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.441+0000 7fb87be00700 1 --2- 172.21.15.117:0/3347173668 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615bdcd7300 0x5615bdcdbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:46.442 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.441+0000 7fb87b400700 1 -- 172.21.15.117:0/3347173668 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615bdcd7300 msgr2=0x5615bdcdbd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:46.443 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.441+0000 7fb87aa00700 1 --2- 172.21.15.117:0/3347173668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615bdccc5b0 0x5615bdcdb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:46.443 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.441+0000 7fb87b400700 1 --2- 172.21.15.117:0/3347173668 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615bdcd7300 0x5615bdcdbd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:46.443 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.442+0000 7fb87b400700 1 -- 172.21.15.117:0/3347173668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615bdccc5b0 msgr2=0x5615bdcdb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:46.443 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.442+0000 7fb87b400700 1 --2- 172.21.15.117:0/3347173668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615bdccc5b0 0x5615bdcdb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:46.443 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.442+0000 7fb87b400700 1 -- 172.21.15.117:0/3347173668 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615bdd504c0 con 0x5615bdc48e40 2024-06-03T03:26:46.443 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.442+0000 7fb87aa00700 1 --2- 172.21.15.117:0/3347173668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615bdccc5b0 0x5615bdcdb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T03:26:46.443 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.442+0000 7fb87b400700 1 --2- 172.21.15.117:0/3347173668 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5615bdc48e40 0x5615bdcdb280 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb86c009e70 tx=0x7fb86c009f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d1b655e0ecb18bd server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:26:46.444 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.442+0000 7fb865600700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-06-03T03:26:46.444 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.442+0000 7fb865600700 1 -- 172.21.15.117:0/3347173668 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb86c003d20 con 0x5615bdc48e40 2024-06-03T03:26:46.444 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.443+0000 7fb865600700 1 -- 172.21.15.117:0/3347173668 <== mon.2 v2:172.21.15.151:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7fb86c0054d0 con 0x5615bdc48e40 2024-06-03T03:26:46.444 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.444+0000 7fb88c95fb80 1 -- 172.21.15.117:0/3347173668 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb87409cce0 con 0x5615bdc48e40 2024-06-03T03:26:46.445 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.444+0000 7fb865600700 1 -- 172.21.15.117:0/3347173668 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fb86c02d450 con 0x5615bdc48e40 2024-06-03T03:26:46.445 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.444+0000 7fb88c95fb80 10 client.25511 fetch_fsmap learned FSMap version 31 2024-06-03T03:26:46.445 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.444+0000 7fb88c95fb80 1 -- 172.21.15.117:0/3347173668 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5615bdd517d0 con 0x5615bdc48e40 2024-06-03T03:26:46.445 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.445+0000 7fb865600700 1 -- 172.21.15.117:0/3347173668 <== mon.2 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb87409cce0 con 0x5615bdc48e40 2024-06-03T03:26:46.446 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.445+0000 7fb865600700 1 -- 172.21.15.117:0/3347173668 <== mon.2 v2:172.21.15.151:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fb86c018040 con 0x5615bdc48e40 2024-06-03T03:26:46.446 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.445+0000 7fb88c95fb80 10 client.25511 fetch_fsmap finished waiting for FSMap version 31 2024-06-03T03:26:46.446 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.445+0000 7fb88c95fb80 10 client.25511 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-03T03:26:46.446 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.445+0000 7fb88c95fb80 20 client.25511 populate_metadata read hostname 'smithi117' 2024-06-03T03:26:46.446 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.446+0000 7fb88c95fb80 1 --2- 172.21.15.117:0/3347173668 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x5615bdd51e80 0x5615bdd542a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:46.447 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.446+0000 7fb88c95fb80 4 client.25511 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-03T03:26:46.447 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.446+0000 7fb88c95fb80 1 -- 172.21.15.117:0/3347173668 --> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5615bdc26580 con 0x5615bdd51e80 2024-06-03T03:26:46.447 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.446+0000 7fb87aa00700 1 --2- 172.21.15.117:0/3347173668 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x5615bdd51e80 0x5615bdd542a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:46.448 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.447+0000 7fb87aa00700 1 --2- 172.21.15.117:0/3347173668 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x5615bdd51e80 0x5615bdd542a0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7995ccc8cd548b9 server_cookie=75ca3868875388c6 in_seq=0 out_seq=0 2024-06-03T03:26:46.448 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.447+0000 7fb865600700 10 client.25511 ms_handle_connect on v2:172.21.15.183:6838/1439615083 2024-06-03T03:26:46.448 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.447+0000 7fb865600700 1 -- 172.21.15.117:0/3347173668 <== mds.0 v2:172.21.15.183:6838/1439615083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5615bdc26580 con 0x5615bdd51e80 2024-06-03T03:26:46.448 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.448+0000 7fb865600700 10 client.25511 handle_command_reply: tid=0 2024-06-03T03:26:46.448 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.448+0000 7fb88c95fb80 1 client.25511 shutdown 2024-06-03T03:26:46.449 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.448+0000 7fb88c95fb80 1 -- 172.21.15.117:0/3347173668 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5615bdc48e40 msgr2=0x5615bdcdb280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:46.449 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.448+0000 7fb88c95fb80 1 --2- 172.21.15.117:0/3347173668 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5615bdc48e40 0x5615bdcdb280 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb86c009e70 tx=0x7fb86c009f60 comp rx=0 tx=0).stop 2024-06-03T03:26:46.449 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.448+0000 7fb88c95fb80 1 -- 172.21.15.117:0/3347173668 shutdown_connections 2024-06-03T03:26:46.449 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.448+0000 7fb88c95fb80 1 --2- 172.21.15.117:0/3347173668 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x5615bdd51e80 0x5615bdd542a0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:46.449 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.448+0000 7fb88c95fb80 1 --2- 172.21.15.117:0/3347173668 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615bdcd7300 0x5615bdcdbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:46.449 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.448+0000 7fb88c95fb80 1 --2- 172.21.15.117:0/3347173668 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5615bdc48e40 0x5615bdcdb280 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:46.449 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.448+0000 7fb88c95fb80 1 --2- 172.21.15.117:0/3347173668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615bdccc5b0 0x5615bdcdb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:46.449 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.448+0000 7fb88c95fb80 1 -- 172.21.15.117:0/3347173668 >> 172.21.15.117:0/3347173668 conn(0x5615bdbf9170 msgr2=0x5615bdcd5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:26:46.450 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.448+0000 7fb865600700 0 client.25511 ms_handle_reset on v2:172.21.15.183:6838/1439615083 2024-06-03T03:26:46.450 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.449+0000 7fb88c95fb80 1 -- 172.21.15.117:0/3347173668 shutdown_connections 2024-06-03T03:26:46.450 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.449+0000 7fb88c95fb80 1 -- 172.21.15.117:0/3347173668 wait complete. 2024-06-03T03:26:46.450 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.450+0000 7fb88c95fb80 20 client.25511 trim_cache size 0 max 16384 2024-06-03T03:26:46.451 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-06-03T03:26:46.451 INFO:teuthology.orchestra.run.smithi117.stdout: "status": "no active scrubs running", 2024-06-03T03:26:46.451 INFO:teuthology.orchestra.run.smithi117.stdout: "scrubs": {} 2024-06-03T03:26:46.451 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-06-03T03:26:46.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.451+0000 7fb882000700 1 -- 172.21.15.117:0/1681343865 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fb8600a2d20 msgr2=0x7fb8600a51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:46.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.451+0000 7fb882000700 1 --2- 172.21.15.117:0/1681343865 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fb8600a2d20 0x7fb8600a51e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x5615bdcc93b0 tx=0x7fb86800f040 comp rx=0 tx=0).stop 2024-06-03T03:26:46.452 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.451+0000 7fb882000700 1 -- 172.21.15.117:0/1681343865 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb87c108500 msgr2=0x7fb87c1969d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:46.452 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.451+0000 7fb882000700 1 --2- 172.21.15.117:0/1681343865 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb87c108500 0x7fb87c1969d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb87400cd30 tx=0x7fb87400cd60 comp rx=0 tx=0).stop 2024-06-03T03:26:46.452 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.451+0000 7fb882000700 1 -- 172.21.15.117:0/1681343865 shutdown_connections 2024-06-03T03:26:46.452 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.451+0000 7fb882000700 1 --2- 172.21.15.117:0/1681343865 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fb8600a2d20 0x7fb8600a51e0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:46.452 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.451+0000 7fb882000700 1 --2- 172.21.15.117:0/1681343865 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb87c108500 0x7fb87c1969d0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:46.453 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.451+0000 7fb882000700 1 --2- 172.21.15.117:0/1681343865 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb87c06a170 0x7fb87c19d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:46.453 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.451+0000 7fb882000700 1 --2- 172.21.15.117:0/1681343865 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb87c06ab30 0x7fb87c19dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:46.453 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.451+0000 7fb882000700 1 -- 172.21.15.117:0/1681343865 >> 172.21.15.117:0/1681343865 conn(0x7fb87c078340 msgr2=0x7fb87c0ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:26:46.453 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.452+0000 7fb882000700 1 -- 172.21.15.117:0/1681343865 shutdown_connections 2024-06-03T03:26:46.453 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:46.452+0000 7fb882000700 1 -- 172.21.15.117:0/1681343865 wait complete. 2024-06-03T03:26:48.308 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:26:48 smithi117 ceph-mon[31649]: pgmap v550: 129 pgs: 129 active+clean; 680 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 341 B/s wr, 10 op/s 2024-06-03T03:26:48.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:26:48 smithi151 ceph-mon[37043]: pgmap v550: 129 pgs: 129 active+clean; 680 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 341 B/s wr, 10 op/s 2024-06-03T03:26:48.408 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.408+0000 7f5f3ca00640 1 -- 172.21.15.117:0/3078144075 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5f38102a50 msgr2=0x7f5f38102ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:48.408 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.408+0000 7f5f3ca00640 1 --2- 172.21.15.117:0/3078144075 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5f38102a50 0x7f5f38102ed0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5f24009860 tx=0x7f5f2402a340 comp rx=0 tx=0).stop 2024-06-03T03:26:48.409 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.408+0000 7f5f3ca00640 1 -- 172.21.15.117:0/3078144075 shutdown_connections 2024-06-03T03:26:48.409 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.408+0000 7f5f3ca00640 1 --2- 172.21.15.117:0/3078144075 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f38103410 0x7f5f38109ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:48.409 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.408+0000 7f5f3ca00640 1 --2- 172.21.15.117:0/3078144075 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5f38102a50 0x7f5f38102ed0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:48.409 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.408+0000 7f5f3ca00640 1 --2- 172.21.15.117:0/3078144075 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5f38101860 0x7f5f38101c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:48.409 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.408+0000 7f5f3ca00640 1 -- 172.21.15.117:0/3078144075 >> 172.21.15.117:0/3078144075 conn(0x7f5f380fcdd0 msgr2=0x7f5f380ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:26:48.409 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.408+0000 7f5f3ca00640 1 -- 172.21.15.117:0/3078144075 shutdown_connections 2024-06-03T03:26:48.409 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.408+0000 7f5f3ca00640 1 -- 172.21.15.117:0/3078144075 wait complete. 2024-06-03T03:26:48.411 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.409+0000 7f5f3ca00640 1 Processor -- start 2024-06-03T03:26:48.411 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.409+0000 7f5f3ca00640 1 -- start start 2024-06-03T03:26:48.411 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.409+0000 7f5f3ca00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5f38101860 0x7f5f38197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:48.411 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.409+0000 7f5f3ca00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f38102a50 0x7f5f381980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:48.411 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.409+0000 7f5f3ca00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5f38103410 0x7f5f3819f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:48.411 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.409+0000 7f5f3ca00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5f3810d4c0 con 0x7f5f38102a50 2024-06-03T03:26:48.411 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.409+0000 7f5f3ca00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5f3810d340 con 0x7f5f38101860 2024-06-03T03:26:48.411 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.409+0000 7f5f3ca00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5f3810d640 con 0x7f5f38103410 2024-06-03T03:26:48.412 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.410+0000 7f5f36800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5f38101860 0x7f5f38197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:48.412 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.410+0000 7f5f36800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5f38101860 0x7f5f38197b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:55212/0 (socket says 172.21.15.117:55212) 2024-06-03T03:26:48.412 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.410+0000 7f5f36800640 1 -- 172.21.15.117:0/1218010143 learned_addr learned my addr 172.21.15.117:0/1218010143 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:26:48.412 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.410+0000 7f5f37200640 1 --2- 172.21.15.117:0/1218010143 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5f38103410 0x7f5f3819f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:48.412 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.410+0000 7f5f35e00640 1 --2- 172.21.15.117:0/1218010143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f38102a50 0x7f5f381980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:48.412 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.410+0000 7f5f36800640 1 -- 172.21.15.117:0/1218010143 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5f38103410 msgr2=0x7f5f3819f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:48.412 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.410+0000 7f5f36800640 1 --2- 172.21.15.117:0/1218010143 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5f38103410 0x7f5f3819f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:48.412 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.410+0000 7f5f36800640 1 -- 172.21.15.117:0/1218010143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f38102a50 msgr2=0x7f5f381980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:48.412 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.410+0000 7f5f36800640 1 --2- 172.21.15.117:0/1218010143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f38102a50 0x7f5f381980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:48.412 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.410+0000 7f5f36800640 1 -- 172.21.15.117:0/1218010143 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f3819f7b0 con 0x7f5f38101860 2024-06-03T03:26:48.412 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.410+0000 7f5f35e00640 1 --2- 172.21.15.117:0/1218010143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f38102a50 0x7f5f381980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T03:26:48.412 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.411+0000 7f5f36800640 1 --2- 172.21.15.117:0/1218010143 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5f38101860 0x7f5f38197b90 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5f2800e9c0 tx=0x7f5f2800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:26:48.413 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.411+0000 7f5f2f400640 1 -- 172.21.15.117:0/1218010143 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f2800cd80 con 0x7f5f38101860 2024-06-03T03:26:48.413 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.411+0000 7f5f3ca00640 1 -- 172.21.15.117:0/1218010143 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f3819faa0 con 0x7f5f38101860 2024-06-03T03:26:48.413 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.411+0000 7f5f2f400640 1 -- 172.21.15.117:0/1218010143 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f280035f0 con 0x7f5f38101860 2024-06-03T03:26:48.413 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.411+0000 7f5f3ca00640 1 -- 172.21.15.117:0/1218010143 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f381a0010 con 0x7f5f38101860 2024-06-03T03:26:48.413 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.411+0000 7f5f2f400640 1 -- 172.21.15.117:0/1218010143 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f28010690 con 0x7f5f38101860 2024-06-03T03:26:48.414 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.414+0000 7f5f40219740 1 Processor -- start 2024-06-03T03:26:48.414 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.414+0000 7f5f40219740 1 -- start start 2024-06-03T03:26:48.415 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.414+0000 7f5f40219740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f175bc1a0 0x559f175bc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:48.415 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.414+0000 7f5f40219740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559f175bcb90 0x559f175c1690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:48.415 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.414+0000 7f5f40219740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f175c1d00 0x559f175c4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:48.416 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.414+0000 7f5f40219740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559f1751b420 con 0x559f175c1d00 2024-06-03T03:26:48.416 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.414+0000 7f5f40219740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559f174c2c50 con 0x559f175bcb90 2024-06-03T03:26:48.416 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.414+0000 7f5f40219740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559f175173e0 con 0x559f175bc1a0 2024-06-03T03:26:48.416 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.415+0000 7f5f35e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f175c1d00 0x559f175c4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:48.416 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.415+0000 7f5f35e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f175c1d00 0x559f175c4190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:57270/0 (socket says 172.21.15.117:57270) 2024-06-03T03:26:48.416 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.415+0000 7f5f35e00640 1 -- 172.21.15.117:0/3376651703 learned_addr learned my addr 172.21.15.117:0/3376651703 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:26:48.417 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.415+0000 7f5f37200640 1 --2- 172.21.15.117:0/3376651703 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f175bc1a0 0x559f175bc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:48.417 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.415+0000 7f5f35e00640 1 -- 172.21.15.117:0/3376651703 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f175bc1a0 msgr2=0x559f175bc5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:48.417 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.415+0000 7f5f35e00640 1 --2- 172.21.15.117:0/3376651703 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f175bc1a0 0x559f175bc5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:48.417 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.415+0000 7f5f35e00640 1 -- 172.21.15.117:0/3376651703 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559f175bcb90 msgr2=0x559f175c1690 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-03T03:26:48.417 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.415+0000 7f5f35e00640 1 --2- 172.21.15.117:0/3376651703 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559f175bcb90 0x559f175c1690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:48.417 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.415+0000 7f5f35e00640 1 -- 172.21.15.117:0/3376651703 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f1722f210 con 0x559f175c1d00 2024-06-03T03:26:48.417 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.416+0000 7f5f35e00640 1 --2- 172.21.15.117:0/3376651703 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f175c1d00 0x559f175c4190 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5f24009860 tx=0x7f5f24005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62a87ca23c7c9a45 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:26:48.417 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.416+0000 7f5f23400640 1 -- 172.21.15.117:0/3376651703 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f24037040 con 0x559f175c1d00 2024-06-03T03:26:48.417 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.416+0000 7f5f2f400640 1 -- 172.21.15.117:0/1218010143 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f5f28004020 con 0x7f5f38101860 2024-06-03T03:26:48.417 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.416+0000 7f5f23400640 1 -- 172.21.15.117:0/3376651703 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f24004030 con 0x559f175c1d00 2024-06-03T03:26:48.418 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.416+0000 7f5f23400640 1 -- 172.21.15.117:0/3376651703 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f240055e0 con 0x559f175c1d00 2024-06-03T03:26:48.418 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.417+0000 7f5f40219740 1 -- 172.21.15.117:0/3376651703 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f175c1d00 msgr2=0x559f175c4190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:48.418 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.417+0000 7f5f40219740 1 --2- 172.21.15.117:0/3376651703 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f175c1d00 0x559f175c4190 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5f24009860 tx=0x7f5f24005d90 comp rx=0 tx=0).stop 2024-06-03T03:26:48.418 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.417+0000 7f5f40219740 1 -- 172.21.15.117:0/3376651703 shutdown_connections 2024-06-03T03:26:48.418 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.417+0000 7f5f40219740 1 --2- 172.21.15.117:0/3376651703 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f175c1d00 0x559f175c4190 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:48.418 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.417+0000 7f5f40219740 1 --2- 172.21.15.117:0/3376651703 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559f175bcb90 0x559f175c1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:48.419 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.417+0000 7f5f40219740 1 --2- 172.21.15.117:0/3376651703 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f175bc1a0 0x559f175bc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:48.419 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.417+0000 7f5f40219740 1 -- 172.21.15.117:0/3376651703 >> 172.21.15.117:0/3376651703 conn(0x559f1757ff80 msgr2=0x559f175b7c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:26:48.419 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.418+0000 7f5f2f400640 1 --2- 172.21.15.117:0/1218010143 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f5f1c0a2c80 0x7f5f1c0a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:48.419 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.419+0000 7f5f2f400640 1 -- 172.21.15.117:0/1218010143 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f5f280cf7d0 con 0x7f5f38101860 2024-06-03T03:26:48.421 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.419+0000 7f5f35e00640 1 --2- 172.21.15.117:0/1218010143 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f5f1c0a2c80 0x7f5f1c0a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:48.421 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.419+0000 7f5f40219740 1 -- 172.21.15.117:0/3376651703 shutdown_connections 2024-06-03T03:26:48.421 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.420+0000 7f5f35e00640 1 --2- 172.21.15.117:0/1218010143 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f5f1c0a2c80 0x7f5f1c0a5140 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5f24005fd0 tx=0x7f5f2402c040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:26:48.421 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.420+0000 7f5f40219740 1 -- 172.21.15.117:0/3376651703 wait complete. 2024-06-03T03:26:48.422 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.420+0000 7f5f40219740 1 -- start start 2024-06-03T03:26:48.422 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.421+0000 7f5f40219740 1 Processor -- start 2024-06-03T03:26:48.423 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.421+0000 7f5f40219740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559f175bc1a0 0x559f175c6480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:48.423 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.421+0000 7f5f40219740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f175bcb90 0x559f175c69c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:48.423 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.421+0000 7f5f40219740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f175c1d00 0x559f175cda00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:48.424 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.421+0000 7f5f40219740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559f175c4c50 con 0x559f175bcb90 2024-06-03T03:26:48.424 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.421+0000 7f5f40219740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559f174d60b0 con 0x559f175bc1a0 2024-06-03T03:26:48.424 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.421+0000 7f5f40219740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559f175c4dd0 con 0x559f175c1d00 2024-06-03T03:26:48.424 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.421+0000 7f5f40219740 10 client.0 _ll_register_callbacks cb 0x559f174c1f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:26:48.424 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.421+0000 7f5f37200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f175c1d00 0x559f175cda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:48.424 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.421+0000 7f5f37200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f175c1d00 0x559f175cda00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:60238/0 (socket says 172.21.15.117:60238) 2024-06-03T03:26:48.424 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.421+0000 7f5f37200640 1 -- 172.21.15.117:0/3022042361 learned_addr learned my addr 172.21.15.117:0/3022042361 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:26:48.424 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.421+0000 7f5f36800640 1 --2- 172.21.15.117:0/3022042361 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559f175bc1a0 0x559f175c6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:48.424 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.422+0000 7f5f35e00640 1 --2- 172.21.15.117:0/3022042361 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f175bcb90 0x559f175c69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:48.424 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.422+0000 7f5f37200640 1 -- 172.21.15.117:0/3022042361 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559f175bc1a0 msgr2=0x559f175c6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:48.424 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.422+0000 7f5f37200640 1 --2- 172.21.15.117:0/3022042361 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559f175bc1a0 0x559f175c6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:48.425 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.422+0000 7f5f37200640 1 -- 172.21.15.117:0/3022042361 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f175bcb90 msgr2=0x559f175c69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:48.425 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.422+0000 7f5f37200640 1 --2- 172.21.15.117:0/3022042361 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f175bcb90 0x559f175c69c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:48.425 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.422+0000 7f5f37200640 1 -- 172.21.15.117:0/3022042361 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f172fb620 con 0x559f175c1d00 2024-06-03T03:26:48.425 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.422+0000 7f5f35e00640 1 --2- 172.21.15.117:0/3022042361 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f175bcb90 0x559f175c69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T03:26:48.425 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.422+0000 7f5f37200640 1 --2- 172.21.15.117:0/3022042361 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f175c1d00 0x559f175cda00 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5f30009fd0 tx=0x7f5f3000d7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0f8fc01b0f0f35d server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:26:48.425 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.422+0000 7f5f17e00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-06-03T03:26:48.425 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.423+0000 7f5f17e00640 1 -- 172.21.15.117:0/3022042361 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f3001b030 con 0x559f175c1d00 2024-06-03T03:26:48.425 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.423+0000 7f5f17e00640 1 -- 172.21.15.117:0/3022042361 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f5f30004b20 con 0x559f175c1d00 2024-06-03T03:26:48.425 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.423+0000 7f5f40219740 1 -- 172.21.15.117:0/3022042361 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f24004030 con 0x559f175c1d00 2024-06-03T03:26:48.425 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.423+0000 7f5f17e00640 1 -- 172.21.15.117:0/3022042361 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5f30025050 con 0x559f175c1d00 2024-06-03T03:26:48.427 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.423+0000 7f5f40219740 10 client.25558 fetch_fsmap learned FSMap version 31 2024-06-03T03:26:48.427 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.423+0000 7f5f40219740 1 -- 172.21.15.117:0/3022042361 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x559f175b5ff0 con 0x559f175c1d00 2024-06-03T03:26:48.427 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.424+0000 7f5f17e00640 1 -- 172.21.15.117:0/3022042361 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f24004030 con 0x559f175c1d00 2024-06-03T03:26:48.428 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.424+0000 7f5f17e00640 1 -- 172.21.15.117:0/3022042361 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f5f3000b040 con 0x559f175c1d00 2024-06-03T03:26:48.428 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.424+0000 7f5f40219740 10 client.25558 fetch_fsmap finished waiting for FSMap version 31 2024-06-03T03:26:48.428 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.424+0000 7f5f40219740 10 client.25558 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-03T03:26:48.428 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.424+0000 7f5f40219740 20 client.25558 populate_metadata read hostname 'smithi117' 2024-06-03T03:26:48.428 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.424+0000 7f5f40219740 1 --2- 172.21.15.117:0/3022042361 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x559f175d6a60 0x559f175d8e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:48.428 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.424+0000 7f5f40219740 4 client.25558 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T03:26:48.428 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.424+0000 7f5f40219740 1 -- 172.21.15.117:0/3022042361 --> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f17528b50 con 0x559f175d6a60 2024-06-03T03:26:48.428 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.425+0000 7f5f36800640 1 --2- 172.21.15.117:0/3022042361 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x559f175d6a60 0x559f175d8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:48.428 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.425+0000 7f5f36800640 1 --2- 172.21.15.117:0/3022042361 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x559f175d6a60 0x559f175d8e70 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8bbda343ef5f588a server_cookie=83cf672bbbc167d2 in_seq=0 out_seq=0 2024-06-03T03:26:48.428 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.425+0000 7f5f17e00640 10 client.25558 ms_handle_connect on v2:172.21.15.183:6838/1439615083 2024-06-03T03:26:48.428 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.426+0000 7f5f17e00640 1 -- 172.21.15.117:0/3022042361 <== mds.0 v2:172.21.15.183:6838/1439615083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559f17528b50 con 0x559f175d6a60 2024-06-03T03:26:48.428 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.426+0000 7f5f17e00640 10 client.25558 handle_command_reply: tid=0 2024-06-03T03:26:48.428 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.426+0000 7f5f40219740 1 client.25558 shutdown 2024-06-03T03:26:48.429 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.426+0000 7f5f40219740 1 -- 172.21.15.117:0/3022042361 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f175c1d00 msgr2=0x559f175cda00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:48.429 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.426+0000 7f5f40219740 1 --2- 172.21.15.117:0/3022042361 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f175c1d00 0x559f175cda00 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5f30009fd0 tx=0x7f5f3000d7b0 comp rx=0 tx=0).stop 2024-06-03T03:26:48.429 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.426+0000 7f5f40219740 1 -- 172.21.15.117:0/3022042361 shutdown_connections 2024-06-03T03:26:48.429 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.426+0000 7f5f40219740 1 --2- 172.21.15.117:0/3022042361 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x559f175d6a60 0x559f175d8e70 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:48.429 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.427+0000 7f5f40219740 1 --2- 172.21.15.117:0/3022042361 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f175c1d00 0x559f175cda00 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:48.429 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.427+0000 7f5f40219740 1 --2- 172.21.15.117:0/3022042361 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f175bcb90 0x559f175c69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:48.429 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.427+0000 7f5f40219740 1 --2- 172.21.15.117:0/3022042361 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559f175bc1a0 0x559f175c6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:48.429 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.427+0000 7f5f17e00640 0 client.25558 ms_handle_reset on v2:172.21.15.183:6838/1439615083 2024-06-03T03:26:48.429 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.427+0000 7f5f40219740 1 -- 172.21.15.117:0/3022042361 >> 172.21.15.117:0/3022042361 conn(0x559f1757ff80 msgr2=0x559f175b2970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:26:48.429 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.427+0000 7f5f40219740 1 -- 172.21.15.117:0/3022042361 shutdown_connections 2024-06-03T03:26:48.429 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.427+0000 7f5f40219740 1 -- 172.21.15.117:0/3022042361 wait complete. 2024-06-03T03:26:48.429 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.427+0000 7f5f40219740 20 client.25558 trim_cache size 0 max 16384 2024-06-03T03:26:48.443 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.443+0000 7f5f40219740 1 Processor -- start 2024-06-03T03:26:48.444 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.443+0000 7f5f40219740 1 -- start start 2024-06-03T03:26:48.444 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.443+0000 7f5f40219740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559f175bc1a0 0x559f175b76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:48.444 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.443+0000 7f5f40219740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f175bcb90 0x559f175cd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:48.444 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.443+0000 7f5f40219740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f175c1d00 0x559f175c5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:48.444 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.443+0000 7f5f40219740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5f1800fbd0 con 0x559f175bcb90 2024-06-03T03:26:48.444 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.443+0000 7f5f40219740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5f18011710 con 0x559f175bc1a0 2024-06-03T03:26:48.444 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.443+0000 7f5f40219740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5f180102a0 con 0x559f175c1d00 2024-06-03T03:26:48.445 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.444+0000 7f5f36800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559f175bc1a0 0x559f175b76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:48.446 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.444+0000 7f5f36800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559f175bc1a0 0x559f175b76a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:55226/0 (socket says 172.21.15.117:55226) 2024-06-03T03:26:48.446 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.444+0000 7f5f36800640 1 -- 172.21.15.117:0/723298235 learned_addr learned my addr 172.21.15.117:0/723298235 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:26:48.446 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.444+0000 7f5f37200640 1 --2- 172.21.15.117:0/723298235 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f175c1d00 0x559f175c5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:48.446 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.444+0000 7f5f35e00640 1 --2- 172.21.15.117:0/723298235 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f175bcb90 0x559f175cd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:48.446 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.444+0000 7f5f37200640 1 -- 172.21.15.117:0/723298235 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559f175bc1a0 msgr2=0x559f175b76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:48.446 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.444+0000 7f5f37200640 1 --2- 172.21.15.117:0/723298235 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559f175bc1a0 0x559f175b76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:48.446 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.444+0000 7f5f37200640 1 -- 172.21.15.117:0/723298235 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f175bcb90 msgr2=0x559f175cd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:48.446 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.444+0000 7f5f37200640 1 --2- 172.21.15.117:0/723298235 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f175bcb90 0x559f175cd9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:48.446 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.444+0000 7f5f37200640 1 -- 172.21.15.117:0/723298235 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f175b88d0 con 0x559f175c1d00 2024-06-03T03:26:48.446 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.444+0000 7f5f35e00640 1 --2- 172.21.15.117:0/723298235 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f175bcb90 0x559f175cd9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T03:26:48.447 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.445+0000 7f5f37200640 1 --2- 172.21.15.117:0/723298235 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f175c1d00 0x559f175c5770 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5f3000e820 tx=0x7f5f30005bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=518a3ab44e00afea server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:26:48.447 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.445+0000 7f5f22a00640 1 -- 172.21.15.117:0/723298235 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f3001b040 con 0x559f175c1d00 2024-06-03T03:26:48.447 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.445+0000 7f5f22a00640 1 -- 172.21.15.117:0/723298235 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f300096d0 con 0x559f175c1d00 2024-06-03T03:26:48.447 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.445+0000 7f5f22a00640 1 -- 172.21.15.117:0/723298235 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f300052a0 con 0x559f175c1d00 2024-06-03T03:26:48.450 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.446+0000 7f5f40219740 1 -- 172.21.15.117:0/723298235 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f175c1d00 msgr2=0x559f175c5770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:48.450 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.446+0000 7f5f40219740 1 --2- 172.21.15.117:0/723298235 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f175c1d00 0x559f175c5770 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5f3000e820 tx=0x7f5f30005bd0 comp rx=0 tx=0).stop 2024-06-03T03:26:48.450 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.446+0000 7f5f40219740 1 -- 172.21.15.117:0/723298235 shutdown_connections 2024-06-03T03:26:48.450 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.446+0000 7f5f40219740 1 --2- 172.21.15.117:0/723298235 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f175c1d00 0x559f175c5770 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:48.450 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.446+0000 7f5f40219740 1 --2- 172.21.15.117:0/723298235 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f175bcb90 0x559f175cd9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:48.451 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.446+0000 7f5f40219740 1 --2- 172.21.15.117:0/723298235 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559f175bc1a0 0x559f175b76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:48.451 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.446+0000 7f5f40219740 1 -- 172.21.15.117:0/723298235 >> 172.21.15.117:0/723298235 conn(0x559f1757ff80 msgr2=0x559f175c1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:26:48.451 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.446+0000 7f5f40219740 1 -- 172.21.15.117:0/723298235 shutdown_connections 2024-06-03T03:26:48.451 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.446+0000 7f5f40219740 1 -- 172.21.15.117:0/723298235 wait complete. 2024-06-03T03:26:48.451 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.447+0000 7f5f40219740 1 -- start start 2024-06-03T03:26:48.451 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.447+0000 7f5f40219740 1 Processor -- start 2024-06-03T03:26:48.451 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.447+0000 7f5f40219740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f175bc1a0 0x559f175ca9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:48.451 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.447+0000 7f5f40219740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f175bcb90 0x559f175caf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:48.451 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.447+0000 7f5f40219740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559f175c1d00 0x559f1763f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:48.451 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.447+0000 7f5f40219740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5f18011040 con 0x559f175bcb90 2024-06-03T03:26:48.451 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.447+0000 7f5f40219740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5f18010970 con 0x559f175c1d00 2024-06-03T03:26:48.452 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.447+0000 7f5f40219740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5f1800dbb0 con 0x559f175bc1a0 2024-06-03T03:26:48.452 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.447+0000 7f5f40219740 10 client.0 _ll_register_callbacks cb 0x7f5f30009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:26:48.452 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.448+0000 7f5f36800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f175bc1a0 0x559f175ca9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:48.452 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.448+0000 7f5f37200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559f175c1d00 0x559f1763f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:48.452 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.448+0000 7f5f36800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f175bc1a0 0x559f175ca9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:60260/0 (socket says 172.21.15.117:60260) 2024-06-03T03:26:48.452 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.448+0000 7f5f36800640 1 -- 172.21.15.117:0/214374620 learned_addr learned my addr 172.21.15.117:0/214374620 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:26:48.452 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.448+0000 7f5f35e00640 1 --2- 172.21.15.117:0/214374620 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f175bcb90 0x559f175caf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:48.452 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.448+0000 7f5f36800640 1 -- 172.21.15.117:0/214374620 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559f175c1d00 msgr2=0x559f1763f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:48.452 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.448+0000 7f5f36800640 1 --2- 172.21.15.117:0/214374620 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559f175c1d00 0x559f1763f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:48.452 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.448+0000 7f5f36800640 1 -- 172.21.15.117:0/214374620 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f175bcb90 msgr2=0x559f175caf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:48.452 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.448+0000 7f5f36800640 1 --2- 172.21.15.117:0/214374620 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f175bcb90 0x559f175caf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:48.452 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.448+0000 7f5f36800640 1 -- 172.21.15.117:0/214374620 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f1763fc80 con 0x559f175bc1a0 2024-06-03T03:26:48.453 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.448+0000 7f5f35e00640 1 --2- 172.21.15.117:0/214374620 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f175bcb90 0x559f175caf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T03:26:48.453 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.448+0000 7f5f36800640 1 --2- 172.21.15.117:0/214374620 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f175bc1a0 0x559f175ca9c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5f280980a0 tx=0x7f5f2809bd00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46743879d6821d6 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:26:48.453 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.448+0000 7f5f17e00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-06-03T03:26:48.453 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.449+0000 7f5f17e00640 1 -- 172.21.15.117:0/214374620 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f280ccff0 con 0x559f175bc1a0 2024-06-03T03:26:48.453 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.449+0000 7f5f17e00640 1 -- 172.21.15.117:0/214374620 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f5f28086040 con 0x559f175bc1a0 2024-06-03T03:26:48.453 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.449+0000 7f5f40219740 1 -- 172.21.15.117:0/214374620 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f300096d0 con 0x559f175bc1a0 2024-06-03T03:26:48.453 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.449+0000 7f5f17e00640 1 -- 172.21.15.117:0/214374620 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5f280982b0 con 0x559f175bc1a0 2024-06-03T03:26:48.453 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.449+0000 7f5f40219740 10 client.25564 fetch_fsmap learned FSMap version 31 2024-06-03T03:26:48.453 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.449+0000 7f5f40219740 1 -- 172.21.15.117:0/214374620 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x559f17641060 con 0x559f175bc1a0 2024-06-03T03:26:48.453 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.449+0000 7f5f17e00640 1 -- 172.21.15.117:0/214374620 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f300096d0 con 0x559f175bc1a0 2024-06-03T03:26:48.453 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.449+0000 7f5f17e00640 1 -- 172.21.15.117:0/214374620 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f5f2809bea0 con 0x559f175bc1a0 2024-06-03T03:26:48.453 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.449+0000 7f5f40219740 10 client.25564 fetch_fsmap finished waiting for FSMap version 31 2024-06-03T03:26:48.453 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.449+0000 7f5f40219740 10 client.25564 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-03T03:26:48.454 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.449+0000 7f5f40219740 20 client.25564 populate_metadata read hostname 'smithi117' 2024-06-03T03:26:48.454 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.449+0000 7f5f40219740 1 --2- 172.21.15.117:0/214374620 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x559f17641a30 0x559f17643e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:48.454 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.449+0000 7f5f40219740 4 client.25564 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-03T03:26:48.454 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.449+0000 7f5f40219740 1 -- 172.21.15.117:0/214374620 --> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559f17522ea0 con 0x559f17641a30 2024-06-03T03:26:48.454 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.450+0000 7f5f35e00640 1 --2- 172.21.15.117:0/214374620 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x559f17641a30 0x559f17643e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:48.454 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.450+0000 7f5f17e00640 10 client.25564 ms_handle_connect on v2:172.21.15.183:6838/1439615083 2024-06-03T03:26:48.454 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.450+0000 7f5f35e00640 1 --2- 172.21.15.117:0/214374620 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x559f17641a30 0x559f17643e50 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68b39d052d46a616 server_cookie=5a53bcbbf5ca2598 in_seq=0 out_seq=0 2024-06-03T03:26:48.455 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:{ 2024-06-03T03:26:48.456 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "return_code": 0 2024-06-03T03:26:48.456 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:} 2024-06-03T03:26:48.457 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.454+0000 7f5f17e00640 1 -- 172.21.15.117:0/214374620 <== mds.0 v2:172.21.15.183:6838/1439615083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x559f17522ea0 con 0x559f17641a30 2024-06-03T03:26:48.457 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.454+0000 7f5f17e00640 10 client.25564 handle_command_reply: tid=0 2024-06-03T03:26:48.457 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.454+0000 7f5f40219740 1 client.25564 shutdown 2024-06-03T03:26:48.457 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.455+0000 7f5f40219740 1 -- 172.21.15.117:0/214374620 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f175bc1a0 msgr2=0x559f175ca9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:48.457 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.455+0000 7f5f40219740 1 --2- 172.21.15.117:0/214374620 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f175bc1a0 0x559f175ca9c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5f280980a0 tx=0x7f5f2809bd00 comp rx=0 tx=0).stop 2024-06-03T03:26:48.457 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.455+0000 7f5f40219740 1 -- 172.21.15.117:0/214374620 shutdown_connections 2024-06-03T03:26:48.457 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.455+0000 7f5f40219740 1 --2- 172.21.15.117:0/214374620 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x559f17641a30 0x559f17643e50 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:48.457 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.455+0000 7f5f40219740 1 --2- 172.21.15.117:0/214374620 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559f175c1d00 0x559f1763f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:48.457 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.455+0000 7f5f40219740 1 --2- 172.21.15.117:0/214374620 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f175bcb90 0x559f175caf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:48.457 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.455+0000 7f5f17e00640 0 client.25564 ms_handle_reset on v2:172.21.15.183:6838/1439615083 2024-06-03T03:26:48.458 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.455+0000 7f5f40219740 1 --2- 172.21.15.117:0/214374620 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f175bc1a0 0x559f175ca9c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:48.458 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.455+0000 7f5f40219740 1 -- 172.21.15.117:0/214374620 >> 172.21.15.117:0/214374620 conn(0x559f1757ff80 msgr2=0x559f175c1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:26:48.458 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.455+0000 7f5f40219740 1 -- 172.21.15.117:0/214374620 shutdown_connections 2024-06-03T03:26:48.458 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.455+0000 7f5f40219740 1 -- 172.21.15.117:0/214374620 wait complete. 2024-06-03T03:26:48.458 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.455+0000 7f5f40219740 20 client.25564 trim_cache size 0 max 16384 2024-06-03T03:26:48.458 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.456+0000 7f5f3ca00640 1 -- 172.21.15.117:0/1218010143 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f5f1c0a2c80 msgr2=0x7f5f1c0a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:48.458 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.456+0000 7f5f3ca00640 1 --2- 172.21.15.117:0/1218010143 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f5f1c0a2c80 0x7f5f1c0a5140 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5f24005fd0 tx=0x7f5f2402c040 comp rx=0 tx=0).stop 2024-06-03T03:26:48.458 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.456+0000 7f5f3ca00640 1 -- 172.21.15.117:0/1218010143 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5f38101860 msgr2=0x7f5f38197b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:48.458 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.456+0000 7f5f3ca00640 1 --2- 172.21.15.117:0/1218010143 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5f38101860 0x7f5f38197b90 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5f2800e9c0 tx=0x7f5f2800ee80 comp rx=0 tx=0).stop 2024-06-03T03:26:48.458 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.456+0000 7f5f3ca00640 1 -- 172.21.15.117:0/1218010143 shutdown_connections 2024-06-03T03:26:48.458 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.456+0000 7f5f3ca00640 1 --2- 172.21.15.117:0/1218010143 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5f38103410 0x7f5f3819f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:48.458 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.456+0000 7f5f3ca00640 1 --2- 172.21.15.117:0/1218010143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f38102a50 0x7f5f381980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:48.458 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.456+0000 7f5f3ca00640 1 --2- 172.21.15.117:0/1218010143 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f5f1c0a2c80 0x7f5f1c0a5140 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:48.459 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.456+0000 7f5f3ca00640 1 --2- 172.21.15.117:0/1218010143 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5f38101860 0x7f5f38197b90 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:48.459 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.456+0000 7f5f3ca00640 1 -- 172.21.15.117:0/1218010143 >> 172.21.15.117:0/1218010143 conn(0x7f5f380fcdd0 msgr2=0x7f5f38105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:26:48.459 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.456+0000 7f5f3ca00640 1 -- 172.21.15.117:0/1218010143 shutdown_connections 2024-06-03T03:26:48.459 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:26:48.457+0000 7f5f3ca00640 1 -- 172.21.15.117:0/1218010143 wait complete. 2024-06-03T03:26:48.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:26:48 smithi183 ceph-mon[37326]: pgmap v550: 129 pgs: 129 active+clean; 680 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 341 B/s wr, 10 op/s 2024-06-03T03:26:48.880 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-03T03:26:48.881 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-03T03:26:49.332 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:26:50.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:26:50 smithi151 ceph-mon[37043]: pgmap v551: 129 pgs: 129 active+clean; 680 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 341 B/s wr, 13 op/s 2024-06-03T03:26:50.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:26:50 smithi117 ceph-mon[31649]: pgmap v551: 129 pgs: 129 active+clean; 680 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 341 B/s wr, 13 op/s 2024-06-03T03:26:50.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:26:50 smithi183 ceph-mon[37326]: pgmap v551: 129 pgs: 129 active+clean; 680 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 341 B/s wr, 13 op/s 2024-06-03T03:26:51.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:26:51 smithi151 ceph-mon[37043]: pgmap v552: 129 pgs: 129 active+clean; 680 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 85 B/s wr, 10 op/s 2024-06-03T03:26:51.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:26:51 smithi117 ceph-mon[31649]: pgmap v552: 129 pgs: 129 active+clean; 680 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 85 B/s wr, 10 op/s 2024-06-03T03:26:51.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:26:51 smithi183 ceph-mon[37326]: pgmap v552: 129 pgs: 129 active+clean; 680 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 85 B/s wr, 10 op/s 2024-06-03T03:26:52.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:52.210+0000 7f6e25c00700 1 -- 172.21.15.117:0/3119383810 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e200751e0 msgr2=0x7f6e2010e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:52.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:52.210+0000 7f6e25c00700 1 --2- 172.21.15.117:0/3119383810 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e200751e0 0x7f6e2010e6c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c009990 tx=0x7f6e0c009ca0 comp rx=0 tx=0).stop 2024-06-03T03:26:52.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:52.210+0000 7f6e25c00700 1 -- 172.21.15.117:0/3119383810 shutdown_connections 2024-06-03T03:26:52.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:52.210+0000 7f6e25c00700 1 --2- 172.21.15.117:0/3119383810 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6e2010ec00 0x7f6e20111070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:52.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:52.210+0000 7f6e25c00700 1 --2- 172.21.15.117:0/3119383810 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6e200748c0 0x7f6e20074ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:52.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:52.210+0000 7f6e25c00700 1 --2- 172.21.15.117:0/3119383810 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e200751e0 0x7f6e2010e6c0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:52.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:52.210+0000 7f6e25c00700 1 -- 172.21.15.117:0/3119383810 >> 172.21.15.117:0/3119383810 conn(0x7f6e200fdf10 msgr2=0x7f6e20100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:26:52.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:52.210+0000 7f6e25c00700 1 -- 172.21.15.117:0/3119383810 shutdown_connections 2024-06-03T03:26:52.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:52.210+0000 7f6e25c00700 1 -- 172.21.15.117:0/3119383810 wait complete. 2024-06-03T03:26:52.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:52.212+0000 7f6e25c00700 1 Processor -- start 2024-06-03T03:26:52.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:52.212+0000 7f6e25c00700 1 -- start start 2024-06-03T03:26:52.214 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:52.213+0000 7f6e25c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e200748c0 0x7f6e2019ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:52.214 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:52.213+0000 7f6e25c00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6e200751e0 0x7f6e2019b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:52.214 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:52.213+0000 7f6e25c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6e2010ec00 0x7f6e2019f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:52.214 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:52.213+0000 7f6e25c00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6e20114940 con 0x7f6e200748c0 2024-06-03T03:26:52.214 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:52.213+0000 7f6e25c00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6e201147c0 con 0x7f6e2010ec00 2024-06-03T03:26:52.214 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:52.213+0000 7f6e25c00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6e20114ac0 con 0x7f6e200751e0 2024-06-03T03:26:52.214 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:52.213+0000 7f6e1f400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6e200751e0 0x7f6e2019b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:52.214 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:52.213+0000 7f6e1fe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e200748c0 0x7f6e2019ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:52.214 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:52.213+0000 7f6e1f400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6e200751e0 0x7f6e2019b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:60276/0 (socket says 172.21.15.117:60276) 2024-06-03T03:26:52.214 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:52.213+0000 7f6e1f400700 1 -- 172.21.15.117:0/2633987060 learned_addr learned my addr 172.21.15.117:0/2633987060 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:26:52.214 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:52.213+0000 7f6e25200700 1 --2- 172.21.15.117:0/2633987060 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6e2010ec00 0x7f6e2019f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:52.215 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:52.214+0000 7f6e1f400700 1 -- 172.21.15.117:0/2633987060 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6e2010ec00 msgr2=0x7f6e2019f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:52.215 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:52.214+0000 7f6e1f400700 1 --2- 172.21.15.117:0/2633987060 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6e2010ec00 0x7f6e2019f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:52.216 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:52.214+0000 7f6e1f400700 1 -- 172.21.15.117:0/2633987060 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e200748c0 msgr2=0x7f6e2019ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:52.216 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:52.214+0000 7f6e1f400700 1 --2- 172.21.15.117:0/2633987060 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e200748c0 0x7f6e2019ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:52.216 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:52.214+0000 7f6e1f400700 1 -- 172.21.15.117:0/2633987060 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e2019fe40 con 0x7f6e200751e0 2024-06-03T03:26:52.216 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:52.214+0000 7f6e1f400700 1 --2- 172.21.15.117:0/2633987060 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6e200751e0 0x7f6e2019b370 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c000c00 tx=0x7f6e0c00bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:26:52.216 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:52.215+0000 7f6e1cc00700 1 -- 172.21.15.117:0/2633987060 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0c027070 con 0x7f6e200751e0 2024-06-03T03:26:52.216 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:52.215+0000 7f6e1cc00700 1 -- 172.21.15.117:0/2633987060 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e0c0056d0 con 0x7f6e200751e0 2024-06-03T03:26:52.216 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:52.215+0000 7f6e25c00700 1 -- 172.21.15.117:0/2633987060 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e201a00d0 con 0x7f6e200751e0 2024-06-03T03:26:52.216 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:52.215+0000 7f6e25c00700 1 -- 172.21.15.117:0/2633987060 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e201a79b0 con 0x7f6e200751e0 2024-06-03T03:26:52.216 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:52.215+0000 7f6e1cc00700 1 -- 172.21.15.117:0/2633987060 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0c022e10 con 0x7f6e200751e0 2024-06-03T03:26:52.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:52.216+0000 7f6e25c00700 1 -- 172.21.15.117:0/2633987060 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e00005330 con 0x7f6e200751e0 2024-06-03T03:26:52.219 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:52.218+0000 7f6e1cc00700 1 -- 172.21.15.117:0/2633987060 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f6e0c02a070 con 0x7f6e200751e0 2024-06-03T03:26:52.220 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:52.220+0000 7f6e1cc00700 1 --2- 172.21.15.117:0/2633987060 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f6e040a2c20 0x7f6e040a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:52.220 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:52.220+0000 7f6e1cc00700 1 -- 172.21.15.117:0/2633987060 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f6e0c0d03a0 con 0x7f6e200751e0 2024-06-03T03:26:52.220 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:52.220+0000 7f6e1fe00700 1 --2- 172.21.15.117:0/2633987060 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f6e040a2c20 0x7f6e040a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:52.221 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:52.221+0000 7f6e1fe00700 1 --2- 172.21.15.117:0/2633987060 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f6e040a2c20 0x7f6e040a50e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6e10009c60 tx=0x7f6e10009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:26:52.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:52.221+0000 7f6e1cc00700 1 -- 172.21.15.117:0/2633987060 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6e0c013070 con 0x7f6e200751e0 2024-06-03T03:26:52.497 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:52.496+0000 7f6e25c00700 1 -- 172.21.15.117:0/2633987060 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6e00005620 con 0x7f6e200751e0 2024-06-03T03:26:52.498 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:52.497+0000 7f6e1cc00700 1 -- 172.21.15.117:0/2633987060 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11955 (secure 0 0 0) 0x7f6e0c09cfa0 con 0x7f6e200751e0 2024-06-03T03:26:52.498 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T03:26:52.499 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3353001119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3353001119},{"type":"v1","addr":"172.21.15.117:6837","nonce":3353001119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14388,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6839/405782304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":405782304},{"type":"v1","addr":"172.21.15.117:6839","nonce":405782304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-03T03:19:44.629822+0000","modified":"2024-06-03T03:25:11.576498+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24517,"mds_1":24512,"mds_2":24503,"mds_3":24499,"mds_4":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24431":{"gid":24431,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6833/3355662416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3355662416},{"type":"v1","addr":"172.21.15.151:6833","nonce":3355662416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/2429725085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2429725085},{"type":"v1","addr":"172.21.15.183:6833","nonce":2429725085}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/31153579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":31153579},{"type":"v1","addr":"172.21.15.151:6835","nonce":31153579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"j","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/2884460088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2884460088},{"type":"v1","addr":"172.21.15.117:6835","nonce":2884460088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/2392407848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":2392407848},{"type":"v1","addr":"172.21.15.151:6837","nonce":2392407848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":32,"addr":"172.21.15.183:6835/1009004320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1009004320},{"type":"v1","addr":"172.21.15.183:6835","nonce":1009004320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.183:6837/2697518526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":2697518526},{"type":"v1","addr":"172.21.15.183:6837","nonce":2697518526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"a","rank":2,"incarnation":22,"state":"up:active","state_seq":27,"addr":"172.21.15.117:6841/1212242967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6840","nonce":1212242967},{"type":"v1","addr":"172.21.15.117:6841","nonce":1212242967}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.151:6839/3623444616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":3623444616},{"type":"v1","addr":"172.21.15.151:6839","nonce":3623444616}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":10,"addr":"172.21.15.183:6839/1439615083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1439615083},{"type":"v1","addr":"172.21.15.183:6839","nonce":1439615083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24517,"qdb_cluster":[24499,24487,24512,24503,24517]},"id":1}]} 2024-06-03T03:26:52.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:52.500+0000 7f6e25c00700 1 -- 172.21.15.117:0/2633987060 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f6e040a2c20 msgr2=0x7f6e040a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:52.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:52.500+0000 7f6e25c00700 1 --2- 172.21.15.117:0/2633987060 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f6e040a2c20 0x7f6e040a50e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6e10009c60 tx=0x7f6e10009260 comp rx=0 tx=0).stop 2024-06-03T03:26:52.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:52.501+0000 7f6e25c00700 1 -- 172.21.15.117:0/2633987060 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6e200751e0 msgr2=0x7f6e2019b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:52.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:52.501+0000 7f6e25c00700 1 --2- 172.21.15.117:0/2633987060 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6e200751e0 0x7f6e2019b370 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c000c00 tx=0x7f6e0c00bab0 comp rx=0 tx=0).stop 2024-06-03T03:26:52.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:52.501+0000 7f6e25c00700 1 -- 172.21.15.117:0/2633987060 shutdown_connections 2024-06-03T03:26:52.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:52.501+0000 7f6e25c00700 1 --2- 172.21.15.117:0/2633987060 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f6e040a2c20 0x7f6e040a50e0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:52.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:52.501+0000 7f6e25c00700 1 --2- 172.21.15.117:0/2633987060 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6e2010ec00 0x7f6e2019f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:52.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:52.501+0000 7f6e25c00700 1 --2- 172.21.15.117:0/2633987060 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e200748c0 0x7f6e2019ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:52.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:52.501+0000 7f6e25c00700 1 --2- 172.21.15.117:0/2633987060 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6e200751e0 0x7f6e2019b370 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:52.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:52.501+0000 7f6e25c00700 1 -- 172.21.15.117:0/2633987060 >> 172.21.15.117:0/2633987060 conn(0x7f6e200fdf10 msgr2=0x7f6e200fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:26:52.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:52.501+0000 7f6e25c00700 1 -- 172.21.15.117:0/2633987060 shutdown_connections 2024-06-03T03:26:52.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:52.501+0000 7f6e25c00700 1 -- 172.21.15.117:0/2633987060 wait complete. 2024-06-03T03:26:52.503 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 31 2024-06-03T03:26:52.906 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:26:52 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/2633987060' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T03:26:52.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:26:52 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/2633987060' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T03:26:52.981 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:26:52 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/2633987060' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T03:26:53.439 INFO:tasks.workunit.client.0.smithi117.stderr:Updating files: 62% (7796/12565) Updating files: 63% (7916/12565) Updating files: 64% (8042/12565) Updating files: 65% (8168/12565) Updating files: 66% (8293/12565) Updating files: 67% (8419/12565) Updating files: 68% (8545/12565) Updating files: 69% (8670/12565) Updating files: 70% (8796/12565) Updating files: 71% (8922/12565) Updating files: 72% (9047/12565) Updating files: 73% (9173/12565) Updating files: 74% (9299/12565) Updating files: 75% (9424/12565) Updating files: 76% (9550/12565) Updating files: 77% (9676/12565) Updating files: 78% (9801/12565) Updating files: 79% (9927/12565) Updating files: 80% (10052/12565) Updating files: 81% (10178/12565) Updating files: 82% (10304/12565) Updating files: 83% (10429/12565) Updating files: 84% (10555/12565) Updating files: 85% (10681/12565) Updating files: 86% (10806/12565) Updating files: 87% (10932/12565) Updating files: 88% (11058/12565) Updating files: 89% (11183/12565) Updating files: 90% (11309/12565) Updating files: 91% (11435/12565) Updating files: 92% (11560/12565) Updating files: 93% (11686/12565) Updating files: 94% (11812/12565) Updating files: 95% (11937/12565) Updating files: 96% (12063/12565) Updating files: 97% (12189/12565) Updating files: 97% (12251/12565) Updating files: 98% (12314/12565) Updating files: 99% (12440/12565) Updating files: 100% (12565/12565) Updating files: 100% (12565/12565), done. 2024-06-03T03:26:53.535 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-06-03T03:26:53.906 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:26:53 smithi151 ceph-mon[37043]: pgmap v553: 129 pgs: 129 active+clean; 681 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 255 B/s wr, 12 op/s 2024-06-03T03:26:53.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:26:53 smithi117 ceph-mon[31649]: pgmap v553: 129 pgs: 129 active+clean; 681 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 255 B/s wr, 12 op/s 2024-06-03T03:26:53.961 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:26:53.981 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:26:53 smithi183 ceph-mon[37326]: pgmap v553: 129 pgs: 129 active+clean; 681 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 255 B/s wr, 12 op/s 2024-06-03T03:26:54.207 INFO:tasks.workunit.client.0.smithi117.stderr:Note: switching to 'dfcc4b3c2c55532c4c04fa47551c9df1dffdf746'. 2024-06-03T03:26:54.208 INFO:tasks.workunit.client.0.smithi117.stderr: 2024-06-03T03:26:54.208 INFO:tasks.workunit.client.0.smithi117.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-06-03T03:26:54.208 INFO:tasks.workunit.client.0.smithi117.stderr:changes and commit them, and you can discard any commits you make in this 2024-06-03T03:26:54.208 INFO:tasks.workunit.client.0.smithi117.stderr:state without impacting any branches by switching back to a branch. 2024-06-03T03:26:54.208 INFO:tasks.workunit.client.0.smithi117.stderr: 2024-06-03T03:26:54.208 INFO:tasks.workunit.client.0.smithi117.stderr:If you want to create a new branch to retain commits you create, you may 2024-06-03T03:26:54.208 INFO:tasks.workunit.client.0.smithi117.stderr:do so (now or later) by using -c with the switch command. Example: 2024-06-03T03:26:54.208 INFO:tasks.workunit.client.0.smithi117.stderr: 2024-06-03T03:26:54.208 INFO:tasks.workunit.client.0.smithi117.stderr: git switch -c 2024-06-03T03:26:54.208 INFO:tasks.workunit.client.0.smithi117.stderr: 2024-06-03T03:26:54.208 INFO:tasks.workunit.client.0.smithi117.stderr:Or undo this operation with: 2024-06-03T03:26:54.208 INFO:tasks.workunit.client.0.smithi117.stderr: 2024-06-03T03:26:54.208 INFO:tasks.workunit.client.0.smithi117.stderr: git switch - 2024-06-03T03:26:54.209 INFO:tasks.workunit.client.0.smithi117.stderr: 2024-06-03T03:26:54.209 INFO:tasks.workunit.client.0.smithi117.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-06-03T03:26:54.209 INFO:tasks.workunit.client.0.smithi117.stderr: 2024-06-03T03:26:54.209 INFO:tasks.workunit.client.0.smithi117.stderr:HEAD is now at dfcc4b3c2c5 Merge pull request #57814 from zdover23/wip-doc-2024-06-01-backport-57804-to-squid 2024-06-03T03:26:54.219 DEBUG:teuthology.orchestra.run.smithi117:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-06-03T03:26:54.290 INFO:tasks.workunit.client.0.smithi117.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-06-03T03:26:54.297 INFO:tasks.workunit.client.0.smithi117.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-06-03T03:26:54.297 INFO:tasks.workunit.client.0.smithi117.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-06-03T03:26:54.696 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:26:54 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:26:54.906 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:26:54 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:26:54.981 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:26:54 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:26:55.842 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:26:55 smithi117 ceph-mon[31649]: pgmap v554: 129 pgs: 129 active+clean; 681 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 255 B/s wr, 10 op/s 2024-06-03T03:26:55.842 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:26:55 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:26:55.842 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:26:55 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:26:55.842 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:26:55 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:26:55.906 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:26:55 smithi151 ceph-mon[37043]: pgmap v554: 129 pgs: 129 active+clean; 681 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 255 B/s wr, 10 op/s 2024-06-03T03:26:55.906 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:26:55 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:26:55.906 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:26:55 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:26:55.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:26:55 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:26:55.981 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:26:55 smithi183 ceph-mon[37326]: pgmap v554: 129 pgs: 129 active+clean; 681 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 255 B/s wr, 10 op/s 2024-06-03T03:26:55.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:26:55 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:26:55.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:26:55 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:26:55.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:26:55 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:26:56.351 INFO:tasks.workunit.client.0.smithi117.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-06-03T03:26:56.464 INFO:tasks.workunit.client.0.smithi117.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-06-03T03:26:56.544 INFO:tasks.workunit.client.0.smithi117.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-06-03T03:26:56.549 INFO:tasks.workunit.client.0.smithi117.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-06-03T03:26:56.550 INFO:tasks.workunit.client.0.smithi117.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-06-03T03:26:56.638 INFO:tasks.workunit.client.0.smithi117.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-06-03T03:26:56.648 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-06-03T03:26:56.648 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-06-03T03:26:56.696 INFO:tasks.workunit:Running workunits matching suites/fsync-tester.sh on client.0... 2024-06-03T03:26:56.698 INFO:tasks.workunit:Running workunit suites/fsync-tester.sh... 2024-06-03T03:26:56.698 DEBUG:teuthology.orchestra.run.smithi117:workunit test suites/fsync-tester.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=dfcc4b3c2c55532c4c04fa47551c9df1dffdf746 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsync-tester.sh 2024-06-03T03:26:56.778 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.778+0000 7f72da400700 1 -- 172.21.15.117:0/3078505754 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72d4102e80 msgr2=0x7f72d410f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:56.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.778+0000 7f72da400700 1 --2- 172.21.15.117:0/3078505754 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72d4102e80 0x7f72d410f330 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f72cc00b0a0 tx=0x7f72cc00b3b0 comp rx=0 tx=0).stop 2024-06-03T03:26:56.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.778+0000 7f72da400700 1 -- 172.21.15.117:0/3078505754 shutdown_connections 2024-06-03T03:26:56.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.778+0000 7f72da400700 1 --2- 172.21.15.117:0/3078505754 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72d4102e80 0x7f72d410f330 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:56.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.778+0000 7f72da400700 1 --2- 172.21.15.117:0/3078505754 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f72d41084f0 0x7f72d41088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:56.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.778+0000 7f72da400700 1 --2- 172.21.15.117:0/3078505754 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72d41024e0 0x7f72d4102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:56.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.778+0000 7f72da400700 1 -- 172.21.15.117:0/3078505754 >> 172.21.15.117:0/3078505754 conn(0x7f72d4079530 msgr2=0x7f72d4079940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:26:56.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.778+0000 7f72da400700 1 -- 172.21.15.117:0/3078505754 shutdown_connections 2024-06-03T03:26:56.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.778+0000 7f72da400700 1 -- 172.21.15.117:0/3078505754 wait complete. 2024-06-03T03:26:56.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.779+0000 7f72da400700 1 Processor -- start 2024-06-03T03:26:56.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.779+0000 7f72da400700 1 -- start start 2024-06-03T03:26:56.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.779+0000 7f72da400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72d41024e0 0x7f72d4074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:56.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.779+0000 7f72da400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f72d4102e80 0x7f72d4074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:56.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.780+0000 7f72da400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72d41084f0 0x7f72d4075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:56.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.780+0000 7f72da400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f72d4112790 con 0x7f72d41084f0 2024-06-03T03:26:56.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.780+0000 7f72da400700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f72d4112610 con 0x7f72d4102e80 2024-06-03T03:26:56.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.780+0000 7f72da400700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f72d4112910 con 0x7f72d41024e0 2024-06-03T03:26:56.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.780+0000 7f72d2a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f72d4102e80 0x7f72d4074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:56.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.780+0000 7f72d3e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72d41084f0 0x7f72d4075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:56.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.780+0000 7f72d2a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f72d4102e80 0x7f72d4074cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:55272/0 (socket says 172.21.15.117:55272) 2024-06-03T03:26:56.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.780+0000 7f72d2a00700 1 -- 172.21.15.117:0/3857640561 learned_addr learned my addr 172.21.15.117:0/3857640561 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:26:56.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.780+0000 7f72d3400700 1 --2- 172.21.15.117:0/3857640561 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72d41024e0 0x7f72d4074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:56.782 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.780+0000 7f72d3e00700 1 -- 172.21.15.117:0/3857640561 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72d41024e0 msgr2=0x7f72d4074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:56.782 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.781+0000 7f72d3e00700 1 --2- 172.21.15.117:0/3857640561 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72d41024e0 0x7f72d4074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:56.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.781+0000 7f72d3e00700 1 -- 172.21.15.117:0/3857640561 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f72d4102e80 msgr2=0x7f72d4074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:56.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.781+0000 7f72d3e00700 1 --2- 172.21.15.117:0/3857640561 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f72d4102e80 0x7f72d4074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:56.785 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.781+0000 7f72d3e00700 1 -- 172.21.15.117:0/3857640561 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72d41a70f0 con 0x7f72d41084f0 2024-06-03T03:26:56.785 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.781+0000 7f72d3400700 1 --2- 172.21.15.117:0/3857640561 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72d41024e0 0x7f72d4074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:26:56.785 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.781+0000 7f72d3e00700 1 --2- 172.21.15.117:0/3857640561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72d41084f0 0x7f72d4075210 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f72cc0062a0 tx=0x7f72cc004dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:26:56.785 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.781+0000 7f72cbe00700 1 -- 172.21.15.117:0/3857640561 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72cc021070 con 0x7f72d41084f0 2024-06-03T03:26:56.785 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.781+0000 7f72da400700 1 -- 172.21.15.117:0/3857640561 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72d41a7380 con 0x7f72d41084f0 2024-06-03T03:26:56.785 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.781+0000 7f72cbe00700 1 -- 172.21.15.117:0/3857640561 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72cc007b80 con 0x7f72d41084f0 2024-06-03T03:26:56.785 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.781+0000 7f72cbe00700 1 -- 172.21.15.117:0/3857640561 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72cc0052a0 con 0x7f72d41084f0 2024-06-03T03:26:56.785 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.782+0000 7f72da400700 1 -- 172.21.15.117:0/3857640561 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72d41a7830 con 0x7f72d41084f0 2024-06-03T03:26:56.786 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.785+0000 7f72cbe00700 1 -- 172.21.15.117:0/3857640561 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f72cc01d450 con 0x7f72d41084f0 2024-06-03T03:26:56.787 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.786+0000 7f72e4ce1b80 1 Processor -- start 2024-06-03T03:26:56.787 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.786+0000 7f72e4ce1b80 1 -- start start 2024-06-03T03:26:56.788 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.787+0000 7f72e4ce1b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de65e19e40 0x55de65e9d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:56.788 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.787+0000 7f72e4ce1b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de65e9d5b0 0x55de65e9da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:56.788 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.787+0000 7f72e4ce1b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55de65ea8300 0x55de65eaa6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:56.788 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.787+0000 7f72e4ce1b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55de65daca10 con 0x55de65e19e40 2024-06-03T03:26:56.788 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.787+0000 7f72e4ce1b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55de65defb00 con 0x55de65e9d5b0 2024-06-03T03:26:56.788 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.787+0000 7f72e4ce1b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55de65daa8f0 con 0x55de65ea8300 2024-06-03T03:26:56.788 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.787+0000 7f72d2a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de65e19e40 0x55de65e9d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:56.788 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.787+0000 7f72d2a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de65e19e40 0x55de65e9d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:57334/0 (socket says 172.21.15.117:57334) 2024-06-03T03:26:56.789 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.787+0000 7f72d2a00700 1 -- 172.21.15.117:0/2573763730 learned_addr learned my addr 172.21.15.117:0/2573763730 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:26:56.789 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.787+0000 7f72d3400700 1 --2- 172.21.15.117:0/2573763730 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de65e9d5b0 0x55de65e9da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:56.789 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.788+0000 7f72cbe00700 1 --2- 172.21.15.117:0/3857640561 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f72b80a2c20 0x7f72b80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:56.789 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.788+0000 7f72d2a00700 1 --2- 172.21.15.117:0/2573763730 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55de65ea8300 0x55de65eaa6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:56.789 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.788+0000 7f72cbe00700 1 -- 172.21.15.117:0/3857640561 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f72cc0d0350 con 0x7f72d41084f0 2024-06-03T03:26:56.789 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.788+0000 7f72d3400700 1 -- 172.21.15.117:0/2573763730 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55de65ea8300 msgr2=0x55de65eaa6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:56.789 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.788+0000 7f72d3400700 1 --2- 172.21.15.117:0/2573763730 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55de65ea8300 0x55de65eaa6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:56.790 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.788+0000 7f72d3400700 1 -- 172.21.15.117:0/2573763730 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de65e19e40 msgr2=0x55de65e9d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:56.790 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.788+0000 7f72d3400700 1 --2- 172.21.15.117:0/2573763730 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de65e19e40 0x55de65e9d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:56.790 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.788+0000 7f72d3400700 1 -- 172.21.15.117:0/2573763730 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de65d97f90 con 0x55de65e9d5b0 2024-06-03T03:26:56.790 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.788+0000 7f72d3e00700 1 --2- 172.21.15.117:0/3857640561 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f72b80a2c20 0x7f72b80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:56.790 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.788+0000 7f72d3400700 1 --2- 172.21.15.117:0/2573763730 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de65e9d5b0 0x55de65e9da30 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f72c4007f60 tx=0x7f72c4007f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e782181dfdf61247 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:26:56.790 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.788+0000 7f72c8c00700 1 -- 172.21.15.117:0/2573763730 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c4011440 con 0x55de65e9d5b0 2024-06-03T03:26:56.790 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.789+0000 7f72c8c00700 1 -- 172.21.15.117:0/2573763730 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72c4011ac0 con 0x55de65e9d5b0 2024-06-03T03:26:56.790 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.789+0000 7f72c8c00700 1 -- 172.21.15.117:0/2573763730 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c4019b90 con 0x55de65e9d5b0 2024-06-03T03:26:56.791 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.789+0000 7f72e4ce1b80 1 -- 172.21.15.117:0/2573763730 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de65e9d5b0 msgr2=0x55de65e9da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:56.791 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.789+0000 7f72e4ce1b80 1 --2- 172.21.15.117:0/2573763730 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de65e9d5b0 0x55de65e9da30 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f72c4007f60 tx=0x7f72c4007f90 comp rx=0 tx=0).stop 2024-06-03T03:26:56.791 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.789+0000 7f72d3e00700 1 --2- 172.21.15.117:0/3857640561 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f72b80a2c20 0x7f72b80a50e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f72cc098cb0 tx=0x7f72cc009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:26:56.791 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.790+0000 7f72e4ce1b80 1 -- 172.21.15.117:0/2573763730 shutdown_connections 2024-06-03T03:26:56.791 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.790+0000 7f72e4ce1b80 1 --2- 172.21.15.117:0/2573763730 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55de65ea8300 0x55de65eaa6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:56.792 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.790+0000 7f72e4ce1b80 1 --2- 172.21.15.117:0/2573763730 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de65e19e40 0x55de65e9d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:56.792 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.790+0000 7f72e4ce1b80 1 --2- 172.21.15.117:0/2573763730 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de65e9d5b0 0x55de65e9da30 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:56.792 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.790+0000 7f72e4ce1b80 1 -- 172.21.15.117:0/2573763730 >> 172.21.15.117:0/2573763730 conn(0x55de65dca170 msgr2=0x55de65e994b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:26:56.792 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.790+0000 7f72e4ce1b80 1 -- 172.21.15.117:0/2573763730 shutdown_connections 2024-06-03T03:26:56.792 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.790+0000 7f72e4ce1b80 1 -- 172.21.15.117:0/2573763730 wait complete. 2024-06-03T03:26:56.792 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.790+0000 7f72e4ce1b80 1 -- start start 2024-06-03T03:26:56.792 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.790+0000 7f72e4ce1b80 1 Processor -- start 2024-06-03T03:26:56.792 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.791+0000 7f72e4ce1b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55de65e19e40 0x55de65ea79e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:56.792 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.791+0000 7f72e4ce1b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de65e9d5b0 0x55de65ea4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:56.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.791+0000 7f72e4ce1b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de65ea8300 0x55de65ea5220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:56.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.791+0000 7f72e4ce1b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55de65d8bf60 con 0x55de65e9d5b0 2024-06-03T03:26:56.794 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.791+0000 7f72e4ce1b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55de65debac0 con 0x55de65ea8300 2024-06-03T03:26:56.794 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.791+0000 7f72e4ce1b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55de65eaafc0 con 0x55de65e19e40 2024-06-03T03:26:56.794 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.791+0000 7f72e4ce1b80 10 client.0 _ll_register_callbacks cb 0x55de65d89720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:26:56.794 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.791+0000 7f72d2a00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55de65e19e40 0x55de65ea79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:56.794 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.791+0000 7f72d2a00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55de65e19e40 0x55de65ea79e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:60330/0 (socket says 172.21.15.117:60330) 2024-06-03T03:26:56.794 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.791+0000 7f72d2a00700 1 -- 172.21.15.117:0/4089855302 learned_addr learned my addr 172.21.15.117:0/4089855302 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:26:56.794 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.791+0000 7f72d2a00700 1 --2- 172.21.15.117:0/4089855302 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de65ea8300 0x55de65ea5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:56.794 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.792+0000 7f72d3400700 1 --2- 172.21.15.117:0/4089855302 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de65e9d5b0 0x55de65ea4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:56.794 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.792+0000 7f72d2a00700 1 -- 172.21.15.117:0/4089855302 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de65ea8300 msgr2=0x55de65ea5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:56.794 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.792+0000 7f72d2a00700 1 --2- 172.21.15.117:0/4089855302 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de65ea8300 0x55de65ea5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:56.794 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.792+0000 7f72d2a00700 1 -- 172.21.15.117:0/4089855302 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de65e9d5b0 msgr2=0x55de65ea4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:56.794 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.792+0000 7f72d2a00700 1 --2- 172.21.15.117:0/4089855302 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de65e9d5b0 0x55de65ea4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:56.795 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.792+0000 7f72d2a00700 1 -- 172.21.15.117:0/4089855302 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de65d3e8d0 con 0x55de65e19e40 2024-06-03T03:26:56.795 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.792+0000 7f72d2a00700 1 --2- 172.21.15.117:0/4089855302 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55de65e19e40 0x55de65ea79e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f72c0012e00 tx=0x7f72c00077c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1dc39527afdcf5bf server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:26:56.795 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.792+0000 7f72bd600700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-06-03T03:26:56.795 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.792+0000 7f72bd600700 1 -- 172.21.15.117:0/4089855302 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c001b040 con 0x55de65e19e40 2024-06-03T03:26:56.795 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.792+0000 7f72bd600700 1 -- 172.21.15.117:0/4089855302 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f72c000eb40 con 0x55de65e19e40 2024-06-03T03:26:56.795 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.792+0000 7f72e4ce1b80 1 -- 172.21.15.117:0/4089855302 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72c4011ac0 con 0x55de65e19e40 2024-06-03T03:26:56.795 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.793+0000 7f72bd600700 1 -- 172.21.15.117:0/4089855302 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f72c0029050 con 0x55de65e19e40 2024-06-03T03:26:56.795 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.793+0000 7f72e4ce1b80 10 client.25582 fetch_fsmap learned FSMap version 31 2024-06-03T03:26:56.795 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.793+0000 7f72e4ce1b80 1 -- 172.21.15.117:0/4089855302 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55de65ebdf50 con 0x55de65e19e40 2024-06-03T03:26:56.795 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.793+0000 7f72bd600700 1 -- 172.21.15.117:0/4089855302 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72c4011ac0 con 0x55de65e19e40 2024-06-03T03:26:56.797 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.793+0000 7f72bd600700 1 -- 172.21.15.117:0/4089855302 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f72c00213d0 con 0x55de65e19e40 2024-06-03T03:26:56.797 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.794+0000 7f72e4ce1b80 10 client.25582 fetch_fsmap finished waiting for FSMap version 31 2024-06-03T03:26:56.797 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.794+0000 7f72e4ce1b80 10 client.25582 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-06-03T03:26:56.797 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.794+0000 7f72e4ce1b80 20 client.25582 populate_metadata read hostname 'smithi117' 2024-06-03T03:26:56.797 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.794+0000 7f72e4ce1b80 1 --2- 172.21.15.117:0/4089855302 >> [v2:172.21.15.183:6834/1009004320,v1:172.21.15.183:6835/1009004320] conn(0x55de65ec0590 0x55de65ec2980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:56.797 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.794+0000 7f72e4ce1b80 4 client.25582 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T03:26:56.797 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.794+0000 7f72e4ce1b80 1 -- 172.21.15.117:0/4089855302 --> [v2:172.21.15.183:6834/1009004320,v1:172.21.15.183:6835/1009004320] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55de65dfd000 con 0x55de65ec0590 2024-06-03T03:26:56.797 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.794+0000 7f72d3400700 1 --2- 172.21.15.117:0/4089855302 >> [v2:172.21.15.183:6834/1009004320,v1:172.21.15.183:6835/1009004320] conn(0x55de65ec0590 0x55de65ec2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:56.797 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.795+0000 7f72d3400700 1 --2- 172.21.15.117:0/4089855302 >> [v2:172.21.15.183:6834/1009004320,v1:172.21.15.183:6835/1009004320] conn(0x55de65ec0590 0x55de65ec2980 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4bb6f1ce009fa3cf server_cookie=463a9e05cbcfccb9 in_seq=0 out_seq=0 2024-06-03T03:26:56.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.795+0000 7f72bd600700 10 client.25582 ms_handle_connect on v2:172.21.15.183:6834/1009004320 2024-06-03T03:26:56.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.796+0000 7f72bd600700 1 -- 172.21.15.117:0/4089855302 <== mds.4 v2:172.21.15.183:6834/1009004320 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55de65dfd000 con 0x55de65ec0590 2024-06-03T03:26:56.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.796+0000 7f72bd600700 10 client.25582 handle_command_reply: tid=0 2024-06-03T03:26:56.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.796+0000 7f72e4ce1b80 1 client.25582 shutdown 2024-06-03T03:26:56.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.796+0000 7f72e4ce1b80 1 -- 172.21.15.117:0/4089855302 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55de65e19e40 msgr2=0x55de65ea79e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:56.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.796+0000 7f72e4ce1b80 1 --2- 172.21.15.117:0/4089855302 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55de65e19e40 0x55de65ea79e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f72c0012e00 tx=0x7f72c00077c0 comp rx=0 tx=0).stop 2024-06-03T03:26:56.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.796+0000 7f72e4ce1b80 1 -- 172.21.15.117:0/4089855302 shutdown_connections 2024-06-03T03:26:56.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.796+0000 7f72e4ce1b80 1 --2- 172.21.15.117:0/4089855302 >> [v2:172.21.15.183:6834/1009004320,v1:172.21.15.183:6835/1009004320] conn(0x55de65ec0590 0x55de65ec2980 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:56.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.796+0000 7f72e4ce1b80 1 --2- 172.21.15.117:0/4089855302 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de65ea8300 0x55de65ea5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:56.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.796+0000 7f72e4ce1b80 1 --2- 172.21.15.117:0/4089855302 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55de65e19e40 0x55de65ea79e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:56.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.796+0000 7f72bd600700 0 client.25582 ms_handle_reset on v2:172.21.15.183:6834/1009004320 2024-06-03T03:26:56.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.796+0000 7f72e4ce1b80 1 --2- 172.21.15.117:0/4089855302 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de65e9d5b0 0x55de65ea4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:56.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.796+0000 7f72e4ce1b80 1 -- 172.21.15.117:0/4089855302 >> 172.21.15.117:0/4089855302 conn(0x55de65dca170 msgr2=0x55de65e98740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:26:56.799 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.796+0000 7f72e4ce1b80 1 -- 172.21.15.117:0/4089855302 shutdown_connections 2024-06-03T03:26:56.799 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.796+0000 7f72e4ce1b80 1 -- 172.21.15.117:0/4089855302 wait complete. 2024-06-03T03:26:56.799 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.797+0000 7f72e4ce1b80 20 client.25582 trim_cache size 0 max 16384 2024-06-03T03:26:56.802 INFO:tasks.workunit.client.0.smithi117.stderr:+ wget http://download.ceph.com/qa/fsync-tester.c 2024-06-03T03:26:56.814 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.814+0000 7f72e4ce1b80 1 Processor -- start 2024-06-03T03:26:56.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.814+0000 7f72e4ce1b80 1 -- start start 2024-06-03T03:26:56.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.814+0000 7f72e4ce1b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de65e19e40 0x55de65eaa720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:56.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.814+0000 7f72e4ce1b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55de65e9d5b0 0x55de65ea3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:56.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.814+0000 7f72e4ce1b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de65ea8300 0x55de65ea96f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:56.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.814+0000 7f72e4ce1b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f72d41abf60 con 0x55de65e19e40 2024-06-03T03:26:56.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.814+0000 7f72e4ce1b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f72d41adaa0 con 0x55de65ea8300 2024-06-03T03:26:56.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.814+0000 7f72e4ce1b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f72d41ac630 con 0x55de65e9d5b0 2024-06-03T03:26:56.816 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.814+0000 7f72d2a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de65e19e40 0x55de65eaa720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:56.816 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.815+0000 7f72d2a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de65e19e40 0x55de65eaa720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:57362/0 (socket says 172.21.15.117:57362) 2024-06-03T03:26:56.816 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.815+0000 7f72d2a00700 1 -- 172.21.15.117:0/136904949 learned_addr learned my addr 172.21.15.117:0/136904949 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:26:56.816 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.815+0000 7f72d3400700 1 --2- 172.21.15.117:0/136904949 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55de65e9d5b0 0x55de65ea3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:56.816 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.815+0000 7f72d2a00700 1 --2- 172.21.15.117:0/136904949 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de65ea8300 0x55de65ea96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:56.816 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.815+0000 7f72d3400700 1 -- 172.21.15.117:0/136904949 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de65ea8300 msgr2=0x55de65ea96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:56.817 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.815+0000 7f72d3400700 1 --2- 172.21.15.117:0/136904949 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de65ea8300 0x55de65ea96f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:56.817 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.815+0000 7f72d3400700 1 -- 172.21.15.117:0/136904949 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de65e19e40 msgr2=0x55de65eaa720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:56.817 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.815+0000 7f72d3400700 1 --2- 172.21.15.117:0/136904949 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de65e19e40 0x55de65eaa720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:56.817 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.815+0000 7f72d3400700 1 -- 172.21.15.117:0/136904949 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de65e9ebd0 con 0x55de65e9d5b0 2024-06-03T03:26:56.817 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.815+0000 7f72d2a00700 1 --2- 172.21.15.117:0/136904949 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de65ea8300 0x55de65ea96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T03:26:56.817 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.815+0000 7f72d3400700 1 --2- 172.21.15.117:0/136904949 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55de65e9d5b0 0x55de65ea3ce0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f72c4007f30 tx=0x7f72c40196f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec4c1d5bc10a1377 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:26:56.817 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.815+0000 7f72bfe00700 1 -- 172.21.15.117:0/136904949 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c4005040 con 0x55de65e9d5b0 2024-06-03T03:26:56.817 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.815+0000 7f72bfe00700 1 -- 172.21.15.117:0/136904949 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72c40056c0 con 0x55de65e9d5b0 2024-06-03T03:26:56.817 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.816+0000 7f72bfe00700 1 -- 172.21.15.117:0/136904949 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c4003680 con 0x55de65e9d5b0 2024-06-03T03:26:56.819 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.816+0000 7f72e4ce1b80 1 -- 172.21.15.117:0/136904949 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55de65e9d5b0 msgr2=0x55de65ea3ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:56.819 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.816+0000 7f72e4ce1b80 1 --2- 172.21.15.117:0/136904949 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55de65e9d5b0 0x55de65ea3ce0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f72c4007f30 tx=0x7f72c40196f0 comp rx=0 tx=0).stop 2024-06-03T03:26:56.819 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.816+0000 7f72e4ce1b80 1 -- 172.21.15.117:0/136904949 shutdown_connections 2024-06-03T03:26:56.819 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.816+0000 7f72e4ce1b80 1 --2- 172.21.15.117:0/136904949 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de65ea8300 0x55de65ea96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:56.819 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.816+0000 7f72e4ce1b80 1 --2- 172.21.15.117:0/136904949 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de65e19e40 0x55de65eaa720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:56.819 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.816+0000 7f72e4ce1b80 1 --2- 172.21.15.117:0/136904949 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55de65e9d5b0 0x55de65ea3ce0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:56.819 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.816+0000 7f72e4ce1b80 1 -- 172.21.15.117:0/136904949 >> 172.21.15.117:0/136904949 conn(0x55de65dca170 msgr2=0x55de65ea35d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:26:56.819 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.816+0000 7f72e4ce1b80 1 -- 172.21.15.117:0/136904949 shutdown_connections 2024-06-03T03:26:56.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.817+0000 7f72e4ce1b80 1 -- 172.21.15.117:0/136904949 wait complete. 2024-06-03T03:26:56.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.817+0000 7f72e4ce1b80 1 -- start start 2024-06-03T03:26:56.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.817+0000 7f72e4ce1b80 1 Processor -- start 2024-06-03T03:26:56.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.818+0000 7f72e4ce1b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de65e19e40 0x55de65eb8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:56.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.818+0000 7f72e4ce1b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55de65e9d5b0 0x55de65eb9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:56.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.818+0000 7f72e4ce1b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de65ea8300 0x55de65ea1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:56.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.818+0000 7f72e4ce1b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f72d41ad3d0 con 0x55de65e19e40 2024-06-03T03:26:56.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.818+0000 7f72e4ce1b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f72d41acd00 con 0x55de65ea8300 2024-06-03T03:26:56.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.818+0000 7f72e4ce1b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f72d41a9f80 con 0x55de65e9d5b0 2024-06-03T03:26:56.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.818+0000 7f72e4ce1b80 10 client.0 _ll_register_callbacks cb 0x55de65d89720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:26:56.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.818+0000 7f72d2a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de65e19e40 0x55de65eb8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:56.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.818+0000 7f72d2a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de65e19e40 0x55de65eb8cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:57364/0 (socket says 172.21.15.117:57364) 2024-06-03T03:26:56.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.818+0000 7f72d2a00700 1 -- 172.21.15.117:0/1262782563 learned_addr learned my addr 172.21.15.117:0/1262782563 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:26:56.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.818+0000 7f72d3400700 1 --2- 172.21.15.117:0/1262782563 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55de65e9d5b0 0x55de65eb9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:56.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.818+0000 7f72d2a00700 1 --2- 172.21.15.117:0/1262782563 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de65ea8300 0x55de65ea1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:56.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.819+0000 7f72d3400700 1 -- 172.21.15.117:0/1262782563 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de65ea8300 msgr2=0x55de65ea1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:56.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.819+0000 7f72d3400700 1 --2- 172.21.15.117:0/1262782563 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de65ea8300 0x55de65ea1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:56.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.819+0000 7f72d3400700 1 -- 172.21.15.117:0/1262782563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de65e19e40 msgr2=0x55de65eb8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:26:56.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.819+0000 7f72d3400700 1 --2- 172.21.15.117:0/1262782563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de65e19e40 0x55de65eb8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:56.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.819+0000 7f72d3400700 1 -- 172.21.15.117:0/1262782563 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de65ea2690 con 0x55de65e9d5b0 2024-06-03T03:26:56.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.819+0000 7f72d2a00700 1 --2- 172.21.15.117:0/1262782563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de65e19e40 0x55de65eb8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:26:56.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.819+0000 7f72d3400700 1 --2- 172.21.15.117:0/1262782563 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55de65e9d5b0 0x55de65eb9230 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f72c4007410 tx=0x7f72c4005970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c7edba8c7136925 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:26:56.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.819+0000 7f72bd600700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-06-03T03:26:56.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.819+0000 7f72bd600700 1 -- 172.21.15.117:0/1262782563 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c40052c0 con 0x55de65e9d5b0 2024-06-03T03:26:56.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.819+0000 7f72bd600700 1 -- 172.21.15.117:0/1262782563 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f72c4005460 con 0x55de65e9d5b0 2024-06-03T03:26:56.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.819+0000 7f72e4ce1b80 1 -- 172.21.15.117:0/1262782563 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55de65df7580 con 0x55de65e9d5b0 2024-06-03T03:26:56.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.820+0000 7f72bd600700 1 -- 172.21.15.117:0/1262782563 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f72c4019760 con 0x55de65e9d5b0 2024-06-03T03:26:56.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.820+0000 7f72e4ce1b80 10 client.25588 fetch_fsmap learned FSMap version 31 2024-06-03T03:26:56.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.820+0000 7f72e4ce1b80 1 -- 172.21.15.117:0/1262782563 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55de65ea45d0 con 0x55de65e9d5b0 2024-06-03T03:26:56.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.820+0000 7f72bd600700 1 -- 172.21.15.117:0/1262782563 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55de65df7580 con 0x55de65e9d5b0 2024-06-03T03:26:56.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.820+0000 7f72bd600700 1 -- 172.21.15.117:0/1262782563 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f72c4011b90 con 0x55de65e9d5b0 2024-06-03T03:26:56.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.820+0000 7f72e4ce1b80 10 client.25588 fetch_fsmap finished waiting for FSMap version 31 2024-06-03T03:26:56.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.820+0000 7f72e4ce1b80 10 client.25588 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-06-03T03:26:56.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.820+0000 7f72e4ce1b80 20 client.25588 populate_metadata read hostname 'smithi117' 2024-06-03T03:26:56.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.820+0000 7f72e4ce1b80 1 --2- 172.21.15.117:0/1262782563 >> [v2:172.21.15.183:6834/1009004320,v1:172.21.15.183:6835/1009004320] conn(0x55de65f2b0e0 0x55de65f2d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:26:56.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.820+0000 7f72e4ce1b80 4 client.25588 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T03:26:56.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.820+0000 7f72e4ce1b80 1 -- 172.21.15.117:0/1262782563 --> [v2:172.21.15.183:6834/1009004320,v1:172.21.15.183:6835/1009004320] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55de65c71250 con 0x55de65f2b0e0 2024-06-03T03:26:56.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.821+0000 7f72d2a00700 1 --2- 172.21.15.117:0/1262782563 >> [v2:172.21.15.183:6834/1009004320,v1:172.21.15.183:6835/1009004320] conn(0x55de65f2b0e0 0x55de65f2d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:26:56.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.821+0000 7f72d2a00700 1 --2- 172.21.15.117:0/1262782563 >> [v2:172.21.15.183:6834/1009004320,v1:172.21.15.183:6835/1009004320] conn(0x55de65f2b0e0 0x55de65f2d4d0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=373bed9a21e93878 server_cookie=264a3b840dea6abb in_seq=0 out_seq=0 2024-06-03T03:26:56.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.821+0000 7f72bd600700 10 client.25588 ms_handle_connect on v2:172.21.15.183:6834/1009004320 2024-06-03T03:26:56.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.821+0000 7f72bd600700 1 -- 172.21.15.117:0/1262782563 <== mds.4 v2:172.21.15.183:6834/1009004320 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55de65c71250 con 0x55de65f2b0e0 2024-06-03T03:26:56.825 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.821+0000 7f72bd600700 10 client.25588 handle_command_reply: tid=0 2024-06-03T03:26:56.825 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.821+0000 7f72e4ce1b80 1 client.25588 shutdown 2024-06-03T03:26:56.825 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.822+0000 7f72e4ce1b80 1 -- 172.21.15.117:0/1262782563 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55de65e9d5b0 msgr2=0x55de65eb9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:56.825 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.822+0000 7f72e4ce1b80 1 --2- 172.21.15.117:0/1262782563 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55de65e9d5b0 0x55de65eb9230 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f72c4007410 tx=0x7f72c4005970 comp rx=0 tx=0).stop 2024-06-03T03:26:56.825 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.822+0000 7f72e4ce1b80 1 -- 172.21.15.117:0/1262782563 shutdown_connections 2024-06-03T03:26:56.825 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.822+0000 7f72e4ce1b80 1 --2- 172.21.15.117:0/1262782563 >> [v2:172.21.15.183:6834/1009004320,v1:172.21.15.183:6835/1009004320] conn(0x55de65f2b0e0 0x55de65f2d4d0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:56.825 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.822+0000 7f72e4ce1b80 1 --2- 172.21.15.117:0/1262782563 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de65ea8300 0x55de65ea1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:56.825 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.822+0000 7f72e4ce1b80 1 --2- 172.21.15.117:0/1262782563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55de65e19e40 0x55de65eb8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:56.825 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.822+0000 7f72e4ce1b80 1 --2- 172.21.15.117:0/1262782563 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55de65e9d5b0 0x55de65eb9230 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:56.825 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.822+0000 7f72bd600700 0 client.25588 ms_handle_reset on v2:172.21.15.183:6834/1009004320 2024-06-03T03:26:56.825 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.822+0000 7f72e4ce1b80 1 -- 172.21.15.117:0/1262782563 >> 172.21.15.117:0/1262782563 conn(0x55de65dca170 msgr2=0x55de65e9f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:26:56.825 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.822+0000 7f72e4ce1b80 1 -- 172.21.15.117:0/1262782563 shutdown_connections 2024-06-03T03:26:56.825 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.822+0000 7f72e4ce1b80 1 -- 172.21.15.117:0/1262782563 wait complete. 2024-06-03T03:26:56.826 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.822+0000 7f72e4ce1b80 20 client.25588 trim_cache size 0 max 16384 2024-06-03T03:26:56.826 INFO:teuthology.orchestra.run.smithi117.stdout:[] 2024-06-03T03:26:56.826 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.823+0000 7f72da400700 1 -- 172.21.15.117:0/3857640561 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f72b80a2c20 msgr2=0x7f72b80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:56.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.823+0000 7f72da400700 1 --2- 172.21.15.117:0/3857640561 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f72b80a2c20 0x7f72b80a50e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f72cc098cb0 tx=0x7f72cc009040 comp rx=0 tx=0).stop 2024-06-03T03:26:56.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.823+0000 7f72da400700 1 -- 172.21.15.117:0/3857640561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72d41084f0 msgr2=0x7f72d4075210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:26:56.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.823+0000 7f72da400700 1 --2- 172.21.15.117:0/3857640561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72d41084f0 0x7f72d4075210 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f72cc0062a0 tx=0x7f72cc004dc0 comp rx=0 tx=0).stop 2024-06-03T03:26:56.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.823+0000 7f72da400700 1 -- 172.21.15.117:0/3857640561 shutdown_connections 2024-06-03T03:26:56.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.823+0000 7f72da400700 1 --2- 172.21.15.117:0/3857640561 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f72b80a2c20 0x7f72b80a50e0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:56.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.823+0000 7f72da400700 1 --2- 172.21.15.117:0/3857640561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72d41084f0 0x7f72d4075210 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:56.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.823+0000 7f72da400700 1 --2- 172.21.15.117:0/3857640561 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72d41024e0 0x7f72d4074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:56.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.823+0000 7f72da400700 1 --2- 172.21.15.117:0/3857640561 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f72d4102e80 0x7f72d4074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:26:56.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.823+0000 7f72da400700 1 -- 172.21.15.117:0/3857640561 >> 172.21.15.117:0/3857640561 conn(0x7f72d4079530 msgr2=0x7f72d41004f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:26:56.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.823+0000 7f72da400700 1 -- 172.21.15.117:0/3857640561 shutdown_connections 2024-06-03T03:26:56.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:26:56.823+0000 7f72da400700 1 -- 172.21.15.117:0/3857640561 wait complete. 2024-06-03T03:26:56.846 INFO:tasks.workunit.client.0.smithi117.stderr:--2024-06-03 03:26:56-- http://download.ceph.com/qa/fsync-tester.c 2024-06-03T03:26:56.847 INFO:tasks.workunit.client.0.smithi117.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2024-06-03T03:26:56.884 INFO:tasks.workunit.client.0.smithi117.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2024-06-03T03:26:56.919 INFO:tasks.workunit.client.0.smithi117.stderr:HTTP request sent, awaiting response... 200 OK 2024-06-03T03:26:56.920 INFO:tasks.workunit.client.0.smithi117.stderr:Length: 3024 (3.0K) [application/octet-stream] 2024-06-03T03:26:56.923 INFO:tasks.workunit.client.0.smithi117.stderr:Saving to: ‘fsync-tester.c’ 2024-06-03T03:26:56.930 INFO:tasks.workunit.client.0.smithi117.stderr: 2024-06-03T03:26:56.930 INFO:tasks.workunit.client.0.smithi117.stderr: 0K .. 100% 122M=0s 2024-06-03T03:26:56.930 INFO:tasks.workunit.client.0.smithi117.stderr: 2024-06-03T03:26:56.930 INFO:tasks.workunit.client.0.smithi117.stderr:2024-06-03 03:26:56 (122 MB/s) - ‘fsync-tester.c’ saved [3024/3024] 2024-06-03T03:26:56.930 INFO:tasks.workunit.client.0.smithi117.stderr: 2024-06-03T03:26:56.932 INFO:tasks.workunit.client.0.smithi117.stderr:+ gcc -D_GNU_SOURCE fsync-tester.c -o fsync-tester 2024-06-03T03:26:57.308 INFO:tasks.workunit.client.0.smithi117.stderr:+ ./fsync-tester 2024-06-03T03:26:57.813 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-06-03T03:26:58.311 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:26:58.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:26:58 smithi151 ceph-mon[37043]: pgmap v555: 129 pgs: 129 active+clean; 681 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 255 B/s wr, 9 op/s 2024-06-03T03:26:58.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:26:58 smithi117 ceph-mon[31649]: pgmap v555: 129 pgs: 129 active+clean; 681 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 255 B/s wr, 9 op/s 2024-06-03T03:26:58.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:26:58 smithi183 ceph-mon[37326]: pgmap v555: 129 pgs: 129 active+clean; 681 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 255 B/s wr, 9 op/s 2024-06-03T03:26:59.386 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:26:59 smithi117 ceph-mon[31649]: pgmap v556: 129 pgs: 129 active+clean; 681 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 12 op/s 2024-06-03T03:26:59.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:26:59 smithi151 ceph-mon[37043]: pgmap v556: 129 pgs: 129 active+clean; 681 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 12 op/s 2024-06-03T03:26:59.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:26:59 smithi183 ceph-mon[37326]: pgmap v556: 129 pgs: 129 active+clean; 681 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 12 op/s 2024-06-03T03:27:00.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.717+0000 7fe950a00700 1 -- 172.21.15.117:0/330323108 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe94c074fe0 msgr2=0x7fe94c113120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:00.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.717+0000 7fe950a00700 1 --2- 172.21.15.117:0/330323108 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe94c074fe0 0x7fe94c113120 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe938009990 tx=0x7fe938009ca0 comp rx=0 tx=0).stop 2024-06-03T03:27:00.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.717+0000 7fe950a00700 1 -- 172.21.15.117:0/330323108 shutdown_connections 2024-06-03T03:27:00.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.717+0000 7fe950a00700 1 --2- 172.21.15.117:0/330323108 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe94c113660 0x7fe94c115a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:00.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.717+0000 7fe950a00700 1 --2- 172.21.15.117:0/330323108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe94c0746c0 0x7fe94c074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:00.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.717+0000 7fe950a00700 1 --2- 172.21.15.117:0/330323108 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe94c074fe0 0x7fe94c113120 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:00.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.717+0000 7fe950a00700 1 -- 172.21.15.117:0/330323108 >> 172.21.15.117:0/330323108 conn(0x7fe94c0795f0 msgr2=0x7fe94c079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:27:00.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.717+0000 7fe950a00700 1 -- 172.21.15.117:0/330323108 shutdown_connections 2024-06-03T03:27:00.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.717+0000 7fe950a00700 1 -- 172.21.15.117:0/330323108 wait complete. 2024-06-03T03:27:00.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.718+0000 7fe950a00700 1 Processor -- start 2024-06-03T03:27:00.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.718+0000 7fe950a00700 1 -- start start 2024-06-03T03:27:00.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.719+0000 7fe950a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe94c0746c0 0x7fe94c073d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:00.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.719+0000 7fe950a00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe94c074fe0 0x7fe94c06dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:00.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.719+0000 7fe950a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe94c113660 0x7fe94c06e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:00.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.719+0000 7fe950a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe94c06c7b0 con 0x7fe94c113660 2024-06-03T03:27:00.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.719+0000 7fe950a00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe94c06c630 con 0x7fe94c0746c0 2024-06-03T03:27:00.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.719+0000 7fe950a00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe94c06c930 con 0x7fe94c074fe0 2024-06-03T03:27:00.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.719+0000 7fe949e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe94c0746c0 0x7fe94c073d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:00.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.719+0000 7fe949e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe94c0746c0 0x7fe94c073d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:50486/0 (socket says 172.21.15.117:50486) 2024-06-03T03:27:00.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.719+0000 7fe949e00700 1 -- 172.21.15.117:0/3636118111 learned_addr learned my addr 172.21.15.117:0/3636118111 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:27:00.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.719+0000 7fe949400700 1 --2- 172.21.15.117:0/3636118111 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe94c074fe0 0x7fe94c06dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:00.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.719+0000 7fe94a800700 1 --2- 172.21.15.117:0/3636118111 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe94c113660 0x7fe94c06e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:00.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.720+0000 7fe949e00700 1 -- 172.21.15.117:0/3636118111 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe94c074fe0 msgr2=0x7fe94c06dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:00.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.720+0000 7fe949e00700 1 --2- 172.21.15.117:0/3636118111 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe94c074fe0 0x7fe94c06dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:00.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.720+0000 7fe949e00700 1 -- 172.21.15.117:0/3636118111 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe94c113660 msgr2=0x7fe94c06e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:00.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.720+0000 7fe949e00700 1 --2- 172.21.15.117:0/3636118111 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe94c113660 0x7fe94c06e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:00.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.720+0000 7fe949e00700 1 -- 172.21.15.117:0/3636118111 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe94c06e930 con 0x7fe94c0746c0 2024-06-03T03:27:00.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.720+0000 7fe949e00700 1 --2- 172.21.15.117:0/3636118111 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe94c0746c0 0x7fe94c073d00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe93c00e9a0 tx=0x7fe93c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:27:00.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.720+0000 7fe942a00700 1 -- 172.21.15.117:0/3636118111 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe93c00ca10 con 0x7fe94c0746c0 2024-06-03T03:27:00.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.720+0000 7fe942a00700 1 -- 172.21.15.117:0/3636118111 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe93c00cbb0 con 0x7fe94c0746c0 2024-06-03T03:27:00.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.721+0000 7fe950a00700 1 -- 172.21.15.117:0/3636118111 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe94c1b4260 con 0x7fe94c0746c0 2024-06-03T03:27:00.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.721+0000 7fe942a00700 1 -- 172.21.15.117:0/3636118111 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe93c018680 con 0x7fe94c0746c0 2024-06-03T03:27:00.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.721+0000 7fe950a00700 1 -- 172.21.15.117:0/3636118111 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe94c1b46b0 con 0x7fe94c0746c0 2024-06-03T03:27:00.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.725+0000 7fe942a00700 1 -- 172.21.15.117:0/3636118111 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fe93c018820 con 0x7fe94c0746c0 2024-06-03T03:27:00.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.725+0000 7fe95b37db80 1 Processor -- start 2024-06-03T03:27:00.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.726+0000 7fe95b37db80 1 -- start start 2024-06-03T03:27:00.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.726+0000 7fe95b37db80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55980bf9fb90 0x55980bf9ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:00.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.726+0000 7fe95b37db80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55980bfa04b0 0x55980bf99d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:00.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.726+0000 7fe95b37db80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55980bf9a240 0x55980bf9a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:00.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.726+0000 7fe95b37db80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55980bea8a10 con 0x55980bf9fb90 2024-06-03T03:27:00.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.726+0000 7fe95b37db80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55980beebb00 con 0x55980bf9a240 2024-06-03T03:27:00.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.726+0000 7fe95b37db80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55980bea68f0 con 0x55980bfa04b0 2024-06-03T03:27:00.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.726+0000 7fe94a800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55980bf9fb90 0x55980bf9ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:00.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.726+0000 7fe94a800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55980bf9fb90 0x55980bf9ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:54958/0 (socket says 172.21.15.117:54958) 2024-06-03T03:27:00.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.726+0000 7fe94a800700 1 -- 172.21.15.117:0/257760634 learned_addr learned my addr 172.21.15.117:0/257760634 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:27:00.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.727+0000 7fe949400700 1 --2- 172.21.15.117:0/257760634 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55980bf9a240 0x55980bf9a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:00.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.727+0000 7fe949e00700 1 --2- 172.21.15.117:0/257760634 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55980bfa04b0 0x55980bf99d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:00.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.727+0000 7fe949400700 1 -- 172.21.15.117:0/257760634 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55980bfa04b0 msgr2=0x55980bf99d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:00.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.727+0000 7fe949400700 1 --2- 172.21.15.117:0/257760634 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55980bfa04b0 0x55980bf99d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:00.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.727+0000 7fe949400700 1 -- 172.21.15.117:0/257760634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55980bf9fb90 msgr2=0x55980bf9ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:00.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.727+0000 7fe949400700 1 --2- 172.21.15.117:0/257760634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55980bf9fb90 0x55980bf9ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:00.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.727+0000 7fe949400700 1 -- 172.21.15.117:0/257760634 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55980be93f90 con 0x55980bf9a240 2024-06-03T03:27:00.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.727+0000 7fe942a00700 1 --2- 172.21.15.117:0/3636118111 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fe9300a2cd0 0x7fe9300a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:00.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.727+0000 7fe949400700 1 --2- 172.21.15.117:0/257760634 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55980bf9a240 0x55980bf9a680 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe938006010 tx=0x7fe9380096e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b3e0e310137532 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:27:00.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.727+0000 7fe942a00700 1 -- 172.21.15.117:0/3636118111 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7fe93c0cf590 con 0x7fe94c0746c0 2024-06-03T03:27:00.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.728+0000 7fe937400700 1 -- 172.21.15.117:0/257760634 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe938027070 con 0x55980bf9a240 2024-06-03T03:27:00.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.728+0000 7fe94a800700 1 --2- 172.21.15.117:0/3636118111 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fe9300a2cd0 0x7fe9300a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:00.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.728+0000 7fe937400700 1 -- 172.21.15.117:0/257760634 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe938005550 con 0x55980bf9a240 2024-06-03T03:27:00.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.728+0000 7fe937400700 1 -- 172.21.15.117:0/257760634 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe938003af0 con 0x55980bf9a240 2024-06-03T03:27:00.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.729+0000 7fe94a800700 1 --2- 172.21.15.117:0/3636118111 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fe9300a2cd0 0x7fe9300a5190 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe944007610 tx=0x7fe94400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:27:00.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.729+0000 7fe95b37db80 1 -- 172.21.15.117:0/257760634 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55980bf9a240 msgr2=0x55980bf9a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:00.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.729+0000 7fe95b37db80 1 --2- 172.21.15.117:0/257760634 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55980bf9a240 0x55980bf9a680 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe938006010 tx=0x7fe9380096e0 comp rx=0 tx=0).stop 2024-06-03T03:27:00.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.729+0000 7fe95b37db80 1 -- 172.21.15.117:0/257760634 shutdown_connections 2024-06-03T03:27:00.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.729+0000 7fe95b37db80 1 --2- 172.21.15.117:0/257760634 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55980bf9a240 0x55980bf9a680 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:00.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.729+0000 7fe95b37db80 1 --2- 172.21.15.117:0/257760634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55980bf9fb90 0x55980bf9ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:00.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.729+0000 7fe95b37db80 1 --2- 172.21.15.117:0/257760634 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55980bfa04b0 0x55980bf99d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:00.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.729+0000 7fe95b37db80 1 -- 172.21.15.117:0/257760634 >> 172.21.15.117:0/257760634 conn(0x55980bec6170 msgr2=0x55980bf94ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:27:00.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.729+0000 7fe95b37db80 1 -- 172.21.15.117:0/257760634 shutdown_connections 2024-06-03T03:27:00.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.729+0000 7fe95b37db80 1 -- 172.21.15.117:0/257760634 wait complete. 2024-06-03T03:27:00.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.730+0000 7fe95b37db80 1 -- start start 2024-06-03T03:27:00.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.730+0000 7fe95b37db80 1 Processor -- start 2024-06-03T03:27:00.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.731+0000 7fe95b37db80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55980bf9a240 0x55980bfa6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:00.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.731+0000 7fe95b37db80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55980bf9fb90 0x55980bfa3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:00.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.731+0000 7fe95b37db80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55980bfa04b0 0x55980bfa39a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:00.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.731+0000 7fe95b37db80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55980be87f60 con 0x55980bf9a240 2024-06-03T03:27:00.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.731+0000 7fe95b37db80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55980bee7ac0 con 0x55980bf9fb90 2024-06-03T03:27:00.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.731+0000 7fe95b37db80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55980bfb7e90 con 0x55980bfa04b0 2024-06-03T03:27:00.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.731+0000 7fe95b37db80 10 client.0 _ll_register_callbacks cb 0x55980be81380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:27:00.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.731+0000 7fe949e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55980bf9a240 0x55980bfa6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:00.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.731+0000 7fe949e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55980bf9a240 0x55980bfa6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:54970/0 (socket says 172.21.15.117:54970) 2024-06-03T03:27:00.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.731+0000 7fe949e00700 1 -- 172.21.15.117:0/2988342451 learned_addr learned my addr 172.21.15.117:0/2988342451 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:27:00.735 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.731+0000 7fe949400700 1 --2- 172.21.15.117:0/2988342451 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55980bf9fb90 0x55980bfa3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:00.735 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.731+0000 7fe94a800700 1 --2- 172.21.15.117:0/2988342451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55980bfa04b0 0x55980bfa39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:00.735 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.732+0000 7fe949e00700 1 -- 172.21.15.117:0/2988342451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55980bfa04b0 msgr2=0x55980bfa39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:00.735 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.732+0000 7fe949e00700 1 --2- 172.21.15.117:0/2988342451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55980bfa04b0 0x55980bfa39a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:00.735 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.732+0000 7fe949e00700 1 -- 172.21.15.117:0/2988342451 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55980bf9fb90 msgr2=0x55980bfa3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:00.735 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.732+0000 7fe949e00700 1 --2- 172.21.15.117:0/2988342451 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55980bf9fb90 0x55980bfa3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:00.735 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.732+0000 7fe949e00700 1 -- 172.21.15.117:0/2988342451 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55980be3a8d0 con 0x55980bf9a240 2024-06-03T03:27:00.735 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.732+0000 7fe949e00700 1 --2- 172.21.15.117:0/2988342451 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55980bf9a240 0x55980bfa6160 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe93c097ec0 tx=0x7fe93c09b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12af215104ac693b server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:27:00.735 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.732+0000 7fe92be00700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-06-03T03:27:00.735 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.732+0000 7fe92be00700 1 -- 172.21.15.117:0/2988342451 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe93c09b3c0 con 0x55980bf9a240 2024-06-03T03:27:00.735 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.732+0000 7fe92be00700 1 -- 172.21.15.117:0/2988342451 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7fe93c0cf2c0 con 0x55980bf9a240 2024-06-03T03:27:00.735 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.732+0000 7fe95b37db80 1 -- 172.21.15.117:0/2988342451 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe938005550 con 0x55980bf9a240 2024-06-03T03:27:00.735 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.733+0000 7fe92be00700 1 -- 172.21.15.117:0/2988342451 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe93c09c160 con 0x55980bf9a240 2024-06-03T03:27:00.736 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.733+0000 7fe95b37db80 10 client.15462 fetch_fsmap learned FSMap version 31 2024-06-03T03:27:00.736 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.733+0000 7fe95b37db80 1 -- 172.21.15.117:0/2988342451 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55980bfc25b0 con 0x55980bf9a240 2024-06-03T03:27:00.736 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.733+0000 7fe92be00700 1 -- 172.21.15.117:0/2988342451 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe938005550 con 0x55980bf9a240 2024-06-03T03:27:00.736 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.733+0000 7fe92be00700 1 -- 172.21.15.117:0/2988342451 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fe93c0ba260 con 0x55980bf9a240 2024-06-03T03:27:00.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.734+0000 7fe95b37db80 10 client.15462 fetch_fsmap finished waiting for FSMap version 31 2024-06-03T03:27:00.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.734+0000 7fe95b37db80 10 client.15462 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-06-03T03:27:00.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.734+0000 7fe95b37db80 20 client.15462 populate_metadata read hostname 'smithi117' 2024-06-03T03:27:00.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.734+0000 7fe95b37db80 1 --2- 172.21.15.117:0/2988342451 >> [v2:172.21.15.183:6836/2697518526,v1:172.21.15.183:6837/2697518526] conn(0x55980bfc4b40 0x55980bfc6f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:00.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.734+0000 7fe95b37db80 4 client.15462 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T03:27:00.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.734+0000 7fe95b37db80 1 -- 172.21.15.117:0/2988342451 --> [v2:172.21.15.183:6836/2697518526,v1:172.21.15.183:6837/2697518526] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55980bef9000 con 0x55980bfc4b40 2024-06-03T03:27:00.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.734+0000 7fe949400700 1 --2- 172.21.15.117:0/2988342451 >> [v2:172.21.15.183:6836/2697518526,v1:172.21.15.183:6837/2697518526] conn(0x55980bfc4b40 0x55980bfc6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:00.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.735+0000 7fe92be00700 10 client.15462 ms_handle_connect on v2:172.21.15.183:6836/2697518526 2024-06-03T03:27:00.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.735+0000 7fe949400700 1 --2- 172.21.15.117:0/2988342451 >> [v2:172.21.15.183:6836/2697518526,v1:172.21.15.183:6837/2697518526] conn(0x55980bfc4b40 0x55980bfc6f30 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d4c6606784b03f29 server_cookie=43fc2f00b2f423fa in_seq=0 out_seq=0 2024-06-03T03:27:00.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.736+0000 7fe92be00700 1 -- 172.21.15.117:0/2988342451 <== mds.3 v2:172.21.15.183:6836/2697518526 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55980bef9000 con 0x55980bfc4b40 2024-06-03T03:27:00.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.736+0000 7fe92be00700 10 client.15462 handle_command_reply: tid=0 2024-06-03T03:27:00.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.736+0000 7fe95b37db80 1 client.15462 shutdown 2024-06-03T03:27:00.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.736+0000 7fe95b37db80 1 -- 172.21.15.117:0/2988342451 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55980bf9a240 msgr2=0x55980bfa6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:00.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.736+0000 7fe95b37db80 1 --2- 172.21.15.117:0/2988342451 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55980bf9a240 0x55980bfa6160 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe93c097ec0 tx=0x7fe93c09b8b0 comp rx=0 tx=0).stop 2024-06-03T03:27:00.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.736+0000 7fe95b37db80 1 -- 172.21.15.117:0/2988342451 shutdown_connections 2024-06-03T03:27:00.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.736+0000 7fe95b37db80 1 --2- 172.21.15.117:0/2988342451 >> [v2:172.21.15.183:6836/2697518526,v1:172.21.15.183:6837/2697518526] conn(0x55980bfc4b40 0x55980bfc6f30 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:00.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.737+0000 7fe95b37db80 1 --2- 172.21.15.117:0/2988342451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55980bfa04b0 0x55980bfa39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:00.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.737+0000 7fe95b37db80 1 --2- 172.21.15.117:0/2988342451 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55980bf9a240 0x55980bfa6160 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:00.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.737+0000 7fe95b37db80 1 --2- 172.21.15.117:0/2988342451 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55980bf9fb90 0x55980bfa3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:00.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.737+0000 7fe95b37db80 1 -- 172.21.15.117:0/2988342451 >> 172.21.15.117:0/2988342451 conn(0x55980bec6170 msgr2=0x55980bf8e790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:27:00.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.737+0000 7fe92be00700 0 client.15462 ms_handle_reset on v2:172.21.15.183:6836/2697518526 2024-06-03T03:27:00.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.737+0000 7fe95b37db80 1 -- 172.21.15.117:0/2988342451 shutdown_connections 2024-06-03T03:27:00.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.737+0000 7fe95b37db80 1 -- 172.21.15.117:0/2988342451 wait complete. 2024-06-03T03:27:00.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.737+0000 7fe95b37db80 20 client.15462 trim_cache size 0 max 16384 2024-06-03T03:27:00.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.768+0000 7fe95b37db80 1 Processor -- start 2024-06-03T03:27:00.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.768+0000 7fe95b37db80 1 -- start start 2024-06-03T03:27:00.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.769+0000 7fe95b37db80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55980bf9a240 0x55980bf99d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:00.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.769+0000 7fe95b37db80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55980bf9fb90 0x55980bfa23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:00.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.769+0000 7fe95b37db80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55980bfa04b0 0x55980bfa1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:00.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.769+0000 7fe95b37db80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe94c1b4960 con 0x55980bf9a240 2024-06-03T03:27:00.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.769+0000 7fe95b37db80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe94c1b6480 con 0x55980bf9fb90 2024-06-03T03:27:00.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.769+0000 7fe95b37db80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe94c1b5010 con 0x55980bfa04b0 2024-06-03T03:27:00.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.769+0000 7fe949e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55980bf9a240 0x55980bf99d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:00.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.769+0000 7fe949e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55980bf9a240 0x55980bf99d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:54980/0 (socket says 172.21.15.117:54980) 2024-06-03T03:27:00.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.769+0000 7fe949e00700 1 -- 172.21.15.117:0/537159388 learned_addr learned my addr 172.21.15.117:0/537159388 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:27:00.771 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.769+0000 7fe94a800700 1 --2- 172.21.15.117:0/537159388 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55980bfa04b0 0x55980bfa1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:00.771 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.769+0000 7fe949400700 1 --2- 172.21.15.117:0/537159388 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55980bf9fb90 0x55980bfa23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:00.771 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.770+0000 7fe949e00700 1 -- 172.21.15.117:0/537159388 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55980bfa04b0 msgr2=0x55980bfa1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:00.771 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.770+0000 7fe949e00700 1 --2- 172.21.15.117:0/537159388 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55980bfa04b0 0x55980bfa1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:00.771 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.770+0000 7fe949e00700 1 -- 172.21.15.117:0/537159388 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55980bf9fb90 msgr2=0x55980bfa23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:00.772 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.770+0000 7fe949e00700 1 --2- 172.21.15.117:0/537159388 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55980bf9fb90 0x55980bfa23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:00.772 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.770+0000 7fe949e00700 1 -- 172.21.15.117:0/537159388 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55980bfa19f0 con 0x55980bf9a240 2024-06-03T03:27:00.772 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.770+0000 7fe949e00700 1 --2- 172.21.15.117:0/537159388 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55980bf9a240 0x55980bf99d00 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe93c00e970 tx=0x7fe93c0d2c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=675f0d7d8562d10e server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:27:00.772 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.770+0000 7fe936a00700 1 -- 172.21.15.117:0/537159388 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe93c09b3c0 con 0x55980bf9a240 2024-06-03T03:27:00.772 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.771+0000 7fe936a00700 1 -- 172.21.15.117:0/537159388 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe93c09fe10 con 0x55980bf9a240 2024-06-03T03:27:00.772 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.771+0000 7fe936a00700 1 -- 172.21.15.117:0/537159388 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe93c09b5a0 con 0x55980bf9a240 2024-06-03T03:27:00.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.771+0000 7fe95b37db80 1 -- 172.21.15.117:0/537159388 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55980bf9a240 msgr2=0x55980bf99d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:00.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.771+0000 7fe95b37db80 1 --2- 172.21.15.117:0/537159388 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55980bf9a240 0x55980bf99d00 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe93c00e970 tx=0x7fe93c0d2c50 comp rx=0 tx=0).stop 2024-06-03T03:27:00.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.771+0000 7fe95b37db80 1 -- 172.21.15.117:0/537159388 shutdown_connections 2024-06-03T03:27:00.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.771+0000 7fe95b37db80 1 --2- 172.21.15.117:0/537159388 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55980bfa04b0 0x55980bfa1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:00.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.771+0000 7fe95b37db80 1 --2- 172.21.15.117:0/537159388 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55980bf9a240 0x55980bf99d00 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:00.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.771+0000 7fe95b37db80 1 --2- 172.21.15.117:0/537159388 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55980bf9fb90 0x55980bfa23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:00.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.771+0000 7fe95b37db80 1 -- 172.21.15.117:0/537159388 >> 172.21.15.117:0/537159388 conn(0x55980bec6170 msgr2=0x55980bfa0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:27:00.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.772+0000 7fe95b37db80 1 -- 172.21.15.117:0/537159388 shutdown_connections 2024-06-03T03:27:00.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.772+0000 7fe95b37db80 1 -- 172.21.15.117:0/537159388 wait complete. 2024-06-03T03:27:00.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.772+0000 7fe95b37db80 1 -- start start 2024-06-03T03:27:00.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.773+0000 7fe95b37db80 1 Processor -- start 2024-06-03T03:27:00.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.773+0000 7fe95b37db80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55980bf9fb90 0x55980bfbd150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:00.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.773+0000 7fe95b37db80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55980bfa04b0 0x55980bfbd690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:00.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.773+0000 7fe95b37db80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55980bfbdbd0 0x55980bfcb780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:00.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.773+0000 7fe95b37db80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe94c1b5db0 con 0x55980bf9fb90 2024-06-03T03:27:00.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.773+0000 7fe95b37db80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe94c1b56e0 con 0x55980bfbdbd0 2024-06-03T03:27:00.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.773+0000 7fe95b37db80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe94c10d100 con 0x55980bfa04b0 2024-06-03T03:27:00.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.774+0000 7fe95b37db80 10 client.0 _ll_register_callbacks cb 0x55980bf9e000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:27:00.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.774+0000 7fe949400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55980bfa04b0 0x55980bfbd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:00.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.774+0000 7fe949400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55980bfa04b0 0x55980bfbd690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:35908/0 (socket says 172.21.15.117:35908) 2024-06-03T03:27:00.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.774+0000 7fe949400700 1 -- 172.21.15.117:0/4045373101 learned_addr learned my addr 172.21.15.117:0/4045373101 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:27:00.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.774+0000 7fe94a800700 1 --2- 172.21.15.117:0/4045373101 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55980bfbdbd0 0x55980bfcb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:00.776 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.774+0000 7fe949e00700 1 --2- 172.21.15.117:0/4045373101 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55980bf9fb90 0x55980bfbd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:00.776 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.774+0000 7fe949400700 1 -- 172.21.15.117:0/4045373101 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55980bfbdbd0 msgr2=0x55980bfcb780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:00.776 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.774+0000 7fe949400700 1 --2- 172.21.15.117:0/4045373101 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55980bfbdbd0 0x55980bfcb780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:00.776 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.775+0000 7fe949400700 1 -- 172.21.15.117:0/4045373101 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55980bf9fb90 msgr2=0x55980bfbd150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:00.776 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.775+0000 7fe949400700 1 --2- 172.21.15.117:0/4045373101 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55980bf9fb90 0x55980bfbd150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:00.776 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.775+0000 7fe949400700 1 -- 172.21.15.117:0/4045373101 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55980bfcbdc0 con 0x55980bfa04b0 2024-06-03T03:27:00.776 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.775+0000 7fe94a800700 1 --2- 172.21.15.117:0/4045373101 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55980bfbdbd0 0x55980bfcb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:27:00.776 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.775+0000 7fe949400700 1 --2- 172.21.15.117:0/4045373101 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55980bfa04b0 0x55980bfbd690 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe938009750 tx=0x7fe938005980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ab80074e16c5de1 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:27:00.776 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.775+0000 7fe92be00700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-06-03T03:27:00.777 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.775+0000 7fe92be00700 1 -- 172.21.15.117:0/4045373101 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe938031070 con 0x55980bfa04b0 2024-06-03T03:27:00.777 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.775+0000 7fe92be00700 1 -- 172.21.15.117:0/4045373101 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7fe93800bce0 con 0x55980bfa04b0 2024-06-03T03:27:00.777 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.776+0000 7fe95b37db80 1 -- 172.21.15.117:0/4045373101 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe93c09fe10 con 0x55980bfa04b0 2024-06-03T03:27:00.777 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.776+0000 7fe92be00700 1 -- 172.21.15.117:0/4045373101 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe93803b050 con 0x55980bfa04b0 2024-06-03T03:27:00.777 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.776+0000 7fe95b37db80 10 client.25606 fetch_fsmap learned FSMap version 31 2024-06-03T03:27:00.777 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.776+0000 7fe95b37db80 1 -- 172.21.15.117:0/4045373101 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55980c02f080 con 0x55980bfa04b0 2024-06-03T03:27:00.777 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.776+0000 7fe92be00700 1 -- 172.21.15.117:0/4045373101 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe93c09fe10 con 0x55980bfa04b0 2024-06-03T03:27:00.778 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.777+0000 7fe92be00700 1 -- 172.21.15.117:0/4045373101 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fe938021030 con 0x55980bfa04b0 2024-06-03T03:27:00.778 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.777+0000 7fe95b37db80 10 client.25606 fetch_fsmap finished waiting for FSMap version 31 2024-06-03T03:27:00.778 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.777+0000 7fe95b37db80 10 client.25606 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-06-03T03:27:00.778 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.777+0000 7fe95b37db80 20 client.25606 populate_metadata read hostname 'smithi117' 2024-06-03T03:27:00.778 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.777+0000 7fe95b37db80 1 --2- 172.21.15.117:0/4045373101 >> [v2:172.21.15.183:6836/2697518526,v1:172.21.15.183:6837/2697518526] conn(0x55980c02f360 0x55980c031750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:00.778 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.777+0000 7fe95b37db80 4 client.25606 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T03:27:00.778 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.777+0000 7fe95b37db80 1 -- 172.21.15.117:0/4045373101 --> [v2:172.21.15.183:6836/2697518526,v1:172.21.15.183:6837/2697518526] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55980bef3580 con 0x55980c02f360 2024-06-03T03:27:00.778 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.778+0000 7fe949e00700 1 --2- 172.21.15.117:0/4045373101 >> [v2:172.21.15.183:6836/2697518526,v1:172.21.15.183:6837/2697518526] conn(0x55980c02f360 0x55980c031750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:00.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.778+0000 7fe949e00700 1 --2- 172.21.15.117:0/4045373101 >> [v2:172.21.15.183:6836/2697518526,v1:172.21.15.183:6837/2697518526] conn(0x55980c02f360 0x55980c031750 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f14011158546ba50 server_cookie=8e81b8b59306a2af in_seq=0 out_seq=0 2024-06-03T03:27:00.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.778+0000 7fe92be00700 10 client.25606 ms_handle_connect on v2:172.21.15.183:6836/2697518526 2024-06-03T03:27:00.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.779+0000 7fe92be00700 1 -- 172.21.15.117:0/4045373101 <== mds.3 v2:172.21.15.183:6836/2697518526 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55980bef3580 con 0x55980c02f360 2024-06-03T03:27:00.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.779+0000 7fe92be00700 10 client.25606 handle_command_reply: tid=0 2024-06-03T03:27:00.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.779+0000 7fe95b37db80 1 client.25606 shutdown 2024-06-03T03:27:00.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.779+0000 7fe95b37db80 1 -- 172.21.15.117:0/4045373101 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55980bfa04b0 msgr2=0x55980bfbd690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:00.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.779+0000 7fe95b37db80 1 --2- 172.21.15.117:0/4045373101 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55980bfa04b0 0x55980bfbd690 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe938009750 tx=0x7fe938005980 comp rx=0 tx=0).stop 2024-06-03T03:27:00.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.779+0000 7fe95b37db80 1 -- 172.21.15.117:0/4045373101 shutdown_connections 2024-06-03T03:27:00.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.779+0000 7fe95b37db80 1 --2- 172.21.15.117:0/4045373101 >> [v2:172.21.15.183:6836/2697518526,v1:172.21.15.183:6837/2697518526] conn(0x55980c02f360 0x55980c031750 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:00.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.779+0000 7fe95b37db80 1 --2- 172.21.15.117:0/4045373101 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55980bfbdbd0 0x55980bfcb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:00.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.779+0000 7fe95b37db80 1 --2- 172.21.15.117:0/4045373101 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55980bf9fb90 0x55980bfbd150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:00.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.779+0000 7fe92be00700 0 client.25606 ms_handle_reset on v2:172.21.15.183:6836/2697518526 2024-06-03T03:27:00.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.779+0000 7fe95b37db80 1 --2- 172.21.15.117:0/4045373101 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55980bfa04b0 0x55980bfbd690 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:00.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.779+0000 7fe95b37db80 1 -- 172.21.15.117:0/4045373101 >> 172.21.15.117:0/4045373101 conn(0x55980bec6170 msgr2=0x55980bfa2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:27:00.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.779+0000 7fe95b37db80 1 -- 172.21.15.117:0/4045373101 shutdown_connections 2024-06-03T03:27:00.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.779+0000 7fe95b37db80 1 -- 172.21.15.117:0/4045373101 wait complete. 2024-06-03T03:27:00.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.780+0000 7fe95b37db80 20 client.25606 trim_cache size 0 max 16384 2024-06-03T03:27:00.782 INFO:teuthology.orchestra.run.smithi117.stdout:[] 2024-06-03T03:27:00.782 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.780+0000 7fe950a00700 1 -- 172.21.15.117:0/3636118111 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fe9300a2cd0 msgr2=0x7fe9300a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:00.782 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.780+0000 7fe950a00700 1 --2- 172.21.15.117:0/3636118111 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fe9300a2cd0 0x7fe9300a5190 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe944007610 tx=0x7fe94400b040 comp rx=0 tx=0).stop 2024-06-03T03:27:00.782 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.780+0000 7fe950a00700 1 -- 172.21.15.117:0/3636118111 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe94c0746c0 msgr2=0x7fe94c073d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:00.782 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.780+0000 7fe950a00700 1 --2- 172.21.15.117:0/3636118111 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe94c0746c0 0x7fe94c073d00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe93c00e9a0 tx=0x7fe93c00ecb0 comp rx=0 tx=0).stop 2024-06-03T03:27:00.782 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.781+0000 7fe950a00700 1 -- 172.21.15.117:0/3636118111 shutdown_connections 2024-06-03T03:27:00.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.781+0000 7fe950a00700 1 --2- 172.21.15.117:0/3636118111 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fe9300a2cd0 0x7fe9300a5190 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:00.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.781+0000 7fe950a00700 1 --2- 172.21.15.117:0/3636118111 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe94c113660 0x7fe94c06e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:00.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.781+0000 7fe950a00700 1 --2- 172.21.15.117:0/3636118111 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe94c0746c0 0x7fe94c073d00 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:00.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.781+0000 7fe950a00700 1 --2- 172.21.15.117:0/3636118111 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe94c074fe0 0x7fe94c06dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:00.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.781+0000 7fe950a00700 1 -- 172.21.15.117:0/3636118111 >> 172.21.15.117:0/3636118111 conn(0x7fe94c0795f0 msgr2=0x7fe94c112420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:27:00.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.782+0000 7fe950a00700 1 -- 172.21.15.117:0/3636118111 shutdown_connections 2024-06-03T03:27:00.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:00.782+0000 7fe950a00700 1 -- 172.21.15.117:0/3636118111 wait complete. 2024-06-03T03:27:01.598 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.597+0000 7f288b800640 1 -- 172.21.15.117:0/1528832275 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2884101a00 msgr2=0x7f2884101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:01.598 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.597+0000 7f288b800640 1 --2- 172.21.15.117:0/1528832275 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2884101a00 0x7f2884101e20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2878009900 tx=0x7f287802a3f0 comp rx=0 tx=0).stop 2024-06-03T03:27:01.598 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.598+0000 7f288b800640 1 -- 172.21.15.117:0/1528832275 shutdown_connections 2024-06-03T03:27:01.598 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.598+0000 7f288b800640 1 --2- 172.21.15.117:0/1528832275 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28841035b0 0x7f2884109e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:01.599 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.598+0000 7f288b800640 1 --2- 172.21.15.117:0/1528832275 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2884102bf0 0x7f2884103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:01.599 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.598+0000 7f288b800640 1 --2- 172.21.15.117:0/1528832275 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2884101a00 0x7f2884101e20 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:01.599 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.598+0000 7f288b800640 1 -- 172.21.15.117:0/1528832275 >> 172.21.15.117:0/1528832275 conn(0x7f28840fcf70 msgr2=0x7f28840ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:27:01.599 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.598+0000 7f288b800640 1 -- 172.21.15.117:0/1528832275 shutdown_connections 2024-06-03T03:27:01.599 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.598+0000 7f288b800640 1 -- 172.21.15.117:0/1528832275 wait complete. 2024-06-03T03:27:01.601 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.599+0000 7f288b800640 1 Processor -- start 2024-06-03T03:27:01.601 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.599+0000 7f288b800640 1 -- start start 2024-06-03T03:27:01.601 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.599+0000 7f288b800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2884101a00 0x7f2884199e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:01.601 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.599+0000 7f288b800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2884102bf0 0x7f288419a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:01.601 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.599+0000 7f288b800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28841035b0 0x7f28841a1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:01.601 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.599+0000 7f288b800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f288410d650 con 0x7f28841035b0 2024-06-03T03:27:01.601 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.599+0000 7f288b800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f288410d4d0 con 0x7f2884102bf0 2024-06-03T03:27:01.601 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.599+0000 7f288b800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f288410d7d0 con 0x7f2884101a00 2024-06-03T03:27:01.602 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.599+0000 7f2888e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2884101a00 0x7f2884199e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:01.602 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.599+0000 7f2888e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2884101a00 0x7f2884199e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:35920/0 (socket says 172.21.15.117:35920) 2024-06-03T03:27:01.602 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.599+0000 7f2888e00640 1 -- 172.21.15.117:0/2480485632 learned_addr learned my addr 172.21.15.117:0/2480485632 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:27:01.602 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.599+0000 7f2889800640 1 --2- 172.21.15.117:0/2480485632 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28841035b0 0x7f28841a1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:01.602 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.599+0000 7f2883e00640 1 --2- 172.21.15.117:0/2480485632 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2884102bf0 0x7f288419a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:01.602 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.600+0000 7f2888e00640 1 -- 172.21.15.117:0/2480485632 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2884102bf0 msgr2=0x7f288419a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:01.602 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.600+0000 7f2888e00640 1 --2- 172.21.15.117:0/2480485632 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2884102bf0 0x7f288419a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:01.602 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.600+0000 7f2888e00640 1 -- 172.21.15.117:0/2480485632 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28841035b0 msgr2=0x7f28841a1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:01.602 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.600+0000 7f2888e00640 1 --2- 172.21.15.117:0/2480485632 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28841035b0 0x7f28841a1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:01.602 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.600+0000 7f2888e00640 1 -- 172.21.15.117:0/2480485632 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28841a1aa0 con 0x7f2884101a00 2024-06-03T03:27:01.602 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.600+0000 7f2888e00640 1 --2- 172.21.15.117:0/2480485632 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2884101a00 0x7f2884199e80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2878031610 tx=0x7f2878004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:27:01.602 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.600+0000 7f2881600640 1 -- 172.21.15.117:0/2480485632 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f287803f030 con 0x7f2884101a00 2024-06-03T03:27:01.603 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.600+0000 7f2881600640 1 -- 172.21.15.117:0/2480485632 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2878003ad0 con 0x7f2884101a00 2024-06-03T03:27:01.603 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.600+0000 7f288b800640 1 -- 172.21.15.117:0/2480485632 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28841a1d30 con 0x7f2884101a00 2024-06-03T03:27:01.603 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.600+0000 7f2881600640 1 -- 172.21.15.117:0/2480485632 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2878035070 con 0x7f2884101a00 2024-06-03T03:27:01.603 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.601+0000 7f288b800640 1 -- 172.21.15.117:0/2480485632 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28841a21d0 con 0x7f2884101a00 2024-06-03T03:27:01.605 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.603+0000 7f288e678740 1 Processor -- start 2024-06-03T03:27:01.605 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.604+0000 7f288e678740 1 -- start start 2024-06-03T03:27:01.605 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.604+0000 7f2881600640 1 -- 172.21.15.117:0/2480485632 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f2878042030 con 0x7f2884101a00 2024-06-03T03:27:01.605 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.604+0000 7f288e678740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b553bd5bd0 0x55b553bd5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:01.605 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.604+0000 7f288e678740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b553bd65c0 0x55b553bdb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:01.605 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.604+0000 7f288e678740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b553bdb730 0x55b553bddbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:01.605 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.604+0000 7f288e678740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b553b39420 con 0x55b553bd65c0 2024-06-03T03:27:01.605 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.604+0000 7f288e678740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b553ae0c50 con 0x55b553bdb730 2024-06-03T03:27:01.605 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.604+0000 7f288e678740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b553b353e0 con 0x55b553bd5bd0 2024-06-03T03:27:01.605 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.604+0000 7f2888e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b553bd65c0 0x55b553bdb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:01.606 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.604+0000 7f2888e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b553bd65c0 0x55b553bdb0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:55012/0 (socket says 172.21.15.117:55012) 2024-06-03T03:27:01.606 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.604+0000 7f2888e00640 1 -- 172.21.15.117:0/75316495 learned_addr learned my addr 172.21.15.117:0/75316495 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:27:01.606 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.604+0000 7f2889800640 1 --2- 172.21.15.117:0/75316495 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b553bd5bd0 0x55b553bd5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:01.606 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.605+0000 7f2883e00640 1 --2- 172.21.15.117:0/75316495 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b553bdb730 0x55b553bddbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:01.608 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.605+0000 7f2888e00640 1 -- 172.21.15.117:0/75316495 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b553bd5bd0 msgr2=0x55b553bd5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:01.608 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.605+0000 7f2888e00640 1 --2- 172.21.15.117:0/75316495 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b553bd5bd0 0x55b553bd5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:01.608 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.605+0000 7f2888e00640 1 -- 172.21.15.117:0/75316495 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b553bdb730 msgr2=0x55b553bddbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:01.608 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.605+0000 7f2888e00640 1 --2- 172.21.15.117:0/75316495 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b553bdb730 0x55b553bddbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:01.608 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.605+0000 7f2888e00640 1 -- 172.21.15.117:0/75316495 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b55384d210 con 0x55b553bd65c0 2024-06-03T03:27:01.608 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.605+0000 7f2883e00640 1 --2- 172.21.15.117:0/75316495 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b553bdb730 0x55b553bddbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T03:27:01.608 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.605+0000 7f2889800640 1 --2- 172.21.15.117:0/75316495 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b553bd5bd0 0x55b553bd5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:27:01.608 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.605+0000 7f2888e00640 1 --2- 172.21.15.117:0/75316495 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b553bd65c0 0x55b553bdb0c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f28780f5bc0 tx=0x7f28780f5cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3673bdce77964458 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:27:01.608 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.606+0000 7f2881600640 1 --2- 172.21.15.117:0/2480485632 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f286c0a2c30 0x7f286c0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:01.608 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.606+0000 7f2876000640 1 -- 172.21.15.117:0/75316495 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28780ff020 con 0x55b553bd65c0 2024-06-03T03:27:01.608 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.606+0000 7f2876000640 1 -- 172.21.15.117:0/75316495 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f287803be40 con 0x55b553bd65c0 2024-06-03T03:27:01.609 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.606+0000 7f2889800640 1 --2- 172.21.15.117:0/2480485632 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f286c0a2c30 0x7f286c0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:01.609 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.606+0000 7f2876000640 1 -- 172.21.15.117:0/75316495 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28780bb8b0 con 0x55b553bd65c0 2024-06-03T03:27:01.609 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.606+0000 7f2881600640 1 -- 172.21.15.117:0/2480485632 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f28780efa30 con 0x7f2884101a00 2024-06-03T03:27:01.609 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.607+0000 7f288e678740 1 -- 172.21.15.117:0/75316495 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b553bd65c0 msgr2=0x55b553bdb0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:01.609 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.607+0000 7f288e678740 1 --2- 172.21.15.117:0/75316495 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b553bd65c0 0x55b553bdb0c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f28780f5bc0 tx=0x7f28780f5cb0 comp rx=0 tx=0).stop 2024-06-03T03:27:01.609 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.607+0000 7f2889800640 1 --2- 172.21.15.117:0/2480485632 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f286c0a2c30 0x7f286c0a50f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x55b553bd6e20 tx=0x7f287c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:27:01.609 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.607+0000 7f288e678740 1 -- 172.21.15.117:0/75316495 shutdown_connections 2024-06-03T03:27:01.609 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.607+0000 7f288e678740 1 --2- 172.21.15.117:0/75316495 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b553bdb730 0x55b553bddbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:01.609 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.607+0000 7f288e678740 1 --2- 172.21.15.117:0/75316495 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b553bd65c0 0x55b553bdb0c0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:01.609 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.607+0000 7f288e678740 1 --2- 172.21.15.117:0/75316495 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b553bd5bd0 0x55b553bd5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:01.609 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.607+0000 7f288e678740 1 -- 172.21.15.117:0/75316495 >> 172.21.15.117:0/75316495 conn(0x55b553b9df80 msgr2=0x55b553bd1650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:27:01.609 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.607+0000 7f288e678740 1 -- 172.21.15.117:0/75316495 shutdown_connections 2024-06-03T03:27:01.610 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.607+0000 7f288e678740 1 -- 172.21.15.117:0/75316495 wait complete. 2024-06-03T03:27:01.610 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.607+0000 7f288e678740 1 -- start start 2024-06-03T03:27:01.610 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.608+0000 7f288e678740 1 Processor -- start 2024-06-03T03:27:01.613 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.608+0000 7f288e678740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b553bd5bd0 0x55b553bdfc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:01.613 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.608+0000 7f288e678740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b553bd65c0 0x55b553be0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:01.613 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.608+0000 7f288e678740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b553bdb730 0x55b553be7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:01.613 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.608+0000 7f288e678740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b553bde4a0 con 0x55b553bd5bd0 2024-06-03T03:27:01.614 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.608+0000 7f288e678740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b553af40b0 con 0x55b553bdb730 2024-06-03T03:27:01.614 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.608+0000 7f288e678740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b553bde620 con 0x55b553bd65c0 2024-06-03T03:27:01.614 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.608+0000 7f288e678740 10 client.0 _ll_register_callbacks cb 0x55b553adff40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:27:01.614 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.608+0000 7f2888e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b553bd5bd0 0x55b553bdfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:01.614 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.608+0000 7f2888e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b553bd5bd0 0x55b553bdfc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:55018/0 (socket says 172.21.15.117:55018) 2024-06-03T03:27:01.614 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.608+0000 7f2888e00640 1 -- 172.21.15.117:0/1666692197 learned_addr learned my addr 172.21.15.117:0/1666692197 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:27:01.614 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.609+0000 7f2889800640 1 --2- 172.21.15.117:0/1666692197 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b553bdb730 0x55b553be7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:01.614 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.609+0000 7f2883e00640 1 --2- 172.21.15.117:0/1666692197 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b553bd65c0 0x55b553be0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:01.614 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.609+0000 7f2888e00640 1 -- 172.21.15.117:0/1666692197 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b553bd65c0 msgr2=0x55b553be0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:01.614 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.609+0000 7f2888e00640 1 --2- 172.21.15.117:0/1666692197 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b553bd65c0 0x55b553be0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:01.614 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.609+0000 7f2888e00640 1 -- 172.21.15.117:0/1666692197 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b553bdb730 msgr2=0x55b553be7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:01.614 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.609+0000 7f2888e00640 1 --2- 172.21.15.117:0/1666692197 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b553bdb730 0x55b553be7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:01.615 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.609+0000 7f2888e00640 1 -- 172.21.15.117:0/1666692197 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b553919620 con 0x55b553bd5bd0 2024-06-03T03:27:01.615 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.609+0000 7f2889800640 1 --2- 172.21.15.117:0/1666692197 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b553bdb730 0x55b553be7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:27:01.615 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.609+0000 7f2888e00640 1 --2- 172.21.15.117:0/1666692197 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b553bd5bd0 0x55b553bdfc30 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f2878009560 tx=0x7f28780f52a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cad8391a47bc3012 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:27:01.615 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.609+0000 7f286aa00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-06-03T03:27:01.615 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.609+0000 7f286aa00640 1 -- 172.21.15.117:0/1666692197 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2878076b50 con 0x55b553bd5bd0 2024-06-03T03:27:01.615 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.609+0000 7f286aa00640 1 -- 172.21.15.117:0/1666692197 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f287803b460 con 0x55b553bd5bd0 2024-06-03T03:27:01.615 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.610+0000 7f288e678740 1 -- 172.21.15.117:0/1666692197 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b553b46b50 con 0x55b553bd5bd0 2024-06-03T03:27:01.615 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.610+0000 7f286aa00640 1 -- 172.21.15.117:0/1666692197 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f2878049070 con 0x55b553bd5bd0 2024-06-03T03:27:01.615 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.610+0000 7f288e678740 10 client.15480 fetch_fsmap learned FSMap version 31 2024-06-03T03:27:01.615 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.610+0000 7f288e678740 1 -- 172.21.15.117:0/1666692197 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b553bde4a0 con 0x55b553bd5bd0 2024-06-03T03:27:01.615 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.610+0000 7f286aa00640 1 -- 172.21.15.117:0/1666692197 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55b553b46b50 con 0x55b553bd5bd0 2024-06-03T03:27:01.615 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.610+0000 7f286aa00640 1 -- 172.21.15.117:0/1666692197 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f28780ed1c0 con 0x55b553bd5bd0 2024-06-03T03:27:01.615 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.610+0000 7f288e678740 10 client.15480 fetch_fsmap finished waiting for FSMap version 31 2024-06-03T03:27:01.616 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.610+0000 7f288e678740 10 client.15480 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-03T03:27:01.616 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.610+0000 7f288e678740 20 client.15480 populate_metadata read hostname 'smithi117' 2024-06-03T03:27:01.616 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.610+0000 7f288e678740 1 --2- 172.21.15.117:0/1666692197 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55b553beb980 0x55b553bedd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:01.616 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.610+0000 7f288e678740 4 client.15480 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T03:27:01.616 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.610+0000 7f288e678740 1 -- 172.21.15.117:0/1666692197 --> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b553b40ea0 con 0x55b553beb980 2024-06-03T03:27:01.616 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.611+0000 7f2883e00640 1 --2- 172.21.15.117:0/1666692197 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55b553beb980 0x55b553bedd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:01.616 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.611+0000 7f286aa00640 10 client.15480 ms_handle_connect on v2:172.21.15.183:6838/1439615083 2024-06-03T03:27:01.616 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.611+0000 7f2883e00640 1 --2- 172.21.15.117:0/1666692197 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55b553beb980 0x55b553bedd70 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c9bdf68e863ab20 server_cookie=82ef3c4ac503dee2 in_seq=0 out_seq=0 2024-06-03T03:27:01.616 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.612+0000 7f286aa00640 1 -- 172.21.15.117:0/1666692197 <== mds.0 v2:172.21.15.183:6838/1439615083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b553b40ea0 con 0x55b553beb980 2024-06-03T03:27:01.616 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.612+0000 7f286aa00640 10 client.15480 handle_command_reply: tid=0 2024-06-03T03:27:01.616 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.612+0000 7f288e678740 1 client.15480 shutdown 2024-06-03T03:27:01.616 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.613+0000 7f288e678740 1 -- 172.21.15.117:0/1666692197 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b553bd5bd0 msgr2=0x55b553bdfc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:01.616 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.613+0000 7f288e678740 1 --2- 172.21.15.117:0/1666692197 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b553bd5bd0 0x55b553bdfc30 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f2878009560 tx=0x7f28780f52a0 comp rx=0 tx=0).stop 2024-06-03T03:27:01.617 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.613+0000 7f288e678740 1 -- 172.21.15.117:0/1666692197 shutdown_connections 2024-06-03T03:27:01.617 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.613+0000 7f288e678740 1 --2- 172.21.15.117:0/1666692197 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55b553beb980 0x55b553bedd70 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:01.617 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.613+0000 7f288e678740 1 --2- 172.21.15.117:0/1666692197 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b553bdb730 0x55b553be7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:01.617 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.613+0000 7f288e678740 1 --2- 172.21.15.117:0/1666692197 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b553bd65c0 0x55b553be0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:01.617 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.613+0000 7f288e678740 1 --2- 172.21.15.117:0/1666692197 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b553bd5bd0 0x55b553bdfc30 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:01.617 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.613+0000 7f286aa00640 0 client.15480 ms_handle_reset on v2:172.21.15.183:6838/1439615083 2024-06-03T03:27:01.617 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.613+0000 7f288e678740 1 -- 172.21.15.117:0/1666692197 >> 172.21.15.117:0/1666692197 conn(0x55b553b9df80 msgr2=0x55b553bd0810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:27:01.617 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.613+0000 7f288e678740 1 -- 172.21.15.117:0/1666692197 shutdown_connections 2024-06-03T03:27:01.617 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.613+0000 7f288e678740 1 -- 172.21.15.117:0/1666692197 wait complete. 2024-06-03T03:27:01.617 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.613+0000 7f288e678740 20 client.15480 trim_cache size 0 max 16384 2024-06-03T03:27:01.646 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.645+0000 7f288e678740 1 Processor -- start 2024-06-03T03:27:01.648 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.646+0000 7f288e678740 1 -- start start 2024-06-03T03:27:01.648 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.646+0000 7f288e678740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b553bd5bd0 0x55b553bdff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:01.648 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.646+0000 7f288e678740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b553bd65c0 0x55b553be70d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:01.648 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.646+0000 7f288e678740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b553bdb730 0x55b553bf3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:01.648 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.646+0000 7f288e678740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f28841ac670 con 0x55b553bdb730 2024-06-03T03:27:01.648 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.646+0000 7f288e678740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f28841ae1b0 con 0x55b553bd65c0 2024-06-03T03:27:01.648 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.646+0000 7f288e678740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f28841acd40 con 0x55b553bd5bd0 2024-06-03T03:27:01.648 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.647+0000 7f2888e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b553bd5bd0 0x55b553bdff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:01.648 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.647+0000 7f2888e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b553bd5bd0 0x55b553bdff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:35954/0 (socket says 172.21.15.117:35954) 2024-06-03T03:27:01.649 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.647+0000 7f2888e00640 1 -- 172.21.15.117:0/4253093223 learned_addr learned my addr 172.21.15.117:0/4253093223 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:27:01.649 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.647+0000 7f2889800640 1 --2- 172.21.15.117:0/4253093223 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b553bdb730 0x55b553bf3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:01.649 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.647+0000 7f2888e00640 1 -- 172.21.15.117:0/4253093223 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b553bd65c0 msgr2=0x55b553be70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:01.649 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.647+0000 7f2888e00640 1 --2- 172.21.15.117:0/4253093223 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b553bd65c0 0x55b553be70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:01.649 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.647+0000 7f2888e00640 1 -- 172.21.15.117:0/4253093223 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b553bdb730 msgr2=0x55b553bf3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:01.649 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.647+0000 7f2888e00640 1 --2- 172.21.15.117:0/4253093223 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b553bdb730 0x55b553bf3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:01.649 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.647+0000 7f2888e00640 1 -- 172.21.15.117:0/4253093223 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b553bee3c0 con 0x55b553bd5bd0 2024-06-03T03:27:01.649 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.647+0000 7f2888e00640 1 --2- 172.21.15.117:0/4253093223 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b553bd5bd0 0x55b553bdff50 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2878031440 tx=0x7f28780ba460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7d737062560b316 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:27:01.649 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.648+0000 7f2875600640 1 -- 172.21.15.117:0/4253093223 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28780ff070 con 0x55b553bd5bd0 2024-06-03T03:27:01.649 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.648+0000 7f2875600640 1 -- 172.21.15.117:0/4253093223 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28780f7610 con 0x55b553bd5bd0 2024-06-03T03:27:01.649 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.648+0000 7f2875600640 1 -- 172.21.15.117:0/4253093223 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2878073f50 con 0x55b553bd5bd0 2024-06-03T03:27:01.652 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.649+0000 7f288e678740 1 -- 172.21.15.117:0/4253093223 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b553bd5bd0 msgr2=0x55b553bdff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:01.652 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.649+0000 7f288e678740 1 --2- 172.21.15.117:0/4253093223 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b553bd5bd0 0x55b553bdff50 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2878031440 tx=0x7f28780ba460 comp rx=0 tx=0).stop 2024-06-03T03:27:01.652 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.649+0000 7f288e678740 1 -- 172.21.15.117:0/4253093223 shutdown_connections 2024-06-03T03:27:01.652 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.649+0000 7f288e678740 1 --2- 172.21.15.117:0/4253093223 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b553bdb730 0x55b553bf3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:01.652 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.649+0000 7f288e678740 1 --2- 172.21.15.117:0/4253093223 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b553bd65c0 0x55b553be70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:01.652 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.649+0000 7f288e678740 1 --2- 172.21.15.117:0/4253093223 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b553bd5bd0 0x55b553bdff50 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:01.652 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.649+0000 7f288e678740 1 -- 172.21.15.117:0/4253093223 >> 172.21.15.117:0/4253093223 conn(0x55b553b9df80 msgr2=0x55b553bdf840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:27:01.653 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.649+0000 7f288e678740 1 -- 172.21.15.117:0/4253093223 shutdown_connections 2024-06-03T03:27:01.653 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.649+0000 7f288e678740 1 -- 172.21.15.117:0/4253093223 wait complete. 2024-06-03T03:27:01.653 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.650+0000 7f288e678740 1 -- start start 2024-06-03T03:27:01.653 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.650+0000 7f288e678740 1 Processor -- start 2024-06-03T03:27:01.653 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.651+0000 7f288e678740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b553bd5bd0 0x55b553be22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:01.653 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.651+0000 7f288e678740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b553bd65c0 0x55b553be2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:01.653 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.651+0000 7f288e678740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b553bdb730 0x55b553c65620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:01.653 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.651+0000 7f288e678740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f28841adae0 con 0x55b553bdb730 2024-06-03T03:27:01.653 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.651+0000 7f288e678740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f28841ad410 con 0x55b553bd65c0 2024-06-03T03:27:01.653 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.651+0000 7f288e678740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f28841aa650 con 0x55b553bd5bd0 2024-06-03T03:27:01.653 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.651+0000 7f288e678740 10 client.0 _ll_register_callbacks cb 0x7f287c009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:27:01.653 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.651+0000 7f2889800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b553bdb730 0x55b553c65620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:01.654 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.651+0000 7f2889800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b553bdb730 0x55b553c65620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:55046/0 (socket says 172.21.15.117:55046) 2024-06-03T03:27:01.654 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.651+0000 7f2889800640 1 -- 172.21.15.117:0/2149077399 learned_addr learned my addr 172.21.15.117:0/2149077399 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:27:01.654 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.651+0000 7f2883e00640 1 --2- 172.21.15.117:0/2149077399 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b553bd65c0 0x55b553be2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:01.654 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.651+0000 7f2888e00640 1 --2- 172.21.15.117:0/2149077399 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b553bd5bd0 0x55b553be22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:01.656 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.652+0000 7f2883e00640 1 -- 172.21.15.117:0/2149077399 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b553bd5bd0 msgr2=0x55b553be22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:01.656 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.652+0000 7f2883e00640 1 --2- 172.21.15.117:0/2149077399 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b553bd5bd0 0x55b553be22d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:01.656 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.652+0000 7f2883e00640 1 -- 172.21.15.117:0/2149077399 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b553bdb730 msgr2=0x55b553c65620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:01.656 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.652+0000 7f2883e00640 1 --2- 172.21.15.117:0/2149077399 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b553bdb730 0x55b553c65620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:01.656 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.652+0000 7f2883e00640 1 -- 172.21.15.117:0/2149077399 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b553c65c10 con 0x55b553bd65c0 2024-06-03T03:27:01.657 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.652+0000 7f2883e00640 1 --2- 172.21.15.117:0/2149077399 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b553bd65c0 0x55b553be2810 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x55b553bee5c0 tx=0x7f2870012820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c05521af28e3d42 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:27:01.657 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.652+0000 7f286aa00640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-06-03T03:27:01.657 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.652+0000 7f286aa00640 1 -- 172.21.15.117:0/2149077399 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2870023030 con 0x55b553bd65c0 2024-06-03T03:27:01.657 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.652+0000 7f288e678740 1 -- 172.21.15.117:0/2149077399 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28780f7610 con 0x55b553bd65c0 2024-06-03T03:27:01.657 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.652+0000 7f286aa00640 1 -- 172.21.15.117:0/2149077399 <== mon.2 v2:172.21.15.151:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f2870005020 con 0x55b553bd65c0 2024-06-03T03:27:01.657 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.653+0000 7f286aa00640 1 -- 172.21.15.117:0/2149077399 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f287002d050 con 0x55b553bd65c0 2024-06-03T03:27:01.657 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.653+0000 7f288e678740 10 client.25580 fetch_fsmap learned FSMap version 31 2024-06-03T03:27:01.657 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.653+0000 7f288e678740 1 -- 172.21.15.117:0/2149077399 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b553c66fe0 con 0x55b553bd65c0 2024-06-03T03:27:01.657 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.653+0000 7f286aa00640 1 -- 172.21.15.117:0/2149077399 <== mon.2 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28780f7610 con 0x55b553bd65c0 2024-06-03T03:27:01.657 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.653+0000 7f286aa00640 1 -- 172.21.15.117:0/2149077399 <== mon.2 v2:172.21.15.151:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f28700047a0 con 0x55b553bd65c0 2024-06-03T03:27:01.657 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.653+0000 7f288e678740 10 client.25580 fetch_fsmap finished waiting for FSMap version 31 2024-06-03T03:27:01.657 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.653+0000 7f288e678740 10 client.25580 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-03T03:27:01.657 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.653+0000 7f288e678740 20 client.25580 populate_metadata read hostname 'smithi117' 2024-06-03T03:27:01.658 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.653+0000 7f288e678740 1 --2- 172.21.15.117:0/2149077399 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55b553c679e0 0x55b553c69e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:01.658 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.653+0000 7f288e678740 4 client.25580 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-03T03:27:01.658 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.653+0000 7f288e678740 1 -- 172.21.15.117:0/2149077399 --> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b553abe7a0 con 0x55b553c679e0 2024-06-03T03:27:01.658 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.654+0000 7f2888e00640 1 --2- 172.21.15.117:0/2149077399 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55b553c679e0 0x55b553c69e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:01.658 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.655+0000 7f286aa00640 10 client.25580 ms_handle_connect on v2:172.21.15.183:6838/1439615083 2024-06-03T03:27:01.658 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.654+0000 7f2888e00640 1 --2- 172.21.15.117:0/2149077399 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55b553c679e0 0x55b553c69e00 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=625ae3077073e227 server_cookie=77ffd62d9a8ef4fc in_seq=0 out_seq=0 2024-06-03T03:27:01.668 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.667+0000 7f286aa00640 1 -- 172.21.15.117:0/2149077399 <== mds.0 v2:172.21.15.183:6838/1439615083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55b553abe7a0 con 0x55b553c679e0 2024-06-03T03:27:01.668 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.667+0000 7f286aa00640 10 client.25580 handle_command_reply: tid=0 2024-06-03T03:27:01.668 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.667+0000 7f288e678740 1 client.25580 shutdown 2024-06-03T03:27:01.668 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.667+0000 7f288e678740 1 -- 172.21.15.117:0/2149077399 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b553bd65c0 msgr2=0x55b553be2810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:01.668 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.667+0000 7f288e678740 1 --2- 172.21.15.117:0/2149077399 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b553bd65c0 0x55b553be2810 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x55b553bee5c0 tx=0x7f2870012820 comp rx=0 tx=0).stop 2024-06-03T03:27:01.669 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.667+0000 7f288e678740 1 -- 172.21.15.117:0/2149077399 shutdown_connections 2024-06-03T03:27:01.669 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.667+0000 7f288e678740 1 --2- 172.21.15.117:0/2149077399 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55b553c679e0 0x55b553c69e00 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:01.669 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.667+0000 7f288e678740 1 --2- 172.21.15.117:0/2149077399 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b553bdb730 0x55b553c65620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:01.669 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.667+0000 7f288e678740 1 --2- 172.21.15.117:0/2149077399 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b553bd65c0 0x55b553be2810 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:01.669 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.667+0000 7f286aa00640 0 client.25580 ms_handle_reset on v2:172.21.15.183:6838/1439615083 2024-06-03T03:27:01.669 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.667+0000 7f288e678740 1 --2- 172.21.15.117:0/2149077399 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b553bd5bd0 0x55b553be22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:01.669 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.668+0000 7f288e678740 1 -- 172.21.15.117:0/2149077399 >> 172.21.15.117:0/2149077399 conn(0x55b553b9df80 msgr2=0x55b553bd4540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:27:01.669 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.668+0000 7f288e678740 1 -- 172.21.15.117:0/2149077399 shutdown_connections 2024-06-03T03:27:01.669 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.668+0000 7f288e678740 1 -- 172.21.15.117:0/2149077399 wait complete. 2024-06-03T03:27:01.669 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.668+0000 7f288e678740 20 client.25580 trim_cache size 0 max 16384 2024-06-03T03:27:01.670 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:{ 2024-06-03T03:27:01.670 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "return_code": 0 2024-06-03T03:27:01.670 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:} 2024-06-03T03:27:01.670 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.669+0000 7f288b800640 1 -- 172.21.15.117:0/2480485632 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f286c0a2c30 msgr2=0x7f286c0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:01.671 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.669+0000 7f288b800640 1 --2- 172.21.15.117:0/2480485632 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f286c0a2c30 0x7f286c0a50f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x55b553bd6e20 tx=0x7f287c009500 comp rx=0 tx=0).stop 2024-06-03T03:27:01.671 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.669+0000 7f288b800640 1 -- 172.21.15.117:0/2480485632 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2884101a00 msgr2=0x7f2884199e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:01.671 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.669+0000 7f288b800640 1 --2- 172.21.15.117:0/2480485632 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2884101a00 0x7f2884199e80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2878031610 tx=0x7f2878004270 comp rx=0 tx=0).stop 2024-06-03T03:27:01.671 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.669+0000 7f288b800640 1 -- 172.21.15.117:0/2480485632 shutdown_connections 2024-06-03T03:27:01.671 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.669+0000 7f288b800640 1 --2- 172.21.15.117:0/2480485632 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28841035b0 0x7f28841a1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:01.671 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.669+0000 7f288b800640 1 --2- 172.21.15.117:0/2480485632 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f286c0a2c30 0x7f286c0a50f0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:01.671 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.669+0000 7f288b800640 1 --2- 172.21.15.117:0/2480485632 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2884102bf0 0x7f288419a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:01.671 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.669+0000 7f288b800640 1 --2- 172.21.15.117:0/2480485632 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2884101a00 0x7f2884199e80 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:01.671 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.669+0000 7f288b800640 1 -- 172.21.15.117:0/2480485632 >> 172.21.15.117:0/2480485632 conn(0x7f28840fcf70 msgr2=0x7f2884073bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:27:01.671 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.669+0000 7f288b800640 1 -- 172.21.15.117:0/2480485632 shutdown_connections 2024-06-03T03:27:01.671 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:01.669+0000 7f288b800640 1 -- 172.21.15.117:0/2480485632 wait complete. 2024-06-03T03:27:01.985 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-03T03:27:02.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:27:02 smithi151 ceph-mon[37043]: pgmap v557: 129 pgs: 129 active+clean; 1.7 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 171 KiB/s wr, 10 op/s 2024-06-03T03:27:02.412 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:27:02.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:27:02 smithi117 ceph-mon[31649]: pgmap v557: 129 pgs: 129 active+clean; 1.7 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 171 KiB/s wr, 10 op/s 2024-06-03T03:27:02.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:27:02 smithi183 ceph-mon[37326]: pgmap v557: 129 pgs: 129 active+clean; 1.7 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 171 KiB/s wr, 10 op/s 2024-06-03T03:27:03.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:27:03 smithi151 ceph-mon[37043]: pgmap v558: 129 pgs: 129 active+clean; 1.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 178 KiB/s wr, 15 op/s 2024-06-03T03:27:03.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:27:03 smithi117 ceph-mon[31649]: pgmap v558: 129 pgs: 129 active+clean; 1.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 178 KiB/s wr, 15 op/s 2024-06-03T03:27:03.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:27:03 smithi183 ceph-mon[37326]: pgmap v558: 129 pgs: 129 active+clean; 1.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 178 KiB/s wr, 15 op/s 2024-06-03T03:27:04.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.897+0000 7fc306200700 1 -- 172.21.15.117:0/2360480983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc300108500 msgr2=0x7fc3001088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:04.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.897+0000 7fc306200700 1 --2- 172.21.15.117:0/2360480983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc300108500 0x7fc3001088e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc2f0009990 tx=0x7fc2f0009ca0 comp rx=0 tx=0).stop 2024-06-03T03:27:04.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.897+0000 7fc306200700 1 -- 172.21.15.117:0/2360480983 shutdown_connections 2024-06-03T03:27:04.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.897+0000 7fc306200700 1 --2- 172.21.15.117:0/2360480983 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc30006ab30 0x7fc30010f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:04.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.897+0000 7fc306200700 1 --2- 172.21.15.117:0/2360480983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc300108500 0x7fc3001088e0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:04.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.897+0000 7fc306200700 1 --2- 172.21.15.117:0/2360480983 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc30006a170 0x7fc30006a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:04.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.897+0000 7fc306200700 1 -- 172.21.15.117:0/2360480983 >> 172.21.15.117:0/2360480983 conn(0x7fc300078340 msgr2=0x7fc300078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:27:04.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.897+0000 7fc306200700 1 -- 172.21.15.117:0/2360480983 shutdown_connections 2024-06-03T03:27:04.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.897+0000 7fc306200700 1 -- 172.21.15.117:0/2360480983 wait complete. 2024-06-03T03:27:04.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.898+0000 7fc306200700 1 Processor -- start 2024-06-03T03:27:04.923 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.898+0000 7fc306200700 1 -- start start 2024-06-03T03:27:04.923 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.899+0000 7fc306200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc30006a170 0x7fc30019adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:04.923 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.899+0000 7fc306200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc30006ab30 0x7fc30019b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:04.923 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.899+0000 7fc306200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc300108500 0x7fc30019f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:04.923 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.899+0000 7fc306200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc300112760 con 0x7fc30006ab30 2024-06-03T03:27:04.923 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.899+0000 7fc306200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc3001125e0 con 0x7fc30006a170 2024-06-03T03:27:04.923 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.899+0000 7fc306200700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc3001128e0 con 0x7fc300108500 2024-06-03T03:27:04.923 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.899+0000 7fc2fea00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc30006ab30 0x7fc30019b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:04.923 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.899+0000 7fc2ff400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc30006a170 0x7fc30019adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:04.923 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.899+0000 7fc2fea00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc30006ab30 0x7fc30019b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:55064/0 (socket says 172.21.15.117:55064) 2024-06-03T03:27:04.923 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.899+0000 7fc2ff400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc30006a170 0x7fc30019adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:50608/0 (socket says 172.21.15.117:50608) 2024-06-03T03:27:04.923 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.899+0000 7fc2fea00700 1 -- 172.21.15.117:0/2542090438 learned_addr learned my addr 172.21.15.117:0/2542090438 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:27:04.923 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.899+0000 7fc2ffe00700 1 --2- 172.21.15.117:0/2542090438 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc300108500 0x7fc30019f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:04.924 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.900+0000 7fc2fea00700 1 -- 172.21.15.117:0/2542090438 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc300108500 msgr2=0x7fc30019f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:04.924 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.900+0000 7fc2fea00700 1 --2- 172.21.15.117:0/2542090438 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc300108500 0x7fc30019f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:04.924 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.900+0000 7fc2fea00700 1 -- 172.21.15.117:0/2542090438 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc30006a170 msgr2=0x7fc30019adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:04.924 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.900+0000 7fc2fea00700 1 --2- 172.21.15.117:0/2542090438 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc30006a170 0x7fc30019adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:04.924 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.900+0000 7fc2fea00700 1 -- 172.21.15.117:0/2542090438 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc30019fe00 con 0x7fc30006ab30 2024-06-03T03:27:04.924 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.900+0000 7fc2ff400700 1 --2- 172.21.15.117:0/2542090438 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc30006a170 0x7fc30019adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:27:04.924 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.900+0000 7fc2fea00700 1 --2- 172.21.15.117:0/2542090438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc30006ab30 0x7fc30019b330 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec00eb30 tx=0x7fc2ec00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:27:04.924 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.900+0000 7fc2f7e00700 1 -- 172.21.15.117:0/2542090438 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec00cbd0 con 0x7fc30006ab30 2024-06-03T03:27:04.924 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.900+0000 7fc306200700 1 -- 172.21.15.117:0/2542090438 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3001a00f0 con 0x7fc30006ab30 2024-06-03T03:27:04.924 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.900+0000 7fc2f7e00700 1 -- 172.21.15.117:0/2542090438 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2ec004500 con 0x7fc30006ab30 2024-06-03T03:27:04.924 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.900+0000 7fc2f7e00700 1 -- 172.21.15.117:0/2542090438 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec010430 con 0x7fc30006ab30 2024-06-03T03:27:04.924 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.900+0000 7fc306200700 1 -- 172.21.15.117:0/2542090438 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3001a7970 con 0x7fc30006ab30 2024-06-03T03:27:04.924 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.904+0000 7fc2f7e00700 1 -- 172.21.15.117:0/2542090438 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fc2ec003f00 con 0x7fc30006ab30 2024-06-03T03:27:04.925 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.905+0000 7fc310b0eb80 1 Processor -- start 2024-06-03T03:27:04.925 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.905+0000 7fc310b0eb80 1 -- start start 2024-06-03T03:27:04.925 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.905+0000 7fc310b0eb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5609c6d6be40 0x5609c6def070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:04.925 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.906+0000 7fc310b0eb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5609c6def5b0 0x5609c6defa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:04.925 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.906+0000 7fc310b0eb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5609c6dfa300 0x5609c6dfc6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:04.925 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.906+0000 7fc310b0eb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5609c6cfea10 con 0x5609c6def5b0 2024-06-03T03:27:04.925 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.906+0000 7fc310b0eb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5609c6d41b00 con 0x5609c6d6be40 2024-06-03T03:27:04.925 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.906+0000 7fc310b0eb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5609c6cfc8f0 con 0x5609c6dfa300 2024-06-03T03:27:04.925 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.906+0000 7fc2f7e00700 1 --2- 172.21.15.117:0/2542090438 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fc2e40a2ca0 0x7fc2e40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:04.925 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.906+0000 7fc2ff400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5609c6def5b0 0x5609c6defa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:04.925 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.906+0000 7fc2f7e00700 1 -- 172.21.15.117:0/2542090438 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7fc2ec0cf710 con 0x7fc30006ab30 2024-06-03T03:27:04.925 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.906+0000 7fc2ff400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5609c6def5b0 0x5609c6defa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:55080/0 (socket says 172.21.15.117:55080) 2024-06-03T03:27:04.925 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.906+0000 7fc2ff400700 1 -- 172.21.15.117:0/92012005 learned_addr learned my addr 172.21.15.117:0/92012005 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:27:04.926 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.906+0000 7fc2ffe00700 1 --2- 172.21.15.117:0/92012005 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5609c6d6be40 0x5609c6def070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:04.926 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.906+0000 7fc2ffe00700 1 --2- 172.21.15.117:0/2542090438 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fc2e40a2ca0 0x7fc2e40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:04.926 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.906+0000 7fc2ff400700 1 -- 172.21.15.117:0/92012005 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5609c6dfa300 msgr2=0x5609c6dfc6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-03T03:27:04.926 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.906+0000 7fc2ff400700 1 --2- 172.21.15.117:0/92012005 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5609c6dfa300 0x5609c6dfc6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:04.926 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.906+0000 7fc2ff400700 1 -- 172.21.15.117:0/92012005 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5609c6d6be40 msgr2=0x5609c6def070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:04.926 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.906+0000 7fc2ff400700 1 --2- 172.21.15.117:0/92012005 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5609c6d6be40 0x5609c6def070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:04.926 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.907+0000 7fc2ff400700 1 -- 172.21.15.117:0/92012005 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609c6ce9f90 con 0x5609c6def5b0 2024-06-03T03:27:04.926 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.907+0000 7fc2ff400700 1 --2- 172.21.15.117:0/92012005 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5609c6def5b0 0x5609c6defa30 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc2f0009990 tx=0x7fc2f0005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=367146d733ff7bf0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:27:04.926 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.907+0000 7fc2ffe00700 1 --2- 172.21.15.117:0/2542090438 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fc2e40a2ca0 0x7fc2e40a5160 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8009ec0 tx=0x7fc2f800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:27:04.926 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.907+0000 7fc2f4c00700 1 -- 172.21.15.117:0/92012005 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f000b880 con 0x5609c6def5b0 2024-06-03T03:27:04.926 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.907+0000 7fc2f4c00700 1 -- 172.21.15.117:0/92012005 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2f0027070 con 0x5609c6def5b0 2024-06-03T03:27:04.926 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.907+0000 7fc2f4c00700 1 -- 172.21.15.117:0/92012005 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f000f6c0 con 0x5609c6def5b0 2024-06-03T03:27:04.927 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.908+0000 7fc310b0eb80 1 -- 172.21.15.117:0/92012005 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5609c6def5b0 msgr2=0x5609c6defa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:04.927 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.908+0000 7fc310b0eb80 1 --2- 172.21.15.117:0/92012005 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5609c6def5b0 0x5609c6defa30 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc2f0009990 tx=0x7fc2f0005bc0 comp rx=0 tx=0).stop 2024-06-03T03:27:04.927 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.908+0000 7fc310b0eb80 1 -- 172.21.15.117:0/92012005 shutdown_connections 2024-06-03T03:27:04.927 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.908+0000 7fc310b0eb80 1 --2- 172.21.15.117:0/92012005 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5609c6dfa300 0x5609c6dfc6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:04.927 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.908+0000 7fc310b0eb80 1 --2- 172.21.15.117:0/92012005 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5609c6d6be40 0x5609c6def070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:04.927 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.908+0000 7fc310b0eb80 1 --2- 172.21.15.117:0/92012005 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5609c6def5b0 0x5609c6defa30 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:04.927 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.908+0000 7fc310b0eb80 1 -- 172.21.15.117:0/92012005 >> 172.21.15.117:0/92012005 conn(0x5609c6d1c170 msgr2=0x5609c6deb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:27:04.927 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.908+0000 7fc310b0eb80 1 -- 172.21.15.117:0/92012005 shutdown_connections 2024-06-03T03:27:04.927 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.908+0000 7fc310b0eb80 1 -- 172.21.15.117:0/92012005 wait complete. 2024-06-03T03:27:04.927 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.909+0000 7fc310b0eb80 1 -- start start 2024-06-03T03:27:04.927 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.909+0000 7fc310b0eb80 1 Processor -- start 2024-06-03T03:27:04.927 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.910+0000 7fc310b0eb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5609c6d6be40 0x5609c6df9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:04.927 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.910+0000 7fc310b0eb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5609c6def5b0 0x5609c6df6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:04.927 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.910+0000 7fc310b0eb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5609c6dfa300 0x5609c6df7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:04.928 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.910+0000 7fc310b0eb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5609c6cddf60 con 0x5609c6d6be40 2024-06-03T03:27:04.928 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.910+0000 7fc310b0eb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5609c6d3dac0 con 0x5609c6dfa300 2024-06-03T03:27:04.928 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.910+0000 7fc310b0eb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5609c6dfcf60 con 0x5609c6def5b0 2024-06-03T03:27:04.928 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.910+0000 7fc310b0eb80 10 client.0 _ll_register_callbacks cb 0x5609c6cdb720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:27:04.928 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.910+0000 7fc2ff400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5609c6d6be40 0x5609c6df9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:04.928 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.910+0000 7fc2ff400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5609c6d6be40 0x5609c6df9ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:55094/0 (socket says 172.21.15.117:55094) 2024-06-03T03:27:04.928 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.910+0000 7fc2ff400700 1 -- 172.21.15.117:0/322560808 learned_addr learned my addr 172.21.15.117:0/322560808 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:27:04.928 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.910+0000 7fc2ffe00700 1 --2- 172.21.15.117:0/322560808 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5609c6dfa300 0x5609c6df7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:04.928 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.910+0000 7fc2fea00700 1 --2- 172.21.15.117:0/322560808 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5609c6def5b0 0x5609c6df6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:04.928 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.910+0000 7fc2ff400700 1 -- 172.21.15.117:0/322560808 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5609c6def5b0 msgr2=0x5609c6df6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:04.928 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.910+0000 7fc2ff400700 1 --2- 172.21.15.117:0/322560808 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5609c6def5b0 0x5609c6df6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:04.928 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.910+0000 7fc2ff400700 1 -- 172.21.15.117:0/322560808 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5609c6dfa300 msgr2=0x5609c6df7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:04.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.910+0000 7fc2ff400700 1 --2- 172.21.15.117:0/322560808 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5609c6dfa300 0x5609c6df7310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:04.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.910+0000 7fc2ff400700 1 -- 172.21.15.117:0/322560808 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609c6c908d0 con 0x5609c6d6be40 2024-06-03T03:27:04.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.910+0000 7fc2ffe00700 1 --2- 172.21.15.117:0/322560808 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5609c6dfa300 0x5609c6df7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:27:04.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.911+0000 7fc2ff400700 1 --2- 172.21.15.117:0/322560808 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5609c6d6be40 0x5609c6df9ad0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc2f0006010 tx=0x7fc2f000beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61d267fdf20a745c server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:27:04.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.911+0000 7fc2e9600700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-06-03T03:27:04.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.911+0000 7fc2e9600700 1 -- 172.21.15.117:0/322560808 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f0003680 con 0x5609c6d6be40 2024-06-03T03:27:04.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.911+0000 7fc2e9600700 1 -- 172.21.15.117:0/322560808 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7fc2f0035470 con 0x5609c6d6be40 2024-06-03T03:27:04.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.911+0000 7fc310b0eb80 1 -- 172.21.15.117:0/322560808 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2f0027070 con 0x5609c6d6be40 2024-06-03T03:27:04.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.911+0000 7fc2e9600700 1 -- 172.21.15.117:0/322560808 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc2f0035690 con 0x5609c6d6be40 2024-06-03T03:27:04.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.911+0000 7fc310b0eb80 10 client.15498 fetch_fsmap learned FSMap version 31 2024-06-03T03:27:04.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.911+0000 7fc310b0eb80 1 -- 172.21.15.117:0/322560808 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5609c6e07710 con 0x5609c6d6be40 2024-06-03T03:27:04.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.912+0000 7fc2e9600700 1 -- 172.21.15.117:0/322560808 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2f0027070 con 0x5609c6d6be40 2024-06-03T03:27:04.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.912+0000 7fc2e9600700 1 -- 172.21.15.117:0/322560808 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fc2f000f730 con 0x5609c6d6be40 2024-06-03T03:27:04.930 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.912+0000 7fc310b0eb80 10 client.15498 fetch_fsmap finished waiting for FSMap version 31 2024-06-03T03:27:04.930 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.912+0000 7fc310b0eb80 10 client.15498 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.a 2024-06-03T03:27:04.930 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.912+0000 7fc310b0eb80 20 client.15498 populate_metadata read hostname 'smithi117' 2024-06-03T03:27:04.930 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.912+0000 7fc310b0eb80 1 --2- 172.21.15.117:0/322560808 >> [v2:172.21.15.117:6840/1212242967,v1:172.21.15.117:6841/1212242967] conn(0x5609c6e09be0 0x5609c6e0bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:04.930 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.912+0000 7fc310b0eb80 4 client.15498 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T03:27:04.930 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.912+0000 7fc310b0eb80 1 -- 172.21.15.117:0/322560808 --> [v2:172.21.15.117:6840/1212242967,v1:172.21.15.117:6841/1212242967] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5609c6d4f000 con 0x5609c6e09be0 2024-06-03T03:27:04.930 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.912+0000 7fc2fea00700 1 --2- 172.21.15.117:0/322560808 >> [v2:172.21.15.117:6840/1212242967,v1:172.21.15.117:6841/1212242967] conn(0x5609c6e09be0 0x5609c6e0bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:04.930 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.913+0000 7fc2fea00700 1 --2- 172.21.15.117:0/322560808 >> [v2:172.21.15.117:6840/1212242967,v1:172.21.15.117:6841/1212242967] conn(0x5609c6e09be0 0x5609c6e0bfd0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=185ce017918527ef server_cookie=c60000723c3dd28a in_seq=0 out_seq=0 2024-06-03T03:27:04.930 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.913+0000 7fc2e9600700 10 client.15498 ms_handle_connect on v2:172.21.15.117:6840/1212242967 2024-06-03T03:27:04.930 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.914+0000 7fc2e9600700 1 -- 172.21.15.117:0/322560808 <== mds.2 v2:172.21.15.117:6840/1212242967 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5609c6d4f000 con 0x5609c6e09be0 2024-06-03T03:27:04.930 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.914+0000 7fc2e9600700 10 client.15498 handle_command_reply: tid=0 2024-06-03T03:27:04.930 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.914+0000 7fc310b0eb80 1 client.15498 shutdown 2024-06-03T03:27:04.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.914+0000 7fc310b0eb80 1 -- 172.21.15.117:0/322560808 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5609c6d6be40 msgr2=0x5609c6df9ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:04.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.914+0000 7fc310b0eb80 1 --2- 172.21.15.117:0/322560808 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5609c6d6be40 0x5609c6df9ad0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc2f0006010 tx=0x7fc2f000beb0 comp rx=0 tx=0).stop 2024-06-03T03:27:04.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.914+0000 7fc310b0eb80 1 -- 172.21.15.117:0/322560808 shutdown_connections 2024-06-03T03:27:04.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.914+0000 7fc310b0eb80 1 --2- 172.21.15.117:0/322560808 >> [v2:172.21.15.117:6840/1212242967,v1:172.21.15.117:6841/1212242967] conn(0x5609c6e09be0 0x5609c6e0bfd0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:04.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.914+0000 7fc310b0eb80 1 --2- 172.21.15.117:0/322560808 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5609c6dfa300 0x5609c6df7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:04.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.914+0000 7fc310b0eb80 1 --2- 172.21.15.117:0/322560808 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5609c6d6be40 0x5609c6df9ad0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:04.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.914+0000 7fc310b0eb80 1 --2- 172.21.15.117:0/322560808 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5609c6def5b0 0x5609c6df6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:04.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.914+0000 7fc2e9600700 0 client.15498 ms_handle_reset on v2:172.21.15.117:6840/1212242967 2024-06-03T03:27:04.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.914+0000 7fc310b0eb80 1 -- 172.21.15.117:0/322560808 >> 172.21.15.117:0/322560808 conn(0x5609c6d1c170 msgr2=0x5609c6dea740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:27:04.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.915+0000 7fc310b0eb80 1 -- 172.21.15.117:0/322560808 shutdown_connections 2024-06-03T03:27:04.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.915+0000 7fc310b0eb80 1 -- 172.21.15.117:0/322560808 wait complete. 2024-06-03T03:27:04.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.915+0000 7fc310b0eb80 20 client.15498 trim_cache size 0 max 16384 2024-06-03T03:27:04.932 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.932+0000 7fc310b0eb80 1 Processor -- start 2024-06-03T03:27:04.932 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.932+0000 7fc310b0eb80 1 -- start start 2024-06-03T03:27:04.932 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.932+0000 7fc310b0eb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5609c6d6be40 0x5609c6df5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:04.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.932+0000 7fc310b0eb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5609c6def5b0 0x5609c6dfb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:04.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.932+0000 7fc310b0eb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5609c6dfa300 0x5609c6de9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:04.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.932+0000 7fc310b0eb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc3001b04d0 con 0x5609c6def5b0 2024-06-03T03:27:04.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.932+0000 7fc310b0eb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc3001b2010 con 0x5609c6d6be40 2024-06-03T03:27:04.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.932+0000 7fc310b0eb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc3001b0ba0 con 0x5609c6dfa300 2024-06-03T03:27:04.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.932+0000 7fc2fea00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5609c6def5b0 0x5609c6dfb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:04.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.933+0000 7fc2fea00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5609c6def5b0 0x5609c6dfb6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:55098/0 (socket says 172.21.15.117:55098) 2024-06-03T03:27:04.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.933+0000 7fc2fea00700 1 -- 172.21.15.117:0/3078334202 learned_addr learned my addr 172.21.15.117:0/3078334202 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:27:04.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.933+0000 7fc2ff400700 1 --2- 172.21.15.117:0/3078334202 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5609c6d6be40 0x5609c6df5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:04.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.933+0000 7fc2ffe00700 1 --2- 172.21.15.117:0/3078334202 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5609c6dfa300 0x5609c6de9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:04.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.933+0000 7fc2ff400700 1 -- 172.21.15.117:0/3078334202 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5609c6dfa300 msgr2=0x5609c6de9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:04.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.933+0000 7fc2ff400700 1 --2- 172.21.15.117:0/3078334202 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5609c6dfa300 0x5609c6de9eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:04.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.933+0000 7fc2ff400700 1 -- 172.21.15.117:0/3078334202 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5609c6def5b0 msgr2=0x5609c6dfb6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:04.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.933+0000 7fc2ff400700 1 --2- 172.21.15.117:0/3078334202 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5609c6def5b0 0x5609c6dfb6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:04.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.933+0000 7fc2ff400700 1 -- 172.21.15.117:0/3078334202 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609c6df8370 con 0x5609c6d6be40 2024-06-03T03:27:04.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.933+0000 7fc2ff400700 1 --2- 172.21.15.117:0/3078334202 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5609c6d6be40 0x5609c6df5dd0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc2f00098e0 tx=0x7fc2f0005800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ed397efefea7f1e server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:27:04.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.934+0000 7fc2ebe00700 1 -- 172.21.15.117:0/3078334202 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f0003680 con 0x5609c6d6be40 2024-06-03T03:27:04.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.934+0000 7fc2ebe00700 1 -- 172.21.15.117:0/3078334202 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2f0027070 con 0x5609c6d6be40 2024-06-03T03:27:04.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.934+0000 7fc2ebe00700 1 -- 172.21.15.117:0/3078334202 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f0035b20 con 0x5609c6d6be40 2024-06-03T03:27:04.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.934+0000 7fc310b0eb80 1 -- 172.21.15.117:0/3078334202 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5609c6d6be40 msgr2=0x5609c6df5dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:04.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.934+0000 7fc310b0eb80 1 --2- 172.21.15.117:0/3078334202 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5609c6d6be40 0x5609c6df5dd0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc2f00098e0 tx=0x7fc2f0005800 comp rx=0 tx=0).stop 2024-06-03T03:27:04.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.935+0000 7fc310b0eb80 1 -- 172.21.15.117:0/3078334202 shutdown_connections 2024-06-03T03:27:04.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.935+0000 7fc310b0eb80 1 --2- 172.21.15.117:0/3078334202 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5609c6dfa300 0x5609c6de9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:04.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.935+0000 7fc310b0eb80 1 --2- 172.21.15.117:0/3078334202 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5609c6d6be40 0x5609c6df5dd0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:04.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.935+0000 7fc310b0eb80 1 --2- 172.21.15.117:0/3078334202 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5609c6def5b0 0x5609c6dfb6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:04.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.935+0000 7fc310b0eb80 1 -- 172.21.15.117:0/3078334202 >> 172.21.15.117:0/3078334202 conn(0x5609c6d1c170 msgr2=0x5609c6df56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:27:04.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.935+0000 7fc310b0eb80 1 -- 172.21.15.117:0/3078334202 shutdown_connections 2024-06-03T03:27:04.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.935+0000 7fc310b0eb80 1 -- 172.21.15.117:0/3078334202 wait complete. 2024-06-03T03:27:04.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.935+0000 7fc310b0eb80 1 -- start start 2024-06-03T03:27:04.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.935+0000 7fc310b0eb80 1 Processor -- start 2024-06-03T03:27:04.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.936+0000 7fc310b0eb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5609c6d6be40 0x5609c6e0a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:04.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.936+0000 7fc310b0eb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5609c6def5b0 0x5609c6e0a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:04.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.936+0000 7fc310b0eb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5609c6dfa300 0x5609c6e0aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:04.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.936+0000 7fc310b0eb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc3001b1940 con 0x5609c6def5b0 2024-06-03T03:27:04.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.936+0000 7fc310b0eb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc3001b1270 con 0x5609c6d6be40 2024-06-03T03:27:04.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.936+0000 7fc310b0eb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc3001ae470 con 0x5609c6dfa300 2024-06-03T03:27:04.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.936+0000 7fc310b0eb80 10 client.0 _ll_register_callbacks cb 0x5609c6cdb720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:27:04.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.936+0000 7fc2fea00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5609c6def5b0 0x5609c6e0a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:04.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.936+0000 7fc2ff400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5609c6d6be40 0x5609c6e0a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:04.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.936+0000 7fc2fea00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5609c6def5b0 0x5609c6e0a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:55106/0 (socket says 172.21.15.117:55106) 2024-06-03T03:27:04.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.936+0000 7fc2fea00700 1 -- 172.21.15.117:0/3802108787 learned_addr learned my addr 172.21.15.117:0/3802108787 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:27:04.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.936+0000 7fc2ffe00700 1 --2- 172.21.15.117:0/3802108787 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5609c6dfa300 0x5609c6e0aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:04.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.937+0000 7fc2ff400700 1 -- 172.21.15.117:0/3802108787 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5609c6dfa300 msgr2=0x5609c6e0aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:04.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.937+0000 7fc2ff400700 1 --2- 172.21.15.117:0/3802108787 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5609c6dfa300 0x5609c6e0aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:04.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.937+0000 7fc2ff400700 1 -- 172.21.15.117:0/3802108787 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5609c6def5b0 msgr2=0x5609c6e0a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:04.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.937+0000 7fc2ff400700 1 --2- 172.21.15.117:0/3802108787 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5609c6def5b0 0x5609c6e0a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:04.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.937+0000 7fc2ff400700 1 -- 172.21.15.117:0/3802108787 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609c6e72730 con 0x5609c6d6be40 2024-06-03T03:27:04.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.937+0000 7fc2fea00700 1 --2- 172.21.15.117:0/3802108787 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5609c6def5b0 0x5609c6e0a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:27:04.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.937+0000 7fc2ff400700 1 --2- 172.21.15.117:0/3802108787 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5609c6d6be40 0x5609c6e0a000 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc2f000fb30 tx=0x7fc2f0009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be1994cc8a8cba72 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:27:04.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.937+0000 7fc2e9600700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-06-03T03:27:04.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.937+0000 7fc2e9600700 1 -- 172.21.15.117:0/3802108787 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f000f870 con 0x5609c6d6be40 2024-06-03T03:27:04.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.937+0000 7fc310b0eb80 1 -- 172.21.15.117:0/3802108787 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2f0027070 con 0x5609c6d6be40 2024-06-03T03:27:04.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.937+0000 7fc2e9600700 1 -- 172.21.15.117:0/3802108787 <== mon.2 v2:172.21.15.151:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7fc2f0035830 con 0x5609c6d6be40 2024-06-03T03:27:04.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.938+0000 7fc2e9600700 1 -- 172.21.15.117:0/3802108787 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc2f0003e50 con 0x5609c6d6be40 2024-06-03T03:27:04.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.938+0000 7fc310b0eb80 10 client.25598 fetch_fsmap learned FSMap version 31 2024-06-03T03:27:04.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.938+0000 7fc310b0eb80 1 -- 172.21.15.117:0/3802108787 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5609c6e73a30 con 0x5609c6d6be40 2024-06-03T03:27:04.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.938+0000 7fc2e9600700 1 -- 172.21.15.117:0/3802108787 <== mon.2 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2f0027070 con 0x5609c6d6be40 2024-06-03T03:27:04.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.938+0000 7fc2e9600700 1 -- 172.21.15.117:0/3802108787 <== mon.2 v2:172.21.15.151:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fc2f0022710 con 0x5609c6d6be40 2024-06-03T03:27:04.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.938+0000 7fc310b0eb80 10 client.25598 fetch_fsmap finished waiting for FSMap version 31 2024-06-03T03:27:04.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.938+0000 7fc310b0eb80 10 client.25598 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.a 2024-06-03T03:27:04.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.938+0000 7fc310b0eb80 20 client.25598 populate_metadata read hostname 'smithi117' 2024-06-03T03:27:04.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.938+0000 7fc310b0eb80 1 --2- 172.21.15.117:0/3802108787 >> [v2:172.21.15.117:6840/1212242967,v1:172.21.15.117:6841/1212242967] conn(0x5609c6e73e40 0x5609c6e76230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:04.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.938+0000 7fc310b0eb80 4 client.25598 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T03:27:04.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.938+0000 7fc310b0eb80 1 -- 172.21.15.117:0/3802108787 --> [v2:172.21.15.117:6840/1212242967,v1:172.21.15.117:6841/1212242967] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5609c6d49580 con 0x5609c6e73e40 2024-06-03T03:27:04.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.939+0000 7fc2fea00700 1 --2- 172.21.15.117:0/3802108787 >> [v2:172.21.15.117:6840/1212242967,v1:172.21.15.117:6841/1212242967] conn(0x5609c6e73e40 0x5609c6e76230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:04.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.939+0000 7fc2fea00700 1 --2- 172.21.15.117:0/3802108787 >> [v2:172.21.15.117:6840/1212242967,v1:172.21.15.117:6841/1212242967] conn(0x5609c6e73e40 0x5609c6e76230 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4c7be28180bb61e server_cookie=60fd7896bdfb8abc in_seq=0 out_seq=0 2024-06-03T03:27:04.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.939+0000 7fc2e9600700 10 client.25598 ms_handle_connect on v2:172.21.15.117:6840/1212242967 2024-06-03T03:27:04.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.939+0000 7fc2e9600700 1 -- 172.21.15.117:0/3802108787 <== mds.2 v2:172.21.15.117:6840/1212242967 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5609c6d49580 con 0x5609c6e73e40 2024-06-03T03:27:04.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.939+0000 7fc2e9600700 10 client.25598 handle_command_reply: tid=0 2024-06-03T03:27:04.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.939+0000 7fc310b0eb80 1 client.25598 shutdown 2024-06-03T03:27:04.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.939+0000 7fc310b0eb80 1 -- 172.21.15.117:0/3802108787 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5609c6d6be40 msgr2=0x5609c6e0a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:04.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.939+0000 7fc310b0eb80 1 --2- 172.21.15.117:0/3802108787 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5609c6d6be40 0x5609c6e0a000 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc2f000fb30 tx=0x7fc2f0009ca0 comp rx=0 tx=0).stop 2024-06-03T03:27:04.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.940+0000 7fc310b0eb80 1 -- 172.21.15.117:0/3802108787 shutdown_connections 2024-06-03T03:27:04.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.940+0000 7fc310b0eb80 1 --2- 172.21.15.117:0/3802108787 >> [v2:172.21.15.117:6840/1212242967,v1:172.21.15.117:6841/1212242967] conn(0x5609c6e73e40 0x5609c6e76230 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:04.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.940+0000 7fc310b0eb80 1 --2- 172.21.15.117:0/3802108787 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5609c6dfa300 0x5609c6e0aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:04.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.940+0000 7fc310b0eb80 1 --2- 172.21.15.117:0/3802108787 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5609c6d6be40 0x5609c6e0a000 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:04.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.940+0000 7fc310b0eb80 1 --2- 172.21.15.117:0/3802108787 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5609c6def5b0 0x5609c6e0a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:04.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.940+0000 7fc2e9600700 0 client.25598 ms_handle_reset on v2:172.21.15.117:6840/1212242967 2024-06-03T03:27:04.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.940+0000 7fc310b0eb80 1 -- 172.21.15.117:0/3802108787 >> 172.21.15.117:0/3802108787 conn(0x5609c6d1c170 msgr2=0x5609c6df8e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:27:04.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.940+0000 7fc310b0eb80 1 -- 172.21.15.117:0/3802108787 shutdown_connections 2024-06-03T03:27:04.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.940+0000 7fc310b0eb80 1 -- 172.21.15.117:0/3802108787 wait complete. 2024-06-03T03:27:04.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.940+0000 7fc310b0eb80 20 client.25598 trim_cache size 0 max 16384 2024-06-03T03:27:04.944 INFO:teuthology.orchestra.run.smithi117.stdout:[] 2024-06-03T03:27:04.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.941+0000 7fc306200700 1 -- 172.21.15.117:0/2542090438 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fc2e40a2ca0 msgr2=0x7fc2e40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:04.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.941+0000 7fc306200700 1 --2- 172.21.15.117:0/2542090438 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fc2e40a2ca0 0x7fc2e40a5160 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8009ec0 tx=0x7fc2f800b040 comp rx=0 tx=0).stop 2024-06-03T03:27:04.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.941+0000 7fc306200700 1 -- 172.21.15.117:0/2542090438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc30006ab30 msgr2=0x7fc30019b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:04.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.941+0000 7fc306200700 1 --2- 172.21.15.117:0/2542090438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc30006ab30 0x7fc30019b330 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec00eb30 tx=0x7fc2ec00ee40 comp rx=0 tx=0).stop 2024-06-03T03:27:04.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.941+0000 7fc306200700 1 -- 172.21.15.117:0/2542090438 shutdown_connections 2024-06-03T03:27:04.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.941+0000 7fc306200700 1 --2- 172.21.15.117:0/2542090438 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fc2e40a2ca0 0x7fc2e40a5160 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:04.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.941+0000 7fc306200700 1 --2- 172.21.15.117:0/2542090438 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc300108500 0x7fc30019f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:04.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.941+0000 7fc306200700 1 --2- 172.21.15.117:0/2542090438 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc30006a170 0x7fc30019adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:04.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.941+0000 7fc306200700 1 --2- 172.21.15.117:0/2542090438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc30006ab30 0x7fc30019b330 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:04.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.941+0000 7fc306200700 1 -- 172.21.15.117:0/2542090438 >> 172.21.15.117:0/2542090438 conn(0x7fc300078340 msgr2=0x7fc30010d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:27:04.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.941+0000 7fc306200700 1 -- 172.21.15.117:0/2542090438 shutdown_connections 2024-06-03T03:27:04.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:04.941+0000 7fc306200700 1 -- 172.21.15.117:0/2542090438 wait complete. 2024-06-03T03:27:05.928 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-03T03:27:06.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:27:06 smithi117 ceph-mon[31649]: pgmap v559: 129 pgs: 129 active+clean; 1.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 605 KiB/s wr, 14 op/s 2024-06-03T03:27:06.364 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:27:06.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:27:06 smithi151 ceph-mon[37043]: pgmap v559: 129 pgs: 129 active+clean; 1.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 605 KiB/s wr, 14 op/s 2024-06-03T03:27:06.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:27:06 smithi183 ceph-mon[37326]: pgmap v559: 129 pgs: 129 active+clean; 1.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 605 KiB/s wr, 14 op/s 2024-06-03T03:27:07.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:27:07 smithi117 ceph-mon[31649]: pgmap v560: 129 pgs: 129 active+clean; 1.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 605 KiB/s wr, 14 op/s 2024-06-03T03:27:07.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:27:07 smithi151 ceph-mon[37043]: pgmap v560: 129 pgs: 129 active+clean; 1.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 605 KiB/s wr, 14 op/s 2024-06-03T03:27:07.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:27:07 smithi183 ceph-mon[37326]: pgmap v560: 129 pgs: 129 active+clean; 1.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 605 KiB/s wr, 14 op/s 2024-06-03T03:27:08.666 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.665+0000 7f3bb3e00700 1 -- 172.21.15.117:0/2703306962 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bac012650 con 0x7f3bb410ef30 2024-06-03T03:27:08.667 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.666+0000 7f3bbbe00700 1 -- 172.21.15.117:0/2703306962 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3bb410ef30 msgr2=0x7f3bb4111320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:08.667 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.666+0000 7f3bbbe00700 1 --2- 172.21.15.117:0/2703306962 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3bb410ef30 0x7f3bb4111320 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f3bac00b0a0 tx=0x7f3bac00b3b0 comp rx=0 tx=0).stop 2024-06-03T03:27:08.667 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.666+0000 7f3bbbe00700 1 -- 172.21.15.117:0/2703306962 shutdown_connections 2024-06-03T03:27:08.667 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.666+0000 7f3bbbe00700 1 --2- 172.21.15.117:0/2703306962 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3bb410ef30 0x7f3bb4111320 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:08.667 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.666+0000 7f3bbbe00700 1 --2- 172.21.15.117:0/2703306962 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3bb406aa00 0x7f3bb406ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:08.668 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.666+0000 7f3bbbe00700 1 --2- 172.21.15.117:0/2703306962 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3bb406b320 0x7f3bb410e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:08.668 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.666+0000 7f3bbbe00700 1 -- 172.21.15.117:0/2703306962 >> 172.21.15.117:0/2703306962 conn(0x7f3bb4078330 msgr2=0x7f3bb4078740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:27:08.668 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.666+0000 7f3bbbe00700 1 -- 172.21.15.117:0/2703306962 shutdown_connections 2024-06-03T03:27:08.668 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.666+0000 7f3bbbe00700 1 -- 172.21.15.117:0/2703306962 wait complete. 2024-06-03T03:27:08.669 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.667+0000 7f3bbbe00700 1 Processor -- start 2024-06-03T03:27:08.670 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.667+0000 7f3bbbe00700 1 -- start start 2024-06-03T03:27:08.670 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.668+0000 7f3bbbe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3bb406aa00 0x7f3bb410e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:08.670 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.668+0000 7f3bbbe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3bb406b320 0x7f3bb410c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:08.670 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.668+0000 7f3bbbe00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3bb410ef30 0x7f3bb410cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:08.670 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.668+0000 7f3bbbe00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb4103cf0 con 0x7f3bb406aa00 2024-06-03T03:27:08.670 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.668+0000 7f3bbbe00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb4103b70 con 0x7f3bb406b320 2024-06-03T03:27:08.670 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.668+0000 7f3bbbe00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb4103e70 con 0x7f3bb410ef30 2024-06-03T03:27:08.670 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.668+0000 7f3bb9400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3bb406aa00 0x7f3bb410e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:08.670 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.668+0000 7f3bb9400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3bb406aa00 0x7f3bb410e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40692/0 (socket says 172.21.15.117:40692) 2024-06-03T03:27:08.670 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.668+0000 7f3bb8a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3bb406b320 0x7f3bb410c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:08.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.668+0000 7f3bb9400700 1 -- 172.21.15.117:0/3704085439 learned_addr learned my addr 172.21.15.117:0/3704085439 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:27:08.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.668+0000 7f3bb9e00700 1 --2- 172.21.15.117:0/3704085439 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3bb410ef30 0x7f3bb410cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:08.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.668+0000 7f3bb8a00700 1 -- 172.21.15.117:0/3704085439 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3bb410ef30 msgr2=0x7f3bb410cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:08.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.668+0000 7f3bb8a00700 1 --2- 172.21.15.117:0/3704085439 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3bb410ef30 0x7f3bb410cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:08.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.668+0000 7f3bb8a00700 1 -- 172.21.15.117:0/3704085439 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3bb406aa00 msgr2=0x7f3bb410e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:08.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.668+0000 7f3bb8a00700 1 --2- 172.21.15.117:0/3704085439 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3bb406aa00 0x7f3bb410e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:08.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.668+0000 7f3bb8a00700 1 -- 172.21.15.117:0/3704085439 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bb410d520 con 0x7f3bb406b320 2024-06-03T03:27:08.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.668+0000 7f3bb9400700 1 --2- 172.21.15.117:0/3704085439 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3bb406aa00 0x7f3bb410e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:27:08.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.669+0000 7f3bb8a00700 1 --2- 172.21.15.117:0/3704085439 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3bb406b320 0x7f3bb410c7d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3ba000e9f0 tx=0x7f3ba000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:27:08.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.669+0000 7f3bb2000700 1 -- 172.21.15.117:0/3704085439 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba000c9c0 con 0x7f3bb406b320 2024-06-03T03:27:08.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.669+0000 7f3bbbe00700 1 -- 172.21.15.117:0/3704085439 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bb41ab7d0 con 0x7f3bb406b320 2024-06-03T03:27:08.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.669+0000 7f3bb2000700 1 -- 172.21.15.117:0/3704085439 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ba000cb60 con 0x7f3bb406b320 2024-06-03T03:27:08.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.669+0000 7f3bb2000700 1 -- 172.21.15.117:0/3704085439 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba0018680 con 0x7f3bb406b320 2024-06-03T03:27:08.672 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.669+0000 7f3bbbe00700 1 -- 172.21.15.117:0/3704085439 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bb41abd30 con 0x7f3bb406b320 2024-06-03T03:27:08.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.672+0000 7f3bb2000700 1 -- 172.21.15.117:0/3704085439 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f3ba0010430 con 0x7f3bb406b320 2024-06-03T03:27:08.679 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.674+0000 7f3bc6724b80 1 Processor -- start 2024-06-03T03:27:08.679 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.675+0000 7f3bc6724b80 1 -- start start 2024-06-03T03:27:08.679 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.675+0000 7f3bb2000700 1 --2- 172.21.15.117:0/3704085439 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f3b9c0a2ca0 0x7f3b9c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:08.680 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.675+0000 7f3bb2000700 1 -- 172.21.15.117:0/3704085439 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f3ba00cf420 con 0x7f3bb406b320 2024-06-03T03:27:08.680 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.675+0000 7f3bc6724b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b2f812e40 0x560b2f896070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:08.680 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.675+0000 7f3bc6724b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b2f8965b0 0x560b2f896a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:08.680 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.675+0000 7f3bc6724b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560b2f8a1300 0x560b2f8a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:08.680 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.675+0000 7f3bc6724b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x560b2f7a5a10 con 0x560b2f8965b0 2024-06-03T03:27:08.680 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.675+0000 7f3bc6724b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x560b2f7e8b00 con 0x560b2f812e40 2024-06-03T03:27:08.680 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.675+0000 7f3bc6724b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560b2f7a38f0 con 0x560b2f8a1300 2024-06-03T03:27:08.680 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.675+0000 7f3bb9e00700 1 --2- 172.21.15.117:0/3704085439 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f3b9c0a2ca0 0x7f3b9c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:08.680 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.675+0000 7f3bb8a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b2f8965b0 0x560b2f896a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:08.680 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.676+0000 7f3bb8a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b2f8965b0 0x560b2f896a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40698/0 (socket says 172.21.15.117:40698) 2024-06-03T03:27:08.680 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.676+0000 7f3bb8a00700 1 -- 172.21.15.117:0/2312054893 learned_addr learned my addr 172.21.15.117:0/2312054893 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:27:08.680 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.676+0000 7f3bb9400700 1 --2- 172.21.15.117:0/2312054893 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b2f812e40 0x560b2f896070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:08.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.676+0000 7f3bb9e00700 1 --2- 172.21.15.117:0/2312054893 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560b2f8a1300 0x560b2f8a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:08.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.676+0000 7f3bb8a00700 1 -- 172.21.15.117:0/2312054893 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560b2f8a1300 msgr2=0x560b2f8a36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:08.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.676+0000 7f3bb8a00700 1 --2- 172.21.15.117:0/2312054893 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560b2f8a1300 0x560b2f8a36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:08.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.676+0000 7f3bb8a00700 1 -- 172.21.15.117:0/2312054893 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b2f812e40 msgr2=0x560b2f896070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:08.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.676+0000 7f3bb8a00700 1 --2- 172.21.15.117:0/2312054893 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b2f812e40 0x560b2f896070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:08.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.676+0000 7f3bb8a00700 1 -- 172.21.15.117:0/2312054893 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b2f790f90 con 0x560b2f8965b0 2024-06-03T03:27:08.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.676+0000 7f3bb9e00700 1 --2- 172.21.15.117:0/3704085439 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f3b9c0a2ca0 0x7f3b9c0a5160 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3bac009e60 tx=0x7f3bac009360 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:27:08.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.676+0000 7f3bb8a00700 1 --2- 172.21.15.117:0/2312054893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b2f8965b0 0x560b2f896a30 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0097cf0 tx=0x7f3ba0097c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6e867676c1b36f0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:27:08.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.676+0000 7f3ba6a00700 1 -- 172.21.15.117:0/2312054893 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba009b890 con 0x560b2f8965b0 2024-06-03T03:27:08.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.677+0000 7f3ba6a00700 1 -- 172.21.15.117:0/2312054893 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ba009fc40 con 0x560b2f8965b0 2024-06-03T03:27:08.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.677+0000 7f3ba6a00700 1 -- 172.21.15.117:0/2312054893 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba00a4070 con 0x560b2f8965b0 2024-06-03T03:27:08.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.677+0000 7f3bc6724b80 1 -- 172.21.15.117:0/2312054893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b2f8965b0 msgr2=0x560b2f896a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:08.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.677+0000 7f3bc6724b80 1 --2- 172.21.15.117:0/2312054893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b2f8965b0 0x560b2f896a30 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0097cf0 tx=0x7f3ba0097c60 comp rx=0 tx=0).stop 2024-06-03T03:27:08.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.677+0000 7f3bc6724b80 1 -- 172.21.15.117:0/2312054893 shutdown_connections 2024-06-03T03:27:08.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.677+0000 7f3bc6724b80 1 --2- 172.21.15.117:0/2312054893 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560b2f8a1300 0x560b2f8a36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:08.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.677+0000 7f3bc6724b80 1 --2- 172.21.15.117:0/2312054893 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b2f812e40 0x560b2f896070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:08.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.677+0000 7f3bc6724b80 1 --2- 172.21.15.117:0/2312054893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b2f8965b0 0x560b2f896a30 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:08.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.677+0000 7f3bc6724b80 1 -- 172.21.15.117:0/2312054893 >> 172.21.15.117:0/2312054893 conn(0x560b2f7c3170 msgr2=0x560b2f8924b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:27:08.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.678+0000 7f3bc6724b80 1 -- 172.21.15.117:0/2312054893 shutdown_connections 2024-06-03T03:27:08.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.678+0000 7f3bc6724b80 1 -- 172.21.15.117:0/2312054893 wait complete. 2024-06-03T03:27:08.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.678+0000 7f3bc6724b80 1 -- start start 2024-06-03T03:27:08.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.678+0000 7f3bc6724b80 1 Processor -- start 2024-06-03T03:27:08.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.679+0000 7f3bc6724b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b2f812e40 0x560b2f8a0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:08.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.679+0000 7f3bc6724b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b2f8965b0 0x560b2f89dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:08.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.679+0000 7f3bc6724b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560b2f8a1300 0x560b2f89e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:08.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.679+0000 7f3bc6724b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x560b2f784f60 con 0x560b2f8965b0 2024-06-03T03:27:08.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.679+0000 7f3bc6724b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x560b2f7e4ac0 con 0x560b2f812e40 2024-06-03T03:27:08.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.679+0000 7f3bc6724b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560b2f8a3f60 con 0x560b2f8a1300 2024-06-03T03:27:08.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.679+0000 7f3bc6724b80 10 client.0 _ll_register_callbacks cb 0x560b2f782720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:27:08.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.679+0000 7f3bb8a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b2f8965b0 0x560b2f89dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:08.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.679+0000 7f3bb9400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b2f812e40 0x560b2f8a0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:08.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.679+0000 7f3bb8a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b2f8965b0 0x560b2f89dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40712/0 (socket says 172.21.15.117:40712) 2024-06-03T03:27:08.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.679+0000 7f3bb8a00700 1 -- 172.21.15.117:0/2198568515 learned_addr learned my addr 172.21.15.117:0/2198568515 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:27:08.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.680+0000 7f3bb8a00700 1 -- 172.21.15.117:0/2198568515 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560b2f8a1300 msgr2=0x560b2f89e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:08.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.680+0000 7f3bb9e00700 1 --2- 172.21.15.117:0/2198568515 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560b2f8a1300 0x560b2f89e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:08.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.680+0000 7f3bb8a00700 1 --2- 172.21.15.117:0/2198568515 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560b2f8a1300 0x560b2f89e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:08.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.680+0000 7f3bb8a00700 1 -- 172.21.15.117:0/2198568515 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b2f812e40 msgr2=0x560b2f8a0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:08.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.680+0000 7f3bb8a00700 1 --2- 172.21.15.117:0/2198568515 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b2f812e40 0x560b2f8a0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:08.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.680+0000 7f3bb8a00700 1 -- 172.21.15.117:0/2198568515 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b2f7378d0 con 0x560b2f8965b0 2024-06-03T03:27:08.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.680+0000 7f3bb9e00700 1 --2- 172.21.15.117:0/2198568515 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560b2f8a1300 0x560b2f89e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T03:27:08.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.680+0000 7f3bb8a00700 1 --2- 172.21.15.117:0/2198568515 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b2f8965b0 0x560b2f89dd90 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f3ba009bef0 tx=0x7f3ba0009ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=444ffef2ae971644 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:27:08.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.680+0000 7f3b9b400700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-06-03T03:27:08.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.680+0000 7f3b9b400700 1 -- 172.21.15.117:0/2198568515 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba00860b0 con 0x560b2f8965b0 2024-06-03T03:27:08.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.680+0000 7f3b9b400700 1 -- 172.21.15.117:0/2198568515 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f3ba0086250 con 0x560b2f8965b0 2024-06-03T03:27:08.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.680+0000 7f3bc6724b80 1 -- 172.21.15.117:0/2198568515 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ba009fc40 con 0x560b2f8965b0 2024-06-03T03:27:08.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.680+0000 7f3b9b400700 1 -- 172.21.15.117:0/2198568515 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f3ba00ba0d0 con 0x560b2f8965b0 2024-06-03T03:27:08.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.680+0000 7f3bc6724b80 10 client.15516 fetch_fsmap learned FSMap version 31 2024-06-03T03:27:08.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.681+0000 7f3bc6724b80 1 -- 172.21.15.117:0/2198568515 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x560b2f8ae6b0 con 0x560b2f8965b0 2024-06-03T03:27:08.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.681+0000 7f3b9b400700 1 -- 172.21.15.117:0/2198568515 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ba009fc40 con 0x560b2f8965b0 2024-06-03T03:27:08.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.681+0000 7f3b9b400700 1 -- 172.21.15.117:0/2198568515 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f3ba0010840 con 0x560b2f8965b0 2024-06-03T03:27:08.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.681+0000 7f3bc6724b80 10 client.15516 fetch_fsmap finished waiting for FSMap version 31 2024-06-03T03:27:08.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.681+0000 7f3bc6724b80 10 client.15516 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-06-03T03:27:08.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.681+0000 7f3bc6724b80 20 client.15516 populate_metadata read hostname 'smithi117' 2024-06-03T03:27:08.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.681+0000 7f3bc6724b80 1 --2- 172.21.15.117:0/2198568515 >> [v2:172.21.15.151:6838/3623444616,v1:172.21.15.151:6839/3623444616] conn(0x560b2f8b0b80 0x560b2f8b2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:08.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.681+0000 7f3bc6724b80 4 client.15516 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T03:27:08.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.681+0000 7f3bc6724b80 1 -- 172.21.15.117:0/2198568515 --> [v2:172.21.15.151:6838/3623444616,v1:172.21.15.151:6839/3623444616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b2f7f6000 con 0x560b2f8b0b80 2024-06-03T03:27:08.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.682+0000 7f3bb9400700 1 --2- 172.21.15.117:0/2198568515 >> [v2:172.21.15.151:6838/3623444616,v1:172.21.15.151:6839/3623444616] conn(0x560b2f8b0b80 0x560b2f8b2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:08.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.682+0000 7f3bb9400700 1 --2- 172.21.15.117:0/2198568515 >> [v2:172.21.15.151:6838/3623444616,v1:172.21.15.151:6839/3623444616] conn(0x560b2f8b0b80 0x560b2f8b2f70 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=609b3e08dd9ff34b server_cookie=68807ffaf6a422fe in_seq=0 out_seq=0 2024-06-03T03:27:08.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.682+0000 7f3b9b400700 10 client.15516 ms_handle_connect on v2:172.21.15.151:6838/3623444616 2024-06-03T03:27:08.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.683+0000 7f3b9b400700 1 -- 172.21.15.117:0/2198568515 <== mds.1 v2:172.21.15.151:6838/3623444616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560b2f7f6000 con 0x560b2f8b0b80 2024-06-03T03:27:08.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.683+0000 7f3b9b400700 10 client.15516 handle_command_reply: tid=0 2024-06-03T03:27:08.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.683+0000 7f3bc6724b80 1 client.15516 shutdown 2024-06-03T03:27:08.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.683+0000 7f3bc6724b80 1 -- 172.21.15.117:0/2198568515 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b2f8965b0 msgr2=0x560b2f89dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:08.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.683+0000 7f3bc6724b80 1 --2- 172.21.15.117:0/2198568515 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b2f8965b0 0x560b2f89dd90 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f3ba009bef0 tx=0x7f3ba0009ef0 comp rx=0 tx=0).stop 2024-06-03T03:27:08.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.683+0000 7f3bc6724b80 1 -- 172.21.15.117:0/2198568515 shutdown_connections 2024-06-03T03:27:08.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.683+0000 7f3bc6724b80 1 --2- 172.21.15.117:0/2198568515 >> [v2:172.21.15.151:6838/3623444616,v1:172.21.15.151:6839/3623444616] conn(0x560b2f8b0b80 0x560b2f8b2f70 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:08.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.683+0000 7f3bc6724b80 1 --2- 172.21.15.117:0/2198568515 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560b2f8a1300 0x560b2f89e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:08.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.683+0000 7f3bc6724b80 1 --2- 172.21.15.117:0/2198568515 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b2f812e40 0x560b2f8a0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:08.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.683+0000 7f3bc6724b80 1 --2- 172.21.15.117:0/2198568515 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b2f8965b0 0x560b2f89dd90 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:08.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.683+0000 7f3bc6724b80 1 -- 172.21.15.117:0/2198568515 >> 172.21.15.117:0/2198568515 conn(0x560b2f7c3170 msgr2=0x560b2f891740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:27:08.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.684+0000 7f3b9b400700 0 client.15516 ms_handle_reset on v2:172.21.15.151:6838/3623444616 2024-06-03T03:27:08.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.684+0000 7f3bc6724b80 1 -- 172.21.15.117:0/2198568515 shutdown_connections 2024-06-03T03:27:08.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.684+0000 7f3bc6724b80 1 -- 172.21.15.117:0/2198568515 wait complete. 2024-06-03T03:27:08.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.684+0000 7f3bc6724b80 20 client.15516 trim_cache size 0 max 16384 2024-06-03T03:27:08.702 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.701+0000 7f3bc6724b80 1 Processor -- start 2024-06-03T03:27:08.702 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.701+0000 7f3bc6724b80 1 -- start start 2024-06-03T03:27:08.703 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.702+0000 7f3bc6724b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b2f812e40 0x560b2f8a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:08.703 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.702+0000 7f3bc6724b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560b2f8965b0 0x560b2f8a26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:08.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.702+0000 7f3bc6724b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b2f8a1300 0x560b2f898fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:08.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.702+0000 7f3bc6724b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb41b49d0 con 0x560b2f8a1300 2024-06-03T03:27:08.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.702+0000 7f3bc6724b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb41b6510 con 0x560b2f812e40 2024-06-03T03:27:08.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.702+0000 7f3bc6724b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb41b50a0 con 0x560b2f8965b0 2024-06-03T03:27:08.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.702+0000 7f3bb9e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b2f8a1300 0x560b2f898fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:08.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.702+0000 7f3bb9e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b2f8a1300 0x560b2f898fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40724/0 (socket says 172.21.15.117:40724) 2024-06-03T03:27:08.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.702+0000 7f3bb9e00700 1 -- 172.21.15.117:0/3935848336 learned_addr learned my addr 172.21.15.117:0/3935848336 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:27:08.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.702+0000 7f3bb8a00700 1 --2- 172.21.15.117:0/3935848336 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560b2f8965b0 0x560b2f8a26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:08.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.702+0000 7f3bb9400700 1 --2- 172.21.15.117:0/3935848336 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b2f812e40 0x560b2f8a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:08.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.703+0000 7f3bb9400700 1 -- 172.21.15.117:0/3935848336 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560b2f8965b0 msgr2=0x560b2f8a26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:08.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.703+0000 7f3bb9400700 1 --2- 172.21.15.117:0/3935848336 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560b2f8965b0 0x560b2f8a26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:08.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.703+0000 7f3bb9400700 1 -- 172.21.15.117:0/3935848336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b2f8a1300 msgr2=0x560b2f898fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:08.705 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.703+0000 7f3bb9400700 1 --2- 172.21.15.117:0/3935848336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b2f8a1300 0x560b2f898fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:08.705 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.703+0000 7f3bb9400700 1 -- 172.21.15.117:0/3935848336 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b2f893080 con 0x560b2f812e40 2024-06-03T03:27:08.705 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.703+0000 7f3bb8a00700 1 --2- 172.21.15.117:0/3935848336 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560b2f8965b0 0x560b2f8a26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T03:27:08.705 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.703+0000 7f3bb9400700 1 --2- 172.21.15.117:0/3935848336 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b2f812e40 0x560b2f8a05d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x560b2f7e8c30 tx=0x7f3ba8009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82a21cdd37a3c06c server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:27:08.705 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.703+0000 7f3ba6000700 1 -- 172.21.15.117:0/3935848336 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba8004830 con 0x560b2f812e40 2024-06-03T03:27:08.705 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.703+0000 7f3ba6000700 1 -- 172.21.15.117:0/3935848336 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ba8005020 con 0x560b2f812e40 2024-06-03T03:27:08.705 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.703+0000 7f3ba6000700 1 -- 172.21.15.117:0/3935848336 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba80296c0 con 0x560b2f812e40 2024-06-03T03:27:08.707 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.704+0000 7f3bc6724b80 1 -- 172.21.15.117:0/3935848336 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b2f812e40 msgr2=0x560b2f8a05d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:08.707 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.704+0000 7f3bc6724b80 1 --2- 172.21.15.117:0/3935848336 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b2f812e40 0x560b2f8a05d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x560b2f7e8c30 tx=0x7f3ba8009260 comp rx=0 tx=0).stop 2024-06-03T03:27:08.708 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.704+0000 7f3bc6724b80 1 -- 172.21.15.117:0/3935848336 shutdown_connections 2024-06-03T03:27:08.708 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.704+0000 7f3bc6724b80 1 --2- 172.21.15.117:0/3935848336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b2f8a1300 0x560b2f898fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:08.708 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.704+0000 7f3bc6724b80 1 --2- 172.21.15.117:0/3935848336 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b2f812e40 0x560b2f8a05d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:08.708 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.704+0000 7f3bc6724b80 1 --2- 172.21.15.117:0/3935848336 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560b2f8965b0 0x560b2f8a26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:08.708 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.704+0000 7f3bc6724b80 1 -- 172.21.15.117:0/3935848336 >> 172.21.15.117:0/3935848336 conn(0x560b2f7c3170 msgr2=0x560b2f891fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:27:08.708 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.704+0000 7f3bc6724b80 1 -- 172.21.15.117:0/3935848336 shutdown_connections 2024-06-03T03:27:08.708 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.704+0000 7f3bc6724b80 1 -- 172.21.15.117:0/3935848336 wait complete. 2024-06-03T03:27:08.708 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.705+0000 7f3bc6724b80 1 -- start start 2024-06-03T03:27:08.708 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.705+0000 7f3bc6724b80 1 Processor -- start 2024-06-03T03:27:08.708 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.705+0000 7f3bc6724b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560b2f812e40 0x560b2f8b0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:08.708 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.706+0000 7f3bc6724b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b2f8965b0 0x560b2f8b1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:08.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.706+0000 7f3bc6724b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b2f8a1300 0x560b2f9194a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:08.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.706+0000 7f3bc6724b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb41b5e40 con 0x560b2f8a1300 2024-06-03T03:27:08.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.706+0000 7f3bc6724b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb41b5770 con 0x560b2f8965b0 2024-06-03T03:27:08.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.706+0000 7f3bc6724b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb41b2970 con 0x560b2f812e40 2024-06-03T03:27:08.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.706+0000 7f3bc6724b80 10 client.0 _ll_register_callbacks cb 0x560b2f782720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:27:08.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.706+0000 7f3bb9e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b2f8a1300 0x560b2f9194a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:08.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.706+0000 7f3bb8a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b2f8965b0 0x560b2f8b1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:08.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.706+0000 7f3bb8a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b2f8965b0 0x560b2f8b1220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:41332/0 (socket says 172.21.15.117:41332) 2024-06-03T03:27:08.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.706+0000 7f3bb8a00700 1 -- 172.21.15.117:0/1551356907 learned_addr learned my addr 172.21.15.117:0/1551356907 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:27:08.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.706+0000 7f3bb9400700 1 --2- 172.21.15.117:0/1551356907 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560b2f812e40 0x560b2f8b0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:08.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.706+0000 7f3bb9e00700 1 -- 172.21.15.117:0/1551356907 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560b2f812e40 msgr2=0x560b2f8b0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:08.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.706+0000 7f3bb9e00700 1 --2- 172.21.15.117:0/1551356907 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560b2f812e40 0x560b2f8b0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:08.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.706+0000 7f3bb9e00700 1 -- 172.21.15.117:0/1551356907 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b2f8965b0 msgr2=0x560b2f8b1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:08.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.706+0000 7f3bb9e00700 1 --2- 172.21.15.117:0/1551356907 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b2f8965b0 0x560b2f8b1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:08.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.706+0000 7f3bb9e00700 1 -- 172.21.15.117:0/1551356907 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b2f9199e0 con 0x560b2f8a1300 2024-06-03T03:27:08.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.706+0000 7f3bb9400700 1 --2- 172.21.15.117:0/1551356907 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560b2f812e40 0x560b2f8b0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:27:08.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.707+0000 7f3bb9e00700 1 --2- 172.21.15.117:0/1551356907 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b2f8a1300 0x560b2f9194a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3bac009330 tx=0x7f3bac00b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aaea9d0d763c8e8 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:27:08.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.707+0000 7f3b9b400700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-06-03T03:27:08.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.707+0000 7f3b9b400700 1 -- 172.21.15.117:0/1551356907 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bac007a80 con 0x560b2f8a1300 2024-06-03T03:27:08.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.707+0000 7f3b9b400700 1 -- 172.21.15.117:0/1551356907 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f3bac007c20 con 0x560b2f8a1300 2024-06-03T03:27:08.711 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.707+0000 7f3bc6724b80 1 -- 172.21.15.117:0/1551356907 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ba8005020 con 0x560b2f8a1300 2024-06-03T03:27:08.712 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.707+0000 7f3b9b400700 1 -- 172.21.15.117:0/1551356907 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f3bac007e20 con 0x560b2f8a1300 2024-06-03T03:27:08.712 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.707+0000 7f3bc6724b80 10 client.15522 fetch_fsmap learned FSMap version 31 2024-06-03T03:27:08.712 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.707+0000 7f3bc6724b80 1 -- 172.21.15.117:0/1551356907 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x560b2f91ab90 con 0x560b2f8a1300 2024-06-03T03:27:08.712 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.708+0000 7f3b9b400700 1 -- 172.21.15.117:0/1551356907 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ba8005020 con 0x560b2f8a1300 2024-06-03T03:27:08.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.708+0000 7f3b9b400700 1 -- 172.21.15.117:0/1551356907 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f3bac011070 con 0x560b2f8a1300 2024-06-03T03:27:08.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.708+0000 7f3bc6724b80 10 client.15522 fetch_fsmap finished waiting for FSMap version 31 2024-06-03T03:27:08.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.708+0000 7f3bc6724b80 10 client.15522 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-06-03T03:27:08.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.708+0000 7f3bc6724b80 20 client.15522 populate_metadata read hostname 'smithi117' 2024-06-03T03:27:08.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.708+0000 7f3bc6724b80 1 --2- 172.21.15.117:0/1551356907 >> [v2:172.21.15.151:6838/3623444616,v1:172.21.15.151:6839/3623444616] conn(0x560b2f91b270 0x560b2f91d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:08.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.708+0000 7f3bc6724b80 4 client.15522 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T03:27:08.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.708+0000 7f3bc6724b80 1 -- 172.21.15.117:0/1551356907 --> [v2:172.21.15.151:6838/3623444616,v1:172.21.15.151:6839/3623444616] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560b2f7f0580 con 0x560b2f91b270 2024-06-03T03:27:08.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.709+0000 7f3bb9400700 1 --2- 172.21.15.117:0/1551356907 >> [v2:172.21.15.151:6838/3623444616,v1:172.21.15.151:6839/3623444616] conn(0x560b2f91b270 0x560b2f91d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:08.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.709+0000 7f3bb9400700 1 --2- 172.21.15.117:0/1551356907 >> [v2:172.21.15.151:6838/3623444616,v1:172.21.15.151:6839/3623444616] conn(0x560b2f91b270 0x560b2f91d690 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f31b318fb03c8d7b server_cookie=8817a4b0701e43a9 in_seq=0 out_seq=0 2024-06-03T03:27:08.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.709+0000 7f3b9b400700 10 client.15522 ms_handle_connect on v2:172.21.15.151:6838/3623444616 2024-06-03T03:27:08.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.709+0000 7f3b9b400700 1 -- 172.21.15.117:0/1551356907 <== mds.1 v2:172.21.15.151:6838/3623444616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560b2f7f0580 con 0x560b2f91b270 2024-06-03T03:27:08.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.709+0000 7f3b9b400700 10 client.15522 handle_command_reply: tid=0 2024-06-03T03:27:08.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.709+0000 7f3bc6724b80 1 client.15522 shutdown 2024-06-03T03:27:08.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.709+0000 7f3bc6724b80 1 -- 172.21.15.117:0/1551356907 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b2f8a1300 msgr2=0x560b2f9194a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:08.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.709+0000 7f3bc6724b80 1 --2- 172.21.15.117:0/1551356907 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b2f8a1300 0x560b2f9194a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3bac009330 tx=0x7f3bac00b3b0 comp rx=0 tx=0).stop 2024-06-03T03:27:08.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.710+0000 7f3bc6724b80 1 -- 172.21.15.117:0/1551356907 shutdown_connections 2024-06-03T03:27:08.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.710+0000 7f3bc6724b80 1 --2- 172.21.15.117:0/1551356907 >> [v2:172.21.15.151:6838/3623444616,v1:172.21.15.151:6839/3623444616] conn(0x560b2f91b270 0x560b2f91d690 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:08.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.710+0000 7f3bc6724b80 1 --2- 172.21.15.117:0/1551356907 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b2f8a1300 0x560b2f9194a0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:08.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.710+0000 7f3bc6724b80 1 --2- 172.21.15.117:0/1551356907 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560b2f812e40 0x560b2f8b0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:08.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.710+0000 7f3bc6724b80 1 --2- 172.21.15.117:0/1551356907 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b2f8965b0 0x560b2f8b1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:08.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.710+0000 7f3b9b400700 0 client.15522 ms_handle_reset on v2:172.21.15.151:6838/3623444616 2024-06-03T03:27:08.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.710+0000 7f3bc6724b80 1 -- 172.21.15.117:0/1551356907 >> 172.21.15.117:0/1551356907 conn(0x560b2f7c3170 msgr2=0x560b2f890eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:27:08.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.710+0000 7f3bc6724b80 1 -- 172.21.15.117:0/1551356907 shutdown_connections 2024-06-03T03:27:08.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.710+0000 7f3bc6724b80 1 -- 172.21.15.117:0/1551356907 wait complete. 2024-06-03T03:27:08.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.710+0000 7f3bc6724b80 20 client.15522 trim_cache size 0 max 16384 2024-06-03T03:27:08.715 INFO:teuthology.orchestra.run.smithi117.stdout:[] 2024-06-03T03:27:08.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.711+0000 7f3bbbe00700 1 -- 172.21.15.117:0/3704085439 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f3b9c0a2ca0 msgr2=0x7f3b9c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:08.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.711+0000 7f3bbbe00700 1 --2- 172.21.15.117:0/3704085439 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f3b9c0a2ca0 0x7f3b9c0a5160 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3bac009e60 tx=0x7f3bac009360 comp rx=0 tx=0).stop 2024-06-03T03:27:08.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.711+0000 7f3bbbe00700 1 -- 172.21.15.117:0/3704085439 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3bb406b320 msgr2=0x7f3bb410c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:08.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.711+0000 7f3bbbe00700 1 --2- 172.21.15.117:0/3704085439 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3bb406b320 0x7f3bb410c7d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3ba000e9f0 tx=0x7f3ba000ed00 comp rx=0 tx=0).stop 2024-06-03T03:27:08.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.711+0000 7f3bbbe00700 1 -- 172.21.15.117:0/3704085439 shutdown_connections 2024-06-03T03:27:08.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.711+0000 7f3bbbe00700 1 --2- 172.21.15.117:0/3704085439 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f3b9c0a2ca0 0x7f3b9c0a5160 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:08.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.711+0000 7f3bbbe00700 1 --2- 172.21.15.117:0/3704085439 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3bb410ef30 0x7f3bb410cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:08.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.711+0000 7f3bbbe00700 1 --2- 172.21.15.117:0/3704085439 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3bb406aa00 0x7f3bb410e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:08.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.711+0000 7f3bbbe00700 1 --2- 172.21.15.117:0/3704085439 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3bb406b320 0x7f3bb410c7d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:08.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.711+0000 7f3bbbe00700 1 -- 172.21.15.117:0/3704085439 >> 172.21.15.117:0/3704085439 conn(0x7f3bb4078330 msgr2=0x7f3bb4110340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:27:08.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.711+0000 7f3bbbe00700 1 -- 172.21.15.117:0/3704085439 shutdown_connections 2024-06-03T03:27:08.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:08.712+0000 7f3bbbe00700 1 -- 172.21.15.117:0/3704085439 wait complete. 2024-06-03T03:27:09.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:27:09 smithi151 ceph-mon[37043]: pgmap v561: 129 pgs: 129 active+clean; 1.8 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 607 KiB/s wr, 18 op/s 2024-06-03T03:27:09.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:27:09 smithi117 ceph-mon[31649]: pgmap v561: 129 pgs: 129 active+clean; 1.8 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 607 KiB/s wr, 18 op/s 2024-06-03T03:27:09.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:27:09 smithi183 ceph-mon[37326]: pgmap v561: 129 pgs: 129 active+clean; 1.8 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 607 KiB/s wr, 18 op/s 2024-06-03T03:27:09.718 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-03T03:27:10.152 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:27:11.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:27:11 smithi151 ceph-mon[37043]: pgmap v562: 129 pgs: 129 active+clean; 1.8 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 1.0 MiB/s wr, 16 op/s 2024-06-03T03:27:11.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:27:11 smithi117 ceph-mon[31649]: pgmap v562: 129 pgs: 129 active+clean; 1.8 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 1.0 MiB/s wr, 16 op/s 2024-06-03T03:27:11.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:27:11 smithi183 ceph-mon[37326]: pgmap v562: 129 pgs: 129 active+clean; 1.8 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 1.0 MiB/s wr, 16 op/s 2024-06-03T03:27:12.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.606+0000 7f9494600700 1 -- 172.21.15.117:0/3153914764 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f948c1024e0 msgr2=0x7f948c102960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:12.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.606+0000 7f9494600700 1 --2- 172.21.15.117:0/3153914764 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f948c1024e0 0x7f948c102960 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9478009990 tx=0x7f9478009ca0 comp rx=0 tx=0).stop 2024-06-03T03:27:12.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.607+0000 7f9494600700 1 -- 172.21.15.117:0/3153914764 shutdown_connections 2024-06-03T03:27:12.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.607+0000 7f9494600700 1 --2- 172.21.15.117:0/3153914764 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f948c102ea0 0x7f948c10f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:12.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.607+0000 7f9494600700 1 --2- 172.21.15.117:0/3153914764 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f948c1085f0 0x7f948c1089d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:12.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.607+0000 7f9494600700 1 --2- 172.21.15.117:0/3153914764 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f948c1024e0 0x7f948c102960 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:12.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.607+0000 7f9494600700 1 -- 172.21.15.117:0/3153914764 >> 172.21.15.117:0/3153914764 conn(0x7f948c079640 msgr2=0x7f948c079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:27:12.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.607+0000 7f9494600700 1 -- 172.21.15.117:0/3153914764 shutdown_connections 2024-06-03T03:27:12.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.607+0000 7f9494600700 1 -- 172.21.15.117:0/3153914764 wait complete. 2024-06-03T03:27:12.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.608+0000 7f9494600700 1 Processor -- start 2024-06-03T03:27:12.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.609+0000 7f9494600700 1 -- start start 2024-06-03T03:27:12.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.609+0000 7f9494600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f948c1024e0 0x7f948c19ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:12.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.609+0000 7f9494600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f948c102ea0 0x7f948c19b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:12.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.609+0000 7f9494600700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f948c1085f0 0x7f948c19f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:12.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.609+0000 7f9494600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f948c1127e0 con 0x7f948c1024e0 2024-06-03T03:27:12.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.609+0000 7f9494600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f948c112660 con 0x7f948c102ea0 2024-06-03T03:27:12.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.609+0000 7f9494600700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f948c112960 con 0x7f948c1085f0 2024-06-03T03:27:12.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.609+0000 7f9491c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f948c1024e0 0x7f948c19ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:12.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.609+0000 7f9491c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f948c1024e0 0x7f948c19ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40746/0 (socket says 172.21.15.117:40746) 2024-06-03T03:27:12.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.609+0000 7f9491200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f948c102ea0 0x7f948c19b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:12.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.609+0000 7f9491c00700 1 -- 172.21.15.117:0/2348944148 learned_addr learned my addr 172.21.15.117:0/2348944148 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:27:12.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.610+0000 7f9492600700 1 --2- 172.21.15.117:0/2348944148 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f948c1085f0 0x7f948c19f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:12.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.610+0000 7f9491200700 1 -- 172.21.15.117:0/2348944148 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f948c1085f0 msgr2=0x7f948c19f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:12.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.610+0000 7f9491200700 1 --2- 172.21.15.117:0/2348944148 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f948c1085f0 0x7f948c19f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:12.612 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.610+0000 7f9491200700 1 -- 172.21.15.117:0/2348944148 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f948c1024e0 msgr2=0x7f948c19ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:12.612 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.610+0000 7f9491200700 1 --2- 172.21.15.117:0/2348944148 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f948c1024e0 0x7f948c19ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:12.612 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.610+0000 7f9491200700 1 -- 172.21.15.117:0/2348944148 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f948c19fe90 con 0x7f948c102ea0 2024-06-03T03:27:12.612 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.610+0000 7f9491200700 1 --2- 172.21.15.117:0/2348944148 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f948c102ea0 0x7f948c19b3c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9478000c00 tx=0x7f947800bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:27:12.612 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.611+0000 7f948a000700 1 -- 172.21.15.117:0/2348944148 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9478026070 con 0x7f948c102ea0 2024-06-03T03:27:12.612 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.611+0000 7f948a000700 1 -- 172.21.15.117:0/2348944148 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9478003d70 con 0x7f948c102ea0 2024-06-03T03:27:12.612 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.611+0000 7f9494600700 1 -- 172.21.15.117:0/2348944148 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f948c1a0120 con 0x7f948c102ea0 2024-06-03T03:27:12.612 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.611+0000 7f9494600700 1 -- 172.21.15.117:0/2348944148 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f948c1a7a00 con 0x7f948c102ea0 2024-06-03T03:27:12.612 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.611+0000 7f948a000700 1 -- 172.21.15.117:0/2348944148 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9478021410 con 0x7f948c102ea0 2024-06-03T03:27:12.615 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.614+0000 7f948a000700 1 -- 172.21.15.117:0/2348944148 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f9478029070 con 0x7f948c102ea0 2024-06-03T03:27:12.615 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.615+0000 7f949ef63b80 1 Processor -- start 2024-06-03T03:27:12.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.615+0000 7f949ef63b80 1 -- start start 2024-06-03T03:27:12.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.616+0000 7f949ef63b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f43ec97e40 0x55f43ed1b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:12.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.616+0000 7f949ef63b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f43ed1b5b0 0x55f43ed1ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:12.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.616+0000 7f949ef63b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f43ed26300 0x55f43ed286f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:12.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.616+0000 7f949ef63b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f43ec2aa10 con 0x55f43ed26300 2024-06-03T03:27:12.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.616+0000 7f949ef63b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f43ec6db00 con 0x55f43ec97e40 2024-06-03T03:27:12.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.616+0000 7f949ef63b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f43ec288f0 con 0x55f43ed1b5b0 2024-06-03T03:27:12.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.616+0000 7f9491c00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f43ed1b5b0 0x55f43ed1ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:12.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.616+0000 7f9491c00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f43ed1b5b0 0x55f43ed1ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:54802/0 (socket says 172.21.15.117:54802) 2024-06-03T03:27:12.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.616+0000 7f9492600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f43ec97e40 0x55f43ed1b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:12.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.616+0000 7f9491c00700 1 -- 172.21.15.117:0/2498404583 learned_addr learned my addr 172.21.15.117:0/2498404583 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:27:12.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.617+0000 7f9491200700 1 --2- 172.21.15.117:0/2498404583 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f43ed26300 0x55f43ed286f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:12.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.617+0000 7f9491c00700 1 -- 172.21.15.117:0/2498404583 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f43ec97e40 msgr2=0x55f43ed1b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:12.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.617+0000 7f9491c00700 1 --2- 172.21.15.117:0/2498404583 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f43ec97e40 0x55f43ed1b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:12.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.617+0000 7f9491c00700 1 -- 172.21.15.117:0/2498404583 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f43ed26300 msgr2=0x55f43ed286f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:12.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.617+0000 7f9491c00700 1 --2- 172.21.15.117:0/2498404583 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f43ed26300 0x55f43ed286f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:12.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.617+0000 7f948a000700 1 --2- 172.21.15.117:0/2348944148 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f94740a2ca0 0x7f94740a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:12.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.617+0000 7f9491c00700 1 -- 172.21.15.117:0/2498404583 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f43ec15f90 con 0x55f43ed1b5b0 2024-06-03T03:27:12.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.617+0000 7f9491200700 1 --2- 172.21.15.117:0/2498404583 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f43ed26300 0x55f43ed286f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T03:27:12.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.617+0000 7f948a000700 1 -- 172.21.15.117:0/2348944148 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f94780d00e0 con 0x7f948c102ea0 2024-06-03T03:27:12.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.617+0000 7f9491c00700 1 --2- 172.21.15.117:0/2498404583 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f43ed1b5b0 0x55f43ed1ba30 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9480007d20 tx=0x7f94800106b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57629ecbb9c5ab34 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:27:12.620 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.618+0000 7f9492600700 1 --2- 172.21.15.117:0/2348944148 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f94740a2ca0 0x7f94740a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:12.620 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.618+0000 7f947ea00700 1 -- 172.21.15.117:0/2498404583 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9480010e10 con 0x55f43ed1b5b0 2024-06-03T03:27:12.620 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.618+0000 7f947ea00700 1 -- 172.21.15.117:0/2498404583 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9480011920 con 0x55f43ed1b5b0 2024-06-03T03:27:12.620 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.618+0000 7f947ea00700 1 -- 172.21.15.117:0/2498404583 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94800198b0 con 0x55f43ed1b5b0 2024-06-03T03:27:12.620 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.618+0000 7f949ef63b80 1 -- 172.21.15.117:0/2498404583 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f43ed1b5b0 msgr2=0x55f43ed1ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:12.620 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.618+0000 7f949ef63b80 1 --2- 172.21.15.117:0/2498404583 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f43ed1b5b0 0x55f43ed1ba30 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9480007d20 tx=0x7f94800106b0 comp rx=0 tx=0).stop 2024-06-03T03:27:12.620 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.619+0000 7f949ef63b80 1 -- 172.21.15.117:0/2498404583 shutdown_connections 2024-06-03T03:27:12.620 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.619+0000 7f949ef63b80 1 --2- 172.21.15.117:0/2498404583 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f43ed26300 0x55f43ed286f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:12.620 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.619+0000 7f949ef63b80 1 --2- 172.21.15.117:0/2498404583 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f43ec97e40 0x55f43ed1b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:12.620 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.619+0000 7f949ef63b80 1 --2- 172.21.15.117:0/2498404583 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f43ed1b5b0 0x55f43ed1ba30 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:12.620 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.619+0000 7f949ef63b80 1 -- 172.21.15.117:0/2498404583 >> 172.21.15.117:0/2498404583 conn(0x55f43ec48170 msgr2=0x55f43ed174b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:27:12.620 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.619+0000 7f9492600700 1 --2- 172.21.15.117:0/2348944148 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f94740a2ca0 0x7f94740a5160 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x55f43ed183b0 tx=0x7f948400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:27:12.620 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.619+0000 7f949ef63b80 1 -- 172.21.15.117:0/2498404583 shutdown_connections 2024-06-03T03:27:12.621 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.619+0000 7f949ef63b80 1 -- 172.21.15.117:0/2498404583 wait complete. 2024-06-03T03:27:12.621 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.619+0000 7f949ef63b80 1 -- start start 2024-06-03T03:27:12.622 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.620+0000 7f949ef63b80 1 Processor -- start 2024-06-03T03:27:12.622 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.620+0000 7f949ef63b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f43ec97e40 0x55f43ed25aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:12.622 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.620+0000 7f949ef63b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f43ed1b5b0 0x55f43ed22da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:12.623 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.620+0000 7f949ef63b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f43ed26300 0x55f43ed232e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:12.623 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.620+0000 7f949ef63b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f43ec09f60 con 0x55f43ed1b5b0 2024-06-03T03:27:12.623 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.620+0000 7f949ef63b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f43ec69ac0 con 0x55f43ed26300 2024-06-03T03:27:12.623 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.620+0000 7f949ef63b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f43ed28fc0 con 0x55f43ec97e40 2024-06-03T03:27:12.623 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.620+0000 7f949ef63b80 10 client.0 _ll_register_callbacks cb 0x55f43ec07720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:27:12.623 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.620+0000 7f9491200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f43ed1b5b0 0x55f43ed22da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:12.623 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.620+0000 7f9491c00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f43ec97e40 0x55f43ed25aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:12.623 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.620+0000 7f9491c00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f43ec97e40 0x55f43ed25aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:54808/0 (socket says 172.21.15.117:54808) 2024-06-03T03:27:12.623 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.621+0000 7f9491c00700 1 -- 172.21.15.117:0/3483272979 learned_addr learned my addr 172.21.15.117:0/3483272979 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:27:12.623 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.621+0000 7f9492600700 1 --2- 172.21.15.117:0/3483272979 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f43ed26300 0x55f43ed232e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:12.623 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.621+0000 7f9491c00700 1 -- 172.21.15.117:0/3483272979 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f43ed26300 msgr2=0x55f43ed232e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:12.623 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.621+0000 7f9491c00700 1 --2- 172.21.15.117:0/3483272979 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f43ed26300 0x55f43ed232e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:12.623 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.621+0000 7f9491c00700 1 -- 172.21.15.117:0/3483272979 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f43ed1b5b0 msgr2=0x55f43ed22da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:12.624 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.621+0000 7f9491c00700 1 --2- 172.21.15.117:0/3483272979 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f43ed1b5b0 0x55f43ed22da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:12.624 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.621+0000 7f9491c00700 1 -- 172.21.15.117:0/3483272979 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f43ebbc8d0 con 0x55f43ec97e40 2024-06-03T03:27:12.624 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.621+0000 7f9491200700 1 --2- 172.21.15.117:0/3483272979 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f43ed1b5b0 0x55f43ed22da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T03:27:12.624 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.621+0000 7f9491c00700 1 --2- 172.21.15.117:0/3483272979 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f43ec97e40 0x55f43ed25aa0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9480000c00 tx=0x7f9480010d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d43dbda271e662ab server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:27:12.624 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.621+0000 7f9473400700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-06-03T03:27:12.624 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.622+0000 7f9473400700 1 -- 172.21.15.117:0/3483272979 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9480028060 con 0x55f43ec97e40 2024-06-03T03:27:12.624 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.622+0000 7f949ef63b80 1 -- 172.21.15.117:0/3483272979 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9480011920 con 0x55f43ec97e40 2024-06-03T03:27:12.624 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.622+0000 7f9473400700 1 -- 172.21.15.117:0/3483272979 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f9480011640 con 0x55f43ec97e40 2024-06-03T03:27:12.624 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.622+0000 7f9473400700 1 -- 172.21.15.117:0/3483272979 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f948002f050 con 0x55f43ec97e40 2024-06-03T03:27:12.624 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.622+0000 7f949ef63b80 10 client.25666 fetch_fsmap learned FSMap version 31 2024-06-03T03:27:12.624 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.622+0000 7f949ef63b80 1 -- 172.21.15.117:0/3483272979 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55f43ed33680 con 0x55f43ec97e40 2024-06-03T03:27:12.626 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.622+0000 7f9473400700 1 -- 172.21.15.117:0/3483272979 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9480011920 con 0x55f43ec97e40 2024-06-03T03:27:12.626 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.623+0000 7f9473400700 1 -- 172.21.15.117:0/3483272979 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f9480022030 con 0x55f43ec97e40 2024-06-03T03:27:12.626 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.623+0000 7f949ef63b80 10 client.25666 fetch_fsmap finished waiting for FSMap version 31 2024-06-03T03:27:12.626 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.623+0000 7f949ef63b80 10 client.25666 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-03T03:27:12.626 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.623+0000 7f949ef63b80 20 client.25666 populate_metadata read hostname 'smithi117' 2024-06-03T03:27:12.626 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.623+0000 7f949ef63b80 1 --2- 172.21.15.117:0/3483272979 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55f43ed35bb0 0x55f43ed37fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:12.626 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.623+0000 7f949ef63b80 4 client.25666 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T03:27:12.626 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.623+0000 7f949ef63b80 1 -- 172.21.15.117:0/3483272979 --> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f43ec7b000 con 0x55f43ed35bb0 2024-06-03T03:27:12.627 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.623+0000 7f9491200700 1 --2- 172.21.15.117:0/3483272979 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55f43ed35bb0 0x55f43ed37fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:12.627 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.624+0000 7f9491200700 1 --2- 172.21.15.117:0/3483272979 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55f43ed35bb0 0x55f43ed37fa0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef22a6185b065403 server_cookie=12c0e1239ef693a6 in_seq=0 out_seq=0 2024-06-03T03:27:12.627 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.624+0000 7f9473400700 10 client.25666 ms_handle_connect on v2:172.21.15.183:6838/1439615083 2024-06-03T03:27:12.627 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.625+0000 7f9473400700 1 -- 172.21.15.117:0/3483272979 <== mds.0 v2:172.21.15.183:6838/1439615083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f43ec7b000 con 0x55f43ed35bb0 2024-06-03T03:27:12.627 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.625+0000 7f9473400700 10 client.25666 handle_command_reply: tid=0 2024-06-03T03:27:12.627 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.625+0000 7f949ef63b80 1 client.25666 shutdown 2024-06-03T03:27:12.627 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.625+0000 7f949ef63b80 1 -- 172.21.15.117:0/3483272979 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f43ec97e40 msgr2=0x55f43ed25aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:12.627 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.625+0000 7f949ef63b80 1 --2- 172.21.15.117:0/3483272979 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f43ec97e40 0x55f43ed25aa0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9480000c00 tx=0x7f9480010d70 comp rx=0 tx=0).stop 2024-06-03T03:27:12.627 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.625+0000 7f949ef63b80 1 -- 172.21.15.117:0/3483272979 shutdown_connections 2024-06-03T03:27:12.627 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.625+0000 7f949ef63b80 1 --2- 172.21.15.117:0/3483272979 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55f43ed35bb0 0x55f43ed37fa0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:12.627 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.625+0000 7f949ef63b80 1 --2- 172.21.15.117:0/3483272979 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f43ed26300 0x55f43ed232e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:12.627 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.625+0000 7f949ef63b80 1 --2- 172.21.15.117:0/3483272979 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f43ec97e40 0x55f43ed25aa0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:12.627 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.625+0000 7f949ef63b80 1 --2- 172.21.15.117:0/3483272979 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f43ed1b5b0 0x55f43ed22da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:12.628 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.625+0000 7f949ef63b80 1 -- 172.21.15.117:0/3483272979 >> 172.21.15.117:0/3483272979 conn(0x55f43ec48170 msgr2=0x55f43ed16740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:27:12.628 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.625+0000 7f9473400700 0 client.25666 ms_handle_reset on v2:172.21.15.183:6838/1439615083 2024-06-03T03:27:12.628 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.625+0000 7f949ef63b80 1 -- 172.21.15.117:0/3483272979 shutdown_connections 2024-06-03T03:27:12.628 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.625+0000 7f949ef63b80 1 -- 172.21.15.117:0/3483272979 wait complete. 2024-06-03T03:27:12.628 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.626+0000 7f949ef63b80 20 client.25666 trim_cache size 0 max 16384 2024-06-03T03:27:12.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.643+0000 7f949ef63b80 1 Processor -- start 2024-06-03T03:27:12.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.643+0000 7f949ef63b80 1 -- start start 2024-06-03T03:27:12.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.643+0000 7f949ef63b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f43ec97e40 0x55f43ed17620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:12.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.643+0000 7f949ef63b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f43ed1b5b0 0x55f43ed21f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:12.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.643+0000 7f949ef63b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f43ed26300 0x55f43ed165c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:12.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.643+0000 7f949ef63b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f948c1b04a0 con 0x55f43ec97e40 2024-06-03T03:27:12.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.643+0000 7f949ef63b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f948c1b1fe0 con 0x55f43ed26300 2024-06-03T03:27:12.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.643+0000 7f949ef63b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f948c1b0b70 con 0x55f43ed1b5b0 2024-06-03T03:27:12.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.644+0000 7f9491c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f43ec97e40 0x55f43ed17620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:12.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.644+0000 7f9491200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f43ed1b5b0 0x55f43ed21f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:12.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.644+0000 7f9491200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f43ed1b5b0 0x55f43ed21f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:54822/0 (socket says 172.21.15.117:54822) 2024-06-03T03:27:12.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.644+0000 7f9491200700 1 -- 172.21.15.117:0/2317859770 learned_addr learned my addr 172.21.15.117:0/2317859770 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:27:12.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.644+0000 7f9492600700 1 --2- 172.21.15.117:0/2317859770 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f43ed26300 0x55f43ed165c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:12.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.644+0000 7f9491200700 1 -- 172.21.15.117:0/2317859770 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f43ed26300 msgr2=0x55f43ed165c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:12.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.644+0000 7f9491200700 1 --2- 172.21.15.117:0/2317859770 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f43ed26300 0x55f43ed165c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:12.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.644+0000 7f9491200700 1 -- 172.21.15.117:0/2317859770 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f43ec97e40 msgr2=0x55f43ed17620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:12.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.644+0000 7f9491200700 1 --2- 172.21.15.117:0/2317859770 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f43ec97e40 0x55f43ed17620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:12.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.644+0000 7f9491200700 1 -- 172.21.15.117:0/2317859770 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f43ed0ee10 con 0x55f43ed1b5b0 2024-06-03T03:27:12.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.644+0000 7f9491200700 1 --2- 172.21.15.117:0/2317859770 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f43ed1b5b0 0x55f43ed21f20 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f947809c2e0 tx=0x7f947800f970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74c111e20bf79155 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:27:12.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.644+0000 7f947e000700 1 -- 172.21.15.117:0/2317859770 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f947809bf40 con 0x55f43ed1b5b0 2024-06-03T03:27:12.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.645+0000 7f947e000700 1 -- 172.21.15.117:0/2317859770 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94780043b0 con 0x55f43ed1b5b0 2024-06-03T03:27:12.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.645+0000 7f947e000700 1 -- 172.21.15.117:0/2317859770 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94780cd900 con 0x55f43ed1b5b0 2024-06-03T03:27:12.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.645+0000 7f949ef63b80 1 -- 172.21.15.117:0/2317859770 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f43ed1b5b0 msgr2=0x55f43ed21f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:12.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.645+0000 7f949ef63b80 1 --2- 172.21.15.117:0/2317859770 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f43ed1b5b0 0x55f43ed21f20 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f947809c2e0 tx=0x7f947800f970 comp rx=0 tx=0).stop 2024-06-03T03:27:12.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.646+0000 7f949ef63b80 1 -- 172.21.15.117:0/2317859770 shutdown_connections 2024-06-03T03:27:12.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.646+0000 7f949ef63b80 1 --2- 172.21.15.117:0/2317859770 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f43ed26300 0x55f43ed165c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:12.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.646+0000 7f949ef63b80 1 --2- 172.21.15.117:0/2317859770 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f43ec97e40 0x55f43ed17620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:12.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.646+0000 7f949ef63b80 1 --2- 172.21.15.117:0/2317859770 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f43ed1b5b0 0x55f43ed21f20 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:12.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.646+0000 7f949ef63b80 1 -- 172.21.15.117:0/2317859770 >> 172.21.15.117:0/2317859770 conn(0x55f43ec48170 msgr2=0x55f43ed15eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:27:12.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.646+0000 7f949ef63b80 1 -- 172.21.15.117:0/2317859770 shutdown_connections 2024-06-03T03:27:12.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.646+0000 7f949ef63b80 1 -- 172.21.15.117:0/2317859770 wait complete. 2024-06-03T03:27:12.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.646+0000 7f949ef63b80 1 -- start start 2024-06-03T03:27:12.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.646+0000 7f949ef63b80 1 Processor -- start 2024-06-03T03:27:12.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.647+0000 7f949ef63b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f43ec97e40 0x55f43ed1f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:12.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.647+0000 7f949ef63b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f43ed1b5b0 0x55f43ed1f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:12.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.647+0000 7f949ef63b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f43ed26300 0x55f43ed1fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:12.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.647+0000 7f949ef63b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f948c1b1910 con 0x55f43ed26300 2024-06-03T03:27:12.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.647+0000 7f949ef63b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f948c1b1240 con 0x55f43ec97e40 2024-06-03T03:27:12.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.647+0000 7f949ef63b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f948c1ae440 con 0x55f43ed1b5b0 2024-06-03T03:27:12.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.647+0000 7f949ef63b80 10 client.0 _ll_register_callbacks cb 0x55f43ec07720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:27:12.650 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.647+0000 7f9491200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f43ed1b5b0 0x55f43ed1f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:12.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.647+0000 7f9491c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f43ec97e40 0x55f43ed1f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:12.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.647+0000 7f9491c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f43ec97e40 0x55f43ed1f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:41408/0 (socket says 172.21.15.117:41408) 2024-06-03T03:27:12.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.647+0000 7f9491c00700 1 -- 172.21.15.117:0/3870935842 learned_addr learned my addr 172.21.15.117:0/3870935842 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:27:12.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.647+0000 7f9491200700 1 --2- 172.21.15.117:0/3870935842 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f43ed1b5b0 0x55f43ed1f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:54832/0 (socket says 172.21.15.117:54832) 2024-06-03T03:27:12.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.647+0000 7f9492600700 1 --2- 172.21.15.117:0/3870935842 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f43ed26300 0x55f43ed1fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:12.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.648+0000 7f9491c00700 1 -- 172.21.15.117:0/3870935842 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f43ed1b5b0 msgr2=0x55f43ed1f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:12.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.648+0000 7f9491c00700 1 --2- 172.21.15.117:0/3870935842 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f43ed1b5b0 0x55f43ed1f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:12.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.648+0000 7f9491c00700 1 -- 172.21.15.117:0/3870935842 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f43ed26300 msgr2=0x55f43ed1fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:12.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.648+0000 7f9491c00700 1 --2- 172.21.15.117:0/3870935842 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f43ed26300 0x55f43ed1fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:12.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.648+0000 7f9491c00700 1 -- 172.21.15.117:0/3870935842 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f43ed2d500 con 0x55f43ec97e40 2024-06-03T03:27:12.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.648+0000 7f9491c00700 1 --2- 172.21.15.117:0/3870935842 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f43ec97e40 0x55f43ed1f1d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f94800036c0 tx=0x7f9480003cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da8e29a9ec6d79d0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:27:12.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.648+0000 7f9473400700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-06-03T03:27:12.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.648+0000 7f9473400700 1 -- 172.21.15.117:0/3870935842 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9480038060 con 0x55f43ec97e40 2024-06-03T03:27:12.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.649+0000 7f949ef63b80 1 -- 172.21.15.117:0/3870935842 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94780043b0 con 0x55f43ec97e40 2024-06-03T03:27:12.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.649+0000 7f9473400700 1 -- 172.21.15.117:0/3870935842 <== mon.2 v2:172.21.15.151:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f94800054d0 con 0x55f43ec97e40 2024-06-03T03:27:12.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.649+0000 7f9473400700 1 -- 172.21.15.117:0/3870935842 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f948003f050 con 0x55f43ec97e40 2024-06-03T03:27:12.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.649+0000 7f949ef63b80 10 client.25634 fetch_fsmap learned FSMap version 31 2024-06-03T03:27:12.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.649+0000 7f949ef63b80 1 -- 172.21.15.117:0/3870935842 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55f43ed2e800 con 0x55f43ec97e40 2024-06-03T03:27:12.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.649+0000 7f9473400700 1 -- 172.21.15.117:0/3870935842 <== mon.2 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94780043b0 con 0x55f43ec97e40 2024-06-03T03:27:12.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.650+0000 7f9473400700 1 -- 172.21.15.117:0/3870935842 <== mon.2 v2:172.21.15.151:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f9480022070 con 0x55f43ec97e40 2024-06-03T03:27:12.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.650+0000 7f949ef63b80 10 client.25634 fetch_fsmap finished waiting for FSMap version 31 2024-06-03T03:27:12.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.650+0000 7f949ef63b80 10 client.25634 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-03T03:27:12.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.650+0000 7f949ef63b80 20 client.25634 populate_metadata read hostname 'smithi117' 2024-06-03T03:27:12.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.650+0000 7f949ef63b80 1 --2- 172.21.15.117:0/3870935842 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55f43eda0510 0x55f43eda2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:12.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.650+0000 7f949ef63b80 4 client.25634 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T03:27:12.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.650+0000 7f949ef63b80 1 -- 172.21.15.117:0/3870935842 --> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f43ec75580 con 0x55f43eda0510 2024-06-03T03:27:12.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.650+0000 7f9491200700 1 --2- 172.21.15.117:0/3870935842 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55f43eda0510 0x55f43eda2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:12.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.651+0000 7f9491200700 1 --2- 172.21.15.117:0/3870935842 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55f43eda0510 0x55f43eda2900 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74d9ebc0caa3a8e5 server_cookie=7baa99708bd6b31d in_seq=0 out_seq=0 2024-06-03T03:27:12.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.651+0000 7f9473400700 10 client.25634 ms_handle_connect on v2:172.21.15.183:6838/1439615083 2024-06-03T03:27:12.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.651+0000 7f9473400700 1 -- 172.21.15.117:0/3870935842 <== mds.0 v2:172.21.15.183:6838/1439615083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f43ec75580 con 0x55f43eda0510 2024-06-03T03:27:12.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.651+0000 7f9473400700 10 client.25634 handle_command_reply: tid=0 2024-06-03T03:27:12.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.651+0000 7f949ef63b80 1 client.25634 shutdown 2024-06-03T03:27:12.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.651+0000 7f949ef63b80 1 -- 172.21.15.117:0/3870935842 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f43ec97e40 msgr2=0x55f43ed1f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:12.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.651+0000 7f949ef63b80 1 --2- 172.21.15.117:0/3870935842 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f43ec97e40 0x55f43ed1f1d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f94800036c0 tx=0x7f9480003cb0 comp rx=0 tx=0).stop 2024-06-03T03:27:12.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.651+0000 7f949ef63b80 1 -- 172.21.15.117:0/3870935842 shutdown_connections 2024-06-03T03:27:12.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.651+0000 7f949ef63b80 1 --2- 172.21.15.117:0/3870935842 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55f43eda0510 0x55f43eda2900 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:12.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.651+0000 7f949ef63b80 1 --2- 172.21.15.117:0/3870935842 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f43ed26300 0x55f43ed1fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:12.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.651+0000 7f949ef63b80 1 --2- 172.21.15.117:0/3870935842 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f43ec97e40 0x55f43ed1f1d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:12.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.651+0000 7f949ef63b80 1 --2- 172.21.15.117:0/3870935842 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f43ed1b5b0 0x55f43ed1f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:12.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.651+0000 7f949ef63b80 1 -- 172.21.15.117:0/3870935842 >> 172.21.15.117:0/3870935842 conn(0x55f43ec48170 msgr2=0x55f43ed1cfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:27:12.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.651+0000 7f9473400700 0 client.25634 ms_handle_reset on v2:172.21.15.183:6838/1439615083 2024-06-03T03:27:12.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.651+0000 7f949ef63b80 1 -- 172.21.15.117:0/3870935842 shutdown_connections 2024-06-03T03:27:12.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.651+0000 7f949ef63b80 1 -- 172.21.15.117:0/3870935842 wait complete. 2024-06-03T03:27:12.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.652+0000 7f949ef63b80 20 client.25634 trim_cache size 0 max 16384 2024-06-03T03:27:12.655 INFO:teuthology.orchestra.run.smithi117.stdout:[] 2024-06-03T03:27:12.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.652+0000 7f9494600700 1 -- 172.21.15.117:0/2348944148 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f94740a2ca0 msgr2=0x7f94740a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:12.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.652+0000 7f9494600700 1 --2- 172.21.15.117:0/2348944148 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f94740a2ca0 0x7f94740a5160 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x55f43ed183b0 tx=0x7f948400b040 comp rx=0 tx=0).stop 2024-06-03T03:27:12.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.652+0000 7f9494600700 1 -- 172.21.15.117:0/2348944148 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f948c102ea0 msgr2=0x7f948c19b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:12.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.652+0000 7f9494600700 1 --2- 172.21.15.117:0/2348944148 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f948c102ea0 0x7f948c19b3c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9478000c00 tx=0x7f947800bab0 comp rx=0 tx=0).stop 2024-06-03T03:27:12.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.652+0000 7f9494600700 1 -- 172.21.15.117:0/2348944148 shutdown_connections 2024-06-03T03:27:12.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.652+0000 7f9494600700 1 --2- 172.21.15.117:0/2348944148 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f94740a2ca0 0x7f94740a5160 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:12.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.652+0000 7f9494600700 1 --2- 172.21.15.117:0/2348944148 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f948c1085f0 0x7f948c19f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:12.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.652+0000 7f9494600700 1 --2- 172.21.15.117:0/2348944148 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f948c1024e0 0x7f948c19ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:12.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.652+0000 7f9494600700 1 --2- 172.21.15.117:0/2348944148 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f948c102ea0 0x7f948c19b3c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:12.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.652+0000 7f9494600700 1 -- 172.21.15.117:0/2348944148 >> 172.21.15.117:0/2348944148 conn(0x7f948c079640 msgr2=0x7f948c100e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:27:12.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.653+0000 7f9494600700 1 -- 172.21.15.117:0/2348944148 shutdown_connections 2024-06-03T03:27:12.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:12.653+0000 7f9494600700 1 -- 172.21.15.117:0/2348944148 wait complete. 2024-06-03T03:27:13.199 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:27:13 smithi117 ceph-mon[31649]: pgmap v563: 129 pgs: 129 active+clean; 1.8 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 866 KiB/s wr, 19 op/s 2024-06-03T03:27:13.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:27:13 smithi151 ceph-mon[37043]: pgmap v563: 129 pgs: 129 active+clean; 1.8 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 866 KiB/s wr, 19 op/s 2024-06-03T03:27:13.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:27:13 smithi183 ceph-mon[37326]: pgmap v563: 129 pgs: 129 active+clean; 1.8 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 866 KiB/s wr, 19 op/s 2024-06-03T03:27:14.813 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.813+0000 7f5952c00640 1 -- 172.21.15.117:0/1692422822 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f594c1075a0 msgr2=0x7f594c109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:14.813 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.813+0000 7f5952c00640 1 --2- 172.21.15.117:0/1692422822 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f594c1075a0 0x7f594c109990 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5944009900 tx=0x7f594402a3f0 comp rx=0 tx=0).stop 2024-06-03T03:27:14.813 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.813+0000 7f5952c00640 1 -- 172.21.15.117:0/1692422822 shutdown_connections 2024-06-03T03:27:14.814 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.813+0000 7f5952c00640 1 --2- 172.21.15.117:0/1692422822 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f594c1075a0 0x7f594c109990 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:14.814 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.813+0000 7f5952c00640 1 --2- 172.21.15.117:0/1692422822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f594c104c70 0x7f594c107060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:14.814 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.813+0000 7f5952c00640 1 --2- 172.21.15.117:0/1692422822 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f594c102340 0x7f594c104730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:14.814 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.813+0000 7f5952c00640 1 -- 172.21.15.117:0/1692422822 >> 172.21.15.117:0/1692422822 conn(0x7f594c0fbc90 msgr2=0x7f594c0fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:27:14.814 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.813+0000 7f5952c00640 1 -- 172.21.15.117:0/1692422822 shutdown_connections 2024-06-03T03:27:14.814 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.813+0000 7f5952c00640 1 -- 172.21.15.117:0/1692422822 wait complete. 2024-06-03T03:27:14.815 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.814+0000 7f5952c00640 1 Processor -- start 2024-06-03T03:27:14.815 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.814+0000 7f5952c00640 1 -- start start 2024-06-03T03:27:14.819 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.815+0000 7f5952c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f594c102340 0x7f594c101cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:14.819 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.815+0000 7f5952c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f594c104c70 0x7f594c1002f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:14.819 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.815+0000 7f5952c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f594c1075a0 0x7f594c100830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:14.819 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.815+0000 7f5952c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f594c10d510 con 0x7f594c1075a0 2024-06-03T03:27:14.819 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.815+0000 7f5952c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f594c10d390 con 0x7f594c104c70 2024-06-03T03:27:14.819 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.815+0000 7f5952c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f594c10d690 con 0x7f594c102340 2024-06-03T03:27:14.819 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.815+0000 7f5952200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f594c1075a0 0x7f594c100830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:14.819 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.815+0000 7f5950e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f594c104c70 0x7f594c1002f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:14.819 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.815+0000 7f5952200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f594c1075a0 0x7f594c100830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40800/0 (socket says 172.21.15.117:40800) 2024-06-03T03:27:14.819 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.815+0000 7f5952200640 1 -- 172.21.15.117:0/2472793382 learned_addr learned my addr 172.21.15.117:0/2472793382 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:27:14.819 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.815+0000 7f5951800640 1 --2- 172.21.15.117:0/2472793382 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f594c102340 0x7f594c101cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:14.819 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.815+0000 7f5950e00640 1 -- 172.21.15.117:0/2472793382 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f594c102340 msgr2=0x7f594c101cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:14.819 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.815+0000 7f5950e00640 1 --2- 172.21.15.117:0/2472793382 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f594c102340 0x7f594c101cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:14.820 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.815+0000 7f5950e00640 1 -- 172.21.15.117:0/2472793382 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f594c1075a0 msgr2=0x7f594c100830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:14.820 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.815+0000 7f5950e00640 1 --2- 172.21.15.117:0/2472793382 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f594c1075a0 0x7f594c100830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:14.820 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.815+0000 7f5950e00640 1 -- 172.21.15.117:0/2472793382 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f594c1010c0 con 0x7f594c104c70 2024-06-03T03:27:14.820 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.816+0000 7f5950e00640 1 --2- 172.21.15.117:0/2472793382 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f594c104c70 0x7f594c1002f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f593800e9c0 tx=0x7f593800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:27:14.820 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.816+0000 7f594a000640 1 -- 172.21.15.117:0/2472793382 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f593800ccc0 con 0x7f594c104c70 2024-06-03T03:27:14.820 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.816+0000 7f5952c00640 1 -- 172.21.15.117:0/2472793382 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f594c1a1c10 con 0x7f594c104c70 2024-06-03T03:27:14.820 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.816+0000 7f594a000640 1 -- 172.21.15.117:0/2472793382 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5938010910 con 0x7f594c104c70 2024-06-03T03:27:14.820 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.816+0000 7f594a000640 1 -- 172.21.15.117:0/2472793382 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5938018880 con 0x7f594c104c70 2024-06-03T03:27:14.820 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.816+0000 7f5952c00640 1 -- 172.21.15.117:0/2472793382 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f594c1a21b0 con 0x7f594c104c70 2024-06-03T03:27:14.822 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.820+0000 7f5957185740 1 Processor -- start 2024-06-03T03:27:14.822 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.820+0000 7f5957185740 1 -- start start 2024-06-03T03:27:14.822 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.821+0000 7f5957185740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1f88f0d90 0x55d1f88f11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:14.822 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.821+0000 7f5957185740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1f88f1780 0x55d1f88f6280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:14.822 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.821+0000 7f5957185740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1f88f68f0 0x55d1f88f8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:14.823 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.821+0000 7f5957185740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d1f8854420 con 0x55d1f88f68f0 2024-06-03T03:27:14.823 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.821+0000 7f5957185740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d1f87fbc50 con 0x55d1f88f0d90 2024-06-03T03:27:14.823 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.821+0000 7f5957185740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d1f88503e0 con 0x55d1f88f1780 2024-06-03T03:27:14.823 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.821+0000 7f5952200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1f88f0d90 0x55d1f88f11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:14.823 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.821+0000 7f5952200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1f88f0d90 0x55d1f88f11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:41424/0 (socket says 172.21.15.117:41424) 2024-06-03T03:27:14.823 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.821+0000 7f5952200640 1 -- 172.21.15.117:0/3806149183 learned_addr learned my addr 172.21.15.117:0/3806149183 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:27:14.823 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.821+0000 7f5951800640 1 --2- 172.21.15.117:0/3806149183 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1f88f1780 0x55d1f88f6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:14.823 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.821+0000 7f5952200640 1 -- 172.21.15.117:0/3806149183 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1f88f1780 msgr2=0x55d1f88f6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:14.823 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.821+0000 7f5952200640 1 --2- 172.21.15.117:0/3806149183 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1f88f1780 0x55d1f88f6280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:14.824 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.821+0000 7f594a000640 1 -- 172.21.15.117:0/2472793382 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f5938010ac0 con 0x7f594c104c70 2024-06-03T03:27:14.824 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.821+0000 7f5952200640 1 -- 172.21.15.117:0/3806149183 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1f88f68f0 msgr2=0x55d1f88f8d80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-03T03:27:14.824 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.821+0000 7f5952200640 1 --2- 172.21.15.117:0/3806149183 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1f88f68f0 0x55d1f88f8d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:14.824 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.821+0000 7f5952200640 1 -- 172.21.15.117:0/3806149183 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1f8568210 con 0x55d1f88f0d90 2024-06-03T03:27:14.824 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.822+0000 7f5952200640 1 --2- 172.21.15.117:0/3806149183 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1f88f0d90 0x55d1f88f11b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5944005ea0 tx=0x7f5944005b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57790a78b95a524d server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:27:14.824 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.822+0000 7f5942a00640 1 -- 172.21.15.117:0/3806149183 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5944040070 con 0x55d1f88f0d90 2024-06-03T03:27:14.824 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.822+0000 7f5942a00640 1 -- 172.21.15.117:0/3806149183 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5944004400 con 0x55d1f88f0d90 2024-06-03T03:27:14.824 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.822+0000 7f5942a00640 1 -- 172.21.15.117:0/3806149183 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5944035040 con 0x55d1f88f0d90 2024-06-03T03:27:14.829 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.823+0000 7f594a000640 1 --2- 172.21.15.117:0/2472793382 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f59300a2c00 0x7f59300a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:14.830 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.823+0000 7f5957185740 1 -- 172.21.15.117:0/3806149183 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1f88f0d90 msgr2=0x55d1f88f11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:14.830 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.823+0000 7f5957185740 1 --2- 172.21.15.117:0/3806149183 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1f88f0d90 0x55d1f88f11b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5944005ea0 tx=0x7f5944005b50 comp rx=0 tx=0).stop 2024-06-03T03:27:14.830 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.823+0000 7f594a000640 1 -- 172.21.15.117:0/2472793382 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f59380cf690 con 0x7f594c104c70 2024-06-03T03:27:14.830 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.823+0000 7f5957185740 1 -- 172.21.15.117:0/3806149183 shutdown_connections 2024-06-03T03:27:14.830 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.823+0000 7f5957185740 1 --2- 172.21.15.117:0/3806149183 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1f88f68f0 0x55d1f88f8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:14.830 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.823+0000 7f5957185740 1 --2- 172.21.15.117:0/3806149183 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1f88f1780 0x55d1f88f6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:14.830 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.823+0000 7f5957185740 1 --2- 172.21.15.117:0/3806149183 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1f88f0d90 0x55d1f88f11b0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:14.830 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.823+0000 7f5957185740 1 -- 172.21.15.117:0/3806149183 >> 172.21.15.117:0/3806149183 conn(0x55d1f88b8f80 msgr2=0x55d1f88ec810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:27:14.831 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.823+0000 7f5957185740 1 -- 172.21.15.117:0/3806149183 shutdown_connections 2024-06-03T03:27:14.831 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.823+0000 7f5951800640 1 --2- 172.21.15.117:0/2472793382 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f59300a2c00 0x7f59300a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:14.831 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.823+0000 7f5957185740 1 -- 172.21.15.117:0/3806149183 wait complete. 2024-06-03T03:27:14.831 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.824+0000 7f5957185740 1 -- start start 2024-06-03T03:27:14.831 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.824+0000 7f5951800640 1 --2- 172.21.15.117:0/2472793382 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f59300a2c00 0x7f59300a50c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x55d1f88ed6c0 tx=0x7f593c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:27:14.831 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.824+0000 7f5957185740 1 Processor -- start 2024-06-03T03:27:14.831 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.824+0000 7f5957185740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1f88f0d90 0x55d1f88fb120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:14.831 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.824+0000 7f5957185740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1f88f1780 0x55d1f88fb660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:14.831 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.824+0000 7f5957185740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1f88f68f0 0x55d1f89026a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:14.831 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.824+0000 7f5957185740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d1f88f9840 con 0x55d1f88f0d90 2024-06-03T03:27:14.831 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.824+0000 7f5957185740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d1f880f0b0 con 0x55d1f88f1780 2024-06-03T03:27:14.831 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.824+0000 7f5957185740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d1f88f99c0 con 0x55d1f88f68f0 2024-06-03T03:27:14.832 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.824+0000 7f5957185740 10 client.0 _ll_register_callbacks cb 0x55d1f87faf40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:27:14.832 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.825+0000 7f5951800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1f88f0d90 0x55d1f88fb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:14.832 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.825+0000 7f5951800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1f88f0d90 0x55d1f88fb120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40816/0 (socket says 172.21.15.117:40816) 2024-06-03T03:27:14.832 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.825+0000 7f5951800640 1 -- 172.21.15.117:0/1403149345 learned_addr learned my addr 172.21.15.117:0/1403149345 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:27:14.832 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.825+0000 7f5952200640 1 --2- 172.21.15.117:0/1403149345 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1f88f68f0 0x55d1f89026a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:14.832 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.825+0000 7f5950e00640 1 --2- 172.21.15.117:0/1403149345 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1f88f1780 0x55d1f88fb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:14.832 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.825+0000 7f5951800640 1 -- 172.21.15.117:0/1403149345 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1f88f68f0 msgr2=0x55d1f89026a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:14.832 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.825+0000 7f5951800640 1 --2- 172.21.15.117:0/1403149345 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1f88f68f0 0x55d1f89026a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:14.832 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.825+0000 7f5951800640 1 -- 172.21.15.117:0/1403149345 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1f88f1780 msgr2=0x55d1f88fb660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:14.833 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.825+0000 7f5951800640 1 --2- 172.21.15.117:0/1403149345 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1f88f1780 0x55d1f88fb660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:14.833 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.825+0000 7f5951800640 1 -- 172.21.15.117:0/1403149345 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1f8634620 con 0x55d1f88f0d90 2024-06-03T03:27:14.833 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.825+0000 7f5952200640 1 --2- 172.21.15.117:0/1403149345 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1f88f68f0 0x55d1f89026a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:27:14.833 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.825+0000 7f5950e00640 1 --2- 172.21.15.117:0/1403149345 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1f88f1780 0x55d1f88fb660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:27:14.833 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.825+0000 7f5951800640 1 --2- 172.21.15.117:0/1403149345 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1f88f0d90 0x55d1f88fb120 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f593c003f90 tx=0x7f593c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7e77fd3ff5728b3 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:27:14.833 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.825+0000 7f5937400640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-06-03T03:27:14.833 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.826+0000 7f5937400640 1 -- 172.21.15.117:0/1403149345 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f593c02c060 con 0x55d1f88f0d90 2024-06-03T03:27:14.833 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.826+0000 7f5937400640 1 -- 172.21.15.117:0/1403149345 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f593c01d430 con 0x55d1f88f0d90 2024-06-03T03:27:14.833 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.826+0000 7f5957185740 1 -- 172.21.15.117:0/1403149345 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5944004400 con 0x55d1f88f0d90 2024-06-03T03:27:14.833 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.826+0000 7f5937400640 1 -- 172.21.15.117:0/1403149345 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f593c033050 con 0x55d1f88f0d90 2024-06-03T03:27:14.833 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.826+0000 7f5957185740 10 client.15546 fetch_fsmap learned FSMap version 31 2024-06-03T03:27:14.833 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.826+0000 7f5957185740 1 -- 172.21.15.117:0/1403149345 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d1f8905ae0 con 0x55d1f88f0d90 2024-06-03T03:27:14.833 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.826+0000 7f5937400640 1 -- 172.21.15.117:0/1403149345 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5944004400 con 0x55d1f88f0d90 2024-06-03T03:27:14.834 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.826+0000 7f5937400640 1 -- 172.21.15.117:0/1403149345 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f593c026030 con 0x55d1f88f0d90 2024-06-03T03:27:14.834 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.827+0000 7f5957185740 10 client.15546 fetch_fsmap finished waiting for FSMap version 31 2024-06-03T03:27:14.834 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.827+0000 7f5957185740 10 client.15546 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-03T03:27:14.834 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.827+0000 7f5957185740 20 client.15546 populate_metadata read hostname 'smithi117' 2024-06-03T03:27:14.834 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.827+0000 7f5957185740 1 --2- 172.21.15.117:0/1403149345 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55d1f89071b0 0x55d1f8909620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:14.834 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.827+0000 7f5957185740 4 client.15546 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T03:27:14.834 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.827+0000 7f5957185740 1 -- 172.21.15.117:0/1403149345 --> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1f8861b50 con 0x55d1f89071b0 2024-06-03T03:27:14.834 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.827+0000 7f5950e00640 1 --2- 172.21.15.117:0/1403149345 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55d1f89071b0 0x55d1f8909620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:14.834 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.828+0000 7f5950e00640 1 --2- 172.21.15.117:0/1403149345 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55d1f89071b0 0x55d1f8909620 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d219ebc4ff275df2 server_cookie=615bbe7516fc98ee in_seq=0 out_seq=0 2024-06-03T03:27:14.834 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.828+0000 7f5937400640 10 client.15546 ms_handle_connect on v2:172.21.15.183:6838/1439615083 2024-06-03T03:27:14.834 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.828+0000 7f5937400640 1 -- 172.21.15.117:0/1403149345 <== mds.0 v2:172.21.15.183:6838/1439615083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d1f8861b50 con 0x55d1f89071b0 2024-06-03T03:27:14.834 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.828+0000 7f5937400640 10 client.15546 handle_command_reply: tid=0 2024-06-03T03:27:14.834 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.828+0000 7f5957185740 1 client.15546 shutdown 2024-06-03T03:27:14.835 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.829+0000 7f5957185740 1 -- 172.21.15.117:0/1403149345 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1f88f0d90 msgr2=0x55d1f88fb120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:14.835 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.829+0000 7f5957185740 1 --2- 172.21.15.117:0/1403149345 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1f88f0d90 0x55d1f88fb120 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f593c003f90 tx=0x7f593c004ab0 comp rx=0 tx=0).stop 2024-06-03T03:27:14.835 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.829+0000 7f5957185740 1 -- 172.21.15.117:0/1403149345 shutdown_connections 2024-06-03T03:27:14.835 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.829+0000 7f5957185740 1 --2- 172.21.15.117:0/1403149345 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55d1f89071b0 0x55d1f8909620 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:14.835 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.829+0000 7f5957185740 1 --2- 172.21.15.117:0/1403149345 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1f88f68f0 0x55d1f89026a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:14.835 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.829+0000 7f5957185740 1 --2- 172.21.15.117:0/1403149345 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1f88f1780 0x55d1f88fb660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:14.835 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.829+0000 7f5937400640 0 client.15546 ms_handle_reset on v2:172.21.15.183:6838/1439615083 2024-06-03T03:27:14.835 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.829+0000 7f5957185740 1 --2- 172.21.15.117:0/1403149345 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1f88f0d90 0x55d1f88fb120 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:14.836 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.829+0000 7f5957185740 1 -- 172.21.15.117:0/1403149345 >> 172.21.15.117:0/1403149345 conn(0x55d1f88b8f80 msgr2=0x55d1f88eb970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:27:14.836 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.829+0000 7f5957185740 1 -- 172.21.15.117:0/1403149345 shutdown_connections 2024-06-03T03:27:14.836 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.829+0000 7f5957185740 1 -- 172.21.15.117:0/1403149345 wait complete. 2024-06-03T03:27:14.836 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.829+0000 7f5957185740 20 client.15546 trim_cache size 0 max 16384 2024-06-03T03:27:14.847 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.846+0000 7f5957185740 1 Processor -- start 2024-06-03T03:27:14.847 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.846+0000 7f5957185740 1 -- start start 2024-06-03T03:27:14.847 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.846+0000 7f5957185740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1f88f0d90 0x55d1f88ec0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:14.848 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.846+0000 7f5957185740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1f88f1780 0x55d1f8902660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:14.848 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.846+0000 7f5957185740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1f88f68f0 0x55d1f88fb440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:14.848 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.846+0000 7f5957185740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f594c1ac6a0 con 0x55d1f88f1780 2024-06-03T03:27:14.848 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.846+0000 7f5957185740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f594c1ae1e0 con 0x55d1f88f68f0 2024-06-03T03:27:14.848 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.846+0000 7f5957185740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f594c1acd70 con 0x55d1f88f0d90 2024-06-03T03:27:14.848 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.846+0000 7f5950e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1f88f1780 0x55d1f8902660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:14.848 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.846+0000 7f5950e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1f88f1780 0x55d1f8902660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40822/0 (socket says 172.21.15.117:40822) 2024-06-03T03:27:14.848 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.846+0000 7f5950e00640 1 -- 172.21.15.117:0/1395131797 learned_addr learned my addr 172.21.15.117:0/1395131797 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:27:14.849 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.847+0000 7f5951800640 1 --2- 172.21.15.117:0/1395131797 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1f88f0d90 0x55d1f88ec0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:14.849 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.847+0000 7f5952200640 1 --2- 172.21.15.117:0/1395131797 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1f88f68f0 0x55d1f88fb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:14.849 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.847+0000 7f5950e00640 1 -- 172.21.15.117:0/1395131797 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1f88f0d90 msgr2=0x55d1f88ec0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:14.849 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.847+0000 7f5950e00640 1 --2- 172.21.15.117:0/1395131797 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1f88f0d90 0x55d1f88ec0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:14.849 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.847+0000 7f5950e00640 1 -- 172.21.15.117:0/1395131797 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1f88f68f0 msgr2=0x55d1f88fb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:14.849 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.847+0000 7f5950e00640 1 --2- 172.21.15.117:0/1395131797 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1f88f68f0 0x55d1f88fb440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:14.849 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.847+0000 7f5950e00640 1 -- 172.21.15.117:0/1395131797 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1f88ed4c0 con 0x55d1f88f1780 2024-06-03T03:27:14.854 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.847+0000 7f5950e00640 1 --2- 172.21.15.117:0/1395131797 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1f88f1780 0x55d1f8902660 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f593809bba0 tx=0x7f593800efb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3c725b17fde131e server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:27:14.855 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.847+0000 7f5942000640 1 -- 172.21.15.117:0/1395131797 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5938099f10 con 0x55d1f88f1780 2024-06-03T03:27:14.855 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.847+0000 7f5942000640 1 -- 172.21.15.117:0/1395131797 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59380cf3a0 con 0x55d1f88f1780 2024-06-03T03:27:14.855 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.848+0000 7f5942000640 1 -- 172.21.15.117:0/1395131797 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f593809aad0 con 0x55d1f88f1780 2024-06-03T03:27:14.855 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.848+0000 7f5957185740 1 -- 172.21.15.117:0/1395131797 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1f88f1780 msgr2=0x55d1f8902660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:14.855 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.848+0000 7f5957185740 1 --2- 172.21.15.117:0/1395131797 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1f88f1780 0x55d1f8902660 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f593809bba0 tx=0x7f593800efb0 comp rx=0 tx=0).stop 2024-06-03T03:27:14.855 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.848+0000 7f5957185740 1 -- 172.21.15.117:0/1395131797 shutdown_connections 2024-06-03T03:27:14.856 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.848+0000 7f5957185740 1 --2- 172.21.15.117:0/1395131797 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1f88f68f0 0x55d1f88fb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:14.856 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.848+0000 7f5957185740 1 --2- 172.21.15.117:0/1395131797 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1f88f1780 0x55d1f8902660 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:14.856 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.848+0000 7f5957185740 1 --2- 172.21.15.117:0/1395131797 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1f88f0d90 0x55d1f88ec0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:14.856 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.848+0000 7f5957185740 1 -- 172.21.15.117:0/1395131797 >> 172.21.15.117:0/1395131797 conn(0x55d1f88b8f80 msgr2=0x55d1f88fad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:27:14.856 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.848+0000 7f5957185740 1 -- 172.21.15.117:0/1395131797 shutdown_connections 2024-06-03T03:27:14.856 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.848+0000 7f5957185740 1 -- 172.21.15.117:0/1395131797 wait complete. 2024-06-03T03:27:14.856 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.849+0000 7f5957185740 1 -- start start 2024-06-03T03:27:14.857 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.849+0000 7f5957185740 1 Processor -- start 2024-06-03T03:27:14.857 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.849+0000 7f5957185740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1f88f0d90 0x55d1f88ff3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:14.857 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.849+0000 7f5957185740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1f88f1780 0x55d1f88ff910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:14.857 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.849+0000 7f5957185740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1f88f68f0 0x55d1f896fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:14.857 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.849+0000 7f5957185740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f594c1adb10 con 0x55d1f88f68f0 2024-06-03T03:27:14.857 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.849+0000 7f5957185740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f594c1ad440 con 0x55d1f88f1780 2024-06-03T03:27:14.857 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.849+0000 7f5957185740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f594c1aa680 con 0x55d1f88f0d90 2024-06-03T03:27:14.857 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.849+0000 7f5957185740 10 client.0 _ll_register_callbacks cb 0x7f593c012ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:27:14.857 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.850+0000 7f5952200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1f88f68f0 0x55d1f896fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:14.857 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.850+0000 7f5952200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1f88f68f0 0x55d1f896fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40830/0 (socket says 172.21.15.117:40830) 2024-06-03T03:27:14.858 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.850+0000 7f5952200640 1 -- 172.21.15.117:0/2622325299 learned_addr learned my addr 172.21.15.117:0/2622325299 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:27:14.858 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.850+0000 7f5951800640 1 --2- 172.21.15.117:0/2622325299 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1f88f0d90 0x55d1f88ff3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:14.858 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.850+0000 7f5950e00640 1 --2- 172.21.15.117:0/2622325299 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1f88f1780 0x55d1f88ff910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:14.858 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.850+0000 7f5952200640 1 -- 172.21.15.117:0/2622325299 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1f88f0d90 msgr2=0x55d1f88ff3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:14.858 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.850+0000 7f5952200640 1 --2- 172.21.15.117:0/2622325299 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1f88f0d90 0x55d1f88ff3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:14.858 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.850+0000 7f5952200640 1 -- 172.21.15.117:0/2622325299 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1f88f1780 msgr2=0x55d1f88ff910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:14.858 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.850+0000 7f5952200640 1 --2- 172.21.15.117:0/2622325299 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1f88f1780 0x55d1f88ff910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:14.858 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.850+0000 7f5952200640 1 -- 172.21.15.117:0/2622325299 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1f8970430 con 0x55d1f88f68f0 2024-06-03T03:27:14.858 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.850+0000 7f5950e00640 1 --2- 172.21.15.117:0/2622325299 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1f88f1780 0x55d1f88ff910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T03:27:14.858 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.850+0000 7f5952200640 1 --2- 172.21.15.117:0/2622325299 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1f88f68f0 0x55d1f896fef0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f5944005ea0 tx=0x7f594402a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fbca64935a21d24 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:27:14.858 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.850+0000 7f5937400640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-06-03T03:27:14.859 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.851+0000 7f5937400640 1 -- 172.21.15.117:0/2622325299 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5944054060 con 0x55d1f88f68f0 2024-06-03T03:27:14.859 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.851+0000 7f5937400640 1 -- 172.21.15.117:0/2622325299 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f59440037f0 con 0x55d1f88f68f0 2024-06-03T03:27:14.859 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.851+0000 7f5957185740 1 -- 172.21.15.117:0/2622325299 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59380cf3a0 con 0x55d1f88f68f0 2024-06-03T03:27:14.859 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.851+0000 7f5937400640 1 -- 172.21.15.117:0/2622325299 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f594405b050 con 0x55d1f88f68f0 2024-06-03T03:27:14.859 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.851+0000 7f5957185740 10 client.15552 fetch_fsmap learned FSMap version 31 2024-06-03T03:27:14.859 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.851+0000 7f5957185740 1 -- 172.21.15.117:0/2622325299 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d1f8971570 con 0x55d1f88f68f0 2024-06-03T03:27:14.859 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.851+0000 7f5937400640 1 -- 172.21.15.117:0/2622325299 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59380cf3a0 con 0x55d1f88f68f0 2024-06-03T03:27:14.859 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.851+0000 7f5937400640 1 -- 172.21.15.117:0/2622325299 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f594403c940 con 0x55d1f88f68f0 2024-06-03T03:27:14.859 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.851+0000 7f5957185740 10 client.15552 fetch_fsmap finished waiting for FSMap version 31 2024-06-03T03:27:14.859 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.851+0000 7f5957185740 10 client.15552 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-03T03:27:14.860 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.851+0000 7f5957185740 20 client.15552 populate_metadata read hostname 'smithi117' 2024-06-03T03:27:14.860 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.851+0000 7f5957185740 1 --2- 172.21.15.117:0/2622325299 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55d1f8971b30 0x55d1f8973f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:14.860 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.851+0000 7f5957185740 4 client.15552 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-03T03:27:14.860 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.851+0000 7f5957185740 1 -- 172.21.15.117:0/2622325299 --> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d1f885bea0 con 0x55d1f8971b30 2024-06-03T03:27:14.860 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.852+0000 7f5951800640 1 --2- 172.21.15.117:0/2622325299 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55d1f8971b30 0x55d1f8973f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:14.860 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.852+0000 7f5951800640 1 --2- 172.21.15.117:0/2622325299 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55d1f8971b30 0x55d1f8973f20 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a506b2402132385 server_cookie=ebb9c0c131583eb8 in_seq=0 out_seq=0 2024-06-03T03:27:14.860 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.852+0000 7f5937400640 10 client.15552 ms_handle_connect on v2:172.21.15.183:6838/1439615083 2024-06-03T03:27:14.864 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.863+0000 7f5937400640 1 -- 172.21.15.117:0/2622325299 <== mds.0 v2:172.21.15.183:6838/1439615083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55d1f885bea0 con 0x55d1f8971b30 2024-06-03T03:27:14.864 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.863+0000 7f5937400640 10 client.15552 handle_command_reply: tid=0 2024-06-03T03:27:14.864 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.863+0000 7f5957185740 1 client.15552 shutdown 2024-06-03T03:27:14.864 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.863+0000 7f5957185740 1 -- 172.21.15.117:0/2622325299 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1f88f68f0 msgr2=0x55d1f896fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:14.864 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.863+0000 7f5957185740 1 --2- 172.21.15.117:0/2622325299 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1f88f68f0 0x55d1f896fef0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f5944005ea0 tx=0x7f594402a3f0 comp rx=0 tx=0).stop 2024-06-03T03:27:14.864 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.863+0000 7f5957185740 1 -- 172.21.15.117:0/2622325299 shutdown_connections 2024-06-03T03:27:14.864 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.863+0000 7f5957185740 1 --2- 172.21.15.117:0/2622325299 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55d1f8971b30 0x55d1f8973f20 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:14.864 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.863+0000 7f5957185740 1 --2- 172.21.15.117:0/2622325299 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1f88f68f0 0x55d1f896fef0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:14.864 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.863+0000 7f5957185740 1 --2- 172.21.15.117:0/2622325299 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1f88f1780 0x55d1f88ff910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:14.864 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.863+0000 7f5937400640 0 client.15552 ms_handle_reset on v2:172.21.15.183:6838/1439615083 2024-06-03T03:27:14.864 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.863+0000 7f5957185740 1 --2- 172.21.15.117:0/2622325299 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1f88f0d90 0x55d1f88ff3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:14.864 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.863+0000 7f5957185740 1 -- 172.21.15.117:0/2622325299 >> 172.21.15.117:0/2622325299 conn(0x55d1f88b8f80 msgr2=0x55d1f88ef700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:27:14.865 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.863+0000 7f5957185740 1 -- 172.21.15.117:0/2622325299 shutdown_connections 2024-06-03T03:27:14.865 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.863+0000 7f5957185740 1 -- 172.21.15.117:0/2622325299 wait complete. 2024-06-03T03:27:14.865 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.863+0000 7f5957185740 20 client.15552 trim_cache size 0 max 16384 2024-06-03T03:27:14.865 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:{ 2024-06-03T03:27:14.865 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "return_code": 0 2024-06-03T03:27:14.865 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:} 2024-06-03T03:27:14.866 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.864+0000 7f5952c00640 1 -- 172.21.15.117:0/2472793382 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f59300a2c00 msgr2=0x7f59300a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:14.866 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.864+0000 7f5952c00640 1 --2- 172.21.15.117:0/2472793382 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f59300a2c00 0x7f59300a50c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x55d1f88ed6c0 tx=0x7f593c009500 comp rx=0 tx=0).stop 2024-06-03T03:27:14.866 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.864+0000 7f5952c00640 1 -- 172.21.15.117:0/2472793382 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f594c104c70 msgr2=0x7f594c1002f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:14.866 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.864+0000 7f5952c00640 1 --2- 172.21.15.117:0/2472793382 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f594c104c70 0x7f594c1002f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f593800e9c0 tx=0x7f593800ee80 comp rx=0 tx=0).stop 2024-06-03T03:27:14.866 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.864+0000 7f5952c00640 1 -- 172.21.15.117:0/2472793382 shutdown_connections 2024-06-03T03:27:14.866 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.864+0000 7f5952c00640 1 --2- 172.21.15.117:0/2472793382 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f594c1075a0 0x7f594c100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:14.866 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.864+0000 7f5952c00640 1 --2- 172.21.15.117:0/2472793382 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f59300a2c00 0x7f59300a50c0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:14.866 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.864+0000 7f5952c00640 1 --2- 172.21.15.117:0/2472793382 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f594c104c70 0x7f594c1002f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:14.866 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.864+0000 7f5952c00640 1 --2- 172.21.15.117:0/2472793382 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f594c102340 0x7f594c101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:14.866 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.864+0000 7f5952c00640 1 -- 172.21.15.117:0/2472793382 >> 172.21.15.117:0/2472793382 conn(0x7f594c0fbc90 msgr2=0x7f594c102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:27:14.867 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.864+0000 7f5952c00640 1 -- 172.21.15.117:0/2472793382 shutdown_connections 2024-06-03T03:27:14.867 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:14.864+0000 7f5952c00640 1 -- 172.21.15.117:0/2472793382 wait complete. 2024-06-03T03:27:15.006 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-03T03:27:15.006 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-03T03:27:15.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:27:15 smithi151 ceph-mon[37043]: pgmap v564: 129 pgs: 129 active+clean; 1.8 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1.3 MiB/s wr, 14 op/s 2024-06-03T03:27:15.443 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:27:15.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:27:15 smithi117 ceph-mon[31649]: pgmap v564: 129 pgs: 129 active+clean; 1.8 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1.3 MiB/s wr, 14 op/s 2024-06-03T03:27:15.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:27:15 smithi183 ceph-mon[37326]: pgmap v564: 129 pgs: 129 active+clean; 1.8 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1.3 MiB/s wr, 14 op/s 2024-06-03T03:27:17.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:27:17 smithi151 ceph-mon[37043]: pgmap v565: 129 pgs: 129 active+clean; 1.8 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 859 KiB/s wr, 13 op/s 2024-06-03T03:27:17.427 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:27:17 smithi117 ceph-mon[31649]: pgmap v565: 129 pgs: 129 active+clean; 1.8 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 859 KiB/s wr, 13 op/s 2024-06-03T03:27:17.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:27:17 smithi183 ceph-mon[37326]: pgmap v565: 129 pgs: 129 active+clean; 1.8 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 859 KiB/s wr, 13 op/s 2024-06-03T03:27:18.030 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.027+0000 7ff430000700 1 -- 172.21.15.117:0/2605415105 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff42806ab30 msgr2=0x7ff42810f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:18.030 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.027+0000 7ff430000700 1 --2- 172.21.15.117:0/2605415105 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff42806ab30 0x7ff42810f320 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff42000b0a0 tx=0x7ff42000b3b0 comp rx=0 tx=0).stop 2024-06-03T03:27:18.030 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.028+0000 7ff430000700 1 -- 172.21.15.117:0/2605415105 shutdown_connections 2024-06-03T03:27:18.030 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.028+0000 7ff430000700 1 --2- 172.21.15.117:0/2605415105 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff42806ab30 0x7ff42810f320 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:18.030 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.028+0000 7ff430000700 1 --2- 172.21.15.117:0/2605415105 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff428108500 0x7ff4281088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:18.030 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.028+0000 7ff430000700 1 --2- 172.21.15.117:0/2605415105 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff42806a170 0x7ff42806a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:18.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.028+0000 7ff430000700 1 -- 172.21.15.117:0/2605415105 >> 172.21.15.117:0/2605415105 conn(0x7ff428078340 msgr2=0x7ff428078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:27:18.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.028+0000 7ff430000700 1 -- 172.21.15.117:0/2605415105 shutdown_connections 2024-06-03T03:27:18.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.028+0000 7ff430000700 1 -- 172.21.15.117:0/2605415105 wait complete. 2024-06-03T03:27:18.032 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.029+0000 7ff430000700 1 Processor -- start 2024-06-03T03:27:18.032 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.029+0000 7ff430000700 1 -- start start 2024-06-03T03:27:18.033 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.029+0000 7ff430000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff42806a170 0x7ff42819e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:18.033 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.029+0000 7ff430000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff42806ab30 0x7ff42819eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:18.033 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.029+0000 7ff430000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff428108500 0x7ff428198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:18.033 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.029+0000 7ff430000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff4281127d0 con 0x7ff428108500 2024-06-03T03:27:18.033 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.029+0000 7ff430000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff428112650 con 0x7ff42806a170 2024-06-03T03:27:18.033 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.029+0000 7ff430000700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff428112950 con 0x7ff42806ab30 2024-06-03T03:27:18.033 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.030+0000 7ff42e000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff428108500 0x7ff428198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:18.033 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.030+0000 7ff42d600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff42806a170 0x7ff42819e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:18.033 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.030+0000 7ff42cc00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff42806ab30 0x7ff42819eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:18.033 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.030+0000 7ff42d600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff42806a170 0x7ff42819e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:56758/0 (socket says 172.21.15.117:56758) 2024-06-03T03:27:18.033 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.030+0000 7ff42e000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff428108500 0x7ff428198a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53984/0 (socket says 172.21.15.117:53984) 2024-06-03T03:27:18.033 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.030+0000 7ff42d600700 1 -- 172.21.15.117:0/1140853896 learned_addr learned my addr 172.21.15.117:0/1140853896 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:27:18.034 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.030+0000 7ff42d600700 1 -- 172.21.15.117:0/1140853896 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff42806ab30 msgr2=0x7ff42819eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:18.034 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.030+0000 7ff42d600700 1 --2- 172.21.15.117:0/1140853896 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff42806ab30 0x7ff42819eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:18.034 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.030+0000 7ff42d600700 1 -- 172.21.15.117:0/1140853896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff428108500 msgr2=0x7ff428198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:18.034 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.030+0000 7ff42d600700 1 --2- 172.21.15.117:0/1140853896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff428108500 0x7ff428198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:18.034 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.030+0000 7ff42d600700 1 -- 172.21.15.117:0/1140853896 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4281991d0 con 0x7ff42806a170 2024-06-03T03:27:18.034 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.030+0000 7ff42e000700 1 --2- 172.21.15.117:0/1140853896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff428108500 0x7ff428198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:27:18.034 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.031+0000 7ff42d600700 1 --2- 172.21.15.117:0/1140853896 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff42806a170 0x7ff42819e860 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff41c00e9c0 tx=0x7ff41c00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:27:18.034 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.031+0000 7ff426000700 1 -- 172.21.15.117:0/1140853896 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff41c00cbd0 con 0x7ff42806a170 2024-06-03T03:27:18.034 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.031+0000 7ff430000700 1 -- 172.21.15.117:0/1140853896 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4281994c0 con 0x7ff42806a170 2024-06-03T03:27:18.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.031+0000 7ff426000700 1 -- 172.21.15.117:0/1140853896 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff41c00ee60 con 0x7ff42806a170 2024-06-03T03:27:18.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.031+0000 7ff426000700 1 -- 172.21.15.117:0/1140853896 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff41c005630 con 0x7ff42806a170 2024-06-03T03:27:18.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.031+0000 7ff430000700 1 -- 172.21.15.117:0/1140853896 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4281a5730 con 0x7ff42806a170 2024-06-03T03:27:18.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.034+0000 7ff426000700 1 -- 172.21.15.117:0/1140853896 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7ff41c004750 con 0x7ff42806a170 2024-06-03T03:27:18.036 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.035+0000 7ff43a92fb80 1 Processor -- start 2024-06-03T03:27:18.036 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.035+0000 7ff43a92fb80 1 -- start start 2024-06-03T03:27:18.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.036+0000 7ff426000700 1 --2- 172.21.15.117:0/1140853896 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7ff4100a2c20 0x7ff4100a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:18.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.036+0000 7ff43a92fb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2fee36ec0 0x55e2feeba0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:18.038 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.036+0000 7ff43a92fb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e2feeba630 0x55e2feebaab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:18.038 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.036+0000 7ff43a92fb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e2feec9790 0x55e2feecbb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:18.038 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.036+0000 7ff43a92fb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e2fedc9a90 con 0x55e2feec9790 2024-06-03T03:27:18.038 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.036+0000 7ff426000700 1 -- 172.21.15.117:0/1140853896 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7ff41c0cf610 con 0x7ff42806a170 2024-06-03T03:27:18.038 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.036+0000 7ff43a92fb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e2fee0cb80 con 0x55e2feeba630 2024-06-03T03:27:18.038 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.036+0000 7ff43a92fb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e2fedc7970 con 0x55e2fee36ec0 2024-06-03T03:27:18.038 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.036+0000 7ff42e000700 1 --2- 172.21.15.117:0/1140853896 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7ff4100a2c20 0x7ff4100a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:18.038 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.036+0000 7ff42cc00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e2feeba630 0x55e2feebaab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:18.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.036+0000 7ff42d600700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2fee36ec0 0x55e2feeba0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:18.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.036+0000 7ff42cc00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e2feeba630 0x55e2feebaab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:56766/0 (socket says 172.21.15.117:56766) 2024-06-03T03:27:18.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.036+0000 7ff42d600700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2fee36ec0 0x55e2feeba0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:43220/0 (socket says 172.21.15.117:43220) 2024-06-03T03:27:18.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.036+0000 7ff42cc00700 1 -- 172.21.15.117:0/2632413615 learned_addr learned my addr 172.21.15.117:0/2632413615 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:27:18.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.036+0000 7ff42e000700 1 --2- 172.21.15.117:0/2632413615 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e2feec9790 0x55e2feecbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:18.041 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.037+0000 7ff42cc00700 1 -- 172.21.15.117:0/2632413615 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2fee36ec0 msgr2=0x55e2feeba0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:18.041 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.037+0000 7ff42cc00700 1 --2- 172.21.15.117:0/2632413615 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2fee36ec0 0x55e2feeba0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:18.041 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.037+0000 7ff42cc00700 1 -- 172.21.15.117:0/2632413615 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e2feec9790 msgr2=0x55e2feecbb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:18.041 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.037+0000 7ff42cc00700 1 --2- 172.21.15.117:0/2632413615 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e2feec9790 0x55e2feecbb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:18.041 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.037+0000 7ff42cc00700 1 -- 172.21.15.117:0/2632413615 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2fedb5010 con 0x55e2feeba630 2024-06-03T03:27:18.041 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.037+0000 7ff42d600700 1 --2- 172.21.15.117:0/2632413615 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2fee36ec0 0x55e2feeba0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:27:18.041 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.037+0000 7ff42cc00700 1 --2- 172.21.15.117:0/2632413615 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e2feeba630 0x55e2feebaab0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff414007c90 tx=0x7ff4140103d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8677f1f03fb0a6d server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:27:18.042 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.037+0000 7ff41aa00700 1 -- 172.21.15.117:0/2632413615 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff414010cb0 con 0x55e2feeba630 2024-06-03T03:27:18.042 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.037+0000 7ff41aa00700 1 -- 172.21.15.117:0/2632413615 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff414015070 con 0x55e2feeba630 2024-06-03T03:27:18.042 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.037+0000 7ff41aa00700 1 -- 172.21.15.117:0/2632413615 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff41401a470 con 0x55e2feeba630 2024-06-03T03:27:18.042 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.038+0000 7ff43a92fb80 1 -- 172.21.15.117:0/2632413615 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e2feeba630 msgr2=0x55e2feebaab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:18.042 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.038+0000 7ff42e000700 1 --2- 172.21.15.117:0/1140853896 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7ff4100a2c20 0x7ff4100a50e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff420009850 tx=0x7ff4200097c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:27:18.042 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.038+0000 7ff43a92fb80 1 --2- 172.21.15.117:0/2632413615 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e2feeba630 0x55e2feebaab0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff414007c90 tx=0x7ff4140103d0 comp rx=0 tx=0).stop 2024-06-03T03:27:18.042 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.038+0000 7ff43a92fb80 1 -- 172.21.15.117:0/2632413615 shutdown_connections 2024-06-03T03:27:18.042 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.038+0000 7ff43a92fb80 1 --2- 172.21.15.117:0/2632413615 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e2feec9790 0x55e2feecbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:18.042 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.038+0000 7ff43a92fb80 1 --2- 172.21.15.117:0/2632413615 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2fee36ec0 0x55e2feeba0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:18.042 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.038+0000 7ff43a92fb80 1 --2- 172.21.15.117:0/2632413615 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e2feeba630 0x55e2feebaab0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:18.042 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.038+0000 7ff43a92fb80 1 -- 172.21.15.117:0/2632413615 >> 172.21.15.117:0/2632413615 conn(0x55e2fede71f0 msgr2=0x55e2feeb6530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:27:18.042 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.038+0000 7ff43a92fb80 1 -- 172.21.15.117:0/2632413615 shutdown_connections 2024-06-03T03:27:18.043 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.038+0000 7ff43a92fb80 1 -- 172.21.15.117:0/2632413615 wait complete. 2024-06-03T03:27:18.043 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.039+0000 7ff43a92fb80 1 -- start start 2024-06-03T03:27:18.043 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.039+0000 7ff43a92fb80 1 Processor -- start 2024-06-03T03:27:18.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.040+0000 7ff43a92fb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e2fee36ec0 0x55e2feec8f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:18.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.040+0000 7ff43a92fb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2feeba630 0x55e2feec6200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:18.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.040+0000 7ff43a92fb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e2feec9790 0x55e2feec6740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:18.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.040+0000 7ff43a92fb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e2feda8fe0 con 0x55e2fee36ec0 2024-06-03T03:27:18.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.040+0000 7ff43a92fb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e2fee08b40 con 0x55e2feec9790 2024-06-03T03:27:18.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.040+0000 7ff43a92fb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e2feecc450 con 0x55e2feeba630 2024-06-03T03:27:18.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.040+0000 7ff43a92fb80 10 client.0 _ll_register_callbacks cb 0x55e2feda67a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:27:18.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.040+0000 7ff42d600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e2fee36ec0 0x55e2feec8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:18.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.040+0000 7ff42e000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e2feec9790 0x55e2feec6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:18.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.040+0000 7ff42e000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e2feec9790 0x55e2feec6740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:56772/0 (socket says 172.21.15.117:56772) 2024-06-03T03:27:18.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.040+0000 7ff42e000700 1 -- 172.21.15.117:0/813010080 learned_addr learned my addr 172.21.15.117:0/813010080 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:27:18.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.040+0000 7ff42cc00700 1 --2- 172.21.15.117:0/813010080 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2feeba630 0x55e2feec6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:18.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.040+0000 7ff42e000700 1 -- 172.21.15.117:0/813010080 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2feeba630 msgr2=0x55e2feec6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:18.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.040+0000 7ff42e000700 1 --2- 172.21.15.117:0/813010080 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2feeba630 0x55e2feec6200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:18.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.040+0000 7ff42e000700 1 -- 172.21.15.117:0/813010080 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e2fee36ec0 msgr2=0x55e2feec8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:18.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.040+0000 7ff42e000700 1 --2- 172.21.15.117:0/813010080 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e2fee36ec0 0x55e2feec8f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:18.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.040+0000 7ff42e000700 1 -- 172.21.15.117:0/813010080 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2fed5b950 con 0x55e2feec9790 2024-06-03T03:27:18.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.041+0000 7ff42e000700 1 --2- 172.21.15.117:0/813010080 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e2feec9790 0x55e2feec6740 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff420009d30 tx=0x7ff42000b3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b4da8f221e95872 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:27:18.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.041+0000 7ff40f400700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-06-03T03:27:18.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.041+0000 7ff40f400700 1 -- 172.21.15.117:0/813010080 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff42002f020 con 0x55e2feec9790 2024-06-03T03:27:18.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.041+0000 7ff40f400700 1 -- 172.21.15.117:0/813010080 <== mon.2 v2:172.21.15.151:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7ff42001e980 con 0x55e2feec9790 2024-06-03T03:27:18.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.041+0000 7ff43a92fb80 1 -- 172.21.15.117:0/813010080 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff414015070 con 0x55e2feec9790 2024-06-03T03:27:18.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.042+0000 7ff40f400700 1 -- 172.21.15.117:0/813010080 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff42003a050 con 0x55e2feec9790 2024-06-03T03:27:18.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.042+0000 7ff43a92fb80 10 client.25664 fetch_fsmap learned FSMap version 31 2024-06-03T03:27:18.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.042+0000 7ff43a92fb80 1 -- 172.21.15.117:0/813010080 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55e2feed6b80 con 0x55e2feec9790 2024-06-03T03:27:18.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.042+0000 7ff40f400700 1 -- 172.21.15.117:0/813010080 <== mon.2 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff414015070 con 0x55e2feec9790 2024-06-03T03:27:18.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.042+0000 7ff40f400700 1 -- 172.21.15.117:0/813010080 <== mon.2 v2:172.21.15.151:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7ff420028070 con 0x55e2feec9790 2024-06-03T03:27:18.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.042+0000 7ff43a92fb80 10 client.25664 fetch_fsmap finished waiting for FSMap version 31 2024-06-03T03:27:18.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.042+0000 7ff43a92fb80 10 client.25664 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-03T03:27:18.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.042+0000 7ff43a92fb80 20 client.25664 populate_metadata read hostname 'smithi117' 2024-06-03T03:27:18.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.042+0000 7ff43a92fb80 1 --2- 172.21.15.117:0/813010080 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55e2feed91c0 0x55e2feedb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:18.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.042+0000 7ff43a92fb80 4 client.25664 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T03:27:18.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.042+0000 7ff43a92fb80 1 -- 172.21.15.117:0/813010080 --> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e2fee1a080 con 0x55e2feed91c0 2024-06-03T03:27:18.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.043+0000 7ff42d600700 1 --2- 172.21.15.117:0/813010080 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55e2feed91c0 0x55e2feedb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:18.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.043+0000 7ff42d600700 1 --2- 172.21.15.117:0/813010080 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55e2feed91c0 0x55e2feedb5b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b839b8b94a391c48 server_cookie=cdfcd8b4edfea71a in_seq=0 out_seq=0 2024-06-03T03:27:18.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.043+0000 7ff40f400700 10 client.25664 ms_handle_connect on v2:172.21.15.183:6838/1439615083 2024-06-03T03:27:18.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.044+0000 7ff40f400700 1 -- 172.21.15.117:0/813010080 <== mds.0 v2:172.21.15.183:6838/1439615083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e2fee1a080 con 0x55e2feed91c0 2024-06-03T03:27:18.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.044+0000 7ff40f400700 10 client.25664 handle_command_reply: tid=0 2024-06-03T03:27:18.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.044+0000 7ff43a92fb80 1 client.25664 shutdown 2024-06-03T03:27:18.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.044+0000 7ff43a92fb80 1 -- 172.21.15.117:0/813010080 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e2feec9790 msgr2=0x55e2feec6740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:18.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.044+0000 7ff43a92fb80 1 --2- 172.21.15.117:0/813010080 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e2feec9790 0x55e2feec6740 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff420009d30 tx=0x7ff42000b3b0 comp rx=0 tx=0).stop 2024-06-03T03:27:18.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.044+0000 7ff43a92fb80 1 -- 172.21.15.117:0/813010080 shutdown_connections 2024-06-03T03:27:18.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.044+0000 7ff43a92fb80 1 --2- 172.21.15.117:0/813010080 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55e2feed91c0 0x55e2feedb5b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:18.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.044+0000 7ff43a92fb80 1 --2- 172.21.15.117:0/813010080 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e2feec9790 0x55e2feec6740 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:18.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.044+0000 7ff43a92fb80 1 --2- 172.21.15.117:0/813010080 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e2fee36ec0 0x55e2feec8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:18.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.044+0000 7ff43a92fb80 1 --2- 172.21.15.117:0/813010080 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2feeba630 0x55e2feec6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:18.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.045+0000 7ff40f400700 0 client.25664 ms_handle_reset on v2:172.21.15.183:6838/1439615083 2024-06-03T03:27:18.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.045+0000 7ff43a92fb80 1 -- 172.21.15.117:0/813010080 >> 172.21.15.117:0/813010080 conn(0x55e2fede71f0 msgr2=0x55e2feeb57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:27:18.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.045+0000 7ff43a92fb80 1 -- 172.21.15.117:0/813010080 shutdown_connections 2024-06-03T03:27:18.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.045+0000 7ff43a92fb80 1 -- 172.21.15.117:0/813010080 wait complete. 2024-06-03T03:27:18.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.045+0000 7ff43a92fb80 20 client.25664 trim_cache size 0 max 16384 2024-06-03T03:27:18.070 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.068+0000 7ff43a92fb80 1 Processor -- start 2024-06-03T03:27:18.070 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.068+0000 7ff43a92fb80 1 -- start start 2024-06-03T03:27:18.070 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.069+0000 7ff43a92fb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e2fee36ec0 0x55e2feeb66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:18.070 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.069+0000 7ff43a92fb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e2feeba630 0x55e2feeb5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:18.071 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.069+0000 7ff43a92fb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2feec9790 0x55e2feec8a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:18.072 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.069+0000 7ff43a92fb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff4281ac100 con 0x55e2fee36ec0 2024-06-03T03:27:18.072 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.069+0000 7ff43a92fb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff4281adc40 con 0x55e2feeba630 2024-06-03T03:27:18.072 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.069+0000 7ff43a92fb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff4281ac7d0 con 0x55e2feec9790 2024-06-03T03:27:18.072 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.069+0000 7ff42cc00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e2feeba630 0x55e2feeb5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:18.072 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.069+0000 7ff42d600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e2fee36ec0 0x55e2feeb66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:18.072 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.069+0000 7ff42cc00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e2feeba630 0x55e2feeb5640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:56778/0 (socket says 172.21.15.117:56778) 2024-06-03T03:27:18.072 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.069+0000 7ff42cc00700 1 -- 172.21.15.117:0/2007895273 learned_addr learned my addr 172.21.15.117:0/2007895273 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:27:18.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.069+0000 7ff42e000700 1 --2- 172.21.15.117:0/2007895273 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2feec9790 0x55e2feec8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:18.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.069+0000 7ff42d600700 1 -- 172.21.15.117:0/2007895273 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2feec9790 msgr2=0x55e2feec8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:18.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.069+0000 7ff42d600700 1 --2- 172.21.15.117:0/2007895273 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2feec9790 0x55e2feec8a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:18.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.069+0000 7ff42d600700 1 -- 172.21.15.117:0/2007895273 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e2feeba630 msgr2=0x55e2feeb5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:18.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.069+0000 7ff42d600700 1 --2- 172.21.15.117:0/2007895273 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e2feeba630 0x55e2feeb5640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:18.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.069+0000 7ff42d600700 1 -- 172.21.15.117:0/2007895273 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2feec70e0 con 0x55e2fee36ec0 2024-06-03T03:27:18.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.069+0000 7ff42cc00700 1 --2- 172.21.15.117:0/2007895273 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e2feeba630 0x55e2feeb5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:27:18.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.070+0000 7ff42d600700 1 --2- 172.21.15.117:0/2007895273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e2fee36ec0 0x55e2feeb66a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff41c009fd0 tx=0x7ff41c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b313107e8519ea2 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:27:18.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.070+0000 7ff41a000700 1 -- 172.21.15.117:0/2007895273 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff41c056070 con 0x55e2fee36ec0 2024-06-03T03:27:18.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.070+0000 7ff41a000700 1 -- 172.21.15.117:0/2007895273 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff41c09fe30 con 0x55e2fee36ec0 2024-06-03T03:27:18.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.070+0000 7ff41a000700 1 -- 172.21.15.117:0/2007895273 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff41c0980f0 con 0x55e2fee36ec0 2024-06-03T03:27:18.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.071+0000 7ff43a92fb80 1 -- 172.21.15.117:0/2007895273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e2fee36ec0 msgr2=0x55e2feeb66a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:18.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.071+0000 7ff43a92fb80 1 --2- 172.21.15.117:0/2007895273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e2fee36ec0 0x55e2feeb66a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff41c009fd0 tx=0x7ff41c00cf90 comp rx=0 tx=0).stop 2024-06-03T03:27:18.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.071+0000 7ff43a92fb80 1 -- 172.21.15.117:0/2007895273 shutdown_connections 2024-06-03T03:27:18.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.071+0000 7ff43a92fb80 1 --2- 172.21.15.117:0/2007895273 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2feec9790 0x55e2feec8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:18.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.071+0000 7ff43a92fb80 1 --2- 172.21.15.117:0/2007895273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e2fee36ec0 0x55e2feeb66a0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:18.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.071+0000 7ff43a92fb80 1 --2- 172.21.15.117:0/2007895273 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e2feeba630 0x55e2feeb5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:18.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.071+0000 7ff43a92fb80 1 -- 172.21.15.117:0/2007895273 >> 172.21.15.117:0/2007895273 conn(0x55e2fede71f0 msgr2=0x55e2feeb4f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:27:18.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.071+0000 7ff43a92fb80 1 -- 172.21.15.117:0/2007895273 shutdown_connections 2024-06-03T03:27:18.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.071+0000 7ff43a92fb80 1 -- 172.21.15.117:0/2007895273 wait complete. 2024-06-03T03:27:18.077 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.071+0000 7ff43a92fb80 1 -- start start 2024-06-03T03:27:18.077 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.072+0000 7ff43a92fb80 1 Processor -- start 2024-06-03T03:27:18.077 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.072+0000 7ff43a92fb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e2fee36ec0 0x55e2feec3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:18.077 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.072+0000 7ff43a92fb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e2feeba630 0x55e2feed16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:18.077 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.072+0000 7ff43a92fb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2feec9790 0x55e2feed1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:18.077 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.072+0000 7ff43a92fb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff4281ad570 con 0x55e2fee36ec0 2024-06-03T03:27:18.078 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.072+0000 7ff43a92fb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff4281acea0 con 0x55e2feeba630 2024-06-03T03:27:18.078 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.072+0000 7ff43a92fb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff4281aa0a0 con 0x55e2feec9790 2024-06-03T03:27:18.078 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.072+0000 7ff43a92fb80 10 client.0 _ll_register_callbacks cb 0x55e2feda67a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:27:18.078 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.072+0000 7ff42d600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e2fee36ec0 0x55e2feec3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:18.078 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.072+0000 7ff42d600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e2fee36ec0 0x55e2feec3b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:54018/0 (socket says 172.21.15.117:54018) 2024-06-03T03:27:18.078 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.072+0000 7ff42d600700 1 -- 172.21.15.117:0/333756042 learned_addr learned my addr 172.21.15.117:0/333756042 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:27:18.078 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.072+0000 7ff42cc00700 1 --2- 172.21.15.117:0/333756042 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e2feeba630 0x55e2feed16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:18.078 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.073+0000 7ff42e000700 1 --2- 172.21.15.117:0/333756042 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2feec9790 0x55e2feed1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:18.078 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.073+0000 7ff42d600700 1 -- 172.21.15.117:0/333756042 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2feec9790 msgr2=0x55e2feed1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:18.078 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.073+0000 7ff42d600700 1 --2- 172.21.15.117:0/333756042 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2feec9790 0x55e2feed1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:18.078 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.073+0000 7ff42d600700 1 -- 172.21.15.117:0/333756042 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e2feeba630 msgr2=0x55e2feed16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:18.079 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.073+0000 7ff42d600700 1 --2- 172.21.15.117:0/333756042 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e2feeba630 0x55e2feed16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:18.079 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.073+0000 7ff42d600700 1 -- 172.21.15.117:0/333756042 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2feed2210 con 0x55e2fee36ec0 2024-06-03T03:27:18.079 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.073+0000 7ff42d600700 1 --2- 172.21.15.117:0/333756042 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e2fee36ec0 0x55e2feec3b30 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff41c00c5b0 tx=0x7ff41c0a4770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a0c0cc7b208735f server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:27:18.079 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.073+0000 7ff40f400700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-06-03T03:27:18.079 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.073+0000 7ff40f400700 1 -- 172.21.15.117:0/333756042 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff41c0b4060 con 0x55e2fee36ec0 2024-06-03T03:27:18.079 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.073+0000 7ff40f400700 1 -- 172.21.15.117:0/333756042 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7ff41c0980f0 con 0x55e2fee36ec0 2024-06-03T03:27:18.079 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.073+0000 7ff43a92fb80 1 -- 172.21.15.117:0/333756042 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff41c09fe30 con 0x55e2fee36ec0 2024-06-03T03:27:18.079 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.074+0000 7ff40f400700 1 -- 172.21.15.117:0/333756042 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff41c05f070 con 0x55e2fee36ec0 2024-06-03T03:27:18.079 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.074+0000 7ff43a92fb80 10 client.15570 fetch_fsmap learned FSMap version 31 2024-06-03T03:27:18.079 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.074+0000 7ff43a92fb80 1 -- 172.21.15.117:0/333756042 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55e2fef44e80 con 0x55e2fee36ec0 2024-06-03T03:27:18.079 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.074+0000 7ff40f400700 1 -- 172.21.15.117:0/333756042 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff41c09fe30 con 0x55e2fee36ec0 2024-06-03T03:27:18.079 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.074+0000 7ff40f400700 1 -- 172.21.15.117:0/333756042 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7ff41c0a4260 con 0x55e2fee36ec0 2024-06-03T03:27:18.079 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.074+0000 7ff43a92fb80 10 client.15570 fetch_fsmap finished waiting for FSMap version 31 2024-06-03T03:27:18.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.074+0000 7ff43a92fb80 10 client.15570 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-03T03:27:18.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.074+0000 7ff43a92fb80 20 client.15570 populate_metadata read hostname 'smithi117' 2024-06-03T03:27:18.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.074+0000 7ff43a92fb80 1 --2- 172.21.15.117:0/333756042 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55e2fef45480 0x55e2fef478a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:18.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.074+0000 7ff43a92fb80 4 client.15570 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-03T03:27:18.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.074+0000 7ff43a92fb80 1 -- 172.21.15.117:0/333756042 --> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e2fee14600 con 0x55e2fef45480 2024-06-03T03:27:18.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.075+0000 7ff42cc00700 1 --2- 172.21.15.117:0/333756042 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55e2fef45480 0x55e2fef478a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:18.081 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.075+0000 7ff42cc00700 1 --2- 172.21.15.117:0/333756042 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55e2fef45480 0x55e2fef478a0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfcd50cd21b5a7e6 server_cookie=858be5d93110d041 in_seq=0 out_seq=0 2024-06-03T03:27:18.081 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.075+0000 7ff40f400700 10 client.15570 ms_handle_connect on v2:172.21.15.183:6838/1439615083 2024-06-03T03:27:18.081 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.076+0000 7ff40f400700 1 -- 172.21.15.117:0/333756042 <== mds.0 v2:172.21.15.183:6838/1439615083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e2fee14600 con 0x55e2fef45480 2024-06-03T03:27:18.081 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.076+0000 7ff40f400700 10 client.15570 handle_command_reply: tid=0 2024-06-03T03:27:18.082 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.076+0000 7ff43a92fb80 1 client.15570 shutdown 2024-06-03T03:27:18.082 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.077+0000 7ff43a92fb80 1 -- 172.21.15.117:0/333756042 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e2fee36ec0 msgr2=0x55e2feec3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:18.082 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.077+0000 7ff43a92fb80 1 --2- 172.21.15.117:0/333756042 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e2fee36ec0 0x55e2feec3b30 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff41c00c5b0 tx=0x7ff41c0a4770 comp rx=0 tx=0).stop 2024-06-03T03:27:18.082 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.077+0000 7ff43a92fb80 1 -- 172.21.15.117:0/333756042 shutdown_connections 2024-06-03T03:27:18.082 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.077+0000 7ff43a92fb80 1 --2- 172.21.15.117:0/333756042 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55e2fef45480 0x55e2fef478a0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:18.082 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.077+0000 7ff43a92fb80 1 --2- 172.21.15.117:0/333756042 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2feec9790 0x55e2feed1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:18.083 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.077+0000 7ff43a92fb80 1 --2- 172.21.15.117:0/333756042 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e2fee36ec0 0x55e2feec3b30 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:18.083 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.077+0000 7ff43a92fb80 1 --2- 172.21.15.117:0/333756042 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e2feeba630 0x55e2feed16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:18.083 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.077+0000 7ff43a92fb80 1 -- 172.21.15.117:0/333756042 >> 172.21.15.117:0/333756042 conn(0x55e2fede71f0 msgr2=0x55e2feeb57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:27:18.083 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.077+0000 7ff40f400700 0 client.15570 ms_handle_reset on v2:172.21.15.183:6838/1439615083 2024-06-03T03:27:18.083 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.077+0000 7ff43a92fb80 1 -- 172.21.15.117:0/333756042 shutdown_connections 2024-06-03T03:27:18.083 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.077+0000 7ff43a92fb80 1 -- 172.21.15.117:0/333756042 wait complete. 2024-06-03T03:27:18.083 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.078+0000 7ff43a92fb80 20 client.15570 trim_cache size 0 max 16384 2024-06-03T03:27:18.083 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-06-03T03:27:18.083 INFO:teuthology.orchestra.run.smithi117.stdout: "return_code": 0, 2024-06-03T03:27:18.083 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_tag": "7ac9de29-438a-45f0-839f-bb9b5c7f7030", 2024-06-03T03:27:18.084 INFO:teuthology.orchestra.run.smithi117.stdout: "mode": "asynchronous" 2024-06-03T03:27:18.084 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-06-03T03:27:18.084 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.078+0000 7ff430000700 1 -- 172.21.15.117:0/1140853896 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7ff4100a2c20 msgr2=0x7ff4100a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:18.084 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.078+0000 7ff430000700 1 --2- 172.21.15.117:0/1140853896 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7ff4100a2c20 0x7ff4100a50e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff420009850 tx=0x7ff4200097c0 comp rx=0 tx=0).stop 2024-06-03T03:27:18.084 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.078+0000 7ff430000700 1 -- 172.21.15.117:0/1140853896 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff42806a170 msgr2=0x7ff42819e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:18.084 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.078+0000 7ff430000700 1 --2- 172.21.15.117:0/1140853896 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff42806a170 0x7ff42819e860 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff41c00e9c0 tx=0x7ff41c00ecd0 comp rx=0 tx=0).stop 2024-06-03T03:27:18.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.078+0000 7ff430000700 1 -- 172.21.15.117:0/1140853896 shutdown_connections 2024-06-03T03:27:18.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.078+0000 7ff430000700 1 --2- 172.21.15.117:0/1140853896 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7ff4100a2c20 0x7ff4100a50e0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:18.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.078+0000 7ff430000700 1 --2- 172.21.15.117:0/1140853896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff428108500 0x7ff428198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:18.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.079+0000 7ff430000700 1 --2- 172.21.15.117:0/1140853896 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff42806a170 0x7ff42819e860 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:18.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.079+0000 7ff430000700 1 --2- 172.21.15.117:0/1140853896 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff42806ab30 0x7ff42819eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:18.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.079+0000 7ff430000700 1 -- 172.21.15.117:0/1140853896 >> 172.21.15.117:0/1140853896 conn(0x7ff428078340 msgr2=0x7ff428100000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:27:18.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.079+0000 7ff430000700 1 -- 172.21.15.117:0/1140853896 shutdown_connections 2024-06-03T03:27:18.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:18.079+0000 7ff430000700 1 -- 172.21.15.117:0/1140853896 wait complete. 2024-06-03T03:27:18.906 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:27:18 smithi151 ceph-mon[37043]: scrub queued for path: / 2024-06-03T03:27:18.906 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:27:18 smithi151 ceph-mon[37043]: scrub summary: idle+waiting paths [/] 2024-06-03T03:27:18.906 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:27:18 smithi151 ceph-mon[37043]: scrub summary: active paths [/] 2024-06-03T03:27:18.907 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:27:18 smithi151 ceph-mon[37043]: scrub summary: idle+waiting paths [/] 2024-06-03T03:27:18.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:27:18 smithi117 ceph-mon[31649]: scrub queued for path: / 2024-06-03T03:27:18.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:27:18 smithi117 ceph-mon[31649]: scrub summary: idle+waiting paths [/] 2024-06-03T03:27:18.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:27:18 smithi117 ceph-mon[31649]: scrub summary: active paths [/] 2024-06-03T03:27:18.948 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:27:18 smithi117 ceph-mon[31649]: scrub summary: idle+waiting paths [/] 2024-06-03T03:27:18.981 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:27:18 smithi183 ceph-mon[37326]: scrub queued for path: / 2024-06-03T03:27:18.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:27:18 smithi183 ceph-mon[37326]: scrub summary: idle+waiting paths [/] 2024-06-03T03:27:18.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:27:18 smithi183 ceph-mon[37326]: scrub summary: active paths [/] 2024-06-03T03:27:18.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:27:18 smithi183 ceph-mon[37326]: scrub summary: idle+waiting paths [/] 2024-06-03T03:27:19.058 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-03T03:27:19.481 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:27:19.863 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:27:19 smithi117 ceph-mon[31649]: pgmap v566: 129 pgs: 129 active+clean; 1.8 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 863 KiB/s wr, 17 op/s 2024-06-03T03:27:19.906 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:27:19 smithi151 ceph-mon[37043]: pgmap v566: 129 pgs: 129 active+clean; 1.8 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 863 KiB/s wr, 17 op/s 2024-06-03T03:27:19.981 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:27:19 smithi183 ceph-mon[37326]: pgmap v566: 129 pgs: 129 active+clean; 1.8 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 863 KiB/s wr, 17 op/s 2024-06-03T03:27:21.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:27:21 smithi151 ceph-mon[37043]: pgmap v567: 129 pgs: 129 active+clean; 1.8 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 1.3 MiB/s wr, 15 op/s 2024-06-03T03:27:21.412 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:27:21 smithi117 ceph-mon[31649]: pgmap v567: 129 pgs: 129 active+clean; 1.8 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 1.3 MiB/s wr, 15 op/s 2024-06-03T03:27:21.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:27:21 smithi183 ceph-mon[37326]: pgmap v567: 129 pgs: 129 active+clean; 1.8 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 1.3 MiB/s wr, 15 op/s 2024-06-03T03:27:21.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.918+0000 7f458c600700 1 -- 172.21.15.117:0/1128932280 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4584107840 msgr2=0x7f458410f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:21.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.918+0000 7f458c600700 1 --2- 172.21.15.117:0/1128932280 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4584107840 0x7f458410f3c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f457c009f90 tx=0x7f457c00b330 comp rx=0 tx=0).stop 2024-06-03T03:27:21.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.918+0000 7f458c600700 1 -- 172.21.15.117:0/1128932280 shutdown_connections 2024-06-03T03:27:21.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.918+0000 7f458c600700 1 --2- 172.21.15.117:0/1128932280 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4584107840 0x7f458410f3c0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:21.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.918+0000 7f458c600700 1 --2- 172.21.15.117:0/1128932280 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4584102740 0x7f4584102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:21.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.918+0000 7f458c600700 1 --2- 172.21.15.117:0/1128932280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45841030f0 0x7f4584107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:21.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.918+0000 7f458c600700 1 -- 172.21.15.117:0/1128932280 >> 172.21.15.117:0/1128932280 conn(0x7f45840795f0 msgr2=0x7f4584079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:27:21.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.919+0000 7f458c600700 1 -- 172.21.15.117:0/1128932280 shutdown_connections 2024-06-03T03:27:21.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.919+0000 7f458c600700 1 -- 172.21.15.117:0/1128932280 wait complete. 2024-06-03T03:27:21.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.920+0000 7f458c600700 1 Processor -- start 2024-06-03T03:27:21.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.920+0000 7f458c600700 1 -- start start 2024-06-03T03:27:21.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.921+0000 7f458c600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4584102740 0x7f458419ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:21.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.921+0000 7f458c600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45841030f0 0x7f458419b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:21.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.921+0000 7f458c600700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4584107840 0x7f458419f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:21.923 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.921+0000 7f458c600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f45841127b0 con 0x7f45841030f0 2024-06-03T03:27:21.923 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.921+0000 7f458c600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4584112630 con 0x7f4584102740 2024-06-03T03:27:21.923 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.921+0000 7f458c600700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4584112930 con 0x7f4584107840 2024-06-03T03:27:21.923 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.921+0000 7f4589200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45841030f0 0x7f458419b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:21.923 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.921+0000 7f4589c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4584102740 0x7f458419ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:21.924 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.921+0000 7f4589c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4584102740 0x7f458419ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:56820/0 (socket says 172.21.15.117:56820) 2024-06-03T03:27:21.924 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.921+0000 7f4589c00700 1 -- 172.21.15.117:0/3933532134 learned_addr learned my addr 172.21.15.117:0/3933532134 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:27:21.924 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.921+0000 7f4589200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45841030f0 0x7f458419b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:54036/0 (socket says 172.21.15.117:54036) 2024-06-03T03:27:21.924 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.921+0000 7f458a600700 1 --2- 172.21.15.117:0/3933532134 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4584107840 0x7f458419f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:21.924 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.922+0000 7f4589c00700 1 -- 172.21.15.117:0/3933532134 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4584107840 msgr2=0x7f458419f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:21.924 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.922+0000 7f4589c00700 1 --2- 172.21.15.117:0/3933532134 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4584107840 0x7f458419f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:21.924 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.922+0000 7f4589c00700 1 -- 172.21.15.117:0/3933532134 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45841030f0 msgr2=0x7f458419b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:21.924 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.922+0000 7f4589c00700 1 --2- 172.21.15.117:0/3933532134 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45841030f0 0x7f458419b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:21.924 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.922+0000 7f4589c00700 1 -- 172.21.15.117:0/3933532134 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f458419fd30 con 0x7f4584102740 2024-06-03T03:27:21.924 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.922+0000 7f4589c00700 1 --2- 172.21.15.117:0/3933532134 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4584102740 0x7f458419ad20 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f457800ec10 tx=0x7f457800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:27:21.924 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.922+0000 7f4582000700 1 -- 172.21.15.117:0/3933532134 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f457800cc90 con 0x7f4584102740 2024-06-03T03:27:21.924 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.922+0000 7f4582000700 1 -- 172.21.15.117:0/3933532134 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f457800ce30 con 0x7f4584102740 2024-06-03T03:27:21.925 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.922+0000 7f458c600700 1 -- 172.21.15.117:0/3933532134 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45841a0020 con 0x7f4584102740 2024-06-03T03:27:21.925 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.922+0000 7f4582000700 1 -- 172.21.15.117:0/3933532134 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4578010690 con 0x7f4584102740 2024-06-03T03:27:21.925 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.922+0000 7f458c600700 1 -- 172.21.15.117:0/3933532134 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45841a78a0 con 0x7f4584102740 2024-06-03T03:27:21.927 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.926+0000 7f4582000700 1 -- 172.21.15.117:0/3933532134 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f4578003680 con 0x7f4584102740 2024-06-03T03:27:21.928 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.927+0000 7f4582000700 1 --2- 172.21.15.117:0/3933532134 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f456c0a2c20 0x7f456c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:21.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.927+0000 7f4582000700 1 -- 172.21.15.117:0/3933532134 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f45780cf790 con 0x7f4584102740 2024-06-03T03:27:21.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.927+0000 7f4589200700 1 --2- 172.21.15.117:0/3933532134 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f456c0a2c20 0x7f456c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:21.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.928+0000 7f4589200700 1 --2- 172.21.15.117:0/3933532134 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f456c0a2c20 0x7f456c0a50e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4584108e90 tx=0x7f4570009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:27:21.930 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.929+0000 7f4596eccb80 1 Processor -- start 2024-06-03T03:27:21.930 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.929+0000 7f4596eccb80 1 -- start start 2024-06-03T03:27:21.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.929+0000 7f4596eccb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cca7a81e40 0x55cca7b05070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:21.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.929+0000 7f4596eccb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cca7b055b0 0x55cca7b05a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:21.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.929+0000 7f4596eccb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cca7b10300 0x55cca7b126f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:21.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.929+0000 7f4596eccb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55cca7a14a10 con 0x55cca7a81e40 2024-06-03T03:27:21.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.930+0000 7f4596eccb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55cca7a57b00 con 0x55cca7b10300 2024-06-03T03:27:21.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.930+0000 7f4596eccb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55cca7a128f0 con 0x55cca7b055b0 2024-06-03T03:27:21.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.930+0000 7f4589c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cca7a81e40 0x55cca7b05070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:21.932 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.930+0000 7f4589c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cca7a81e40 0x55cca7b05070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:54038/0 (socket says 172.21.15.117:54038) 2024-06-03T03:27:21.932 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.930+0000 7f4589200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cca7b055b0 0x55cca7b05a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:21.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.930+0000 7f4589c00700 1 -- 172.21.15.117:0/2743564099 learned_addr learned my addr 172.21.15.117:0/2743564099 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:27:21.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.930+0000 7f458a600700 1 --2- 172.21.15.117:0/2743564099 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cca7b10300 0x55cca7b126f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:21.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.930+0000 7f4589c00700 1 -- 172.21.15.117:0/2743564099 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cca7b055b0 msgr2=0x55cca7b05a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:21.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.930+0000 7f4589c00700 1 --2- 172.21.15.117:0/2743564099 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cca7b055b0 0x55cca7b05a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:21.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.930+0000 7f4589c00700 1 -- 172.21.15.117:0/2743564099 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cca7b10300 msgr2=0x55cca7b126f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:21.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.930+0000 7f4589c00700 1 --2- 172.21.15.117:0/2743564099 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cca7b10300 0x55cca7b126f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:21.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.930+0000 7f4589c00700 1 -- 172.21.15.117:0/2743564099 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cca79fff90 con 0x55cca7a81e40 2024-06-03T03:27:21.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.931+0000 7f4589c00700 1 --2- 172.21.15.117:0/2743564099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cca7a81e40 0x55cca7b05070 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f45780a4340 tx=0x7f457809bb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4356d7d7e78ef676 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:27:21.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.931+0000 7f4576a00700 1 -- 172.21.15.117:0/2743564099 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f457809c300 con 0x55cca7a81e40 2024-06-03T03:27:21.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.931+0000 7f4576a00700 1 -- 172.21.15.117:0/2743564099 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45780b2070 con 0x55cca7a81e40 2024-06-03T03:27:21.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.931+0000 7f4576a00700 1 -- 172.21.15.117:0/2743564099 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45780cf470 con 0x55cca7a81e40 2024-06-03T03:27:21.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.932+0000 7f4596eccb80 1 -- 172.21.15.117:0/2743564099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cca7a81e40 msgr2=0x55cca7b05070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:21.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.932+0000 7f4596eccb80 1 --2- 172.21.15.117:0/2743564099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cca7a81e40 0x55cca7b05070 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f45780a4340 tx=0x7f457809bb00 comp rx=0 tx=0).stop 2024-06-03T03:27:21.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.932+0000 7f4596eccb80 1 -- 172.21.15.117:0/2743564099 shutdown_connections 2024-06-03T03:27:21.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.932+0000 7f4596eccb80 1 --2- 172.21.15.117:0/2743564099 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cca7b10300 0x55cca7b126f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:21.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.932+0000 7f4596eccb80 1 --2- 172.21.15.117:0/2743564099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cca7a81e40 0x55cca7b05070 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:21.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.932+0000 7f4596eccb80 1 --2- 172.21.15.117:0/2743564099 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cca7b055b0 0x55cca7b05a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:21.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.932+0000 7f4596eccb80 1 -- 172.21.15.117:0/2743564099 >> 172.21.15.117:0/2743564099 conn(0x55cca7a32170 msgr2=0x55cca7b014b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:27:21.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.932+0000 7f4596eccb80 1 -- 172.21.15.117:0/2743564099 shutdown_connections 2024-06-03T03:27:21.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.933+0000 7f4596eccb80 1 -- 172.21.15.117:0/2743564099 wait complete. 2024-06-03T03:27:21.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.933+0000 7f4596eccb80 1 -- start start 2024-06-03T03:27:21.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.933+0000 7f4596eccb80 1 Processor -- start 2024-06-03T03:27:21.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.934+0000 7f4596eccb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cca7a81e40 0x55cca7b0fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:21.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.934+0000 7f4596eccb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cca7b055b0 0x55cca7b0ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:21.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.934+0000 7f4596eccb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cca7b10300 0x55cca7b0d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:21.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.934+0000 7f4596eccb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55cca79f3f60 con 0x55cca7a81e40 2024-06-03T03:27:21.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.934+0000 7f4596eccb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55cca7a53ac0 con 0x55cca7b10300 2024-06-03T03:27:21.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.934+0000 7f4596eccb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55cca7b12fc0 con 0x55cca7b055b0 2024-06-03T03:27:21.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.934+0000 7f4596eccb80 10 client.0 _ll_register_callbacks cb 0x55cca79f1720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:27:21.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.934+0000 7f4589c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cca7a81e40 0x55cca7b0fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:21.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.934+0000 7f458a600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cca7b10300 0x55cca7b0d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:21.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.934+0000 7f4589c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cca7a81e40 0x55cca7b0fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:54048/0 (socket says 172.21.15.117:54048) 2024-06-03T03:27:21.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.934+0000 7f4589c00700 1 -- 172.21.15.117:0/4075786287 learned_addr learned my addr 172.21.15.117:0/4075786287 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:27:21.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.934+0000 7f4589200700 1 --2- 172.21.15.117:0/4075786287 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cca7b055b0 0x55cca7b0ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:21.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.935+0000 7f458a600700 1 -- 172.21.15.117:0/4075786287 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cca7b055b0 msgr2=0x55cca7b0ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:21.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.935+0000 7f458a600700 1 --2- 172.21.15.117:0/4075786287 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cca7b055b0 0x55cca7b0ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:21.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.935+0000 7f458a600700 1 -- 172.21.15.117:0/4075786287 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cca7a81e40 msgr2=0x55cca7b0fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:21.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.935+0000 7f458a600700 1 --2- 172.21.15.117:0/4075786287 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cca7a81e40 0x55cca7b0fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:21.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.935+0000 7f458a600700 1 -- 172.21.15.117:0/4075786287 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cca79a68d0 con 0x55cca7b10300 2024-06-03T03:27:21.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.935+0000 7f4589c00700 1 --2- 172.21.15.117:0/4075786287 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cca7a81e40 0x55cca7b0fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:27:21.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.935+0000 7f458a600700 1 --2- 172.21.15.117:0/4075786287 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cca7b10300 0x55cca7b0d3a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f457c00b6c0 tx=0x7f457c009510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea7013cb1e8a5c4a server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:27:21.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.935+0000 7f456b400700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-06-03T03:27:21.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.935+0000 7f456b400700 1 -- 172.21.15.117:0/4075786287 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f457c005240 con 0x55cca7b10300 2024-06-03T03:27:21.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.936+0000 7f456b400700 1 -- 172.21.15.117:0/4075786287 <== mon.2 v2:172.21.15.151:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f457c0053e0 con 0x55cca7b10300 2024-06-03T03:27:21.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.936+0000 7f4596eccb80 1 -- 172.21.15.117:0/4075786287 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45780b2070 con 0x55cca7b10300 2024-06-03T03:27:21.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.936+0000 7f456b400700 1 -- 172.21.15.117:0/4075786287 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f457c0055e0 con 0x55cca7b10300 2024-06-03T03:27:21.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.936+0000 7f4596eccb80 10 client.25682 fetch_fsmap learned FSMap version 31 2024-06-03T03:27:21.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.936+0000 7f4596eccb80 1 -- 172.21.15.117:0/4075786287 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55cca7b1d7f0 con 0x55cca7b10300 2024-06-03T03:27:21.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.936+0000 7f456b400700 1 -- 172.21.15.117:0/4075786287 <== mon.2 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45780b2070 con 0x55cca7b10300 2024-06-03T03:27:21.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.936+0000 7f456b400700 1 -- 172.21.15.117:0/4075786287 <== mon.2 v2:172.21.15.151:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f457c011070 con 0x55cca7b10300 2024-06-03T03:27:21.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.937+0000 7f4596eccb80 10 client.25682 fetch_fsmap finished waiting for FSMap version 31 2024-06-03T03:27:21.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.937+0000 7f4596eccb80 10 client.25682 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-03T03:27:21.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.937+0000 7f4596eccb80 20 client.25682 populate_metadata read hostname 'smithi117' 2024-06-03T03:27:21.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.937+0000 7f4596eccb80 1 --2- 172.21.15.117:0/4075786287 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55cca7b1fe30 0x55cca7b22260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:21.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.937+0000 7f4596eccb80 4 client.25682 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T03:27:21.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.937+0000 7f4596eccb80 1 -- 172.21.15.117:0/4075786287 --> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cca7a65000 con 0x55cca7b1fe30 2024-06-03T03:27:21.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.937+0000 7f4589c00700 1 --2- 172.21.15.117:0/4075786287 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55cca7b1fe30 0x55cca7b22260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:21.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.938+0000 7f4589c00700 1 --2- 172.21.15.117:0/4075786287 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55cca7b1fe30 0x55cca7b22260 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=377f734d42d8b921 server_cookie=d78e0ca0ff897722 in_seq=0 out_seq=0 2024-06-03T03:27:21.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.938+0000 7f456b400700 10 client.25682 ms_handle_connect on v2:172.21.15.183:6838/1439615083 2024-06-03T03:27:21.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.938+0000 7f456b400700 1 -- 172.21.15.117:0/4075786287 <== mds.0 v2:172.21.15.183:6838/1439615083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55cca7a65000 con 0x55cca7b1fe30 2024-06-03T03:27:21.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.938+0000 7f456b400700 10 client.25682 handle_command_reply: tid=0 2024-06-03T03:27:21.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.938+0000 7f4596eccb80 1 client.25682 shutdown 2024-06-03T03:27:21.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.939+0000 7f4596eccb80 1 -- 172.21.15.117:0/4075786287 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cca7b10300 msgr2=0x55cca7b0d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:21.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.939+0000 7f4596eccb80 1 --2- 172.21.15.117:0/4075786287 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cca7b10300 0x55cca7b0d3a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f457c00b6c0 tx=0x7f457c009510 comp rx=0 tx=0).stop 2024-06-03T03:27:21.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.939+0000 7f4596eccb80 1 -- 172.21.15.117:0/4075786287 shutdown_connections 2024-06-03T03:27:21.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.939+0000 7f4596eccb80 1 --2- 172.21.15.117:0/4075786287 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55cca7b1fe30 0x55cca7b22260 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:21.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.939+0000 7f4596eccb80 1 --2- 172.21.15.117:0/4075786287 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cca7b10300 0x55cca7b0d3a0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:21.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.939+0000 7f4596eccb80 1 --2- 172.21.15.117:0/4075786287 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cca7a81e40 0x55cca7b0fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:21.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.939+0000 7f4596eccb80 1 --2- 172.21.15.117:0/4075786287 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cca7b055b0 0x55cca7b0ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:21.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.939+0000 7f4596eccb80 1 -- 172.21.15.117:0/4075786287 >> 172.21.15.117:0/4075786287 conn(0x55cca7a32170 msgr2=0x55cca7b00740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:27:21.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.939+0000 7f456b400700 0 client.25682 ms_handle_reset on v2:172.21.15.183:6838/1439615083 2024-06-03T03:27:21.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.939+0000 7f4596eccb80 1 -- 172.21.15.117:0/4075786287 shutdown_connections 2024-06-03T03:27:21.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.939+0000 7f4596eccb80 1 -- 172.21.15.117:0/4075786287 wait complete. 2024-06-03T03:27:21.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.939+0000 7f4596eccb80 20 client.25682 trim_cache size 0 max 16384 2024-06-03T03:27:21.963 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.961+0000 7f4596eccb80 1 Processor -- start 2024-06-03T03:27:21.963 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.961+0000 7f4596eccb80 1 -- start start 2024-06-03T03:27:21.964 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.962+0000 7f4596eccb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cca7a81e40 0x55cca7b1c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:21.964 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.962+0000 7f4596eccb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cca7b055b0 0x55cca7b0f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:21.964 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.962+0000 7f4596eccb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cca7b10300 0x55cca7b116f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:21.964 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.962+0000 7f4596eccb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f45841b0510 con 0x55cca7a81e40 2024-06-03T03:27:21.964 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.962+0000 7f4596eccb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f45841b2050 con 0x55cca7b10300 2024-06-03T03:27:21.964 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.962+0000 7f4596eccb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f45841b0be0 con 0x55cca7b055b0 2024-06-03T03:27:21.964 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.962+0000 7f4589c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cca7a81e40 0x55cca7b1c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:21.964 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.962+0000 7f458a600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cca7b10300 0x55cca7b116f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:21.964 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.962+0000 7f458a600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cca7b10300 0x55cca7b116f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:56854/0 (socket says 172.21.15.117:56854) 2024-06-03T03:27:21.964 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.962+0000 7f4589c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cca7a81e40 0x55cca7b1c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:54064/0 (socket says 172.21.15.117:54064) 2024-06-03T03:27:21.964 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.962+0000 7f458a600700 1 -- 172.21.15.117:0/2690052553 learned_addr learned my addr 172.21.15.117:0/2690052553 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:27:21.965 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.962+0000 7f4589200700 1 --2- 172.21.15.117:0/2690052553 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cca7b055b0 0x55cca7b0f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:21.965 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.962+0000 7f458a600700 1 -- 172.21.15.117:0/2690052553 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cca7b055b0 msgr2=0x55cca7b0f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:21.965 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.962+0000 7f458a600700 1 --2- 172.21.15.117:0/2690052553 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cca7b055b0 0x55cca7b0f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:21.965 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.962+0000 7f458a600700 1 -- 172.21.15.117:0/2690052553 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cca7a81e40 msgr2=0x55cca7b1c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:21.965 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.962+0000 7f458a600700 1 --2- 172.21.15.117:0/2690052553 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cca7a81e40 0x55cca7b1c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:21.965 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.962+0000 7f458a600700 1 -- 172.21.15.117:0/2690052553 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cca7af8e10 con 0x55cca7b10300 2024-06-03T03:27:21.966 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.963+0000 7f458a600700 1 --2- 172.21.15.117:0/2690052553 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cca7b10300 0x55cca7b116f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f457c009ac0 tx=0x7f457c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50a71a2d52b432fb server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:27:21.966 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.963+0000 7f4576000700 1 -- 172.21.15.117:0/2690052553 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f457c03a070 con 0x55cca7b10300 2024-06-03T03:27:21.966 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.963+0000 7f4576000700 1 -- 172.21.15.117:0/2690052553 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f457c027470 con 0x55cca7b10300 2024-06-03T03:27:21.966 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.963+0000 7f4576000700 1 -- 172.21.15.117:0/2690052553 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f457c005f20 con 0x55cca7b10300 2024-06-03T03:27:21.966 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.964+0000 7f4596eccb80 1 -- 172.21.15.117:0/2690052553 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cca7b10300 msgr2=0x55cca7b116f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:21.966 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.964+0000 7f4596eccb80 1 --2- 172.21.15.117:0/2690052553 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cca7b10300 0x55cca7b116f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f457c009ac0 tx=0x7f457c00b330 comp rx=0 tx=0).stop 2024-06-03T03:27:21.967 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.964+0000 7f4596eccb80 1 -- 172.21.15.117:0/2690052553 shutdown_connections 2024-06-03T03:27:21.967 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.964+0000 7f4596eccb80 1 --2- 172.21.15.117:0/2690052553 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cca7b10300 0x55cca7b116f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:21.967 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.964+0000 7f4596eccb80 1 --2- 172.21.15.117:0/2690052553 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cca7a81e40 0x55cca7b1c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:21.967 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.964+0000 7f4596eccb80 1 --2- 172.21.15.117:0/2690052553 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cca7b055b0 0x55cca7b0f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:21.967 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.964+0000 7f4596eccb80 1 -- 172.21.15.117:0/2690052553 >> 172.21.15.117:0/2690052553 conn(0x55cca7a32170 msgr2=0x55cca7b00fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:27:21.967 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.964+0000 7f4596eccb80 1 -- 172.21.15.117:0/2690052553 shutdown_connections 2024-06-03T03:27:21.967 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.964+0000 7f4596eccb80 1 -- 172.21.15.117:0/2690052553 wait complete. 2024-06-03T03:27:21.967 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.965+0000 7f4596eccb80 1 -- start start 2024-06-03T03:27:21.967 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.965+0000 7f4596eccb80 1 Processor -- start 2024-06-03T03:27:21.968 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.965+0000 7f4596eccb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cca7a81e40 0x55cca7b18250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:21.968 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.965+0000 7f4596eccb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cca7b055b0 0x55cca7b18790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:21.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.965+0000 7f4596eccb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cca7b10300 0x55cca7b0a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:21.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.965+0000 7f4596eccb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f45841b1980 con 0x55cca7a81e40 2024-06-03T03:27:21.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.965+0000 7f4596eccb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f45841b12b0 con 0x55cca7b055b0 2024-06-03T03:27:21.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.965+0000 7f4596eccb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f45841ae4b0 con 0x55cca7b10300 2024-06-03T03:27:21.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.965+0000 7f4596eccb80 10 client.0 _ll_register_callbacks cb 0x55cca79f1720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:27:21.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.966+0000 7f4589c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cca7a81e40 0x55cca7b18250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:21.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.966+0000 7f4589200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cca7b055b0 0x55cca7b18790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:21.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.966+0000 7f4589c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cca7a81e40 0x55cca7b18250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:54074/0 (socket says 172.21.15.117:54074) 2024-06-03T03:27:21.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.966+0000 7f4589c00700 1 -- 172.21.15.117:0/1641366284 learned_addr learned my addr 172.21.15.117:0/1641366284 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:27:21.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.966+0000 7f458a600700 1 --2- 172.21.15.117:0/1641366284 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cca7b10300 0x55cca7b0a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:21.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.966+0000 7f4589200700 1 -- 172.21.15.117:0/1641366284 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cca7b10300 msgr2=0x55cca7b0a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:21.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.966+0000 7f4589200700 1 --2- 172.21.15.117:0/1641366284 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cca7b10300 0x55cca7b0a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:21.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.966+0000 7f4589200700 1 -- 172.21.15.117:0/1641366284 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cca7a81e40 msgr2=0x55cca7b18250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:21.970 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.966+0000 7f4589200700 1 --2- 172.21.15.117:0/1641366284 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cca7a81e40 0x55cca7b18250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:21.970 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.966+0000 7f4589200700 1 -- 172.21.15.117:0/1641366284 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cca7b28480 con 0x55cca7b055b0 2024-06-03T03:27:21.970 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.966+0000 7f4589c00700 1 --2- 172.21.15.117:0/1641366284 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cca7a81e40 0x55cca7b18250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:27:21.970 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.966+0000 7f4589200700 1 --2- 172.21.15.117:0/1641366284 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cca7b055b0 0x55cca7b18790 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x55cca7b0e430 tx=0x7f45700047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5d1fdce8396d0e2 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:27:21.970 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.966+0000 7f456b400700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-06-03T03:27:21.970 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.967+0000 7f456b400700 1 -- 172.21.15.117:0/1641366284 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f457002d060 con 0x55cca7b055b0 2024-06-03T03:27:21.970 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.967+0000 7f4596eccb80 1 -- 172.21.15.117:0/1641366284 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f457c027470 con 0x55cca7b055b0 2024-06-03T03:27:21.970 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.967+0000 7f456b400700 1 -- 172.21.15.117:0/1641366284 <== mon.2 v2:172.21.15.151:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f4570028660 con 0x55cca7b055b0 2024-06-03T03:27:21.970 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.967+0000 7f456b400700 1 -- 172.21.15.117:0/1641366284 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f4570034050 con 0x55cca7b055b0 2024-06-03T03:27:21.970 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.967+0000 7f4596eccb80 10 client.25688 fetch_fsmap learned FSMap version 31 2024-06-03T03:27:21.970 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.967+0000 7f4596eccb80 1 -- 172.21.15.117:0/1641366284 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55cca7b26ec0 con 0x55cca7b055b0 2024-06-03T03:27:21.972 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.968+0000 7f456b400700 1 -- 172.21.15.117:0/1641366284 <== mon.2 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f457c027470 con 0x55cca7b055b0 2024-06-03T03:27:21.972 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.968+0000 7f456b400700 1 -- 172.21.15.117:0/1641366284 <== mon.2 v2:172.21.15.151:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f457001d930 con 0x55cca7b055b0 2024-06-03T03:27:21.972 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.968+0000 7f4596eccb80 10 client.25688 fetch_fsmap finished waiting for FSMap version 31 2024-06-03T03:27:21.972 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.968+0000 7f4596eccb80 10 client.25688 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-03T03:27:21.972 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.968+0000 7f4596eccb80 20 client.25688 populate_metadata read hostname 'smithi117' 2024-06-03T03:27:21.972 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.968+0000 7f4596eccb80 1 --2- 172.21.15.117:0/1641366284 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55cca7b27570 0x55cca7b8cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:21.972 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.968+0000 7f4596eccb80 4 client.25688 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-03T03:27:21.972 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.968+0000 7f4596eccb80 1 -- 172.21.15.117:0/1641366284 --> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cca7a5f580 con 0x55cca7b27570 2024-06-03T03:27:21.972 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.968+0000 7f4589c00700 1 --2- 172.21.15.117:0/1641366284 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55cca7b27570 0x55cca7b8cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:21.973 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.969+0000 7f4589c00700 1 --2- 172.21.15.117:0/1641366284 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55cca7b27570 0x55cca7b8cb10 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2270fa0ec49e5a51 server_cookie=d5721a579679cf39 in_seq=0 out_seq=0 2024-06-03T03:27:21.973 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.969+0000 7f456b400700 10 client.25688 ms_handle_connect on v2:172.21.15.183:6838/1439615083 2024-06-03T03:27:21.973 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.969+0000 7f456b400700 1 -- 172.21.15.117:0/1641366284 <== mds.0 v2:172.21.15.183:6838/1439615083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55cca7a5f580 con 0x55cca7b27570 2024-06-03T03:27:21.973 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.969+0000 7f456b400700 10 client.25688 handle_command_reply: tid=0 2024-06-03T03:27:21.973 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.969+0000 7f4596eccb80 1 client.25688 shutdown 2024-06-03T03:27:21.973 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.969+0000 7f4596eccb80 1 -- 172.21.15.117:0/1641366284 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cca7b055b0 msgr2=0x55cca7b18790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:21.973 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.969+0000 7f4596eccb80 1 --2- 172.21.15.117:0/1641366284 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cca7b055b0 0x55cca7b18790 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x55cca7b0e430 tx=0x7f45700047c0 comp rx=0 tx=0).stop 2024-06-03T03:27:21.973 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.969+0000 7f4596eccb80 1 -- 172.21.15.117:0/1641366284 shutdown_connections 2024-06-03T03:27:21.973 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.969+0000 7f4596eccb80 1 --2- 172.21.15.117:0/1641366284 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55cca7b27570 0x55cca7b8cb10 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:21.973 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.969+0000 7f4596eccb80 1 --2- 172.21.15.117:0/1641366284 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cca7b10300 0x55cca7b0a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:21.973 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.969+0000 7f456b400700 0 client.25688 ms_handle_reset on v2:172.21.15.183:6838/1439615083 2024-06-03T03:27:21.973 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.969+0000 7f4596eccb80 1 --2- 172.21.15.117:0/1641366284 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cca7a81e40 0x55cca7b18250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:21.973 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.969+0000 7f4596eccb80 1 --2- 172.21.15.117:0/1641366284 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cca7b055b0 0x55cca7b18790 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:21.973 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.969+0000 7f4596eccb80 1 -- 172.21.15.117:0/1641366284 >> 172.21.15.117:0/1641366284 conn(0x55cca7a32170 msgr2=0x55cca7b0b750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:27:21.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.969+0000 7f4596eccb80 1 -- 172.21.15.117:0/1641366284 shutdown_connections 2024-06-03T03:27:21.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.969+0000 7f4596eccb80 1 -- 172.21.15.117:0/1641366284 wait complete. 2024-06-03T03:27:21.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.970+0000 7f4596eccb80 20 client.25688 trim_cache size 0 max 16384 2024-06-03T03:27:21.974 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-06-03T03:27:21.974 INFO:teuthology.orchestra.run.smithi117.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-03T03:27:21.974 INFO:teuthology.orchestra.run.smithi117.stdout: "scrubs": { 2024-06-03T03:27:21.974 INFO:teuthology.orchestra.run.smithi117.stdout: "7ac9de29-438a-45f0-839f-bb9b5c7f7030": { 2024-06-03T03:27:21.974 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-06-03T03:27:21.974 INFO:teuthology.orchestra.run.smithi117.stdout: "tag": "7ac9de29-438a-45f0-839f-bb9b5c7f7030", 2024-06-03T03:27:21.974 INFO:teuthology.orchestra.run.smithi117.stdout: "options": "recursive,force" 2024-06-03T03:27:21.975 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:27:21.975 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:27:21.975 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-06-03T03:27:21.975 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.970+0000 7f458c600700 1 -- 172.21.15.117:0/3933532134 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f456c0a2c20 msgr2=0x7f456c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:21.975 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.970+0000 7f458c600700 1 --2- 172.21.15.117:0/3933532134 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f456c0a2c20 0x7f456c0a50e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4584108e90 tx=0x7f4570009260 comp rx=0 tx=0).stop 2024-06-03T03:27:21.975 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.970+0000 7f458c600700 1 -- 172.21.15.117:0/3933532134 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4584102740 msgr2=0x7f458419ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:21.975 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.970+0000 7f458c600700 1 --2- 172.21.15.117:0/3933532134 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4584102740 0x7f458419ad20 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f457800ec10 tx=0x7f457800ef20 comp rx=0 tx=0).stop 2024-06-03T03:27:21.976 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.970+0000 7f458c600700 1 -- 172.21.15.117:0/3933532134 shutdown_connections 2024-06-03T03:27:21.976 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.970+0000 7f458c600700 1 --2- 172.21.15.117:0/3933532134 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f456c0a2c20 0x7f456c0a50e0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:21.976 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.970+0000 7f458c600700 1 --2- 172.21.15.117:0/3933532134 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4584107840 0x7f458419f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:21.976 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.970+0000 7f458c600700 1 --2- 172.21.15.117:0/3933532134 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4584102740 0x7f458419ad20 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:21.976 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.970+0000 7f458c600700 1 --2- 172.21.15.117:0/3933532134 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45841030f0 0x7f458419b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:21.976 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.970+0000 7f458c600700 1 -- 172.21.15.117:0/3933532134 >> 172.21.15.117:0/3933532134 conn(0x7f45840795f0 msgr2=0x7f4584100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:27:21.976 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.971+0000 7f458c600700 1 -- 172.21.15.117:0/3933532134 shutdown_connections 2024-06-03T03:27:21.976 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:21.971+0000 7f458c600700 1 -- 172.21.15.117:0/3933532134 wait complete. 2024-06-03T03:27:22.696 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:27:22 smithi117 ceph-mon[31649]: scrub summary: idle 2024-06-03T03:27:22.793 INFO:tasks.cephfs.filesystem:scrub status for tag:7ac9de29-438a-45f0-839f-bb9b5c7f7030 - {'path': '/', 'tag': '7ac9de29-438a-45f0-839f-bb9b5c7f7030', 'options': 'recursive,force'} 2024-06-03T03:27:22.906 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:27:22 smithi151 ceph-mon[37043]: scrub summary: idle 2024-06-03T03:27:22.981 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:27:22 smithi183 ceph-mon[37326]: scrub summary: idle 2024-06-03T03:27:23.906 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:27:23 smithi151 ceph-mon[37043]: pgmap v568: 129 pgs: 129 active+clean; 1.9 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 864 KiB/s wr, 18 op/s 2024-06-03T03:27:23.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:27:23 smithi117 ceph-mon[31649]: pgmap v568: 129 pgs: 129 active+clean; 1.9 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 864 KiB/s wr, 18 op/s 2024-06-03T03:27:23.981 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:27:23 smithi183 ceph-mon[37326]: pgmap v568: 129 pgs: 129 active+clean; 1.9 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 864 KiB/s wr, 18 op/s 2024-06-03T03:27:25.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:27:25 smithi151 ceph-mon[37043]: pgmap v569: 129 pgs: 129 active+clean; 1.9 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 1.3 MiB/s wr, 16 op/s 2024-06-03T03:27:25.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:27:25 smithi117 ceph-mon[31649]: pgmap v569: 129 pgs: 129 active+clean; 1.9 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 1.3 MiB/s wr, 16 op/s 2024-06-03T03:27:25.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:27:25 smithi183 ceph-mon[37326]: pgmap v569: 129 pgs: 129 active+clean; 1.9 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 1.3 MiB/s wr, 16 op/s 2024-06-03T03:27:27.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:27:27 smithi151 ceph-mon[37043]: pgmap v570: 129 pgs: 129 active+clean; 1.9 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 860 KiB/s wr, 14 op/s 2024-06-03T03:27:27.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:27:27 smithi117 ceph-mon[31649]: pgmap v570: 129 pgs: 129 active+clean; 1.9 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 860 KiB/s wr, 14 op/s 2024-06-03T03:27:27.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:27:27 smithi183 ceph-mon[37326]: pgmap v570: 129 pgs: 129 active+clean; 1.9 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 860 KiB/s wr, 14 op/s 2024-06-03T03:27:28.041 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.040+0000 7fcc25800640 1 -- 172.21.15.117:0/3021351580 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc2010dcc0 msgr2=0x7fcc2010e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:28.041 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.040+0000 7fcc25800640 1 --2- 172.21.15.117:0/3021351580 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc2010dcc0 0x7fcc2010e140 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fcc10009ca0 tx=0x7fcc1002a8c0 comp rx=0 tx=0).stop 2024-06-03T03:27:28.041 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.041+0000 7fcc25800640 1 -- 172.21.15.117:0/3021351580 shutdown_connections 2024-06-03T03:27:28.042 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.041+0000 7fcc25800640 1 --2- 172.21.15.117:0/3021351580 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc2010dcc0 0x7fcc2010e140 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:28.042 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.041+0000 7fcc25800640 1 --2- 172.21.15.117:0/3021351580 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc2010b310 0x7fcc2010d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:28.042 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.041+0000 7fcc25800640 1 --2- 172.21.15.117:0/3021351580 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcc201089a0 0x7fcc2010add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:28.042 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.041+0000 7fcc25800640 1 -- 172.21.15.117:0/3021351580 >> 172.21.15.117:0/3021351580 conn(0x7fcc2006ea10 msgr2=0x7fcc20070e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:27:28.043 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.042+0000 7fcc25800640 1 -- 172.21.15.117:0/3021351580 shutdown_connections 2024-06-03T03:27:28.043 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.043+0000 7fcc25800640 1 -- 172.21.15.117:0/3021351580 wait complete. 2024-06-03T03:27:28.044 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.044+0000 7fcc25800640 1 Processor -- start 2024-06-03T03:27:28.044 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.044+0000 7fcc25800640 1 -- start start 2024-06-03T03:27:28.045 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.044+0000 7fcc25800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc201089a0 0x7fcc201a6a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:28.045 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.045+0000 7fcc25800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc2010b310 0x7fcc201a6f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:28.045 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.045+0000 7fcc25800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcc2010dcc0 0x7fcc201adf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:28.045 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.045+0000 7fcc25800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2011c4e0 con 0x7fcc201089a0 2024-06-03T03:27:28.045 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.045+0000 7fcc25800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2011c360 con 0x7fcc2010dcc0 2024-06-03T03:27:28.045 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.045+0000 7fcc25800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2011c660 con 0x7fcc2010b310 2024-06-03T03:27:28.045 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.045+0000 7fcc1f400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcc2010dcc0 0x7fcc201adf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.045+0000 7fcc1f400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcc2010dcc0 0x7fcc201adf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:40124/0 (socket says 172.21.15.117:40124) 2024-06-03T03:27:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.045+0000 7fcc1e000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc2010b310 0x7fcc201a6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.045+0000 7fcc1f400640 1 -- 172.21.15.117:0/1151016934 learned_addr learned my addr 172.21.15.117:0/1151016934 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:27:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.045+0000 7fcc1ea00640 1 --2- 172.21.15.117:0/1151016934 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc201089a0 0x7fcc201a6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.045+0000 7fcc1e000640 1 -- 172.21.15.117:0/1151016934 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcc2010dcc0 msgr2=0x7fcc201adf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.045+0000 7fcc1e000640 1 --2- 172.21.15.117:0/1151016934 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcc2010dcc0 0x7fcc201adf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.045+0000 7fcc1e000640 1 -- 172.21.15.117:0/1151016934 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc201089a0 msgr2=0x7fcc201a6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.046+0000 7fcc1e000640 1 --2- 172.21.15.117:0/1151016934 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc201089a0 0x7fcc201a6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.046+0000 7fcc1e000640 1 -- 172.21.15.117:0/1151016934 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc201ae5e0 con 0x7fcc2010b310 2024-06-03T03:27:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.046+0000 7fcc1f400640 1 --2- 172.21.15.117:0/1151016934 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcc2010dcc0 0x7fcc201adf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:27:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.046+0000 7fcc1ea00640 1 --2- 172.21.15.117:0/1151016934 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc201089a0 0x7fcc201a6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:27:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.046+0000 7fcc1e000640 1 --2- 172.21.15.117:0/1151016934 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc2010b310 0x7fcc201a6f50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fcc1800d6d0 tx=0x7fcc1800db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:27:28.048 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.046+0000 7fcc17400640 1 -- 172.21.15.117:0/1151016934 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc18017070 con 0x7fcc2010b310 2024-06-03T03:27:28.048 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.046+0000 7fcc17400640 1 -- 172.21.15.117:0/1151016934 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc18009820 con 0x7fcc2010b310 2024-06-03T03:27:28.048 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.046+0000 7fcc17400640 1 -- 172.21.15.117:0/1151016934 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc18004d00 con 0x7fcc2010b310 2024-06-03T03:27:28.048 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.048+0000 7fcc25800640 1 -- 172.21.15.117:0/1151016934 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc201ae8d0 con 0x7fcc2010b310 2024-06-03T03:27:28.049 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.048+0000 7fcc25800640 1 -- 172.21.15.117:0/1151016934 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc201aeda0 con 0x7fcc2010b310 2024-06-03T03:27:28.053 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.053+0000 7fcc285cd740 1 Processor -- start 2024-06-03T03:27:28.058 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.057+0000 7fcc285cd740 1 -- start start 2024-06-03T03:27:28.058 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.057+0000 7fcc285cd740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fc54cb0bd0 0x55fc54cb0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:28.058 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.057+0000 7fcc285cd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc54cb15c0 0x55fc54cb60c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:28.059 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.057+0000 7fcc285cd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc54cb6730 0x55fc54cb8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:28.059 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.057+0000 7fcc285cd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55fc54c14420 con 0x55fc54cb15c0 2024-06-03T03:27:28.059 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.057+0000 7fcc285cd740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55fc54bbbc50 con 0x55fc54cb0bd0 2024-06-03T03:27:28.059 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.057+0000 7fcc285cd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55fc54c103e0 con 0x55fc54cb6730 2024-06-03T03:27:28.059 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.058+0000 7fcc1f400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fc54cb0bd0 0x55fc54cb0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:28.060 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.058+0000 7fcc1f400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fc54cb0bd0 0x55fc54cb0ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:40132/0 (socket says 172.21.15.117:40132) 2024-06-03T03:27:28.060 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.058+0000 7fcc1f400640 1 -- 172.21.15.117:0/2810313194 learned_addr learned my addr 172.21.15.117:0/2810313194 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:27:28.060 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.059+0000 7fcc1ea00640 1 --2- 172.21.15.117:0/2810313194 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc54cb15c0 0x55fc54cb60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:28.060 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.059+0000 7fcc1f400640 1 -- 172.21.15.117:0/2810313194 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc54cb6730 msgr2=0x55fc54cb8bc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-03T03:27:28.060 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.059+0000 7fcc1f400640 1 --2- 172.21.15.117:0/2810313194 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc54cb6730 0x55fc54cb8bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:28.060 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.059+0000 7fcc1f400640 1 -- 172.21.15.117:0/2810313194 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc54cb15c0 msgr2=0x55fc54cb60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:28.060 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.059+0000 7fcc1f400640 1 --2- 172.21.15.117:0/2810313194 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc54cb15c0 0x55fc54cb60c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:28.060 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.059+0000 7fcc1f400640 1 -- 172.21.15.117:0/2810313194 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc54928210 con 0x55fc54cb0bd0 2024-06-03T03:27:28.060 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.059+0000 7fcc1f400640 1 --2- 172.21.15.117:0/2810313194 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fc54cb0bd0 0x55fc54cb0ff0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fcc1002f530 tx=0x7fcc10009800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c729fec60f89eb2 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:27:28.061 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.060+0000 7fcc17400640 1 -- 172.21.15.117:0/1151016934 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fcc180099d0 con 0x7fcc2010b310 2024-06-03T03:27:28.061 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.060+0000 7fcc0fe00640 1 -- 172.21.15.117:0/2810313194 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc10031650 con 0x55fc54cb0bd0 2024-06-03T03:27:28.061 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.060+0000 7fcc0fe00640 1 -- 172.21.15.117:0/2810313194 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc10036040 con 0x55fc54cb0bd0 2024-06-03T03:27:28.061 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.060+0000 7fcc0fe00640 1 -- 172.21.15.117:0/2810313194 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc10004810 con 0x55fc54cb0bd0 2024-06-03T03:27:28.062 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.061+0000 7fcc285cd740 1 -- 172.21.15.117:0/2810313194 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fc54cb0bd0 msgr2=0x55fc54cb0ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:28.062 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.061+0000 7fcc285cd740 1 --2- 172.21.15.117:0/2810313194 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fc54cb0bd0 0x55fc54cb0ff0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fcc1002f530 tx=0x7fcc10009800 comp rx=0 tx=0).stop 2024-06-03T03:27:28.062 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.062+0000 7fcc285cd740 1 -- 172.21.15.117:0/2810313194 shutdown_connections 2024-06-03T03:27:28.062 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.062+0000 7fcc285cd740 1 --2- 172.21.15.117:0/2810313194 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc54cb6730 0x55fc54cb8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:28.062 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.062+0000 7fcc285cd740 1 --2- 172.21.15.117:0/2810313194 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc54cb15c0 0x55fc54cb60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:28.062 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.062+0000 7fcc285cd740 1 --2- 172.21.15.117:0/2810313194 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fc54cb0bd0 0x55fc54cb0ff0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:28.062 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.062+0000 7fcc285cd740 1 -- 172.21.15.117:0/2810313194 >> 172.21.15.117:0/2810313194 conn(0x55fc54c78f80 msgr2=0x55fc54cac650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:27:28.063 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.062+0000 7fcc285cd740 1 -- 172.21.15.117:0/2810313194 shutdown_connections 2024-06-03T03:27:28.063 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.062+0000 7fcc285cd740 1 -- 172.21.15.117:0/2810313194 wait complete. 2024-06-03T03:27:28.063 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.063+0000 7fcc285cd740 1 -- start start 2024-06-03T03:27:28.064 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.064+0000 7fcc285cd740 1 Processor -- start 2024-06-03T03:27:28.072 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.067+0000 7fcc17400640 1 --2- 172.21.15.117:0/1151016934 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fcc040a2bd0 0x7fcc040a5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:28.072 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.068+0000 7fcc17400640 1 -- 172.21.15.117:0/1151016934 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7fcc180cefa0 con 0x7fcc2010b310 2024-06-03T03:27:28.072 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.070+0000 7fcc285cd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc54cb0bd0 0x55fc54cbae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:28.072 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.070+0000 7fcc1f400640 1 --2- 172.21.15.117:0/1151016934 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fcc040a2bd0 0x7fcc040a5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:28.073 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.070+0000 7fcc285cd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc54cb15c0 0x55fc54cbb350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:28.073 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.070+0000 7fcc285cd740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fc54cb6730 0x55fc54cc2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:28.073 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.070+0000 7fcc285cd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55fc54cb9500 con 0x55fc54cb0bd0 2024-06-03T03:27:28.073 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.070+0000 7fcc285cd740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55fc54bcf0b0 con 0x55fc54cb6730 2024-06-03T03:27:28.073 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.070+0000 7fcc285cd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55fc54cb9680 con 0x55fc54cb15c0 2024-06-03T03:27:28.073 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.070+0000 7fcc285cd740 10 client.0 _ll_register_callbacks cb 0x55fc54bbaf40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:27:28.073 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.070+0000 7fcc1f400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fc54cb6730 0x55fc54cc2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:28.073 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.070+0000 7fcc1e000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc54cb15c0 0x55fc54cbb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:28.073 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.070+0000 7fcc1f400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fc54cb6730 0x55fc54cc2390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:40146/0 (socket says 172.21.15.117:40146) 2024-06-03T03:27:28.073 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.070+0000 7fcc1f400640 1 -- 172.21.15.117:0/856634730 learned_addr learned my addr 172.21.15.117:0/856634730 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:27:28.073 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.071+0000 7fcc1ea00640 1 --2- 172.21.15.117:0/856634730 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc54cb0bd0 0x55fc54cbae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:28.073 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.071+0000 7fcc1e000640 1 -- 172.21.15.117:0/856634730 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fc54cb6730 msgr2=0x55fc54cc2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:28.073 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.071+0000 7fcc1e000640 1 --2- 172.21.15.117:0/856634730 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fc54cb6730 0x55fc54cc2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:28.074 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.071+0000 7fcc1e000640 1 -- 172.21.15.117:0/856634730 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc54cb0bd0 msgr2=0x55fc54cbae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:28.074 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.071+0000 7fcc1e000640 1 --2- 172.21.15.117:0/856634730 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc54cb0bd0 0x55fc54cbae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:28.074 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.071+0000 7fcc1e000640 1 -- 172.21.15.117:0/856634730 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc549f4620 con 0x55fc54cb15c0 2024-06-03T03:27:28.074 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.071+0000 7fcc1ea00640 1 --2- 172.21.15.117:0/856634730 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc54cb0bd0 0x55fc54cbae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T03:27:28.074 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.071+0000 7fcc1f400640 1 --2- 172.21.15.117:0/1151016934 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fcc040a2bd0 0x7fcc040a5090 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fcc10009620 tx=0x7fcc10004770 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:27:28.074 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.071+0000 7fcc1e000640 1 --2- 172.21.15.117:0/856634730 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc54cb15c0 0x55fc54cbb350 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fcc180cc6b0 tx=0x7fcc1809ad70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9cc75d055679223 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:27:28.076 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.073+0000 7fcc0cc00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-06-03T03:27:28.076 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.073+0000 7fcc0cc00640 1 -- 172.21.15.117:0/856634730 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc1809f7c0 con 0x55fc54cb15c0 2024-06-03T03:27:28.076 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.073+0000 7fcc0cc00640 1 -- 172.21.15.117:0/856634730 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7fcc18097a80 con 0x55fc54cb15c0 2024-06-03T03:27:28.076 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.073+0000 7fcc285cd740 1 -- 172.21.15.117:0/856634730 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc10036040 con 0x55fc54cb15c0 2024-06-03T03:27:28.076 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.073+0000 7fcc0cc00640 1 -- 172.21.15.117:0/856634730 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fcc180a4080 con 0x55fc54cb15c0 2024-06-03T03:27:28.076 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.073+0000 7fcc285cd740 10 client.25735 fetch_fsmap learned FSMap version 31 2024-06-03T03:27:28.076 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.073+0000 7fcc285cd740 1 -- 172.21.15.117:0/856634730 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55fc54cc5590 con 0x55fc54cb15c0 2024-06-03T03:27:28.076 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.073+0000 7fcc0cc00640 1 -- 172.21.15.117:0/856634730 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc10036040 con 0x55fc54cb15c0 2024-06-03T03:27:28.076 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.073+0000 7fcc0cc00640 1 -- 172.21.15.117:0/856634730 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fcc1809a860 con 0x55fc54cb15c0 2024-06-03T03:27:28.076 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.074+0000 7fcc285cd740 10 client.25735 fetch_fsmap finished waiting for FSMap version 31 2024-06-03T03:27:28.076 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.074+0000 7fcc285cd740 10 client.25735 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-03T03:27:28.076 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.074+0000 7fcc285cd740 20 client.25735 populate_metadata read hostname 'smithi117' 2024-06-03T03:27:28.077 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.074+0000 7fcc285cd740 1 --2- 172.21.15.117:0/856634730 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55fc54cc6c60 0x55fc54cc9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:28.077 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.074+0000 7fcc285cd740 4 client.25735 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T03:27:28.077 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.074+0000 7fcc285cd740 1 -- 172.21.15.117:0/856634730 --> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc54c21b50 con 0x55fc54cc6c60 2024-06-03T03:27:28.077 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.074+0000 7fcc1ea00640 1 --2- 172.21.15.117:0/856634730 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55fc54cc6c60 0x55fc54cc9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:28.077 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.075+0000 7fcc1ea00640 1 --2- 172.21.15.117:0/856634730 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55fc54cc6c60 0x55fc54cc9050 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2aa82d1f63b4d8a9 server_cookie=443588289c2761c5 in_seq=0 out_seq=0 2024-06-03T03:27:28.077 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.075+0000 7fcc0cc00640 10 client.25735 ms_handle_connect on v2:172.21.15.183:6838/1439615083 2024-06-03T03:27:28.077 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.075+0000 7fcc0cc00640 1 -- 172.21.15.117:0/856634730 <== mds.0 v2:172.21.15.183:6838/1439615083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55fc54c21b50 con 0x55fc54cc6c60 2024-06-03T03:27:28.077 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.075+0000 7fcc0cc00640 10 client.25735 handle_command_reply: tid=0 2024-06-03T03:27:28.077 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.076+0000 7fcc285cd740 1 client.25735 shutdown 2024-06-03T03:27:28.078 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.076+0000 7fcc285cd740 1 -- 172.21.15.117:0/856634730 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc54cb15c0 msgr2=0x55fc54cbb350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:28.078 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.076+0000 7fcc285cd740 1 --2- 172.21.15.117:0/856634730 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc54cb15c0 0x55fc54cbb350 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fcc180cc6b0 tx=0x7fcc1809ad70 comp rx=0 tx=0).stop 2024-06-03T03:27:28.078 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.076+0000 7fcc285cd740 1 -- 172.21.15.117:0/856634730 shutdown_connections 2024-06-03T03:27:28.078 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.076+0000 7fcc285cd740 1 --2- 172.21.15.117:0/856634730 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55fc54cc6c60 0x55fc54cc9050 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:28.078 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.076+0000 7fcc285cd740 1 --2- 172.21.15.117:0/856634730 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fc54cb6730 0x55fc54cc2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:28.078 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.076+0000 7fcc285cd740 1 --2- 172.21.15.117:0/856634730 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc54cb15c0 0x55fc54cbb350 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:28.078 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.076+0000 7fcc285cd740 1 --2- 172.21.15.117:0/856634730 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc54cb0bd0 0x55fc54cbae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:28.078 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.076+0000 7fcc285cd740 1 -- 172.21.15.117:0/856634730 >> 172.21.15.117:0/856634730 conn(0x55fc54c78f80 msgr2=0x55fc54cab810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:27:28.079 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.076+0000 7fcc0cc00640 0 client.25735 ms_handle_reset on v2:172.21.15.183:6838/1439615083 2024-06-03T03:27:28.079 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.076+0000 7fcc285cd740 1 -- 172.21.15.117:0/856634730 shutdown_connections 2024-06-03T03:27:28.079 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.076+0000 7fcc285cd740 1 -- 172.21.15.117:0/856634730 wait complete. 2024-06-03T03:27:28.079 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.078+0000 7fcc285cd740 20 client.25735 trim_cache size 0 max 16384 2024-06-03T03:27:28.121 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.120+0000 7fcc285cd740 1 Processor -- start 2024-06-03T03:27:28.126 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.122+0000 7fcc285cd740 1 -- start start 2024-06-03T03:27:28.127 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.123+0000 7fcc285cd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc54cb0bd0 0x55fc54cabf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:28.127 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.123+0000 7fcc285cd740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fc54cb15c0 0x55fc54cc22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:28.127 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.124+0000 7fcc285cd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc54cb6730 0x55fc54cac4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:28.127 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.124+0000 7fcc285cd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2007deb0 con 0x55fc54cb6730 2024-06-03T03:27:28.127 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.124+0000 7fcc285cd740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2007f9f0 con 0x55fc54cb15c0 2024-06-03T03:27:28.127 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.124+0000 7fcc285cd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2007e580 con 0x55fc54cb0bd0 2024-06-03T03:27:28.129 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.129+0000 7fcc1ea00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc54cb0bd0 0x55fc54cabf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.129+0000 7fcc1ea00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc54cb0bd0 0x55fc54cabf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:33312/0 (socket says 172.21.15.117:33312) 2024-06-03T03:27:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.129+0000 7fcc1ea00640 1 -- 172.21.15.117:0/3026878155 learned_addr learned my addr 172.21.15.117:0/3026878155 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:27:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.129+0000 7fcc1e000640 1 --2- 172.21.15.117:0/3026878155 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fc54cb15c0 0x55fc54cc22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.130+0000 7fcc1ea00640 1 -- 172.21.15.117:0/3026878155 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fc54cb15c0 msgr2=0x55fc54cc22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.130+0000 7fcc1ea00640 1 --2- 172.21.15.117:0/3026878155 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fc54cb15c0 0x55fc54cc22b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.130+0000 7fcc1ea00640 1 -- 172.21.15.117:0/3026878155 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc54cb6730 msgr2=0x55fc54cac4d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-03T03:27:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.130+0000 7fcc1ea00640 1 --2- 172.21.15.117:0/3026878155 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc54cb6730 0x55fc54cac4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.130+0000 7fcc1ea00640 1 -- 172.21.15.117:0/3026878155 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc54cb1e50 con 0x55fc54cb0bd0 2024-06-03T03:27:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.130+0000 7fcc1ea00640 1 --2- 172.21.15.117:0/3026878155 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc54cb0bd0 0x55fc54cabf90 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fcc08005d00 tx=0x7fcc08013890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e27abd3f4655aba8 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:27:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.131+0000 7fcc0f400640 1 -- 172.21.15.117:0/3026878155 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc08023030 con 0x55fc54cb0bd0 2024-06-03T03:27:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.132+0000 7fcc0f400640 1 -- 172.21.15.117:0/3026878155 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc08005020 con 0x55fc54cb0bd0 2024-06-03T03:27:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.132+0000 7fcc0f400640 1 -- 172.21.15.117:0/3026878155 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc080049e0 con 0x55fc54cb0bd0 2024-06-03T03:27:28.134 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.133+0000 7fcc285cd740 1 -- 172.21.15.117:0/3026878155 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc54cb0bd0 msgr2=0x55fc54cabf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:28.134 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.133+0000 7fcc285cd740 1 --2- 172.21.15.117:0/3026878155 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc54cb0bd0 0x55fc54cabf90 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fcc08005d00 tx=0x7fcc08013890 comp rx=0 tx=0).stop 2024-06-03T03:27:28.134 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.134+0000 7fcc285cd740 1 -- 172.21.15.117:0/3026878155 shutdown_connections 2024-06-03T03:27:28.134 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.134+0000 7fcc285cd740 1 --2- 172.21.15.117:0/3026878155 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc54cb6730 0x55fc54cac4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:28.134 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.134+0000 7fcc285cd740 1 --2- 172.21.15.117:0/3026878155 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fc54cb15c0 0x55fc54cc22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:28.134 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.134+0000 7fcc285cd740 1 --2- 172.21.15.117:0/3026878155 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc54cb0bd0 0x55fc54cabf90 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:28.135 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.134+0000 7fcc285cd740 1 -- 172.21.15.117:0/3026878155 >> 172.21.15.117:0/3026878155 conn(0x55fc54c78f80 msgr2=0x55fc54cb2980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:27:28.135 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.134+0000 7fcc285cd740 1 -- 172.21.15.117:0/3026878155 shutdown_connections 2024-06-03T03:27:28.135 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.134+0000 7fcc285cd740 1 -- 172.21.15.117:0/3026878155 wait complete. 2024-06-03T03:27:28.135 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.135+0000 7fcc285cd740 1 -- start start 2024-06-03T03:27:28.135 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.135+0000 7fcc285cd740 1 Processor -- start 2024-06-03T03:27:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.135+0000 7fcc285cd740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fc54cb0bd0 0x55fc54cbefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.135+0000 7fcc285cd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc54cb15c0 0x55fc54cbf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.135+0000 7fcc285cd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc54cb6730 0x55fc54d2f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.135+0000 7fcc285cd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2007f320 con 0x55fc54cb15c0 2024-06-03T03:27:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.135+0000 7fcc285cd740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2007ec50 con 0x55fc54cb0bd0 2024-06-03T03:27:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.135+0000 7fcc285cd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2007be90 con 0x55fc54cb6730 2024-06-03T03:27:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.136+0000 7fcc285cd740 10 client.0 _ll_register_callbacks cb 0x7fcc180057f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:27:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.137+0000 7fcc1ea00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fc54cb0bd0 0x55fc54cbefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.137+0000 7fcc1f400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc54cb6730 0x55fc54d2f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:28.138 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.137+0000 7fcc1f400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc54cb6730 0x55fc54d2f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:33322/0 (socket says 172.21.15.117:33322) 2024-06-03T03:27:28.138 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.137+0000 7fcc1f400640 1 -- 172.21.15.117:0/520172476 learned_addr learned my addr 172.21.15.117:0/520172476 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:27:28.138 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.138+0000 7fcc1ea00640 1 -- 172.21.15.117:0/520172476 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc54cb6730 msgr2=0x55fc54d2f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:28.138 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.138+0000 7fcc1ea00640 1 --2- 172.21.15.117:0/520172476 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc54cb6730 0x55fc54d2f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:28.138 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.138+0000 7fcc1ea00640 1 -- 172.21.15.117:0/520172476 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc54cb15c0 msgr2=0x55fc54cbf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:28.139 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.138+0000 7fcc1ea00640 1 --2- 172.21.15.117:0/520172476 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc54cb15c0 0x55fc54cbf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:28.139 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.138+0000 7fcc1ea00640 1 -- 172.21.15.117:0/520172476 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc54cc2590 con 0x55fc54cb0bd0 2024-06-03T03:27:28.139 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.138+0000 7fcc1ea00640 1 --2- 172.21.15.117:0/520172476 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fc54cb0bd0 0x55fc54cbefd0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fcc08000c00 tx=0x7fcc08012140 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94422cc2380adb58 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:27:28.140 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.138+0000 7fcc0cc00640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-06-03T03:27:28.140 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.139+0000 7fcc0cc00640 1 -- 172.21.15.117:0/520172476 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc08023050 con 0x55fc54cb0bd0 2024-06-03T03:27:28.140 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.139+0000 7fcc0cc00640 1 -- 172.21.15.117:0/520172476 <== mon.2 v2:172.21.15.151:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7fcc080053d0 con 0x55fc54cb0bd0 2024-06-03T03:27:28.140 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.139+0000 7fcc285cd740 1 -- 172.21.15.117:0/520172476 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc08005020 con 0x55fc54cb0bd0 2024-06-03T03:27:28.140 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.139+0000 7fcc0cc00640 1 -- 172.21.15.117:0/520172476 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fcc080055d0 con 0x55fc54cb0bd0 2024-06-03T03:27:28.141 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.141+0000 7fcc285cd740 10 client.25706 fetch_fsmap learned FSMap version 31 2024-06-03T03:27:28.141 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.141+0000 7fcc285cd740 1 -- 172.21.15.117:0/520172476 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55fc54d30d10 con 0x55fc54cb0bd0 2024-06-03T03:27:28.142 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.142+0000 7fcc0cc00640 1 -- 172.21.15.117:0/520172476 <== mon.2 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc08005020 con 0x55fc54cb0bd0 2024-06-03T03:27:28.142 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.142+0000 7fcc0cc00640 1 -- 172.21.15.117:0/520172476 <== mon.2 v2:172.21.15.151:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fcc0802f020 con 0x55fc54cb0bd0 2024-06-03T03:27:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.142+0000 7fcc285cd740 10 client.25706 fetch_fsmap finished waiting for FSMap version 31 2024-06-03T03:27:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.143+0000 7fcc285cd740 10 client.25706 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-03T03:27:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.143+0000 7fcc285cd740 20 client.25706 populate_metadata read hostname 'smithi117' 2024-06-03T03:27:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.143+0000 7fcc285cd740 1 --2- 172.21.15.117:0/520172476 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55fc54d315b0 0x55fc54d339d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.143+0000 7fcc285cd740 4 client.25706 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-03T03:27:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.143+0000 7fcc285cd740 1 -- 172.21.15.117:0/520172476 --> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fc54c1bea0 con 0x55fc54d315b0 2024-06-03T03:27:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.143+0000 7fcc1e000640 1 --2- 172.21.15.117:0/520172476 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55fc54d315b0 0x55fc54d339d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.144+0000 7fcc1e000640 1 --2- 172.21.15.117:0/520172476 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55fc54d315b0 0x55fc54d339d0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f03caca27df01f9 server_cookie=82c880c6e7dd1e92 in_seq=0 out_seq=0 2024-06-03T03:27:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.144+0000 7fcc0cc00640 10 client.25706 ms_handle_connect on v2:172.21.15.183:6838/1439615083 2024-06-03T03:27:28.194 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.194+0000 7fcc0cc00640 1 -- 172.21.15.117:0/520172476 <== mds.0 v2:172.21.15.183:6838/1439615083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55fc54c1bea0 con 0x55fc54d315b0 2024-06-03T03:27:28.195 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.194+0000 7fcc0cc00640 10 client.25706 handle_command_reply: tid=0 2024-06-03T03:27:28.195 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.194+0000 7fcc285cd740 1 client.25706 shutdown 2024-06-03T03:27:28.196 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.194+0000 7fcc285cd740 1 -- 172.21.15.117:0/520172476 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fc54cb0bd0 msgr2=0x55fc54cbefd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:28.196 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.194+0000 7fcc285cd740 1 --2- 172.21.15.117:0/520172476 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fc54cb0bd0 0x55fc54cbefd0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fcc08000c00 tx=0x7fcc08012140 comp rx=0 tx=0).stop 2024-06-03T03:27:28.196 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.195+0000 7fcc285cd740 1 -- 172.21.15.117:0/520172476 shutdown_connections 2024-06-03T03:27:28.196 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.195+0000 7fcc285cd740 1 --2- 172.21.15.117:0/520172476 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55fc54d315b0 0x55fc54d339d0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:28.196 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.195+0000 7fcc0cc00640 0 client.25706 ms_handle_reset on v2:172.21.15.183:6838/1439615083 2024-06-03T03:27:28.196 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.195+0000 7fcc285cd740 1 --2- 172.21.15.117:0/520172476 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc54cb6730 0x55fc54d2f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:28.196 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.195+0000 7fcc285cd740 1 --2- 172.21.15.117:0/520172476 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc54cb15c0 0x55fc54cbf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:28.196 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.195+0000 7fcc285cd740 1 --2- 172.21.15.117:0/520172476 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fc54cb0bd0 0x55fc54cbefd0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:28.196 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.195+0000 7fcc285cd740 1 -- 172.21.15.117:0/520172476 >> 172.21.15.117:0/520172476 conn(0x55fc54c78f80 msgr2=0x55fc54caf620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:27:28.196 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.195+0000 7fcc285cd740 1 -- 172.21.15.117:0/520172476 shutdown_connections 2024-06-03T03:27:28.197 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.195+0000 7fcc285cd740 1 -- 172.21.15.117:0/520172476 wait complete. 2024-06-03T03:27:28.197 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.195+0000 7fcc285cd740 20 client.25706 trim_cache size 0 max 16384 2024-06-03T03:27:28.197 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:{ 2024-06-03T03:27:28.197 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "return_code": 0 2024-06-03T03:27:28.197 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:} 2024-06-03T03:27:28.199 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.198+0000 7fcc25800640 1 -- 172.21.15.117:0/1151016934 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fcc040a2bd0 msgr2=0x7fcc040a5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:28.199 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.198+0000 7fcc25800640 1 --2- 172.21.15.117:0/1151016934 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fcc040a2bd0 0x7fcc040a5090 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fcc10009620 tx=0x7fcc10004770 comp rx=0 tx=0).stop 2024-06-03T03:27:28.199 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.198+0000 7fcc25800640 1 -- 172.21.15.117:0/1151016934 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc2010b310 msgr2=0x7fcc201a6f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:28.199 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.198+0000 7fcc25800640 1 --2- 172.21.15.117:0/1151016934 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc2010b310 0x7fcc201a6f50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fcc1800d6d0 tx=0x7fcc1800db90 comp rx=0 tx=0).stop 2024-06-03T03:27:28.201 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.199+0000 7fcc25800640 1 -- 172.21.15.117:0/1151016934 shutdown_connections 2024-06-03T03:27:28.201 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.199+0000 7fcc25800640 1 --2- 172.21.15.117:0/1151016934 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fcc040a2bd0 0x7fcc040a5090 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:28.201 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.199+0000 7fcc25800640 1 --2- 172.21.15.117:0/1151016934 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcc2010dcc0 0x7fcc201adf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:28.201 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.199+0000 7fcc25800640 1 --2- 172.21.15.117:0/1151016934 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc2010b310 0x7fcc201a6f50 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:28.201 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.199+0000 7fcc25800640 1 --2- 172.21.15.117:0/1151016934 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc201089a0 0x7fcc201a6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:28.201 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.199+0000 7fcc25800640 1 -- 172.21.15.117:0/1151016934 >> 172.21.15.117:0/1151016934 conn(0x7fcc2006ea10 msgr2=0x7fcc2010c140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:27:28.202 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.201+0000 7fcc25800640 1 -- 172.21.15.117:0/1151016934 shutdown_connections 2024-06-03T03:27:28.202 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:28.202+0000 7fcc25800640 1 -- 172.21.15.117:0/1151016934 wait complete. 2024-06-03T03:27:29.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:27:29 smithi151 ceph-mon[37043]: pgmap v571: 129 pgs: 129 active+clean; 9.9 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 1.5 MiB/s wr, 17 op/s 2024-06-03T03:27:29.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:27:29 smithi117 ceph-mon[31649]: pgmap v571: 129 pgs: 129 active+clean; 9.9 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 1.5 MiB/s wr, 17 op/s 2024-06-03T03:27:29.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:27:29 smithi183 ceph-mon[37326]: pgmap v571: 129 pgs: 129 active+clean; 9.9 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 1.5 MiB/s wr, 17 op/s 2024-06-03T03:27:31.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:27:31 smithi151 ceph-mon[37043]: pgmap v572: 129 pgs: 129 active+clean; 142 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 21 op/s 2024-06-03T03:27:31.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:27:31 smithi117 ceph-mon[31649]: pgmap v572: 129 pgs: 129 active+clean; 142 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 21 op/s 2024-06-03T03:27:31.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:27:31 smithi183 ceph-mon[37326]: pgmap v572: 129 pgs: 129 active+clean; 142 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 21 op/s 2024-06-03T03:27:33.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:27:33 smithi151 ceph-mon[37043]: pgmap v573: 129 pgs: 129 active+clean; 254 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 22 MiB/s wr, 31 op/s 2024-06-03T03:27:33.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:27:33 smithi117 ceph-mon[31649]: pgmap v573: 129 pgs: 129 active+clean; 254 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 22 MiB/s wr, 31 op/s 2024-06-03T03:27:33.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:27:33 smithi183 ceph-mon[37326]: pgmap v573: 129 pgs: 129 active+clean; 254 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 22 MiB/s wr, 31 op/s 2024-06-03T03:27:35.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:27:35 smithi151 ceph-mon[37043]: pgmap v574: 129 pgs: 129 active+clean; 258 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 MiB/s wr, 28 op/s 2024-06-03T03:27:35.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:27:35 smithi117 ceph-mon[31649]: pgmap v574: 129 pgs: 129 active+clean; 258 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 MiB/s wr, 28 op/s 2024-06-03T03:27:35.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:27:35 smithi183 ceph-mon[37326]: pgmap v574: 129 pgs: 129 active+clean; 258 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 MiB/s wr, 28 op/s 2024-06-03T03:27:37.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:27:37 smithi151 ceph-mon[37043]: pgmap v575: 129 pgs: 129 active+clean; 258 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 MiB/s wr, 27 op/s 2024-06-03T03:27:37.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:27:37 smithi117 ceph-mon[31649]: pgmap v575: 129 pgs: 129 active+clean; 258 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 MiB/s wr, 27 op/s 2024-06-03T03:27:37.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:27:37 smithi183 ceph-mon[37326]: pgmap v575: 129 pgs: 129 active+clean; 258 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 MiB/s wr, 27 op/s 2024-06-03T03:27:39.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:27:39 smithi151 ceph-mon[37043]: pgmap v576: 129 pgs: 129 active+clean; 258 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 MiB/s wr, 31 op/s 2024-06-03T03:27:39.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:27:39 smithi117 ceph-mon[31649]: pgmap v576: 129 pgs: 129 active+clean; 258 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 MiB/s wr, 31 op/s 2024-06-03T03:27:39.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:27:39 smithi183 ceph-mon[37326]: pgmap v576: 129 pgs: 129 active+clean; 258 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 MiB/s wr, 31 op/s 2024-06-03T03:27:41.349 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.348+0000 7f8e73000640 1 -- 172.21.15.117:0/101315336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e6c100a50 msgr2=0x7f8e6c100eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:41.350 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.348+0000 7f8e73000640 1 --2- 172.21.15.117:0/101315336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e6c100a50 0x7f8e6c100eb0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f8e58009900 tx=0x7f8e5802a3f0 comp rx=0 tx=0).stop 2024-06-03T03:27:41.350 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.348+0000 7f8e73000640 1 -- 172.21.15.117:0/101315336 shutdown_connections 2024-06-03T03:27:41.350 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.348+0000 7f8e73000640 1 --2- 172.21.15.117:0/101315336 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e6c1013f0 0x7f8e6c101870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:41.350 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.348+0000 7f8e73000640 1 --2- 172.21.15.117:0/101315336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e6c100a50 0x7f8e6c100eb0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:41.350 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.348+0000 7f8e73000640 1 --2- 172.21.15.117:0/101315336 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8e6c1000f0 0x7f8e6c100510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:41.350 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.348+0000 7f8e73000640 1 -- 172.21.15.117:0/101315336 >> 172.21.15.117:0/101315336 conn(0x7f8e6c0fbc90 msgr2=0x7f8e6c0fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:27:41.351 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.348+0000 7f8e73000640 1 -- 172.21.15.117:0/101315336 shutdown_connections 2024-06-03T03:27:41.351 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.348+0000 7f8e73000640 1 -- 172.21.15.117:0/101315336 wait complete. 2024-06-03T03:27:41.351 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.349+0000 7f8e73000640 1 Processor -- start 2024-06-03T03:27:41.351 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.349+0000 7f8e73000640 1 -- start start 2024-06-03T03:27:41.354 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.349+0000 7f8e73000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e6c1000f0 0x7f8e6c199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:41.354 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.349+0000 7f8e73000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e6c100a50 0x7f8e6c19a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:41.354 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.350+0000 7f8e73000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8e6c1013f0 0x7f8e6c1a1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:41.354 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.350+0000 7f8e73000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8e6c10f6b0 con 0x7f8e6c100a50 2024-06-03T03:27:41.354 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.350+0000 7f8e73000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8e6c10f530 con 0x7f8e6c1000f0 2024-06-03T03:27:41.354 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.350+0000 7f8e73000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8e6c10f830 con 0x7f8e6c1013f0 2024-06-03T03:27:41.355 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.350+0000 7f8e71200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e6c100a50 0x7f8e6c19a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:41.355 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.350+0000 7f8e71200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e6c100a50 0x7f8e6c19a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:49676/0 (socket says 172.21.15.117:49676) 2024-06-03T03:27:41.355 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.350+0000 7f8e71200640 1 -- 172.21.15.117:0/1219655011 learned_addr learned my addr 172.21.15.117:0/1219655011 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:27:41.355 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.350+0000 7f8e71c00640 1 --2- 172.21.15.117:0/1219655011 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e6c1000f0 0x7f8e6c199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:41.355 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.350+0000 7f8e72600640 1 --2- 172.21.15.117:0/1219655011 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8e6c1013f0 0x7f8e6c1a1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:41.355 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.350+0000 7f8e71200640 1 -- 172.21.15.117:0/1219655011 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8e6c1013f0 msgr2=0x7f8e6c1a1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:41.355 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.350+0000 7f8e71200640 1 --2- 172.21.15.117:0/1219655011 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8e6c1013f0 0x7f8e6c1a1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:41.356 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.350+0000 7f8e71200640 1 -- 172.21.15.117:0/1219655011 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e6c1000f0 msgr2=0x7f8e6c199d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:41.356 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.350+0000 7f8e71200640 1 --2- 172.21.15.117:0/1219655011 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e6c1000f0 0x7f8e6c199d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:41.356 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.350+0000 7f8e71200640 1 -- 172.21.15.117:0/1219655011 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e6c1a1970 con 0x7f8e6c100a50 2024-06-03T03:27:41.356 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.350+0000 7f8e71c00640 1 --2- 172.21.15.117:0/1219655011 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e6c1000f0 0x7f8e6c199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:27:41.356 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.351+0000 7f8e71200640 1 --2- 172.21.15.117:0/1219655011 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e6c100a50 0x7f8e6c19a290 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f8e58031440 tx=0x7f8e58032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:27:41.356 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.351+0000 7f8e6a000640 1 -- 172.21.15.117:0/1219655011 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e5803f030 con 0x7f8e6c100a50 2024-06-03T03:27:41.356 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.351+0000 7f8e6a000640 1 -- 172.21.15.117:0/1219655011 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e58003ce0 con 0x7f8e6c100a50 2024-06-03T03:27:41.356 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.351+0000 7f8e73000640 1 -- 172.21.15.117:0/1219655011 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e6c1a1c00 con 0x7f8e6c100a50 2024-06-03T03:27:41.357 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.351+0000 7f8e6a000640 1 -- 172.21.15.117:0/1219655011 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e58035070 con 0x7f8e6c100a50 2024-06-03T03:27:41.357 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.351+0000 7f8e73000640 1 -- 172.21.15.117:0/1219655011 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e6c1a2140 con 0x7f8e6c100a50 2024-06-03T03:27:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.354+0000 7f8e6a000640 1 -- 172.21.15.117:0/1219655011 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f8e58042030 con 0x7f8e6c100a50 2024-06-03T03:27:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.355+0000 7f8e6a000640 1 --2- 172.21.15.117:0/1219655011 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f8e540a2c00 0x7f8e540a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.355+0000 7f8e775c0740 1 Processor -- start 2024-06-03T03:27:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.356+0000 7f8e6a000640 1 -- 172.21.15.117:0/1219655011 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f8e580ef830 con 0x7f8e6c100a50 2024-06-03T03:27:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.356+0000 7f8e775c0740 1 -- start start 2024-06-03T03:27:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.356+0000 7f8e71c00640 1 --2- 172.21.15.117:0/1219655011 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f8e540a2c00 0x7f8e540a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.356+0000 7f8e775c0740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a867b08d90 0x55a867b091b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.356+0000 7f8e775c0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a867b09780 0x55a867b0e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.356+0000 7f8e775c0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a867b0e8f0 0x55a867b10d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.356+0000 7f8e775c0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a867a6c420 con 0x55a867b0e8f0 2024-06-03T03:27:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.356+0000 7f8e775c0740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a867a13c50 con 0x55a867b08d90 2024-06-03T03:27:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.356+0000 7f8e775c0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a867a683e0 con 0x55a867b09780 2024-06-03T03:27:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.356+0000 7f8e72600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a867b0e8f0 0x55a867b10d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.356+0000 7f8e71200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a867b09780 0x55a867b0e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.356+0000 7f8e72600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a867b0e8f0 0x55a867b10d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:49686/0 (socket says 172.21.15.117:49686) 2024-06-03T03:27:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.356+0000 7f8e72600640 1 -- 172.21.15.117:0/1256089606 learned_addr learned my addr 172.21.15.117:0/1256089606 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:27:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.357+0000 7f8e71c00640 1 --2- 172.21.15.117:0/1219655011 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f8e540a2c00 0x7f8e540a50c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f8e60011490 tx=0x7f8e60005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:27:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.357+0000 7f8e71c00640 1 --2- 172.21.15.117:0/1256089606 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a867b08d90 0x55a867b091b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.357+0000 7f8e71200640 1 -- 172.21.15.117:0/1256089606 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a867b08d90 msgr2=0x55a867b091b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.357+0000 7f8e71200640 1 --2- 172.21.15.117:0/1256089606 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a867b08d90 0x55a867b091b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.357+0000 7f8e71200640 1 -- 172.21.15.117:0/1256089606 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a867b0e8f0 msgr2=0x55a867b10d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.357+0000 7f8e71200640 1 --2- 172.21.15.117:0/1256089606 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a867b0e8f0 0x55a867b10d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.357+0000 7f8e71200640 1 -- 172.21.15.117:0/1256089606 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a867780210 con 0x55a867b09780 2024-06-03T03:27:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.357+0000 7f8e71200640 1 --2- 172.21.15.117:0/1256089606 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a867b09780 0x55a867b0e280 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8e58006010 tx=0x7f8e580bb8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=310e41cc79cb603b server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:27:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.357+0000 7f8e5ea00640 1 -- 172.21.15.117:0/1256089606 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e58049040 con 0x55a867b09780 2024-06-03T03:27:41.367 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.357+0000 7f8e5ea00640 1 -- 172.21.15.117:0/1256089606 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e580bbca0 con 0x55a867b09780 2024-06-03T03:27:41.367 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.357+0000 7f8e5ea00640 1 -- 172.21.15.117:0/1256089606 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e5803b580 con 0x55a867b09780 2024-06-03T03:27:41.367 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.358+0000 7f8e775c0740 1 -- 172.21.15.117:0/1256089606 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a867b09780 msgr2=0x55a867b0e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:41.367 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.358+0000 7f8e775c0740 1 --2- 172.21.15.117:0/1256089606 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a867b09780 0x55a867b0e280 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8e58006010 tx=0x7f8e580bb8a0 comp rx=0 tx=0).stop 2024-06-03T03:27:41.367 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.358+0000 7f8e775c0740 1 -- 172.21.15.117:0/1256089606 shutdown_connections 2024-06-03T03:27:41.367 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.358+0000 7f8e775c0740 1 --2- 172.21.15.117:0/1256089606 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a867b0e8f0 0x55a867b10d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:41.367 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.358+0000 7f8e775c0740 1 --2- 172.21.15.117:0/1256089606 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a867b09780 0x55a867b0e280 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:41.367 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.358+0000 7f8e775c0740 1 --2- 172.21.15.117:0/1256089606 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a867b08d90 0x55a867b091b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:41.367 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.358+0000 7f8e775c0740 1 -- 172.21.15.117:0/1256089606 >> 172.21.15.117:0/1256089606 conn(0x55a867ad0f80 msgr2=0x55a867b04810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:27:41.367 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.359+0000 7f8e775c0740 1 -- 172.21.15.117:0/1256089606 shutdown_connections 2024-06-03T03:27:41.367 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.359+0000 7f8e775c0740 1 -- 172.21.15.117:0/1256089606 wait complete. 2024-06-03T03:27:41.367 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.359+0000 7f8e775c0740 1 -- start start 2024-06-03T03:27:41.368 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.359+0000 7f8e775c0740 1 Processor -- start 2024-06-03T03:27:41.368 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.360+0000 7f8e775c0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a867b08d90 0x55a867b12dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:41.368 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.360+0000 7f8e775c0740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a867b09780 0x55a867b13300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:41.368 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.360+0000 7f8e775c0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a867b0e8f0 0x55a867b1a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:41.368 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.360+0000 7f8e775c0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a867b11660 con 0x55a867b0e8f0 2024-06-03T03:27:41.368 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.360+0000 7f8e775c0740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a867a270b0 con 0x55a867b09780 2024-06-03T03:27:41.368 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.360+0000 7f8e775c0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a867b117e0 con 0x55a867b08d90 2024-06-03T03:27:41.368 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.360+0000 7f8e775c0740 10 client.0 _ll_register_callbacks cb 0x55a867a12f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:27:41.368 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.360+0000 7f8e72600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a867b0e8f0 0x55a867b1a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:41.369 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.360+0000 7f8e72600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a867b0e8f0 0x55a867b1a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:49688/0 (socket says 172.21.15.117:49688) 2024-06-03T03:27:41.369 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.360+0000 7f8e72600640 1 -- 172.21.15.117:0/2390970733 learned_addr learned my addr 172.21.15.117:0/2390970733 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:27:41.369 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.360+0000 7f8e71200640 1 --2- 172.21.15.117:0/2390970733 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a867b09780 0x55a867b13300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:41.369 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.360+0000 7f8e71c00640 1 --2- 172.21.15.117:0/2390970733 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a867b08d90 0x55a867b12dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:41.369 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.361+0000 7f8e72600640 1 -- 172.21.15.117:0/2390970733 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a867b08d90 msgr2=0x55a867b12dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:41.369 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.361+0000 7f8e72600640 1 --2- 172.21.15.117:0/2390970733 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a867b08d90 0x55a867b12dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:41.369 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.361+0000 7f8e72600640 1 -- 172.21.15.117:0/2390970733 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a867b09780 msgr2=0x55a867b13300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:41.369 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.361+0000 7f8e72600640 1 --2- 172.21.15.117:0/2390970733 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a867b09780 0x55a867b13300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:41.369 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.361+0000 7f8e72600640 1 -- 172.21.15.117:0/2390970733 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a86784c620 con 0x55a867b0e8f0 2024-06-03T03:27:41.369 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.361+0000 7f8e71200640 1 --2- 172.21.15.117:0/2390970733 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a867b09780 0x55a867b13300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:27:41.369 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.361+0000 7f8e72600640 1 --2- 172.21.15.117:0/2390970733 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a867b0e8f0 0x55a867b1a340 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f8e64012d20 tx=0x7f8e6400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97024b95d35a01cd server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:27:41.370 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.361+0000 7f8e53400640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-06-03T03:27:41.370 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.361+0000 7f8e53400640 1 -- 172.21.15.117:0/2390970733 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e64009e40 con 0x55a867b0e8f0 2024-06-03T03:27:41.370 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.361+0000 7f8e53400640 1 -- 172.21.15.117:0/2390970733 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f8e64012110 con 0x55a867b0e8f0 2024-06-03T03:27:41.370 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.361+0000 7f8e775c0740 1 -- 172.21.15.117:0/2390970733 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e580bbca0 con 0x55a867b0e8f0 2024-06-03T03:27:41.370 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.362+0000 7f8e53400640 1 -- 172.21.15.117:0/2390970733 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8e64025050 con 0x55a867b0e8f0 2024-06-03T03:27:41.370 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.362+0000 7f8e775c0740 10 client.15606 fetch_fsmap learned FSMap version 31 2024-06-03T03:27:41.370 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.362+0000 7f8e775c0740 1 -- 172.21.15.117:0/2390970733 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a867b1d780 con 0x55a867b0e8f0 2024-06-03T03:27:41.370 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.362+0000 7f8e53400640 1 -- 172.21.15.117:0/2390970733 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e580bbca0 con 0x55a867b0e8f0 2024-06-03T03:27:41.370 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.362+0000 7f8e53400640 1 -- 172.21.15.117:0/2390970733 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f8e64004690 con 0x55a867b0e8f0 2024-06-03T03:27:41.370 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.362+0000 7f8e775c0740 10 client.15606 fetch_fsmap finished waiting for FSMap version 31 2024-06-03T03:27:41.370 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.362+0000 7f8e775c0740 10 client.15606 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-03T03:27:41.370 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.362+0000 7f8e775c0740 20 client.15606 populate_metadata read hostname 'smithi117' 2024-06-03T03:27:41.371 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.362+0000 7f8e775c0740 1 --2- 172.21.15.117:0/2390970733 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55a867b1eec0 0x55a867b212b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:41.371 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.362+0000 7f8e775c0740 4 client.15606 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T03:27:41.371 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.362+0000 7f8e775c0740 1 -- 172.21.15.117:0/2390970733 --> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a867a79b50 con 0x55a867b1eec0 2024-06-03T03:27:41.371 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.363+0000 7f8e71c00640 1 --2- 172.21.15.117:0/2390970733 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55a867b1eec0 0x55a867b212b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:41.371 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.363+0000 7f8e71c00640 1 --2- 172.21.15.117:0/2390970733 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55a867b1eec0 0x55a867b212b0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f21c2ee4682c607 server_cookie=85b4ef113ca5229b in_seq=0 out_seq=0 2024-06-03T03:27:41.371 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.363+0000 7f8e53400640 10 client.15606 ms_handle_connect on v2:172.21.15.183:6838/1439615083 2024-06-03T03:27:41.371 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.364+0000 7f8e53400640 1 -- 172.21.15.117:0/2390970733 <== mds.0 v2:172.21.15.183:6838/1439615083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a867a79b50 con 0x55a867b1eec0 2024-06-03T03:27:41.371 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.364+0000 7f8e53400640 10 client.15606 handle_command_reply: tid=0 2024-06-03T03:27:41.371 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.364+0000 7f8e775c0740 1 client.15606 shutdown 2024-06-03T03:27:41.372 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.364+0000 7f8e775c0740 1 -- 172.21.15.117:0/2390970733 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a867b0e8f0 msgr2=0x55a867b1a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:41.372 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.364+0000 7f8e775c0740 1 --2- 172.21.15.117:0/2390970733 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a867b0e8f0 0x55a867b1a340 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f8e64012d20 tx=0x7f8e6400f790 comp rx=0 tx=0).stop 2024-06-03T03:27:41.372 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.364+0000 7f8e775c0740 1 -- 172.21.15.117:0/2390970733 shutdown_connections 2024-06-03T03:27:41.372 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.364+0000 7f8e775c0740 1 --2- 172.21.15.117:0/2390970733 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55a867b1eec0 0x55a867b212b0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:41.372 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.364+0000 7f8e775c0740 1 --2- 172.21.15.117:0/2390970733 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a867b0e8f0 0x55a867b1a340 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:41.372 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.364+0000 7f8e775c0740 1 --2- 172.21.15.117:0/2390970733 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a867b09780 0x55a867b13300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:41.372 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.364+0000 7f8e775c0740 1 --2- 172.21.15.117:0/2390970733 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a867b08d90 0x55a867b12dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:41.372 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.364+0000 7f8e53400640 0 client.15606 ms_handle_reset on v2:172.21.15.183:6838/1439615083 2024-06-03T03:27:41.372 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.364+0000 7f8e775c0740 1 -- 172.21.15.117:0/2390970733 >> 172.21.15.117:0/2390970733 conn(0x55a867ad0f80 msgr2=0x55a867b03970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:27:41.373 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.365+0000 7f8e775c0740 1 -- 172.21.15.117:0/2390970733 shutdown_connections 2024-06-03T03:27:41.373 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.365+0000 7f8e775c0740 1 -- 172.21.15.117:0/2390970733 wait complete. 2024-06-03T03:27:41.373 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.365+0000 7f8e775c0740 20 client.15606 trim_cache size 0 max 16384 2024-06-03T03:27:41.382 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.381+0000 7f8e775c0740 1 Processor -- start 2024-06-03T03:27:41.382 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.381+0000 7f8e775c0740 1 -- start start 2024-06-03T03:27:41.382 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.381+0000 7f8e775c0740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a867b08d90 0x55a867b12dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:41.382 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.381+0000 7f8e775c0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a867b09780 0x55a867b1a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:41.382 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.381+0000 7f8e775c0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a867b0e8f0 0x55a867b13300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:41.382 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.381+0000 7f8e775c0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8e6c1ac630 con 0x55a867b09780 2024-06-03T03:27:41.382 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.381+0000 7f8e775c0740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8e6c1ae170 con 0x55a867b08d90 2024-06-03T03:27:41.382 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.381+0000 7f8e775c0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8e6c1acd00 con 0x55a867b0e8f0 2024-06-03T03:27:41.384 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.382+0000 7f8e71200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a867b09780 0x55a867b1a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:41.384 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.382+0000 7f8e72600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a867b0e8f0 0x55a867b13300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:41.384 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.382+0000 7f8e71200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a867b09780 0x55a867b1a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:49696/0 (socket says 172.21.15.117:49696) 2024-06-03T03:27:41.384 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.382+0000 7f8e71200640 1 -- 172.21.15.117:0/3866638377 learned_addr learned my addr 172.21.15.117:0/3866638377 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:27:41.384 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.382+0000 7f8e71c00640 1 --2- 172.21.15.117:0/3866638377 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a867b08d90 0x55a867b12dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:41.384 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.382+0000 7f8e72600640 1 -- 172.21.15.117:0/3866638377 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a867b08d90 msgr2=0x55a867b12dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:41.384 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.382+0000 7f8e72600640 1 --2- 172.21.15.117:0/3866638377 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a867b08d90 0x55a867b12dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:41.384 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.382+0000 7f8e72600640 1 -- 172.21.15.117:0/3866638377 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a867b09780 msgr2=0x55a867b1a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:41.384 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.382+0000 7f8e72600640 1 --2- 172.21.15.117:0/3866638377 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a867b09780 0x55a867b1a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:41.384 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.382+0000 7f8e72600640 1 -- 172.21.15.117:0/3866638377 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a867b21930 con 0x55a867b0e8f0 2024-06-03T03:27:41.384 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.382+0000 7f8e71c00640 1 --2- 172.21.15.117:0/3866638377 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a867b08d90 0x55a867b12dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T03:27:41.384 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.382+0000 7f8e71200640 1 --2- 172.21.15.117:0/3866638377 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a867b09780 0x55a867b1a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:27:41.385 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.382+0000 7f8e72600640 1 --2- 172.21.15.117:0/3866638377 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a867b0e8f0 0x55a867b13300 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8e640106f0 tx=0x7f8e64009bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ed55da8189cbaad server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:27:41.385 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.383+0000 7f8e5e000640 1 -- 172.21.15.117:0/3866638377 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e64009e40 con 0x55a867b0e8f0 2024-06-03T03:27:41.385 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.383+0000 7f8e5e000640 1 -- 172.21.15.117:0/3866638377 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e64003e50 con 0x55a867b0e8f0 2024-06-03T03:27:41.385 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.383+0000 7f8e5e000640 1 -- 172.21.15.117:0/3866638377 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e64010d10 con 0x55a867b0e8f0 2024-06-03T03:27:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.384+0000 7f8e775c0740 1 -- 172.21.15.117:0/3866638377 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a867b0e8f0 msgr2=0x55a867b13300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.384+0000 7f8e775c0740 1 --2- 172.21.15.117:0/3866638377 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a867b0e8f0 0x55a867b13300 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8e640106f0 tx=0x7f8e64009bf0 comp rx=0 tx=0).stop 2024-06-03T03:27:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.384+0000 7f8e775c0740 1 -- 172.21.15.117:0/3866638377 shutdown_connections 2024-06-03T03:27:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.384+0000 7f8e775c0740 1 --2- 172.21.15.117:0/3866638377 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a867b0e8f0 0x55a867b13300 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.384+0000 7f8e775c0740 1 --2- 172.21.15.117:0/3866638377 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a867b09780 0x55a867b1a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.384+0000 7f8e775c0740 1 --2- 172.21.15.117:0/3866638377 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a867b08d90 0x55a867b12dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.384+0000 7f8e775c0740 1 -- 172.21.15.117:0/3866638377 >> 172.21.15.117:0/3866638377 conn(0x55a867ad0f80 msgr2=0x55a867b0aaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:27:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.384+0000 7f8e775c0740 1 -- 172.21.15.117:0/3866638377 shutdown_connections 2024-06-03T03:27:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.384+0000 7f8e775c0740 1 -- 172.21.15.117:0/3866638377 wait complete. 2024-06-03T03:27:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.385+0000 7f8e775c0740 1 -- start start 2024-06-03T03:27:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.385+0000 7f8e775c0740 1 Processor -- start 2024-06-03T03:27:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.385+0000 7f8e775c0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a867b08d90 0x55a867b17180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.385+0000 7f8e775c0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a867b09780 0x55a867b176c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:41.390 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.385+0000 7f8e775c0740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a867b0e8f0 0x55a867b87b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:41.390 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.385+0000 7f8e775c0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8e6c1adaa0 con 0x55a867b09780 2024-06-03T03:27:41.390 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.385+0000 7f8e775c0740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8e6c1ad3d0 con 0x55a867b0e8f0 2024-06-03T03:27:41.390 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.385+0000 7f8e775c0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8e6c1aa610 con 0x55a867b08d90 2024-06-03T03:27:41.390 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.386+0000 7f8e775c0740 10 client.0 _ll_register_callbacks cb 0x7f8e64009cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:27:41.390 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.386+0000 7f8e71200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a867b09780 0x55a867b176c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:41.390 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.386+0000 7f8e71200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a867b09780 0x55a867b176c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:49712/0 (socket says 172.21.15.117:49712) 2024-06-03T03:27:41.390 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.386+0000 7f8e71200640 1 -- 172.21.15.117:0/3382458976 learned_addr learned my addr 172.21.15.117:0/3382458976 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:27:41.390 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.386+0000 7f8e71c00640 1 --2- 172.21.15.117:0/3382458976 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a867b08d90 0x55a867b17180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:41.390 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.386+0000 7f8e72600640 1 --2- 172.21.15.117:0/3382458976 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a867b0e8f0 0x55a867b87b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:41.390 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.386+0000 7f8e71200640 1 -- 172.21.15.117:0/3382458976 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a867b08d90 msgr2=0x55a867b17180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:41.391 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.386+0000 7f8e71200640 1 --2- 172.21.15.117:0/3382458976 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a867b08d90 0x55a867b17180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:41.391 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.386+0000 7f8e71200640 1 -- 172.21.15.117:0/3382458976 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a867b0e8f0 msgr2=0x55a867b87b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:41.391 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.386+0000 7f8e71200640 1 --2- 172.21.15.117:0/3382458976 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a867b0e8f0 0x55a867b87b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:41.391 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.386+0000 7f8e71200640 1 -- 172.21.15.117:0/3382458976 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a867b880c0 con 0x55a867b09780 2024-06-03T03:27:41.391 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.386+0000 7f8e71c00640 1 --2- 172.21.15.117:0/3382458976 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a867b08d90 0x55a867b17180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:27:41.391 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.386+0000 7f8e71200640 1 --2- 172.21.15.117:0/3382458976 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a867b09780 0x55a867b176c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f8e58005ec0 tx=0x7f8e58005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff2302ed2aa238ff server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:27:41.391 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.386+0000 7f8e53400640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-06-03T03:27:41.391 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.387+0000 7f8e53400640 1 -- 172.21.15.117:0/3382458976 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e58049040 con 0x55a867b09780 2024-06-03T03:27:41.391 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.387+0000 7f8e53400640 1 -- 172.21.15.117:0/3382458976 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f8e580b8310 con 0x55a867b09780 2024-06-03T03:27:41.391 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.387+0000 7f8e775c0740 1 -- 172.21.15.117:0/3382458976 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e64003e50 con 0x55a867b09780 2024-06-03T03:27:41.391 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.387+0000 7f8e53400640 1 -- 172.21.15.117:0/3382458976 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8e580bba70 con 0x55a867b09780 2024-06-03T03:27:41.391 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.387+0000 7f8e775c0740 10 client.15612 fetch_fsmap learned FSMap version 31 2024-06-03T03:27:41.391 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.387+0000 7f8e775c0740 1 -- 172.21.15.117:0/3382458976 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a867b89220 con 0x55a867b09780 2024-06-03T03:27:41.391 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.387+0000 7f8e53400640 1 -- 172.21.15.117:0/3382458976 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e64003e50 con 0x55a867b09780 2024-06-03T03:27:41.391 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.387+0000 7f8e53400640 1 -- 172.21.15.117:0/3382458976 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f8e5803bc60 con 0x55a867b09780 2024-06-03T03:27:41.392 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.387+0000 7f8e775c0740 10 client.15612 fetch_fsmap finished waiting for FSMap version 31 2024-06-03T03:27:41.392 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.387+0000 7f8e775c0740 10 client.15612 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-03T03:27:41.392 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.387+0000 7f8e775c0740 20 client.15612 populate_metadata read hostname 'smithi117' 2024-06-03T03:27:41.392 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.388+0000 7f8e775c0740 1 --2- 172.21.15.117:0/3382458976 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55a867b89ac0 0x55a867b8bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:41.392 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.388+0000 7f8e775c0740 4 client.15612 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-03T03:27:41.392 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.388+0000 7f8e775c0740 1 -- 172.21.15.117:0/3382458976 --> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a867a73ea0 con 0x55a867b89ac0 2024-06-03T03:27:41.392 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.388+0000 7f8e71c00640 1 --2- 172.21.15.117:0/3382458976 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55a867b89ac0 0x55a867b8bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:41.392 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.388+0000 7f8e71c00640 1 --2- 172.21.15.117:0/3382458976 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55a867b89ac0 0x55a867b8bee0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ecade2516527d1d server_cookie=63af7ee91ead98d9 in_seq=0 out_seq=0 2024-06-03T03:27:41.392 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.388+0000 7f8e53400640 10 client.15612 ms_handle_connect on v2:172.21.15.183:6838/1439615083 2024-06-03T03:27:41.401 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.400+0000 7f8e53400640 1 -- 172.21.15.117:0/3382458976 <== mds.0 v2:172.21.15.183:6838/1439615083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55a867a73ea0 con 0x55a867b89ac0 2024-06-03T03:27:41.402 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.400+0000 7f8e53400640 10 client.15612 handle_command_reply: tid=0 2024-06-03T03:27:41.402 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.400+0000 7f8e775c0740 1 client.15612 shutdown 2024-06-03T03:27:41.402 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.401+0000 7f8e775c0740 1 -- 172.21.15.117:0/3382458976 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a867b09780 msgr2=0x55a867b176c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:41.402 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.401+0000 7f8e775c0740 1 --2- 172.21.15.117:0/3382458976 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a867b09780 0x55a867b176c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f8e58005ec0 tx=0x7f8e58005e50 comp rx=0 tx=0).stop 2024-06-03T03:27:41.402 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.401+0000 7f8e775c0740 1 -- 172.21.15.117:0/3382458976 shutdown_connections 2024-06-03T03:27:41.402 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.401+0000 7f8e775c0740 1 --2- 172.21.15.117:0/3382458976 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55a867b89ac0 0x55a867b8bee0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:41.402 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.401+0000 7f8e775c0740 1 --2- 172.21.15.117:0/3382458976 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a867b0e8f0 0x55a867b87b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:41.402 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.401+0000 7f8e775c0740 1 --2- 172.21.15.117:0/3382458976 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a867b09780 0x55a867b176c0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:41.402 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.401+0000 7f8e775c0740 1 --2- 172.21.15.117:0/3382458976 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a867b08d90 0x55a867b17180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:41.402 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.401+0000 7f8e53400640 0 client.15612 ms_handle_reset on v2:172.21.15.183:6838/1439615083 2024-06-03T03:27:41.402 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.401+0000 7f8e775c0740 1 -- 172.21.15.117:0/3382458976 >> 172.21.15.117:0/3382458976 conn(0x55a867ad0f80 msgr2=0x55a867b12dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:27:41.402 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.401+0000 7f8e775c0740 1 -- 172.21.15.117:0/3382458976 shutdown_connections 2024-06-03T03:27:41.403 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.401+0000 7f8e775c0740 1 -- 172.21.15.117:0/3382458976 wait complete. 2024-06-03T03:27:41.403 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.401+0000 7f8e775c0740 20 client.15612 trim_cache size 0 max 16384 2024-06-03T03:27:41.403 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:{ 2024-06-03T03:27:41.403 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "return_code": 0 2024-06-03T03:27:41.403 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:} 2024-06-03T03:27:41.404 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.402+0000 7f8e73000640 1 -- 172.21.15.117:0/1219655011 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f8e540a2c00 msgr2=0x7f8e540a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:41.404 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.402+0000 7f8e73000640 1 --2- 172.21.15.117:0/1219655011 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f8e540a2c00 0x7f8e540a50c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f8e60011490 tx=0x7f8e60005f90 comp rx=0 tx=0).stop 2024-06-03T03:27:41.404 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.402+0000 7f8e73000640 1 -- 172.21.15.117:0/1219655011 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e6c100a50 msgr2=0x7f8e6c19a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:41.404 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.402+0000 7f8e73000640 1 --2- 172.21.15.117:0/1219655011 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e6c100a50 0x7f8e6c19a290 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f8e58031440 tx=0x7f8e58032ce0 comp rx=0 tx=0).stop 2024-06-03T03:27:41.404 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.402+0000 7f8e73000640 1 -- 172.21.15.117:0/1219655011 shutdown_connections 2024-06-03T03:27:41.404 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.402+0000 7f8e73000640 1 --2- 172.21.15.117:0/1219655011 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8e6c1013f0 0x7f8e6c1a1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:41.404 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.402+0000 7f8e73000640 1 --2- 172.21.15.117:0/1219655011 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e6c100a50 0x7f8e6c19a290 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:41.404 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.402+0000 7f8e73000640 1 --2- 172.21.15.117:0/1219655011 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f8e540a2c00 0x7f8e540a50c0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:41.404 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.402+0000 7f8e73000640 1 --2- 172.21.15.117:0/1219655011 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e6c1000f0 0x7f8e6c199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:41.405 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.402+0000 7f8e73000640 1 -- 172.21.15.117:0/1219655011 >> 172.21.15.117:0/1219655011 conn(0x7f8e6c0fbc90 msgr2=0x7f8e6c0fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:27:41.405 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.402+0000 7f8e73000640 1 -- 172.21.15.117:0/1219655011 shutdown_connections 2024-06-03T03:27:41.405 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:41.402+0000 7f8e73000640 1 -- 172.21.15.117:0/1219655011 wait complete. 2024-06-03T03:27:41.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:27:41 smithi151 ceph-mon[37043]: pgmap v577: 129 pgs: 129 active+clean; 258 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 MiB/s wr, 28 op/s 2024-06-03T03:27:41.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:27:41 smithi117 ceph-mon[31649]: pgmap v577: 129 pgs: 129 active+clean; 258 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 MiB/s wr, 28 op/s 2024-06-03T03:27:41.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:27:41 smithi183 ceph-mon[37326]: pgmap v577: 129 pgs: 129 active+clean; 258 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 MiB/s wr, 28 op/s 2024-06-03T03:27:43.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:27:43 smithi151 ceph-mon[37043]: pgmap v578: 129 pgs: 129 active+clean; 258 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 11 MiB/s wr, 24 op/s 2024-06-03T03:27:43.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:27:43 smithi117 ceph-mon[31649]: pgmap v578: 129 pgs: 129 active+clean; 258 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 11 MiB/s wr, 24 op/s 2024-06-03T03:27:43.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:27:43 smithi183 ceph-mon[37326]: pgmap v578: 129 pgs: 129 active+clean; 258 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 11 MiB/s wr, 24 op/s 2024-06-03T03:27:45.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:27:45 smithi151 ceph-mon[37043]: pgmap v579: 129 pgs: 129 active+clean; 258 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1.5 MiB/s wr, 14 op/s 2024-06-03T03:27:45.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:27:45 smithi117 ceph-mon[31649]: pgmap v579: 129 pgs: 129 active+clean; 258 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1.5 MiB/s wr, 14 op/s 2024-06-03T03:27:45.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:27:45 smithi183 ceph-mon[37326]: pgmap v579: 129 pgs: 129 active+clean; 258 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1.5 MiB/s wr, 14 op/s 2024-06-03T03:27:47.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:27:47 smithi151 ceph-mon[37043]: pgmap v580: 129 pgs: 129 active+clean; 258 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 775 KiB/s wr, 13 op/s 2024-06-03T03:27:47.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:27:47 smithi117 ceph-mon[31649]: pgmap v580: 129 pgs: 129 active+clean; 258 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 775 KiB/s wr, 13 op/s 2024-06-03T03:27:47.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:27:47 smithi183 ceph-mon[37326]: pgmap v580: 129 pgs: 129 active+clean; 258 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 775 KiB/s wr, 13 op/s 2024-06-03T03:27:49.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:27:49 smithi151 ceph-mon[37043]: pgmap v581: 129 pgs: 129 active+clean; 258 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 778 KiB/s wr, 17 op/s 2024-06-03T03:27:49.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:27:49 smithi117 ceph-mon[31649]: pgmap v581: 129 pgs: 129 active+clean; 258 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 778 KiB/s wr, 17 op/s 2024-06-03T03:27:49.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:27:49 smithi183 ceph-mon[37326]: pgmap v581: 129 pgs: 129 active+clean; 258 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 778 KiB/s wr, 17 op/s 2024-06-03T03:27:51.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:27:51 smithi151 ceph-mon[37043]: pgmap v582: 129 pgs: 129 active+clean; 258 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1.2 MiB/s wr, 14 op/s 2024-06-03T03:27:51.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:27:51 smithi117 ceph-mon[31649]: pgmap v582: 129 pgs: 129 active+clean; 258 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1.2 MiB/s wr, 14 op/s 2024-06-03T03:27:51.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:27:51 smithi183 ceph-mon[37326]: pgmap v582: 129 pgs: 129 active+clean; 258 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1.2 MiB/s wr, 14 op/s 2024-06-03T03:27:52.795 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-03T03:27:53.250 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:27:53.405 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:27:53 smithi117 ceph-mon[31649]: pgmap v583: 129 pgs: 129 active+clean; 258 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 779 KiB/s wr, 17 op/s 2024-06-03T03:27:53.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:27:53 smithi151 ceph-mon[37043]: pgmap v583: 129 pgs: 129 active+clean; 258 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 779 KiB/s wr, 17 op/s 2024-06-03T03:27:53.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:27:53 smithi183 ceph-mon[37326]: pgmap v583: 129 pgs: 129 active+clean; 258 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 779 KiB/s wr, 17 op/s 2024-06-03T03:27:54.541 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.540+0000 7f3e3e000640 1 -- 172.21.15.117:0/2902922379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e38074380 msgr2=0x7f3e380747e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:54.541 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.540+0000 7f3e3e000640 1 --2- 172.21.15.117:0/2902922379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e38074380 0x7f3e380747e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f3e24009900 tx=0x7f3e2402a480 comp rx=0 tx=0).stop 2024-06-03T03:27:54.541 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.540+0000 7f3e3e000640 1 -- 172.21.15.117:0/2902922379 shutdown_connections 2024-06-03T03:27:54.542 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.540+0000 7f3e3e000640 1 --2- 172.21.15.117:0/2902922379 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e381098b0 0x7f3e3810bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:54.542 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.540+0000 7f3e3e000640 1 --2- 172.21.15.117:0/2902922379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e38074380 0x7f3e380747e0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:54.542 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.540+0000 7f3e3e000640 1 --2- 172.21.15.117:0/2902922379 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3e38075960 0x7f3e38073db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:54.542 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.540+0000 7f3e3e000640 1 -- 172.21.15.117:0/2902922379 >> 172.21.15.117:0/2902922379 conn(0x7f3e380fcf90 msgr2=0x7f3e380ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:27:54.542 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.540+0000 7f3e3e000640 1 -- 172.21.15.117:0/2902922379 shutdown_connections 2024-06-03T03:27:54.542 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.540+0000 7f3e3e000640 1 -- 172.21.15.117:0/2902922379 wait complete. 2024-06-03T03:27:54.542 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.541+0000 7f3e3e000640 1 Processor -- start 2024-06-03T03:27:54.542 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.541+0000 7f3e3e000640 1 -- start start 2024-06-03T03:27:54.544 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.541+0000 7f3e3e000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e38074380 0x7f3e38199e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:54.544 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.541+0000 7f3e3e000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3e38075960 0x7f3e3819a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:54.544 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.541+0000 7f3e3e000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e381098b0 0x7f3e381a13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:54.544 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.541+0000 7f3e3e000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3e3810f890 con 0x7f3e381098b0 2024-06-03T03:27:54.544 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.541+0000 7f3e3e000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3e3810f710 con 0x7f3e38075960 2024-06-03T03:27:54.545 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.541+0000 7f3e3e000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3e3810fa10 con 0x7f3e38074380 2024-06-03T03:27:54.545 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.542+0000 7f3e37e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e381098b0 0x7f3e381a13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:54.545 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.542+0000 7f3e37e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e381098b0 0x7f3e381a13d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56832/0 (socket says 172.21.15.117:56832) 2024-06-03T03:27:54.545 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.542+0000 7f3e37e00640 1 -- 172.21.15.117:0/4108120868 learned_addr learned my addr 172.21.15.117:0/4108120868 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:27:54.545 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.542+0000 7f3e36a00640 1 --2- 172.21.15.117:0/4108120868 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3e38075960 0x7f3e3819a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:54.545 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.542+0000 7f3e37400640 1 --2- 172.21.15.117:0/4108120868 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e38074380 0x7f3e38199e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:54.545 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.542+0000 7f3e37e00640 1 -- 172.21.15.117:0/4108120868 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e38074380 msgr2=0x7f3e38199e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:54.545 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.542+0000 7f3e37e00640 1 --2- 172.21.15.117:0/4108120868 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e38074380 0x7f3e38199e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:54.545 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.542+0000 7f3e37e00640 1 -- 172.21.15.117:0/4108120868 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3e38075960 msgr2=0x7f3e3819a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:54.545 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.542+0000 7f3e37e00640 1 --2- 172.21.15.117:0/4108120868 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3e38075960 0x7f3e3819a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:54.545 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.542+0000 7f3e37e00640 1 -- 172.21.15.117:0/4108120868 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e381a1a40 con 0x7f3e381098b0 2024-06-03T03:27:54.545 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.542+0000 7f3e37e00640 1 --2- 172.21.15.117:0/4108120868 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e381098b0 0x7f3e381a13d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f3e3000eb10 tx=0x7f3e3000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:27:54.545 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.543+0000 7f3e2fe00640 1 -- 172.21.15.117:0/4108120868 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e30004b20 con 0x7f3e381098b0 2024-06-03T03:27:54.546 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.543+0000 7f3e2fe00640 1 -- 172.21.15.117:0/4108120868 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e30004cc0 con 0x7f3e381098b0 2024-06-03T03:27:54.546 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.543+0000 7f3e3e000640 1 -- 172.21.15.117:0/4108120868 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e381a1d30 con 0x7f3e381098b0 2024-06-03T03:27:54.546 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.543+0000 7f3e2fe00640 1 -- 172.21.15.117:0/4108120868 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e30010760 con 0x7f3e381098b0 2024-06-03T03:27:54.546 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.543+0000 7f3e3e000640 1 -- 172.21.15.117:0/4108120868 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e381a21f0 con 0x7f3e381098b0 2024-06-03T03:27:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.547+0000 7f3e40fdb740 1 Processor -- start 2024-06-03T03:27:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.547+0000 7f3e2fe00640 1 -- 172.21.15.117:0/4108120868 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f3e30005020 con 0x7f3e381098b0 2024-06-03T03:27:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.547+0000 7f3e40fdb740 1 -- start start 2024-06-03T03:27:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.548+0000 7f3e40fdb740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a541c65bd0 0x55a541c65ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.548+0000 7f3e40fdb740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a541c665c0 0x55a541c6b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.548+0000 7f3e40fdb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a541c6b730 0x55a541c6dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.548+0000 7f3e40fdb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a541bc9420 con 0x55a541c6b730 2024-06-03T03:27:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.548+0000 7f3e40fdb740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a541b70c50 con 0x55a541c65bd0 2024-06-03T03:27:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.548+0000 7f3e40fdb740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a541bc53e0 con 0x55a541c665c0 2024-06-03T03:27:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.548+0000 7f3e36a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a541c65bd0 0x55a541c65ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.548+0000 7f3e37400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a541c665c0 0x55a541c6b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.548+0000 7f3e37400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a541c665c0 0x55a541c6b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:57888/0 (socket says 172.21.15.117:57888) 2024-06-03T03:27:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.548+0000 7f3e37400640 1 -- 172.21.15.117:0/1617984865 learned_addr learned my addr 172.21.15.117:0/1617984865 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:27:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.548+0000 7f3e36a00640 1 --2- 172.21.15.117:0/1617984865 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a541c6b730 0x55a541c6dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:54.551 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.549+0000 7f3e37400640 1 -- 172.21.15.117:0/1617984865 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a541c65bd0 msgr2=0x55a541c65ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:54.551 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.549+0000 7f3e37400640 1 --2- 172.21.15.117:0/1617984865 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a541c65bd0 0x55a541c65ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:54.551 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.549+0000 7f3e37400640 1 -- 172.21.15.117:0/1617984865 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a541c6b730 msgr2=0x55a541c6dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:54.551 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.549+0000 7f3e37400640 1 --2- 172.21.15.117:0/1617984865 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a541c6b730 0x55a541c6dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:54.551 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.549+0000 7f3e37400640 1 -- 172.21.15.117:0/1617984865 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5418dd210 con 0x55a541c665c0 2024-06-03T03:27:54.551 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.549+0000 7f3e36a00640 1 --2- 172.21.15.117:0/1617984865 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a541c6b730 0x55a541c6dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T03:27:54.551 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.549+0000 7f3e2fe00640 1 --2- 172.21.15.117:0/4108120868 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f3e1c0a2c30 0x7f3e1c0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.549+0000 7f3e2fe00640 1 -- 172.21.15.117:0/4108120868 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f3e300cfa70 con 0x7f3e381098b0 2024-06-03T03:27:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.549+0000 7f3e37400640 1 --2- 172.21.15.117:0/1617984865 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a541c665c0 0x55a541c6b0c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3e28009cf0 tx=0x7f3e28011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b34a1ec8c1308633 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:27:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.549+0000 7f3e36a00640 1 --2- 172.21.15.117:0/4108120868 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f3e1c0a2c30 0x7f3e1c0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.549+0000 7f3e23e00640 1 -- 172.21.15.117:0/1617984865 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e28004230 con 0x55a541c665c0 2024-06-03T03:27:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.549+0000 7f3e23e00640 1 -- 172.21.15.117:0/1617984865 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e280043d0 con 0x55a541c665c0 2024-06-03T03:27:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.549+0000 7f3e23e00640 1 -- 172.21.15.117:0/1617984865 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e280198c0 con 0x55a541c665c0 2024-06-03T03:27:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.550+0000 7f3e40fdb740 1 -- 172.21.15.117:0/1617984865 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a541c665c0 msgr2=0x55a541c6b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.550+0000 7f3e36a00640 1 --2- 172.21.15.117:0/4108120868 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f3e1c0a2c30 0x7f3e1c0a50f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3e240314e0 tx=0x7f3e24000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:27:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.550+0000 7f3e40fdb740 1 --2- 172.21.15.117:0/1617984865 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a541c665c0 0x55a541c6b0c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3e28009cf0 tx=0x7f3e28011810 comp rx=0 tx=0).stop 2024-06-03T03:27:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.550+0000 7f3e40fdb740 1 -- 172.21.15.117:0/1617984865 shutdown_connections 2024-06-03T03:27:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.550+0000 7f3e40fdb740 1 --2- 172.21.15.117:0/1617984865 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a541c6b730 0x55a541c6dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.550+0000 7f3e40fdb740 1 --2- 172.21.15.117:0/1617984865 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a541c665c0 0x55a541c6b0c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.550+0000 7f3e40fdb740 1 --2- 172.21.15.117:0/1617984865 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a541c65bd0 0x55a541c65ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.550+0000 7f3e40fdb740 1 -- 172.21.15.117:0/1617984865 >> 172.21.15.117:0/1617984865 conn(0x55a541c2df80 msgr2=0x55a541c61650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:27:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.550+0000 7f3e40fdb740 1 -- 172.21.15.117:0/1617984865 shutdown_connections 2024-06-03T03:27:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.550+0000 7f3e40fdb740 1 -- 172.21.15.117:0/1617984865 wait complete. 2024-06-03T03:27:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.551+0000 7f3e40fdb740 1 -- start start 2024-06-03T03:27:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.551+0000 7f3e40fdb740 1 Processor -- start 2024-06-03T03:27:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.551+0000 7f3e40fdb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a541c65bd0 0x55a541c6ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.551+0000 7f3e40fdb740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a541c665c0 0x55a541c70450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.551+0000 7f3e40fdb740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a541c6b730 0x55a541c77420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.551+0000 7f3e40fdb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a541c6e6e0 con 0x55a541c65bd0 2024-06-03T03:27:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.551+0000 7f3e40fdb740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a541b840b0 con 0x55a541c665c0 2024-06-03T03:27:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.551+0000 7f3e40fdb740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a541c6e860 con 0x55a541c6b730 2024-06-03T03:27:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.552+0000 7f3e40fdb740 10 client.0 _ll_register_callbacks cb 0x55a541b6ff40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:27:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.552+0000 7f3e37400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a541c65bd0 0x55a541c6ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.552+0000 7f3e37400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a541c65bd0 0x55a541c6ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56854/0 (socket says 172.21.15.117:56854) 2024-06-03T03:27:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.552+0000 7f3e37400640 1 -- 172.21.15.117:0/1961412347 learned_addr learned my addr 172.21.15.117:0/1961412347 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:27:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.552+0000 7f3e36a00640 1 --2- 172.21.15.117:0/1961412347 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a541c665c0 0x55a541c70450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.552+0000 7f3e37e00640 1 --2- 172.21.15.117:0/1961412347 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a541c6b730 0x55a541c77420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.552+0000 7f3e37400640 1 -- 172.21.15.117:0/1961412347 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a541c6b730 msgr2=0x55a541c77420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.552+0000 7f3e37400640 1 --2- 172.21.15.117:0/1961412347 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a541c6b730 0x55a541c77420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.552+0000 7f3e37400640 1 -- 172.21.15.117:0/1961412347 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a541c665c0 msgr2=0x55a541c70450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.552+0000 7f3e37400640 1 --2- 172.21.15.117:0/1961412347 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a541c665c0 0x55a541c70450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.552+0000 7f3e37400640 1 -- 172.21.15.117:0/1961412347 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5419a9620 con 0x55a541c65bd0 2024-06-03T03:27:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.552+0000 7f3e37e00640 1 --2- 172.21.15.117:0/1961412347 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a541c6b730 0x55a541c77420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T03:27:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.552+0000 7f3e37400640 1 --2- 172.21.15.117:0/1961412347 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a541c65bd0 0x55a541c6ff10 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f3e2801a000 tx=0x7f3e28003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63b761afaa00b9c2 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:27:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.552+0000 7f3e20c00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-06-03T03:27:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.553+0000 7f3e20c00640 1 -- 172.21.15.117:0/1961412347 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e28027070 con 0x55a541c65bd0 2024-06-03T03:27:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.553+0000 7f3e20c00640 1 -- 172.21.15.117:0/1961412347 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f3e28005240 con 0x55a541c65bd0 2024-06-03T03:27:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.553+0000 7f3e40fdb740 1 -- 172.21.15.117:0/1961412347 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e280043d0 con 0x55a541c65bd0 2024-06-03T03:27:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.553+0000 7f3e20c00640 1 -- 172.21.15.117:0/1961412347 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f3e28033050 con 0x55a541c65bd0 2024-06-03T03:27:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.553+0000 7f3e40fdb740 10 client.15624 fetch_fsmap learned FSMap version 31 2024-06-03T03:27:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.553+0000 7f3e40fdb740 1 -- 172.21.15.117:0/1961412347 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a541c7a790 con 0x55a541c65bd0 2024-06-03T03:27:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.553+0000 7f3e20c00640 1 -- 172.21.15.117:0/1961412347 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e280043d0 con 0x55a541c65bd0 2024-06-03T03:27:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.553+0000 7f3e20c00640 1 -- 172.21.15.117:0/1961412347 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f3e28004580 con 0x55a541c65bd0 2024-06-03T03:27:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.554+0000 7f3e40fdb740 10 client.15624 fetch_fsmap finished waiting for FSMap version 31 2024-06-03T03:27:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.554+0000 7f3e40fdb740 10 client.15624 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-03T03:27:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.554+0000 7f3e40fdb740 20 client.15624 populate_metadata read hostname 'smithi117' 2024-06-03T03:27:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.554+0000 7f3e40fdb740 1 --2- 172.21.15.117:0/1961412347 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55a541c7bcf0 0x55a541c7e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.554+0000 7f3e40fdb740 4 client.15624 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T03:27:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.554+0000 7f3e40fdb740 1 -- 172.21.15.117:0/1961412347 --> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a541bd6b50 con 0x55a541c7bcf0 2024-06-03T03:27:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.554+0000 7f3e36a00640 1 --2- 172.21.15.117:0/1961412347 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55a541c7bcf0 0x55a541c7e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.554+0000 7f3e36a00640 1 --2- 172.21.15.117:0/1961412347 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55a541c7bcf0 0x55a541c7e100 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db14c06d727b195d server_cookie=a1083f9844efdc4e in_seq=0 out_seq=0 2024-06-03T03:27:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.555+0000 7f3e20c00640 10 client.15624 ms_handle_connect on v2:172.21.15.183:6838/1439615083 2024-06-03T03:27:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.555+0000 7f3e20c00640 1 -- 172.21.15.117:0/1961412347 <== mds.0 v2:172.21.15.183:6838/1439615083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a541bd6b50 con 0x55a541c7bcf0 2024-06-03T03:27:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.555+0000 7f3e20c00640 10 client.15624 handle_command_reply: tid=0 2024-06-03T03:27:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.555+0000 7f3e40fdb740 1 client.15624 shutdown 2024-06-03T03:27:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.556+0000 7f3e40fdb740 1 -- 172.21.15.117:0/1961412347 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a541c65bd0 msgr2=0x55a541c6ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.556+0000 7f3e40fdb740 1 --2- 172.21.15.117:0/1961412347 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a541c65bd0 0x55a541c6ff10 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f3e2801a000 tx=0x7f3e28003820 comp rx=0 tx=0).stop 2024-06-03T03:27:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.556+0000 7f3e40fdb740 1 -- 172.21.15.117:0/1961412347 shutdown_connections 2024-06-03T03:27:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.556+0000 7f3e40fdb740 1 --2- 172.21.15.117:0/1961412347 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55a541c7bcf0 0x55a541c7e100 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.556+0000 7f3e40fdb740 1 --2- 172.21.15.117:0/1961412347 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a541c6b730 0x55a541c77420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.556+0000 7f3e40fdb740 1 --2- 172.21.15.117:0/1961412347 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a541c665c0 0x55a541c70450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.556+0000 7f3e20c00640 0 client.15624 ms_handle_reset on v2:172.21.15.183:6838/1439615083 2024-06-03T03:27:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.556+0000 7f3e40fdb740 1 --2- 172.21.15.117:0/1961412347 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a541c65bd0 0x55a541c6ff10 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.556+0000 7f3e40fdb740 1 -- 172.21.15.117:0/1961412347 >> 172.21.15.117:0/1961412347 conn(0x55a541c2df80 msgr2=0x55a541c60810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:27:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.556+0000 7f3e40fdb740 1 -- 172.21.15.117:0/1961412347 shutdown_connections 2024-06-03T03:27:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.556+0000 7f3e40fdb740 1 -- 172.21.15.117:0/1961412347 wait complete. 2024-06-03T03:27:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.556+0000 7f3e40fdb740 20 client.15624 trim_cache size 0 max 16384 2024-06-03T03:27:54.573 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.572+0000 7f3e40fdb740 1 Processor -- start 2024-06-03T03:27:54.573 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.572+0000 7f3e40fdb740 1 -- start start 2024-06-03T03:27:54.573 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.573+0000 7f3e40fdb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a541c65bd0 0x55a541c64e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:54.573 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.573+0000 7f3e40fdb740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a541c665c0 0x55a541c77340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:54.573 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.573+0000 7f3e40fdb740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a541c6b730 0x55a541c60690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:54.573 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.573+0000 7f3e40fdb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3e381ac710 con 0x55a541c65bd0 2024-06-03T03:27:54.574 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.573+0000 7f3e40fdb740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3e381ae250 con 0x55a541c6b730 2024-06-03T03:27:54.574 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.573+0000 7f3e40fdb740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3e381acde0 con 0x55a541c665c0 2024-06-03T03:27:54.575 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.573+0000 7f3e37400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a541c65bd0 0x55a541c64e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:54.575 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.573+0000 7f3e37400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a541c65bd0 0x55a541c64e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56858/0 (socket says 172.21.15.117:56858) 2024-06-03T03:27:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.573+0000 7f3e37400640 1 -- 172.21.15.117:0/1735034451 learned_addr learned my addr 172.21.15.117:0/1735034451 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:27:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.573+0000 7f3e36a00640 1 --2- 172.21.15.117:0/1735034451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a541c665c0 0x55a541c77340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.573+0000 7f3e37e00640 1 --2- 172.21.15.117:0/1735034451 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a541c6b730 0x55a541c60690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.574+0000 7f3e36a00640 1 -- 172.21.15.117:0/1735034451 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a541c6b730 msgr2=0x55a541c60690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.574+0000 7f3e36a00640 1 --2- 172.21.15.117:0/1735034451 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a541c6b730 0x55a541c60690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.574+0000 7f3e36a00640 1 -- 172.21.15.117:0/1735034451 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a541c65bd0 msgr2=0x55a541c64e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.574+0000 7f3e36a00640 1 --2- 172.21.15.117:0/1735034451 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a541c65bd0 0x55a541c64e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.574+0000 7f3e36a00640 1 -- 172.21.15.117:0/1735034451 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a541c61ba0 con 0x55a541c665c0 2024-06-03T03:27:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.574+0000 7f3e36a00640 1 --2- 172.21.15.117:0/1735034451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a541c665c0 0x55a541c77340 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3e24031440 tx=0x7f3e2403f070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cbf1ffb7c9d3fed server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:27:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.574+0000 7f3e23400640 1 -- 172.21.15.117:0/1735034451 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e24035040 con 0x55a541c665c0 2024-06-03T03:27:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.574+0000 7f3e23400640 1 -- 172.21.15.117:0/1735034451 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e24059070 con 0x55a541c665c0 2024-06-03T03:27:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.574+0000 7f3e23400640 1 -- 172.21.15.117:0/1735034451 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e24054ba0 con 0x55a541c665c0 2024-06-03T03:27:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.575+0000 7f3e40fdb740 1 -- 172.21.15.117:0/1735034451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a541c665c0 msgr2=0x55a541c77340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.575+0000 7f3e40fdb740 1 --2- 172.21.15.117:0/1735034451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a541c665c0 0x55a541c77340 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3e24031440 tx=0x7f3e2403f070 comp rx=0 tx=0).stop 2024-06-03T03:27:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.575+0000 7f3e40fdb740 1 -- 172.21.15.117:0/1735034451 shutdown_connections 2024-06-03T03:27:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.575+0000 7f3e40fdb740 1 --2- 172.21.15.117:0/1735034451 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a541c6b730 0x55a541c60690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.575+0000 7f3e40fdb740 1 --2- 172.21.15.117:0/1735034451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a541c665c0 0x55a541c77340 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.575+0000 7f3e40fdb740 1 --2- 172.21.15.117:0/1735034451 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a541c65bd0 0x55a541c64e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.575+0000 7f3e40fdb740 1 -- 172.21.15.117:0/1735034451 >> 172.21.15.117:0/1735034451 conn(0x55a541c2df80 msgr2=0x55a541c5ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:27:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.575+0000 7f3e40fdb740 1 -- 172.21.15.117:0/1735034451 shutdown_connections 2024-06-03T03:27:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.575+0000 7f3e40fdb740 1 -- 172.21.15.117:0/1735034451 wait complete. 2024-06-03T03:27:54.580 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.576+0000 7f3e40fdb740 1 -- start start 2024-06-03T03:27:54.580 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.576+0000 7f3e40fdb740 1 Processor -- start 2024-06-03T03:27:54.580 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.576+0000 7f3e40fdb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a541c65bd0 0x55a541c74b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:54.580 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.576+0000 7f3e40fdb740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a541c665c0 0x55a541c750d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:54.580 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.576+0000 7f3e40fdb740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a541c6b730 0x55a541c760e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:54.580 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.576+0000 7f3e40fdb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3e381adb80 con 0x55a541c65bd0 2024-06-03T03:27:54.580 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.576+0000 7f3e40fdb740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3e381ad4b0 con 0x55a541c6b730 2024-06-03T03:27:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.576+0000 7f3e40fdb740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3e381aa6f0 con 0x55a541c665c0 2024-06-03T03:27:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.576+0000 7f3e40fdb740 10 client.0 _ll_register_callbacks cb 0x7f3e300093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:27:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.576+0000 7f3e37400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a541c65bd0 0x55a541c74b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.576+0000 7f3e37400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a541c65bd0 0x55a541c74b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56864/0 (socket says 172.21.15.117:56864) 2024-06-03T03:27:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.576+0000 7f3e37400640 1 -- 172.21.15.117:0/2595610254 learned_addr learned my addr 172.21.15.117:0/2595610254 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:27:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.577+0000 7f3e37e00640 1 --2- 172.21.15.117:0/2595610254 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a541c6b730 0x55a541c760e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.577+0000 7f3e36a00640 1 --2- 172.21.15.117:0/2595610254 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a541c665c0 0x55a541c750d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.577+0000 7f3e37400640 1 -- 172.21.15.117:0/2595610254 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a541c665c0 msgr2=0x55a541c750d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.577+0000 7f3e37400640 1 --2- 172.21.15.117:0/2595610254 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a541c665c0 0x55a541c750d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.577+0000 7f3e37400640 1 -- 172.21.15.117:0/2595610254 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a541c6b730 msgr2=0x55a541c760e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.577+0000 7f3e37400640 1 --2- 172.21.15.117:0/2595610254 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a541c6b730 0x55a541c760e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.577+0000 7f3e37400640 1 -- 172.21.15.117:0/2595610254 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a541c766d0 con 0x55a541c65bd0 2024-06-03T03:27:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.577+0000 7f3e37400640 1 --2- 172.21.15.117:0/2595610254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a541c65bd0 0x55a541c74b90 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3e28004b90 tx=0x7f3e28003e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fa0e82c5b533aa4 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:27:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.577+0000 7f3e20c00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-06-03T03:27:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.577+0000 7f3e20c00640 1 -- 172.21.15.117:0/2595610254 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e28027040 con 0x55a541c65bd0 2024-06-03T03:27:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.577+0000 7f3e40fdb740 1 -- 172.21.15.117:0/2595610254 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e24059070 con 0x55a541c65bd0 2024-06-03T03:27:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.577+0000 7f3e20c00640 1 -- 172.21.15.117:0/2595610254 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f3e28019420 con 0x55a541c65bd0 2024-06-03T03:27:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.578+0000 7f3e20c00640 1 -- 172.21.15.117:0/2595610254 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f3e28004790 con 0x55a541c65bd0 2024-06-03T03:27:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.578+0000 7f3e40fdb740 10 client.15630 fetch_fsmap learned FSMap version 31 2024-06-03T03:27:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.578+0000 7f3e40fdb740 1 -- 172.21.15.117:0/2595610254 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a541cef240 con 0x55a541c65bd0 2024-06-03T03:27:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.578+0000 7f3e20c00640 1 -- 172.21.15.117:0/2595610254 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e24059070 con 0x55a541c65bd0 2024-06-03T03:27:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.578+0000 7f3e20c00640 1 -- 172.21.15.117:0/2595610254 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f3e2802c6e0 con 0x55a541c65bd0 2024-06-03T03:27:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.578+0000 7f3e40fdb740 10 client.15630 fetch_fsmap finished waiting for FSMap version 31 2024-06-03T03:27:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.578+0000 7f3e40fdb740 10 client.15630 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-03T03:27:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.578+0000 7f3e40fdb740 20 client.15630 populate_metadata read hostname 'smithi117' 2024-06-03T03:27:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.578+0000 7f3e40fdb740 1 --2- 172.21.15.117:0/2595610254 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55a541cefb80 0x55a541cf1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.578+0000 7f3e40fdb740 4 client.15630 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-03T03:27:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.578+0000 7f3e40fdb740 1 -- 172.21.15.117:0/2595610254 --> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a541bd0ea0 con 0x55a541cefb80 2024-06-03T03:27:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.579+0000 7f3e36a00640 1 --2- 172.21.15.117:0/2595610254 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55a541cefb80 0x55a541cf1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.579+0000 7f3e20c00640 10 client.15630 ms_handle_connect on v2:172.21.15.183:6838/1439615083 2024-06-03T03:27:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.579+0000 7f3e36a00640 1 --2- 172.21.15.117:0/2595610254 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55a541cefb80 0x55a541cf1f70 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9401297760c4537 server_cookie=de4fbf0211cef2c0 in_seq=0 out_seq=0 2024-06-03T03:27:54.591 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.590+0000 7f3e20c00640 1 -- 172.21.15.117:0/2595610254 <== mds.0 v2:172.21.15.183:6838/1439615083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55a541bd0ea0 con 0x55a541cefb80 2024-06-03T03:27:54.591 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.590+0000 7f3e20c00640 10 client.15630 handle_command_reply: tid=0 2024-06-03T03:27:54.591 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.590+0000 7f3e40fdb740 1 client.15630 shutdown 2024-06-03T03:27:54.591 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.590+0000 7f3e40fdb740 1 -- 172.21.15.117:0/2595610254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a541c65bd0 msgr2=0x55a541c74b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:54.591 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.590+0000 7f3e40fdb740 1 --2- 172.21.15.117:0/2595610254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a541c65bd0 0x55a541c74b90 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3e28004b90 tx=0x7f3e28003e80 comp rx=0 tx=0).stop 2024-06-03T03:27:54.591 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.590+0000 7f3e40fdb740 1 -- 172.21.15.117:0/2595610254 shutdown_connections 2024-06-03T03:27:54.591 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.590+0000 7f3e40fdb740 1 --2- 172.21.15.117:0/2595610254 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x55a541cefb80 0x55a541cf1f70 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:54.591 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.590+0000 7f3e40fdb740 1 --2- 172.21.15.117:0/2595610254 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a541c6b730 0x55a541c760e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:54.591 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.590+0000 7f3e40fdb740 1 --2- 172.21.15.117:0/2595610254 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a541c665c0 0x55a541c750d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:54.591 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.590+0000 7f3e20c00640 0 client.15630 ms_handle_reset on v2:172.21.15.183:6838/1439615083 2024-06-03T03:27:54.591 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.590+0000 7f3e40fdb740 1 --2- 172.21.15.117:0/2595610254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a541c65bd0 0x55a541c74b90 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:54.591 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.590+0000 7f3e40fdb740 1 -- 172.21.15.117:0/2595610254 >> 172.21.15.117:0/2595610254 conn(0x55a541c2df80 msgr2=0x55a541c60810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:27:54.592 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.590+0000 7f3e40fdb740 1 -- 172.21.15.117:0/2595610254 shutdown_connections 2024-06-03T03:27:54.592 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.590+0000 7f3e40fdb740 1 -- 172.21.15.117:0/2595610254 wait complete. 2024-06-03T03:27:54.592 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.590+0000 7f3e40fdb740 20 client.15630 trim_cache size 0 max 16384 2024-06-03T03:27:54.592 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:{ 2024-06-03T03:27:54.592 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "return_code": 0 2024-06-03T03:27:54.592 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:} 2024-06-03T03:27:54.593 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.591+0000 7f3e3e000640 1 -- 172.21.15.117:0/4108120868 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f3e1c0a2c30 msgr2=0x7f3e1c0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:54.593 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.591+0000 7f3e3e000640 1 --2- 172.21.15.117:0/4108120868 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f3e1c0a2c30 0x7f3e1c0a50f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3e240314e0 tx=0x7f3e24000bc0 comp rx=0 tx=0).stop 2024-06-03T03:27:54.593 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.591+0000 7f3e3e000640 1 -- 172.21.15.117:0/4108120868 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e381098b0 msgr2=0x7f3e381a13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:54.593 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.591+0000 7f3e3e000640 1 --2- 172.21.15.117:0/4108120868 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e381098b0 0x7f3e381a13d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f3e3000eb10 tx=0x7f3e3000c2d0 comp rx=0 tx=0).stop 2024-06-03T03:27:54.593 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.591+0000 7f3e3e000640 1 -- 172.21.15.117:0/4108120868 shutdown_connections 2024-06-03T03:27:54.593 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.591+0000 7f3e3e000640 1 --2- 172.21.15.117:0/4108120868 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e381098b0 0x7f3e381a13d0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:54.593 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.591+0000 7f3e3e000640 1 --2- 172.21.15.117:0/4108120868 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f3e1c0a2c30 0x7f3e1c0a50f0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:54.593 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.591+0000 7f3e3e000640 1 --2- 172.21.15.117:0/4108120868 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3e38075960 0x7f3e3819a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:54.593 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.591+0000 7f3e3e000640 1 --2- 172.21.15.117:0/4108120868 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e38074380 0x7f3e38199e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:54.593 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.591+0000 7f3e3e000640 1 -- 172.21.15.117:0/4108120868 >> 172.21.15.117:0/4108120868 conn(0x7f3e380fcf90 msgr2=0x7f3e38105840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:27:54.593 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.591+0000 7f3e3e000640 1 -- 172.21.15.117:0/4108120868 shutdown_connections 2024-06-03T03:27:54.593 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-06-03T03:27:54.591+0000 7f3e3e000640 1 -- 172.21.15.117:0/4108120868 wait complete. 2024-06-03T03:27:55.392 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:27:55 smithi117 ceph-mon[31649]: pgmap v584: 129 pgs: 129 active+clean; 258 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1.2 MiB/s wr, 14 op/s 2024-06-03T03:27:55.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:27:55 smithi151 ceph-mon[37043]: pgmap v584: 129 pgs: 129 active+clean; 258 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1.2 MiB/s wr, 14 op/s 2024-06-03T03:27:55.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:27:55 smithi183 ceph-mon[37326]: pgmap v584: 129 pgs: 129 active+clean; 258 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1.2 MiB/s wr, 14 op/s 2024-06-03T03:27:55.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.724+0000 7f788c400700 1 -- 172.21.15.117:0/1945217871 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f78841084f0 msgr2=0x7f78841088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:55.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.724+0000 7f788c400700 1 --2- 172.21.15.117:0/1945217871 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f78841084f0 0x7f78841088d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f78780098f0 tx=0x7f7878009c00 comp rx=0 tx=0).stop 2024-06-03T03:27:55.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.724+0000 7f788c400700 1 -- 172.21.15.117:0/1945217871 shutdown_connections 2024-06-03T03:27:55.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.725+0000 7f788c400700 1 --2- 172.21.15.117:0/1945217871 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f788406b3c0 0x7f788410f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:55.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.725+0000 7f788c400700 1 --2- 172.21.15.117:0/1945217871 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f78841084f0 0x7f78841088d0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:55.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.725+0000 7f788c400700 1 --2- 172.21.15.117:0/1945217871 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f788406aa00 0x7f788406ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:55.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.725+0000 7f788c400700 1 -- 172.21.15.117:0/1945217871 >> 172.21.15.117:0/1945217871 conn(0x7f7884078300 msgr2=0x7f7884078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:27:55.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.725+0000 7f788c400700 1 -- 172.21.15.117:0/1945217871 shutdown_connections 2024-06-03T03:27:55.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.725+0000 7f788c400700 1 -- 172.21.15.117:0/1945217871 wait complete. 2024-06-03T03:27:55.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.726+0000 7f788c400700 1 Processor -- start 2024-06-03T03:27:55.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.726+0000 7f788c400700 1 -- start start 2024-06-03T03:27:55.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.726+0000 7f788c400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f788406aa00 0x7f7884101450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:55.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.726+0000 7f788c400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f788406b3c0 0x7f7884101990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:55.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.727+0000 7f788c400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78841084f0 0x7f7884101ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:55.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.727+0000 7f788c400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7884112770 con 0x7f788406b3c0 2024-06-03T03:27:55.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.727+0000 7f788c400700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f78841125f0 con 0x7f788406aa00 2024-06-03T03:27:55.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.727+0000 7f788c400700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f78841128f0 con 0x7f78841084f0 2024-06-03T03:27:55.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.727+0000 7f7889a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f788406aa00 0x7f7884101450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:55.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.727+0000 7f7889a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f788406aa00 0x7f7884101450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:53614/0 (socket says 172.21.15.117:53614) 2024-06-03T03:27:55.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.727+0000 7f7889a00700 1 -- 172.21.15.117:0/1501412443 learned_addr learned my addr 172.21.15.117:0/1501412443 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:27:55.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.727+0000 7f788a400700 1 --2- 172.21.15.117:0/1501412443 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78841084f0 0x7f7884101ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:55.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.727+0000 7f7889000700 1 --2- 172.21.15.117:0/1501412443 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f788406b3c0 0x7f7884101990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:55.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.727+0000 7f7889a00700 1 -- 172.21.15.117:0/1501412443 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78841084f0 msgr2=0x7f7884101ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:55.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.727+0000 7f7889a00700 1 --2- 172.21.15.117:0/1501412443 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78841084f0 0x7f7884101ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:55.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.727+0000 7f7889a00700 1 -- 172.21.15.117:0/1501412443 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f788406b3c0 msgr2=0x7f7884101990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:55.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.727+0000 7f7889a00700 1 --2- 172.21.15.117:0/1501412443 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f788406b3c0 0x7f7884101990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:55.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.727+0000 7f7889a00700 1 -- 172.21.15.117:0/1501412443 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78841a71d0 con 0x7f788406aa00 2024-06-03T03:27:55.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.727+0000 7f7889000700 1 --2- 172.21.15.117:0/1501412443 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f788406b3c0 0x7f7884101990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T03:27:55.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.728+0000 7f7889a00700 1 --2- 172.21.15.117:0/1501412443 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f788406aa00 0x7f7884101450 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7878009230 tx=0x7f787800fa70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:27:55.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.728+0000 7f7882000700 1 -- 172.21.15.117:0/1501412443 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7878005200 con 0x7f788406aa00 2024-06-03T03:27:55.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.728+0000 7f7882000700 1 -- 172.21.15.117:0/1501412443 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f787800b560 con 0x7f788406aa00 2024-06-03T03:27:55.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.728+0000 7f788c400700 1 -- 172.21.15.117:0/1501412443 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78841a7460 con 0x7f788406aa00 2024-06-03T03:27:55.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.728+0000 7f7882000700 1 -- 172.21.15.117:0/1501412443 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7878003b10 con 0x7f788406aa00 2024-06-03T03:27:55.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.728+0000 7f788c400700 1 -- 172.21.15.117:0/1501412443 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78841a78b0 con 0x7f788406aa00 2024-06-03T03:27:55.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.732+0000 7f7882000700 1 -- 172.21.15.117:0/1501412443 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f7878029060 con 0x7f788406aa00 2024-06-03T03:27:55.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.733+0000 7f7896d98b80 1 Processor -- start 2024-06-03T03:27:55.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.734+0000 7f7896d98b80 1 -- start start 2024-06-03T03:27:55.735 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.734+0000 7f7882000700 1 --2- 172.21.15.117:0/1501412443 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f786c0a2ca0 0x7f786c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:55.735 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.734+0000 7f7896d98b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556b68612b90 0x556b68612f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:55.735 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.734+0000 7f7896d98b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556b686134b0 0x556b6860cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:55.735 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.734+0000 7f7896d98b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b6860d240 0x556b6860d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:55.735 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.734+0000 7f7896d98b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556b6851ba10 con 0x556b6860d240 2024-06-03T03:27:55.735 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.734+0000 7f7896d98b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x556b6855eb00 con 0x556b68612b90 2024-06-03T03:27:55.735 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.734+0000 7f7896d98b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556b685198f0 con 0x556b686134b0 2024-06-03T03:27:55.735 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.734+0000 7f7882000700 1 -- 172.21.15.117:0/1501412443 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f78780cfc40 con 0x7f788406aa00 2024-06-03T03:27:55.736 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.735+0000 7f7889000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556b686134b0 0x556b6860cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:55.736 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.735+0000 7f7889000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556b686134b0 0x556b6860cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:57934/0 (socket says 172.21.15.117:57934) 2024-06-03T03:27:55.736 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.735+0000 7f7889000700 1 -- 172.21.15.117:0/1898229637 learned_addr learned my addr 172.21.15.117:0/1898229637 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:27:55.736 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.735+0000 7f788a400700 1 --2- 172.21.15.117:0/1898229637 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b6860d240 0x556b6860d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:55.736 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.735+0000 7f788a400700 1 --2- 172.21.15.117:0/1501412443 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f786c0a2ca0 0x7f786c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:55.736 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.735+0000 7f7889000700 1 -- 172.21.15.117:0/1898229637 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556b68612b90 msgr2=0x556b68612f70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-03T03:27:55.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.735+0000 7f7889000700 1 --2- 172.21.15.117:0/1898229637 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556b68612b90 0x556b68612f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:55.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.735+0000 7f7889000700 1 -- 172.21.15.117:0/1898229637 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b6860d240 msgr2=0x556b6860d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:55.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.735+0000 7f7889000700 1 --2- 172.21.15.117:0/1898229637 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b6860d240 0x556b6860d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:55.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.735+0000 7f7889000700 1 -- 172.21.15.117:0/1898229637 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b68506f90 con 0x556b686134b0 2024-06-03T03:27:55.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.735+0000 7f7889000700 1 --2- 172.21.15.117:0/1898229637 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556b686134b0 0x556b6860cd00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7870009d30 tx=0x7f78700116c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9a1441aae7b5300 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:27:55.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.736+0000 7f788a400700 1 --2- 172.21.15.117:0/1501412443 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f786c0a2ca0 0x7f786c0a5160 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f787c009550 tx=0x7f787c006040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:27:55.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.736+0000 7f7876a00700 1 -- 172.21.15.117:0/1898229637 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7870012450 con 0x556b686134b0 2024-06-03T03:27:55.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.736+0000 7f7876a00700 1 -- 172.21.15.117:0/1898229637 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7870012ad0 con 0x556b686134b0 2024-06-03T03:27:55.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.736+0000 7f7876a00700 1 -- 172.21.15.117:0/1898229637 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f787000f970 con 0x556b686134b0 2024-06-03T03:27:55.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.736+0000 7f7896d98b80 1 -- 172.21.15.117:0/1898229637 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556b686134b0 msgr2=0x556b6860cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:55.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.736+0000 7f7896d98b80 1 --2- 172.21.15.117:0/1898229637 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556b686134b0 0x556b6860cd00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7870009d30 tx=0x7f78700116c0 comp rx=0 tx=0).stop 2024-06-03T03:27:55.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.737+0000 7f7896d98b80 1 -- 172.21.15.117:0/1898229637 shutdown_connections 2024-06-03T03:27:55.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.737+0000 7f7896d98b80 1 --2- 172.21.15.117:0/1898229637 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b6860d240 0x556b6860d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:55.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.737+0000 7f7896d98b80 1 --2- 172.21.15.117:0/1898229637 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556b68612b90 0x556b68612f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:55.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.737+0000 7f7896d98b80 1 --2- 172.21.15.117:0/1898229637 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556b686134b0 0x556b6860cd00 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:55.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.737+0000 7f7896d98b80 1 -- 172.21.15.117:0/1898229637 >> 172.21.15.117:0/1898229637 conn(0x556b68539170 msgr2=0x556b68607ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:27:55.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.737+0000 7f7896d98b80 1 -- 172.21.15.117:0/1898229637 shutdown_connections 2024-06-03T03:27:55.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.737+0000 7f7896d98b80 1 -- 172.21.15.117:0/1898229637 wait complete. 2024-06-03T03:27:55.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.737+0000 7f7896d98b80 1 -- start start 2024-06-03T03:27:55.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.738+0000 7f7896d98b80 1 Processor -- start 2024-06-03T03:27:55.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.738+0000 7f7896d98b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556b6860d240 0x556b68619270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:55.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.738+0000 7f7896d98b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556b68612b90 0x556b68616570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:55.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.738+0000 7f7896d98b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b686134b0 0x556b68616ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:55.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.738+0000 7f7896d98b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556b684faf60 con 0x556b686134b0 2024-06-03T03:27:55.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.738+0000 7f7896d98b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x556b6855aac0 con 0x556b6860d240 2024-06-03T03:27:55.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.738+0000 7f7896d98b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556b68619e90 con 0x556b68612b90 2024-06-03T03:27:55.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.738+0000 7f7896d98b80 10 client.0 _ll_register_callbacks cb 0x556b684f4380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:27:55.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.739+0000 7f7889000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556b68612b90 0x556b68616570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:55.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.739+0000 7f7889000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556b68612b90 0x556b68616570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:57936/0 (socket says 172.21.15.117:57936) 2024-06-03T03:27:55.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.739+0000 7f7889000700 1 -- 172.21.15.117:0/2526103734 learned_addr learned my addr 172.21.15.117:0/2526103734 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:27:55.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.739+0000 7f7889a00700 1 --2- 172.21.15.117:0/2526103734 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556b6860d240 0x556b68619270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:55.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.739+0000 7f788a400700 1 --2- 172.21.15.117:0/2526103734 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b686134b0 0x556b68616ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:55.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.739+0000 7f7889000700 1 -- 172.21.15.117:0/2526103734 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556b6860d240 msgr2=0x556b68619270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:55.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.739+0000 7f7889000700 1 --2- 172.21.15.117:0/2526103734 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556b6860d240 0x556b68619270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:55.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.739+0000 7f7889000700 1 -- 172.21.15.117:0/2526103734 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b686134b0 msgr2=0x556b68616ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:55.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.739+0000 7f7889000700 1 --2- 172.21.15.117:0/2526103734 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b686134b0 0x556b68616ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:55.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.739+0000 7f7889000700 1 -- 172.21.15.117:0/2526103734 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b684ad8d0 con 0x556b68612b90 2024-06-03T03:27:55.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.739+0000 7f788a400700 1 --2- 172.21.15.117:0/2526103734 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b686134b0 0x556b68616ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T03:27:55.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.739+0000 7f7889000700 1 --2- 172.21.15.117:0/2526103734 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556b68612b90 0x556b68616570 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7870005b40 tx=0x7f7870011fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c437cc1ac595e8eb server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:27:55.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.740+0000 7f786b400700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-06-03T03:27:55.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.740+0000 7f786b400700 1 -- 172.21.15.117:0/2526103734 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7870028020 con 0x556b68612b90 2024-06-03T03:27:55.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.740+0000 7f786b400700 1 -- 172.21.15.117:0/2526103734 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f7870004750 con 0x556b68612b90 2024-06-03T03:27:55.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.740+0000 7f7896d98b80 1 -- 172.21.15.117:0/2526103734 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7870012ad0 con 0x556b68612b90 2024-06-03T03:27:55.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.740+0000 7f786b400700 1 -- 172.21.15.117:0/2526103734 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7870033050 con 0x556b68612b90 2024-06-03T03:27:55.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.740+0000 7f7896d98b80 10 client.25789 fetch_fsmap learned FSMap version 31 2024-06-03T03:27:55.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.740+0000 7f7896d98b80 1 -- 172.21.15.117:0/2526103734 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x556b68624670 con 0x556b68612b90 2024-06-03T03:27:55.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.741+0000 7f786b400700 1 -- 172.21.15.117:0/2526103734 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7870012ad0 con 0x556b68612b90 2024-06-03T03:27:55.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.741+0000 7f786b400700 1 -- 172.21.15.117:0/2526103734 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f787000f5f0 con 0x556b68612b90 2024-06-03T03:27:55.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.741+0000 7f7896d98b80 10 client.25789 fetch_fsmap finished waiting for FSMap version 31 2024-06-03T03:27:55.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.741+0000 7f7896d98b80 10 client.25789 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-03T03:27:55.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.741+0000 7f7896d98b80 20 client.25789 populate_metadata read hostname 'smithi117' 2024-06-03T03:27:55.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.741+0000 7f7896d98b80 1 --2- 172.21.15.117:0/2526103734 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x556b68626cb0 0x556b686290a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:55.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.741+0000 7f7896d98b80 4 client.25789 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T03:27:55.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.741+0000 7f7896d98b80 1 -- 172.21.15.117:0/2526103734 --> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b6856c000 con 0x556b68626cb0 2024-06-03T03:27:55.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.741+0000 7f7889a00700 1 --2- 172.21.15.117:0/2526103734 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x556b68626cb0 0x556b686290a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:55.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.742+0000 7f7889a00700 1 --2- 172.21.15.117:0/2526103734 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x556b68626cb0 0x556b686290a0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5179f963f7e19d14 server_cookie=8c4948d50d766803 in_seq=0 out_seq=0 2024-06-03T03:27:55.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.742+0000 7f786b400700 10 client.25789 ms_handle_connect on v2:172.21.15.183:6838/1439615083 2024-06-03T03:27:55.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.743+0000 7f786b400700 1 -- 172.21.15.117:0/2526103734 <== mds.0 v2:172.21.15.183:6838/1439615083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556b6856c000 con 0x556b68626cb0 2024-06-03T03:27:55.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.743+0000 7f786b400700 10 client.25789 handle_command_reply: tid=0 2024-06-03T03:27:55.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.743+0000 7f7896d98b80 1 client.25789 shutdown 2024-06-03T03:27:55.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.743+0000 7f7896d98b80 1 -- 172.21.15.117:0/2526103734 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556b68612b90 msgr2=0x556b68616570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:55.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.743+0000 7f7896d98b80 1 --2- 172.21.15.117:0/2526103734 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556b68612b90 0x556b68616570 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7870005b40 tx=0x7f7870011fb0 comp rx=0 tx=0).stop 2024-06-03T03:27:55.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.743+0000 7f7896d98b80 1 -- 172.21.15.117:0/2526103734 shutdown_connections 2024-06-03T03:27:55.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.743+0000 7f7896d98b80 1 --2- 172.21.15.117:0/2526103734 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x556b68626cb0 0x556b686290a0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:55.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.743+0000 7f7896d98b80 1 --2- 172.21.15.117:0/2526103734 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b686134b0 0x556b68616ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:55.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.743+0000 7f7896d98b80 1 --2- 172.21.15.117:0/2526103734 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556b6860d240 0x556b68619270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:55.746 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.743+0000 7f7896d98b80 1 --2- 172.21.15.117:0/2526103734 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556b68612b90 0x556b68616570 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:55.746 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.743+0000 7f7896d98b80 1 -- 172.21.15.117:0/2526103734 >> 172.21.15.117:0/2526103734 conn(0x556b68539170 msgr2=0x556b68601790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:27:55.746 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.743+0000 7f786b400700 0 client.25789 ms_handle_reset on v2:172.21.15.183:6838/1439615083 2024-06-03T03:27:55.746 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.743+0000 7f7896d98b80 1 -- 172.21.15.117:0/2526103734 shutdown_connections 2024-06-03T03:27:55.746 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.744+0000 7f7896d98b80 1 -- 172.21.15.117:0/2526103734 wait complete. 2024-06-03T03:27:55.746 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.744+0000 7f7896d98b80 20 client.25789 trim_cache size 0 max 16384 2024-06-03T03:27:55.790 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.789+0000 7f7896d98b80 1 Processor -- start 2024-06-03T03:27:55.790 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.790+0000 7f7896d98b80 1 -- start start 2024-06-03T03:27:55.791 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.790+0000 7f7896d98b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556b6860d240 0x556b6860cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:55.791 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.790+0000 7f7896d98b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556b68612b90 0x556b686168a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:55.791 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.790+0000 7f7896d98b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b686134b0 0x556b686141c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:55.791 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.790+0000 7f7896d98b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f78841b0380 con 0x556b686134b0 2024-06-03T03:27:55.791 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.790+0000 7f7896d98b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f78841b1ec0 con 0x556b6860d240 2024-06-03T03:27:55.792 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.790+0000 7f7896d98b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f78841b0a50 con 0x556b68612b90 2024-06-03T03:27:55.792 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.791+0000 7f7889a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556b6860d240 0x556b6860cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:55.792 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.791+0000 7f7889000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556b68612b90 0x556b686168a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:55.792 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.791+0000 7f7889a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556b6860d240 0x556b6860cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:53638/0 (socket says 172.21.15.117:53638) 2024-06-03T03:27:55.792 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.791+0000 7f7889000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556b68612b90 0x556b686168a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:57944/0 (socket says 172.21.15.117:57944) 2024-06-03T03:27:55.792 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.791+0000 7f7889a00700 1 -- 172.21.15.117:0/534019061 learned_addr learned my addr 172.21.15.117:0/534019061 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:27:55.792 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.791+0000 7f788a400700 1 --2- 172.21.15.117:0/534019061 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b686134b0 0x556b686141c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:55.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.791+0000 7f7889000700 1 -- 172.21.15.117:0/534019061 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556b6860d240 msgr2=0x556b6860cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:55.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.791+0000 7f7889000700 1 --2- 172.21.15.117:0/534019061 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556b6860d240 0x556b6860cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:55.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.791+0000 7f7889000700 1 -- 172.21.15.117:0/534019061 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b686134b0 msgr2=0x556b686141c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:55.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.791+0000 7f7889000700 1 --2- 172.21.15.117:0/534019061 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b686134b0 0x556b686141c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:55.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.791+0000 7f7889000700 1 -- 172.21.15.117:0/534019061 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b686174b0 con 0x556b68612b90 2024-06-03T03:27:55.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.791+0000 7f7889a00700 1 --2- 172.21.15.117:0/534019061 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556b6860d240 0x556b6860cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:27:55.794 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.791+0000 7f7889000700 1 --2- 172.21.15.117:0/534019061 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556b68612b90 0x556b686168a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7870006010 tx=0x7f7870012500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b51e023cb5d4a39b server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:27:55.794 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.792+0000 7f7876000700 1 -- 172.21.15.117:0/534019061 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f787003c070 con 0x556b68612b90 2024-06-03T03:27:55.794 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.792+0000 7f7876000700 1 -- 172.21.15.117:0/534019061 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f787000fc30 con 0x556b68612b90 2024-06-03T03:27:55.794 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.792+0000 7f7876000700 1 -- 172.21.15.117:0/534019061 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7870028040 con 0x556b68612b90 2024-06-03T03:27:55.794 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.793+0000 7f7896d98b80 1 -- 172.21.15.117:0/534019061 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556b68612b90 msgr2=0x556b686168a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:55.794 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.793+0000 7f7896d98b80 1 --2- 172.21.15.117:0/534019061 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556b68612b90 0x556b686168a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7870006010 tx=0x7f7870012500 comp rx=0 tx=0).stop 2024-06-03T03:27:55.794 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.793+0000 7f7896d98b80 1 -- 172.21.15.117:0/534019061 shutdown_connections 2024-06-03T03:27:55.794 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.793+0000 7f7896d98b80 1 --2- 172.21.15.117:0/534019061 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b686134b0 0x556b686141c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:55.794 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.793+0000 7f7896d98b80 1 --2- 172.21.15.117:0/534019061 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556b6860d240 0x556b6860cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:55.794 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.793+0000 7f7896d98b80 1 --2- 172.21.15.117:0/534019061 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556b68612b90 0x556b686168a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:55.794 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.793+0000 7f7896d98b80 1 -- 172.21.15.117:0/534019061 >> 172.21.15.117:0/534019061 conn(0x556b68539170 msgr2=0x556b68614e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:27:55.794 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.793+0000 7f7896d98b80 1 -- 172.21.15.117:0/534019061 shutdown_connections 2024-06-03T03:27:55.795 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.793+0000 7f7896d98b80 1 -- 172.21.15.117:0/534019061 wait complete. 2024-06-03T03:27:55.795 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.794+0000 7f7896d98b80 1 -- start start 2024-06-03T03:27:55.795 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.794+0000 7f7896d98b80 1 Processor -- start 2024-06-03T03:27:55.796 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.795+0000 7f7896d98b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b6860d240 0x556b6861f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:55.796 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.795+0000 7f7896d98b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556b68612b90 0x556b6861f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:55.796 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.795+0000 7f7896d98b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556b686134b0 0x556b6862da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:55.796 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.795+0000 7f7896d98b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f78841b17f0 con 0x556b6860d240 2024-06-03T03:27:55.796 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.795+0000 7f7896d98b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f78841b1120 con 0x556b68612b90 2024-06-03T03:27:55.796 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.795+0000 7f7896d98b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f78841ae320 con 0x556b686134b0 2024-06-03T03:27:55.796 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.795+0000 7f7896d98b80 10 client.0 _ll_register_callbacks cb 0x556b684f4380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:27:55.796 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.795+0000 7f7889000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556b68612b90 0x556b6861f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:55.796 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.795+0000 7f7889000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556b68612b90 0x556b6861f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:53654/0 (socket says 172.21.15.117:53654) 2024-06-03T03:27:55.796 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.795+0000 7f7889000700 1 -- 172.21.15.117:0/239577937 learned_addr learned my addr 172.21.15.117:0/239577937 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:27:55.796 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.795+0000 7f788a400700 1 --2- 172.21.15.117:0/239577937 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556b686134b0 0x556b6862da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:55.796 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.795+0000 7f7889a00700 1 --2- 172.21.15.117:0/239577937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b6860d240 0x556b6861f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:55.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.795+0000 7f7889000700 1 -- 172.21.15.117:0/239577937 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556b686134b0 msgr2=0x556b6862da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:55.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.795+0000 7f7889000700 1 --2- 172.21.15.117:0/239577937 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556b686134b0 0x556b6862da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:55.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.795+0000 7f7889000700 1 -- 172.21.15.117:0/239577937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b6860d240 msgr2=0x556b6861f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:27:55.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.796+0000 7f7889000700 1 --2- 172.21.15.117:0/239577937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b6860d240 0x556b6861f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:55.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.796+0000 7f7889000700 1 -- 172.21.15.117:0/239577937 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b6862e2a0 con 0x556b68612b90 2024-06-03T03:27:55.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.796+0000 7f7889a00700 1 --2- 172.21.15.117:0/239577937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b6860d240 0x556b6861f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T03:27:55.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.796+0000 7f7889000700 1 --2- 172.21.15.117:0/239577937 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556b68612b90 0x556b6861f8b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f78700095c0 tx=0x7f787000fe00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56f3c4d360307f6d server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:27:55.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.796+0000 7f786b400700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-06-03T03:27:55.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.796+0000 7f786b400700 1 -- 172.21.15.117:0/239577937 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f787003c070 con 0x556b68612b90 2024-06-03T03:27:55.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.796+0000 7f786b400700 1 -- 172.21.15.117:0/239577937 <== mon.2 v2:172.21.15.151:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f787000f750 con 0x556b68612b90 2024-06-03T03:27:55.799 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.796+0000 7f7896d98b80 1 -- 172.21.15.117:0/239577937 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f787000fc30 con 0x556b68612b90 2024-06-03T03:27:55.799 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.797+0000 7f786b400700 1 -- 172.21.15.117:0/239577937 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f787004a050 con 0x556b68612b90 2024-06-03T03:27:55.799 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.797+0000 7f7896d98b80 10 client.25757 fetch_fsmap learned FSMap version 31 2024-06-03T03:27:55.799 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.797+0000 7f7896d98b80 1 -- 172.21.15.117:0/239577937 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x556b68691120 con 0x556b68612b90 2024-06-03T03:27:55.799 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.797+0000 7f786b400700 1 -- 172.21.15.117:0/239577937 <== mon.2 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f787000fc30 con 0x556b68612b90 2024-06-03T03:27:55.799 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.797+0000 7f786b400700 1 -- 172.21.15.117:0/239577937 <== mon.2 v2:172.21.15.151:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f7870004be0 con 0x556b68612b90 2024-06-03T03:27:55.799 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.797+0000 7f7896d98b80 10 client.25757 fetch_fsmap finished waiting for FSMap version 31 2024-06-03T03:27:55.799 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.797+0000 7f7896d98b80 10 client.25757 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-03T03:27:55.799 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.797+0000 7f7896d98b80 20 client.25757 populate_metadata read hostname 'smithi117' 2024-06-03T03:27:55.799 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.797+0000 7f7896d98b80 1 --2- 172.21.15.117:0/239577937 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x556b68691740 0x556b68693b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:55.799 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.798+0000 7f7896d98b80 4 client.25757 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-03T03:27:55.799 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.798+0000 7f7896d98b80 1 -- 172.21.15.117:0/239577937 --> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556b68566580 con 0x556b68691740 2024-06-03T03:27:55.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.798+0000 7f7889a00700 1 --2- 172.21.15.117:0/239577937 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x556b68691740 0x556b68693b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:55.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.798+0000 7f786b400700 10 client.25757 ms_handle_connect on v2:172.21.15.183:6838/1439615083 2024-06-03T03:27:55.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.798+0000 7f7889a00700 1 --2- 172.21.15.117:0/239577937 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x556b68691740 0x556b68693b60 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f457c7520c626079 server_cookie=d3d3fa5734648d91 in_seq=0 out_seq=0 2024-06-03T03:27:55.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.799+0000 7f786b400700 1 -- 172.21.15.117:0/239577937 <== mds.0 v2:172.21.15.183:6838/1439615083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x556b68566580 con 0x556b68691740 2024-06-03T03:27:55.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.799+0000 7f786b400700 10 client.25757 handle_command_reply: tid=0 2024-06-03T03:27:55.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.799+0000 7f7896d98b80 1 client.25757 shutdown 2024-06-03T03:27:55.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.799+0000 7f7896d98b80 1 -- 172.21.15.117:0/239577937 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556b68612b90 msgr2=0x556b6861f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:55.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.799+0000 7f7896d98b80 1 --2- 172.21.15.117:0/239577937 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556b68612b90 0x556b6861f8b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f78700095c0 tx=0x7f787000fe00 comp rx=0 tx=0).stop 2024-06-03T03:27:55.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.799+0000 7f7896d98b80 1 -- 172.21.15.117:0/239577937 shutdown_connections 2024-06-03T03:27:55.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.799+0000 7f7896d98b80 1 --2- 172.21.15.117:0/239577937 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x556b68691740 0x556b68693b60 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:55.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.799+0000 7f7896d98b80 1 --2- 172.21.15.117:0/239577937 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556b686134b0 0x556b6862da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:55.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.799+0000 7f7896d98b80 1 --2- 172.21.15.117:0/239577937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b6860d240 0x556b6861f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:55.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.799+0000 7f7896d98b80 1 --2- 172.21.15.117:0/239577937 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556b68612b90 0x556b6861f8b0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:55.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.799+0000 7f7896d98b80 1 -- 172.21.15.117:0/239577937 >> 172.21.15.117:0/239577937 conn(0x556b68539170 msgr2=0x556b68615710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:27:55.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.799+0000 7f786b400700 0 client.25757 ms_handle_reset on v2:172.21.15.183:6838/1439615083 2024-06-03T03:27:55.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.799+0000 7f7896d98b80 1 -- 172.21.15.117:0/239577937 shutdown_connections 2024-06-03T03:27:55.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.799+0000 7f7896d98b80 1 -- 172.21.15.117:0/239577937 wait complete. 2024-06-03T03:27:55.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.800+0000 7f7896d98b80 20 client.25757 trim_cache size 0 max 16384 2024-06-03T03:27:55.802 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-06-03T03:27:55.802 INFO:teuthology.orchestra.run.smithi117.stdout: "status": "no active scrubs running", 2024-06-03T03:27:55.802 INFO:teuthology.orchestra.run.smithi117.stdout: "scrubs": {} 2024-06-03T03:27:55.802 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-06-03T03:27:55.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.800+0000 7f788c400700 1 -- 172.21.15.117:0/1501412443 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f786c0a2ca0 msgr2=0x7f786c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:55.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.800+0000 7f788c400700 1 --2- 172.21.15.117:0/1501412443 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f786c0a2ca0 0x7f786c0a5160 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f787c009550 tx=0x7f787c006040 comp rx=0 tx=0).stop 2024-06-03T03:27:55.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.800+0000 7f788c400700 1 -- 172.21.15.117:0/1501412443 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f788406aa00 msgr2=0x7f7884101450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:55.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.800+0000 7f788c400700 1 --2- 172.21.15.117:0/1501412443 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f788406aa00 0x7f7884101450 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7878009230 tx=0x7f787800fa70 comp rx=0 tx=0).stop 2024-06-03T03:27:55.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.800+0000 7f788c400700 1 -- 172.21.15.117:0/1501412443 shutdown_connections 2024-06-03T03:27:55.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.800+0000 7f788c400700 1 --2- 172.21.15.117:0/1501412443 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f786c0a2ca0 0x7f786c0a5160 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:55.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.800+0000 7f788c400700 1 --2- 172.21.15.117:0/1501412443 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78841084f0 0x7f7884101ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:55.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.800+0000 7f788c400700 1 --2- 172.21.15.117:0/1501412443 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f788406aa00 0x7f7884101450 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:55.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.800+0000 7f788c400700 1 --2- 172.21.15.117:0/1501412443 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f788406b3c0 0x7f7884101990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:55.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.800+0000 7f788c400700 1 -- 172.21.15.117:0/1501412443 >> 172.21.15.117:0/1501412443 conn(0x7f7884078300 msgr2=0x7f7884100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:27:55.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.801+0000 7f788c400700 1 -- 172.21.15.117:0/1501412443 shutdown_connections 2024-06-03T03:27:55.804 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:55.801+0000 7f788c400700 1 -- 172.21.15.117:0/1501412443 wait complete. 2024-06-03T03:27:56.380 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:27:56 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:27:56.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:27:56 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:27:56.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:27:56 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:27:56.950 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-03T03:27:56.950 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-03T03:27:57.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:27:57 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:27:57.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:27:57 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:27:57.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:27:57 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:27:57.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:27:57 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:27:57.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:27:57 smithi117 ceph-mon[31649]: Adjusting osd_memory_target on smithi117 to 176.4M 2024-06-03T03:27:57.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:27:57 smithi117 ceph-mon[31649]: Unable to set osd_memory_target on smithi117 to 185061990: error parsing value: Value '185061990' is below minimum 939524096 2024-06-03T03:27:57.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:27:57 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:27:57.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:27:57 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:27:57.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:27:57 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:27:57.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:27:57 smithi117 ceph-mon[31649]: pgmap v585: 129 pgs: 129 active+clean; 258 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 860 KiB/s wr, 13 op/s 2024-06-03T03:27:57.372 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:27:57.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:27:57 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:27:57.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:27:57 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:27:57.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:27:57 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:27:57.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:27:57 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:27:57.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:27:57 smithi151 ceph-mon[37043]: Adjusting osd_memory_target on smithi117 to 176.4M 2024-06-03T03:27:57.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:27:57 smithi151 ceph-mon[37043]: Unable to set osd_memory_target on smithi117 to 185061990: error parsing value: Value '185061990' is below minimum 939524096 2024-06-03T03:27:57.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:27:57 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:27:57.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:27:57 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:27:57.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:27:57 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:27:57.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:27:57 smithi151 ceph-mon[37043]: pgmap v585: 129 pgs: 129 active+clean; 258 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 860 KiB/s wr, 13 op/s 2024-06-03T03:27:57.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:27:57 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:27:57.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:27:57 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:27:57.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:27:57 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:27:57.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:27:57 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:27:57.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:27:57 smithi183 ceph-mon[37326]: Adjusting osd_memory_target on smithi117 to 176.4M 2024-06-03T03:27:57.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:27:57 smithi183 ceph-mon[37326]: Unable to set osd_memory_target on smithi117 to 185061990: error parsing value: Value '185061990' is below minimum 939524096 2024-06-03T03:27:57.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:27:57 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:27:57.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:27:57 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:27:57.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:27:57 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:27:57.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:27:57 smithi183 ceph-mon[37326]: pgmap v585: 129 pgs: 129 active+clean; 258 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 860 KiB/s wr, 13 op/s 2024-06-03T03:27:58.202 INFO:tasks.workunit.client.0.smithi117.stdout:setting up random write file 2024-06-03T03:27:58.202 INFO:tasks.workunit.client.0.smithi117.stdout:done setting up random write file 2024-06-03T03:27:58.202 INFO:tasks.workunit.client.0.smithi117.stdout:starting fsync run 2024-06-03T03:27:58.202 INFO:tasks.workunit.client.0.smithi117.stdout:write time: 0.0012s fsync time: 0.0284s 2024-06-03T03:27:58.202 INFO:tasks.workunit.client.0.smithi117.stdout:write time: 0.0006s fsync time: 0.0242s 2024-06-03T03:27:58.202 INFO:tasks.workunit.client.0.smithi117.stdout:write time: 0.0020s fsync time: 0.0281s 2024-06-03T03:27:58.202 INFO:tasks.workunit.client.0.smithi117.stdout:write time: 0.0008s fsync time: 0.0197s 2024-06-03T03:27:58.203 INFO:tasks.workunit.client.0.smithi117.stdout:write time: 0.0007s fsync time: 0.0172s 2024-06-03T03:27:58.203 INFO:tasks.workunit.client.0.smithi117.stdout:write time: 0.0006s fsync time: 0.0121s 2024-06-03T03:27:58.203 INFO:tasks.workunit.client.0.smithi117.stdout:write time: 0.0009s fsync time: 0.0114s 2024-06-03T03:27:58.203 INFO:tasks.workunit.client.0.smithi117.stdout:write time: 0.0009s fsync time: 0.0188s 2024-06-03T03:27:58.203 INFO:tasks.workunit.client.0.smithi117.stdout:write time: 0.0009s fsync time: 0.0176s 2024-06-03T03:27:58.203 INFO:tasks.workunit.client.0.smithi117.stdout:write time: 0.0013s fsync time: 0.0214s 2024-06-03T03:27:58.203 INFO:tasks.workunit.client.0.smithi117.stdout:write time: 0.0014s fsync time: 0.0215s 2024-06-03T03:27:58.203 INFO:tasks.workunit.client.0.smithi117.stdout:write time: 0.0013s fsync time: 0.0227s 2024-06-03T03:27:58.203 INFO:tasks.workunit.client.0.smithi117.stdout:write time: 0.0007s fsync time: 0.0150s 2024-06-03T03:27:58.203 INFO:tasks.workunit.client.0.smithi117.stdout:write time: 0.0013s fsync time: 0.0160s 2024-06-03T03:27:58.203 INFO:tasks.workunit.client.0.smithi117.stdout:write time: 0.0016s fsync time: 0.0164s 2024-06-03T03:27:58.203 INFO:tasks.workunit.client.0.smithi117.stdout:write time: 0.0009s fsync time: 0.0159s 2024-06-03T03:27:58.203 INFO:tasks.workunit.client.0.smithi117.stdout:write time: 0.0014s fsync time: 0.0233s 2024-06-03T03:27:58.203 INFO:tasks.workunit.client.0.smithi117.stdout:write time: 0.0008s fsync time: 0.0143s 2024-06-03T03:27:58.204 INFO:tasks.workunit.client.0.smithi117.stdout:write time: 0.0013s fsync time: 0.0228s 2024-06-03T03:27:58.204 INFO:tasks.workunit.client.0.smithi117.stdout:write time: 0.0007s fsync time: 0.0100s 2024-06-03T03:27:58.204 INFO:tasks.workunit.client.0.smithi117.stdout:write time: 0.0013s fsync time: 0.0210s 2024-06-03T03:27:58.204 INFO:tasks.workunit.client.0.smithi117.stdout:write time: 0.0006s fsync time: 0.0113s 2024-06-03T03:27:58.204 INFO:tasks.workunit.client.0.smithi117.stdout:write time: 0.0014s fsync time: 0.0215s 2024-06-03T03:27:58.204 INFO:tasks.workunit.client.0.smithi117.stdout:write time: 0.0009s fsync time: 0.0167s 2024-06-03T03:27:58.204 INFO:tasks.workunit.client.0.smithi117.stdout:write time: 0.0013s fsync time: 0.0214s 2024-06-03T03:27:58.204 INFO:tasks.workunit.client.0.smithi117.stdout:write time: 0.0014s fsync time: 0.0203s 2024-06-03T03:27:58.204 INFO:tasks.workunit.client.0.smithi117.stdout:write time: 0.0017s fsync time: 0.0259s 2024-06-03T03:27:58.204 INFO:tasks.workunit.client.0.smithi117.stdout:write time: 0.0013s fsync time: 0.0221s 2024-06-03T03:27:58.204 INFO:tasks.workunit.client.0.smithi117.stdout:write time: 0.0015s fsync time: 0.0210s 2024-06-03T03:27:58.204 INFO:tasks.workunit.client.0.smithi117.stdout:write time: 0.0014s fsync time: 0.0204s 2024-06-03T03:27:58.204 INFO:tasks.workunit.client.0.smithi117.stdout:write time: 0.0007s fsync time: 0.0115s 2024-06-03T03:27:58.204 INFO:tasks.workunit.client.0.smithi117.stdout:write time: 0.0014s fsync time: 0.0151s 2024-06-03T03:27:58.205 INFO:tasks.workunit.client.0.smithi117.stdout:write time: 0.0014s fsync time: 0.0276s 2024-06-03T03:27:58.205 INFO:tasks.workunit.client.0.smithi117.stdout:write time: 0.0014s fsync time: 0.0220s 2024-06-03T03:27:58.205 INFO:tasks.workunit.client.0.smithi117.stdout:write time: 0.0013s fsync time: 0.0259s 2024-06-03T03:27:58.205 INFO:tasks.workunit.client.0.smithi117.stdout:write time: 0.0020s fsync time: 0.0231s 2024-06-03T03:27:58.205 INFO:tasks.workunit.client.0.smithi117.stdout:write time: 0.0014s fsync time: 0.0198s 2024-06-03T03:27:58.205 INFO:tasks.workunit.client.0.smithi117.stdout:write time: 0.0014s fsync time: 0.0212s 2024-06-03T03:27:58.205 INFO:tasks.workunit.client.0.smithi117.stdout:write time: 0.0014s fsync time: 0.0207s 2024-06-03T03:27:58.205 INFO:tasks.workunit.client.0.smithi117.stdout:write time: 0.0013s fsync time: 0.0220s 2024-06-03T03:27:58.205 INFO:tasks.workunit.client.0.smithi117.stdout:write time: 0.0013s fsync time: 0.0180s 2024-06-03T03:27:58.205 INFO:tasks.workunit.client.0.smithi117.stdout:write time: 0.0014s fsync time: 0.0208s 2024-06-03T03:27:58.205 INFO:tasks.workunit.client.0.smithi117.stdout:write time: 0.0013s fsync time: 0.0198s 2024-06-03T03:27:58.205 INFO:tasks.workunit.client.0.smithi117.stdout:write time: 0.0013s fsync time: 0.0205s 2024-06-03T03:27:58.205 INFO:tasks.workunit.client.0.smithi117.stdout:write time: 0.0014s fsync time: 0.0272s 2024-06-03T03:27:58.205 INFO:tasks.workunit.client.0.smithi117.stdout:write time: 0.0013s fsync time: 0.0195s 2024-06-03T03:27:58.206 INFO:tasks.workunit.client.0.smithi117.stdout:write time: 0.0014s fsync time: 0.0210s 2024-06-03T03:27:58.206 INFO:tasks.workunit.client.0.smithi117.stdout:write time: 0.0015s fsync time: 0.0211s 2024-06-03T03:27:58.206 INFO:tasks.workunit.client.0.smithi117.stdout:write time: 0.0014s fsync time: 0.0212s 2024-06-03T03:27:58.206 INFO:tasks.workunit.client.0.smithi117.stdout:write time: 0.0014s fsync time: 0.0195s 2024-06-03T03:27:58.206 INFO:tasks.workunit.client.0.smithi117.stdout:write time: 0.0014s fsync time: 0.0288s 2024-06-03T03:27:58.206 INFO:tasks.workunit.client.0.smithi117.stdout:write time: 0.0014s fsync time: 0.0203s 2024-06-03T03:27:58.206 INFO:tasks.workunit.client.0.smithi117.stdout:write time: 0.0014s fsync time: 0.0237s 2024-06-03T03:27:58.206 INFO:tasks.workunit.client.0.smithi117.stdout:write time: 0.0013s fsync time: 0.0219s 2024-06-03T03:27:58.206 INFO:tasks.workunit.client.0.smithi117.stdout:write time: 0.0007s fsync time: 0.0115s 2024-06-03T03:27:58.206 INFO:tasks.workunit.client.0.smithi117.stdout:write time: 0.0014s fsync time: 0.0198s 2024-06-03T03:27:58.206 INFO:tasks.workunit.client.0.smithi117.stdout:write time: 0.0014s fsync time: 0.0215s 2024-06-03T03:27:58.206 INFO:tasks.workunit.client.0.smithi117.stdout:write time: 0.0009s fsync time: 0.0129s 2024-06-03T03:27:58.206 INFO:tasks.workunit.client.0.smithi117.stdout:write time: 0.0014s fsync time: 0.0248s 2024-06-03T03:27:58.207 INFO:tasks.workunit.client.0.smithi117.stdout:write time: 0.0014s fsync time: 0.0184s 2024-06-03T03:27:58.207 INFO:tasks.workunit.client.0.smithi117.stdout:run done 60 fsyncs total, killing random writer 2024-06-03T03:27:58.207 INFO:tasks.workunit.client.0.smithi117.stderr:+ echo /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2024-06-03T03:27:58.207 INFO:tasks.workunit.client.0.smithi117.stdout:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2024-06-03T03:27:58.208 INFO:tasks.workunit.client.0.smithi117.stderr:+ whereis lsof 2024-06-03T03:27:58.297 INFO:tasks.workunit.client.0.smithi117.stdout:lsof: /usr/bin/lsof /usr/share/man/man1/lsof.1.gz 2024-06-03T03:27:58.298 INFO:tasks.workunit.client.0.smithi117.stderr:+ lsof 2024-06-03T03:27:58.325 INFO:tasks.workunit.client.0.smithi117.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/e04d43a5b19f3b780ca167f4bef91f12f650f7f9963bc85d2f34d8e864bc563f/merged 2024-06-03T03:27:58.325 INFO:tasks.workunit.client.0.smithi117.stderr: Output information may be incomplete. 2024-06-03T03:27:58.325 INFO:tasks.workunit.client.0.smithi117.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/49aecbe5b1a6a9bac4e30a1c37d239059c0a14b1e08f120c0d08b189b29d8540/merged 2024-06-03T03:27:58.325 INFO:tasks.workunit.client.0.smithi117.stderr: Output information may be incomplete. 2024-06-03T03:27:58.327 INFO:tasks.workunit.client.0.smithi117.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/988becf3e01266ecda231e6268d8dcff387213d36bf0f70ac4171d813d1beb3e/merged 2024-06-03T03:27:58.327 INFO:tasks.workunit.client.0.smithi117.stderr: Output information may be incomplete. 2024-06-03T03:27:58.334 INFO:tasks.workunit.client.0.smithi117.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/b1b4ab210a1eb2a153ba97d3b9c52198533f348cced3914c75c1c3684edc5035/merged 2024-06-03T03:27:58.334 INFO:tasks.workunit.client.0.smithi117.stderr: Output information may be incomplete. 2024-06-03T03:27:58.334 INFO:tasks.workunit.client.0.smithi117.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/22da1ac732882fda7547c910fac99e88a57d1f105276353914a52c72167ff0d1/merged 2024-06-03T03:27:58.334 INFO:tasks.workunit.client.0.smithi117.stderr: Output information may be incomplete. 2024-06-03T03:27:58.334 INFO:tasks.workunit.client.0.smithi117.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/f267d15561608e24f453521b3855dba25a998885a8587e5c23a9b63ae482f1fb/merged 2024-06-03T03:27:58.334 INFO:tasks.workunit.client.0.smithi117.stderr: Output information may be incomplete. 2024-06-03T03:27:58.334 INFO:tasks.workunit.client.0.smithi117.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/06e0c3911621dba29f7c4059876e6ef1f2e14a1cf1e8d7b7ff4435e0d4110957/merged 2024-06-03T03:27:58.334 INFO:tasks.workunit.client.0.smithi117.stderr: Output information may be incomplete. 2024-06-03T03:27:58.334 INFO:tasks.workunit.client.0.smithi117.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/631353c7704a89cf52580563c2a1ed61ea6b7f8a6f75a8ffbb041209507baac8/merged 2024-06-03T03:27:58.334 INFO:tasks.workunit.client.0.smithi117.stderr: Output information may be incomplete. 2024-06-03T03:27:58.334 INFO:tasks.workunit.client.0.smithi117.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/919ad2c37db6bf9bb7a3ad8e4d8fb93bd047080e2b5b61f53d3cdc16db3dc898/merged 2024-06-03T03:27:58.335 INFO:tasks.workunit.client.0.smithi117.stderr: Output information may be incomplete. 2024-06-03T03:27:58.335 INFO:tasks.workunit.client.0.smithi117.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/2cca29e878b378cdb9d06281a8e1d9cd8ad948ec2b6d0b955f99495982a95579/merged 2024-06-03T03:27:58.335 INFO:tasks.workunit.client.0.smithi117.stderr: Output information may be incomplete. 2024-06-03T03:27:58.335 INFO:tasks.workunit.client.0.smithi117.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/6984b3b5de97e5eb905208cb3792ef06f986e5f469d4ffc140520df7cfab9806/merged 2024-06-03T03:27:58.335 INFO:tasks.workunit.client.0.smithi117.stderr: Output information may be incomplete. 2024-06-03T03:27:58.487 INFO:tasks.workunit.client.0.smithi117.stdout:COMMAND PID TID TASKCMD USER FD TYPE DEVICE SIZE/OFF NODE NAME 2024-06-03T03:27:58.487 INFO:tasks.workunit.client.0.smithi117.stdout:systemd 1 root cwd unknown /proc/1/cwd (readlink: Permission denied) 2024-06-03T03:27:58.487 INFO:tasks.workunit.client.0.smithi117.stdout:systemd 1 root rtd unknown /proc/1/root (readlink: Permission denied) 2024-06-03T03:27:58.488 INFO:tasks.workunit.client.0.smithi117.stdout:systemd 1 root txt unknown /proc/1/exe (readlink: Permission denied) 2024-06-03T03:27:58.488 INFO:tasks.workunit.client.0.smithi117.stdout:systemd 1 root NOFD /proc/1/fd (opendir: Permission denied) 2024-06-03T03:27:58.488 INFO:tasks.workunit.client.0.smithi117.stdout:kthreadd 2 root cwd unknown /proc/2/cwd (readlink: Permission denied) 2024-06-03T03:27:58.488 INFO:tasks.workunit.client.0.smithi117.stdout:kthreadd 2 root rtd unknown /proc/2/root (readlink: Permission denied) 2024-06-03T03:27:58.488 INFO:tasks.workunit.client.0.smithi117.stdout:kthreadd 2 root txt unknown /proc/2/exe (readlink: Permission denied) 2024-06-03T03:27:58.488 INFO:tasks.workunit.client.0.smithi117.stdout:kthreadd 2 root NOFD /proc/2/fd (opendir: Permission denied) 2024-06-03T03:27:58.488 INFO:tasks.workunit.client.0.smithi117.stdout:pool_work 3 root cwd unknown /proc/3/cwd (readlink: Permission denied) 2024-06-03T03:27:58.488 INFO:tasks.workunit.client.0.smithi117.stdout:pool_work 3 root rtd unknown /proc/3/root (readlink: Permission denied) 2024-06-03T03:27:58.488 INFO:tasks.workunit.client.0.smithi117.stdout:pool_work 3 root txt unknown /proc/3/exe (readlink: Permission denied) 2024-06-03T03:27:58.488 INFO:tasks.workunit.client.0.smithi117.stdout:pool_work 3 root NOFD /proc/3/fd (opendir: Permission denied) 2024-06-03T03:27:58.489 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 4 root cwd unknown /proc/4/cwd (readlink: Permission denied) 2024-06-03T03:27:58.489 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 4 root rtd unknown /proc/4/root (readlink: Permission denied) 2024-06-03T03:27:58.489 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 4 root txt unknown /proc/4/exe (readlink: Permission denied) 2024-06-03T03:27:58.489 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 4 root NOFD /proc/4/fd (opendir: Permission denied) 2024-06-03T03:27:58.489 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 5 root cwd unknown /proc/5/cwd (readlink: Permission denied) 2024-06-03T03:27:58.489 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 5 root rtd unknown /proc/5/root (readlink: Permission denied) 2024-06-03T03:27:58.489 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 5 root txt unknown /proc/5/exe (readlink: Permission denied) 2024-06-03T03:27:58.489 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 5 root NOFD /proc/5/fd (opendir: Permission denied) 2024-06-03T03:27:58.489 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 6 root cwd unknown /proc/6/cwd (readlink: Permission denied) 2024-06-03T03:27:58.489 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 6 root rtd unknown /proc/6/root (readlink: Permission denied) 2024-06-03T03:27:58.489 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 6 root txt unknown /proc/6/exe (readlink: Permission denied) 2024-06-03T03:27:58.489 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 6 root NOFD /proc/6/fd (opendir: Permission denied) 2024-06-03T03:27:58.489 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/0 9 root cwd unknown /proc/9/cwd (readlink: Permission denied) 2024-06-03T03:27:58.490 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/0 9 root rtd unknown /proc/9/root (readlink: Permission denied) 2024-06-03T03:27:58.490 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/0 9 root txt unknown /proc/9/exe (readlink: Permission denied) 2024-06-03T03:27:58.490 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/0 9 root NOFD /proc/9/fd (opendir: Permission denied) 2024-06-03T03:27:58.490 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/u 10 root cwd unknown /proc/10/cwd (readlink: Permission denied) 2024-06-03T03:27:58.490 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/u 10 root rtd unknown /proc/10/root (readlink: Permission denied) 2024-06-03T03:27:58.490 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/u 10 root txt unknown /proc/10/exe (readlink: Permission denied) 2024-06-03T03:27:58.490 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/u 10 root NOFD /proc/10/fd (opendir: Permission denied) 2024-06-03T03:27:58.490 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 11 root cwd unknown /proc/11/cwd (readlink: Permission denied) 2024-06-03T03:27:58.490 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 11 root rtd unknown /proc/11/root (readlink: Permission denied) 2024-06-03T03:27:58.490 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 11 root txt unknown /proc/11/exe (readlink: Permission denied) 2024-06-03T03:27:58.490 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 11 root NOFD /proc/11/fd (opendir: Permission denied) 2024-06-03T03:27:58.490 INFO:tasks.workunit.client.0.smithi117.stdout:rcu_tasks 12 root cwd unknown /proc/12/cwd (readlink: Permission denied) 2024-06-03T03:27:58.490 INFO:tasks.workunit.client.0.smithi117.stdout:rcu_tasks 12 root rtd unknown /proc/12/root (readlink: Permission denied) 2024-06-03T03:27:58.490 INFO:tasks.workunit.client.0.smithi117.stdout:rcu_tasks 12 root txt unknown /proc/12/exe (readlink: Permission denied) 2024-06-03T03:27:58.491 INFO:tasks.workunit.client.0.smithi117.stdout:rcu_tasks 12 root NOFD /proc/12/fd (opendir: Permission denied) 2024-06-03T03:27:58.491 INFO:tasks.workunit.client.0.smithi117.stdout:rcu_tasks 13 root cwd unknown /proc/13/cwd (readlink: Permission denied) 2024-06-03T03:27:58.491 INFO:tasks.workunit.client.0.smithi117.stdout:rcu_tasks 13 root rtd unknown /proc/13/root (readlink: Permission denied) 2024-06-03T03:27:58.491 INFO:tasks.workunit.client.0.smithi117.stdout:rcu_tasks 13 root txt unknown /proc/13/exe (readlink: Permission denied) 2024-06-03T03:27:58.491 INFO:tasks.workunit.client.0.smithi117.stdout:rcu_tasks 13 root NOFD /proc/13/fd (opendir: Permission denied) 2024-06-03T03:27:58.491 INFO:tasks.workunit.client.0.smithi117.stdout:rcu_tasks 14 root cwd unknown /proc/14/cwd (readlink: Permission denied) 2024-06-03T03:27:58.491 INFO:tasks.workunit.client.0.smithi117.stdout:rcu_tasks 14 root rtd unknown /proc/14/root (readlink: Permission denied) 2024-06-03T03:27:58.491 INFO:tasks.workunit.client.0.smithi117.stdout:rcu_tasks 14 root txt unknown /proc/14/exe (readlink: Permission denied) 2024-06-03T03:27:58.491 INFO:tasks.workunit.client.0.smithi117.stdout:rcu_tasks 14 root NOFD /proc/14/fd (opendir: Permission denied) 2024-06-03T03:27:58.491 INFO:tasks.workunit.client.0.smithi117.stdout:ksoftirqd 15 root cwd unknown /proc/15/cwd (readlink: Permission denied) 2024-06-03T03:27:58.491 INFO:tasks.workunit.client.0.smithi117.stdout:ksoftirqd 15 root rtd unknown /proc/15/root (readlink: Permission denied) 2024-06-03T03:27:58.491 INFO:tasks.workunit.client.0.smithi117.stdout:ksoftirqd 15 root txt unknown /proc/15/exe (readlink: Permission denied) 2024-06-03T03:27:58.491 INFO:tasks.workunit.client.0.smithi117.stdout:ksoftirqd 15 root NOFD /proc/15/fd (opendir: Permission denied) 2024-06-03T03:27:58.491 INFO:tasks.workunit.client.0.smithi117.stdout:rcu_preem 16 root cwd unknown /proc/16/cwd (readlink: Permission denied) 2024-06-03T03:27:58.492 INFO:tasks.workunit.client.0.smithi117.stdout:rcu_preem 16 root rtd unknown /proc/16/root (readlink: Permission denied) 2024-06-03T03:27:58.492 INFO:tasks.workunit.client.0.smithi117.stdout:rcu_preem 16 root txt unknown /proc/16/exe (readlink: Permission denied) 2024-06-03T03:27:58.492 INFO:tasks.workunit.client.0.smithi117.stdout:rcu_preem 16 root NOFD /proc/16/fd (opendir: Permission denied) 2024-06-03T03:27:58.492 INFO:tasks.workunit.client.0.smithi117.stdout:rcu_exp_p 17 root cwd unknown /proc/17/cwd (readlink: Permission denied) 2024-06-03T03:27:58.492 INFO:tasks.workunit.client.0.smithi117.stdout:rcu_exp_p 17 root rtd unknown /proc/17/root (readlink: Permission denied) 2024-06-03T03:27:58.492 INFO:tasks.workunit.client.0.smithi117.stdout:rcu_exp_p 17 root txt unknown /proc/17/exe (readlink: Permission denied) 2024-06-03T03:27:58.492 INFO:tasks.workunit.client.0.smithi117.stdout:rcu_exp_p 17 root NOFD /proc/17/fd (opendir: Permission denied) 2024-06-03T03:27:58.492 INFO:tasks.workunit.client.0.smithi117.stdout:rcu_exp_g 18 root cwd unknown /proc/18/cwd (readlink: Permission denied) 2024-06-03T03:27:58.492 INFO:tasks.workunit.client.0.smithi117.stdout:rcu_exp_g 18 root rtd unknown /proc/18/root (readlink: Permission denied) 2024-06-03T03:27:58.492 INFO:tasks.workunit.client.0.smithi117.stdout:rcu_exp_g 18 root txt unknown /proc/18/exe (readlink: Permission denied) 2024-06-03T03:27:58.492 INFO:tasks.workunit.client.0.smithi117.stdout:rcu_exp_g 18 root NOFD /proc/18/fd (opendir: Permission denied) 2024-06-03T03:27:58.492 INFO:tasks.workunit.client.0.smithi117.stdout:migration 19 root cwd unknown /proc/19/cwd (readlink: Permission denied) 2024-06-03T03:27:58.492 INFO:tasks.workunit.client.0.smithi117.stdout:migration 19 root rtd unknown /proc/19/root (readlink: Permission denied) 2024-06-03T03:27:58.493 INFO:tasks.workunit.client.0.smithi117.stdout:migration 19 root txt unknown /proc/19/exe (readlink: Permission denied) 2024-06-03T03:27:58.493 INFO:tasks.workunit.client.0.smithi117.stdout:migration 19 root NOFD /proc/19/fd (opendir: Permission denied) 2024-06-03T03:27:58.493 INFO:tasks.workunit.client.0.smithi117.stdout:cpuhp/0 20 root cwd unknown /proc/20/cwd (readlink: Permission denied) 2024-06-03T03:27:58.493 INFO:tasks.workunit.client.0.smithi117.stdout:cpuhp/0 20 root rtd unknown /proc/20/root (readlink: Permission denied) 2024-06-03T03:27:58.493 INFO:tasks.workunit.client.0.smithi117.stdout:cpuhp/0 20 root txt unknown /proc/20/exe (readlink: Permission denied) 2024-06-03T03:27:58.493 INFO:tasks.workunit.client.0.smithi117.stdout:cpuhp/0 20 root NOFD /proc/20/fd (opendir: Permission denied) 2024-06-03T03:27:58.493 INFO:tasks.workunit.client.0.smithi117.stdout:cpuhp/1 21 root cwd unknown /proc/21/cwd (readlink: Permission denied) 2024-06-03T03:27:58.493 INFO:tasks.workunit.client.0.smithi117.stdout:cpuhp/1 21 root rtd unknown /proc/21/root (readlink: Permission denied) 2024-06-03T03:27:58.493 INFO:tasks.workunit.client.0.smithi117.stdout:cpuhp/1 21 root txt unknown /proc/21/exe (readlink: Permission denied) 2024-06-03T03:27:58.493 INFO:tasks.workunit.client.0.smithi117.stdout:cpuhp/1 21 root NOFD /proc/21/fd (opendir: Permission denied) 2024-06-03T03:27:58.493 INFO:tasks.workunit.client.0.smithi117.stdout:migration 22 root cwd unknown /proc/22/cwd (readlink: Permission denied) 2024-06-03T03:27:58.493 INFO:tasks.workunit.client.0.smithi117.stdout:migration 22 root rtd unknown /proc/22/root (readlink: Permission denied) 2024-06-03T03:27:58.493 INFO:tasks.workunit.client.0.smithi117.stdout:migration 22 root txt unknown /proc/22/exe (readlink: Permission denied) 2024-06-03T03:27:58.493 INFO:tasks.workunit.client.0.smithi117.stdout:migration 22 root NOFD /proc/22/fd (opendir: Permission denied) 2024-06-03T03:27:58.494 INFO:tasks.workunit.client.0.smithi117.stdout:ksoftirqd 23 root cwd unknown /proc/23/cwd (readlink: Permission denied) 2024-06-03T03:27:58.494 INFO:tasks.workunit.client.0.smithi117.stdout:ksoftirqd 23 root rtd unknown /proc/23/root (readlink: Permission denied) 2024-06-03T03:27:58.494 INFO:tasks.workunit.client.0.smithi117.stdout:ksoftirqd 23 root txt unknown /proc/23/exe (readlink: Permission denied) 2024-06-03T03:27:58.494 INFO:tasks.workunit.client.0.smithi117.stdout:ksoftirqd 23 root NOFD /proc/23/fd (opendir: Permission denied) 2024-06-03T03:27:58.494 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/1 25 root cwd unknown /proc/25/cwd (readlink: Permission denied) 2024-06-03T03:27:58.494 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/1 25 root rtd unknown /proc/25/root (readlink: Permission denied) 2024-06-03T03:27:58.494 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/1 25 root txt unknown /proc/25/exe (readlink: Permission denied) 2024-06-03T03:27:58.494 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/1 25 root NOFD /proc/25/fd (opendir: Permission denied) 2024-06-03T03:27:58.494 INFO:tasks.workunit.client.0.smithi117.stdout:cpuhp/2 26 root cwd unknown /proc/26/cwd (readlink: Permission denied) 2024-06-03T03:27:58.494 INFO:tasks.workunit.client.0.smithi117.stdout:cpuhp/2 26 root rtd unknown /proc/26/root (readlink: Permission denied) 2024-06-03T03:27:58.494 INFO:tasks.workunit.client.0.smithi117.stdout:cpuhp/2 26 root txt unknown /proc/26/exe (readlink: Permission denied) 2024-06-03T03:27:58.494 INFO:tasks.workunit.client.0.smithi117.stdout:cpuhp/2 26 root NOFD /proc/26/fd (opendir: Permission denied) 2024-06-03T03:27:58.495 INFO:tasks.workunit.client.0.smithi117.stdout:migration 27 root cwd unknown /proc/27/cwd (readlink: Permission denied) 2024-06-03T03:27:58.495 INFO:tasks.workunit.client.0.smithi117.stdout:migration 27 root rtd unknown /proc/27/root (readlink: Permission denied) 2024-06-03T03:27:58.495 INFO:tasks.workunit.client.0.smithi117.stdout:migration 27 root txt unknown /proc/27/exe (readlink: Permission denied) 2024-06-03T03:27:58.495 INFO:tasks.workunit.client.0.smithi117.stdout:migration 27 root NOFD /proc/27/fd (opendir: Permission denied) 2024-06-03T03:27:58.495 INFO:tasks.workunit.client.0.smithi117.stdout:ksoftirqd 28 root cwd unknown /proc/28/cwd (readlink: Permission denied) 2024-06-03T03:27:58.495 INFO:tasks.workunit.client.0.smithi117.stdout:ksoftirqd 28 root rtd unknown /proc/28/root (readlink: Permission denied) 2024-06-03T03:27:58.495 INFO:tasks.workunit.client.0.smithi117.stdout:ksoftirqd 28 root txt unknown /proc/28/exe (readlink: Permission denied) 2024-06-03T03:27:58.495 INFO:tasks.workunit.client.0.smithi117.stdout:ksoftirqd 28 root NOFD /proc/28/fd (opendir: Permission denied) 2024-06-03T03:27:58.495 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/2 30 root cwd unknown /proc/30/cwd (readlink: Permission denied) 2024-06-03T03:27:58.495 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/2 30 root rtd unknown /proc/30/root (readlink: Permission denied) 2024-06-03T03:27:58.495 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/2 30 root txt unknown /proc/30/exe (readlink: Permission denied) 2024-06-03T03:27:58.495 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/2 30 root NOFD /proc/30/fd (opendir: Permission denied) 2024-06-03T03:27:58.495 INFO:tasks.workunit.client.0.smithi117.stdout:cpuhp/3 31 root cwd unknown /proc/31/cwd (readlink: Permission denied) 2024-06-03T03:27:58.495 INFO:tasks.workunit.client.0.smithi117.stdout:cpuhp/3 31 root rtd unknown /proc/31/root (readlink: Permission denied) 2024-06-03T03:27:58.496 INFO:tasks.workunit.client.0.smithi117.stdout:cpuhp/3 31 root txt unknown /proc/31/exe (readlink: Permission denied) 2024-06-03T03:27:58.496 INFO:tasks.workunit.client.0.smithi117.stdout:cpuhp/3 31 root NOFD /proc/31/fd (opendir: Permission denied) 2024-06-03T03:27:58.496 INFO:tasks.workunit.client.0.smithi117.stdout:migration 32 root cwd unknown /proc/32/cwd (readlink: Permission denied) 2024-06-03T03:27:58.496 INFO:tasks.workunit.client.0.smithi117.stdout:migration 32 root rtd unknown /proc/32/root (readlink: Permission denied) 2024-06-03T03:27:58.496 INFO:tasks.workunit.client.0.smithi117.stdout:migration 32 root txt unknown /proc/32/exe (readlink: Permission denied) 2024-06-03T03:27:58.496 INFO:tasks.workunit.client.0.smithi117.stdout:migration 32 root NOFD /proc/32/fd (opendir: Permission denied) 2024-06-03T03:27:58.496 INFO:tasks.workunit.client.0.smithi117.stdout:ksoftirqd 33 root cwd unknown /proc/33/cwd (readlink: Permission denied) 2024-06-03T03:27:58.496 INFO:tasks.workunit.client.0.smithi117.stdout:ksoftirqd 33 root rtd unknown /proc/33/root (readlink: Permission denied) 2024-06-03T03:27:58.496 INFO:tasks.workunit.client.0.smithi117.stdout:ksoftirqd 33 root txt unknown /proc/33/exe (readlink: Permission denied) 2024-06-03T03:27:58.496 INFO:tasks.workunit.client.0.smithi117.stdout:ksoftirqd 33 root NOFD /proc/33/fd (opendir: Permission denied) 2024-06-03T03:27:58.496 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/3 35 root cwd unknown /proc/35/cwd (readlink: Permission denied) 2024-06-03T03:27:58.496 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/3 35 root rtd unknown /proc/35/root (readlink: Permission denied) 2024-06-03T03:27:58.496 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/3 35 root txt unknown /proc/35/exe (readlink: Permission denied) 2024-06-03T03:27:58.497 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/3 35 root NOFD /proc/35/fd (opendir: Permission denied) 2024-06-03T03:27:58.497 INFO:tasks.workunit.client.0.smithi117.stdout:cpuhp/4 36 root cwd unknown /proc/36/cwd (readlink: Permission denied) 2024-06-03T03:27:58.497 INFO:tasks.workunit.client.0.smithi117.stdout:cpuhp/4 36 root rtd unknown /proc/36/root (readlink: Permission denied) 2024-06-03T03:27:58.497 INFO:tasks.workunit.client.0.smithi117.stdout:cpuhp/4 36 root txt unknown /proc/36/exe (readlink: Permission denied) 2024-06-03T03:27:58.497 INFO:tasks.workunit.client.0.smithi117.stdout:cpuhp/4 36 root NOFD /proc/36/fd (opendir: Permission denied) 2024-06-03T03:27:58.497 INFO:tasks.workunit.client.0.smithi117.stdout:migration 37 root cwd unknown /proc/37/cwd (readlink: Permission denied) 2024-06-03T03:27:58.497 INFO:tasks.workunit.client.0.smithi117.stdout:migration 37 root rtd unknown /proc/37/root (readlink: Permission denied) 2024-06-03T03:27:58.497 INFO:tasks.workunit.client.0.smithi117.stdout:migration 37 root txt unknown /proc/37/exe (readlink: Permission denied) 2024-06-03T03:27:58.497 INFO:tasks.workunit.client.0.smithi117.stdout:migration 37 root NOFD /proc/37/fd (opendir: Permission denied) 2024-06-03T03:27:58.497 INFO:tasks.workunit.client.0.smithi117.stdout:ksoftirqd 38 root cwd unknown /proc/38/cwd (readlink: Permission denied) 2024-06-03T03:27:58.497 INFO:tasks.workunit.client.0.smithi117.stdout:ksoftirqd 38 root rtd unknown /proc/38/root (readlink: Permission denied) 2024-06-03T03:27:58.497 INFO:tasks.workunit.client.0.smithi117.stdout:ksoftirqd 38 root txt unknown /proc/38/exe (readlink: Permission denied) 2024-06-03T03:27:58.498 INFO:tasks.workunit.client.0.smithi117.stdout:ksoftirqd 38 root NOFD /proc/38/fd (opendir: Permission denied) 2024-06-03T03:27:58.498 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/4 40 root cwd unknown /proc/40/cwd (readlink: Permission denied) 2024-06-03T03:27:58.498 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/4 40 root rtd unknown /proc/40/root (readlink: Permission denied) 2024-06-03T03:27:58.498 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/4 40 root txt unknown /proc/40/exe (readlink: Permission denied) 2024-06-03T03:27:58.498 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/4 40 root NOFD /proc/40/fd (opendir: Permission denied) 2024-06-03T03:27:58.498 INFO:tasks.workunit.client.0.smithi117.stdout:cpuhp/5 41 root cwd unknown /proc/41/cwd (readlink: Permission denied) 2024-06-03T03:27:58.498 INFO:tasks.workunit.client.0.smithi117.stdout:cpuhp/5 41 root rtd unknown /proc/41/root (readlink: Permission denied) 2024-06-03T03:27:58.498 INFO:tasks.workunit.client.0.smithi117.stdout:cpuhp/5 41 root txt unknown /proc/41/exe (readlink: Permission denied) 2024-06-03T03:27:58.498 INFO:tasks.workunit.client.0.smithi117.stdout:cpuhp/5 41 root NOFD /proc/41/fd (opendir: Permission denied) 2024-06-03T03:27:58.498 INFO:tasks.workunit.client.0.smithi117.stdout:migration 42 root cwd unknown /proc/42/cwd (readlink: Permission denied) 2024-06-03T03:27:58.498 INFO:tasks.workunit.client.0.smithi117.stdout:migration 42 root rtd unknown /proc/42/root (readlink: Permission denied) 2024-06-03T03:27:58.498 INFO:tasks.workunit.client.0.smithi117.stdout:migration 42 root txt unknown /proc/42/exe (readlink: Permission denied) 2024-06-03T03:27:58.499 INFO:tasks.workunit.client.0.smithi117.stdout:migration 42 root NOFD /proc/42/fd (opendir: Permission denied) 2024-06-03T03:27:58.499 INFO:tasks.workunit.client.0.smithi117.stdout:ksoftirqd 43 root cwd unknown /proc/43/cwd (readlink: Permission denied) 2024-06-03T03:27:58.499 INFO:tasks.workunit.client.0.smithi117.stdout:ksoftirqd 43 root rtd unknown /proc/43/root (readlink: Permission denied) 2024-06-03T03:27:58.499 INFO:tasks.workunit.client.0.smithi117.stdout:ksoftirqd 43 root txt unknown /proc/43/exe (readlink: Permission denied) 2024-06-03T03:27:58.499 INFO:tasks.workunit.client.0.smithi117.stdout:ksoftirqd 43 root NOFD /proc/43/fd (opendir: Permission denied) 2024-06-03T03:27:58.499 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/5 45 root cwd unknown /proc/45/cwd (readlink: Permission denied) 2024-06-03T03:27:58.499 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/5 45 root rtd unknown /proc/45/root (readlink: Permission denied) 2024-06-03T03:27:58.499 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/5 45 root txt unknown /proc/45/exe (readlink: Permission denied) 2024-06-03T03:27:58.499 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/5 45 root NOFD /proc/45/fd (opendir: Permission denied) 2024-06-03T03:27:58.499 INFO:tasks.workunit.client.0.smithi117.stdout:cpuhp/6 46 root cwd unknown /proc/46/cwd (readlink: Permission denied) 2024-06-03T03:27:58.499 INFO:tasks.workunit.client.0.smithi117.stdout:cpuhp/6 46 root rtd unknown /proc/46/root (readlink: Permission denied) 2024-06-03T03:27:58.499 INFO:tasks.workunit.client.0.smithi117.stdout:cpuhp/6 46 root txt unknown /proc/46/exe (readlink: Permission denied) 2024-06-03T03:27:58.499 INFO:tasks.workunit.client.0.smithi117.stdout:cpuhp/6 46 root NOFD /proc/46/fd (opendir: Permission denied) 2024-06-03T03:27:58.499 INFO:tasks.workunit.client.0.smithi117.stdout:migration 47 root cwd unknown /proc/47/cwd (readlink: Permission denied) 2024-06-03T03:27:58.500 INFO:tasks.workunit.client.0.smithi117.stdout:migration 47 root rtd unknown /proc/47/root (readlink: Permission denied) 2024-06-03T03:27:58.500 INFO:tasks.workunit.client.0.smithi117.stdout:migration 47 root txt unknown /proc/47/exe (readlink: Permission denied) 2024-06-03T03:27:58.500 INFO:tasks.workunit.client.0.smithi117.stdout:migration 47 root NOFD /proc/47/fd (opendir: Permission denied) 2024-06-03T03:27:58.500 INFO:tasks.workunit.client.0.smithi117.stdout:ksoftirqd 48 root cwd unknown /proc/48/cwd (readlink: Permission denied) 2024-06-03T03:27:58.500 INFO:tasks.workunit.client.0.smithi117.stdout:ksoftirqd 48 root rtd unknown /proc/48/root (readlink: Permission denied) 2024-06-03T03:27:58.500 INFO:tasks.workunit.client.0.smithi117.stdout:ksoftirqd 48 root txt unknown /proc/48/exe (readlink: Permission denied) 2024-06-03T03:27:58.500 INFO:tasks.workunit.client.0.smithi117.stdout:ksoftirqd 48 root NOFD /proc/48/fd (opendir: Permission denied) 2024-06-03T03:27:58.500 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/6 50 root cwd unknown /proc/50/cwd (readlink: Permission denied) 2024-06-03T03:27:58.500 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/6 50 root rtd unknown /proc/50/root (readlink: Permission denied) 2024-06-03T03:27:58.500 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/6 50 root txt unknown /proc/50/exe (readlink: Permission denied) 2024-06-03T03:27:58.500 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/6 50 root NOFD /proc/50/fd (opendir: Permission denied) 2024-06-03T03:27:58.500 INFO:tasks.workunit.client.0.smithi117.stdout:cpuhp/7 51 root cwd unknown /proc/51/cwd (readlink: Permission denied) 2024-06-03T03:27:58.500 INFO:tasks.workunit.client.0.smithi117.stdout:cpuhp/7 51 root rtd unknown /proc/51/root (readlink: Permission denied) 2024-06-03T03:27:58.501 INFO:tasks.workunit.client.0.smithi117.stdout:cpuhp/7 51 root txt unknown /proc/51/exe (readlink: Permission denied) 2024-06-03T03:27:58.501 INFO:tasks.workunit.client.0.smithi117.stdout:cpuhp/7 51 root NOFD /proc/51/fd (opendir: Permission denied) 2024-06-03T03:27:58.501 INFO:tasks.workunit.client.0.smithi117.stdout:migration 52 root cwd unknown /proc/52/cwd (readlink: Permission denied) 2024-06-03T03:27:58.501 INFO:tasks.workunit.client.0.smithi117.stdout:migration 52 root rtd unknown /proc/52/root (readlink: Permission denied) 2024-06-03T03:27:58.501 INFO:tasks.workunit.client.0.smithi117.stdout:migration 52 root txt unknown /proc/52/exe (readlink: Permission denied) 2024-06-03T03:27:58.501 INFO:tasks.workunit.client.0.smithi117.stdout:migration 52 root NOFD /proc/52/fd (opendir: Permission denied) 2024-06-03T03:27:58.501 INFO:tasks.workunit.client.0.smithi117.stdout:ksoftirqd 53 root cwd unknown /proc/53/cwd (readlink: Permission denied) 2024-06-03T03:27:58.501 INFO:tasks.workunit.client.0.smithi117.stdout:ksoftirqd 53 root rtd unknown /proc/53/root (readlink: Permission denied) 2024-06-03T03:27:58.501 INFO:tasks.workunit.client.0.smithi117.stdout:ksoftirqd 53 root txt unknown /proc/53/exe (readlink: Permission denied) 2024-06-03T03:27:58.501 INFO:tasks.workunit.client.0.smithi117.stdout:ksoftirqd 53 root NOFD /proc/53/fd (opendir: Permission denied) 2024-06-03T03:27:58.501 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/7 54 root cwd unknown /proc/54/cwd (readlink: Permission denied) 2024-06-03T03:27:58.501 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/7 54 root rtd unknown /proc/54/root (readlink: Permission denied) 2024-06-03T03:27:58.501 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/7 54 root txt unknown /proc/54/exe (readlink: Permission denied) 2024-06-03T03:27:58.502 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/7 54 root NOFD /proc/54/fd (opendir: Permission denied) 2024-06-03T03:27:58.502 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/7 55 root cwd unknown /proc/55/cwd (readlink: Permission denied) 2024-06-03T03:27:58.502 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/7 55 root rtd unknown /proc/55/root (readlink: Permission denied) 2024-06-03T03:27:58.502 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/7 55 root txt unknown /proc/55/exe (readlink: Permission denied) 2024-06-03T03:27:58.502 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/7 55 root NOFD /proc/55/fd (opendir: Permission denied) 2024-06-03T03:27:58.502 INFO:tasks.workunit.client.0.smithi117.stdout:kdevtmpfs 56 root cwd unknown /proc/56/cwd (readlink: Permission denied) 2024-06-03T03:27:58.502 INFO:tasks.workunit.client.0.smithi117.stdout:kdevtmpfs 56 root rtd unknown /proc/56/root (readlink: Permission denied) 2024-06-03T03:27:58.502 INFO:tasks.workunit.client.0.smithi117.stdout:kdevtmpfs 56 root txt unknown /proc/56/exe (readlink: Permission denied) 2024-06-03T03:27:58.502 INFO:tasks.workunit.client.0.smithi117.stdout:kdevtmpfs 56 root NOFD /proc/56/fd (opendir: Permission denied) 2024-06-03T03:27:58.502 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 57 root cwd unknown /proc/57/cwd (readlink: Permission denied) 2024-06-03T03:27:58.502 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 57 root rtd unknown /proc/57/root (readlink: Permission denied) 2024-06-03T03:27:58.502 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 57 root txt unknown /proc/57/exe (readlink: Permission denied) 2024-06-03T03:27:58.502 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 57 root NOFD /proc/57/fd (opendir: Permission denied) 2024-06-03T03:27:58.503 INFO:tasks.workunit.client.0.smithi117.stdout:kauditd 58 root cwd unknown /proc/58/cwd (readlink: Permission denied) 2024-06-03T03:27:58.503 INFO:tasks.workunit.client.0.smithi117.stdout:kauditd 58 root rtd unknown /proc/58/root (readlink: Permission denied) 2024-06-03T03:27:58.503 INFO:tasks.workunit.client.0.smithi117.stdout:kauditd 58 root txt unknown /proc/58/exe (readlink: Permission denied) 2024-06-03T03:27:58.503 INFO:tasks.workunit.client.0.smithi117.stdout:kauditd 58 root NOFD /proc/58/fd (opendir: Permission denied) 2024-06-03T03:27:58.503 INFO:tasks.workunit.client.0.smithi117.stdout:khungtask 59 root cwd unknown /proc/59/cwd (readlink: Permission denied) 2024-06-03T03:27:58.503 INFO:tasks.workunit.client.0.smithi117.stdout:khungtask 59 root rtd unknown /proc/59/root (readlink: Permission denied) 2024-06-03T03:27:58.503 INFO:tasks.workunit.client.0.smithi117.stdout:khungtask 59 root txt unknown /proc/59/exe (readlink: Permission denied) 2024-06-03T03:27:58.503 INFO:tasks.workunit.client.0.smithi117.stdout:khungtask 59 root NOFD /proc/59/fd (opendir: Permission denied) 2024-06-03T03:27:58.503 INFO:tasks.workunit.client.0.smithi117.stdout:oom_reape 61 root cwd unknown /proc/61/cwd (readlink: Permission denied) 2024-06-03T03:27:58.503 INFO:tasks.workunit.client.0.smithi117.stdout:oom_reape 61 root rtd unknown /proc/61/root (readlink: Permission denied) 2024-06-03T03:27:58.503 INFO:tasks.workunit.client.0.smithi117.stdout:oom_reape 61 root txt unknown /proc/61/exe (readlink: Permission denied) 2024-06-03T03:27:58.503 INFO:tasks.workunit.client.0.smithi117.stdout:oom_reape 61 root NOFD /proc/61/fd (opendir: Permission denied) 2024-06-03T03:27:58.503 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 63 root cwd unknown /proc/63/cwd (readlink: Permission denied) 2024-06-03T03:27:58.503 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 63 root rtd unknown /proc/63/root (readlink: Permission denied) 2024-06-03T03:27:58.504 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 63 root txt unknown /proc/63/exe (readlink: Permission denied) 2024-06-03T03:27:58.504 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 63 root NOFD /proc/63/fd (opendir: Permission denied) 2024-06-03T03:27:58.504 INFO:tasks.workunit.client.0.smithi117.stdout:kcompactd 64 root cwd unknown /proc/64/cwd (readlink: Permission denied) 2024-06-03T03:27:58.504 INFO:tasks.workunit.client.0.smithi117.stdout:kcompactd 64 root rtd unknown /proc/64/root (readlink: Permission denied) 2024-06-03T03:27:58.504 INFO:tasks.workunit.client.0.smithi117.stdout:kcompactd 64 root txt unknown /proc/64/exe (readlink: Permission denied) 2024-06-03T03:27:58.504 INFO:tasks.workunit.client.0.smithi117.stdout:kcompactd 64 root NOFD /proc/64/fd (opendir: Permission denied) 2024-06-03T03:27:58.504 INFO:tasks.workunit.client.0.smithi117.stdout:ksmd 65 root cwd unknown /proc/65/cwd (readlink: Permission denied) 2024-06-03T03:27:58.504 INFO:tasks.workunit.client.0.smithi117.stdout:ksmd 65 root rtd unknown /proc/65/root (readlink: Permission denied) 2024-06-03T03:27:58.504 INFO:tasks.workunit.client.0.smithi117.stdout:ksmd 65 root txt unknown /proc/65/exe (readlink: Permission denied) 2024-06-03T03:27:58.504 INFO:tasks.workunit.client.0.smithi117.stdout:ksmd 65 root NOFD /proc/65/fd (opendir: Permission denied) 2024-06-03T03:27:58.504 INFO:tasks.workunit.client.0.smithi117.stdout:khugepage 66 root cwd unknown /proc/66/cwd (readlink: Permission denied) 2024-06-03T03:27:58.504 INFO:tasks.workunit.client.0.smithi117.stdout:khugepage 66 root rtd unknown /proc/66/root (readlink: Permission denied) 2024-06-03T03:27:58.504 INFO:tasks.workunit.client.0.smithi117.stdout:khugepage 66 root txt unknown /proc/66/exe (readlink: Permission denied) 2024-06-03T03:27:58.505 INFO:tasks.workunit.client.0.smithi117.stdout:khugepage 66 root NOFD /proc/66/fd (opendir: Permission denied) 2024-06-03T03:27:58.505 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 67 root cwd unknown /proc/67/cwd (readlink: Permission denied) 2024-06-03T03:27:58.505 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 67 root rtd unknown /proc/67/root (readlink: Permission denied) 2024-06-03T03:27:58.505 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 67 root txt unknown /proc/67/exe (readlink: Permission denied) 2024-06-03T03:27:58.505 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 67 root NOFD /proc/67/fd (opendir: Permission denied) 2024-06-03T03:27:58.505 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 68 root cwd unknown /proc/68/cwd (readlink: Permission denied) 2024-06-03T03:27:58.505 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 68 root rtd unknown /proc/68/root (readlink: Permission denied) 2024-06-03T03:27:58.505 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 68 root txt unknown /proc/68/exe (readlink: Permission denied) 2024-06-03T03:27:58.505 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 68 root NOFD /proc/68/fd (opendir: Permission denied) 2024-06-03T03:27:58.505 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 69 root cwd unknown /proc/69/cwd (readlink: Permission denied) 2024-06-03T03:27:58.505 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 69 root rtd unknown /proc/69/root (readlink: Permission denied) 2024-06-03T03:27:58.505 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 69 root txt unknown /proc/69/exe (readlink: Permission denied) 2024-06-03T03:27:58.505 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 69 root NOFD /proc/69/fd (opendir: Permission denied) 2024-06-03T03:27:58.506 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 70 root cwd unknown /proc/70/cwd (readlink: Permission denied) 2024-06-03T03:27:58.506 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 70 root rtd unknown /proc/70/root (readlink: Permission denied) 2024-06-03T03:27:58.506 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 70 root txt unknown /proc/70/exe (readlink: Permission denied) 2024-06-03T03:27:58.506 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 70 root NOFD /proc/70/fd (opendir: Permission denied) 2024-06-03T03:27:58.506 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/1 72 root cwd unknown /proc/72/cwd (readlink: Permission denied) 2024-06-03T03:27:58.506 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/1 72 root rtd unknown /proc/72/root (readlink: Permission denied) 2024-06-03T03:27:58.506 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/1 72 root txt unknown /proc/72/exe (readlink: Permission denied) 2024-06-03T03:27:58.506 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/1 72 root NOFD /proc/72/fd (opendir: Permission denied) 2024-06-03T03:27:58.506 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/2 74 root cwd unknown /proc/74/cwd (readlink: Permission denied) 2024-06-03T03:27:58.506 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/2 74 root rtd unknown /proc/74/root (readlink: Permission denied) 2024-06-03T03:27:58.506 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/2 74 root txt unknown /proc/74/exe (readlink: Permission denied) 2024-06-03T03:27:58.506 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/2 74 root NOFD /proc/74/fd (opendir: Permission denied) 2024-06-03T03:27:58.507 INFO:tasks.workunit.client.0.smithi117.stdout:irq/9-acp 78 root cwd unknown /proc/78/cwd (readlink: Permission denied) 2024-06-03T03:27:58.507 INFO:tasks.workunit.client.0.smithi117.stdout:irq/9-acp 78 root rtd unknown /proc/78/root (readlink: Permission denied) 2024-06-03T03:27:58.507 INFO:tasks.workunit.client.0.smithi117.stdout:irq/9-acp 78 root txt unknown /proc/78/exe (readlink: Permission denied) 2024-06-03T03:27:58.507 INFO:tasks.workunit.client.0.smithi117.stdout:irq/9-acp 78 root NOFD /proc/78/fd (opendir: Permission denied) 2024-06-03T03:27:58.507 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 79 root cwd unknown /proc/79/cwd (readlink: Permission denied) 2024-06-03T03:27:58.507 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 79 root rtd unknown /proc/79/root (readlink: Permission denied) 2024-06-03T03:27:58.507 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 79 root txt unknown /proc/79/exe (readlink: Permission denied) 2024-06-03T03:27:58.507 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 79 root NOFD /proc/79/fd (opendir: Permission denied) 2024-06-03T03:27:58.507 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 80 root cwd unknown /proc/80/cwd (readlink: Permission denied) 2024-06-03T03:27:58.507 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 80 root rtd unknown /proc/80/root (readlink: Permission denied) 2024-06-03T03:27:58.507 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 80 root txt unknown /proc/80/exe (readlink: Permission denied) 2024-06-03T03:27:58.507 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 80 root NOFD /proc/80/fd (opendir: Permission denied) 2024-06-03T03:27:58.508 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 81 root cwd unknown /proc/81/cwd (readlink: Permission denied) 2024-06-03T03:27:58.508 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 81 root rtd unknown /proc/81/root (readlink: Permission denied) 2024-06-03T03:27:58.508 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 81 root txt unknown /proc/81/exe (readlink: Permission denied) 2024-06-03T03:27:58.508 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 81 root NOFD /proc/81/fd (opendir: Permission denied) 2024-06-03T03:27:58.508 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 82 root cwd unknown /proc/82/cwd (readlink: Permission denied) 2024-06-03T03:27:58.508 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 82 root rtd unknown /proc/82/root (readlink: Permission denied) 2024-06-03T03:27:58.508 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 82 root txt unknown /proc/82/exe (readlink: Permission denied) 2024-06-03T03:27:58.508 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 82 root NOFD /proc/82/fd (opendir: Permission denied) 2024-06-03T03:27:58.508 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 83 root cwd unknown /proc/83/cwd (readlink: Permission denied) 2024-06-03T03:27:58.508 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 83 root rtd unknown /proc/83/root (readlink: Permission denied) 2024-06-03T03:27:58.508 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 83 root txt unknown /proc/83/exe (readlink: Permission denied) 2024-06-03T03:27:58.508 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 83 root NOFD /proc/83/fd (opendir: Permission denied) 2024-06-03T03:27:58.508 INFO:tasks.workunit.client.0.smithi117.stdout:watchdogd 84 root cwd unknown /proc/84/cwd (readlink: Permission denied) 2024-06-03T03:27:58.509 INFO:tasks.workunit.client.0.smithi117.stdout:watchdogd 84 root rtd unknown /proc/84/root (readlink: Permission denied) 2024-06-03T03:27:58.509 INFO:tasks.workunit.client.0.smithi117.stdout:watchdogd 84 root txt unknown /proc/84/exe (readlink: Permission denied) 2024-06-03T03:27:58.509 INFO:tasks.workunit.client.0.smithi117.stdout:watchdogd 84 root NOFD /proc/84/fd (opendir: Permission denied) 2024-06-03T03:27:58.509 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/0 85 root cwd unknown /proc/85/cwd (readlink: Permission denied) 2024-06-03T03:27:58.509 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/0 85 root rtd unknown /proc/85/root (readlink: Permission denied) 2024-06-03T03:27:58.509 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/0 85 root txt unknown /proc/85/exe (readlink: Permission denied) 2024-06-03T03:27:58.509 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/0 85 root NOFD /proc/85/fd (opendir: Permission denied) 2024-06-03T03:27:58.509 INFO:tasks.workunit.client.0.smithi117.stdout:kswapd0 86 root cwd unknown /proc/86/cwd (readlink: Permission denied) 2024-06-03T03:27:58.509 INFO:tasks.workunit.client.0.smithi117.stdout:kswapd0 86 root rtd unknown /proc/86/root (readlink: Permission denied) 2024-06-03T03:27:58.509 INFO:tasks.workunit.client.0.smithi117.stdout:kswapd0 86 root txt unknown /proc/86/exe (readlink: Permission denied) 2024-06-03T03:27:58.509 INFO:tasks.workunit.client.0.smithi117.stdout:kswapd0 86 root NOFD /proc/86/fd (opendir: Permission denied) 2024-06-03T03:27:58.509 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 88 root cwd unknown /proc/88/cwd (readlink: Permission denied) 2024-06-03T03:27:58.510 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 88 root rtd unknown /proc/88/root (readlink: Permission denied) 2024-06-03T03:27:58.510 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 88 root txt unknown /proc/88/exe (readlink: Permission denied) 2024-06-03T03:27:58.510 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 88 root NOFD /proc/88/fd (opendir: Permission denied) 2024-06-03T03:27:58.510 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 91 root cwd unknown /proc/91/cwd (readlink: Permission denied) 2024-06-03T03:27:58.510 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 91 root rtd unknown /proc/91/root (readlink: Permission denied) 2024-06-03T03:27:58.510 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 91 root txt unknown /proc/91/exe (readlink: Permission denied) 2024-06-03T03:27:58.510 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 91 root NOFD /proc/91/fd (opendir: Permission denied) 2024-06-03T03:27:58.510 INFO:tasks.workunit.client.0.smithi117.stdout:scsi_eh_0 93 root cwd unknown /proc/93/cwd (readlink: Permission denied) 2024-06-03T03:27:58.510 INFO:tasks.workunit.client.0.smithi117.stdout:scsi_eh_0 93 root rtd unknown /proc/93/root (readlink: Permission denied) 2024-06-03T03:27:58.510 INFO:tasks.workunit.client.0.smithi117.stdout:scsi_eh_0 93 root txt unknown /proc/93/exe (readlink: Permission denied) 2024-06-03T03:27:58.510 INFO:tasks.workunit.client.0.smithi117.stdout:scsi_eh_0 93 root NOFD /proc/93/fd (opendir: Permission denied) 2024-06-03T03:27:58.510 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 94 root cwd unknown /proc/94/cwd (readlink: Permission denied) 2024-06-03T03:27:58.511 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 94 root rtd unknown /proc/94/root (readlink: Permission denied) 2024-06-03T03:27:58.511 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 94 root txt unknown /proc/94/exe (readlink: Permission denied) 2024-06-03T03:27:58.511 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 94 root NOFD /proc/94/fd (opendir: Permission denied) 2024-06-03T03:27:58.511 INFO:tasks.workunit.client.0.smithi117.stdout:scsi_eh_1 95 root cwd unknown /proc/95/cwd (readlink: Permission denied) 2024-06-03T03:27:58.511 INFO:tasks.workunit.client.0.smithi117.stdout:scsi_eh_1 95 root rtd unknown /proc/95/root (readlink: Permission denied) 2024-06-03T03:27:58.511 INFO:tasks.workunit.client.0.smithi117.stdout:scsi_eh_1 95 root txt unknown /proc/95/exe (readlink: Permission denied) 2024-06-03T03:27:58.511 INFO:tasks.workunit.client.0.smithi117.stdout:scsi_eh_1 95 root NOFD /proc/95/fd (opendir: Permission denied) 2024-06-03T03:27:58.511 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 96 root cwd unknown /proc/96/cwd (readlink: Permission denied) 2024-06-03T03:27:58.511 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 96 root rtd unknown /proc/96/root (readlink: Permission denied) 2024-06-03T03:27:58.511 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 96 root txt unknown /proc/96/exe (readlink: Permission denied) 2024-06-03T03:27:58.511 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 96 root NOFD /proc/96/fd (opendir: Permission denied) 2024-06-03T03:27:58.511 INFO:tasks.workunit.client.0.smithi117.stdout:scsi_eh_2 97 root cwd unknown /proc/97/cwd (readlink: Permission denied) 2024-06-03T03:27:58.511 INFO:tasks.workunit.client.0.smithi117.stdout:scsi_eh_2 97 root rtd unknown /proc/97/root (readlink: Permission denied) 2024-06-03T03:27:58.512 INFO:tasks.workunit.client.0.smithi117.stdout:scsi_eh_2 97 root txt unknown /proc/97/exe (readlink: Permission denied) 2024-06-03T03:27:58.512 INFO:tasks.workunit.client.0.smithi117.stdout:scsi_eh_2 97 root NOFD /proc/97/fd (opendir: Permission denied) 2024-06-03T03:27:58.512 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 98 root cwd unknown /proc/98/cwd (readlink: Permission denied) 2024-06-03T03:27:58.512 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 98 root rtd unknown /proc/98/root (readlink: Permission denied) 2024-06-03T03:27:58.512 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 98 root txt unknown /proc/98/exe (readlink: Permission denied) 2024-06-03T03:27:58.512 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 98 root NOFD /proc/98/fd (opendir: Permission denied) 2024-06-03T03:27:58.512 INFO:tasks.workunit.client.0.smithi117.stdout:scsi_eh_3 99 root cwd unknown /proc/99/cwd (readlink: Permission denied) 2024-06-03T03:27:58.512 INFO:tasks.workunit.client.0.smithi117.stdout:scsi_eh_3 99 root rtd unknown /proc/99/root (readlink: Permission denied) 2024-06-03T03:27:58.512 INFO:tasks.workunit.client.0.smithi117.stdout:scsi_eh_3 99 root txt unknown /proc/99/exe (readlink: Permission denied) 2024-06-03T03:27:58.512 INFO:tasks.workunit.client.0.smithi117.stdout:scsi_eh_3 99 root NOFD /proc/99/fd (opendir: Permission denied) 2024-06-03T03:27:58.512 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 100 root cwd unknown /proc/100/cwd (readlink: Permission denied) 2024-06-03T03:27:58.512 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 100 root rtd unknown /proc/100/root (readlink: Permission denied) 2024-06-03T03:27:58.512 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 100 root txt unknown /proc/100/exe (readlink: Permission denied) 2024-06-03T03:27:58.512 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 100 root NOFD /proc/100/fd (opendir: Permission denied) 2024-06-03T03:27:58.512 INFO:tasks.workunit.client.0.smithi117.stdout:scsi_eh_4 104 root cwd unknown /proc/104/cwd (readlink: Permission denied) 2024-06-03T03:27:58.513 INFO:tasks.workunit.client.0.smithi117.stdout:scsi_eh_4 104 root rtd unknown /proc/104/root (readlink: Permission denied) 2024-06-03T03:27:58.513 INFO:tasks.workunit.client.0.smithi117.stdout:scsi_eh_4 104 root txt unknown /proc/104/exe (readlink: Permission denied) 2024-06-03T03:27:58.513 INFO:tasks.workunit.client.0.smithi117.stdout:scsi_eh_4 104 root NOFD /proc/104/fd (opendir: Permission denied) 2024-06-03T03:27:58.513 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 105 root cwd unknown /proc/105/cwd (readlink: Permission denied) 2024-06-03T03:27:58.513 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 105 root rtd unknown /proc/105/root (readlink: Permission denied) 2024-06-03T03:27:58.513 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 105 root txt unknown /proc/105/exe (readlink: Permission denied) 2024-06-03T03:27:58.513 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 105 root NOFD /proc/105/fd (opendir: Permission denied) 2024-06-03T03:27:58.513 INFO:tasks.workunit.client.0.smithi117.stdout:scsi_eh_5 106 root cwd unknown /proc/106/cwd (readlink: Permission denied) 2024-06-03T03:27:58.513 INFO:tasks.workunit.client.0.smithi117.stdout:scsi_eh_5 106 root rtd unknown /proc/106/root (readlink: Permission denied) 2024-06-03T03:27:58.513 INFO:tasks.workunit.client.0.smithi117.stdout:scsi_eh_5 106 root txt unknown /proc/106/exe (readlink: Permission denied) 2024-06-03T03:27:58.513 INFO:tasks.workunit.client.0.smithi117.stdout:scsi_eh_5 106 root NOFD /proc/106/fd (opendir: Permission denied) 2024-06-03T03:27:58.513 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 107 root cwd unknown /proc/107/cwd (readlink: Permission denied) 2024-06-03T03:27:58.513 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 107 root rtd unknown /proc/107/root (readlink: Permission denied) 2024-06-03T03:27:58.514 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 107 root txt unknown /proc/107/exe (readlink: Permission denied) 2024-06-03T03:27:58.514 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 107 root NOFD /proc/107/fd (opendir: Permission denied) 2024-06-03T03:27:58.514 INFO:tasks.workunit.client.0.smithi117.stdout:scsi_eh_6 108 root cwd unknown /proc/108/cwd (readlink: Permission denied) 2024-06-03T03:27:58.514 INFO:tasks.workunit.client.0.smithi117.stdout:scsi_eh_6 108 root rtd unknown /proc/108/root (readlink: Permission denied) 2024-06-03T03:27:58.514 INFO:tasks.workunit.client.0.smithi117.stdout:scsi_eh_6 108 root txt unknown /proc/108/exe (readlink: Permission denied) 2024-06-03T03:27:58.514 INFO:tasks.workunit.client.0.smithi117.stdout:scsi_eh_6 108 root NOFD /proc/108/fd (opendir: Permission denied) 2024-06-03T03:27:58.514 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 109 root cwd unknown /proc/109/cwd (readlink: Permission denied) 2024-06-03T03:27:58.514 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 109 root rtd unknown /proc/109/root (readlink: Permission denied) 2024-06-03T03:27:58.514 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 109 root txt unknown /proc/109/exe (readlink: Permission denied) 2024-06-03T03:27:58.514 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 109 root NOFD /proc/109/fd (opendir: Permission denied) 2024-06-03T03:27:58.514 INFO:tasks.workunit.client.0.smithi117.stdout:scsi_eh_7 110 root cwd unknown /proc/110/cwd (readlink: Permission denied) 2024-06-03T03:27:58.514 INFO:tasks.workunit.client.0.smithi117.stdout:scsi_eh_7 110 root rtd unknown /proc/110/root (readlink: Permission denied) 2024-06-03T03:27:58.515 INFO:tasks.workunit.client.0.smithi117.stdout:scsi_eh_7 110 root txt unknown /proc/110/exe (readlink: Permission denied) 2024-06-03T03:27:58.515 INFO:tasks.workunit.client.0.smithi117.stdout:scsi_eh_7 110 root NOFD /proc/110/fd (opendir: Permission denied) 2024-06-03T03:27:58.515 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 111 root cwd unknown /proc/111/cwd (readlink: Permission denied) 2024-06-03T03:27:58.515 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 111 root rtd unknown /proc/111/root (readlink: Permission denied) 2024-06-03T03:27:58.515 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 111 root txt unknown /proc/111/exe (readlink: Permission denied) 2024-06-03T03:27:58.515 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 111 root NOFD /proc/111/fd (opendir: Permission denied) 2024-06-03T03:27:58.515 INFO:tasks.workunit.client.0.smithi117.stdout:scsi_eh_8 112 root cwd unknown /proc/112/cwd (readlink: Permission denied) 2024-06-03T03:27:58.515 INFO:tasks.workunit.client.0.smithi117.stdout:scsi_eh_8 112 root rtd unknown /proc/112/root (readlink: Permission denied) 2024-06-03T03:27:58.515 INFO:tasks.workunit.client.0.smithi117.stdout:scsi_eh_8 112 root txt unknown /proc/112/exe (readlink: Permission denied) 2024-06-03T03:27:58.515 INFO:tasks.workunit.client.0.smithi117.stdout:scsi_eh_8 112 root NOFD /proc/112/fd (opendir: Permission denied) 2024-06-03T03:27:58.515 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 113 root cwd unknown /proc/113/cwd (readlink: Permission denied) 2024-06-03T03:27:58.515 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 113 root rtd unknown /proc/113/root (readlink: Permission denied) 2024-06-03T03:27:58.515 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 113 root txt unknown /proc/113/exe (readlink: Permission denied) 2024-06-03T03:27:58.515 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 113 root NOFD /proc/113/fd (opendir: Permission denied) 2024-06-03T03:27:58.516 INFO:tasks.workunit.client.0.smithi117.stdout:scsi_eh_9 114 root cwd unknown /proc/114/cwd (readlink: Permission denied) 2024-06-03T03:27:58.516 INFO:tasks.workunit.client.0.smithi117.stdout:scsi_eh_9 114 root rtd unknown /proc/114/root (readlink: Permission denied) 2024-06-03T03:27:58.516 INFO:tasks.workunit.client.0.smithi117.stdout:scsi_eh_9 114 root txt unknown /proc/114/exe (readlink: Permission denied) 2024-06-03T03:27:58.516 INFO:tasks.workunit.client.0.smithi117.stdout:scsi_eh_9 114 root NOFD /proc/114/fd (opendir: Permission denied) 2024-06-03T03:27:58.516 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 115 root cwd unknown /proc/115/cwd (readlink: Permission denied) 2024-06-03T03:27:58.516 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 115 root rtd unknown /proc/115/root (readlink: Permission denied) 2024-06-03T03:27:58.516 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 115 root txt unknown /proc/115/exe (readlink: Permission denied) 2024-06-03T03:27:58.516 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 115 root NOFD /proc/115/fd (opendir: Permission denied) 2024-06-03T03:27:58.516 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/u 117 root cwd unknown /proc/117/cwd (readlink: Permission denied) 2024-06-03T03:27:58.516 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/u 117 root rtd unknown /proc/117/root (readlink: Permission denied) 2024-06-03T03:27:58.516 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/u 117 root txt unknown /proc/117/exe (readlink: Permission denied) 2024-06-03T03:27:58.516 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/u 117 root NOFD /proc/117/fd (opendir: Permission denied) 2024-06-03T03:27:58.516 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/u 119 root cwd unknown /proc/119/cwd (readlink: Permission denied) 2024-06-03T03:27:58.516 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/u 119 root rtd unknown /proc/119/root (readlink: Permission denied) 2024-06-03T03:27:58.517 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/u 119 root txt unknown /proc/119/exe (readlink: Permission denied) 2024-06-03T03:27:58.517 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/u 119 root NOFD /proc/119/fd (opendir: Permission denied) 2024-06-03T03:27:58.517 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/u 120 root cwd unknown /proc/120/cwd (readlink: Permission denied) 2024-06-03T03:27:58.517 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/u 120 root rtd unknown /proc/120/root (readlink: Permission denied) 2024-06-03T03:27:58.517 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/u 120 root txt unknown /proc/120/exe (readlink: Permission denied) 2024-06-03T03:27:58.517 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/u 120 root NOFD /proc/120/fd (opendir: Permission denied) 2024-06-03T03:27:58.517 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/4 125 root cwd unknown /proc/125/cwd (readlink: Permission denied) 2024-06-03T03:27:58.517 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/4 125 root rtd unknown /proc/125/root (readlink: Permission denied) 2024-06-03T03:27:58.517 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/4 125 root txt unknown /proc/125/exe (readlink: Permission denied) 2024-06-03T03:27:58.517 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/4 125 root NOFD /proc/125/fd (opendir: Permission denied) 2024-06-03T03:27:58.517 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 128 root cwd unknown /proc/128/cwd (readlink: Permission denied) 2024-06-03T03:27:58.517 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 128 root rtd unknown /proc/128/root (readlink: Permission denied) 2024-06-03T03:27:58.517 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 128 root txt unknown /proc/128/exe (readlink: Permission denied) 2024-06-03T03:27:58.517 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 128 root NOFD /proc/128/fd (opendir: Permission denied) 2024-06-03T03:27:58.518 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 130 root cwd unknown /proc/130/cwd (readlink: Permission denied) 2024-06-03T03:27:58.518 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 130 root rtd unknown /proc/130/root (readlink: Permission denied) 2024-06-03T03:27:58.518 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 130 root txt unknown /proc/130/exe (readlink: Permission denied) 2024-06-03T03:27:58.518 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 130 root NOFD /proc/130/fd (opendir: Permission denied) 2024-06-03T03:27:58.518 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/1 131 root cwd unknown /proc/131/cwd (readlink: Permission denied) 2024-06-03T03:27:58.518 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/1 131 root rtd unknown /proc/131/root (readlink: Permission denied) 2024-06-03T03:27:58.518 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/1 131 root txt unknown /proc/131/exe (readlink: Permission denied) 2024-06-03T03:27:58.518 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/1 131 root NOFD /proc/131/fd (opendir: Permission denied) 2024-06-03T03:27:58.518 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 132 root cwd unknown /proc/132/cwd (readlink: Permission denied) 2024-06-03T03:27:58.518 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 132 root rtd unknown /proc/132/root (readlink: Permission denied) 2024-06-03T03:27:58.518 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 132 root txt unknown /proc/132/exe (readlink: Permission denied) 2024-06-03T03:27:58.518 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 132 root NOFD /proc/132/fd (opendir: Permission denied) 2024-06-03T03:27:58.518 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/u 139 root cwd unknown /proc/139/cwd (readlink: Permission denied) 2024-06-03T03:27:58.519 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/u 139 root rtd unknown /proc/139/root (readlink: Permission denied) 2024-06-03T03:27:58.519 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/u 139 root txt unknown /proc/139/exe (readlink: Permission denied) 2024-06-03T03:27:58.519 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/u 139 root NOFD /proc/139/fd (opendir: Permission denied) 2024-06-03T03:27:58.519 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/2 249 root cwd unknown /proc/249/cwd (readlink: Permission denied) 2024-06-03T03:27:58.519 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/2 249 root rtd unknown /proc/249/root (readlink: Permission denied) 2024-06-03T03:27:58.519 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/2 249 root txt unknown /proc/249/exe (readlink: Permission denied) 2024-06-03T03:27:58.519 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/2 249 root NOFD /proc/249/fd (opendir: Permission denied) 2024-06-03T03:27:58.519 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/3 251 root cwd unknown /proc/251/cwd (readlink: Permission denied) 2024-06-03T03:27:58.519 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/3 251 root rtd unknown /proc/251/root (readlink: Permission denied) 2024-06-03T03:27:58.519 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/3 251 root txt unknown /proc/251/exe (readlink: Permission denied) 2024-06-03T03:27:58.519 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/3 251 root NOFD /proc/251/fd (opendir: Permission denied) 2024-06-03T03:27:58.519 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/5 256 root cwd unknown /proc/256/cwd (readlink: Permission denied) 2024-06-03T03:27:58.519 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/5 256 root rtd unknown /proc/256/root (readlink: Permission denied) 2024-06-03T03:27:58.519 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/5 256 root txt unknown /proc/256/exe (readlink: Permission denied) 2024-06-03T03:27:58.519 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/5 256 root NOFD /proc/256/fd (opendir: Permission denied) 2024-06-03T03:27:58.520 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/7 259 root cwd unknown /proc/259/cwd (readlink: Permission denied) 2024-06-03T03:27:58.520 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/7 259 root rtd unknown /proc/259/root (readlink: Permission denied) 2024-06-03T03:27:58.520 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/7 259 root txt unknown /proc/259/exe (readlink: Permission denied) 2024-06-03T03:27:58.520 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/7 259 root NOFD /proc/259/fd (opendir: Permission denied) 2024-06-03T03:27:58.520 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/6 268 root cwd unknown /proc/268/cwd (readlink: Permission denied) 2024-06-03T03:27:58.520 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/6 268 root rtd unknown /proc/268/root (readlink: Permission denied) 2024-06-03T03:27:58.520 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/6 268 root txt unknown /proc/268/exe (readlink: Permission denied) 2024-06-03T03:27:58.520 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/6 268 root NOFD /proc/268/fd (opendir: Permission denied) 2024-06-03T03:27:58.520 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/5 269 root cwd unknown /proc/269/cwd (readlink: Permission denied) 2024-06-03T03:27:58.520 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/5 269 root rtd unknown /proc/269/root (readlink: Permission denied) 2024-06-03T03:27:58.520 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/5 269 root txt unknown /proc/269/exe (readlink: Permission denied) 2024-06-03T03:27:58.520 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/5 269 root NOFD /proc/269/fd (opendir: Permission denied) 2024-06-03T03:27:58.520 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 411 root cwd unknown /proc/411/cwd (readlink: Permission denied) 2024-06-03T03:27:58.520 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 411 root rtd unknown /proc/411/root (readlink: Permission denied) 2024-06-03T03:27:58.520 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 411 root txt unknown /proc/411/exe (readlink: Permission denied) 2024-06-03T03:27:58.521 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 411 root NOFD /proc/411/fd (opendir: Permission denied) 2024-06-03T03:27:58.521 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 412 root cwd unknown /proc/412/cwd (readlink: Permission denied) 2024-06-03T03:27:58.521 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 412 root rtd unknown /proc/412/root (readlink: Permission denied) 2024-06-03T03:27:58.521 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 412 root txt unknown /proc/412/exe (readlink: Permission denied) 2024-06-03T03:27:58.521 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 412 root NOFD /proc/412/fd (opendir: Permission denied) 2024-06-03T03:27:58.521 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 419 root cwd unknown /proc/419/cwd (readlink: Permission denied) 2024-06-03T03:27:58.521 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 419 root rtd unknown /proc/419/root (readlink: Permission denied) 2024-06-03T03:27:58.521 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 419 root txt unknown /proc/419/exe (readlink: Permission denied) 2024-06-03T03:27:58.521 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 419 root NOFD /proc/419/fd (opendir: Permission denied) 2024-06-03T03:27:58.521 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 432 root cwd unknown /proc/432/cwd (readlink: Permission denied) 2024-06-03T03:27:58.521 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 432 root rtd unknown /proc/432/root (readlink: Permission denied) 2024-06-03T03:27:58.521 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 432 root txt unknown /proc/432/exe (readlink: Permission denied) 2024-06-03T03:27:58.521 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 432 root NOFD /proc/432/fd (opendir: Permission denied) 2024-06-03T03:27:58.522 INFO:tasks.workunit.client.0.smithi117.stdout:jbd2/sda1 446 root cwd unknown /proc/446/cwd (readlink: Permission denied) 2024-06-03T03:27:58.522 INFO:tasks.workunit.client.0.smithi117.stdout:jbd2/sda1 446 root rtd unknown /proc/446/root (readlink: Permission denied) 2024-06-03T03:27:58.522 INFO:tasks.workunit.client.0.smithi117.stdout:jbd2/sda1 446 root txt unknown /proc/446/exe (readlink: Permission denied) 2024-06-03T03:27:58.522 INFO:tasks.workunit.client.0.smithi117.stdout:jbd2/sda1 446 root NOFD /proc/446/fd (opendir: Permission denied) 2024-06-03T03:27:58.522 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 447 root cwd unknown /proc/447/cwd (readlink: Permission denied) 2024-06-03T03:27:58.522 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 447 root rtd unknown /proc/447/root (readlink: Permission denied) 2024-06-03T03:27:58.522 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 447 root txt unknown /proc/447/exe (readlink: Permission denied) 2024-06-03T03:27:58.522 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 447 root NOFD /proc/447/fd (opendir: Permission denied) 2024-06-03T03:27:58.522 INFO:tasks.workunit.client.0.smithi117.stdout:systemd-j 524 root cwd unknown /proc/524/cwd (readlink: Permission denied) 2024-06-03T03:27:58.522 INFO:tasks.workunit.client.0.smithi117.stdout:systemd-j 524 root rtd unknown /proc/524/root (readlink: Permission denied) 2024-06-03T03:27:58.522 INFO:tasks.workunit.client.0.smithi117.stdout:systemd-j 524 root txt unknown /proc/524/exe (readlink: Permission denied) 2024-06-03T03:27:58.522 INFO:tasks.workunit.client.0.smithi117.stdout:systemd-j 524 root NOFD /proc/524/fd (opendir: Permission denied) 2024-06-03T03:27:58.523 INFO:tasks.workunit.client.0.smithi117.stdout:ext4lazyi 531 root cwd unknown /proc/531/cwd (readlink: Permission denied) 2024-06-03T03:27:58.523 INFO:tasks.workunit.client.0.smithi117.stdout:ext4lazyi 531 root rtd unknown /proc/531/root (readlink: Permission denied) 2024-06-03T03:27:58.523 INFO:tasks.workunit.client.0.smithi117.stdout:ext4lazyi 531 root txt unknown /proc/531/exe (readlink: Permission denied) 2024-06-03T03:27:58.523 INFO:tasks.workunit.client.0.smithi117.stdout:ext4lazyi 531 root NOFD /proc/531/fd (opendir: Permission denied) 2024-06-03T03:27:58.523 INFO:tasks.workunit.client.0.smithi117.stdout:systemd-u 547 root cwd unknown /proc/547/cwd (readlink: Permission denied) 2024-06-03T03:27:58.523 INFO:tasks.workunit.client.0.smithi117.stdout:systemd-u 547 root rtd unknown /proc/547/root (readlink: Permission denied) 2024-06-03T03:27:58.523 INFO:tasks.workunit.client.0.smithi117.stdout:systemd-u 547 root txt unknown /proc/547/exe (readlink: Permission denied) 2024-06-03T03:27:58.523 INFO:tasks.workunit.client.0.smithi117.stdout:systemd-u 547 root NOFD /proc/547/fd (opendir: Permission denied) 2024-06-03T03:27:58.523 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 598 root cwd unknown /proc/598/cwd (readlink: Permission denied) 2024-06-03T03:27:58.523 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 598 root rtd unknown /proc/598/root (readlink: Permission denied) 2024-06-03T03:27:58.523 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 598 root txt unknown /proc/598/exe (readlink: Permission denied) 2024-06-03T03:27:58.523 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 598 root NOFD /proc/598/fd (opendir: Permission denied) 2024-06-03T03:27:58.523 INFO:tasks.workunit.client.0.smithi117.stdout:kipmi0 608 root cwd unknown /proc/608/cwd (readlink: Permission denied) 2024-06-03T03:27:58.523 INFO:tasks.workunit.client.0.smithi117.stdout:kipmi0 608 root rtd unknown /proc/608/root (readlink: Permission denied) 2024-06-03T03:27:58.524 INFO:tasks.workunit.client.0.smithi117.stdout:kipmi0 608 root txt unknown /proc/608/exe (readlink: Permission denied) 2024-06-03T03:27:58.524 INFO:tasks.workunit.client.0.smithi117.stdout:kipmi0 608 root NOFD /proc/608/fd (opendir: Permission denied) 2024-06-03T03:27:58.524 INFO:tasks.workunit.client.0.smithi117.stdout:rpcbind 612 rpc cwd unknown /proc/612/cwd (readlink: Permission denied) 2024-06-03T03:27:58.524 INFO:tasks.workunit.client.0.smithi117.stdout:rpcbind 612 rpc rtd unknown /proc/612/root (readlink: Permission denied) 2024-06-03T03:27:58.524 INFO:tasks.workunit.client.0.smithi117.stdout:rpcbind 612 rpc txt unknown /proc/612/exe (readlink: Permission denied) 2024-06-03T03:27:58.524 INFO:tasks.workunit.client.0.smithi117.stdout:rpcbind 612 rpc NOFD /proc/612/fd (opendir: Permission denied) 2024-06-03T03:27:58.524 INFO:tasks.workunit.client.0.smithi117.stdout:auditd 619 root cwd unknown /proc/619/cwd (readlink: Permission denied) 2024-06-03T03:27:58.524 INFO:tasks.workunit.client.0.smithi117.stdout:auditd 619 root rtd unknown /proc/619/root (readlink: Permission denied) 2024-06-03T03:27:58.524 INFO:tasks.workunit.client.0.smithi117.stdout:auditd 619 root txt unknown /proc/619/exe (readlink: Permission denied) 2024-06-03T03:27:58.524 INFO:tasks.workunit.client.0.smithi117.stdout:auditd 619 root NOFD /proc/619/fd (opendir: Permission denied) 2024-06-03T03:27:58.524 INFO:tasks.workunit.client.0.smithi117.stdout:auditd 619 620 auditd root cwd unknown /proc/619/task/620/cwd (readlink: Permission denied) 2024-06-03T03:27:58.524 INFO:tasks.workunit.client.0.smithi117.stdout:auditd 619 620 auditd root rtd unknown /proc/619/task/620/root (readlink: Permission denied) 2024-06-03T03:27:58.524 INFO:tasks.workunit.client.0.smithi117.stdout:auditd 619 620 auditd root txt unknown /proc/619/task/620/exe (readlink: Permission denied) 2024-06-03T03:27:58.525 INFO:tasks.workunit.client.0.smithi117.stdout:auditd 619 620 auditd root NOFD /proc/619/task/620/fd (opendir: Permission denied) 2024-06-03T03:27:58.525 INFO:tasks.workunit.client.0.smithi117.stdout:auditd 619 622 auditd root cwd unknown /proc/619/task/622/cwd (readlink: Permission denied) 2024-06-03T03:27:58.525 INFO:tasks.workunit.client.0.smithi117.stdout:auditd 619 622 auditd root rtd unknown /proc/619/task/622/root (readlink: Permission denied) 2024-06-03T03:27:58.525 INFO:tasks.workunit.client.0.smithi117.stdout:auditd 619 622 auditd root txt unknown /proc/619/task/622/exe (readlink: Permission denied) 2024-06-03T03:27:58.525 INFO:tasks.workunit.client.0.smithi117.stdout:auditd 619 622 auditd root NOFD /proc/619/task/622/fd (opendir: Permission denied) 2024-06-03T03:27:58.525 INFO:tasks.workunit.client.0.smithi117.stdout:sedispatc 621 root cwd unknown /proc/621/cwd (readlink: Permission denied) 2024-06-03T03:27:58.525 INFO:tasks.workunit.client.0.smithi117.stdout:sedispatc 621 root rtd unknown /proc/621/root (readlink: Permission denied) 2024-06-03T03:27:58.525 INFO:tasks.workunit.client.0.smithi117.stdout:sedispatc 621 root txt unknown /proc/621/exe (readlink: Permission denied) 2024-06-03T03:27:58.525 INFO:tasks.workunit.client.0.smithi117.stdout:sedispatc 621 root NOFD /proc/621/fd (opendir: Permission denied) 2024-06-03T03:27:58.525 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 626 root cwd unknown /proc/626/cwd (readlink: Permission denied) 2024-06-03T03:27:58.525 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 626 root rtd unknown /proc/626/root (readlink: Permission denied) 2024-06-03T03:27:58.525 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 626 root txt unknown /proc/626/exe (readlink: Permission denied) 2024-06-03T03:27:58.525 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 626 root NOFD /proc/626/fd (opendir: Permission denied) 2024-06-03T03:27:58.525 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 627 root cwd unknown /proc/627/cwd (readlink: Permission denied) 2024-06-03T03:27:58.526 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 627 root rtd unknown /proc/627/root (readlink: Permission denied) 2024-06-03T03:27:58.526 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 627 root txt unknown /proc/627/exe (readlink: Permission denied) 2024-06-03T03:27:58.526 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 627 root NOFD /proc/627/fd (opendir: Permission denied) 2024-06-03T03:27:58.526 INFO:tasks.workunit.client.0.smithi117.stdout:dbus-brok 662 dbus cwd unknown /proc/662/cwd (readlink: Permission denied) 2024-06-03T03:27:58.526 INFO:tasks.workunit.client.0.smithi117.stdout:dbus-brok 662 dbus rtd unknown /proc/662/root (readlink: Permission denied) 2024-06-03T03:27:58.526 INFO:tasks.workunit.client.0.smithi117.stdout:dbus-brok 662 dbus txt unknown /proc/662/exe (readlink: Permission denied) 2024-06-03T03:27:58.526 INFO:tasks.workunit.client.0.smithi117.stdout:dbus-brok 662 dbus NOFD /proc/662/fd (opendir: Permission denied) 2024-06-03T03:27:58.526 INFO:tasks.workunit.client.0.smithi117.stdout:dbus-brok 663 dbus cwd unknown /proc/663/cwd (readlink: Permission denied) 2024-06-03T03:27:58.526 INFO:tasks.workunit.client.0.smithi117.stdout:dbus-brok 663 dbus rtd unknown /proc/663/root (readlink: Permission denied) 2024-06-03T03:27:58.526 INFO:tasks.workunit.client.0.smithi117.stdout:dbus-brok 663 dbus txt unknown /proc/663/exe (readlink: Permission denied) 2024-06-03T03:27:58.526 INFO:tasks.workunit.client.0.smithi117.stdout:dbus-brok 663 dbus NOFD /proc/663/fd (opendir: Permission denied) 2024-06-03T03:27:58.526 INFO:tasks.workunit.client.0.smithi117.stdout:NetworkMa 664 root cwd unknown /proc/664/cwd (readlink: Permission denied) 2024-06-03T03:27:58.526 INFO:tasks.workunit.client.0.smithi117.stdout:NetworkMa 664 root rtd unknown /proc/664/root (readlink: Permission denied) 2024-06-03T03:27:58.527 INFO:tasks.workunit.client.0.smithi117.stdout:NetworkMa 664 root txt unknown /proc/664/exe (readlink: Permission denied) 2024-06-03T03:27:58.527 INFO:tasks.workunit.client.0.smithi117.stdout:NetworkMa 664 root NOFD /proc/664/fd (opendir: Permission denied) 2024-06-03T03:27:58.527 INFO:tasks.workunit.client.0.smithi117.stdout:NetworkMa 664 679 gmain root cwd unknown /proc/664/task/679/cwd (readlink: Permission denied) 2024-06-03T03:27:58.527 INFO:tasks.workunit.client.0.smithi117.stdout:NetworkMa 664 679 gmain root rtd unknown /proc/664/task/679/root (readlink: Permission denied) 2024-06-03T03:27:58.527 INFO:tasks.workunit.client.0.smithi117.stdout:NetworkMa 664 679 gmain root txt unknown /proc/664/task/679/exe (readlink: Permission denied) 2024-06-03T03:27:58.527 INFO:tasks.workunit.client.0.smithi117.stdout:NetworkMa 664 679 gmain root NOFD /proc/664/task/679/fd (opendir: Permission denied) 2024-06-03T03:27:58.527 INFO:tasks.workunit.client.0.smithi117.stdout:NetworkMa 664 680 gdbus root cwd unknown /proc/664/task/680/cwd (readlink: Permission denied) 2024-06-03T03:27:58.527 INFO:tasks.workunit.client.0.smithi117.stdout:NetworkMa 664 680 gdbus root rtd unknown /proc/664/task/680/root (readlink: Permission denied) 2024-06-03T03:27:58.527 INFO:tasks.workunit.client.0.smithi117.stdout:NetworkMa 664 680 gdbus root txt unknown /proc/664/task/680/exe (readlink: Permission denied) 2024-06-03T03:27:58.527 INFO:tasks.workunit.client.0.smithi117.stdout:NetworkMa 664 680 gdbus root NOFD /proc/664/task/680/fd (opendir: Permission denied) 2024-06-03T03:27:58.527 INFO:tasks.workunit.client.0.smithi117.stdout:irqbalanc 667 root cwd unknown /proc/667/cwd (readlink: Permission denied) 2024-06-03T03:27:58.527 INFO:tasks.workunit.client.0.smithi117.stdout:irqbalanc 667 root rtd unknown /proc/667/root (readlink: Permission denied) 2024-06-03T03:27:58.528 INFO:tasks.workunit.client.0.smithi117.stdout:irqbalanc 667 root txt unknown /proc/667/exe (readlink: Permission denied) 2024-06-03T03:27:58.528 INFO:tasks.workunit.client.0.smithi117.stdout:irqbalanc 667 root NOFD /proc/667/fd (opendir: Permission denied) 2024-06-03T03:27:58.528 INFO:tasks.workunit.client.0.smithi117.stdout:irqbalanc 667 670 gmain root cwd unknown /proc/667/task/670/cwd (readlink: Permission denied) 2024-06-03T03:27:58.528 INFO:tasks.workunit.client.0.smithi117.stdout:irqbalanc 667 670 gmain root rtd unknown /proc/667/task/670/root (readlink: Permission denied) 2024-06-03T03:27:58.528 INFO:tasks.workunit.client.0.smithi117.stdout:irqbalanc 667 670 gmain root txt unknown /proc/667/task/670/exe (readlink: Permission denied) 2024-06-03T03:27:58.528 INFO:tasks.workunit.client.0.smithi117.stdout:irqbalanc 667 670 gmain root NOFD /proc/667/task/670/fd (opendir: Permission denied) 2024-06-03T03:27:58.528 INFO:tasks.workunit.client.0.smithi117.stdout:lsmd 668 libstoragemgmt cwd unknown /proc/668/cwd (readlink: Permission denied) 2024-06-03T03:27:58.528 INFO:tasks.workunit.client.0.smithi117.stdout:lsmd 668 libstoragemgmt rtd unknown /proc/668/root (readlink: Permission denied) 2024-06-03T03:27:58.528 INFO:tasks.workunit.client.0.smithi117.stdout:lsmd 668 libstoragemgmt txt unknown /proc/668/exe (readlink: Permission denied) 2024-06-03T03:27:58.528 INFO:tasks.workunit.client.0.smithi117.stdout:lsmd 668 libstoragemgmt NOFD /proc/668/fd (opendir: Permission denied) 2024-06-03T03:27:58.528 INFO:tasks.workunit.client.0.smithi117.stdout:smartd 669 root cwd unknown /proc/669/cwd (readlink: Permission denied) 2024-06-03T03:27:58.528 INFO:tasks.workunit.client.0.smithi117.stdout:smartd 669 root rtd unknown /proc/669/root (readlink: Permission denied) 2024-06-03T03:27:58.528 INFO:tasks.workunit.client.0.smithi117.stdout:smartd 669 root txt unknown /proc/669/exe (readlink: Permission denied) 2024-06-03T03:27:58.529 INFO:tasks.workunit.client.0.smithi117.stdout:smartd 669 root NOFD /proc/669/fd (opendir: Permission denied) 2024-06-03T03:27:58.529 INFO:tasks.workunit.client.0.smithi117.stdout:systemd-l 672 root cwd unknown /proc/672/cwd (readlink: Permission denied) 2024-06-03T03:27:58.529 INFO:tasks.workunit.client.0.smithi117.stdout:systemd-l 672 root rtd unknown /proc/672/root (readlink: Permission denied) 2024-06-03T03:27:58.529 INFO:tasks.workunit.client.0.smithi117.stdout:systemd-l 672 root txt unknown /proc/672/exe (readlink: Permission denied) 2024-06-03T03:27:58.529 INFO:tasks.workunit.client.0.smithi117.stdout:systemd-l 672 root NOFD /proc/672/fd (opendir: Permission denied) 2024-06-03T03:27:58.529 INFO:tasks.workunit.client.0.smithi117.stdout:sshd 731 root cwd unknown /proc/731/cwd (readlink: Permission denied) 2024-06-03T03:27:58.529 INFO:tasks.workunit.client.0.smithi117.stdout:sshd 731 root rtd unknown /proc/731/root (readlink: Permission denied) 2024-06-03T03:27:58.529 INFO:tasks.workunit.client.0.smithi117.stdout:sshd 731 root txt unknown /proc/731/exe (readlink: Permission denied) 2024-06-03T03:27:58.529 INFO:tasks.workunit.client.0.smithi117.stdout:sshd 731 root NOFD /proc/731/fd (opendir: Permission denied) 2024-06-03T03:27:58.529 INFO:tasks.workunit.client.0.smithi117.stdout:gssproxy 735 root cwd unknown /proc/735/cwd (readlink: Permission denied) 2024-06-03T03:27:58.529 INFO:tasks.workunit.client.0.smithi117.stdout:gssproxy 735 root rtd unknown /proc/735/root (readlink: Permission denied) 2024-06-03T03:27:58.529 INFO:tasks.workunit.client.0.smithi117.stdout:gssproxy 735 root txt unknown /proc/735/exe (readlink: Permission denied) 2024-06-03T03:27:58.529 INFO:tasks.workunit.client.0.smithi117.stdout:gssproxy 735 root NOFD /proc/735/fd (opendir: Permission denied) 2024-06-03T03:27:58.529 INFO:tasks.workunit.client.0.smithi117.stdout:gssproxy 735 736 gssproxy root cwd unknown /proc/735/task/736/cwd (readlink: Permission denied) 2024-06-03T03:27:58.530 INFO:tasks.workunit.client.0.smithi117.stdout:gssproxy 735 736 gssproxy root rtd unknown /proc/735/task/736/root (readlink: Permission denied) 2024-06-03T03:27:58.530 INFO:tasks.workunit.client.0.smithi117.stdout:gssproxy 735 736 gssproxy root txt unknown /proc/735/task/736/exe (readlink: Permission denied) 2024-06-03T03:27:58.530 INFO:tasks.workunit.client.0.smithi117.stdout:gssproxy 735 736 gssproxy root NOFD /proc/735/task/736/fd (opendir: Permission denied) 2024-06-03T03:27:58.530 INFO:tasks.workunit.client.0.smithi117.stdout:gssproxy 735 737 gssproxy root cwd unknown /proc/735/task/737/cwd (readlink: Permission denied) 2024-06-03T03:27:58.530 INFO:tasks.workunit.client.0.smithi117.stdout:gssproxy 735 737 gssproxy root rtd unknown /proc/735/task/737/root (readlink: Permission denied) 2024-06-03T03:27:58.530 INFO:tasks.workunit.client.0.smithi117.stdout:gssproxy 735 737 gssproxy root txt unknown /proc/735/task/737/exe (readlink: Permission denied) 2024-06-03T03:27:58.530 INFO:tasks.workunit.client.0.smithi117.stdout:gssproxy 735 737 gssproxy root NOFD /proc/735/task/737/fd (opendir: Permission denied) 2024-06-03T03:27:58.530 INFO:tasks.workunit.client.0.smithi117.stdout:gssproxy 735 738 gssproxy root cwd unknown /proc/735/task/738/cwd (readlink: Permission denied) 2024-06-03T03:27:58.530 INFO:tasks.workunit.client.0.smithi117.stdout:gssproxy 735 738 gssproxy root rtd unknown /proc/735/task/738/root (readlink: Permission denied) 2024-06-03T03:27:58.530 INFO:tasks.workunit.client.0.smithi117.stdout:gssproxy 735 738 gssproxy root txt unknown /proc/735/task/738/exe (readlink: Permission denied) 2024-06-03T03:27:58.530 INFO:tasks.workunit.client.0.smithi117.stdout:gssproxy 735 738 gssproxy root NOFD /proc/735/task/738/fd (opendir: Permission denied) 2024-06-03T03:27:58.530 INFO:tasks.workunit.client.0.smithi117.stdout:gssproxy 735 739 gssproxy root cwd unknown /proc/735/task/739/cwd (readlink: Permission denied) 2024-06-03T03:27:58.531 INFO:tasks.workunit.client.0.smithi117.stdout:gssproxy 735 739 gssproxy root rtd unknown /proc/735/task/739/root (readlink: Permission denied) 2024-06-03T03:27:58.531 INFO:tasks.workunit.client.0.smithi117.stdout:gssproxy 735 739 gssproxy root txt unknown /proc/735/task/739/exe (readlink: Permission denied) 2024-06-03T03:27:58.531 INFO:tasks.workunit.client.0.smithi117.stdout:gssproxy 735 739 gssproxy root NOFD /proc/735/task/739/fd (opendir: Permission denied) 2024-06-03T03:27:58.531 INFO:tasks.workunit.client.0.smithi117.stdout:gssproxy 735 740 gssproxy root cwd unknown /proc/735/task/740/cwd (readlink: Permission denied) 2024-06-03T03:27:58.531 INFO:tasks.workunit.client.0.smithi117.stdout:gssproxy 735 740 gssproxy root rtd unknown /proc/735/task/740/root (readlink: Permission denied) 2024-06-03T03:27:58.531 INFO:tasks.workunit.client.0.smithi117.stdout:gssproxy 735 740 gssproxy root txt unknown /proc/735/task/740/exe (readlink: Permission denied) 2024-06-03T03:27:58.531 INFO:tasks.workunit.client.0.smithi117.stdout:gssproxy 735 740 gssproxy root NOFD /proc/735/task/740/fd (opendir: Permission denied) 2024-06-03T03:27:58.531 INFO:tasks.workunit.client.0.smithi117.stdout:atd 742 root cwd unknown /proc/742/cwd (readlink: Permission denied) 2024-06-03T03:27:58.531 INFO:tasks.workunit.client.0.smithi117.stdout:atd 742 root rtd unknown /proc/742/root (readlink: Permission denied) 2024-06-03T03:27:58.531 INFO:tasks.workunit.client.0.smithi117.stdout:atd 742 root txt unknown /proc/742/exe (readlink: Permission denied) 2024-06-03T03:27:58.531 INFO:tasks.workunit.client.0.smithi117.stdout:atd 742 root NOFD /proc/742/fd (opendir: Permission denied) 2024-06-03T03:27:58.531 INFO:tasks.workunit.client.0.smithi117.stdout:crond 743 root cwd unknown /proc/743/cwd (readlink: Permission denied) 2024-06-03T03:27:58.531 INFO:tasks.workunit.client.0.smithi117.stdout:crond 743 root rtd unknown /proc/743/root (readlink: Permission denied) 2024-06-03T03:27:58.532 INFO:tasks.workunit.client.0.smithi117.stdout:crond 743 root txt unknown /proc/743/exe (readlink: Permission denied) 2024-06-03T03:27:58.532 INFO:tasks.workunit.client.0.smithi117.stdout:crond 743 root NOFD /proc/743/fd (opendir: Permission denied) 2024-06-03T03:27:58.532 INFO:tasks.workunit.client.0.smithi117.stdout:nrpe 763 nrpe cwd unknown /proc/763/cwd (readlink: Permission denied) 2024-06-03T03:27:58.532 INFO:tasks.workunit.client.0.smithi117.stdout:nrpe 763 nrpe rtd unknown /proc/763/root (readlink: Permission denied) 2024-06-03T03:27:58.532 INFO:tasks.workunit.client.0.smithi117.stdout:nrpe 763 nrpe txt unknown /proc/763/exe (readlink: Permission denied) 2024-06-03T03:27:58.532 INFO:tasks.workunit.client.0.smithi117.stdout:nrpe 763 nrpe NOFD /proc/763/fd (opendir: Permission denied) 2024-06-03T03:27:58.532 INFO:tasks.workunit.client.0.smithi117.stdout:agetty 1087 root cwd unknown /proc/1087/cwd (readlink: Permission denied) 2024-06-03T03:27:58.532 INFO:tasks.workunit.client.0.smithi117.stdout:agetty 1087 root rtd unknown /proc/1087/root (readlink: Permission denied) 2024-06-03T03:27:58.532 INFO:tasks.workunit.client.0.smithi117.stdout:agetty 1087 root txt unknown /proc/1087/exe (readlink: Permission denied) 2024-06-03T03:27:58.532 INFO:tasks.workunit.client.0.smithi117.stdout:agetty 1087 root NOFD /proc/1087/fd (opendir: Permission denied) 2024-06-03T03:27:58.532 INFO:tasks.workunit.client.0.smithi117.stdout:agetty 1091 root cwd unknown /proc/1091/cwd (readlink: Permission denied) 2024-06-03T03:27:58.532 INFO:tasks.workunit.client.0.smithi117.stdout:agetty 1091 root rtd unknown /proc/1091/root (readlink: Permission denied) 2024-06-03T03:27:58.532 INFO:tasks.workunit.client.0.smithi117.stdout:agetty 1091 root txt unknown /proc/1091/exe (readlink: Permission denied) 2024-06-03T03:27:58.532 INFO:tasks.workunit.client.0.smithi117.stdout:agetty 1091 root NOFD /proc/1091/fd (opendir: Permission denied) 2024-06-03T03:27:58.533 INFO:tasks.workunit.client.0.smithi117.stdout:systemd 4235 ubuntu cwd DIR 8,1 4096 2 / 2024-06-03T03:27:58.533 INFO:tasks.workunit.client.0.smithi117.stdout:systemd 4235 ubuntu rtd DIR 8,1 4096 2 / 2024-06-03T03:27:58.533 INFO:tasks.workunit.client.0.smithi117.stdout:systemd 4235 ubuntu txt REG 8,1 98216 18583 /usr/lib/systemd/systemd 2024-06-03T03:27:58.533 INFO:tasks.workunit.client.0.smithi117.stdout:systemd 4235 ubuntu DEL REG 8,1 849 /etc/selinux/targeted/contexts/files/file_contexts.bin 2024-06-03T03:27:58.533 INFO:tasks.workunit.client.0.smithi117.stdout:systemd 4235 ubuntu mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-06-03T03:27:58.533 INFO:tasks.workunit.client.0.smithi117.stdout:systemd 4235 ubuntu mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-06-03T03:27:58.533 INFO:tasks.workunit.client.0.smithi117.stdout:systemd 4235 ubuntu mem REG 8,1 882376 7182 /usr/lib64/libzstd.so.1.5.1 2024-06-03T03:27:58.533 INFO:tasks.workunit.client.0.smithi117.stdout:systemd 4235 ubuntu DEL REG 8,1 7490 /usr/lib64/libcrypto.so.3.0.7 2024-06-03T03:27:58.533 INFO:tasks.workunit.client.0.smithi117.stdout:systemd 4235 ubuntu mem REG 8,1 1714208 7271 /usr/lib64/libp11-kit.so.0.3.1 2024-06-03T03:27:58.533 INFO:tasks.workunit.client.0.smithi117.stdout:systemd 4235 ubuntu mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-06-03T03:27:58.533 INFO:tasks.workunit.client.0.smithi117.stdout:systemd 4235 ubuntu DEL REG 8,1 1130 /etc/selinux/targeted/contexts/files/file_contexts.homedirs.bin 2024-06-03T03:27:58.533 INFO:tasks.workunit.client.0.smithi117.stdout:systemd 4235 ubuntu mem REG 8,1 44784 7268 /usr/lib64/libffi.so.8.1.0 2024-06-03T03:27:58.533 INFO:tasks.workunit.client.0.smithi117.stdout:systemd 4235 ubuntu mem REG 8,1 153600 7292 /usr/lib64/libgpg-error.so.0.32.0 2024-06-03T03:27:58.533 INFO:tasks.workunit.client.0.smithi117.stdout:systemd 4235 ubuntu mem REG 8,1 28552 7287 /usr/lib64/libattr.so.1.1.2501 2024-06-03T03:27:58.534 INFO:tasks.workunit.client.0.smithi117.stdout:systemd 4235 ubuntu mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-06-03T03:27:58.534 INFO:tasks.workunit.client.0.smithi117.stdout:systemd 4235 ubuntu mem REG 8,1 32528 7206 /usr/lib64/libcap-ng.so.0.0.0 2024-06-03T03:27:58.534 INFO:tasks.workunit.client.0.smithi117.stdout:systemd 4235 ubuntu mem REG 8,1 178728 7177 /usr/lib64/liblzma.so.5.2.5 2024-06-03T03:27:58.534 INFO:tasks.workunit.client.0.smithi117.stdout:systemd 4235 ubuntu mem REG 8,1 1288504 7366 /usr/lib64/libgcrypt.so.20.4.0 2024-06-03T03:27:58.534 INFO:tasks.workunit.client.0.smithi117.stdout:systemd 4235 ubuntu mem REG 8,1 3785856 46432 /usr/lib64/systemd/libsystemd-shared-252.so 2024-06-03T03:27:58.534 INFO:tasks.workunit.client.0.smithi117.stdout:systemd 4235 ubuntu mem REG 8,1 40424 8439 /usr/lib64/libeconf.so.0.4.1 2024-06-03T03:27:58.534 INFO:tasks.workunit.client.0.smithi117.stdout:systemd 4235 ubuntu mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-06-03T03:27:58.534 INFO:tasks.workunit.client.0.smithi117.stdout:systemd 4235 ubuntu mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-06-03T03:27:58.534 INFO:tasks.workunit.client.0.smithi117.stdout:systemd 4235 ubuntu mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-06-03T03:27:58.534 INFO:tasks.workunit.client.0.smithi117.stdout:systemd 4235 ubuntu mem REG 8,1 2157904 46429 /usr/lib64/systemd/libsystemd-core-252.so 2024-06-03T03:27:58.534 INFO:tasks.workunit.client.0.smithi117.stdout:systemd 4235 ubuntu mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-06-03T03:27:58.534 INFO:tasks.workunit.client.0.smithi117.stdout:systemd 4235 ubuntu mem REG 8,1 40496 7288 /usr/lib64/libacl.so.1.1.2301 2024-06-03T03:27:58.535 INFO:tasks.workunit.client.0.smithi117.stdout:systemd 4235 ubuntu mem REG 8,1 291512 7639 /usr/lib64/libmount.so.1.1.0 2024-06-03T03:27:58.535 INFO:tasks.workunit.client.0.smithi117.stdout:systemd 4235 ubuntu mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-06-03T03:27:58.535 INFO:tasks.workunit.client.0.smithi117.stdout:systemd 4235 ubuntu mem REG 8,1 107424 7663 /usr/lib64/libkmod.so.2.3.6 2024-06-03T03:27:58.535 INFO:tasks.workunit.client.0.smithi117.stdout:systemd 4235 ubuntu mem REG 8,1 138848 7210 /usr/lib64/libaudit.so.1.0.0 2024-06-03T03:27:58.535 INFO:tasks.workunit.client.0.smithi117.stdout:systemd 4235 ubuntu mem REG 8,1 69592 8448 /usr/lib64/libpam.so.0.85.1 2024-06-03T03:27:58.535 INFO:tasks.workunit.client.0.smithi117.stdout:systemd 4235 ubuntu mem REG 8,1 127680 7455 /usr/lib64/libseccomp.so.2.5.2 2024-06-03T03:27:58.535 INFO:tasks.workunit.client.0.smithi117.stdout:systemd 4235 ubuntu mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-06-03T03:27:58.535 INFO:tasks.workunit.client.0.smithi117.stdout:systemd 4235 ubuntu mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-06-03T03:27:58.535 INFO:tasks.workunit.client.0.smithi117.stdout:systemd 4235 ubuntu mem REG 0,20 0 19 /sys/fs/selinux/status 2024-06-03T03:27:58.535 INFO:tasks.workunit.client.0.smithi117.stdout:systemd 4235 ubuntu 0r CHR 1,3 0t0 4 /dev/null 2024-06-03T03:27:58.535 INFO:tasks.workunit.client.0.smithi117.stdout:systemd 4235 ubuntu 1u unix 0x0000000000000000 0t0 8121 type=STREAM (CONNECTED) 2024-06-03T03:27:58.535 INFO:tasks.workunit.client.0.smithi117.stdout:systemd 4235 ubuntu 2u unix 0x0000000000000000 0t0 8121 type=STREAM (CONNECTED) 2024-06-03T03:27:58.535 INFO:tasks.workunit.client.0.smithi117.stdout:systemd 4235 ubuntu 3u unix 0x0000000000000000 0t0 8150 type=DGRAM (CONNECTED) 2024-06-03T03:27:58.535 INFO:tasks.workunit.client.0.smithi117.stdout:systemd 4235 ubuntu 4u a_inode 0,15 0 50 [eventpoll:5,6,8,11,12,14,15,16,17,19,20,21] 2024-06-03T03:27:58.536 INFO:tasks.workunit.client.0.smithi117.stdout:systemd 4235 ubuntu 5u a_inode 0,15 0 50 [signalfd] 2024-06-03T03:27:58.536 INFO:tasks.workunit.client.0.smithi117.stdout:systemd 4235 ubuntu 6r a_inode 0,15 0 50 inotify 2024-06-03T03:27:58.536 INFO:tasks.workunit.client.0.smithi117.stdout:systemd 4235 ubuntu 7r DIR 0,26 0 3619 /sys/fs/cgroup/user.slice/user-1000.slice/user@1000.service 2024-06-03T03:27:58.536 INFO:tasks.workunit.client.0.smithi117.stdout:systemd 4235 ubuntu 8u a_inode 0,15 0 50 [timerfd] 2024-06-03T03:27:58.536 INFO:tasks.workunit.client.0.smithi117.stdout:systemd 4235 ubuntu 9u a_inode 0,15 0 50 [eventpoll:10,13] 2024-06-03T03:27:58.536 INFO:tasks.workunit.client.0.smithi117.stdout:systemd 4235 ubuntu 10r REG 0,21 0 8139 /proc/4235/mountinfo 2024-06-03T03:27:58.536 INFO:tasks.workunit.client.0.smithi117.stdout:systemd 4235 ubuntu 11r a_inode 0,15 0 50 inotify 2024-06-03T03:27:58.536 INFO:tasks.workunit.client.0.smithi117.stdout:systemd 4235 ubuntu 12u unix 0x0000000000000000 0t0 8172 /run/user/1000/bus type=STREAM (LISTEN) 2024-06-03T03:27:58.536 INFO:tasks.workunit.client.0.smithi117.stdout:systemd 4235 ubuntu 13r a_inode 0,15 0 50 inotify 2024-06-03T03:27:58.536 INFO:tasks.workunit.client.0.smithi117.stdout:systemd 4235 ubuntu 14r REG 0,21 0 4026532078 /proc/swaps 2024-06-03T03:27:58.536 INFO:tasks.workunit.client.0.smithi117.stdout:systemd 4235 ubuntu 15u netlink 0t0 8154 KOBJECT_UEVENT 2024-06-03T03:27:58.536 INFO:tasks.workunit.client.0.smithi117.stdout:systemd 4235 ubuntu 16u unix 0x0000000000000000 0t0 8159 /run/user/1000/systemd/notify type=DGRAM (UNCONNECTED) 2024-06-03T03:27:58.536 INFO:tasks.workunit.client.0.smithi117.stdout:systemd 4235 ubuntu 17u unix 0x0000000000000000 0t0 8160 type=DGRAM (CONNECTED) 2024-06-03T03:27:58.536 INFO:tasks.workunit.client.0.smithi117.stdout:systemd 4235 ubuntu 18u unix 0x0000000000000000 0t0 8161 type=DGRAM (CONNECTED) 2024-06-03T03:27:58.537 INFO:tasks.workunit.client.0.smithi117.stdout:systemd 4235 ubuntu 19u unix 0x0000000000000000 0t0 8162 /run/user/1000/systemd/private type=STREAM (LISTEN) 2024-06-03T03:27:58.537 INFO:tasks.workunit.client.0.smithi117.stdout:systemd 4235 ubuntu 20u unix 0x0000000000000000 0t0 8164 type=STREAM (CONNECTED) 2024-06-03T03:27:58.537 INFO:tasks.workunit.client.0.smithi117.stdout:systemd 4235 ubuntu 21u a_inode 0,15 0 50 [timerfd] 2024-06-03T03:27:58.537 INFO:tasks.workunit.client.0.smithi117.stdout:(sd-pam) 4238 ubuntu cwd unknown /proc/4238/cwd (readlink: Permission denied) 2024-06-03T03:27:58.537 INFO:tasks.workunit.client.0.smithi117.stdout:(sd-pam) 4238 ubuntu rtd unknown /proc/4238/root (readlink: Permission denied) 2024-06-03T03:27:58.537 INFO:tasks.workunit.client.0.smithi117.stdout:(sd-pam) 4238 ubuntu txt unknown /proc/4238/exe (readlink: Permission denied) 2024-06-03T03:27:58.537 INFO:tasks.workunit.client.0.smithi117.stdout:(sd-pam) 4238 ubuntu NOFD /proc/4238/fd (opendir: Permission denied) 2024-06-03T03:27:58.537 INFO:tasks.workunit.client.0.smithi117.stdout:rsyslogd 4481 root cwd unknown /proc/4481/cwd (readlink: Permission denied) 2024-06-03T03:27:58.537 INFO:tasks.workunit.client.0.smithi117.stdout:rsyslogd 4481 root rtd unknown /proc/4481/root (readlink: Permission denied) 2024-06-03T03:27:58.537 INFO:tasks.workunit.client.0.smithi117.stdout:rsyslogd 4481 root txt unknown /proc/4481/exe (readlink: Permission denied) 2024-06-03T03:27:58.537 INFO:tasks.workunit.client.0.smithi117.stdout:rsyslogd 4481 root NOFD /proc/4481/fd (opendir: Permission denied) 2024-06-03T03:27:58.537 INFO:tasks.workunit.client.0.smithi117.stdout:rsyslogd 4481 4483 in:imjour root cwd unknown /proc/4481/task/4483/cwd (readlink: Permission denied) 2024-06-03T03:27:58.537 INFO:tasks.workunit.client.0.smithi117.stdout:rsyslogd 4481 4483 in:imjour root rtd unknown /proc/4481/task/4483/root (readlink: Permission denied) 2024-06-03T03:27:58.537 INFO:tasks.workunit.client.0.smithi117.stdout:rsyslogd 4481 4483 in:imjour root txt unknown /proc/4481/task/4483/exe (readlink: Permission denied) 2024-06-03T03:27:58.538 INFO:tasks.workunit.client.0.smithi117.stdout:rsyslogd 4481 4483 in:imjour root NOFD /proc/4481/task/4483/fd (opendir: Permission denied) 2024-06-03T03:27:58.538 INFO:tasks.workunit.client.0.smithi117.stdout:rsyslogd 4481 4484 rs:main root cwd unknown /proc/4481/task/4484/cwd (readlink: Permission denied) 2024-06-03T03:27:58.538 INFO:tasks.workunit.client.0.smithi117.stdout:rsyslogd 4481 4484 rs:main root rtd unknown /proc/4481/task/4484/root (readlink: Permission denied) 2024-06-03T03:27:58.538 INFO:tasks.workunit.client.0.smithi117.stdout:rsyslogd 4481 4484 rs:main root txt unknown /proc/4481/task/4484/exe (readlink: Permission denied) 2024-06-03T03:27:58.538 INFO:tasks.workunit.client.0.smithi117.stdout:rsyslogd 4481 4484 rs:main root NOFD /proc/4481/task/4484/fd (opendir: Permission denied) 2024-06-03T03:27:58.538 INFO:tasks.workunit.client.0.smithi117.stdout:anacron 5187 root cwd unknown /proc/5187/cwd (readlink: Permission denied) 2024-06-03T03:27:58.538 INFO:tasks.workunit.client.0.smithi117.stdout:anacron 5187 root rtd unknown /proc/5187/root (readlink: Permission denied) 2024-06-03T03:27:58.538 INFO:tasks.workunit.client.0.smithi117.stdout:anacron 5187 root txt unknown /proc/5187/exe (readlink: Permission denied) 2024-06-03T03:27:58.538 INFO:tasks.workunit.client.0.smithi117.stdout:anacron 5187 root NOFD /proc/5187/fd (opendir: Permission denied) 2024-06-03T03:27:58.538 INFO:tasks.workunit.client.0.smithi117.stdout:polkitd 16340 polkitd cwd unknown /proc/16340/cwd (readlink: Permission denied) 2024-06-03T03:27:58.539 INFO:tasks.workunit.client.0.smithi117.stdout:polkitd 16340 polkitd rtd unknown /proc/16340/root (readlink: Permission denied) 2024-06-03T03:27:58.539 INFO:tasks.workunit.client.0.smithi117.stdout:polkitd 16340 polkitd txt unknown /proc/16340/exe (readlink: Permission denied) 2024-06-03T03:27:58.539 INFO:tasks.workunit.client.0.smithi117.stdout:polkitd 16340 polkitd NOFD /proc/16340/fd (opendir: Permission denied) 2024-06-03T03:27:58.539 INFO:tasks.workunit.client.0.smithi117.stdout:polkitd 16340 16342 gmain polkitd cwd unknown /proc/16340/task/16342/cwd (readlink: Permission denied) 2024-06-03T03:27:58.539 INFO:tasks.workunit.client.0.smithi117.stdout:polkitd 16340 16342 gmain polkitd rtd unknown /proc/16340/task/16342/root (readlink: Permission denied) 2024-06-03T03:27:58.539 INFO:tasks.workunit.client.0.smithi117.stdout:polkitd 16340 16342 gmain polkitd txt unknown /proc/16340/task/16342/exe (readlink: Permission denied) 2024-06-03T03:27:58.540 INFO:tasks.workunit.client.0.smithi117.stdout:polkitd 16340 16342 gmain polkitd NOFD /proc/16340/task/16342/fd (opendir: Permission denied) 2024-06-03T03:27:58.540 INFO:tasks.workunit.client.0.smithi117.stdout:polkitd 16340 16343 gdbus polkitd cwd unknown /proc/16340/task/16343/cwd (readlink: Permission denied) 2024-06-03T03:27:58.540 INFO:tasks.workunit.client.0.smithi117.stdout:polkitd 16340 16343 gdbus polkitd rtd unknown /proc/16340/task/16343/root (readlink: Permission denied) 2024-06-03T03:27:58.540 INFO:tasks.workunit.client.0.smithi117.stdout:polkitd 16340 16343 gdbus polkitd txt unknown /proc/16340/task/16343/exe (readlink: Permission denied) 2024-06-03T03:27:58.540 INFO:tasks.workunit.client.0.smithi117.stdout:polkitd 16340 16343 gdbus polkitd NOFD /proc/16340/task/16343/fd (opendir: Permission denied) 2024-06-03T03:27:58.540 INFO:tasks.workunit.client.0.smithi117.stdout:polkitd 16340 16344 JS\x20Hel polkitd cwd unknown /proc/16340/task/16344/cwd (readlink: Permission denied) 2024-06-03T03:27:58.541 INFO:tasks.workunit.client.0.smithi117.stdout:polkitd 16340 16344 JS\x20Hel polkitd rtd unknown /proc/16340/task/16344/root (readlink: Permission denied) 2024-06-03T03:27:58.541 INFO:tasks.workunit.client.0.smithi117.stdout:polkitd 16340 16344 JS\x20Hel polkitd txt unknown /proc/16340/task/16344/exe (readlink: Permission denied) 2024-06-03T03:27:58.541 INFO:tasks.workunit.client.0.smithi117.stdout:polkitd 16340 16344 JS\x20Hel polkitd NOFD /proc/16340/task/16344/fd (opendir: Permission denied) 2024-06-03T03:27:58.541 INFO:tasks.workunit.client.0.smithi117.stdout:polkitd 16340 16345 JS\x20Hel polkitd cwd unknown /proc/16340/task/16345/cwd (readlink: Permission denied) 2024-06-03T03:27:58.541 INFO:tasks.workunit.client.0.smithi117.stdout:polkitd 16340 16345 JS\x20Hel polkitd rtd unknown /proc/16340/task/16345/root (readlink: Permission denied) 2024-06-03T03:27:58.541 INFO:tasks.workunit.client.0.smithi117.stdout:polkitd 16340 16345 JS\x20Hel polkitd txt unknown /proc/16340/task/16345/exe (readlink: Permission denied) 2024-06-03T03:27:58.542 INFO:tasks.workunit.client.0.smithi117.stdout:polkitd 16340 16345 JS\x20Hel polkitd NOFD /proc/16340/task/16345/fd (opendir: Permission denied) 2024-06-03T03:27:58.542 INFO:tasks.workunit.client.0.smithi117.stdout:polkitd 16340 16346 JS\x20Hel polkitd cwd unknown /proc/16340/task/16346/cwd (readlink: Permission denied) 2024-06-03T03:27:58.542 INFO:tasks.workunit.client.0.smithi117.stdout:polkitd 16340 16346 JS\x20Hel polkitd rtd unknown /proc/16340/task/16346/root (readlink: Permission denied) 2024-06-03T03:27:58.542 INFO:tasks.workunit.client.0.smithi117.stdout:polkitd 16340 16346 JS\x20Hel polkitd txt unknown /proc/16340/task/16346/exe (readlink: Permission denied) 2024-06-03T03:27:58.542 INFO:tasks.workunit.client.0.smithi117.stdout:polkitd 16340 16346 JS\x20Hel polkitd NOFD /proc/16340/task/16346/fd (opendir: Permission denied) 2024-06-03T03:27:58.542 INFO:tasks.workunit.client.0.smithi117.stdout:polkitd 16340 16347 JS\x20Hel polkitd cwd unknown /proc/16340/task/16347/cwd (readlink: Permission denied) 2024-06-03T03:27:58.542 INFO:tasks.workunit.client.0.smithi117.stdout:polkitd 16340 16347 JS\x20Hel polkitd rtd unknown /proc/16340/task/16347/root (readlink: Permission denied) 2024-06-03T03:27:58.543 INFO:tasks.workunit.client.0.smithi117.stdout:polkitd 16340 16347 JS\x20Hel polkitd txt unknown /proc/16340/task/16347/exe (readlink: Permission denied) 2024-06-03T03:27:58.543 INFO:tasks.workunit.client.0.smithi117.stdout:polkitd 16340 16347 JS\x20Hel polkitd NOFD /proc/16340/task/16347/fd (opendir: Permission denied) 2024-06-03T03:27:58.543 INFO:tasks.workunit.client.0.smithi117.stdout:polkitd 16340 16348 JS\x20Hel polkitd cwd unknown /proc/16340/task/16348/cwd (readlink: Permission denied) 2024-06-03T03:27:58.543 INFO:tasks.workunit.client.0.smithi117.stdout:polkitd 16340 16348 JS\x20Hel polkitd rtd unknown /proc/16340/task/16348/root (readlink: Permission denied) 2024-06-03T03:27:58.543 INFO:tasks.workunit.client.0.smithi117.stdout:polkitd 16340 16348 JS\x20Hel polkitd txt unknown /proc/16340/task/16348/exe (readlink: Permission denied) 2024-06-03T03:27:58.543 INFO:tasks.workunit.client.0.smithi117.stdout:polkitd 16340 16348 JS\x20Hel polkitd NOFD /proc/16340/task/16348/fd (opendir: Permission denied) 2024-06-03T03:27:58.543 INFO:tasks.workunit.client.0.smithi117.stdout:polkitd 16340 16349 JS\x20Hel polkitd cwd unknown /proc/16340/task/16349/cwd (readlink: Permission denied) 2024-06-03T03:27:58.543 INFO:tasks.workunit.client.0.smithi117.stdout:polkitd 16340 16349 JS\x20Hel polkitd rtd unknown /proc/16340/task/16349/root (readlink: Permission denied) 2024-06-03T03:27:58.544 INFO:tasks.workunit.client.0.smithi117.stdout:polkitd 16340 16349 JS\x20Hel polkitd txt unknown /proc/16340/task/16349/exe (readlink: Permission denied) 2024-06-03T03:27:58.544 INFO:tasks.workunit.client.0.smithi117.stdout:polkitd 16340 16349 JS\x20Hel polkitd NOFD /proc/16340/task/16349/fd (opendir: Permission denied) 2024-06-03T03:27:58.544 INFO:tasks.workunit.client.0.smithi117.stdout:polkitd 16340 16350 JS\x20Hel polkitd cwd unknown /proc/16340/task/16350/cwd (readlink: Permission denied) 2024-06-03T03:27:58.544 INFO:tasks.workunit.client.0.smithi117.stdout:polkitd 16340 16350 JS\x20Hel polkitd rtd unknown /proc/16340/task/16350/root (readlink: Permission denied) 2024-06-03T03:27:58.544 INFO:tasks.workunit.client.0.smithi117.stdout:polkitd 16340 16350 JS\x20Hel polkitd txt unknown /proc/16340/task/16350/exe (readlink: Permission denied) 2024-06-03T03:27:58.544 INFO:tasks.workunit.client.0.smithi117.stdout:polkitd 16340 16350 JS\x20Hel polkitd NOFD /proc/16340/task/16350/fd (opendir: Permission denied) 2024-06-03T03:27:58.544 INFO:tasks.workunit.client.0.smithi117.stdout:polkitd 16340 16351 JS\x20Hel polkitd cwd unknown /proc/16340/task/16351/cwd (readlink: Permission denied) 2024-06-03T03:27:58.544 INFO:tasks.workunit.client.0.smithi117.stdout:polkitd 16340 16351 JS\x20Hel polkitd rtd unknown /proc/16340/task/16351/root (readlink: Permission denied) 2024-06-03T03:27:58.545 INFO:tasks.workunit.client.0.smithi117.stdout:polkitd 16340 16351 JS\x20Hel polkitd txt unknown /proc/16340/task/16351/exe (readlink: Permission denied) 2024-06-03T03:27:58.545 INFO:tasks.workunit.client.0.smithi117.stdout:polkitd 16340 16351 JS\x20Hel polkitd NOFD /proc/16340/task/16351/fd (opendir: Permission denied) 2024-06-03T03:27:58.545 INFO:tasks.workunit.client.0.smithi117.stdout:polkitd 16340 16352 polkitd polkitd cwd unknown /proc/16340/task/16352/cwd (readlink: Permission denied) 2024-06-03T03:27:58.545 INFO:tasks.workunit.client.0.smithi117.stdout:polkitd 16340 16352 polkitd polkitd rtd unknown /proc/16340/task/16352/root (readlink: Permission denied) 2024-06-03T03:27:58.545 INFO:tasks.workunit.client.0.smithi117.stdout:polkitd 16340 16352 polkitd polkitd txt unknown /proc/16340/task/16352/exe (readlink: Permission denied) 2024-06-03T03:27:58.545 INFO:tasks.workunit.client.0.smithi117.stdout:polkitd 16340 16352 polkitd polkitd NOFD /proc/16340/task/16352/fd (opendir: Permission denied) 2024-06-03T03:27:58.546 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/3 17477 root cwd unknown /proc/17477/cwd (readlink: Permission denied) 2024-06-03T03:27:58.546 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/3 17477 root rtd unknown /proc/17477/root (readlink: Permission denied) 2024-06-03T03:27:58.546 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/3 17477 root txt unknown /proc/17477/exe (readlink: Permission denied) 2024-06-03T03:27:58.546 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/3 17477 root NOFD /proc/17477/fd (opendir: Permission denied) 2024-06-03T03:27:58.546 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 20391 root cwd unknown /proc/20391/cwd (readlink: Permission denied) 2024-06-03T03:27:58.546 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 20391 root rtd unknown /proc/20391/root (readlink: Permission denied) 2024-06-03T03:27:58.547 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 20391 root txt unknown /proc/20391/exe (readlink: Permission denied) 2024-06-03T03:27:58.547 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 20391 root NOFD /proc/20391/fd (opendir: Permission denied) 2024-06-03T03:27:58.547 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 20421 root cwd unknown /proc/20421/cwd (readlink: Permission denied) 2024-06-03T03:27:58.547 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 20421 root rtd unknown /proc/20421/root (readlink: Permission denied) 2024-06-03T03:27:58.547 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 20421 root txt unknown /proc/20421/exe (readlink: Permission denied) 2024-06-03T03:27:58.547 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 20421 root NOFD /proc/20421/fd (opendir: Permission denied) 2024-06-03T03:27:58.548 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 20450 root cwd unknown /proc/20450/cwd (readlink: Permission denied) 2024-06-03T03:27:58.548 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 20450 root rtd unknown /proc/20450/root (readlink: Permission denied) 2024-06-03T03:27:58.548 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 20450 root txt unknown /proc/20450/exe (readlink: Permission denied) 2024-06-03T03:27:58.548 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 20450 root NOFD /proc/20450/fd (opendir: Permission denied) 2024-06-03T03:27:58.548 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 20479 root cwd unknown /proc/20479/cwd (readlink: Permission denied) 2024-06-03T03:27:58.548 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 20479 root rtd unknown /proc/20479/root (readlink: Permission denied) 2024-06-03T03:27:58.548 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 20479 root txt unknown /proc/20479/exe (readlink: Permission denied) 2024-06-03T03:27:58.548 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 20479 root NOFD /proc/20479/fd (opendir: Permission denied) 2024-06-03T03:27:58.548 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 20508 root cwd unknown /proc/20508/cwd (readlink: Permission denied) 2024-06-03T03:27:58.548 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 20508 root rtd unknown /proc/20508/root (readlink: Permission denied) 2024-06-03T03:27:58.549 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 20508 root txt unknown /proc/20508/exe (readlink: Permission denied) 2024-06-03T03:27:58.549 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 20508 root NOFD /proc/20508/fd (opendir: Permission denied) 2024-06-03T03:27:58.549 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 20684 root cwd unknown /proc/20684/cwd (readlink: Permission denied) 2024-06-03T03:27:58.549 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 20684 root rtd unknown /proc/20684/root (readlink: Permission denied) 2024-06-03T03:27:58.549 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 20684 root txt unknown /proc/20684/exe (readlink: Permission denied) 2024-06-03T03:27:58.549 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 20684 root NOFD /proc/20684/fd (opendir: Permission denied) 2024-06-03T03:27:58.549 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 20685 root cwd unknown /proc/20685/cwd (readlink: Permission denied) 2024-06-03T03:27:58.549 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 20685 root rtd unknown /proc/20685/root (readlink: Permission denied) 2024-06-03T03:27:58.549 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 20685 root txt unknown /proc/20685/exe (readlink: Permission denied) 2024-06-03T03:27:58.549 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 20685 root NOFD /proc/20685/fd (opendir: Permission denied) 2024-06-03T03:27:58.549 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 20686 root cwd unknown /proc/20686/cwd (readlink: Permission denied) 2024-06-03T03:27:58.549 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 20686 root rtd unknown /proc/20686/root (readlink: Permission denied) 2024-06-03T03:27:58.550 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 20686 root txt unknown /proc/20686/exe (readlink: Permission denied) 2024-06-03T03:27:58.550 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 20686 root NOFD /proc/20686/fd (opendir: Permission denied) 2024-06-03T03:27:58.550 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 20687 root cwd unknown /proc/20687/cwd (readlink: Permission denied) 2024-06-03T03:27:58.550 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 20687 root rtd unknown /proc/20687/root (readlink: Permission denied) 2024-06-03T03:27:58.550 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 20687 root txt unknown /proc/20687/exe (readlink: Permission denied) 2024-06-03T03:27:58.550 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 20687 root NOFD /proc/20687/fd (opendir: Permission denied) 2024-06-03T03:27:58.550 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 20688 root cwd unknown /proc/20688/cwd (readlink: Permission denied) 2024-06-03T03:27:58.551 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 20688 root rtd unknown /proc/20688/root (readlink: Permission denied) 2024-06-03T03:27:58.551 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 20688 root txt unknown /proc/20688/exe (readlink: Permission denied) 2024-06-03T03:27:58.551 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 20688 root NOFD /proc/20688/fd (opendir: Permission denied) 2024-06-03T03:27:58.551 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 20689 root cwd unknown /proc/20689/cwd (readlink: Permission denied) 2024-06-03T03:27:58.551 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 20689 root rtd unknown /proc/20689/root (readlink: Permission denied) 2024-06-03T03:27:58.551 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 20689 root txt unknown /proc/20689/exe (readlink: Permission denied) 2024-06-03T03:27:58.551 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 20689 root NOFD /proc/20689/fd (opendir: Permission denied) 2024-06-03T03:27:58.551 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 20690 root cwd unknown /proc/20690/cwd (readlink: Permission denied) 2024-06-03T03:27:58.551 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 20690 root rtd unknown /proc/20690/root (readlink: Permission denied) 2024-06-03T03:27:58.551 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 20690 root txt unknown /proc/20690/exe (readlink: Permission denied) 2024-06-03T03:27:58.551 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 20690 root NOFD /proc/20690/fd (opendir: Permission denied) 2024-06-03T03:27:58.551 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 20691 root cwd unknown /proc/20691/cwd (readlink: Permission denied) 2024-06-03T03:27:58.551 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 20691 root rtd unknown /proc/20691/root (readlink: Permission denied) 2024-06-03T03:27:58.551 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 20691 root txt unknown /proc/20691/exe (readlink: Permission denied) 2024-06-03T03:27:58.552 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 20691 root NOFD /proc/20691/fd (opendir: Permission denied) 2024-06-03T03:27:58.552 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 20692 root cwd unknown /proc/20692/cwd (readlink: Permission denied) 2024-06-03T03:27:58.552 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 20692 root rtd unknown /proc/20692/root (readlink: Permission denied) 2024-06-03T03:27:58.552 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 20692 root txt unknown /proc/20692/exe (readlink: Permission denied) 2024-06-03T03:27:58.552 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 20692 root NOFD /proc/20692/fd (opendir: Permission denied) 2024-06-03T03:27:58.552 INFO:tasks.workunit.client.0.smithi117.stdout:xfsaild/d 20693 root cwd unknown /proc/20693/cwd (readlink: Permission denied) 2024-06-03T03:27:58.552 INFO:tasks.workunit.client.0.smithi117.stdout:xfsaild/d 20693 root rtd unknown /proc/20693/root (readlink: Permission denied) 2024-06-03T03:27:58.552 INFO:tasks.workunit.client.0.smithi117.stdout:xfsaild/d 20693 root txt unknown /proc/20693/exe (readlink: Permission denied) 2024-06-03T03:27:58.552 INFO:tasks.workunit.client.0.smithi117.stdout:xfsaild/d 20693 root NOFD /proc/20693/fd (opendir: Permission denied) 2024-06-03T03:27:58.552 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/0 21199 root cwd unknown /proc/21199/cwd (readlink: Permission denied) 2024-06-03T03:27:58.552 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/0 21199 root rtd unknown /proc/21199/root (readlink: Permission denied) 2024-06-03T03:27:58.552 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/0 21199 root txt unknown /proc/21199/exe (readlink: Permission denied) 2024-06-03T03:27:58.552 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/0 21199 root NOFD /proc/21199/fd (opendir: Permission denied) 2024-06-03T03:27:58.553 INFO:tasks.workunit.client.0.smithi117.stdout:sshd 23124 root cwd unknown /proc/23124/cwd (readlink: Permission denied) 2024-06-03T03:27:58.553 INFO:tasks.workunit.client.0.smithi117.stdout:sshd 23124 root rtd unknown /proc/23124/root (readlink: Permission denied) 2024-06-03T03:27:58.553 INFO:tasks.workunit.client.0.smithi117.stdout:sshd 23124 root txt unknown /proc/23124/exe (readlink: Permission denied) 2024-06-03T03:27:58.553 INFO:tasks.workunit.client.0.smithi117.stdout:sshd 23124 root NOFD /proc/23124/fd (opendir: Permission denied) 2024-06-03T03:27:58.553 INFO:tasks.workunit.client.0.smithi117.stdout:sshd 23127 ubuntu cwd unknown /proc/23127/cwd (readlink: Permission denied) 2024-06-03T03:27:58.553 INFO:tasks.workunit.client.0.smithi117.stdout:sshd 23127 ubuntu rtd unknown /proc/23127/root (readlink: Permission denied) 2024-06-03T03:27:58.553 INFO:tasks.workunit.client.0.smithi117.stdout:sshd 23127 ubuntu txt unknown /proc/23127/exe (readlink: Permission denied) 2024-06-03T03:27:58.553 INFO:tasks.workunit.client.0.smithi117.stdout:sshd 23127 ubuntu NOFD /proc/23127/fd (opendir: Permission denied) 2024-06-03T03:27:58.553 INFO:tasks.workunit.client.0.smithi117.stdout:chronyd 23187 chrony cwd unknown /proc/23187/cwd (readlink: Permission denied) 2024-06-03T03:27:58.553 INFO:tasks.workunit.client.0.smithi117.stdout:chronyd 23187 chrony rtd unknown /proc/23187/root (readlink: Permission denied) 2024-06-03T03:27:58.553 INFO:tasks.workunit.client.0.smithi117.stdout:chronyd 23187 chrony txt unknown /proc/23187/exe (readlink: Permission denied) 2024-06-03T03:27:58.553 INFO:tasks.workunit.client.0.smithi117.stdout:chronyd 23187 chrony NOFD /proc/23187/fd (opendir: Permission denied) 2024-06-03T03:27:58.553 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-cras 23523 ceph cwd unknown /proc/23523/cwd (readlink: Permission denied) 2024-06-03T03:27:58.554 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-cras 23523 ceph rtd unknown /proc/23523/root (readlink: Permission denied) 2024-06-03T03:27:58.554 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-cras 23523 ceph txt unknown /proc/23523/exe (readlink: Permission denied) 2024-06-03T03:27:58.554 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-cras 23523 ceph NOFD /proc/23523/fd (opendir: Permission denied) 2024-06-03T03:27:58.554 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/4 24802 root cwd unknown /proc/24802/cwd (readlink: Permission denied) 2024-06-03T03:27:58.554 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/4 24802 root rtd unknown /proc/24802/root (readlink: Permission denied) 2024-06-03T03:27:58.554 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/4 24802 root txt unknown /proc/24802/exe (readlink: Permission denied) 2024-06-03T03:27:58.554 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/4 24802 root NOFD /proc/24802/fd (opendir: Permission denied) 2024-06-03T03:27:58.554 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/4 24803 root cwd unknown /proc/24803/cwd (readlink: Permission denied) 2024-06-03T03:27:58.554 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/4 24803 root rtd unknown /proc/24803/root (readlink: Permission denied) 2024-06-03T03:27:58.554 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/4 24803 root txt unknown /proc/24803/exe (readlink: Permission denied) 2024-06-03T03:27:58.554 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/4 24803 root NOFD /proc/24803/fd (opendir: Permission denied) 2024-06-03T03:27:58.554 INFO:tasks.workunit.client.0.smithi117.stdout:sudo 30663 root cwd unknown /proc/30663/cwd (readlink: Permission denied) 2024-06-03T03:27:58.554 INFO:tasks.workunit.client.0.smithi117.stdout:sudo 30663 root rtd unknown /proc/30663/root (readlink: Permission denied) 2024-06-03T03:27:58.554 INFO:tasks.workunit.client.0.smithi117.stdout:sudo 30663 root txt unknown /proc/30663/exe (readlink: Permission denied) 2024-06-03T03:27:58.555 INFO:tasks.workunit.client.0.smithi117.stdout:sudo 30663 root NOFD /proc/30663/fd (opendir: Permission denied) 2024-06-03T03:27:58.555 INFO:tasks.workunit.client.0.smithi117.stdout:sudo 30679 root cwd unknown /proc/30679/cwd (readlink: Permission denied) 2024-06-03T03:27:58.555 INFO:tasks.workunit.client.0.smithi117.stdout:sudo 30679 root rtd unknown /proc/30679/root (readlink: Permission denied) 2024-06-03T03:27:58.555 INFO:tasks.workunit.client.0.smithi117.stdout:sudo 30679 root txt unknown /proc/30679/exe (readlink: Permission denied) 2024-06-03T03:27:58.555 INFO:tasks.workunit.client.0.smithi117.stdout:sudo 30679 root NOFD /proc/30679/fd (opendir: Permission denied) 2024-06-03T03:27:58.555 INFO:tasks.workunit.client.0.smithi117.stdout:journalct 30688 root cwd unknown /proc/30688/cwd (readlink: Permission denied) 2024-06-03T03:27:58.555 INFO:tasks.workunit.client.0.smithi117.stdout:journalct 30688 root rtd unknown /proc/30688/root (readlink: Permission denied) 2024-06-03T03:27:58.555 INFO:tasks.workunit.client.0.smithi117.stdout:journalct 30688 root txt unknown /proc/30688/exe (readlink: Permission denied) 2024-06-03T03:27:58.555 INFO:tasks.workunit.client.0.smithi117.stdout:journalct 30688 root NOFD /proc/30688/fd (opendir: Permission denied) 2024-06-03T03:27:58.555 INFO:tasks.workunit.client.0.smithi117.stdout:journalct 30709 root cwd unknown /proc/30709/cwd (readlink: Permission denied) 2024-06-03T03:27:58.555 INFO:tasks.workunit.client.0.smithi117.stdout:journalct 30709 root rtd unknown /proc/30709/root (readlink: Permission denied) 2024-06-03T03:27:58.555 INFO:tasks.workunit.client.0.smithi117.stdout:journalct 30709 root txt unknown /proc/30709/exe (readlink: Permission denied) 2024-06-03T03:27:58.555 INFO:tasks.workunit.client.0.smithi117.stdout:journalct 30709 root NOFD /proc/30709/fd (opendir: Permission denied) 2024-06-03T03:27:58.555 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/5 30927 root cwd unknown /proc/30927/cwd (readlink: Permission denied) 2024-06-03T03:27:58.556 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/5 30927 root rtd unknown /proc/30927/root (readlink: Permission denied) 2024-06-03T03:27:58.556 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/5 30927 root txt unknown /proc/30927/exe (readlink: Permission denied) 2024-06-03T03:27:58.556 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/5 30927 root NOFD /proc/30927/fd (opendir: Permission denied) 2024-06-03T03:27:58.556 INFO:tasks.workunit.client.0.smithi117.stdout:conmon 31645 root cwd unknown /proc/31645/cwd (readlink: Permission denied) 2024-06-03T03:27:58.556 INFO:tasks.workunit.client.0.smithi117.stdout:conmon 31645 root rtd unknown /proc/31645/root (readlink: Permission denied) 2024-06-03T03:27:58.556 INFO:tasks.workunit.client.0.smithi117.stdout:conmon 31645 root txt unknown /proc/31645/exe (readlink: Permission denied) 2024-06-03T03:27:58.556 INFO:tasks.workunit.client.0.smithi117.stdout:conmon 31645 root NOFD /proc/31645/fd (opendir: Permission denied) 2024-06-03T03:27:58.556 INFO:tasks.workunit.client.0.smithi117.stdout:podman-in 31647 root cwd unknown /proc/31647/cwd (readlink: Permission denied) 2024-06-03T03:27:58.556 INFO:tasks.workunit.client.0.smithi117.stdout:podman-in 31647 root rtd unknown /proc/31647/root (readlink: Permission denied) 2024-06-03T03:27:58.556 INFO:tasks.workunit.client.0.smithi117.stdout:podman-in 31647 root txt unknown /proc/31647/exe (readlink: Permission denied) 2024-06-03T03:27:58.556 INFO:tasks.workunit.client.0.smithi117.stdout:podman-in 31647 root NOFD /proc/31647/fd (opendir: Permission denied) 2024-06-03T03:27:58.556 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 ceph cwd unknown /proc/31649/cwd (readlink: Permission denied) 2024-06-03T03:27:58.556 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 ceph rtd unknown /proc/31649/root (readlink: Permission denied) 2024-06-03T03:27:58.557 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 ceph txt unknown /proc/31649/exe (readlink: Permission denied) 2024-06-03T03:27:58.557 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 ceph NOFD /proc/31649/fd (opendir: Permission denied) 2024-06-03T03:27:58.557 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31650 log ceph cwd unknown /proc/31649/task/31650/cwd (readlink: Permission denied) 2024-06-03T03:27:58.557 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31650 log ceph rtd unknown /proc/31649/task/31650/root (readlink: Permission denied) 2024-06-03T03:27:58.557 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31650 log ceph txt unknown /proc/31649/task/31650/exe (readlink: Permission denied) 2024-06-03T03:27:58.557 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31650 log ceph NOFD /proc/31649/task/31650/fd (opendir: Permission denied) 2024-06-03T03:27:58.557 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31651 service ceph cwd unknown /proc/31649/task/31651/cwd (readlink: Permission denied) 2024-06-03T03:27:58.557 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31651 service ceph rtd unknown /proc/31649/task/31651/root (readlink: Permission denied) 2024-06-03T03:27:58.557 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31651 service ceph txt unknown /proc/31649/task/31651/exe (readlink: Permission denied) 2024-06-03T03:27:58.557 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31651 service ceph NOFD /proc/31649/task/31651/fd (opendir: Permission denied) 2024-06-03T03:27:58.557 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31652 admin_soc ceph cwd unknown /proc/31649/task/31652/cwd (readlink: Permission denied) 2024-06-03T03:27:58.557 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31652 admin_soc ceph rtd unknown /proc/31649/task/31652/root (readlink: Permission denied) 2024-06-03T03:27:58.557 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31652 admin_soc ceph txt unknown /proc/31649/task/31652/exe (readlink: Permission denied) 2024-06-03T03:27:58.557 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31652 admin_soc ceph NOFD /proc/31649/task/31652/fd (opendir: Permission denied) 2024-06-03T03:27:58.558 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31653 signal_ha ceph cwd unknown /proc/31649/task/31653/cwd (readlink: Permission denied) 2024-06-03T03:27:58.558 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31653 signal_ha ceph rtd unknown /proc/31649/task/31653/root (readlink: Permission denied) 2024-06-03T03:27:58.558 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31653 signal_ha ceph txt unknown /proc/31649/task/31653/exe (readlink: Permission denied) 2024-06-03T03:27:58.558 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31653 signal_ha ceph NOFD /proc/31649/task/31653/fd (opendir: Permission denied) 2024-06-03T03:27:58.558 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31654 rocksdb:l ceph cwd unknown /proc/31649/task/31654/cwd (readlink: Permission denied) 2024-06-03T03:27:58.558 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31654 rocksdb:l ceph rtd unknown /proc/31649/task/31654/root (readlink: Permission denied) 2024-06-03T03:27:58.558 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31654 rocksdb:l ceph txt unknown /proc/31649/task/31654/exe (readlink: Permission denied) 2024-06-03T03:27:58.558 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31654 rocksdb:l ceph NOFD /proc/31649/task/31654/fd (opendir: Permission denied) 2024-06-03T03:27:58.558 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31655 rocksdb:h ceph cwd unknown /proc/31649/task/31655/cwd (readlink: Permission denied) 2024-06-03T03:27:58.558 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31655 rocksdb:h ceph rtd unknown /proc/31649/task/31655/root (readlink: Permission denied) 2024-06-03T03:27:58.558 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31655 rocksdb:h ceph txt unknown /proc/31649/task/31655/exe (readlink: Permission denied) 2024-06-03T03:27:58.558 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31655 rocksdb:h ceph NOFD /proc/31649/task/31655/fd (opendir: Permission denied) 2024-06-03T03:27:58.559 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31671 ceph-mon ceph cwd unknown /proc/31649/task/31671/cwd (readlink: Permission denied) 2024-06-03T03:27:58.559 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31671 ceph-mon ceph rtd unknown /proc/31649/task/31671/root (readlink: Permission denied) 2024-06-03T03:27:58.559 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31671 ceph-mon ceph txt unknown /proc/31649/task/31671/exe (readlink: Permission denied) 2024-06-03T03:27:58.559 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31671 ceph-mon ceph NOFD /proc/31649/task/31671/fd (opendir: Permission denied) 2024-06-03T03:27:58.559 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31672 fn_monsto ceph cwd unknown /proc/31649/task/31672/cwd (readlink: Permission denied) 2024-06-03T03:27:58.559 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31672 fn_monsto ceph rtd unknown /proc/31649/task/31672/root (readlink: Permission denied) 2024-06-03T03:27:58.559 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31672 fn_monsto ceph txt unknown /proc/31649/task/31672/exe (readlink: Permission denied) 2024-06-03T03:27:58.559 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31672 fn_monsto ceph NOFD /proc/31649/task/31672/fd (opendir: Permission denied) 2024-06-03T03:27:58.559 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31673 msgr-work ceph cwd unknown /proc/31649/task/31673/cwd (readlink: Permission denied) 2024-06-03T03:27:58.559 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31673 msgr-work ceph rtd unknown /proc/31649/task/31673/root (readlink: Permission denied) 2024-06-03T03:27:58.559 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31673 msgr-work ceph txt unknown /proc/31649/task/31673/exe (readlink: Permission denied) 2024-06-03T03:27:58.559 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31673 msgr-work ceph NOFD /proc/31649/task/31673/fd (opendir: Permission denied) 2024-06-03T03:27:58.559 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31674 msgr-work ceph cwd unknown /proc/31649/task/31674/cwd (readlink: Permission denied) 2024-06-03T03:27:58.559 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31674 msgr-work ceph rtd unknown /proc/31649/task/31674/root (readlink: Permission denied) 2024-06-03T03:27:58.559 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31674 msgr-work ceph txt unknown /proc/31649/task/31674/exe (readlink: Permission denied) 2024-06-03T03:27:58.560 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31674 msgr-work ceph NOFD /proc/31649/task/31674/fd (opendir: Permission denied) 2024-06-03T03:27:58.560 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31675 msgr-work ceph cwd unknown /proc/31649/task/31675/cwd (readlink: Permission denied) 2024-06-03T03:27:58.560 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31675 msgr-work ceph rtd unknown /proc/31649/task/31675/root (readlink: Permission denied) 2024-06-03T03:27:58.560 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31675 msgr-work ceph txt unknown /proc/31649/task/31675/exe (readlink: Permission denied) 2024-06-03T03:27:58.560 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31675 msgr-work ceph NOFD /proc/31649/task/31675/fd (opendir: Permission denied) 2024-06-03T03:27:58.560 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31676 OpHistory ceph cwd unknown /proc/31649/task/31676/cwd (readlink: Permission denied) 2024-06-03T03:27:58.561 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31676 OpHistory ceph rtd unknown /proc/31649/task/31676/root (readlink: Permission denied) 2024-06-03T03:27:58.561 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31676 OpHistory ceph txt unknown /proc/31649/task/31676/exe (readlink: Permission denied) 2024-06-03T03:27:58.561 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31676 OpHistory ceph NOFD /proc/31649/task/31676/fd (opendir: Permission denied) 2024-06-03T03:27:58.561 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31677 fin ceph cwd unknown /proc/31649/task/31677/cwd (readlink: Permission denied) 2024-06-03T03:27:58.561 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31677 fin ceph rtd unknown /proc/31649/task/31677/root (readlink: Permission denied) 2024-06-03T03:27:58.561 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31677 fin ceph txt unknown /proc/31649/task/31677/exe (readlink: Permission denied) 2024-06-03T03:27:58.561 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31677 fin ceph NOFD /proc/31649/task/31677/fd (opendir: Permission denied) 2024-06-03T03:27:58.561 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31678 safe_time ceph cwd unknown /proc/31649/task/31678/cwd (readlink: Permission denied) 2024-06-03T03:27:58.561 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31678 safe_time ceph rtd unknown /proc/31649/task/31678/root (readlink: Permission denied) 2024-06-03T03:27:58.561 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31678 safe_time ceph txt unknown /proc/31649/task/31678/exe (readlink: Permission denied) 2024-06-03T03:27:58.561 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31678 safe_time ceph NOFD /proc/31649/task/31678/fd (opendir: Permission denied) 2024-06-03T03:27:58.561 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31679 cpu_tp ceph cwd unknown /proc/31649/task/31679/cwd (readlink: Permission denied) 2024-06-03T03:27:58.562 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31679 cpu_tp ceph rtd unknown /proc/31649/task/31679/root (readlink: Permission denied) 2024-06-03T03:27:58.562 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31679 cpu_tp ceph txt unknown /proc/31649/task/31679/exe (readlink: Permission denied) 2024-06-03T03:27:58.562 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31679 cpu_tp ceph NOFD /proc/31649/task/31679/fd (opendir: Permission denied) 2024-06-03T03:27:58.562 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31680 cpu_tp ceph cwd unknown /proc/31649/task/31680/cwd (readlink: Permission denied) 2024-06-03T03:27:58.562 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31680 cpu_tp ceph rtd unknown /proc/31649/task/31680/root (readlink: Permission denied) 2024-06-03T03:27:58.562 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31680 cpu_tp ceph txt unknown /proc/31649/task/31680/exe (readlink: Permission denied) 2024-06-03T03:27:58.563 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31680 cpu_tp ceph NOFD /proc/31649/task/31680/fd (opendir: Permission denied) 2024-06-03T03:27:58.563 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31681 cpu_tp ceph cwd unknown /proc/31649/task/31681/cwd (readlink: Permission denied) 2024-06-03T03:27:58.563 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31681 cpu_tp ceph rtd unknown /proc/31649/task/31681/root (readlink: Permission denied) 2024-06-03T03:27:58.563 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31681 cpu_tp ceph txt unknown /proc/31649/task/31681/exe (readlink: Permission denied) 2024-06-03T03:27:58.563 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31681 cpu_tp ceph NOFD /proc/31649/task/31681/fd (opendir: Permission denied) 2024-06-03T03:27:58.564 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31682 cpu_tp ceph cwd unknown /proc/31649/task/31682/cwd (readlink: Permission denied) 2024-06-03T03:27:58.564 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31682 cpu_tp ceph rtd unknown /proc/31649/task/31682/root (readlink: Permission denied) 2024-06-03T03:27:58.564 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31682 cpu_tp ceph txt unknown /proc/31649/task/31682/exe (readlink: Permission denied) 2024-06-03T03:27:58.564 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31682 cpu_tp ceph NOFD /proc/31649/task/31682/fd (opendir: Permission denied) 2024-06-03T03:27:58.564 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31683 ms_dispat ceph cwd unknown /proc/31649/task/31683/cwd (readlink: Permission denied) 2024-06-03T03:27:58.564 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31683 ms_dispat ceph rtd unknown /proc/31649/task/31683/root (readlink: Permission denied) 2024-06-03T03:27:58.564 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31683 ms_dispat ceph txt unknown /proc/31649/task/31683/exe (readlink: Permission denied) 2024-06-03T03:27:58.565 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31683 ms_dispat ceph NOFD /proc/31649/task/31683/fd (opendir: Permission denied) 2024-06-03T03:27:58.565 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31684 ms_local ceph cwd unknown /proc/31649/task/31684/cwd (readlink: Permission denied) 2024-06-03T03:27:58.565 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31684 ms_local ceph rtd unknown /proc/31649/task/31684/root (readlink: Permission denied) 2024-06-03T03:27:58.565 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31684 ms_local ceph txt unknown /proc/31649/task/31684/exe (readlink: Permission denied) 2024-06-03T03:27:58.565 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31684 ms_local ceph NOFD /proc/31649/task/31684/fd (opendir: Permission denied) 2024-06-03T03:27:58.565 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31685 safe_time ceph cwd unknown /proc/31649/task/31685/cwd (readlink: Permission denied) 2024-06-03T03:27:58.565 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31685 safe_time ceph rtd unknown /proc/31649/task/31685/root (readlink: Permission denied) 2024-06-03T03:27:58.565 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31685 safe_time ceph txt unknown /proc/31649/task/31685/exe (readlink: Permission denied) 2024-06-03T03:27:58.565 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31685 safe_time ceph NOFD /proc/31649/task/31685/fd (opendir: Permission denied) 2024-06-03T03:27:58.565 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31686 ms_dispat ceph cwd unknown /proc/31649/task/31686/cwd (readlink: Permission denied) 2024-06-03T03:27:58.565 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31686 ms_dispat ceph rtd unknown /proc/31649/task/31686/root (readlink: Permission denied) 2024-06-03T03:27:58.565 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31686 ms_dispat ceph txt unknown /proc/31649/task/31686/exe (readlink: Permission denied) 2024-06-03T03:27:58.565 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31686 ms_dispat ceph NOFD /proc/31649/task/31686/fd (opendir: Permission denied) 2024-06-03T03:27:58.565 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31687 ms_local ceph cwd unknown /proc/31649/task/31687/cwd (readlink: Permission denied) 2024-06-03T03:27:58.566 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31687 ms_local ceph rtd unknown /proc/31649/task/31687/root (readlink: Permission denied) 2024-06-03T03:27:58.566 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31687 ms_local ceph txt unknown /proc/31649/task/31687/exe (readlink: Permission denied) 2024-06-03T03:27:58.566 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 31687 ms_local ceph NOFD /proc/31649/task/31687/fd (opendir: Permission denied) 2024-06-03T03:27:58.566 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 65899 rstore_co ceph cwd unknown /proc/31649/task/65899/cwd (readlink: Permission denied) 2024-06-03T03:27:58.566 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 65899 rstore_co ceph rtd unknown /proc/31649/task/65899/root (readlink: Permission denied) 2024-06-03T03:27:58.566 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 65899 rstore_co ceph txt unknown /proc/31649/task/65899/exe (readlink: Permission denied) 2024-06-03T03:27:58.566 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mon 31649 65899 rstore_co ceph NOFD /proc/31649/task/65899/fd (opendir: Permission denied) 2024-06-03T03:27:58.566 INFO:tasks.workunit.client.0.smithi117.stdout:conmon 31883 root cwd unknown /proc/31883/cwd (readlink: Permission denied) 2024-06-03T03:27:58.566 INFO:tasks.workunit.client.0.smithi117.stdout:conmon 31883 root rtd unknown /proc/31883/root (readlink: Permission denied) 2024-06-03T03:27:58.566 INFO:tasks.workunit.client.0.smithi117.stdout:conmon 31883 root txt unknown /proc/31883/exe (readlink: Permission denied) 2024-06-03T03:27:58.566 INFO:tasks.workunit.client.0.smithi117.stdout:conmon 31883 root NOFD /proc/31883/fd (opendir: Permission denied) 2024-06-03T03:27:58.566 INFO:tasks.workunit.client.0.smithi117.stdout:podman-in 31885 root cwd unknown /proc/31885/cwd (readlink: Permission denied) 2024-06-03T03:27:58.567 INFO:tasks.workunit.client.0.smithi117.stdout:podman-in 31885 root rtd unknown /proc/31885/root (readlink: Permission denied) 2024-06-03T03:27:58.567 INFO:tasks.workunit.client.0.smithi117.stdout:podman-in 31885 root txt unknown /proc/31885/exe (readlink: Permission denied) 2024-06-03T03:27:58.567 INFO:tasks.workunit.client.0.smithi117.stdout:podman-in 31885 root NOFD /proc/31885/fd (opendir: Permission denied) 2024-06-03T03:27:58.567 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 ceph cwd unknown /proc/31887/cwd (readlink: Permission denied) 2024-06-03T03:27:58.567 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 ceph rtd unknown /proc/31887/root (readlink: Permission denied) 2024-06-03T03:27:58.567 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 ceph txt unknown /proc/31887/exe (readlink: Permission denied) 2024-06-03T03:27:58.567 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 ceph NOFD /proc/31887/fd (opendir: Permission denied) 2024-06-03T03:27:58.567 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34248 log ceph cwd unknown /proc/31887/task/34248/cwd (readlink: Permission denied) 2024-06-03T03:27:58.567 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34248 log ceph rtd unknown /proc/31887/task/34248/root (readlink: Permission denied) 2024-06-03T03:27:58.567 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34248 log ceph txt unknown /proc/31887/task/34248/exe (readlink: Permission denied) 2024-06-03T03:27:58.567 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34248 log ceph NOFD /proc/31887/task/34248/fd (opendir: Permission denied) 2024-06-03T03:27:58.567 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34250 msgr-work ceph cwd unknown /proc/31887/task/34250/cwd (readlink: Permission denied) 2024-06-03T03:27:58.567 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34250 msgr-work ceph rtd unknown /proc/31887/task/34250/root (readlink: Permission denied) 2024-06-03T03:27:58.568 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34250 msgr-work ceph txt unknown /proc/31887/task/34250/exe (readlink: Permission denied) 2024-06-03T03:27:58.568 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34250 msgr-work ceph NOFD /proc/31887/task/34250/fd (opendir: Permission denied) 2024-06-03T03:27:58.568 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34251 msgr-work ceph cwd unknown /proc/31887/task/34251/cwd (readlink: Permission denied) 2024-06-03T03:27:58.568 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34251 msgr-work ceph rtd unknown /proc/31887/task/34251/root (readlink: Permission denied) 2024-06-03T03:27:58.568 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34251 msgr-work ceph txt unknown /proc/31887/task/34251/exe (readlink: Permission denied) 2024-06-03T03:27:58.568 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34251 msgr-work ceph NOFD /proc/31887/task/34251/fd (opendir: Permission denied) 2024-06-03T03:27:58.568 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34252 msgr-work ceph cwd unknown /proc/31887/task/34252/cwd (readlink: Permission denied) 2024-06-03T03:27:58.568 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34252 msgr-work ceph rtd unknown /proc/31887/task/34252/root (readlink: Permission denied) 2024-06-03T03:27:58.568 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34252 msgr-work ceph txt unknown /proc/31887/task/34252/exe (readlink: Permission denied) 2024-06-03T03:27:58.568 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34252 msgr-work ceph NOFD /proc/31887/task/34252/fd (opendir: Permission denied) 2024-06-03T03:27:58.568 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34256 service ceph cwd unknown /proc/31887/task/34256/cwd (readlink: Permission denied) 2024-06-03T03:27:58.569 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34256 service ceph rtd unknown /proc/31887/task/34256/root (readlink: Permission denied) 2024-06-03T03:27:58.569 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34256 service ceph txt unknown /proc/31887/task/34256/exe (readlink: Permission denied) 2024-06-03T03:27:58.569 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34256 service ceph NOFD /proc/31887/task/34256/fd (opendir: Permission denied) 2024-06-03T03:27:58.569 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34257 admin_soc ceph cwd unknown /proc/31887/task/34257/cwd (readlink: Permission denied) 2024-06-03T03:27:58.569 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34257 admin_soc ceph rtd unknown /proc/31887/task/34257/root (readlink: Permission denied) 2024-06-03T03:27:58.569 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34257 admin_soc ceph txt unknown /proc/31887/task/34257/exe (readlink: Permission denied) 2024-06-03T03:27:58.569 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34257 admin_soc ceph NOFD /proc/31887/task/34257/fd (opendir: Permission denied) 2024-06-03T03:27:58.569 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34258 ceph_time ceph cwd unknown /proc/31887/task/34258/cwd (readlink: Permission denied) 2024-06-03T03:27:58.569 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34258 ceph_time ceph rtd unknown /proc/31887/task/34258/root (readlink: Permission denied) 2024-06-03T03:27:58.569 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34258 ceph_time ceph txt unknown /proc/31887/task/34258/exe (readlink: Permission denied) 2024-06-03T03:27:58.570 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34258 ceph_time ceph NOFD /proc/31887/task/34258/fd (opendir: Permission denied) 2024-06-03T03:27:58.570 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34259 fn_anonym ceph cwd unknown /proc/31887/task/34259/cwd (readlink: Permission denied) 2024-06-03T03:27:58.570 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34259 fn_anonym ceph rtd unknown /proc/31887/task/34259/root (readlink: Permission denied) 2024-06-03T03:27:58.570 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34259 fn_anonym ceph txt unknown /proc/31887/task/34259/exe (readlink: Permission denied) 2024-06-03T03:27:58.570 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34259 fn_anonym ceph NOFD /proc/31887/task/34259/fd (opendir: Permission denied) 2024-06-03T03:27:58.570 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34260 signal_ha ceph cwd unknown /proc/31887/task/34260/cwd (readlink: Permission denied) 2024-06-03T03:27:58.570 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34260 signal_ha ceph rtd unknown /proc/31887/task/34260/root (readlink: Permission denied) 2024-06-03T03:27:58.570 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34260 signal_ha ceph txt unknown /proc/31887/task/34260/exe (readlink: Permission denied) 2024-06-03T03:27:58.570 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34260 signal_ha ceph NOFD /proc/31887/task/34260/fd (opendir: Permission denied) 2024-06-03T03:27:58.570 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34261 mgrsb-fin ceph cwd unknown /proc/31887/task/34261/cwd (readlink: Permission denied) 2024-06-03T03:27:58.570 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34261 mgrsb-fin ceph rtd unknown /proc/31887/task/34261/root (readlink: Permission denied) 2024-06-03T03:27:58.570 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34261 mgrsb-fin ceph txt unknown /proc/31887/task/34261/exe (readlink: Permission denied) 2024-06-03T03:27:58.571 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34261 mgrsb-fin ceph NOFD /proc/31887/task/34261/fd (opendir: Permission denied) 2024-06-03T03:27:58.571 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34262 ms_dispat ceph cwd unknown /proc/31887/task/34262/cwd (readlink: Permission denied) 2024-06-03T03:27:58.571 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34262 ms_dispat ceph rtd unknown /proc/31887/task/34262/root (readlink: Permission denied) 2024-06-03T03:27:58.571 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34262 ms_dispat ceph txt unknown /proc/31887/task/34262/exe (readlink: Permission denied) 2024-06-03T03:27:58.571 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34262 ms_dispat ceph NOFD /proc/31887/task/34262/fd (opendir: Permission denied) 2024-06-03T03:27:58.571 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34263 ms_local ceph cwd unknown /proc/31887/task/34263/cwd (readlink: Permission denied) 2024-06-03T03:27:58.571 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34263 ms_local ceph rtd unknown /proc/31887/task/34263/root (readlink: Permission denied) 2024-06-03T03:27:58.571 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34263 ms_local ceph txt unknown /proc/31887/task/34263/exe (readlink: Permission denied) 2024-06-03T03:27:58.571 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34263 ms_local ceph NOFD /proc/31887/task/34263/fd (opendir: Permission denied) 2024-06-03T03:27:58.571 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34264 io_contex ceph cwd unknown /proc/31887/task/34264/cwd (readlink: Permission denied) 2024-06-03T03:27:58.571 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34264 io_contex ceph rtd unknown /proc/31887/task/34264/root (readlink: Permission denied) 2024-06-03T03:27:58.571 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34264 io_contex ceph txt unknown /proc/31887/task/34264/exe (readlink: Permission denied) 2024-06-03T03:27:58.571 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34264 io_contex ceph NOFD /proc/31887/task/34264/fd (opendir: Permission denied) 2024-06-03T03:27:58.572 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34265 io_contex ceph cwd unknown /proc/31887/task/34265/cwd (readlink: Permission denied) 2024-06-03T03:27:58.572 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34265 io_contex ceph rtd unknown /proc/31887/task/34265/root (readlink: Permission denied) 2024-06-03T03:27:58.572 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34265 io_contex ceph txt unknown /proc/31887/task/34265/exe (readlink: Permission denied) 2024-06-03T03:27:58.572 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34265 io_contex ceph NOFD /proc/31887/task/34265/fd (opendir: Permission denied) 2024-06-03T03:27:58.572 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34266 safe_time ceph cwd unknown /proc/31887/task/34266/cwd (readlink: Permission denied) 2024-06-03T03:27:58.572 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34266 safe_time ceph rtd unknown /proc/31887/task/34266/root (readlink: Permission denied) 2024-06-03T03:27:58.572 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34266 safe_time ceph txt unknown /proc/31887/task/34266/exe (readlink: Permission denied) 2024-06-03T03:27:58.572 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34266 safe_time ceph NOFD /proc/31887/task/34266/fd (opendir: Permission denied) 2024-06-03T03:27:58.572 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34267 safe_time ceph cwd unknown /proc/31887/task/34267/cwd (readlink: Permission denied) 2024-06-03T03:27:58.572 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34267 safe_time ceph rtd unknown /proc/31887/task/34267/root (readlink: Permission denied) 2024-06-03T03:27:58.572 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34267 safe_time ceph txt unknown /proc/31887/task/34267/exe (readlink: Permission denied) 2024-06-03T03:27:58.572 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34267 safe_time ceph NOFD /proc/31887/task/34267/fd (opendir: Permission denied) 2024-06-03T03:27:58.572 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34268 safe_time ceph cwd unknown /proc/31887/task/34268/cwd (readlink: Permission denied) 2024-06-03T03:27:58.572 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34268 safe_time ceph rtd unknown /proc/31887/task/34268/root (readlink: Permission denied) 2024-06-03T03:27:58.572 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34268 safe_time ceph txt unknown /proc/31887/task/34268/exe (readlink: Permission denied) 2024-06-03T03:27:58.573 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34268 safe_time ceph NOFD /proc/31887/task/34268/fd (opendir: Permission denied) 2024-06-03T03:27:58.573 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34269 fn_anonym ceph cwd unknown /proc/31887/task/34269/cwd (readlink: Permission denied) 2024-06-03T03:27:58.573 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34269 fn_anonym ceph rtd unknown /proc/31887/task/34269/root (readlink: Permission denied) 2024-06-03T03:27:58.573 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34269 fn_anonym ceph txt unknown /proc/31887/task/34269/exe (readlink: Permission denied) 2024-06-03T03:27:58.573 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34269 fn_anonym ceph NOFD /proc/31887/task/34269/fd (opendir: Permission denied) 2024-06-03T03:27:58.573 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34270 flusher ceph cwd unknown /proc/31887/task/34270/cwd (readlink: Permission denied) 2024-06-03T03:27:58.573 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34270 flusher ceph rtd unknown /proc/31887/task/34270/root (readlink: Permission denied) 2024-06-03T03:27:58.573 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34270 flusher ceph txt unknown /proc/31887/task/34270/exe (readlink: Permission denied) 2024-06-03T03:27:58.573 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34270 flusher ceph NOFD /proc/31887/task/34270/fd (opendir: Permission denied) 2024-06-03T03:27:58.573 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34271 safe_time ceph cwd unknown /proc/31887/task/34271/cwd (readlink: Permission denied) 2024-06-03T03:27:58.573 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34271 safe_time ceph rtd unknown /proc/31887/task/34271/root (readlink: Permission denied) 2024-06-03T03:27:58.573 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34271 safe_time ceph txt unknown /proc/31887/task/34271/exe (readlink: Permission denied) 2024-06-03T03:27:58.573 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34271 safe_time ceph NOFD /proc/31887/task/34271/fd (opendir: Permission denied) 2024-06-03T03:27:58.573 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34445 mgr-fin ceph cwd unknown /proc/31887/task/34445/cwd (readlink: Permission denied) 2024-06-03T03:27:58.573 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34445 mgr-fin ceph rtd unknown /proc/31887/task/34445/root (readlink: Permission denied) 2024-06-03T03:27:58.574 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34445 mgr-fin ceph txt unknown /proc/31887/task/34445/exe (readlink: Permission denied) 2024-06-03T03:27:58.574 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34445 mgr-fin ceph NOFD /proc/31887/task/34445/fd (opendir: Permission denied) 2024-06-03T03:27:58.574 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34446 ms_dispat ceph cwd unknown /proc/31887/task/34446/cwd (readlink: Permission denied) 2024-06-03T03:27:58.574 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34446 ms_dispat ceph rtd unknown /proc/31887/task/34446/root (readlink: Permission denied) 2024-06-03T03:27:58.574 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34446 ms_dispat ceph txt unknown /proc/31887/task/34446/exe (readlink: Permission denied) 2024-06-03T03:27:58.574 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34446 ms_dispat ceph NOFD /proc/31887/task/34446/fd (opendir: Permission denied) 2024-06-03T03:27:58.574 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34447 ms_local ceph cwd unknown /proc/31887/task/34447/cwd (readlink: Permission denied) 2024-06-03T03:27:58.574 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34447 ms_local ceph rtd unknown /proc/31887/task/34447/root (readlink: Permission denied) 2024-06-03T03:27:58.574 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34447 ms_local ceph txt unknown /proc/31887/task/34447/exe (readlink: Permission denied) 2024-06-03T03:27:58.574 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34447 ms_local ceph NOFD /proc/31887/task/34447/fd (opendir: Permission denied) 2024-06-03T03:27:58.574 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34448 safe_time ceph cwd unknown /proc/31887/task/34448/cwd (readlink: Permission denied) 2024-06-03T03:27:58.575 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34448 safe_time ceph rtd unknown /proc/31887/task/34448/root (readlink: Permission denied) 2024-06-03T03:27:58.575 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34448 safe_time ceph txt unknown /proc/31887/task/34448/exe (readlink: Permission denied) 2024-06-03T03:27:58.575 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34448 safe_time ceph NOFD /proc/31887/task/34448/fd (opendir: Permission denied) 2024-06-03T03:27:58.575 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34449 cmdfin ceph cwd unknown /proc/31887/task/34449/cwd (readlink: Permission denied) 2024-06-03T03:27:58.575 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34449 cmdfin ceph rtd unknown /proc/31887/task/34449/root (readlink: Permission denied) 2024-06-03T03:27:58.575 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34449 cmdfin ceph txt unknown /proc/31887/task/34449/exe (readlink: Permission denied) 2024-06-03T03:27:58.575 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34449 cmdfin ceph NOFD /proc/31887/task/34449/fd (opendir: Permission denied) 2024-06-03T03:27:58.575 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34453 io_contex ceph cwd unknown /proc/31887/task/34453/cwd (readlink: Permission denied) 2024-06-03T03:27:58.575 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34453 io_contex ceph rtd unknown /proc/31887/task/34453/root (readlink: Permission denied) 2024-06-03T03:27:58.575 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34453 io_contex ceph txt unknown /proc/31887/task/34453/exe (readlink: Permission denied) 2024-06-03T03:27:58.575 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34453 io_contex ceph NOFD /proc/31887/task/34453/fd (opendir: Permission denied) 2024-06-03T03:27:58.575 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34454 io_contex ceph cwd unknown /proc/31887/task/34454/cwd (readlink: Permission denied) 2024-06-03T03:27:58.576 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34454 io_contex ceph rtd unknown /proc/31887/task/34454/root (readlink: Permission denied) 2024-06-03T03:27:58.576 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34454 io_contex ceph txt unknown /proc/31887/task/34454/exe (readlink: Permission denied) 2024-06-03T03:27:58.576 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34454 io_contex ceph NOFD /proc/31887/task/34454/fd (opendir: Permission denied) 2024-06-03T03:27:58.576 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34455 ceph_time ceph cwd unknown /proc/31887/task/34455/cwd (readlink: Permission denied) 2024-06-03T03:27:58.576 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34455 ceph_time ceph rtd unknown /proc/31887/task/34455/root (readlink: Permission denied) 2024-06-03T03:27:58.576 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34455 ceph_time ceph txt unknown /proc/31887/task/34455/exe (readlink: Permission denied) 2024-06-03T03:27:58.576 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34455 ceph_time ceph NOFD /proc/31887/task/34455/fd (opendir: Permission denied) 2024-06-03T03:27:58.576 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34456 ms_dispat ceph cwd unknown /proc/31887/task/34456/cwd (readlink: Permission denied) 2024-06-03T03:27:58.576 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34456 ms_dispat ceph rtd unknown /proc/31887/task/34456/root (readlink: Permission denied) 2024-06-03T03:27:58.576 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34456 ms_dispat ceph txt unknown /proc/31887/task/34456/exe (readlink: Permission denied) 2024-06-03T03:27:58.576 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34456 ms_dispat ceph NOFD /proc/31887/task/34456/fd (opendir: Permission denied) 2024-06-03T03:27:58.576 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34457 ms_local ceph cwd unknown /proc/31887/task/34457/cwd (readlink: Permission denied) 2024-06-03T03:27:58.577 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34457 ms_local ceph rtd unknown /proc/31887/task/34457/root (readlink: Permission denied) 2024-06-03T03:27:58.577 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34457 ms_local ceph txt unknown /proc/31887/task/34457/exe (readlink: Permission denied) 2024-06-03T03:27:58.577 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34457 ms_local ceph NOFD /proc/31887/task/34457/fd (opendir: Permission denied) 2024-06-03T03:27:58.577 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34458 safe_time ceph cwd unknown /proc/31887/task/34458/cwd (readlink: Permission denied) 2024-06-03T03:27:58.577 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34458 safe_time ceph rtd unknown /proc/31887/task/34458/root (readlink: Permission denied) 2024-06-03T03:27:58.577 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34458 safe_time ceph txt unknown /proc/31887/task/34458/exe (readlink: Permission denied) 2024-06-03T03:27:58.577 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34458 safe_time ceph NOFD /proc/31887/task/34458/fd (opendir: Permission denied) 2024-06-03T03:27:58.577 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34459 safe_time ceph cwd unknown /proc/31887/task/34459/cwd (readlink: Permission denied) 2024-06-03T03:27:58.577 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34459 safe_time ceph rtd unknown /proc/31887/task/34459/root (readlink: Permission denied) 2024-06-03T03:27:58.577 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34459 safe_time ceph txt unknown /proc/31887/task/34459/exe (readlink: Permission denied) 2024-06-03T03:27:58.577 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34459 safe_time ceph NOFD /proc/31887/task/34459/fd (opendir: Permission denied) 2024-06-03T03:27:58.577 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34460 balancer ceph cwd unknown /proc/31887/task/34460/cwd (readlink: Permission denied) 2024-06-03T03:27:58.577 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34460 balancer ceph rtd unknown /proc/31887/task/34460/root (readlink: Permission denied) 2024-06-03T03:27:58.577 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34460 balancer ceph txt unknown /proc/31887/task/34460/exe (readlink: Permission denied) 2024-06-03T03:27:58.578 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34460 balancer ceph NOFD /proc/31887/task/34460/fd (opendir: Permission denied) 2024-06-03T03:27:58.578 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34461 m-fin-bal ceph cwd unknown /proc/31887/task/34461/cwd (readlink: Permission denied) 2024-06-03T03:27:58.578 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34461 m-fin-bal ceph rtd unknown /proc/31887/task/34461/root (readlink: Permission denied) 2024-06-03T03:27:58.578 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34461 m-fin-bal ceph txt unknown /proc/31887/task/34461/exe (readlink: Permission denied) 2024-06-03T03:27:58.578 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34461 m-fin-bal ceph NOFD /proc/31887/task/34461/fd (opendir: Permission denied) 2024-06-03T03:27:58.578 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34463 mgr-fin ceph cwd unknown /proc/31887/task/34463/cwd (readlink: Permission denied) 2024-06-03T03:27:58.578 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34463 mgr-fin ceph rtd unknown /proc/31887/task/34463/root (readlink: Permission denied) 2024-06-03T03:27:58.578 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34463 mgr-fin ceph txt unknown /proc/31887/task/34463/exe (readlink: Permission denied) 2024-06-03T03:27:58.578 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34463 mgr-fin ceph NOFD /proc/31887/task/34463/fd (opendir: Permission denied) 2024-06-03T03:27:58.578 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34464 mgr-fin ceph cwd unknown /proc/31887/task/34464/cwd (readlink: Permission denied) 2024-06-03T03:27:58.578 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34464 mgr-fin ceph rtd unknown /proc/31887/task/34464/root (readlink: Permission denied) 2024-06-03T03:27:58.579 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34464 mgr-fin ceph txt unknown /proc/31887/task/34464/exe (readlink: Permission denied) 2024-06-03T03:27:58.579 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34464 mgr-fin ceph NOFD /proc/31887/task/34464/fd (opendir: Permission denied) 2024-06-03T03:27:58.579 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34465 mgr-fin ceph cwd unknown /proc/31887/task/34465/cwd (readlink: Permission denied) 2024-06-03T03:27:58.579 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34465 mgr-fin ceph rtd unknown /proc/31887/task/34465/root (readlink: Permission denied) 2024-06-03T03:27:58.579 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34465 mgr-fin ceph txt unknown /proc/31887/task/34465/exe (readlink: Permission denied) 2024-06-03T03:27:58.579 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34465 mgr-fin ceph NOFD /proc/31887/task/34465/fd (opendir: Permission denied) 2024-06-03T03:27:58.579 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34466 mgr-fin ceph cwd unknown /proc/31887/task/34466/cwd (readlink: Permission denied) 2024-06-03T03:27:58.579 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34466 mgr-fin ceph rtd unknown /proc/31887/task/34466/root (readlink: Permission denied) 2024-06-03T03:27:58.579 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34466 mgr-fin ceph txt unknown /proc/31887/task/34466/exe (readlink: Permission denied) 2024-06-03T03:27:58.579 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34466 mgr-fin ceph NOFD /proc/31887/task/34466/fd (opendir: Permission denied) 2024-06-03T03:27:58.579 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34467 mgr-fin ceph cwd unknown /proc/31887/task/34467/cwd (readlink: Permission denied) 2024-06-03T03:27:58.579 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34467 mgr-fin ceph rtd unknown /proc/31887/task/34467/root (readlink: Permission denied) 2024-06-03T03:27:58.579 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34467 mgr-fin ceph txt unknown /proc/31887/task/34467/exe (readlink: Permission denied) 2024-06-03T03:27:58.580 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34467 mgr-fin ceph NOFD /proc/31887/task/34467/fd (opendir: Permission denied) 2024-06-03T03:27:58.580 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34468 mgr-fin ceph cwd unknown /proc/31887/task/34468/cwd (readlink: Permission denied) 2024-06-03T03:27:58.580 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34468 mgr-fin ceph rtd unknown /proc/31887/task/34468/root (readlink: Permission denied) 2024-06-03T03:27:58.580 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34468 mgr-fin ceph txt unknown /proc/31887/task/34468/exe (readlink: Permission denied) 2024-06-03T03:27:58.580 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34468 mgr-fin ceph NOFD /proc/31887/task/34468/fd (opendir: Permission denied) 2024-06-03T03:27:58.580 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34469 mgr-fin ceph cwd unknown /proc/31887/task/34469/cwd (readlink: Permission denied) 2024-06-03T03:27:58.580 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34469 mgr-fin ceph rtd unknown /proc/31887/task/34469/root (readlink: Permission denied) 2024-06-03T03:27:58.580 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34469 mgr-fin ceph txt unknown /proc/31887/task/34469/exe (readlink: Permission denied) 2024-06-03T03:27:58.580 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34469 mgr-fin ceph NOFD /proc/31887/task/34469/fd (opendir: Permission denied) 2024-06-03T03:27:58.580 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34470 mgr-fin ceph cwd unknown /proc/31887/task/34470/cwd (readlink: Permission denied) 2024-06-03T03:27:58.580 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34470 mgr-fin ceph rtd unknown /proc/31887/task/34470/root (readlink: Permission denied) 2024-06-03T03:27:58.580 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34470 mgr-fin ceph txt unknown /proc/31887/task/34470/exe (readlink: Permission denied) 2024-06-03T03:27:58.580 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34470 mgr-fin ceph NOFD /proc/31887/task/34470/fd (opendir: Permission denied) 2024-06-03T03:27:58.581 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34471 mgr-fin ceph cwd unknown /proc/31887/task/34471/cwd (readlink: Permission denied) 2024-06-03T03:27:58.581 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34471 mgr-fin ceph rtd unknown /proc/31887/task/34471/root (readlink: Permission denied) 2024-06-03T03:27:58.581 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34471 mgr-fin ceph txt unknown /proc/31887/task/34471/exe (readlink: Permission denied) 2024-06-03T03:27:58.581 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34471 mgr-fin ceph NOFD /proc/31887/task/34471/fd (opendir: Permission denied) 2024-06-03T03:27:58.581 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34472 mgr-fin ceph cwd unknown /proc/31887/task/34472/cwd (readlink: Permission denied) 2024-06-03T03:27:58.581 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34472 mgr-fin ceph rtd unknown /proc/31887/task/34472/root (readlink: Permission denied) 2024-06-03T03:27:58.581 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34472 mgr-fin ceph txt unknown /proc/31887/task/34472/exe (readlink: Permission denied) 2024-06-03T03:27:58.581 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34472 mgr-fin ceph NOFD /proc/31887/task/34472/fd (opendir: Permission denied) 2024-06-03T03:27:58.581 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34473 mgr-fin ceph cwd unknown /proc/31887/task/34473/cwd (readlink: Permission denied) 2024-06-03T03:27:58.581 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34473 mgr-fin ceph rtd unknown /proc/31887/task/34473/root (readlink: Permission denied) 2024-06-03T03:27:58.581 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34473 mgr-fin ceph txt unknown /proc/31887/task/34473/exe (readlink: Permission denied) 2024-06-03T03:27:58.581 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34473 mgr-fin ceph NOFD /proc/31887/task/34473/fd (opendir: Permission denied) 2024-06-03T03:27:58.581 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34474 mgr-fin ceph cwd unknown /proc/31887/task/34474/cwd (readlink: Permission denied) 2024-06-03T03:27:58.582 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34474 mgr-fin ceph rtd unknown /proc/31887/task/34474/root (readlink: Permission denied) 2024-06-03T03:27:58.582 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34474 mgr-fin ceph txt unknown /proc/31887/task/34474/exe (readlink: Permission denied) 2024-06-03T03:27:58.582 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34474 mgr-fin ceph NOFD /proc/31887/task/34474/fd (opendir: Permission denied) 2024-06-03T03:27:58.582 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34475 mgr-fin ceph cwd unknown /proc/31887/task/34475/cwd (readlink: Permission denied) 2024-06-03T03:27:58.582 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34475 mgr-fin ceph rtd unknown /proc/31887/task/34475/root (readlink: Permission denied) 2024-06-03T03:27:58.582 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34475 mgr-fin ceph txt unknown /proc/31887/task/34475/exe (readlink: Permission denied) 2024-06-03T03:27:58.582 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34475 mgr-fin ceph NOFD /proc/31887/task/34475/fd (opendir: Permission denied) 2024-06-03T03:27:58.582 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34476 mgr-fin ceph cwd unknown /proc/31887/task/34476/cwd (readlink: Permission denied) 2024-06-03T03:27:58.582 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34476 mgr-fin ceph rtd unknown /proc/31887/task/34476/root (readlink: Permission denied) 2024-06-03T03:27:58.582 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34476 mgr-fin ceph txt unknown /proc/31887/task/34476/exe (readlink: Permission denied) 2024-06-03T03:27:58.582 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34476 mgr-fin ceph NOFD /proc/31887/task/34476/fd (opendir: Permission denied) 2024-06-03T03:27:58.582 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34478 mgr-fin ceph cwd unknown /proc/31887/task/34478/cwd (readlink: Permission denied) 2024-06-03T03:27:58.583 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34478 mgr-fin ceph rtd unknown /proc/31887/task/34478/root (readlink: Permission denied) 2024-06-03T03:27:58.583 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34478 mgr-fin ceph txt unknown /proc/31887/task/34478/exe (readlink: Permission denied) 2024-06-03T03:27:58.583 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34478 mgr-fin ceph NOFD /proc/31887/task/34478/fd (opendir: Permission denied) 2024-06-03T03:27:58.583 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34479 mgr-fin ceph cwd unknown /proc/31887/task/34479/cwd (readlink: Permission denied) 2024-06-03T03:27:58.583 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34479 mgr-fin ceph rtd unknown /proc/31887/task/34479/root (readlink: Permission denied) 2024-06-03T03:27:58.583 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34479 mgr-fin ceph txt unknown /proc/31887/task/34479/exe (readlink: Permission denied) 2024-06-03T03:27:58.583 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34479 mgr-fin ceph NOFD /proc/31887/task/34479/fd (opendir: Permission denied) 2024-06-03T03:27:58.583 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34480 cephadm ceph cwd unknown /proc/31887/task/34480/cwd (readlink: Permission denied) 2024-06-03T03:27:58.583 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34480 cephadm ceph rtd unknown /proc/31887/task/34480/root (readlink: Permission denied) 2024-06-03T03:27:58.583 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34480 cephadm ceph txt unknown /proc/31887/task/34480/exe (readlink: Permission denied) 2024-06-03T03:27:58.584 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34480 cephadm ceph NOFD /proc/31887/task/34480/fd (opendir: Permission denied) 2024-06-03T03:27:58.584 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34481 m-fin-cep ceph cwd unknown /proc/31887/task/34481/cwd (readlink: Permission denied) 2024-06-03T03:27:58.584 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34481 m-fin-cep ceph rtd unknown /proc/31887/task/34481/root (readlink: Permission denied) 2024-06-03T03:27:58.584 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34481 m-fin-cep ceph txt unknown /proc/31887/task/34481/exe (readlink: Permission denied) 2024-06-03T03:27:58.584 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34481 m-fin-cep ceph NOFD /proc/31887/task/34481/fd (opendir: Permission denied) 2024-06-03T03:27:58.584 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34482 mgr-fin ceph cwd unknown /proc/31887/task/34482/cwd (readlink: Permission denied) 2024-06-03T03:27:58.584 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34482 mgr-fin ceph rtd unknown /proc/31887/task/34482/root (readlink: Permission denied) 2024-06-03T03:27:58.584 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34482 mgr-fin ceph txt unknown /proc/31887/task/34482/exe (readlink: Permission denied) 2024-06-03T03:27:58.584 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34482 mgr-fin ceph NOFD /proc/31887/task/34482/fd (opendir: Permission denied) 2024-06-03T03:27:58.584 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34483 mgr-fin ceph cwd unknown /proc/31887/task/34483/cwd (readlink: Permission denied) 2024-06-03T03:27:58.584 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34483 mgr-fin ceph rtd unknown /proc/31887/task/34483/root (readlink: Permission denied) 2024-06-03T03:27:58.584 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34483 mgr-fin ceph txt unknown /proc/31887/task/34483/exe (readlink: Permission denied) 2024-06-03T03:27:58.584 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34483 mgr-fin ceph NOFD /proc/31887/task/34483/fd (opendir: Permission denied) 2024-06-03T03:27:58.584 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34484 crash ceph cwd unknown /proc/31887/task/34484/cwd (readlink: Permission denied) 2024-06-03T03:27:58.585 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34484 crash ceph rtd unknown /proc/31887/task/34484/root (readlink: Permission denied) 2024-06-03T03:27:58.585 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34484 crash ceph txt unknown /proc/31887/task/34484/exe (readlink: Permission denied) 2024-06-03T03:27:58.585 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34484 crash ceph NOFD /proc/31887/task/34484/fd (opendir: Permission denied) 2024-06-03T03:27:58.585 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34485 m-fin-cra ceph cwd unknown /proc/31887/task/34485/cwd (readlink: Permission denied) 2024-06-03T03:27:58.585 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34485 m-fin-cra ceph rtd unknown /proc/31887/task/34485/root (readlink: Permission denied) 2024-06-03T03:27:58.585 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34485 m-fin-cra ceph txt unknown /proc/31887/task/34485/exe (readlink: Permission denied) 2024-06-03T03:27:58.585 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34485 m-fin-cra ceph NOFD /proc/31887/task/34485/fd (opendir: Permission denied) 2024-06-03T03:27:58.585 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34486 mgr-fin ceph cwd unknown /proc/31887/task/34486/cwd (readlink: Permission denied) 2024-06-03T03:27:58.585 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34486 mgr-fin ceph rtd unknown /proc/31887/task/34486/root (readlink: Permission denied) 2024-06-03T03:27:58.585 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34486 mgr-fin ceph txt unknown /proc/31887/task/34486/exe (readlink: Permission denied) 2024-06-03T03:27:58.585 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34486 mgr-fin ceph NOFD /proc/31887/task/34486/fd (opendir: Permission denied) 2024-06-03T03:27:58.585 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34487 dashboard ceph cwd unknown /proc/31887/task/34487/cwd (readlink: Permission denied) 2024-06-03T03:27:58.585 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34487 dashboard ceph rtd unknown /proc/31887/task/34487/root (readlink: Permission denied) 2024-06-03T03:27:58.585 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34487 dashboard ceph txt unknown /proc/31887/task/34487/exe (readlink: Permission denied) 2024-06-03T03:27:58.586 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34487 dashboard ceph NOFD /proc/31887/task/34487/fd (opendir: Permission denied) 2024-06-03T03:27:58.586 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34488 m-fin-das ceph cwd unknown /proc/31887/task/34488/cwd (readlink: Permission denied) 2024-06-03T03:27:58.586 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34488 m-fin-das ceph rtd unknown /proc/31887/task/34488/root (readlink: Permission denied) 2024-06-03T03:27:58.586 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34488 m-fin-das ceph txt unknown /proc/31887/task/34488/exe (readlink: Permission denied) 2024-06-03T03:27:58.586 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34488 m-fin-das ceph NOFD /proc/31887/task/34488/fd (opendir: Permission denied) 2024-06-03T03:27:58.586 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34490 cephadm ceph cwd unknown /proc/31887/task/34490/cwd (readlink: Permission denied) 2024-06-03T03:27:58.586 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34490 cephadm ceph rtd unknown /proc/31887/task/34490/root (readlink: Permission denied) 2024-06-03T03:27:58.586 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34490 cephadm ceph txt unknown /proc/31887/task/34490/exe (readlink: Permission denied) 2024-06-03T03:27:58.586 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34490 cephadm ceph NOFD /proc/31887/task/34490/fd (opendir: Permission denied) 2024-06-03T03:27:58.586 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34492 devicehea ceph cwd unknown /proc/31887/task/34492/cwd (readlink: Permission denied) 2024-06-03T03:27:58.586 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34492 devicehea ceph rtd unknown /proc/31887/task/34492/root (readlink: Permission denied) 2024-06-03T03:27:58.586 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34492 devicehea ceph txt unknown /proc/31887/task/34492/exe (readlink: Permission denied) 2024-06-03T03:27:58.587 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34492 devicehea ceph NOFD /proc/31887/task/34492/fd (opendir: Permission denied) 2024-06-03T03:27:58.587 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34493 m-fin-dev ceph cwd unknown /proc/31887/task/34493/cwd (readlink: Permission denied) 2024-06-03T03:27:58.587 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34493 m-fin-dev ceph rtd unknown /proc/31887/task/34493/root (readlink: Permission denied) 2024-06-03T03:27:58.587 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34493 m-fin-dev ceph txt unknown /proc/31887/task/34493/exe (readlink: Permission denied) 2024-06-03T03:27:58.587 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34493 m-fin-dev ceph NOFD /proc/31887/task/34493/fd (opendir: Permission denied) 2024-06-03T03:27:58.587 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34495 mgr-fin ceph cwd unknown /proc/31887/task/34495/cwd (readlink: Permission denied) 2024-06-03T03:27:58.587 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34495 mgr-fin ceph rtd unknown /proc/31887/task/34495/root (readlink: Permission denied) 2024-06-03T03:27:58.587 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34495 mgr-fin ceph txt unknown /proc/31887/task/34495/exe (readlink: Permission denied) 2024-06-03T03:27:58.587 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34495 mgr-fin ceph NOFD /proc/31887/task/34495/fd (opendir: Permission denied) 2024-06-03T03:27:58.587 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34496 m-fin-ios ceph cwd unknown /proc/31887/task/34496/cwd (readlink: Permission denied) 2024-06-03T03:27:58.587 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34496 m-fin-ios ceph rtd unknown /proc/31887/task/34496/root (readlink: Permission denied) 2024-06-03T03:27:58.587 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34496 m-fin-ios ceph txt unknown /proc/31887/task/34496/exe (readlink: Permission denied) 2024-06-03T03:27:58.587 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34496 m-fin-ios ceph NOFD /proc/31887/task/34496/fd (opendir: Permission denied) 2024-06-03T03:27:58.587 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34497 mgr-fin ceph cwd unknown /proc/31887/task/34497/cwd (readlink: Permission denied) 2024-06-03T03:27:58.588 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34497 mgr-fin ceph rtd unknown /proc/31887/task/34497/root (readlink: Permission denied) 2024-06-03T03:27:58.588 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34497 mgr-fin ceph txt unknown /proc/31887/task/34497/exe (readlink: Permission denied) 2024-06-03T03:27:58.588 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34497 mgr-fin ceph NOFD /proc/31887/task/34497/fd (opendir: Permission denied) 2024-06-03T03:27:58.588 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34498 mgr-fin ceph cwd unknown /proc/31887/task/34498/cwd (readlink: Permission denied) 2024-06-03T03:27:58.588 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34498 mgr-fin ceph rtd unknown /proc/31887/task/34498/root (readlink: Permission denied) 2024-06-03T03:27:58.588 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34498 mgr-fin ceph txt unknown /proc/31887/task/34498/exe (readlink: Permission denied) 2024-06-03T03:27:58.588 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34498 mgr-fin ceph NOFD /proc/31887/task/34498/fd (opendir: Permission denied) 2024-06-03T03:27:58.588 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34500 mgr-fin ceph cwd unknown /proc/31887/task/34500/cwd (readlink: Permission denied) 2024-06-03T03:27:58.588 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34500 mgr-fin ceph rtd unknown /proc/31887/task/34500/root (readlink: Permission denied) 2024-06-03T03:27:58.588 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34500 mgr-fin ceph txt unknown /proc/31887/task/34500/exe (readlink: Permission denied) 2024-06-03T03:27:58.588 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34500 mgr-fin ceph NOFD /proc/31887/task/34500/fd (opendir: Permission denied) 2024-06-03T03:27:58.588 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34501 m-fin-nfs ceph cwd unknown /proc/31887/task/34501/cwd (readlink: Permission denied) 2024-06-03T03:27:58.588 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34501 m-fin-nfs ceph rtd unknown /proc/31887/task/34501/root (readlink: Permission denied) 2024-06-03T03:27:58.589 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34501 m-fin-nfs ceph txt unknown /proc/31887/task/34501/exe (readlink: Permission denied) 2024-06-03T03:27:58.589 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34501 m-fin-nfs ceph NOFD /proc/31887/task/34501/fd (opendir: Permission denied) 2024-06-03T03:27:58.589 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34504 m-fin-orc ceph cwd unknown /proc/31887/task/34504/cwd (readlink: Permission denied) 2024-06-03T03:27:58.589 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34504 m-fin-orc ceph rtd unknown /proc/31887/task/34504/root (readlink: Permission denied) 2024-06-03T03:27:58.589 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34504 m-fin-orc ceph txt unknown /proc/31887/task/34504/exe (readlink: Permission denied) 2024-06-03T03:27:58.589 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34504 m-fin-orc ceph NOFD /proc/31887/task/34504/fd (opendir: Permission denied) 2024-06-03T03:27:58.589 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34505 mgr-fin ceph cwd unknown /proc/31887/task/34505/cwd (readlink: Permission denied) 2024-06-03T03:27:58.589 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34505 mgr-fin ceph rtd unknown /proc/31887/task/34505/root (readlink: Permission denied) 2024-06-03T03:27:58.589 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34505 mgr-fin ceph txt unknown /proc/31887/task/34505/exe (readlink: Permission denied) 2024-06-03T03:27:58.589 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34505 mgr-fin ceph NOFD /proc/31887/task/34505/fd (opendir: Permission denied) 2024-06-03T03:27:58.589 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34506 pg_autosc ceph cwd unknown /proc/31887/task/34506/cwd (readlink: Permission denied) 2024-06-03T03:27:58.589 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34506 pg_autosc ceph rtd unknown /proc/31887/task/34506/root (readlink: Permission denied) 2024-06-03T03:27:58.589 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34506 pg_autosc ceph txt unknown /proc/31887/task/34506/exe (readlink: Permission denied) 2024-06-03T03:27:58.589 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34506 pg_autosc ceph NOFD /proc/31887/task/34506/fd (opendir: Permission denied) 2024-06-03T03:27:58.590 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34507 m-fin-pg_ ceph cwd unknown /proc/31887/task/34507/cwd (readlink: Permission denied) 2024-06-03T03:27:58.590 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34507 m-fin-pg_ ceph rtd unknown /proc/31887/task/34507/root (readlink: Permission denied) 2024-06-03T03:27:58.590 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34507 m-fin-pg_ ceph txt unknown /proc/31887/task/34507/exe (readlink: Permission denied) 2024-06-03T03:27:58.590 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34507 m-fin-pg_ ceph NOFD /proc/31887/task/34507/fd (opendir: Permission denied) 2024-06-03T03:27:58.590 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34508 mgr-fin ceph cwd unknown /proc/31887/task/34508/cwd (readlink: Permission denied) 2024-06-03T03:27:58.590 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34508 mgr-fin ceph rtd unknown /proc/31887/task/34508/root (readlink: Permission denied) 2024-06-03T03:27:58.590 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34508 mgr-fin ceph txt unknown /proc/31887/task/34508/exe (readlink: Permission denied) 2024-06-03T03:27:58.590 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34508 mgr-fin ceph NOFD /proc/31887/task/34508/fd (opendir: Permission denied) 2024-06-03T03:27:58.590 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34509 progress ceph cwd unknown /proc/31887/task/34509/cwd (readlink: Permission denied) 2024-06-03T03:27:58.590 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34509 progress ceph rtd unknown /proc/31887/task/34509/root (readlink: Permission denied) 2024-06-03T03:27:58.590 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34509 progress ceph txt unknown /proc/31887/task/34509/exe (readlink: Permission denied) 2024-06-03T03:27:58.591 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34509 progress ceph NOFD /proc/31887/task/34509/fd (opendir: Permission denied) 2024-06-03T03:27:58.591 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34510 m-fin-pro ceph cwd unknown /proc/31887/task/34510/cwd (readlink: Permission denied) 2024-06-03T03:27:58.591 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34510 m-fin-pro ceph rtd unknown /proc/31887/task/34510/root (readlink: Permission denied) 2024-06-03T03:27:58.591 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34510 m-fin-pro ceph txt unknown /proc/31887/task/34510/exe (readlink: Permission denied) 2024-06-03T03:27:58.591 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34510 m-fin-pro ceph NOFD /proc/31887/task/34510/fd (opendir: Permission denied) 2024-06-03T03:27:58.591 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34511 mgr-fin ceph cwd unknown /proc/31887/task/34511/cwd (readlink: Permission denied) 2024-06-03T03:27:58.591 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34511 mgr-fin ceph rtd unknown /proc/31887/task/34511/root (readlink: Permission denied) 2024-06-03T03:27:58.591 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34511 mgr-fin ceph txt unknown /proc/31887/task/34511/exe (readlink: Permission denied) 2024-06-03T03:27:58.591 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34511 mgr-fin ceph NOFD /proc/31887/task/34511/fd (opendir: Permission denied) 2024-06-03T03:27:58.591 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34512 mgr-fin ceph cwd unknown /proc/31887/task/34512/cwd (readlink: Permission denied) 2024-06-03T03:27:58.591 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34512 mgr-fin ceph rtd unknown /proc/31887/task/34512/root (readlink: Permission denied) 2024-06-03T03:27:58.591 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34512 mgr-fin ceph txt unknown /proc/31887/task/34512/exe (readlink: Permission denied) 2024-06-03T03:27:58.591 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34512 mgr-fin ceph NOFD /proc/31887/task/34512/fd (opendir: Permission denied) 2024-06-03T03:27:58.592 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34516 mgr-fin ceph cwd unknown /proc/31887/task/34516/cwd (readlink: Permission denied) 2024-06-03T03:27:58.592 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34516 mgr-fin ceph rtd unknown /proc/31887/task/34516/root (readlink: Permission denied) 2024-06-03T03:27:58.592 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34516 mgr-fin ceph txt unknown /proc/31887/task/34516/exe (readlink: Permission denied) 2024-06-03T03:27:58.592 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34516 mgr-fin ceph NOFD /proc/31887/task/34516/fd (opendir: Permission denied) 2024-06-03T03:27:58.592 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34517 io_contex ceph cwd unknown /proc/31887/task/34517/cwd (readlink: Permission denied) 2024-06-03T03:27:58.592 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34517 io_contex ceph rtd unknown /proc/31887/task/34517/root (readlink: Permission denied) 2024-06-03T03:27:58.592 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34517 io_contex ceph txt unknown /proc/31887/task/34517/exe (readlink: Permission denied) 2024-06-03T03:27:58.592 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34517 io_contex ceph NOFD /proc/31887/task/34517/fd (opendir: Permission denied) 2024-06-03T03:27:58.592 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34518 io_contex ceph cwd unknown /proc/31887/task/34518/cwd (readlink: Permission denied) 2024-06-03T03:27:58.592 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34518 io_contex ceph rtd unknown /proc/31887/task/34518/root (readlink: Permission denied) 2024-06-03T03:27:58.592 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34518 io_contex ceph txt unknown /proc/31887/task/34518/exe (readlink: Permission denied) 2024-06-03T03:27:58.592 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34518 io_contex ceph NOFD /proc/31887/task/34518/fd (opendir: Permission denied) 2024-06-03T03:27:58.592 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34519 ceph_time ceph cwd unknown /proc/31887/task/34519/cwd (readlink: Permission denied) 2024-06-03T03:27:58.592 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34519 ceph_time ceph rtd unknown /proc/31887/task/34519/root (readlink: Permission denied) 2024-06-03T03:27:58.593 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34519 ceph_time ceph txt unknown /proc/31887/task/34519/exe (readlink: Permission denied) 2024-06-03T03:27:58.593 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34519 ceph_time ceph NOFD /proc/31887/task/34519/fd (opendir: Permission denied) 2024-06-03T03:27:58.593 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34520 ms_dispat ceph cwd unknown /proc/31887/task/34520/cwd (readlink: Permission denied) 2024-06-03T03:27:58.593 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34520 ms_dispat ceph rtd unknown /proc/31887/task/34520/root (readlink: Permission denied) 2024-06-03T03:27:58.593 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34520 ms_dispat ceph txt unknown /proc/31887/task/34520/exe (readlink: Permission denied) 2024-06-03T03:27:58.593 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34520 ms_dispat ceph NOFD /proc/31887/task/34520/fd (opendir: Permission denied) 2024-06-03T03:27:58.593 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34521 ms_local ceph cwd unknown /proc/31887/task/34521/cwd (readlink: Permission denied) 2024-06-03T03:27:58.593 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34521 ms_local ceph rtd unknown /proc/31887/task/34521/root (readlink: Permission denied) 2024-06-03T03:27:58.593 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34521 ms_local ceph txt unknown /proc/31887/task/34521/exe (readlink: Permission denied) 2024-06-03T03:27:58.593 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34521 ms_local ceph NOFD /proc/31887/task/34521/fd (opendir: Permission denied) 2024-06-03T03:27:58.593 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34522 safe_time ceph cwd unknown /proc/31887/task/34522/cwd (readlink: Permission denied) 2024-06-03T03:27:58.593 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34522 safe_time ceph rtd unknown /proc/31887/task/34522/root (readlink: Permission denied) 2024-06-03T03:27:58.593 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34522 safe_time ceph txt unknown /proc/31887/task/34522/exe (readlink: Permission denied) 2024-06-03T03:27:58.594 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34522 safe_time ceph NOFD /proc/31887/task/34522/fd (opendir: Permission denied) 2024-06-03T03:27:58.594 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34523 safe_time ceph cwd unknown /proc/31887/task/34523/cwd (readlink: Permission denied) 2024-06-03T03:27:58.594 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34523 safe_time ceph rtd unknown /proc/31887/task/34523/root (readlink: Permission denied) 2024-06-03T03:27:58.594 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34523 safe_time ceph txt unknown /proc/31887/task/34523/exe (readlink: Permission denied) 2024-06-03T03:27:58.594 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34523 safe_time ceph NOFD /proc/31887/task/34523/fd (opendir: Permission denied) 2024-06-03T03:27:58.594 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34524 mgr-fin ceph cwd unknown /proc/31887/task/34524/cwd (readlink: Permission denied) 2024-06-03T03:27:58.594 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34524 mgr-fin ceph rtd unknown /proc/31887/task/34524/root (readlink: Permission denied) 2024-06-03T03:27:58.594 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34524 mgr-fin ceph txt unknown /proc/31887/task/34524/exe (readlink: Permission denied) 2024-06-03T03:27:58.594 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34524 mgr-fin ceph NOFD /proc/31887/task/34524/fd (opendir: Permission denied) 2024-06-03T03:27:58.594 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34526 m-fin-rbd ceph cwd unknown /proc/31887/task/34526/cwd (readlink: Permission denied) 2024-06-03T03:27:58.594 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34526 m-fin-rbd ceph rtd unknown /proc/31887/task/34526/root (readlink: Permission denied) 2024-06-03T03:27:58.594 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34526 m-fin-rbd ceph txt unknown /proc/31887/task/34526/exe (readlink: Permission denied) 2024-06-03T03:27:58.595 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34526 m-fin-rbd ceph NOFD /proc/31887/task/34526/fd (opendir: Permission denied) 2024-06-03T03:27:58.595 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34527 restful ceph cwd unknown /proc/31887/task/34527/cwd (readlink: Permission denied) 2024-06-03T03:27:58.595 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34527 restful ceph rtd unknown /proc/31887/task/34527/root (readlink: Permission denied) 2024-06-03T03:27:58.595 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34527 restful ceph txt unknown /proc/31887/task/34527/exe (readlink: Permission denied) 2024-06-03T03:27:58.595 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34527 restful ceph NOFD /proc/31887/task/34527/fd (opendir: Permission denied) 2024-06-03T03:27:58.595 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34528 m-fin-res ceph cwd unknown /proc/31887/task/34528/cwd (readlink: Permission denied) 2024-06-03T03:27:58.595 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34528 m-fin-res ceph rtd unknown /proc/31887/task/34528/root (readlink: Permission denied) 2024-06-03T03:27:58.595 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34528 m-fin-res ceph txt unknown /proc/31887/task/34528/exe (readlink: Permission denied) 2024-06-03T03:27:58.595 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34528 m-fin-res ceph NOFD /proc/31887/task/34528/fd (opendir: Permission denied) 2024-06-03T03:27:58.595 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34530 m-fin-sta ceph cwd unknown /proc/31887/task/34530/cwd (readlink: Permission denied) 2024-06-03T03:27:58.595 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34530 m-fin-sta ceph rtd unknown /proc/31887/task/34530/root (readlink: Permission denied) 2024-06-03T03:27:58.595 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34530 m-fin-sta ceph txt unknown /proc/31887/task/34530/exe (readlink: Permission denied) 2024-06-03T03:27:58.595 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34530 m-fin-sta ceph NOFD /proc/31887/task/34530/fd (opendir: Permission denied) 2024-06-03T03:27:58.595 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34531 telemetry ceph cwd unknown /proc/31887/task/34531/cwd (readlink: Permission denied) 2024-06-03T03:27:58.596 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34531 telemetry ceph rtd unknown /proc/31887/task/34531/root (readlink: Permission denied) 2024-06-03T03:27:58.596 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34531 telemetry ceph txt unknown /proc/31887/task/34531/exe (readlink: Permission denied) 2024-06-03T03:27:58.596 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34531 telemetry ceph NOFD /proc/31887/task/34531/fd (opendir: Permission denied) 2024-06-03T03:27:58.596 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34532 m-fin-tel ceph cwd unknown /proc/31887/task/34532/cwd (readlink: Permission denied) 2024-06-03T03:27:58.596 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34532 m-fin-tel ceph rtd unknown /proc/31887/task/34532/root (readlink: Permission denied) 2024-06-03T03:27:58.596 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34532 m-fin-tel ceph txt unknown /proc/31887/task/34532/exe (readlink: Permission denied) 2024-06-03T03:27:58.596 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34532 m-fin-tel ceph NOFD /proc/31887/task/34532/fd (opendir: Permission denied) 2024-06-03T03:27:58.596 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34533 mgr-fin ceph cwd unknown /proc/31887/task/34533/cwd (readlink: Permission denied) 2024-06-03T03:27:58.596 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34533 mgr-fin ceph rtd unknown /proc/31887/task/34533/root (readlink: Permission denied) 2024-06-03T03:27:58.596 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34533 mgr-fin ceph txt unknown /proc/31887/task/34533/exe (readlink: Permission denied) 2024-06-03T03:27:58.596 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34533 mgr-fin ceph NOFD /proc/31887/task/34533/fd (opendir: Permission denied) 2024-06-03T03:27:58.596 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34534 mgr-fin ceph cwd unknown /proc/31887/task/34534/cwd (readlink: Permission denied) 2024-06-03T03:27:58.596 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34534 mgr-fin ceph rtd unknown /proc/31887/task/34534/root (readlink: Permission denied) 2024-06-03T03:27:58.597 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34534 mgr-fin ceph txt unknown /proc/31887/task/34534/exe (readlink: Permission denied) 2024-06-03T03:27:58.597 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34534 mgr-fin ceph NOFD /proc/31887/task/34534/fd (opendir: Permission denied) 2024-06-03T03:27:58.597 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34535 mgr-fin ceph cwd unknown /proc/31887/task/34535/cwd (readlink: Permission denied) 2024-06-03T03:27:58.597 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34535 mgr-fin ceph rtd unknown /proc/31887/task/34535/root (readlink: Permission denied) 2024-06-03T03:27:58.597 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34535 mgr-fin ceph txt unknown /proc/31887/task/34535/exe (readlink: Permission denied) 2024-06-03T03:27:58.597 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34535 mgr-fin ceph NOFD /proc/31887/task/34535/fd (opendir: Permission denied) 2024-06-03T03:27:58.597 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34536 mgr-fin ceph cwd unknown /proc/31887/task/34536/cwd (readlink: Permission denied) 2024-06-03T03:27:58.597 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34536 mgr-fin ceph rtd unknown /proc/31887/task/34536/root (readlink: Permission denied) 2024-06-03T03:27:58.597 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34536 mgr-fin ceph txt unknown /proc/31887/task/34536/exe (readlink: Permission denied) 2024-06-03T03:27:58.597 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34536 mgr-fin ceph NOFD /proc/31887/task/34536/fd (opendir: Permission denied) 2024-06-03T03:27:58.597 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34540 mgr-fin ceph cwd unknown /proc/31887/task/34540/cwd (readlink: Permission denied) 2024-06-03T03:27:58.597 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34540 mgr-fin ceph rtd unknown /proc/31887/task/34540/root (readlink: Permission denied) 2024-06-03T03:27:58.597 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34540 mgr-fin ceph txt unknown /proc/31887/task/34540/exe (readlink: Permission denied) 2024-06-03T03:27:58.597 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34540 mgr-fin ceph NOFD /proc/31887/task/34540/fd (opendir: Permission denied) 2024-06-03T03:27:58.598 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34541 io_contex ceph cwd unknown /proc/31887/task/34541/cwd (readlink: Permission denied) 2024-06-03T03:27:58.598 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34541 io_contex ceph rtd unknown /proc/31887/task/34541/root (readlink: Permission denied) 2024-06-03T03:27:58.598 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34541 io_contex ceph txt unknown /proc/31887/task/34541/exe (readlink: Permission denied) 2024-06-03T03:27:58.598 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34541 io_contex ceph NOFD /proc/31887/task/34541/fd (opendir: Permission denied) 2024-06-03T03:27:58.598 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34542 io_contex ceph cwd unknown /proc/31887/task/34542/cwd (readlink: Permission denied) 2024-06-03T03:27:58.598 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34542 io_contex ceph rtd unknown /proc/31887/task/34542/root (readlink: Permission denied) 2024-06-03T03:27:58.598 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34542 io_contex ceph txt unknown /proc/31887/task/34542/exe (readlink: Permission denied) 2024-06-03T03:27:58.598 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34542 io_contex ceph NOFD /proc/31887/task/34542/fd (opendir: Permission denied) 2024-06-03T03:27:58.598 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34543 ceph_time ceph cwd unknown /proc/31887/task/34543/cwd (readlink: Permission denied) 2024-06-03T03:27:58.598 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34543 ceph_time ceph rtd unknown /proc/31887/task/34543/root (readlink: Permission denied) 2024-06-03T03:27:58.598 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34543 ceph_time ceph txt unknown /proc/31887/task/34543/exe (readlink: Permission denied) 2024-06-03T03:27:58.598 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34543 ceph_time ceph NOFD /proc/31887/task/34543/fd (opendir: Permission denied) 2024-06-03T03:27:58.599 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34544 ms_dispat ceph cwd unknown /proc/31887/task/34544/cwd (readlink: Permission denied) 2024-06-03T03:27:58.599 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34544 ms_dispat ceph rtd unknown /proc/31887/task/34544/root (readlink: Permission denied) 2024-06-03T03:27:58.599 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34544 ms_dispat ceph txt unknown /proc/31887/task/34544/exe (readlink: Permission denied) 2024-06-03T03:27:58.599 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34544 ms_dispat ceph NOFD /proc/31887/task/34544/fd (opendir: Permission denied) 2024-06-03T03:27:58.599 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34545 ms_local ceph cwd unknown /proc/31887/task/34545/cwd (readlink: Permission denied) 2024-06-03T03:27:58.599 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34545 ms_local ceph rtd unknown /proc/31887/task/34545/root (readlink: Permission denied) 2024-06-03T03:27:58.599 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34545 ms_local ceph txt unknown /proc/31887/task/34545/exe (readlink: Permission denied) 2024-06-03T03:27:58.599 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34545 ms_local ceph NOFD /proc/31887/task/34545/fd (opendir: Permission denied) 2024-06-03T03:27:58.599 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34546 safe_time ceph cwd unknown /proc/31887/task/34546/cwd (readlink: Permission denied) 2024-06-03T03:27:58.599 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34546 safe_time ceph rtd unknown /proc/31887/task/34546/root (readlink: Permission denied) 2024-06-03T03:27:58.599 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34546 safe_time ceph txt unknown /proc/31887/task/34546/exe (readlink: Permission denied) 2024-06-03T03:27:58.599 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34546 safe_time ceph NOFD /proc/31887/task/34546/fd (opendir: Permission denied) 2024-06-03T03:27:58.599 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34548 safe_time ceph cwd unknown /proc/31887/task/34548/cwd (readlink: Permission denied) 2024-06-03T03:27:58.600 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34548 safe_time ceph rtd unknown /proc/31887/task/34548/root (readlink: Permission denied) 2024-06-03T03:27:58.600 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34548 safe_time ceph txt unknown /proc/31887/task/34548/exe (readlink: Permission denied) 2024-06-03T03:27:58.600 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34548 safe_time ceph NOFD /proc/31887/task/34548/fd (opendir: Permission denied) 2024-06-03T03:27:58.600 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34549 mgr-fin ceph cwd unknown /proc/31887/task/34549/cwd (readlink: Permission denied) 2024-06-03T03:27:58.600 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34549 mgr-fin ceph rtd unknown /proc/31887/task/34549/root (readlink: Permission denied) 2024-06-03T03:27:58.600 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34549 mgr-fin ceph txt unknown /proc/31887/task/34549/exe (readlink: Permission denied) 2024-06-03T03:27:58.600 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34549 mgr-fin ceph NOFD /proc/31887/task/34549/fd (opendir: Permission denied) 2024-06-03T03:27:58.600 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34550 mgr-fin ceph cwd unknown /proc/31887/task/34550/cwd (readlink: Permission denied) 2024-06-03T03:27:58.600 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34550 mgr-fin ceph rtd unknown /proc/31887/task/34550/root (readlink: Permission denied) 2024-06-03T03:27:58.600 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34550 mgr-fin ceph txt unknown /proc/31887/task/34550/exe (readlink: Permission denied) 2024-06-03T03:27:58.600 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34550 mgr-fin ceph NOFD /proc/31887/task/34550/fd (opendir: Permission denied) 2024-06-03T03:27:58.600 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34551 mgr-fin ceph cwd unknown /proc/31887/task/34551/cwd (readlink: Permission denied) 2024-06-03T03:27:58.600 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34551 mgr-fin ceph rtd unknown /proc/31887/task/34551/root (readlink: Permission denied) 2024-06-03T03:27:58.601 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34551 mgr-fin ceph txt unknown /proc/31887/task/34551/exe (readlink: Permission denied) 2024-06-03T03:27:58.601 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34551 mgr-fin ceph NOFD /proc/31887/task/34551/fd (opendir: Permission denied) 2024-06-03T03:27:58.601 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34552 mgr-fin ceph cwd unknown /proc/31887/task/34552/cwd (readlink: Permission denied) 2024-06-03T03:27:58.601 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34552 mgr-fin ceph rtd unknown /proc/31887/task/34552/root (readlink: Permission denied) 2024-06-03T03:27:58.601 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34552 mgr-fin ceph txt unknown /proc/31887/task/34552/exe (readlink: Permission denied) 2024-06-03T03:27:58.601 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34552 mgr-fin ceph NOFD /proc/31887/task/34552/fd (opendir: Permission denied) 2024-06-03T03:27:58.601 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34553 mgr-fin ceph cwd unknown /proc/31887/task/34553/cwd (readlink: Permission denied) 2024-06-03T03:27:58.601 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34553 mgr-fin ceph rtd unknown /proc/31887/task/34553/root (readlink: Permission denied) 2024-06-03T03:27:58.601 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34553 mgr-fin ceph txt unknown /proc/31887/task/34553/exe (readlink: Permission denied) 2024-06-03T03:27:58.601 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34553 mgr-fin ceph NOFD /proc/31887/task/34553/fd (opendir: Permission denied) 2024-06-03T03:27:58.601 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34554 mgr-fin ceph cwd unknown /proc/31887/task/34554/cwd (readlink: Permission denied) 2024-06-03T03:27:58.601 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34554 mgr-fin ceph rtd unknown /proc/31887/task/34554/root (readlink: Permission denied) 2024-06-03T03:27:58.602 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34554 mgr-fin ceph txt unknown /proc/31887/task/34554/exe (readlink: Permission denied) 2024-06-03T03:27:58.602 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34554 mgr-fin ceph NOFD /proc/31887/task/34554/fd (opendir: Permission denied) 2024-06-03T03:27:58.602 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34555 mgr-fin ceph cwd unknown /proc/31887/task/34555/cwd (readlink: Permission denied) 2024-06-03T03:27:58.602 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34555 mgr-fin ceph rtd unknown /proc/31887/task/34555/root (readlink: Permission denied) 2024-06-03T03:27:58.602 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34555 mgr-fin ceph txt unknown /proc/31887/task/34555/exe (readlink: Permission denied) 2024-06-03T03:27:58.602 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34555 mgr-fin ceph NOFD /proc/31887/task/34555/fd (opendir: Permission denied) 2024-06-03T03:27:58.602 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34556 mgr-fin ceph cwd unknown /proc/31887/task/34556/cwd (readlink: Permission denied) 2024-06-03T03:27:58.602 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34556 mgr-fin ceph rtd unknown /proc/31887/task/34556/root (readlink: Permission denied) 2024-06-03T03:27:58.602 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34556 mgr-fin ceph txt unknown /proc/31887/task/34556/exe (readlink: Permission denied) 2024-06-03T03:27:58.602 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34556 mgr-fin ceph NOFD /proc/31887/task/34556/fd (opendir: Permission denied) 2024-06-03T03:27:58.602 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34557 mgr-fin ceph cwd unknown /proc/31887/task/34557/cwd (readlink: Permission denied) 2024-06-03T03:27:58.603 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34557 mgr-fin ceph rtd unknown /proc/31887/task/34557/root (readlink: Permission denied) 2024-06-03T03:27:58.603 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34557 mgr-fin ceph txt unknown /proc/31887/task/34557/exe (readlink: Permission denied) 2024-06-03T03:27:58.603 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34557 mgr-fin ceph NOFD /proc/31887/task/34557/fd (opendir: Permission denied) 2024-06-03T03:27:58.603 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34558 mgr-fin ceph cwd unknown /proc/31887/task/34558/cwd (readlink: Permission denied) 2024-06-03T03:27:58.603 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34558 mgr-fin ceph rtd unknown /proc/31887/task/34558/root (readlink: Permission denied) 2024-06-03T03:27:58.603 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34558 mgr-fin ceph txt unknown /proc/31887/task/34558/exe (readlink: Permission denied) 2024-06-03T03:27:58.603 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34558 mgr-fin ceph NOFD /proc/31887/task/34558/fd (opendir: Permission denied) 2024-06-03T03:27:58.603 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34559 mgr-fin ceph cwd unknown /proc/31887/task/34559/cwd (readlink: Permission denied) 2024-06-03T03:27:58.603 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34559 mgr-fin ceph rtd unknown /proc/31887/task/34559/root (readlink: Permission denied) 2024-06-03T03:27:58.603 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34559 mgr-fin ceph txt unknown /proc/31887/task/34559/exe (readlink: Permission denied) 2024-06-03T03:27:58.603 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34559 mgr-fin ceph NOFD /proc/31887/task/34559/fd (opendir: Permission denied) 2024-06-03T03:27:58.603 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34560 mgr-fin ceph cwd unknown /proc/31887/task/34560/cwd (readlink: Permission denied) 2024-06-03T03:27:58.604 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34560 mgr-fin ceph rtd unknown /proc/31887/task/34560/root (readlink: Permission denied) 2024-06-03T03:27:58.604 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34560 mgr-fin ceph txt unknown /proc/31887/task/34560/exe (readlink: Permission denied) 2024-06-03T03:27:58.604 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34560 mgr-fin ceph NOFD /proc/31887/task/34560/fd (opendir: Permission denied) 2024-06-03T03:27:58.604 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34562 m-fin-vol ceph cwd unknown /proc/31887/task/34562/cwd (readlink: Permission denied) 2024-06-03T03:27:58.604 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34562 m-fin-vol ceph rtd unknown /proc/31887/task/34562/root (readlink: Permission denied) 2024-06-03T03:27:58.604 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34562 m-fin-vol ceph txt unknown /proc/31887/task/34562/exe (readlink: Permission denied) 2024-06-03T03:27:58.604 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34562 m-fin-vol ceph NOFD /proc/31887/task/34562/fd (opendir: Permission denied) 2024-06-03T03:27:58.604 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34581 mgr-fin ceph cwd unknown /proc/31887/task/34581/cwd (readlink: Permission denied) 2024-06-03T03:27:58.604 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34581 mgr-fin ceph rtd unknown /proc/31887/task/34581/root (readlink: Permission denied) 2024-06-03T03:27:58.604 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34581 mgr-fin ceph txt unknown /proc/31887/task/34581/exe (readlink: Permission denied) 2024-06-03T03:27:58.604 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34581 mgr-fin ceph NOFD /proc/31887/task/34581/fd (opendir: Permission denied) 2024-06-03T03:27:58.604 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34583 mgr-fin ceph cwd unknown /proc/31887/task/34583/cwd (readlink: Permission denied) 2024-06-03T03:27:58.604 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34583 mgr-fin ceph rtd unknown /proc/31887/task/34583/root (readlink: Permission denied) 2024-06-03T03:27:58.604 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34583 mgr-fin ceph txt unknown /proc/31887/task/34583/exe (readlink: Permission denied) 2024-06-03T03:27:58.605 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34583 mgr-fin ceph NOFD /proc/31887/task/34583/fd (opendir: Permission denied) 2024-06-03T03:27:58.605 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34584 mgr-fin ceph cwd unknown /proc/31887/task/34584/cwd (readlink: Permission denied) 2024-06-03T03:27:58.605 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34584 mgr-fin ceph rtd unknown /proc/31887/task/34584/root (readlink: Permission denied) 2024-06-03T03:27:58.605 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34584 mgr-fin ceph txt unknown /proc/31887/task/34584/exe (readlink: Permission denied) 2024-06-03T03:27:58.605 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34584 mgr-fin ceph NOFD /proc/31887/task/34584/fd (opendir: Permission denied) 2024-06-03T03:27:58.605 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34585 mgr-fin ceph cwd unknown /proc/31887/task/34585/cwd (readlink: Permission denied) 2024-06-03T03:27:58.605 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34585 mgr-fin ceph rtd unknown /proc/31887/task/34585/root (readlink: Permission denied) 2024-06-03T03:27:58.605 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34585 mgr-fin ceph txt unknown /proc/31887/task/34585/exe (readlink: Permission denied) 2024-06-03T03:27:58.605 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34585 mgr-fin ceph NOFD /proc/31887/task/34585/fd (opendir: Permission denied) 2024-06-03T03:27:58.605 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34586 mgr-fin ceph cwd unknown /proc/31887/task/34586/cwd (readlink: Permission denied) 2024-06-03T03:27:58.605 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34586 mgr-fin ceph rtd unknown /proc/31887/task/34586/root (readlink: Permission denied) 2024-06-03T03:27:58.605 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34586 mgr-fin ceph txt unknown /proc/31887/task/34586/exe (readlink: Permission denied) 2024-06-03T03:27:58.605 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34586 mgr-fin ceph NOFD /proc/31887/task/34586/fd (opendir: Permission denied) 2024-06-03T03:27:58.605 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34587 mgr-fin ceph cwd unknown /proc/31887/task/34587/cwd (readlink: Permission denied) 2024-06-03T03:27:58.606 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34587 mgr-fin ceph rtd unknown /proc/31887/task/34587/root (readlink: Permission denied) 2024-06-03T03:27:58.606 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34587 mgr-fin ceph txt unknown /proc/31887/task/34587/exe (readlink: Permission denied) 2024-06-03T03:27:58.606 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34587 mgr-fin ceph NOFD /proc/31887/task/34587/fd (opendir: Permission denied) 2024-06-03T03:27:58.606 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34588 mgr-fin ceph cwd unknown /proc/31887/task/34588/cwd (readlink: Permission denied) 2024-06-03T03:27:58.606 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34588 mgr-fin ceph rtd unknown /proc/31887/task/34588/root (readlink: Permission denied) 2024-06-03T03:27:58.606 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34588 mgr-fin ceph txt unknown /proc/31887/task/34588/exe (readlink: Permission denied) 2024-06-03T03:27:58.606 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34588 mgr-fin ceph NOFD /proc/31887/task/34588/fd (opendir: Permission denied) 2024-06-03T03:27:58.606 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34589 mgr-fin ceph cwd unknown /proc/31887/task/34589/cwd (readlink: Permission denied) 2024-06-03T03:27:58.606 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34589 mgr-fin ceph rtd unknown /proc/31887/task/34589/root (readlink: Permission denied) 2024-06-03T03:27:58.606 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34589 mgr-fin ceph txt unknown /proc/31887/task/34589/exe (readlink: Permission denied) 2024-06-03T03:27:58.606 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34589 mgr-fin ceph NOFD /proc/31887/task/34589/fd (opendir: Permission denied) 2024-06-03T03:27:58.606 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34590 mgr-fin ceph cwd unknown /proc/31887/task/34590/cwd (readlink: Permission denied) 2024-06-03T03:27:58.606 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34590 mgr-fin ceph rtd unknown /proc/31887/task/34590/root (readlink: Permission denied) 2024-06-03T03:27:58.607 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34590 mgr-fin ceph txt unknown /proc/31887/task/34590/exe (readlink: Permission denied) 2024-06-03T03:27:58.607 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34590 mgr-fin ceph NOFD /proc/31887/task/34590/fd (opendir: Permission denied) 2024-06-03T03:27:58.607 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34591 mgr-fin ceph cwd unknown /proc/31887/task/34591/cwd (readlink: Permission denied) 2024-06-03T03:27:58.607 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34591 mgr-fin ceph rtd unknown /proc/31887/task/34591/root (readlink: Permission denied) 2024-06-03T03:27:58.607 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34591 mgr-fin ceph txt unknown /proc/31887/task/34591/exe (readlink: Permission denied) 2024-06-03T03:27:58.607 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34591 mgr-fin ceph NOFD /proc/31887/task/34591/fd (opendir: Permission denied) 2024-06-03T03:27:58.609 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34592 mgr-fin ceph cwd unknown /proc/31887/task/34592/cwd (readlink: Permission denied) 2024-06-03T03:27:58.609 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34592 mgr-fin ceph rtd unknown /proc/31887/task/34592/root (readlink: Permission denied) 2024-06-03T03:27:58.609 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34592 mgr-fin ceph txt unknown /proc/31887/task/34592/exe (readlink: Permission denied) 2024-06-03T03:27:58.609 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34592 mgr-fin ceph NOFD /proc/31887/task/34592/fd (opendir: Permission denied) 2024-06-03T03:27:58.609 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34646 mgr-fin ceph cwd unknown /proc/31887/task/34646/cwd (readlink: Permission denied) 2024-06-03T03:27:58.609 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34646 mgr-fin ceph rtd unknown /proc/31887/task/34646/root (readlink: Permission denied) 2024-06-03T03:27:58.609 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34646 mgr-fin ceph txt unknown /proc/31887/task/34646/exe (readlink: Permission denied) 2024-06-03T03:27:58.609 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34646 mgr-fin ceph NOFD /proc/31887/task/34646/fd (opendir: Permission denied) 2024-06-03T03:27:58.609 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34647 mgr-fin ceph cwd unknown /proc/31887/task/34647/cwd (readlink: Permission denied) 2024-06-03T03:27:58.609 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34647 mgr-fin ceph rtd unknown /proc/31887/task/34647/root (readlink: Permission denied) 2024-06-03T03:27:58.609 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34647 mgr-fin ceph txt unknown /proc/31887/task/34647/exe (readlink: Permission denied) 2024-06-03T03:27:58.610 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34647 mgr-fin ceph NOFD /proc/31887/task/34647/fd (opendir: Permission denied) 2024-06-03T03:27:58.610 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34648 mgr-fin ceph cwd unknown /proc/31887/task/34648/cwd (readlink: Permission denied) 2024-06-03T03:27:58.610 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34648 mgr-fin ceph rtd unknown /proc/31887/task/34648/root (readlink: Permission denied) 2024-06-03T03:27:58.610 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34648 mgr-fin ceph txt unknown /proc/31887/task/34648/exe (readlink: Permission denied) 2024-06-03T03:27:58.610 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34648 mgr-fin ceph NOFD /proc/31887/task/34648/fd (opendir: Permission denied) 2024-06-03T03:27:58.610 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34649 mgr-fin ceph cwd unknown /proc/31887/task/34649/cwd (readlink: Permission denied) 2024-06-03T03:27:58.610 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34649 mgr-fin ceph rtd unknown /proc/31887/task/34649/root (readlink: Permission denied) 2024-06-03T03:27:58.610 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34649 mgr-fin ceph txt unknown /proc/31887/task/34649/exe (readlink: Permission denied) 2024-06-03T03:27:58.610 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34649 mgr-fin ceph NOFD /proc/31887/task/34649/fd (opendir: Permission denied) 2024-06-03T03:27:58.610 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34650 mgr-fin ceph cwd unknown /proc/31887/task/34650/cwd (readlink: Permission denied) 2024-06-03T03:27:58.610 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34650 mgr-fin ceph rtd unknown /proc/31887/task/34650/root (readlink: Permission denied) 2024-06-03T03:27:58.610 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34650 mgr-fin ceph txt unknown /proc/31887/task/34650/exe (readlink: Permission denied) 2024-06-03T03:27:58.610 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34650 mgr-fin ceph NOFD /proc/31887/task/34650/fd (opendir: Permission denied) 2024-06-03T03:27:58.610 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34651 mgr-fin ceph cwd unknown /proc/31887/task/34651/cwd (readlink: Permission denied) 2024-06-03T03:27:58.611 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34651 mgr-fin ceph rtd unknown /proc/31887/task/34651/root (readlink: Permission denied) 2024-06-03T03:27:58.611 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34651 mgr-fin ceph txt unknown /proc/31887/task/34651/exe (readlink: Permission denied) 2024-06-03T03:27:58.611 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34651 mgr-fin ceph NOFD /proc/31887/task/34651/fd (opendir: Permission denied) 2024-06-03T03:27:58.611 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34652 mgr-fin ceph cwd unknown /proc/31887/task/34652/cwd (readlink: Permission denied) 2024-06-03T03:27:58.611 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34652 mgr-fin ceph rtd unknown /proc/31887/task/34652/root (readlink: Permission denied) 2024-06-03T03:27:58.611 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34652 mgr-fin ceph txt unknown /proc/31887/task/34652/exe (readlink: Permission denied) 2024-06-03T03:27:58.611 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34652 mgr-fin ceph NOFD /proc/31887/task/34652/fd (opendir: Permission denied) 2024-06-03T03:27:58.611 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34653 mgr-fin ceph cwd unknown /proc/31887/task/34653/cwd (readlink: Permission denied) 2024-06-03T03:27:58.611 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34653 mgr-fin ceph rtd unknown /proc/31887/task/34653/root (readlink: Permission denied) 2024-06-03T03:27:58.611 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34653 mgr-fin ceph txt unknown /proc/31887/task/34653/exe (readlink: Permission denied) 2024-06-03T03:27:58.611 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34653 mgr-fin ceph NOFD /proc/31887/task/34653/fd (opendir: Permission denied) 2024-06-03T03:27:58.611 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34654 mgr-fin ceph cwd unknown /proc/31887/task/34654/cwd (readlink: Permission denied) 2024-06-03T03:27:58.611 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34654 mgr-fin ceph rtd unknown /proc/31887/task/34654/root (readlink: Permission denied) 2024-06-03T03:27:58.611 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34654 mgr-fin ceph txt unknown /proc/31887/task/34654/exe (readlink: Permission denied) 2024-06-03T03:27:58.612 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34654 mgr-fin ceph NOFD /proc/31887/task/34654/fd (opendir: Permission denied) 2024-06-03T03:27:58.612 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34655 mgr-fin ceph cwd unknown /proc/31887/task/34655/cwd (readlink: Permission denied) 2024-06-03T03:27:58.612 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34655 mgr-fin ceph rtd unknown /proc/31887/task/34655/root (readlink: Permission denied) 2024-06-03T03:27:58.612 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34655 mgr-fin ceph txt unknown /proc/31887/task/34655/exe (readlink: Permission denied) 2024-06-03T03:27:58.612 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34655 mgr-fin ceph NOFD /proc/31887/task/34655/fd (opendir: Permission denied) 2024-06-03T03:27:58.612 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34656 mgr-fin ceph cwd unknown /proc/31887/task/34656/cwd (readlink: Permission denied) 2024-06-03T03:27:58.612 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34656 mgr-fin ceph rtd unknown /proc/31887/task/34656/root (readlink: Permission denied) 2024-06-03T03:27:58.612 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34656 mgr-fin ceph txt unknown /proc/31887/task/34656/exe (readlink: Permission denied) 2024-06-03T03:27:58.612 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 34656 mgr-fin ceph NOFD /proc/31887/task/34656/fd (opendir: Permission denied) 2024-06-03T03:27:58.612 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 35146 dashboard ceph cwd unknown /proc/31887/task/35146/cwd (readlink: Permission denied) 2024-06-03T03:27:58.612 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 35146 dashboard ceph rtd unknown /proc/31887/task/35146/root (readlink: Permission denied) 2024-06-03T03:27:58.612 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 35146 dashboard ceph txt unknown /proc/31887/task/35146/exe (readlink: Permission denied) 2024-06-03T03:27:58.612 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 35146 dashboard ceph NOFD /proc/31887/task/35146/fd (opendir: Permission denied) 2024-06-03T03:27:58.613 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 35147 dashboard ceph cwd unknown /proc/31887/task/35147/cwd (readlink: Permission denied) 2024-06-03T03:27:58.613 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 35147 dashboard ceph rtd unknown /proc/31887/task/35147/root (readlink: Permission denied) 2024-06-03T03:27:58.613 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 35147 dashboard ceph txt unknown /proc/31887/task/35147/exe (readlink: Permission denied) 2024-06-03T03:27:58.613 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 35147 dashboard ceph NOFD /proc/31887/task/35147/fd (opendir: Permission denied) 2024-06-03T03:27:58.613 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 35148 dashboard ceph cwd unknown /proc/31887/task/35148/cwd (readlink: Permission denied) 2024-06-03T03:27:58.613 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 35148 dashboard ceph rtd unknown /proc/31887/task/35148/root (readlink: Permission denied) 2024-06-03T03:27:58.613 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 35148 dashboard ceph txt unknown /proc/31887/task/35148/exe (readlink: Permission denied) 2024-06-03T03:27:58.613 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 35148 dashboard ceph NOFD /proc/31887/task/35148/fd (opendir: Permission denied) 2024-06-03T03:27:58.613 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 35149 dashboard ceph cwd unknown /proc/31887/task/35149/cwd (readlink: Permission denied) 2024-06-03T03:27:58.613 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 35149 dashboard ceph rtd unknown /proc/31887/task/35149/root (readlink: Permission denied) 2024-06-03T03:27:58.613 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 35149 dashboard ceph txt unknown /proc/31887/task/35149/exe (readlink: Permission denied) 2024-06-03T03:27:58.614 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 35149 dashboard ceph NOFD /proc/31887/task/35149/fd (opendir: Permission denied) 2024-06-03T03:27:58.614 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 35150 dashboard ceph cwd unknown /proc/31887/task/35150/cwd (readlink: Permission denied) 2024-06-03T03:27:58.614 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 35150 dashboard ceph rtd unknown /proc/31887/task/35150/root (readlink: Permission denied) 2024-06-03T03:27:58.614 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 35150 dashboard ceph txt unknown /proc/31887/task/35150/exe (readlink: Permission denied) 2024-06-03T03:27:58.614 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 35150 dashboard ceph NOFD /proc/31887/task/35150/fd (opendir: Permission denied) 2024-06-03T03:27:58.614 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 35151 dashboard ceph cwd unknown /proc/31887/task/35151/cwd (readlink: Permission denied) 2024-06-03T03:27:58.614 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 35151 dashboard ceph rtd unknown /proc/31887/task/35151/root (readlink: Permission denied) 2024-06-03T03:27:58.614 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 35151 dashboard ceph txt unknown /proc/31887/task/35151/exe (readlink: Permission denied) 2024-06-03T03:27:58.614 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 35151 dashboard ceph NOFD /proc/31887/task/35151/fd (opendir: Permission denied) 2024-06-03T03:27:58.614 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 35152 dashboard ceph cwd unknown /proc/31887/task/35152/cwd (readlink: Permission denied) 2024-06-03T03:27:58.614 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 35152 dashboard ceph rtd unknown /proc/31887/task/35152/root (readlink: Permission denied) 2024-06-03T03:27:58.615 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 35152 dashboard ceph txt unknown /proc/31887/task/35152/exe (readlink: Permission denied) 2024-06-03T03:27:58.615 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 35152 dashboard ceph NOFD /proc/31887/task/35152/fd (opendir: Permission denied) 2024-06-03T03:27:58.615 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 35153 dashboard ceph cwd unknown /proc/31887/task/35153/cwd (readlink: Permission denied) 2024-06-03T03:27:58.615 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 35153 dashboard ceph rtd unknown /proc/31887/task/35153/root (readlink: Permission denied) 2024-06-03T03:27:58.615 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 35153 dashboard ceph txt unknown /proc/31887/task/35153/exe (readlink: Permission denied) 2024-06-03T03:27:58.615 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 35153 dashboard ceph NOFD /proc/31887/task/35153/fd (opendir: Permission denied) 2024-06-03T03:27:58.615 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 35154 dashboard ceph cwd unknown /proc/31887/task/35154/cwd (readlink: Permission denied) 2024-06-03T03:27:58.615 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 35154 dashboard ceph rtd unknown /proc/31887/task/35154/root (readlink: Permission denied) 2024-06-03T03:27:58.615 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 35154 dashboard ceph txt unknown /proc/31887/task/35154/exe (readlink: Permission denied) 2024-06-03T03:27:58.615 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 35154 dashboard ceph NOFD /proc/31887/task/35154/fd (opendir: Permission denied) 2024-06-03T03:27:58.615 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 35155 dashboard ceph cwd unknown /proc/31887/task/35155/cwd (readlink: Permission denied) 2024-06-03T03:27:58.615 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 35155 dashboard ceph rtd unknown /proc/31887/task/35155/root (readlink: Permission denied) 2024-06-03T03:27:58.615 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 35155 dashboard ceph txt unknown /proc/31887/task/35155/exe (readlink: Permission denied) 2024-06-03T03:27:58.615 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 35155 dashboard ceph NOFD /proc/31887/task/35155/fd (opendir: Permission denied) 2024-06-03T03:27:58.616 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 35156 dashboard ceph cwd unknown /proc/31887/task/35156/cwd (readlink: Permission denied) 2024-06-03T03:27:58.616 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 35156 dashboard ceph rtd unknown /proc/31887/task/35156/root (readlink: Permission denied) 2024-06-03T03:27:58.616 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 35156 dashboard ceph txt unknown /proc/31887/task/35156/exe (readlink: Permission denied) 2024-06-03T03:27:58.616 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 35156 dashboard ceph NOFD /proc/31887/task/35156/fd (opendir: Permission denied) 2024-06-03T03:27:58.616 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 35158 dashboard ceph cwd unknown /proc/31887/task/35158/cwd (readlink: Permission denied) 2024-06-03T03:27:58.616 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 35158 dashboard ceph rtd unknown /proc/31887/task/35158/root (readlink: Permission denied) 2024-06-03T03:27:58.616 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 35158 dashboard ceph txt unknown /proc/31887/task/35158/exe (readlink: Permission denied) 2024-06-03T03:27:58.616 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 35158 dashboard ceph NOFD /proc/31887/task/35158/fd (opendir: Permission denied) 2024-06-03T03:27:58.616 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 39403 cephadm ceph cwd unknown /proc/31887/task/39403/cwd (readlink: Permission denied) 2024-06-03T03:27:58.616 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 39403 cephadm ceph rtd unknown /proc/31887/task/39403/root (readlink: Permission denied) 2024-06-03T03:27:58.616 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 39403 cephadm ceph txt unknown /proc/31887/task/39403/exe (readlink: Permission denied) 2024-06-03T03:27:58.616 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 39403 cephadm ceph NOFD /proc/31887/task/39403/fd (opendir: Permission denied) 2024-06-03T03:27:58.616 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 39404 cephadm ceph cwd unknown /proc/31887/task/39404/cwd (readlink: Permission denied) 2024-06-03T03:27:58.617 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 39404 cephadm ceph rtd unknown /proc/31887/task/39404/root (readlink: Permission denied) 2024-06-03T03:27:58.617 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 39404 cephadm ceph txt unknown /proc/31887/task/39404/exe (readlink: Permission denied) 2024-06-03T03:27:58.617 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 39404 cephadm ceph NOFD /proc/31887/task/39404/fd (opendir: Permission denied) 2024-06-03T03:27:58.617 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 39405 cephadm ceph cwd unknown /proc/31887/task/39405/cwd (readlink: Permission denied) 2024-06-03T03:27:58.617 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 39405 cephadm ceph rtd unknown /proc/31887/task/39405/root (readlink: Permission denied) 2024-06-03T03:27:58.617 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 39405 cephadm ceph txt unknown /proc/31887/task/39405/exe (readlink: Permission denied) 2024-06-03T03:27:58.617 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 39405 cephadm ceph NOFD /proc/31887/task/39405/fd (opendir: Permission denied) 2024-06-03T03:27:58.617 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 39406 cephadm ceph cwd unknown /proc/31887/task/39406/cwd (readlink: Permission denied) 2024-06-03T03:27:58.617 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 39406 cephadm ceph rtd unknown /proc/31887/task/39406/root (readlink: Permission denied) 2024-06-03T03:27:58.617 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 39406 cephadm ceph txt unknown /proc/31887/task/39406/exe (readlink: Permission denied) 2024-06-03T03:27:58.617 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 39406 cephadm ceph NOFD /proc/31887/task/39406/fd (opendir: Permission denied) 2024-06-03T03:27:58.617 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 39407 cephadm ceph cwd unknown /proc/31887/task/39407/cwd (readlink: Permission denied) 2024-06-03T03:27:58.617 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 39407 cephadm ceph rtd unknown /proc/31887/task/39407/root (readlink: Permission denied) 2024-06-03T03:27:58.617 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 39407 cephadm ceph txt unknown /proc/31887/task/39407/exe (readlink: Permission denied) 2024-06-03T03:27:58.618 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 39407 cephadm ceph NOFD /proc/31887/task/39407/fd (opendir: Permission denied) 2024-06-03T03:27:58.618 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 39408 cephadm ceph cwd unknown /proc/31887/task/39408/cwd (readlink: Permission denied) 2024-06-03T03:27:58.618 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 39408 cephadm ceph rtd unknown /proc/31887/task/39408/root (readlink: Permission denied) 2024-06-03T03:27:58.618 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 39408 cephadm ceph txt unknown /proc/31887/task/39408/exe (readlink: Permission denied) 2024-06-03T03:27:58.618 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 39408 cephadm ceph NOFD /proc/31887/task/39408/fd (opendir: Permission denied) 2024-06-03T03:27:58.618 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 39409 cephadm ceph cwd unknown /proc/31887/task/39409/cwd (readlink: Permission denied) 2024-06-03T03:27:58.618 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 39409 cephadm ceph rtd unknown /proc/31887/task/39409/root (readlink: Permission denied) 2024-06-03T03:27:58.618 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 39409 cephadm ceph txt unknown /proc/31887/task/39409/exe (readlink: Permission denied) 2024-06-03T03:27:58.618 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 39409 cephadm ceph NOFD /proc/31887/task/39409/fd (opendir: Permission denied) 2024-06-03T03:27:58.618 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 39410 cephadm ceph cwd unknown /proc/31887/task/39410/cwd (readlink: Permission denied) 2024-06-03T03:27:58.618 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 39410 cephadm ceph rtd unknown /proc/31887/task/39410/root (readlink: Permission denied) 2024-06-03T03:27:58.618 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 39410 cephadm ceph txt unknown /proc/31887/task/39410/exe (readlink: Permission denied) 2024-06-03T03:27:58.618 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 39410 cephadm ceph NOFD /proc/31887/task/39410/fd (opendir: Permission denied) 2024-06-03T03:27:58.619 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 39411 cephadm ceph cwd unknown /proc/31887/task/39411/cwd (readlink: Permission denied) 2024-06-03T03:27:58.619 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 39411 cephadm ceph rtd unknown /proc/31887/task/39411/root (readlink: Permission denied) 2024-06-03T03:27:58.619 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 39411 cephadm ceph txt unknown /proc/31887/task/39411/exe (readlink: Permission denied) 2024-06-03T03:27:58.619 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 39411 cephadm ceph NOFD /proc/31887/task/39411/fd (opendir: Permission denied) 2024-06-03T03:27:58.619 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 39412 cephadm ceph cwd unknown /proc/31887/task/39412/cwd (readlink: Permission denied) 2024-06-03T03:27:58.619 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 39412 cephadm ceph rtd unknown /proc/31887/task/39412/root (readlink: Permission denied) 2024-06-03T03:27:58.619 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 39412 cephadm ceph txt unknown /proc/31887/task/39412/exe (readlink: Permission denied) 2024-06-03T03:27:58.619 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 39412 cephadm ceph NOFD /proc/31887/task/39412/fd (opendir: Permission denied) 2024-06-03T03:27:58.619 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 39734 cephadm ceph cwd unknown /proc/31887/task/39734/cwd (readlink: Permission denied) 2024-06-03T03:27:58.619 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 39734 cephadm ceph rtd unknown /proc/31887/task/39734/root (readlink: Permission denied) 2024-06-03T03:27:58.619 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 39734 cephadm ceph txt unknown /proc/31887/task/39734/exe (readlink: Permission denied) 2024-06-03T03:27:58.619 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 39734 cephadm ceph NOFD /proc/31887/task/39734/fd (opendir: Permission denied) 2024-06-03T03:27:58.619 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 39735 cephadm ceph cwd unknown /proc/31887/task/39735/cwd (readlink: Permission denied) 2024-06-03T03:27:58.620 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 39735 cephadm ceph rtd unknown /proc/31887/task/39735/root (readlink: Permission denied) 2024-06-03T03:27:58.620 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 39735 cephadm ceph txt unknown /proc/31887/task/39735/exe (readlink: Permission denied) 2024-06-03T03:27:58.620 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 39735 cephadm ceph NOFD /proc/31887/task/39735/fd (opendir: Permission denied) 2024-06-03T03:27:58.620 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 39768 cephadm ceph cwd unknown /proc/31887/task/39768/cwd (readlink: Permission denied) 2024-06-03T03:27:58.620 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 39768 cephadm ceph rtd unknown /proc/31887/task/39768/root (readlink: Permission denied) 2024-06-03T03:27:58.620 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 39768 cephadm ceph txt unknown /proc/31887/task/39768/exe (readlink: Permission denied) 2024-06-03T03:27:58.620 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 39768 cephadm ceph NOFD /proc/31887/task/39768/fd (opendir: Permission denied) 2024-06-03T03:27:58.620 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 39769 cephadm ceph cwd unknown /proc/31887/task/39769/cwd (readlink: Permission denied) 2024-06-03T03:27:58.620 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 39769 cephadm ceph rtd unknown /proc/31887/task/39769/root (readlink: Permission denied) 2024-06-03T03:27:58.620 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 39769 cephadm ceph txt unknown /proc/31887/task/39769/exe (readlink: Permission denied) 2024-06-03T03:27:58.620 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 39769 cephadm ceph NOFD /proc/31887/task/39769/fd (opendir: Permission denied) 2024-06-03T03:27:58.620 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 39770 cephadm ceph cwd unknown /proc/31887/task/39770/cwd (readlink: Permission denied) 2024-06-03T03:27:58.620 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 39770 cephadm ceph rtd unknown /proc/31887/task/39770/root (readlink: Permission denied) 2024-06-03T03:27:58.620 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 39770 cephadm ceph txt unknown /proc/31887/task/39770/exe (readlink: Permission denied) 2024-06-03T03:27:58.621 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 39770 cephadm ceph NOFD /proc/31887/task/39770/fd (opendir: Permission denied) 2024-06-03T03:27:58.621 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 39771 cephadm ceph cwd unknown /proc/31887/task/39771/cwd (readlink: Permission denied) 2024-06-03T03:27:58.621 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 39771 cephadm ceph rtd unknown /proc/31887/task/39771/root (readlink: Permission denied) 2024-06-03T03:27:58.621 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 39771 cephadm ceph txt unknown /proc/31887/task/39771/exe (readlink: Permission denied) 2024-06-03T03:27:58.621 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 39771 cephadm ceph NOFD /proc/31887/task/39771/fd (opendir: Permission denied) 2024-06-03T03:27:58.621 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 39772 cephadm ceph cwd unknown /proc/31887/task/39772/cwd (readlink: Permission denied) 2024-06-03T03:27:58.621 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 39772 cephadm ceph rtd unknown /proc/31887/task/39772/root (readlink: Permission denied) 2024-06-03T03:27:58.621 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 39772 cephadm ceph txt unknown /proc/31887/task/39772/exe (readlink: Permission denied) 2024-06-03T03:27:58.621 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 39772 cephadm ceph NOFD /proc/31887/task/39772/fd (opendir: Permission denied) 2024-06-03T03:27:58.621 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 39773 cephadm ceph cwd unknown /proc/31887/task/39773/cwd (readlink: Permission denied) 2024-06-03T03:27:58.621 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 39773 cephadm ceph rtd unknown /proc/31887/task/39773/root (readlink: Permission denied) 2024-06-03T03:27:58.622 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 39773 cephadm ceph txt unknown /proc/31887/task/39773/exe (readlink: Permission denied) 2024-06-03T03:27:58.622 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 39773 cephadm ceph NOFD /proc/31887/task/39773/fd (opendir: Permission denied) 2024-06-03T03:27:58.622 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 39774 cephadm ceph cwd unknown /proc/31887/task/39774/cwd (readlink: Permission denied) 2024-06-03T03:27:58.622 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 39774 cephadm ceph rtd unknown /proc/31887/task/39774/root (readlink: Permission denied) 2024-06-03T03:27:58.622 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 39774 cephadm ceph txt unknown /proc/31887/task/39774/exe (readlink: Permission denied) 2024-06-03T03:27:58.622 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 39774 cephadm ceph NOFD /proc/31887/task/39774/fd (opendir: Permission denied) 2024-06-03T03:27:58.622 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 39775 cephadm ceph cwd unknown /proc/31887/task/39775/cwd (readlink: Permission denied) 2024-06-03T03:27:58.622 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 39775 cephadm ceph rtd unknown /proc/31887/task/39775/root (readlink: Permission denied) 2024-06-03T03:27:58.622 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 39775 cephadm ceph txt unknown /proc/31887/task/39775/exe (readlink: Permission denied) 2024-06-03T03:27:58.622 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 39775 cephadm ceph NOFD /proc/31887/task/39775/fd (opendir: Permission denied) 2024-06-03T03:27:58.622 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 48672 devicehea ceph cwd unknown /proc/31887/task/48672/cwd (readlink: Permission denied) 2024-06-03T03:27:58.623 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 48672 devicehea ceph rtd unknown /proc/31887/task/48672/root (readlink: Permission denied) 2024-06-03T03:27:58.623 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 48672 devicehea ceph txt unknown /proc/31887/task/48672/exe (readlink: Permission denied) 2024-06-03T03:27:58.623 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 48672 devicehea ceph NOFD /proc/31887/task/48672/fd (opendir: Permission denied) 2024-06-03T03:27:58.623 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 48676 io_contex ceph cwd unknown /proc/31887/task/48676/cwd (readlink: Permission denied) 2024-06-03T03:27:58.623 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 48676 io_contex ceph rtd unknown /proc/31887/task/48676/root (readlink: Permission denied) 2024-06-03T03:27:58.623 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 48676 io_contex ceph txt unknown /proc/31887/task/48676/exe (readlink: Permission denied) 2024-06-03T03:27:58.623 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 48676 io_contex ceph NOFD /proc/31887/task/48676/fd (opendir: Permission denied) 2024-06-03T03:27:58.623 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 48677 io_contex ceph cwd unknown /proc/31887/task/48677/cwd (readlink: Permission denied) 2024-06-03T03:27:58.623 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 48677 io_contex ceph rtd unknown /proc/31887/task/48677/root (readlink: Permission denied) 2024-06-03T03:27:58.623 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 48677 io_contex ceph txt unknown /proc/31887/task/48677/exe (readlink: Permission denied) 2024-06-03T03:27:58.623 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 48677 io_contex ceph NOFD /proc/31887/task/48677/fd (opendir: Permission denied) 2024-06-03T03:27:58.623 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 48678 ceph_time ceph cwd unknown /proc/31887/task/48678/cwd (readlink: Permission denied) 2024-06-03T03:27:58.623 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 48678 ceph_time ceph rtd unknown /proc/31887/task/48678/root (readlink: Permission denied) 2024-06-03T03:27:58.624 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 48678 ceph_time ceph txt unknown /proc/31887/task/48678/exe (readlink: Permission denied) 2024-06-03T03:27:58.624 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 48678 ceph_time ceph NOFD /proc/31887/task/48678/fd (opendir: Permission denied) 2024-06-03T03:27:58.624 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 48679 ms_dispat ceph cwd unknown /proc/31887/task/48679/cwd (readlink: Permission denied) 2024-06-03T03:27:58.624 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 48679 ms_dispat ceph rtd unknown /proc/31887/task/48679/root (readlink: Permission denied) 2024-06-03T03:27:58.624 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 48679 ms_dispat ceph txt unknown /proc/31887/task/48679/exe (readlink: Permission denied) 2024-06-03T03:27:58.624 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 48679 ms_dispat ceph NOFD /proc/31887/task/48679/fd (opendir: Permission denied) 2024-06-03T03:27:58.624 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 48680 ms_local ceph cwd unknown /proc/31887/task/48680/cwd (readlink: Permission denied) 2024-06-03T03:27:58.624 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 48680 ms_local ceph rtd unknown /proc/31887/task/48680/root (readlink: Permission denied) 2024-06-03T03:27:58.624 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 48680 ms_local ceph txt unknown /proc/31887/task/48680/exe (readlink: Permission denied) 2024-06-03T03:27:58.624 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 48680 ms_local ceph NOFD /proc/31887/task/48680/fd (opendir: Permission denied) 2024-06-03T03:27:58.624 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 48681 safe_time ceph cwd unknown /proc/31887/task/48681/cwd (readlink: Permission denied) 2024-06-03T03:27:58.624 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 48681 safe_time ceph rtd unknown /proc/31887/task/48681/root (readlink: Permission denied) 2024-06-03T03:27:58.624 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 48681 safe_time ceph txt unknown /proc/31887/task/48681/exe (readlink: Permission denied) 2024-06-03T03:27:58.624 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 48681 safe_time ceph NOFD /proc/31887/task/48681/fd (opendir: Permission denied) 2024-06-03T03:27:58.625 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 48682 safe_time ceph cwd unknown /proc/31887/task/48682/cwd (readlink: Permission denied) 2024-06-03T03:27:58.625 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 48682 safe_time ceph rtd unknown /proc/31887/task/48682/root (readlink: Permission denied) 2024-06-03T03:27:58.625 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 48682 safe_time ceph txt unknown /proc/31887/task/48682/exe (readlink: Permission denied) 2024-06-03T03:27:58.625 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mgr 31887 48682 safe_time ceph NOFD /proc/31887/task/48682/fd (opendir: Permission denied) 2024-06-03T03:27:58.625 INFO:tasks.workunit.client.0.smithi117.stdout:systemd 33025 root cwd unknown /proc/33025/cwd (readlink: Permission denied) 2024-06-03T03:27:58.625 INFO:tasks.workunit.client.0.smithi117.stdout:systemd 33025 root rtd unknown /proc/33025/root (readlink: Permission denied) 2024-06-03T03:27:58.625 INFO:tasks.workunit.client.0.smithi117.stdout:systemd 33025 root txt unknown /proc/33025/exe (readlink: Permission denied) 2024-06-03T03:27:58.625 INFO:tasks.workunit.client.0.smithi117.stdout:systemd 33025 root NOFD /proc/33025/fd (opendir: Permission denied) 2024-06-03T03:27:58.625 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/5 33026 root cwd unknown /proc/33026/cwd (readlink: Permission denied) 2024-06-03T03:27:58.625 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/5 33026 root rtd unknown /proc/33026/root (readlink: Permission denied) 2024-06-03T03:27:58.625 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/5 33026 root txt unknown /proc/33026/exe (readlink: Permission denied) 2024-06-03T03:27:58.625 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/5 33026 root NOFD /proc/33026/fd (opendir: Permission denied) 2024-06-03T03:27:58.625 INFO:tasks.workunit.client.0.smithi117.stdout:(sd-pam) 33028 root cwd unknown /proc/33028/cwd (readlink: Permission denied) 2024-06-03T03:27:58.626 INFO:tasks.workunit.client.0.smithi117.stdout:(sd-pam) 33028 root rtd unknown /proc/33028/root (readlink: Permission denied) 2024-06-03T03:27:58.626 INFO:tasks.workunit.client.0.smithi117.stdout:(sd-pam) 33028 root txt unknown /proc/33028/exe (readlink: Permission denied) 2024-06-03T03:27:58.626 INFO:tasks.workunit.client.0.smithi117.stdout:(sd-pam) 33028 root NOFD /proc/33028/fd (opendir: Permission denied) 2024-06-03T03:27:58.626 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/2 34247 root cwd unknown /proc/34247/cwd (readlink: Permission denied) 2024-06-03T03:27:58.626 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/2 34247 root rtd unknown /proc/34247/root (readlink: Permission denied) 2024-06-03T03:27:58.626 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/2 34247 root txt unknown /proc/34247/exe (readlink: Permission denied) 2024-06-03T03:27:58.626 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/2 34247 root NOFD /proc/34247/fd (opendir: Permission denied) 2024-06-03T03:27:58.626 INFO:tasks.workunit.client.0.smithi117.stdout:sshd 34547 root cwd unknown /proc/34547/cwd (readlink: Permission denied) 2024-06-03T03:27:58.626 INFO:tasks.workunit.client.0.smithi117.stdout:sshd 34547 root rtd unknown /proc/34547/root (readlink: Permission denied) 2024-06-03T03:27:58.626 INFO:tasks.workunit.client.0.smithi117.stdout:sshd 34547 root txt unknown /proc/34547/exe (readlink: Permission denied) 2024-06-03T03:27:58.626 INFO:tasks.workunit.client.0.smithi117.stdout:sshd 34547 root NOFD /proc/34547/fd (opendir: Permission denied) 2024-06-03T03:27:58.626 INFO:tasks.workunit.client.0.smithi117.stdout:sshd 34565 root cwd unknown /proc/34565/cwd (readlink: Permission denied) 2024-06-03T03:27:58.626 INFO:tasks.workunit.client.0.smithi117.stdout:sshd 34565 root rtd unknown /proc/34565/root (readlink: Permission denied) 2024-06-03T03:27:58.627 INFO:tasks.workunit.client.0.smithi117.stdout:sshd 34565 root txt unknown /proc/34565/exe (readlink: Permission denied) 2024-06-03T03:27:58.627 INFO:tasks.workunit.client.0.smithi117.stdout:sshd 34565 root NOFD /proc/34565/fd (opendir: Permission denied) 2024-06-03T03:27:58.627 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/6 40123 root cwd unknown /proc/40123/cwd (readlink: Permission denied) 2024-06-03T03:27:58.627 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/6 40123 root rtd unknown /proc/40123/root (readlink: Permission denied) 2024-06-03T03:27:58.627 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/6 40123 root txt unknown /proc/40123/exe (readlink: Permission denied) 2024-06-03T03:27:58.627 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/6 40123 root NOFD /proc/40123/fd (opendir: Permission denied) 2024-06-03T03:27:58.627 INFO:tasks.workunit.client.0.smithi117.stdout:conmon 44092 root cwd unknown /proc/44092/cwd (readlink: Permission denied) 2024-06-03T03:27:58.627 INFO:tasks.workunit.client.0.smithi117.stdout:conmon 44092 root rtd unknown /proc/44092/root (readlink: Permission denied) 2024-06-03T03:27:58.627 INFO:tasks.workunit.client.0.smithi117.stdout:conmon 44092 root txt unknown /proc/44092/exe (readlink: Permission denied) 2024-06-03T03:27:58.627 INFO:tasks.workunit.client.0.smithi117.stdout:conmon 44092 root NOFD /proc/44092/fd (opendir: Permission denied) 2024-06-03T03:27:58.627 INFO:tasks.workunit.client.0.smithi117.stdout:podman-in 44094 root cwd unknown /proc/44094/cwd (readlink: Permission denied) 2024-06-03T03:27:58.627 INFO:tasks.workunit.client.0.smithi117.stdout:podman-in 44094 root rtd unknown /proc/44094/root (readlink: Permission denied) 2024-06-03T03:27:58.627 INFO:tasks.workunit.client.0.smithi117.stdout:podman-in 44094 root txt unknown /proc/44094/exe (readlink: Permission denied) 2024-06-03T03:27:58.627 INFO:tasks.workunit.client.0.smithi117.stdout:podman-in 44094 root NOFD /proc/44094/fd (opendir: Permission denied) 2024-06-03T03:27:58.628 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 ceph cwd unknown /proc/44096/cwd (readlink: Permission denied) 2024-06-03T03:27:58.628 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 ceph rtd unknown /proc/44096/root (readlink: Permission denied) 2024-06-03T03:27:58.628 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 ceph txt unknown /proc/44096/exe (readlink: Permission denied) 2024-06-03T03:27:58.628 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 ceph NOFD /proc/44096/fd (opendir: Permission denied) 2024-06-03T03:27:58.628 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44097 log ceph cwd unknown /proc/44096/task/44097/cwd (readlink: Permission denied) 2024-06-03T03:27:58.628 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44097 log ceph rtd unknown /proc/44096/task/44097/root (readlink: Permission denied) 2024-06-03T03:27:58.628 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44097 log ceph txt unknown /proc/44096/task/44097/exe (readlink: Permission denied) 2024-06-03T03:27:58.628 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44097 log ceph NOFD /proc/44096/task/44097/fd (opendir: Permission denied) 2024-06-03T03:27:58.628 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44099 msgr-work ceph cwd unknown /proc/44096/task/44099/cwd (readlink: Permission denied) 2024-06-03T03:27:58.628 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44099 msgr-work ceph rtd unknown /proc/44096/task/44099/root (readlink: Permission denied) 2024-06-03T03:27:58.628 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44099 msgr-work ceph txt unknown /proc/44096/task/44099/exe (readlink: Permission denied) 2024-06-03T03:27:58.628 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44099 msgr-work ceph NOFD /proc/44096/task/44099/fd (opendir: Permission denied) 2024-06-03T03:27:58.628 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44100 msgr-work ceph cwd unknown /proc/44096/task/44100/cwd (readlink: Permission denied) 2024-06-03T03:27:58.629 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44100 msgr-work ceph rtd unknown /proc/44096/task/44100/root (readlink: Permission denied) 2024-06-03T03:27:58.629 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44100 msgr-work ceph txt unknown /proc/44096/task/44100/exe (readlink: Permission denied) 2024-06-03T03:27:58.629 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44100 msgr-work ceph NOFD /proc/44096/task/44100/fd (opendir: Permission denied) 2024-06-03T03:27:58.629 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44101 msgr-work ceph cwd unknown /proc/44096/task/44101/cwd (readlink: Permission denied) 2024-06-03T03:27:58.629 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44101 msgr-work ceph rtd unknown /proc/44096/task/44101/root (readlink: Permission denied) 2024-06-03T03:27:58.629 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44101 msgr-work ceph txt unknown /proc/44096/task/44101/exe (readlink: Permission denied) 2024-06-03T03:27:58.629 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44101 msgr-work ceph NOFD /proc/44096/task/44101/fd (opendir: Permission denied) 2024-06-03T03:27:58.629 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44105 service ceph cwd unknown /proc/44096/task/44105/cwd (readlink: Permission denied) 2024-06-03T03:27:58.629 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44105 service ceph rtd unknown /proc/44096/task/44105/root (readlink: Permission denied) 2024-06-03T03:27:58.629 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44105 service ceph txt unknown /proc/44096/task/44105/exe (readlink: Permission denied) 2024-06-03T03:27:58.629 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44105 service ceph NOFD /proc/44096/task/44105/fd (opendir: Permission denied) 2024-06-03T03:27:58.629 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44106 admin_soc ceph cwd unknown /proc/44096/task/44106/cwd (readlink: Permission denied) 2024-06-03T03:27:58.629 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44106 admin_soc ceph rtd unknown /proc/44096/task/44106/root (readlink: Permission denied) 2024-06-03T03:27:58.630 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44106 admin_soc ceph txt unknown /proc/44096/task/44106/exe (readlink: Permission denied) 2024-06-03T03:27:58.630 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44106 admin_soc ceph NOFD /proc/44096/task/44106/fd (opendir: Permission denied) 2024-06-03T03:27:58.630 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44249 signal_ha ceph cwd unknown /proc/44096/task/44249/cwd (readlink: Permission denied) 2024-06-03T03:27:58.630 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44249 signal_ha ceph rtd unknown /proc/44096/task/44249/root (readlink: Permission denied) 2024-06-03T03:27:58.630 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44249 signal_ha ceph txt unknown /proc/44096/task/44249/exe (readlink: Permission denied) 2024-06-03T03:27:58.630 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44249 signal_ha ceph NOFD /proc/44096/task/44249/fd (opendir: Permission denied) 2024-06-03T03:27:58.630 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44250 io_contex ceph cwd unknown /proc/44096/task/44250/cwd (readlink: Permission denied) 2024-06-03T03:27:58.630 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44250 io_contex ceph rtd unknown /proc/44096/task/44250/root (readlink: Permission denied) 2024-06-03T03:27:58.630 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44250 io_contex ceph txt unknown /proc/44096/task/44250/exe (readlink: Permission denied) 2024-06-03T03:27:58.630 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44250 io_contex ceph NOFD /proc/44096/task/44250/fd (opendir: Permission denied) 2024-06-03T03:27:58.630 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44251 io_contex ceph cwd unknown /proc/44096/task/44251/cwd (readlink: Permission denied) 2024-06-03T03:27:58.630 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44251 io_contex ceph rtd unknown /proc/44096/task/44251/root (readlink: Permission denied) 2024-06-03T03:27:58.630 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44251 io_contex ceph txt unknown /proc/44096/task/44251/exe (readlink: Permission denied) 2024-06-03T03:27:58.631 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44251 io_contex ceph NOFD /proc/44096/task/44251/fd (opendir: Permission denied) 2024-06-03T03:27:58.631 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44254 OpHistory ceph cwd unknown /proc/44096/task/44254/cwd (readlink: Permission denied) 2024-06-03T03:27:58.631 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44254 OpHistory ceph rtd unknown /proc/44096/task/44254/root (readlink: Permission denied) 2024-06-03T03:27:58.631 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44254 OpHistory ceph txt unknown /proc/44096/task/44254/exe (readlink: Permission denied) 2024-06-03T03:27:58.631 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44254 OpHistory ceph NOFD /proc/44096/task/44254/fd (opendir: Permission denied) 2024-06-03T03:27:58.631 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44255 ceph_time ceph cwd unknown /proc/44096/task/44255/cwd (readlink: Permission denied) 2024-06-03T03:27:58.631 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44255 ceph_time ceph rtd unknown /proc/44096/task/44255/root (readlink: Permission denied) 2024-06-03T03:27:58.631 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44255 ceph_time ceph txt unknown /proc/44096/task/44255/exe (readlink: Permission denied) 2024-06-03T03:27:58.631 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44255 ceph_time ceph NOFD /proc/44096/task/44255/fd (opendir: Permission denied) 2024-06-03T03:27:58.631 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44477 ceph-osd ceph cwd unknown /proc/44096/task/44477/cwd (readlink: Permission denied) 2024-06-03T03:27:58.631 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44477 ceph-osd ceph rtd unknown /proc/44096/task/44477/root (readlink: Permission denied) 2024-06-03T03:27:58.631 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44477 ceph-osd ceph txt unknown /proc/44096/task/44477/exe (readlink: Permission denied) 2024-06-03T03:27:58.631 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44477 ceph-osd ceph NOFD /proc/44096/task/44477/fd (opendir: Permission denied) 2024-06-03T03:27:58.632 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44478 safe_time ceph cwd unknown /proc/44096/task/44478/cwd (readlink: Permission denied) 2024-06-03T03:27:58.632 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44478 safe_time ceph rtd unknown /proc/44096/task/44478/root (readlink: Permission denied) 2024-06-03T03:27:58.632 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44478 safe_time ceph txt unknown /proc/44096/task/44478/exe (readlink: Permission denied) 2024-06-03T03:27:58.632 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44478 safe_time ceph NOFD /proc/44096/task/44478/fd (opendir: Permission denied) 2024-06-03T03:27:58.632 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44479 safe_time ceph cwd unknown /proc/44096/task/44479/cwd (readlink: Permission denied) 2024-06-03T03:27:58.632 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44479 safe_time ceph rtd unknown /proc/44096/task/44479/root (readlink: Permission denied) 2024-06-03T03:27:58.632 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44479 safe_time ceph txt unknown /proc/44096/task/44479/exe (readlink: Permission denied) 2024-06-03T03:27:58.632 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44479 safe_time ceph NOFD /proc/44096/task/44479/fd (opendir: Permission denied) 2024-06-03T03:27:58.632 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44480 safe_time ceph cwd unknown /proc/44096/task/44480/cwd (readlink: Permission denied) 2024-06-03T03:27:58.632 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44480 safe_time ceph rtd unknown /proc/44096/task/44480/root (readlink: Permission denied) 2024-06-03T03:27:58.632 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44480 safe_time ceph txt unknown /proc/44096/task/44480/exe (readlink: Permission denied) 2024-06-03T03:27:58.632 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44480 safe_time ceph NOFD /proc/44096/task/44480/fd (opendir: Permission denied) 2024-06-03T03:27:58.632 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44481 safe_time ceph cwd unknown /proc/44096/task/44481/cwd (readlink: Permission denied) 2024-06-03T03:27:58.632 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44481 safe_time ceph rtd unknown /proc/44096/task/44481/root (readlink: Permission denied) 2024-06-03T03:27:58.633 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44481 safe_time ceph txt unknown /proc/44096/task/44481/exe (readlink: Permission denied) 2024-06-03T03:27:58.633 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44481 safe_time ceph NOFD /proc/44096/task/44481/fd (opendir: Permission denied) 2024-06-03T03:27:58.633 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44482 fn_anonym ceph cwd unknown /proc/44096/task/44482/cwd (readlink: Permission denied) 2024-06-03T03:27:58.633 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44482 fn_anonym ceph rtd unknown /proc/44096/task/44482/root (readlink: Permission denied) 2024-06-03T03:27:58.633 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44482 fn_anonym ceph txt unknown /proc/44096/task/44482/exe (readlink: Permission denied) 2024-06-03T03:27:58.633 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44482 fn_anonym ceph NOFD /proc/44096/task/44482/fd (opendir: Permission denied) 2024-06-03T03:27:58.633 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44488 rocksdb:l ceph cwd unknown /proc/44096/task/44488/cwd (readlink: Permission denied) 2024-06-03T03:27:58.633 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44488 rocksdb:l ceph rtd unknown /proc/44096/task/44488/root (readlink: Permission denied) 2024-06-03T03:27:58.633 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44488 rocksdb:l ceph txt unknown /proc/44096/task/44488/exe (readlink: Permission denied) 2024-06-03T03:27:58.633 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44488 rocksdb:l ceph NOFD /proc/44096/task/44488/fd (opendir: Permission denied) 2024-06-03T03:27:58.633 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44489 rocksdb:l ceph cwd unknown /proc/44096/task/44489/cwd (readlink: Permission denied) 2024-06-03T03:27:58.633 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44489 rocksdb:l ceph rtd unknown /proc/44096/task/44489/root (readlink: Permission denied) 2024-06-03T03:27:58.633 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44489 rocksdb:l ceph txt unknown /proc/44096/task/44489/exe (readlink: Permission denied) 2024-06-03T03:27:58.634 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44489 rocksdb:l ceph NOFD /proc/44096/task/44489/fd (opendir: Permission denied) 2024-06-03T03:27:58.634 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44490 rocksdb:l ceph cwd unknown /proc/44096/task/44490/cwd (readlink: Permission denied) 2024-06-03T03:27:58.634 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44490 rocksdb:l ceph rtd unknown /proc/44096/task/44490/root (readlink: Permission denied) 2024-06-03T03:27:58.634 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44490 rocksdb:l ceph txt unknown /proc/44096/task/44490/exe (readlink: Permission denied) 2024-06-03T03:27:58.634 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44490 rocksdb:l ceph NOFD /proc/44096/task/44490/fd (opendir: Permission denied) 2024-06-03T03:27:58.634 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44491 rocksdb:h ceph cwd unknown /proc/44096/task/44491/cwd (readlink: Permission denied) 2024-06-03T03:27:58.634 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44491 rocksdb:h ceph rtd unknown /proc/44096/task/44491/root (readlink: Permission denied) 2024-06-03T03:27:58.634 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44491 rocksdb:h ceph txt unknown /proc/44096/task/44491/exe (readlink: Permission denied) 2024-06-03T03:27:58.634 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44491 rocksdb:h ceph NOFD /proc/44096/task/44491/fd (opendir: Permission denied) 2024-06-03T03:27:58.634 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44939 bstore_ai ceph cwd unknown /proc/44096/task/44939/cwd (readlink: Permission denied) 2024-06-03T03:27:58.634 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44939 bstore_ai ceph rtd unknown /proc/44096/task/44939/root (readlink: Permission denied) 2024-06-03T03:27:58.634 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44939 bstore_ai ceph txt unknown /proc/44096/task/44939/exe (readlink: Permission denied) 2024-06-03T03:27:58.634 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 44939 bstore_ai ceph NOFD /proc/44096/task/44939/fd (opendir: Permission denied) 2024-06-03T03:27:58.635 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45141 bstore_ai ceph cwd unknown /proc/44096/task/45141/cwd (readlink: Permission denied) 2024-06-03T03:27:58.635 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45141 bstore_ai ceph rtd unknown /proc/44096/task/45141/root (readlink: Permission denied) 2024-06-03T03:27:58.635 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45141 bstore_ai ceph txt unknown /proc/44096/task/45141/exe (readlink: Permission denied) 2024-06-03T03:27:58.635 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45141 bstore_ai ceph NOFD /proc/44096/task/45141/fd (opendir: Permission denied) 2024-06-03T03:27:58.635 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45327 ceph-osd ceph cwd unknown /proc/44096/task/45327/cwd (readlink: Permission denied) 2024-06-03T03:27:58.635 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45327 ceph-osd ceph rtd unknown /proc/44096/task/45327/root (readlink: Permission denied) 2024-06-03T03:27:58.635 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45327 ceph-osd ceph txt unknown /proc/44096/task/45327/exe (readlink: Permission denied) 2024-06-03T03:27:58.635 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45327 ceph-osd ceph NOFD /proc/44096/task/45327/fd (opendir: Permission denied) 2024-06-03T03:27:58.635 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45328 cfin ceph cwd unknown /proc/44096/task/45328/cwd (readlink: Permission denied) 2024-06-03T03:27:58.635 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45328 cfin ceph rtd unknown /proc/44096/task/45328/root (readlink: Permission denied) 2024-06-03T03:27:58.635 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45328 cfin ceph txt unknown /proc/44096/task/45328/exe (readlink: Permission denied) 2024-06-03T03:27:58.635 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45328 cfin ceph NOFD /proc/44096/task/45328/fd (opendir: Permission denied) 2024-06-03T03:27:58.635 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45329 bstore_kv ceph cwd unknown /proc/44096/task/45329/cwd (readlink: Permission denied) 2024-06-03T03:27:58.636 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45329 bstore_kv ceph rtd unknown /proc/44096/task/45329/root (readlink: Permission denied) 2024-06-03T03:27:58.636 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45329 bstore_kv ceph txt unknown /proc/44096/task/45329/exe (readlink: Permission denied) 2024-06-03T03:27:58.636 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45329 bstore_kv ceph NOFD /proc/44096/task/45329/fd (opendir: Permission denied) 2024-06-03T03:27:58.636 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45330 bstore_kv ceph cwd unknown /proc/44096/task/45330/cwd (readlink: Permission denied) 2024-06-03T03:27:58.636 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45330 bstore_kv ceph rtd unknown /proc/44096/task/45330/root (readlink: Permission denied) 2024-06-03T03:27:58.636 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45330 bstore_kv ceph txt unknown /proc/44096/task/45330/exe (readlink: Permission denied) 2024-06-03T03:27:58.636 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45330 bstore_kv ceph NOFD /proc/44096/task/45330/fd (opendir: Permission denied) 2024-06-03T03:27:58.636 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45331 bstore_me ceph cwd unknown /proc/44096/task/45331/cwd (readlink: Permission denied) 2024-06-03T03:27:58.636 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45331 bstore_me ceph rtd unknown /proc/44096/task/45331/root (readlink: Permission denied) 2024-06-03T03:27:58.636 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45331 bstore_me ceph txt unknown /proc/44096/task/45331/exe (readlink: Permission denied) 2024-06-03T03:27:58.636 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45331 bstore_me ceph NOFD /proc/44096/task/45331/fd (opendir: Permission denied) 2024-06-03T03:27:58.636 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45332 ms_dispat ceph cwd unknown /proc/44096/task/45332/cwd (readlink: Permission denied) 2024-06-03T03:27:58.636 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45332 ms_dispat ceph rtd unknown /proc/44096/task/45332/root (readlink: Permission denied) 2024-06-03T03:27:58.636 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45332 ms_dispat ceph txt unknown /proc/44096/task/45332/exe (readlink: Permission denied) 2024-06-03T03:27:58.637 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45332 ms_dispat ceph NOFD /proc/44096/task/45332/fd (opendir: Permission denied) 2024-06-03T03:27:58.637 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45333 ms_local ceph cwd unknown /proc/44096/task/45333/cwd (readlink: Permission denied) 2024-06-03T03:27:58.637 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45333 ms_local ceph rtd unknown /proc/44096/task/45333/root (readlink: Permission denied) 2024-06-03T03:27:58.637 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45333 ms_local ceph txt unknown /proc/44096/task/45333/exe (readlink: Permission denied) 2024-06-03T03:27:58.637 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45333 ms_local ceph NOFD /proc/44096/task/45333/fd (opendir: Permission denied) 2024-06-03T03:27:58.637 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45334 safe_time ceph cwd unknown /proc/44096/task/45334/cwd (readlink: Permission denied) 2024-06-03T03:27:58.637 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45334 safe_time ceph rtd unknown /proc/44096/task/45334/root (readlink: Permission denied) 2024-06-03T03:27:58.637 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45334 safe_time ceph txt unknown /proc/44096/task/45334/exe (readlink: Permission denied) 2024-06-03T03:27:58.637 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45334 safe_time ceph NOFD /proc/44096/task/45334/fd (opendir: Permission denied) 2024-06-03T03:27:58.637 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45335 safe_time ceph cwd unknown /proc/44096/task/45335/cwd (readlink: Permission denied) 2024-06-03T03:27:58.637 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45335 safe_time ceph rtd unknown /proc/44096/task/45335/root (readlink: Permission denied) 2024-06-03T03:27:58.637 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45335 safe_time ceph txt unknown /proc/44096/task/45335/exe (readlink: Permission denied) 2024-06-03T03:27:58.637 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45335 safe_time ceph NOFD /proc/44096/task/45335/fd (opendir: Permission denied) 2024-06-03T03:27:58.637 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45336 ms_dispat ceph cwd unknown /proc/44096/task/45336/cwd (readlink: Permission denied) 2024-06-03T03:27:58.638 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45336 ms_dispat ceph rtd unknown /proc/44096/task/45336/root (readlink: Permission denied) 2024-06-03T03:27:58.638 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45336 ms_dispat ceph txt unknown /proc/44096/task/45336/exe (readlink: Permission denied) 2024-06-03T03:27:58.638 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45336 ms_dispat ceph NOFD /proc/44096/task/45336/fd (opendir: Permission denied) 2024-06-03T03:27:58.638 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45337 ms_local ceph cwd unknown /proc/44096/task/45337/cwd (readlink: Permission denied) 2024-06-03T03:27:58.638 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45337 ms_local ceph rtd unknown /proc/44096/task/45337/root (readlink: Permission denied) 2024-06-03T03:27:58.638 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45337 ms_local ceph txt unknown /proc/44096/task/45337/exe (readlink: Permission denied) 2024-06-03T03:27:58.638 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45337 ms_local ceph NOFD /proc/44096/task/45337/fd (opendir: Permission denied) 2024-06-03T03:27:58.638 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45338 ms_dispat ceph cwd unknown /proc/44096/task/45338/cwd (readlink: Permission denied) 2024-06-03T03:27:58.638 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45338 ms_dispat ceph rtd unknown /proc/44096/task/45338/root (readlink: Permission denied) 2024-06-03T03:27:58.638 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45338 ms_dispat ceph txt unknown /proc/44096/task/45338/exe (readlink: Permission denied) 2024-06-03T03:27:58.638 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45338 ms_dispat ceph NOFD /proc/44096/task/45338/fd (opendir: Permission denied) 2024-06-03T03:27:58.638 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45339 ms_local ceph cwd unknown /proc/44096/task/45339/cwd (readlink: Permission denied) 2024-06-03T03:27:58.638 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45339 ms_local ceph rtd unknown /proc/44096/task/45339/root (readlink: Permission denied) 2024-06-03T03:27:58.639 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45339 ms_local ceph txt unknown /proc/44096/task/45339/exe (readlink: Permission denied) 2024-06-03T03:27:58.639 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45339 ms_local ceph NOFD /proc/44096/task/45339/fd (opendir: Permission denied) 2024-06-03T03:27:58.639 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45340 ms_dispat ceph cwd unknown /proc/44096/task/45340/cwd (readlink: Permission denied) 2024-06-03T03:27:58.639 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45340 ms_dispat ceph rtd unknown /proc/44096/task/45340/root (readlink: Permission denied) 2024-06-03T03:27:58.639 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45340 ms_dispat ceph txt unknown /proc/44096/task/45340/exe (readlink: Permission denied) 2024-06-03T03:27:58.639 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45340 ms_dispat ceph NOFD /proc/44096/task/45340/fd (opendir: Permission denied) 2024-06-03T03:27:58.639 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45341 ms_local ceph cwd unknown /proc/44096/task/45341/cwd (readlink: Permission denied) 2024-06-03T03:27:58.639 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45341 ms_local ceph rtd unknown /proc/44096/task/45341/root (readlink: Permission denied) 2024-06-03T03:27:58.639 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45341 ms_local ceph txt unknown /proc/44096/task/45341/exe (readlink: Permission denied) 2024-06-03T03:27:58.640 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45341 ms_local ceph NOFD /proc/44096/task/45341/fd (opendir: Permission denied) 2024-06-03T03:27:58.640 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45342 ms_dispat ceph cwd unknown /proc/44096/task/45342/cwd (readlink: Permission denied) 2024-06-03T03:27:58.640 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45342 ms_dispat ceph rtd unknown /proc/44096/task/45342/root (readlink: Permission denied) 2024-06-03T03:27:58.640 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45342 ms_dispat ceph txt unknown /proc/44096/task/45342/exe (readlink: Permission denied) 2024-06-03T03:27:58.640 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45342 ms_dispat ceph NOFD /proc/44096/task/45342/fd (opendir: Permission denied) 2024-06-03T03:27:58.640 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45343 ms_local ceph cwd unknown /proc/44096/task/45343/cwd (readlink: Permission denied) 2024-06-03T03:27:58.640 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45343 ms_local ceph rtd unknown /proc/44096/task/45343/root (readlink: Permission denied) 2024-06-03T03:27:58.640 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45343 ms_local ceph txt unknown /proc/44096/task/45343/exe (readlink: Permission denied) 2024-06-03T03:27:58.640 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45343 ms_local ceph NOFD /proc/44096/task/45343/fd (opendir: Permission denied) 2024-06-03T03:27:58.640 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45344 ms_dispat ceph cwd unknown /proc/44096/task/45344/cwd (readlink: Permission denied) 2024-06-03T03:27:58.640 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45344 ms_dispat ceph rtd unknown /proc/44096/task/45344/root (readlink: Permission denied) 2024-06-03T03:27:58.640 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45344 ms_dispat ceph txt unknown /proc/44096/task/45344/exe (readlink: Permission denied) 2024-06-03T03:27:58.641 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45344 ms_dispat ceph NOFD /proc/44096/task/45344/fd (opendir: Permission denied) 2024-06-03T03:27:58.641 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45345 ms_local ceph cwd unknown /proc/44096/task/45345/cwd (readlink: Permission denied) 2024-06-03T03:27:58.641 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45345 ms_local ceph rtd unknown /proc/44096/task/45345/root (readlink: Permission denied) 2024-06-03T03:27:58.641 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45345 ms_local ceph txt unknown /proc/44096/task/45345/exe (readlink: Permission denied) 2024-06-03T03:27:58.641 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45345 ms_local ceph NOFD /proc/44096/task/45345/fd (opendir: Permission denied) 2024-06-03T03:27:58.641 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45346 ms_dispat ceph cwd unknown /proc/44096/task/45346/cwd (readlink: Permission denied) 2024-06-03T03:27:58.641 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45346 ms_dispat ceph rtd unknown /proc/44096/task/45346/root (readlink: Permission denied) 2024-06-03T03:27:58.641 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45346 ms_dispat ceph txt unknown /proc/44096/task/45346/exe (readlink: Permission denied) 2024-06-03T03:27:58.641 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45346 ms_dispat ceph NOFD /proc/44096/task/45346/fd (opendir: Permission denied) 2024-06-03T03:27:58.641 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45347 ms_local ceph cwd unknown /proc/44096/task/45347/cwd (readlink: Permission denied) 2024-06-03T03:27:58.641 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45347 ms_local ceph rtd unknown /proc/44096/task/45347/root (readlink: Permission denied) 2024-06-03T03:27:58.641 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45347 ms_local ceph txt unknown /proc/44096/task/45347/exe (readlink: Permission denied) 2024-06-03T03:27:58.642 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45347 ms_local ceph NOFD /proc/44096/task/45347/fd (opendir: Permission denied) 2024-06-03T03:27:58.642 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45348 fn_anonym ceph cwd unknown /proc/44096/task/45348/cwd (readlink: Permission denied) 2024-06-03T03:27:58.642 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45348 fn_anonym ceph rtd unknown /proc/44096/task/45348/root (readlink: Permission denied) 2024-06-03T03:27:58.642 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45348 fn_anonym ceph txt unknown /proc/44096/task/45348/exe (readlink: Permission denied) 2024-06-03T03:27:58.642 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45348 fn_anonym ceph NOFD /proc/44096/task/45348/fd (opendir: Permission denied) 2024-06-03T03:27:58.642 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45349 finisher ceph cwd unknown /proc/44096/task/45349/cwd (readlink: Permission denied) 2024-06-03T03:27:58.642 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45349 finisher ceph rtd unknown /proc/44096/task/45349/root (readlink: Permission denied) 2024-06-03T03:27:58.642 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45349 finisher ceph txt unknown /proc/44096/task/45349/exe (readlink: Permission denied) 2024-06-03T03:27:58.642 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45349 finisher ceph NOFD /proc/44096/task/45349/fd (opendir: Permission denied) 2024-06-03T03:27:58.642 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45350 safe_time ceph cwd unknown /proc/44096/task/45350/cwd (readlink: Permission denied) 2024-06-03T03:27:58.642 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45350 safe_time ceph rtd unknown /proc/44096/task/45350/root (readlink: Permission denied) 2024-06-03T03:27:58.642 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45350 safe_time ceph txt unknown /proc/44096/task/45350/exe (readlink: Permission denied) 2024-06-03T03:27:58.642 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45350 safe_time ceph NOFD /proc/44096/task/45350/fd (opendir: Permission denied) 2024-06-03T03:27:58.643 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45351 safe_time ceph cwd unknown /proc/44096/task/45351/cwd (readlink: Permission denied) 2024-06-03T03:27:58.643 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45351 safe_time ceph rtd unknown /proc/44096/task/45351/root (readlink: Permission denied) 2024-06-03T03:27:58.643 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45351 safe_time ceph txt unknown /proc/44096/task/45351/exe (readlink: Permission denied) 2024-06-03T03:27:58.643 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45351 safe_time ceph NOFD /proc/44096/task/45351/fd (opendir: Permission denied) 2024-06-03T03:27:58.643 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45352 ceph-osd ceph cwd unknown /proc/44096/task/45352/cwd (readlink: Permission denied) 2024-06-03T03:27:58.643 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45352 ceph-osd ceph rtd unknown /proc/44096/task/45352/root (readlink: Permission denied) 2024-06-03T03:27:58.643 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45352 ceph-osd ceph txt unknown /proc/44096/task/45352/exe (readlink: Permission denied) 2024-06-03T03:27:58.643 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45352 ceph-osd ceph NOFD /proc/44096/task/45352/fd (opendir: Permission denied) 2024-06-03T03:27:58.643 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45353 osd_srv_a ceph cwd unknown /proc/44096/task/45353/cwd (readlink: Permission denied) 2024-06-03T03:27:58.643 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45353 osd_srv_a ceph rtd unknown /proc/44096/task/45353/root (readlink: Permission denied) 2024-06-03T03:27:58.643 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45353 osd_srv_a ceph txt unknown /proc/44096/task/45353/exe (readlink: Permission denied) 2024-06-03T03:27:58.643 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45353 osd_srv_a ceph NOFD /proc/44096/task/45353/fd (opendir: Permission denied) 2024-06-03T03:27:58.643 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45354 tp_osd_tp ceph cwd unknown /proc/44096/task/45354/cwd (readlink: Permission denied) 2024-06-03T03:27:58.644 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45354 tp_osd_tp ceph rtd unknown /proc/44096/task/45354/root (readlink: Permission denied) 2024-06-03T03:27:58.644 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45354 tp_osd_tp ceph txt unknown /proc/44096/task/45354/exe (readlink: Permission denied) 2024-06-03T03:27:58.644 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45354 tp_osd_tp ceph NOFD /proc/44096/task/45354/fd (opendir: Permission denied) 2024-06-03T03:27:58.644 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45355 tp_osd_tp ceph cwd unknown /proc/44096/task/45355/cwd (readlink: Permission denied) 2024-06-03T03:27:58.644 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45355 tp_osd_tp ceph rtd unknown /proc/44096/task/45355/root (readlink: Permission denied) 2024-06-03T03:27:58.644 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45355 tp_osd_tp ceph txt unknown /proc/44096/task/45355/exe (readlink: Permission denied) 2024-06-03T03:27:58.644 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45355 tp_osd_tp ceph NOFD /proc/44096/task/45355/fd (opendir: Permission denied) 2024-06-03T03:27:58.644 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45356 tp_osd_tp ceph cwd unknown /proc/44096/task/45356/cwd (readlink: Permission denied) 2024-06-03T03:27:58.644 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45356 tp_osd_tp ceph rtd unknown /proc/44096/task/45356/root (readlink: Permission denied) 2024-06-03T03:27:58.644 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45356 tp_osd_tp ceph txt unknown /proc/44096/task/45356/exe (readlink: Permission denied) 2024-06-03T03:27:58.644 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45356 tp_osd_tp ceph NOFD /proc/44096/task/45356/fd (opendir: Permission denied) 2024-06-03T03:27:58.644 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45357 tp_osd_tp ceph cwd unknown /proc/44096/task/45357/cwd (readlink: Permission denied) 2024-06-03T03:27:58.644 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45357 tp_osd_tp ceph rtd unknown /proc/44096/task/45357/root (readlink: Permission denied) 2024-06-03T03:27:58.645 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45357 tp_osd_tp ceph txt unknown /proc/44096/task/45357/exe (readlink: Permission denied) 2024-06-03T03:27:58.645 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45357 tp_osd_tp ceph NOFD /proc/44096/task/45357/fd (opendir: Permission denied) 2024-06-03T03:27:58.645 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45358 tp_osd_tp ceph cwd unknown /proc/44096/task/45358/cwd (readlink: Permission denied) 2024-06-03T03:27:58.645 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45358 tp_osd_tp ceph rtd unknown /proc/44096/task/45358/root (readlink: Permission denied) 2024-06-03T03:27:58.645 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45358 tp_osd_tp ceph txt unknown /proc/44096/task/45358/exe (readlink: Permission denied) 2024-06-03T03:27:58.645 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45358 tp_osd_tp ceph NOFD /proc/44096/task/45358/fd (opendir: Permission denied) 2024-06-03T03:27:58.645 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45359 tp_osd_tp ceph cwd unknown /proc/44096/task/45359/cwd (readlink: Permission denied) 2024-06-03T03:27:58.645 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45359 tp_osd_tp ceph rtd unknown /proc/44096/task/45359/root (readlink: Permission denied) 2024-06-03T03:27:58.645 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45359 tp_osd_tp ceph txt unknown /proc/44096/task/45359/exe (readlink: Permission denied) 2024-06-03T03:27:58.645 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45359 tp_osd_tp ceph NOFD /proc/44096/task/45359/fd (opendir: Permission denied) 2024-06-03T03:27:58.645 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45360 tp_osd_tp ceph cwd unknown /proc/44096/task/45360/cwd (readlink: Permission denied) 2024-06-03T03:27:58.645 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45360 tp_osd_tp ceph rtd unknown /proc/44096/task/45360/root (readlink: Permission denied) 2024-06-03T03:27:58.645 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45360 tp_osd_tp ceph txt unknown /proc/44096/task/45360/exe (readlink: Permission denied) 2024-06-03T03:27:58.645 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45360 tp_osd_tp ceph NOFD /proc/44096/task/45360/fd (opendir: Permission denied) 2024-06-03T03:27:58.646 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45361 tp_osd_tp ceph cwd unknown /proc/44096/task/45361/cwd (readlink: Permission denied) 2024-06-03T03:27:58.646 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45361 tp_osd_tp ceph rtd unknown /proc/44096/task/45361/root (readlink: Permission denied) 2024-06-03T03:27:58.646 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45361 tp_osd_tp ceph txt unknown /proc/44096/task/45361/exe (readlink: Permission denied) 2024-06-03T03:27:58.646 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45361 tp_osd_tp ceph NOFD /proc/44096/task/45361/fd (opendir: Permission denied) 2024-06-03T03:27:58.646 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45362 tp_osd_tp ceph cwd unknown /proc/44096/task/45362/cwd (readlink: Permission denied) 2024-06-03T03:27:58.646 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45362 tp_osd_tp ceph rtd unknown /proc/44096/task/45362/root (readlink: Permission denied) 2024-06-03T03:27:58.646 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45362 tp_osd_tp ceph txt unknown /proc/44096/task/45362/exe (readlink: Permission denied) 2024-06-03T03:27:58.646 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45362 tp_osd_tp ceph NOFD /proc/44096/task/45362/fd (opendir: Permission denied) 2024-06-03T03:27:58.646 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45363 tp_osd_tp ceph cwd unknown /proc/44096/task/45363/cwd (readlink: Permission denied) 2024-06-03T03:27:58.646 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45363 tp_osd_tp ceph rtd unknown /proc/44096/task/45363/root (readlink: Permission denied) 2024-06-03T03:27:58.646 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45363 tp_osd_tp ceph txt unknown /proc/44096/task/45363/exe (readlink: Permission denied) 2024-06-03T03:27:58.646 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45363 tp_osd_tp ceph NOFD /proc/44096/task/45363/fd (opendir: Permission denied) 2024-06-03T03:27:58.646 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45364 tp_osd_tp ceph cwd unknown /proc/44096/task/45364/cwd (readlink: Permission denied) 2024-06-03T03:27:58.647 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45364 tp_osd_tp ceph rtd unknown /proc/44096/task/45364/root (readlink: Permission denied) 2024-06-03T03:27:58.647 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45364 tp_osd_tp ceph txt unknown /proc/44096/task/45364/exe (readlink: Permission denied) 2024-06-03T03:27:58.647 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45364 tp_osd_tp ceph NOFD /proc/44096/task/45364/fd (opendir: Permission denied) 2024-06-03T03:27:58.647 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45365 tp_osd_tp ceph cwd unknown /proc/44096/task/45365/cwd (readlink: Permission denied) 2024-06-03T03:27:58.647 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45365 tp_osd_tp ceph rtd unknown /proc/44096/task/45365/root (readlink: Permission denied) 2024-06-03T03:27:58.647 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45365 tp_osd_tp ceph txt unknown /proc/44096/task/45365/exe (readlink: Permission denied) 2024-06-03T03:27:58.647 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45365 tp_osd_tp ceph NOFD /proc/44096/task/45365/fd (opendir: Permission denied) 2024-06-03T03:27:58.647 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45366 tp_osd_tp ceph cwd unknown /proc/44096/task/45366/cwd (readlink: Permission denied) 2024-06-03T03:27:58.647 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45366 tp_osd_tp ceph rtd unknown /proc/44096/task/45366/root (readlink: Permission denied) 2024-06-03T03:27:58.647 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45366 tp_osd_tp ceph txt unknown /proc/44096/task/45366/exe (readlink: Permission denied) 2024-06-03T03:27:58.647 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45366 tp_osd_tp ceph NOFD /proc/44096/task/45366/fd (opendir: Permission denied) 2024-06-03T03:27:58.647 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45367 tp_osd_tp ceph cwd unknown /proc/44096/task/45367/cwd (readlink: Permission denied) 2024-06-03T03:27:58.647 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45367 tp_osd_tp ceph rtd unknown /proc/44096/task/45367/root (readlink: Permission denied) 2024-06-03T03:27:58.648 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45367 tp_osd_tp ceph txt unknown /proc/44096/task/45367/exe (readlink: Permission denied) 2024-06-03T03:27:58.648 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45367 tp_osd_tp ceph NOFD /proc/44096/task/45367/fd (opendir: Permission denied) 2024-06-03T03:27:58.648 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45368 tp_osd_tp ceph cwd unknown /proc/44096/task/45368/cwd (readlink: Permission denied) 2024-06-03T03:27:58.648 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45368 tp_osd_tp ceph rtd unknown /proc/44096/task/45368/root (readlink: Permission denied) 2024-06-03T03:27:58.648 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45368 tp_osd_tp ceph txt unknown /proc/44096/task/45368/exe (readlink: Permission denied) 2024-06-03T03:27:58.648 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45368 tp_osd_tp ceph NOFD /proc/44096/task/45368/fd (opendir: Permission denied) 2024-06-03T03:27:58.648 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45369 tp_osd_tp ceph cwd unknown /proc/44096/task/45369/cwd (readlink: Permission denied) 2024-06-03T03:27:58.648 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45369 tp_osd_tp ceph rtd unknown /proc/44096/task/45369/root (readlink: Permission denied) 2024-06-03T03:27:58.648 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45369 tp_osd_tp ceph txt unknown /proc/44096/task/45369/exe (readlink: Permission denied) 2024-06-03T03:27:58.648 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45369 tp_osd_tp ceph NOFD /proc/44096/task/45369/fd (opendir: Permission denied) 2024-06-03T03:27:58.648 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45370 osd_srv_h ceph cwd unknown /proc/44096/task/45370/cwd (readlink: Permission denied) 2024-06-03T03:27:58.648 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45370 osd_srv_h ceph rtd unknown /proc/44096/task/45370/root (readlink: Permission denied) 2024-06-03T03:27:58.648 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45370 osd_srv_h ceph txt unknown /proc/44096/task/45370/exe (readlink: Permission denied) 2024-06-03T03:27:58.649 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 44096 45370 osd_srv_h ceph NOFD /proc/44096/task/45370/fd (opendir: Permission denied) 2024-06-03T03:27:58.649 INFO:tasks.workunit.client.0.smithi117.stdout:sudo 45488 root cwd unknown /proc/45488/cwd (readlink: Permission denied) 2024-06-03T03:27:58.649 INFO:tasks.workunit.client.0.smithi117.stdout:sudo 45488 root rtd unknown /proc/45488/root (readlink: Permission denied) 2024-06-03T03:27:58.649 INFO:tasks.workunit.client.0.smithi117.stdout:sudo 45488 root txt unknown /proc/45488/exe (readlink: Permission denied) 2024-06-03T03:27:58.649 INFO:tasks.workunit.client.0.smithi117.stdout:sudo 45488 root NOFD /proc/45488/fd (opendir: Permission denied) 2024-06-03T03:27:58.649 INFO:tasks.workunit.client.0.smithi117.stdout:journalct 45519 root cwd unknown /proc/45519/cwd (readlink: Permission denied) 2024-06-03T03:27:58.649 INFO:tasks.workunit.client.0.smithi117.stdout:journalct 45519 root rtd unknown /proc/45519/root (readlink: Permission denied) 2024-06-03T03:27:58.649 INFO:tasks.workunit.client.0.smithi117.stdout:journalct 45519 root txt unknown /proc/45519/exe (readlink: Permission denied) 2024-06-03T03:27:58.649 INFO:tasks.workunit.client.0.smithi117.stdout:journalct 45519 root NOFD /proc/45519/fd (opendir: Permission denied) 2024-06-03T03:27:58.649 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/1 49056 root cwd unknown /proc/49056/cwd (readlink: Permission denied) 2024-06-03T03:27:58.649 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/1 49056 root rtd unknown /proc/49056/root (readlink: Permission denied) 2024-06-03T03:27:58.649 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/1 49056 root txt unknown /proc/49056/exe (readlink: Permission denied) 2024-06-03T03:27:58.649 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/1 49056 root NOFD /proc/49056/fd (opendir: Permission denied) 2024-06-03T03:27:58.649 INFO:tasks.workunit.client.0.smithi117.stdout:conmon 51024 root cwd unknown /proc/51024/cwd (readlink: Permission denied) 2024-06-03T03:27:58.650 INFO:tasks.workunit.client.0.smithi117.stdout:conmon 51024 root rtd unknown /proc/51024/root (readlink: Permission denied) 2024-06-03T03:27:58.650 INFO:tasks.workunit.client.0.smithi117.stdout:conmon 51024 root txt unknown /proc/51024/exe (readlink: Permission denied) 2024-06-03T03:27:58.650 INFO:tasks.workunit.client.0.smithi117.stdout:conmon 51024 root NOFD /proc/51024/fd (opendir: Permission denied) 2024-06-03T03:27:58.650 INFO:tasks.workunit.client.0.smithi117.stdout:podman-in 51026 root cwd unknown /proc/51026/cwd (readlink: Permission denied) 2024-06-03T03:27:58.650 INFO:tasks.workunit.client.0.smithi117.stdout:podman-in 51026 root rtd unknown /proc/51026/root (readlink: Permission denied) 2024-06-03T03:27:58.650 INFO:tasks.workunit.client.0.smithi117.stdout:podman-in 51026 root txt unknown /proc/51026/exe (readlink: Permission denied) 2024-06-03T03:27:58.650 INFO:tasks.workunit.client.0.smithi117.stdout:podman-in 51026 root NOFD /proc/51026/fd (opendir: Permission denied) 2024-06-03T03:27:58.650 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 ceph cwd unknown /proc/51028/cwd (readlink: Permission denied) 2024-06-03T03:27:58.650 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 ceph rtd unknown /proc/51028/root (readlink: Permission denied) 2024-06-03T03:27:58.650 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 ceph txt unknown /proc/51028/exe (readlink: Permission denied) 2024-06-03T03:27:58.651 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 ceph NOFD /proc/51028/fd (opendir: Permission denied) 2024-06-03T03:27:58.651 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51029 log ceph cwd unknown /proc/51028/task/51029/cwd (readlink: Permission denied) 2024-06-03T03:27:58.651 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51029 log ceph rtd unknown /proc/51028/task/51029/root (readlink: Permission denied) 2024-06-03T03:27:58.651 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51029 log ceph txt unknown /proc/51028/task/51029/exe (readlink: Permission denied) 2024-06-03T03:27:58.651 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51029 log ceph NOFD /proc/51028/task/51029/fd (opendir: Permission denied) 2024-06-03T03:27:58.651 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51031 msgr-work ceph cwd unknown /proc/51028/task/51031/cwd (readlink: Permission denied) 2024-06-03T03:27:58.651 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51031 msgr-work ceph rtd unknown /proc/51028/task/51031/root (readlink: Permission denied) 2024-06-03T03:27:58.651 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51031 msgr-work ceph txt unknown /proc/51028/task/51031/exe (readlink: Permission denied) 2024-06-03T03:27:58.651 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51031 msgr-work ceph NOFD /proc/51028/task/51031/fd (opendir: Permission denied) 2024-06-03T03:27:58.651 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51032 msgr-work ceph cwd unknown /proc/51028/task/51032/cwd (readlink: Permission denied) 2024-06-03T03:27:58.651 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51032 msgr-work ceph rtd unknown /proc/51028/task/51032/root (readlink: Permission denied) 2024-06-03T03:27:58.651 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51032 msgr-work ceph txt unknown /proc/51028/task/51032/exe (readlink: Permission denied) 2024-06-03T03:27:58.651 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51032 msgr-work ceph NOFD /proc/51028/task/51032/fd (opendir: Permission denied) 2024-06-03T03:27:58.651 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51033 msgr-work ceph cwd unknown /proc/51028/task/51033/cwd (readlink: Permission denied) 2024-06-03T03:27:58.652 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51033 msgr-work ceph rtd unknown /proc/51028/task/51033/root (readlink: Permission denied) 2024-06-03T03:27:58.652 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51033 msgr-work ceph txt unknown /proc/51028/task/51033/exe (readlink: Permission denied) 2024-06-03T03:27:58.652 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51033 msgr-work ceph NOFD /proc/51028/task/51033/fd (opendir: Permission denied) 2024-06-03T03:27:58.652 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51037 service ceph cwd unknown /proc/51028/task/51037/cwd (readlink: Permission denied) 2024-06-03T03:27:58.652 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51037 service ceph rtd unknown /proc/51028/task/51037/root (readlink: Permission denied) 2024-06-03T03:27:58.652 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51037 service ceph txt unknown /proc/51028/task/51037/exe (readlink: Permission denied) 2024-06-03T03:27:58.652 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51037 service ceph NOFD /proc/51028/task/51037/fd (opendir: Permission denied) 2024-06-03T03:27:58.652 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51038 admin_soc ceph cwd unknown /proc/51028/task/51038/cwd (readlink: Permission denied) 2024-06-03T03:27:58.652 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51038 admin_soc ceph rtd unknown /proc/51028/task/51038/root (readlink: Permission denied) 2024-06-03T03:27:58.652 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51038 admin_soc ceph txt unknown /proc/51028/task/51038/exe (readlink: Permission denied) 2024-06-03T03:27:58.652 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51038 admin_soc ceph NOFD /proc/51028/task/51038/fd (opendir: Permission denied) 2024-06-03T03:27:58.652 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51181 signal_ha ceph cwd unknown /proc/51028/task/51181/cwd (readlink: Permission denied) 2024-06-03T03:27:58.652 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51181 signal_ha ceph rtd unknown /proc/51028/task/51181/root (readlink: Permission denied) 2024-06-03T03:27:58.653 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51181 signal_ha ceph txt unknown /proc/51028/task/51181/exe (readlink: Permission denied) 2024-06-03T03:27:58.653 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51181 signal_ha ceph NOFD /proc/51028/task/51181/fd (opendir: Permission denied) 2024-06-03T03:27:58.653 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51182 io_contex ceph cwd unknown /proc/51028/task/51182/cwd (readlink: Permission denied) 2024-06-03T03:27:58.653 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51182 io_contex ceph rtd unknown /proc/51028/task/51182/root (readlink: Permission denied) 2024-06-03T03:27:58.653 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51182 io_contex ceph txt unknown /proc/51028/task/51182/exe (readlink: Permission denied) 2024-06-03T03:27:58.653 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51182 io_contex ceph NOFD /proc/51028/task/51182/fd (opendir: Permission denied) 2024-06-03T03:27:58.653 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51183 io_contex ceph cwd unknown /proc/51028/task/51183/cwd (readlink: Permission denied) 2024-06-03T03:27:58.653 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51183 io_contex ceph rtd unknown /proc/51028/task/51183/root (readlink: Permission denied) 2024-06-03T03:27:58.653 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51183 io_contex ceph txt unknown /proc/51028/task/51183/exe (readlink: Permission denied) 2024-06-03T03:27:58.653 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51183 io_contex ceph NOFD /proc/51028/task/51183/fd (opendir: Permission denied) 2024-06-03T03:27:58.653 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51188 OpHistory ceph cwd unknown /proc/51028/task/51188/cwd (readlink: Permission denied) 2024-06-03T03:27:58.653 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51188 OpHistory ceph rtd unknown /proc/51028/task/51188/root (readlink: Permission denied) 2024-06-03T03:27:58.653 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51188 OpHistory ceph txt unknown /proc/51028/task/51188/exe (readlink: Permission denied) 2024-06-03T03:27:58.654 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51188 OpHistory ceph NOFD /proc/51028/task/51188/fd (opendir: Permission denied) 2024-06-03T03:27:58.654 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51189 ceph_time ceph cwd unknown /proc/51028/task/51189/cwd (readlink: Permission denied) 2024-06-03T03:27:58.654 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51189 ceph_time ceph rtd unknown /proc/51028/task/51189/root (readlink: Permission denied) 2024-06-03T03:27:58.654 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51189 ceph_time ceph txt unknown /proc/51028/task/51189/exe (readlink: Permission denied) 2024-06-03T03:27:58.654 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51189 ceph_time ceph NOFD /proc/51028/task/51189/fd (opendir: Permission denied) 2024-06-03T03:27:58.654 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51406 ceph-osd ceph cwd unknown /proc/51028/task/51406/cwd (readlink: Permission denied) 2024-06-03T03:27:58.654 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51406 ceph-osd ceph rtd unknown /proc/51028/task/51406/root (readlink: Permission denied) 2024-06-03T03:27:58.654 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51406 ceph-osd ceph txt unknown /proc/51028/task/51406/exe (readlink: Permission denied) 2024-06-03T03:27:58.654 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51406 ceph-osd ceph NOFD /proc/51028/task/51406/fd (opendir: Permission denied) 2024-06-03T03:27:58.654 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51407 safe_time ceph cwd unknown /proc/51028/task/51407/cwd (readlink: Permission denied) 2024-06-03T03:27:58.654 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51407 safe_time ceph rtd unknown /proc/51028/task/51407/root (readlink: Permission denied) 2024-06-03T03:27:58.655 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51407 safe_time ceph txt unknown /proc/51028/task/51407/exe (readlink: Permission denied) 2024-06-03T03:27:58.655 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51407 safe_time ceph NOFD /proc/51028/task/51407/fd (opendir: Permission denied) 2024-06-03T03:27:58.655 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51408 safe_time ceph cwd unknown /proc/51028/task/51408/cwd (readlink: Permission denied) 2024-06-03T03:27:58.655 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51408 safe_time ceph rtd unknown /proc/51028/task/51408/root (readlink: Permission denied) 2024-06-03T03:27:58.655 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51408 safe_time ceph txt unknown /proc/51028/task/51408/exe (readlink: Permission denied) 2024-06-03T03:27:58.655 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51408 safe_time ceph NOFD /proc/51028/task/51408/fd (opendir: Permission denied) 2024-06-03T03:27:58.655 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51409 safe_time ceph cwd unknown /proc/51028/task/51409/cwd (readlink: Permission denied) 2024-06-03T03:27:58.655 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51409 safe_time ceph rtd unknown /proc/51028/task/51409/root (readlink: Permission denied) 2024-06-03T03:27:58.655 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51409 safe_time ceph txt unknown /proc/51028/task/51409/exe (readlink: Permission denied) 2024-06-03T03:27:58.655 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51409 safe_time ceph NOFD /proc/51028/task/51409/fd (opendir: Permission denied) 2024-06-03T03:27:58.655 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51410 safe_time ceph cwd unknown /proc/51028/task/51410/cwd (readlink: Permission denied) 2024-06-03T03:27:58.655 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51410 safe_time ceph rtd unknown /proc/51028/task/51410/root (readlink: Permission denied) 2024-06-03T03:27:58.655 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51410 safe_time ceph txt unknown /proc/51028/task/51410/exe (readlink: Permission denied) 2024-06-03T03:27:58.656 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51410 safe_time ceph NOFD /proc/51028/task/51410/fd (opendir: Permission denied) 2024-06-03T03:27:58.656 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51411 fn_anonym ceph cwd unknown /proc/51028/task/51411/cwd (readlink: Permission denied) 2024-06-03T03:27:58.656 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51411 fn_anonym ceph rtd unknown /proc/51028/task/51411/root (readlink: Permission denied) 2024-06-03T03:27:58.656 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51411 fn_anonym ceph txt unknown /proc/51028/task/51411/exe (readlink: Permission denied) 2024-06-03T03:27:58.656 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51411 fn_anonym ceph NOFD /proc/51028/task/51411/fd (opendir: Permission denied) 2024-06-03T03:27:58.656 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51416 rocksdb:l ceph cwd unknown /proc/51028/task/51416/cwd (readlink: Permission denied) 2024-06-03T03:27:58.656 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51416 rocksdb:l ceph rtd unknown /proc/51028/task/51416/root (readlink: Permission denied) 2024-06-03T03:27:58.656 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51416 rocksdb:l ceph txt unknown /proc/51028/task/51416/exe (readlink: Permission denied) 2024-06-03T03:27:58.656 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51416 rocksdb:l ceph NOFD /proc/51028/task/51416/fd (opendir: Permission denied) 2024-06-03T03:27:58.657 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51417 rocksdb:l ceph cwd unknown /proc/51028/task/51417/cwd (readlink: Permission denied) 2024-06-03T03:27:58.657 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51417 rocksdb:l ceph rtd unknown /proc/51028/task/51417/root (readlink: Permission denied) 2024-06-03T03:27:58.657 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51417 rocksdb:l ceph txt unknown /proc/51028/task/51417/exe (readlink: Permission denied) 2024-06-03T03:27:58.657 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51417 rocksdb:l ceph NOFD /proc/51028/task/51417/fd (opendir: Permission denied) 2024-06-03T03:27:58.657 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51418 rocksdb:l ceph cwd unknown /proc/51028/task/51418/cwd (readlink: Permission denied) 2024-06-03T03:27:58.657 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51418 rocksdb:l ceph rtd unknown /proc/51028/task/51418/root (readlink: Permission denied) 2024-06-03T03:27:58.657 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51418 rocksdb:l ceph txt unknown /proc/51028/task/51418/exe (readlink: Permission denied) 2024-06-03T03:27:58.657 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51418 rocksdb:l ceph NOFD /proc/51028/task/51418/fd (opendir: Permission denied) 2024-06-03T03:27:58.657 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51419 rocksdb:h ceph cwd unknown /proc/51028/task/51419/cwd (readlink: Permission denied) 2024-06-03T03:27:58.657 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51419 rocksdb:h ceph rtd unknown /proc/51028/task/51419/root (readlink: Permission denied) 2024-06-03T03:27:58.657 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51419 rocksdb:h ceph txt unknown /proc/51028/task/51419/exe (readlink: Permission denied) 2024-06-03T03:27:58.657 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51419 rocksdb:h ceph NOFD /proc/51028/task/51419/fd (opendir: Permission denied) 2024-06-03T03:27:58.658 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51842 bstore_ai ceph cwd unknown /proc/51028/task/51842/cwd (readlink: Permission denied) 2024-06-03T03:27:58.658 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51842 bstore_ai ceph rtd unknown /proc/51028/task/51842/root (readlink: Permission denied) 2024-06-03T03:27:58.658 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51842 bstore_ai ceph txt unknown /proc/51028/task/51842/exe (readlink: Permission denied) 2024-06-03T03:27:58.658 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 51842 bstore_ai ceph NOFD /proc/51028/task/51842/fd (opendir: Permission denied) 2024-06-03T03:27:58.658 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52112 bstore_ai ceph cwd unknown /proc/51028/task/52112/cwd (readlink: Permission denied) 2024-06-03T03:27:58.658 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52112 bstore_ai ceph rtd unknown /proc/51028/task/52112/root (readlink: Permission denied) 2024-06-03T03:27:58.658 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52112 bstore_ai ceph txt unknown /proc/51028/task/52112/exe (readlink: Permission denied) 2024-06-03T03:27:58.658 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52112 bstore_ai ceph NOFD /proc/51028/task/52112/fd (opendir: Permission denied) 2024-06-03T03:27:58.658 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52301 ceph-osd ceph cwd unknown /proc/51028/task/52301/cwd (readlink: Permission denied) 2024-06-03T03:27:58.658 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52301 ceph-osd ceph rtd unknown /proc/51028/task/52301/root (readlink: Permission denied) 2024-06-03T03:27:58.658 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52301 ceph-osd ceph txt unknown /proc/51028/task/52301/exe (readlink: Permission denied) 2024-06-03T03:27:58.658 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52301 ceph-osd ceph NOFD /proc/51028/task/52301/fd (opendir: Permission denied) 2024-06-03T03:27:58.658 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52302 cfin ceph cwd unknown /proc/51028/task/52302/cwd (readlink: Permission denied) 2024-06-03T03:27:58.659 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52302 cfin ceph rtd unknown /proc/51028/task/52302/root (readlink: Permission denied) 2024-06-03T03:27:58.659 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52302 cfin ceph txt unknown /proc/51028/task/52302/exe (readlink: Permission denied) 2024-06-03T03:27:58.659 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52302 cfin ceph NOFD /proc/51028/task/52302/fd (opendir: Permission denied) 2024-06-03T03:27:58.659 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52303 bstore_kv ceph cwd unknown /proc/51028/task/52303/cwd (readlink: Permission denied) 2024-06-03T03:27:58.659 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52303 bstore_kv ceph rtd unknown /proc/51028/task/52303/root (readlink: Permission denied) 2024-06-03T03:27:58.659 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52303 bstore_kv ceph txt unknown /proc/51028/task/52303/exe (readlink: Permission denied) 2024-06-03T03:27:58.659 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52303 bstore_kv ceph NOFD /proc/51028/task/52303/fd (opendir: Permission denied) 2024-06-03T03:27:58.659 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52304 bstore_kv ceph cwd unknown /proc/51028/task/52304/cwd (readlink: Permission denied) 2024-06-03T03:27:58.659 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52304 bstore_kv ceph rtd unknown /proc/51028/task/52304/root (readlink: Permission denied) 2024-06-03T03:27:58.659 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52304 bstore_kv ceph txt unknown /proc/51028/task/52304/exe (readlink: Permission denied) 2024-06-03T03:27:58.659 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52304 bstore_kv ceph NOFD /proc/51028/task/52304/fd (opendir: Permission denied) 2024-06-03T03:27:58.659 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52305 bstore_me ceph cwd unknown /proc/51028/task/52305/cwd (readlink: Permission denied) 2024-06-03T03:27:58.659 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52305 bstore_me ceph rtd unknown /proc/51028/task/52305/root (readlink: Permission denied) 2024-06-03T03:27:58.659 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52305 bstore_me ceph txt unknown /proc/51028/task/52305/exe (readlink: Permission denied) 2024-06-03T03:27:58.660 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52305 bstore_me ceph NOFD /proc/51028/task/52305/fd (opendir: Permission denied) 2024-06-03T03:27:58.660 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52308 ms_dispat ceph cwd unknown /proc/51028/task/52308/cwd (readlink: Permission denied) 2024-06-03T03:27:58.660 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52308 ms_dispat ceph rtd unknown /proc/51028/task/52308/root (readlink: Permission denied) 2024-06-03T03:27:58.660 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52308 ms_dispat ceph txt unknown /proc/51028/task/52308/exe (readlink: Permission denied) 2024-06-03T03:27:58.660 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52308 ms_dispat ceph NOFD /proc/51028/task/52308/fd (opendir: Permission denied) 2024-06-03T03:27:58.660 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52309 ms_local ceph cwd unknown /proc/51028/task/52309/cwd (readlink: Permission denied) 2024-06-03T03:27:58.660 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52309 ms_local ceph rtd unknown /proc/51028/task/52309/root (readlink: Permission denied) 2024-06-03T03:27:58.660 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52309 ms_local ceph txt unknown /proc/51028/task/52309/exe (readlink: Permission denied) 2024-06-03T03:27:58.660 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52309 ms_local ceph NOFD /proc/51028/task/52309/fd (opendir: Permission denied) 2024-06-03T03:27:58.660 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52310 safe_time ceph cwd unknown /proc/51028/task/52310/cwd (readlink: Permission denied) 2024-06-03T03:27:58.660 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52310 safe_time ceph rtd unknown /proc/51028/task/52310/root (readlink: Permission denied) 2024-06-03T03:27:58.660 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52310 safe_time ceph txt unknown /proc/51028/task/52310/exe (readlink: Permission denied) 2024-06-03T03:27:58.660 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52310 safe_time ceph NOFD /proc/51028/task/52310/fd (opendir: Permission denied) 2024-06-03T03:27:58.661 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52311 safe_time ceph cwd unknown /proc/51028/task/52311/cwd (readlink: Permission denied) 2024-06-03T03:27:58.661 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52311 safe_time ceph rtd unknown /proc/51028/task/52311/root (readlink: Permission denied) 2024-06-03T03:27:58.661 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52311 safe_time ceph txt unknown /proc/51028/task/52311/exe (readlink: Permission denied) 2024-06-03T03:27:58.661 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52311 safe_time ceph NOFD /proc/51028/task/52311/fd (opendir: Permission denied) 2024-06-03T03:27:58.661 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52313 ms_dispat ceph cwd unknown /proc/51028/task/52313/cwd (readlink: Permission denied) 2024-06-03T03:27:58.661 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52313 ms_dispat ceph rtd unknown /proc/51028/task/52313/root (readlink: Permission denied) 2024-06-03T03:27:58.661 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52313 ms_dispat ceph txt unknown /proc/51028/task/52313/exe (readlink: Permission denied) 2024-06-03T03:27:58.661 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52313 ms_dispat ceph NOFD /proc/51028/task/52313/fd (opendir: Permission denied) 2024-06-03T03:27:58.661 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52314 ms_local ceph cwd unknown /proc/51028/task/52314/cwd (readlink: Permission denied) 2024-06-03T03:27:58.661 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52314 ms_local ceph rtd unknown /proc/51028/task/52314/root (readlink: Permission denied) 2024-06-03T03:27:58.661 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52314 ms_local ceph txt unknown /proc/51028/task/52314/exe (readlink: Permission denied) 2024-06-03T03:27:58.662 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52314 ms_local ceph NOFD /proc/51028/task/52314/fd (opendir: Permission denied) 2024-06-03T03:27:58.662 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52315 ms_dispat ceph cwd unknown /proc/51028/task/52315/cwd (readlink: Permission denied) 2024-06-03T03:27:58.662 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52315 ms_dispat ceph rtd unknown /proc/51028/task/52315/root (readlink: Permission denied) 2024-06-03T03:27:58.662 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52315 ms_dispat ceph txt unknown /proc/51028/task/52315/exe (readlink: Permission denied) 2024-06-03T03:27:58.662 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52315 ms_dispat ceph NOFD /proc/51028/task/52315/fd (opendir: Permission denied) 2024-06-03T03:27:58.662 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52316 ms_local ceph cwd unknown /proc/51028/task/52316/cwd (readlink: Permission denied) 2024-06-03T03:27:58.662 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52316 ms_local ceph rtd unknown /proc/51028/task/52316/root (readlink: Permission denied) 2024-06-03T03:27:58.662 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52316 ms_local ceph txt unknown /proc/51028/task/52316/exe (readlink: Permission denied) 2024-06-03T03:27:58.662 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52316 ms_local ceph NOFD /proc/51028/task/52316/fd (opendir: Permission denied) 2024-06-03T03:27:58.662 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52317 ms_dispat ceph cwd unknown /proc/51028/task/52317/cwd (readlink: Permission denied) 2024-06-03T03:27:58.662 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52317 ms_dispat ceph rtd unknown /proc/51028/task/52317/root (readlink: Permission denied) 2024-06-03T03:27:58.662 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52317 ms_dispat ceph txt unknown /proc/51028/task/52317/exe (readlink: Permission denied) 2024-06-03T03:27:58.662 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52317 ms_dispat ceph NOFD /proc/51028/task/52317/fd (opendir: Permission denied) 2024-06-03T03:27:58.663 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52318 ms_local ceph cwd unknown /proc/51028/task/52318/cwd (readlink: Permission denied) 2024-06-03T03:27:58.663 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52318 ms_local ceph rtd unknown /proc/51028/task/52318/root (readlink: Permission denied) 2024-06-03T03:27:58.663 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52318 ms_local ceph txt unknown /proc/51028/task/52318/exe (readlink: Permission denied) 2024-06-03T03:27:58.663 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52318 ms_local ceph NOFD /proc/51028/task/52318/fd (opendir: Permission denied) 2024-06-03T03:27:58.663 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52319 ms_dispat ceph cwd unknown /proc/51028/task/52319/cwd (readlink: Permission denied) 2024-06-03T03:27:58.663 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52319 ms_dispat ceph rtd unknown /proc/51028/task/52319/root (readlink: Permission denied) 2024-06-03T03:27:58.663 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52319 ms_dispat ceph txt unknown /proc/51028/task/52319/exe (readlink: Permission denied) 2024-06-03T03:27:58.663 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52319 ms_dispat ceph NOFD /proc/51028/task/52319/fd (opendir: Permission denied) 2024-06-03T03:27:58.663 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52320 ms_local ceph cwd unknown /proc/51028/task/52320/cwd (readlink: Permission denied) 2024-06-03T03:27:58.663 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52320 ms_local ceph rtd unknown /proc/51028/task/52320/root (readlink: Permission denied) 2024-06-03T03:27:58.663 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52320 ms_local ceph txt unknown /proc/51028/task/52320/exe (readlink: Permission denied) 2024-06-03T03:27:58.663 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52320 ms_local ceph NOFD /proc/51028/task/52320/fd (opendir: Permission denied) 2024-06-03T03:27:58.663 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52321 ms_dispat ceph cwd unknown /proc/51028/task/52321/cwd (readlink: Permission denied) 2024-06-03T03:27:58.663 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52321 ms_dispat ceph rtd unknown /proc/51028/task/52321/root (readlink: Permission denied) 2024-06-03T03:27:58.664 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52321 ms_dispat ceph txt unknown /proc/51028/task/52321/exe (readlink: Permission denied) 2024-06-03T03:27:58.664 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52321 ms_dispat ceph NOFD /proc/51028/task/52321/fd (opendir: Permission denied) 2024-06-03T03:27:58.664 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52322 ms_local ceph cwd unknown /proc/51028/task/52322/cwd (readlink: Permission denied) 2024-06-03T03:27:58.664 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52322 ms_local ceph rtd unknown /proc/51028/task/52322/root (readlink: Permission denied) 2024-06-03T03:27:58.664 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52322 ms_local ceph txt unknown /proc/51028/task/52322/exe (readlink: Permission denied) 2024-06-03T03:27:58.664 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52322 ms_local ceph NOFD /proc/51028/task/52322/fd (opendir: Permission denied) 2024-06-03T03:27:58.664 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52323 ms_dispat ceph cwd unknown /proc/51028/task/52323/cwd (readlink: Permission denied) 2024-06-03T03:27:58.664 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52323 ms_dispat ceph rtd unknown /proc/51028/task/52323/root (readlink: Permission denied) 2024-06-03T03:27:58.664 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52323 ms_dispat ceph txt unknown /proc/51028/task/52323/exe (readlink: Permission denied) 2024-06-03T03:27:58.664 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52323 ms_dispat ceph NOFD /proc/51028/task/52323/fd (opendir: Permission denied) 2024-06-03T03:27:58.664 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52324 ms_local ceph cwd unknown /proc/51028/task/52324/cwd (readlink: Permission denied) 2024-06-03T03:27:58.664 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52324 ms_local ceph rtd unknown /proc/51028/task/52324/root (readlink: Permission denied) 2024-06-03T03:27:58.664 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52324 ms_local ceph txt unknown /proc/51028/task/52324/exe (readlink: Permission denied) 2024-06-03T03:27:58.665 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52324 ms_local ceph NOFD /proc/51028/task/52324/fd (opendir: Permission denied) 2024-06-03T03:27:58.665 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52325 fn_anonym ceph cwd unknown /proc/51028/task/52325/cwd (readlink: Permission denied) 2024-06-03T03:27:58.665 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52325 fn_anonym ceph rtd unknown /proc/51028/task/52325/root (readlink: Permission denied) 2024-06-03T03:27:58.665 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52325 fn_anonym ceph txt unknown /proc/51028/task/52325/exe (readlink: Permission denied) 2024-06-03T03:27:58.665 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52325 fn_anonym ceph NOFD /proc/51028/task/52325/fd (opendir: Permission denied) 2024-06-03T03:27:58.665 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52326 finisher ceph cwd unknown /proc/51028/task/52326/cwd (readlink: Permission denied) 2024-06-03T03:27:58.665 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52326 finisher ceph rtd unknown /proc/51028/task/52326/root (readlink: Permission denied) 2024-06-03T03:27:58.665 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52326 finisher ceph txt unknown /proc/51028/task/52326/exe (readlink: Permission denied) 2024-06-03T03:27:58.665 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52326 finisher ceph NOFD /proc/51028/task/52326/fd (opendir: Permission denied) 2024-06-03T03:27:58.665 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52327 safe_time ceph cwd unknown /proc/51028/task/52327/cwd (readlink: Permission denied) 2024-06-03T03:27:58.665 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52327 safe_time ceph rtd unknown /proc/51028/task/52327/root (readlink: Permission denied) 2024-06-03T03:27:58.666 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52327 safe_time ceph txt unknown /proc/51028/task/52327/exe (readlink: Permission denied) 2024-06-03T03:27:58.666 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52327 safe_time ceph NOFD /proc/51028/task/52327/fd (opendir: Permission denied) 2024-06-03T03:27:58.666 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52328 safe_time ceph cwd unknown /proc/51028/task/52328/cwd (readlink: Permission denied) 2024-06-03T03:27:58.666 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52328 safe_time ceph rtd unknown /proc/51028/task/52328/root (readlink: Permission denied) 2024-06-03T03:27:58.666 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52328 safe_time ceph txt unknown /proc/51028/task/52328/exe (readlink: Permission denied) 2024-06-03T03:27:58.666 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52328 safe_time ceph NOFD /proc/51028/task/52328/fd (opendir: Permission denied) 2024-06-03T03:27:58.666 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52329 ceph-osd ceph cwd unknown /proc/51028/task/52329/cwd (readlink: Permission denied) 2024-06-03T03:27:58.666 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52329 ceph-osd ceph rtd unknown /proc/51028/task/52329/root (readlink: Permission denied) 2024-06-03T03:27:58.666 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52329 ceph-osd ceph txt unknown /proc/51028/task/52329/exe (readlink: Permission denied) 2024-06-03T03:27:58.666 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52329 ceph-osd ceph NOFD /proc/51028/task/52329/fd (opendir: Permission denied) 2024-06-03T03:27:58.666 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52330 osd_srv_a ceph cwd unknown /proc/51028/task/52330/cwd (readlink: Permission denied) 2024-06-03T03:27:58.666 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52330 osd_srv_a ceph rtd unknown /proc/51028/task/52330/root (readlink: Permission denied) 2024-06-03T03:27:58.667 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52330 osd_srv_a ceph txt unknown /proc/51028/task/52330/exe (readlink: Permission denied) 2024-06-03T03:27:58.667 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52330 osd_srv_a ceph NOFD /proc/51028/task/52330/fd (opendir: Permission denied) 2024-06-03T03:27:58.667 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52331 tp_osd_tp ceph cwd unknown /proc/51028/task/52331/cwd (readlink: Permission denied) 2024-06-03T03:27:58.667 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52331 tp_osd_tp ceph rtd unknown /proc/51028/task/52331/root (readlink: Permission denied) 2024-06-03T03:27:58.667 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52331 tp_osd_tp ceph txt unknown /proc/51028/task/52331/exe (readlink: Permission denied) 2024-06-03T03:27:58.667 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52331 tp_osd_tp ceph NOFD /proc/51028/task/52331/fd (opendir: Permission denied) 2024-06-03T03:27:58.667 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52332 tp_osd_tp ceph cwd unknown /proc/51028/task/52332/cwd (readlink: Permission denied) 2024-06-03T03:27:58.667 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52332 tp_osd_tp ceph rtd unknown /proc/51028/task/52332/root (readlink: Permission denied) 2024-06-03T03:27:58.667 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52332 tp_osd_tp ceph txt unknown /proc/51028/task/52332/exe (readlink: Permission denied) 2024-06-03T03:27:58.667 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52332 tp_osd_tp ceph NOFD /proc/51028/task/52332/fd (opendir: Permission denied) 2024-06-03T03:27:58.667 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52333 tp_osd_tp ceph cwd unknown /proc/51028/task/52333/cwd (readlink: Permission denied) 2024-06-03T03:27:58.667 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52333 tp_osd_tp ceph rtd unknown /proc/51028/task/52333/root (readlink: Permission denied) 2024-06-03T03:27:58.667 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52333 tp_osd_tp ceph txt unknown /proc/51028/task/52333/exe (readlink: Permission denied) 2024-06-03T03:27:58.667 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52333 tp_osd_tp ceph NOFD /proc/51028/task/52333/fd (opendir: Permission denied) 2024-06-03T03:27:58.667 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52334 tp_osd_tp ceph cwd unknown /proc/51028/task/52334/cwd (readlink: Permission denied) 2024-06-03T03:27:58.668 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52334 tp_osd_tp ceph rtd unknown /proc/51028/task/52334/root (readlink: Permission denied) 2024-06-03T03:27:58.668 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52334 tp_osd_tp ceph txt unknown /proc/51028/task/52334/exe (readlink: Permission denied) 2024-06-03T03:27:58.668 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52334 tp_osd_tp ceph NOFD /proc/51028/task/52334/fd (opendir: Permission denied) 2024-06-03T03:27:58.668 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52335 tp_osd_tp ceph cwd unknown /proc/51028/task/52335/cwd (readlink: Permission denied) 2024-06-03T03:27:58.668 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52335 tp_osd_tp ceph rtd unknown /proc/51028/task/52335/root (readlink: Permission denied) 2024-06-03T03:27:58.668 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52335 tp_osd_tp ceph txt unknown /proc/51028/task/52335/exe (readlink: Permission denied) 2024-06-03T03:27:58.668 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52335 tp_osd_tp ceph NOFD /proc/51028/task/52335/fd (opendir: Permission denied) 2024-06-03T03:27:58.668 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52336 tp_osd_tp ceph cwd unknown /proc/51028/task/52336/cwd (readlink: Permission denied) 2024-06-03T03:27:58.668 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52336 tp_osd_tp ceph rtd unknown /proc/51028/task/52336/root (readlink: Permission denied) 2024-06-03T03:27:58.668 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52336 tp_osd_tp ceph txt unknown /proc/51028/task/52336/exe (readlink: Permission denied) 2024-06-03T03:27:58.668 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52336 tp_osd_tp ceph NOFD /proc/51028/task/52336/fd (opendir: Permission denied) 2024-06-03T03:27:58.669 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52337 tp_osd_tp ceph cwd unknown /proc/51028/task/52337/cwd (readlink: Permission denied) 2024-06-03T03:27:58.669 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52337 tp_osd_tp ceph rtd unknown /proc/51028/task/52337/root (readlink: Permission denied) 2024-06-03T03:27:58.669 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52337 tp_osd_tp ceph txt unknown /proc/51028/task/52337/exe (readlink: Permission denied) 2024-06-03T03:27:58.669 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52337 tp_osd_tp ceph NOFD /proc/51028/task/52337/fd (opendir: Permission denied) 2024-06-03T03:27:58.669 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52338 tp_osd_tp ceph cwd unknown /proc/51028/task/52338/cwd (readlink: Permission denied) 2024-06-03T03:27:58.669 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52338 tp_osd_tp ceph rtd unknown /proc/51028/task/52338/root (readlink: Permission denied) 2024-06-03T03:27:58.669 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52338 tp_osd_tp ceph txt unknown /proc/51028/task/52338/exe (readlink: Permission denied) 2024-06-03T03:27:58.669 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52338 tp_osd_tp ceph NOFD /proc/51028/task/52338/fd (opendir: Permission denied) 2024-06-03T03:27:58.669 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52339 tp_osd_tp ceph cwd unknown /proc/51028/task/52339/cwd (readlink: Permission denied) 2024-06-03T03:27:58.669 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52339 tp_osd_tp ceph rtd unknown /proc/51028/task/52339/root (readlink: Permission denied) 2024-06-03T03:27:58.669 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52339 tp_osd_tp ceph txt unknown /proc/51028/task/52339/exe (readlink: Permission denied) 2024-06-03T03:27:58.669 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52339 tp_osd_tp ceph NOFD /proc/51028/task/52339/fd (opendir: Permission denied) 2024-06-03T03:27:58.669 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52340 tp_osd_tp ceph cwd unknown /proc/51028/task/52340/cwd (readlink: Permission denied) 2024-06-03T03:27:58.669 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52340 tp_osd_tp ceph rtd unknown /proc/51028/task/52340/root (readlink: Permission denied) 2024-06-03T03:27:58.670 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52340 tp_osd_tp ceph txt unknown /proc/51028/task/52340/exe (readlink: Permission denied) 2024-06-03T03:27:58.670 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52340 tp_osd_tp ceph NOFD /proc/51028/task/52340/fd (opendir: Permission denied) 2024-06-03T03:27:58.670 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52341 tp_osd_tp ceph cwd unknown /proc/51028/task/52341/cwd (readlink: Permission denied) 2024-06-03T03:27:58.670 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52341 tp_osd_tp ceph rtd unknown /proc/51028/task/52341/root (readlink: Permission denied) 2024-06-03T03:27:58.670 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52341 tp_osd_tp ceph txt unknown /proc/51028/task/52341/exe (readlink: Permission denied) 2024-06-03T03:27:58.670 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52341 tp_osd_tp ceph NOFD /proc/51028/task/52341/fd (opendir: Permission denied) 2024-06-03T03:27:58.670 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52342 tp_osd_tp ceph cwd unknown /proc/51028/task/52342/cwd (readlink: Permission denied) 2024-06-03T03:27:58.670 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52342 tp_osd_tp ceph rtd unknown /proc/51028/task/52342/root (readlink: Permission denied) 2024-06-03T03:27:58.671 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52342 tp_osd_tp ceph txt unknown /proc/51028/task/52342/exe (readlink: Permission denied) 2024-06-03T03:27:58.671 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52342 tp_osd_tp ceph NOFD /proc/51028/task/52342/fd (opendir: Permission denied) 2024-06-03T03:27:58.671 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52343 tp_osd_tp ceph cwd unknown /proc/51028/task/52343/cwd (readlink: Permission denied) 2024-06-03T03:27:58.671 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52343 tp_osd_tp ceph rtd unknown /proc/51028/task/52343/root (readlink: Permission denied) 2024-06-03T03:27:58.671 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52343 tp_osd_tp ceph txt unknown /proc/51028/task/52343/exe (readlink: Permission denied) 2024-06-03T03:27:58.671 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52343 tp_osd_tp ceph NOFD /proc/51028/task/52343/fd (opendir: Permission denied) 2024-06-03T03:27:58.671 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52344 tp_osd_tp ceph cwd unknown /proc/51028/task/52344/cwd (readlink: Permission denied) 2024-06-03T03:27:58.671 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52344 tp_osd_tp ceph rtd unknown /proc/51028/task/52344/root (readlink: Permission denied) 2024-06-03T03:27:58.671 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52344 tp_osd_tp ceph txt unknown /proc/51028/task/52344/exe (readlink: Permission denied) 2024-06-03T03:27:58.671 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52344 tp_osd_tp ceph NOFD /proc/51028/task/52344/fd (opendir: Permission denied) 2024-06-03T03:27:58.671 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52345 tp_osd_tp ceph cwd unknown /proc/51028/task/52345/cwd (readlink: Permission denied) 2024-06-03T03:27:58.671 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52345 tp_osd_tp ceph rtd unknown /proc/51028/task/52345/root (readlink: Permission denied) 2024-06-03T03:27:58.671 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52345 tp_osd_tp ceph txt unknown /proc/51028/task/52345/exe (readlink: Permission denied) 2024-06-03T03:27:58.671 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52345 tp_osd_tp ceph NOFD /proc/51028/task/52345/fd (opendir: Permission denied) 2024-06-03T03:27:58.671 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52346 tp_osd_tp ceph cwd unknown /proc/51028/task/52346/cwd (readlink: Permission denied) 2024-06-03T03:27:58.672 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52346 tp_osd_tp ceph rtd unknown /proc/51028/task/52346/root (readlink: Permission denied) 2024-06-03T03:27:58.672 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52346 tp_osd_tp ceph txt unknown /proc/51028/task/52346/exe (readlink: Permission denied) 2024-06-03T03:27:58.672 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52346 tp_osd_tp ceph NOFD /proc/51028/task/52346/fd (opendir: Permission denied) 2024-06-03T03:27:58.672 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52347 osd_srv_h ceph cwd unknown /proc/51028/task/52347/cwd (readlink: Permission denied) 2024-06-03T03:27:58.672 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52347 osd_srv_h ceph rtd unknown /proc/51028/task/52347/root (readlink: Permission denied) 2024-06-03T03:27:58.672 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52347 osd_srv_h ceph txt unknown /proc/51028/task/52347/exe (readlink: Permission denied) 2024-06-03T03:27:58.672 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 51028 52347 osd_srv_h ceph NOFD /proc/51028/task/52347/fd (opendir: Permission denied) 2024-06-03T03:27:58.672 INFO:tasks.workunit.client.0.smithi117.stdout:sudo 52526 root cwd unknown /proc/52526/cwd (readlink: Permission denied) 2024-06-03T03:27:58.672 INFO:tasks.workunit.client.0.smithi117.stdout:sudo 52526 root rtd unknown /proc/52526/root (readlink: Permission denied) 2024-06-03T03:27:58.672 INFO:tasks.workunit.client.0.smithi117.stdout:sudo 52526 root txt unknown /proc/52526/exe (readlink: Permission denied) 2024-06-03T03:27:58.672 INFO:tasks.workunit.client.0.smithi117.stdout:sudo 52526 root NOFD /proc/52526/fd (opendir: Permission denied) 2024-06-03T03:27:58.672 INFO:tasks.workunit.client.0.smithi117.stdout:journalct 52556 root cwd unknown /proc/52556/cwd (readlink: Permission denied) 2024-06-03T03:27:58.672 INFO:tasks.workunit.client.0.smithi117.stdout:journalct 52556 root rtd unknown /proc/52556/root (readlink: Permission denied) 2024-06-03T03:27:58.672 INFO:tasks.workunit.client.0.smithi117.stdout:journalct 52556 root txt unknown /proc/52556/exe (readlink: Permission denied) 2024-06-03T03:27:58.672 INFO:tasks.workunit.client.0.smithi117.stdout:journalct 52556 root NOFD /proc/52556/fd (opendir: Permission denied) 2024-06-03T03:27:58.673 INFO:tasks.workunit.client.0.smithi117.stdout:conmon 57519 root cwd unknown /proc/57519/cwd (readlink: Permission denied) 2024-06-03T03:27:58.673 INFO:tasks.workunit.client.0.smithi117.stdout:conmon 57519 root rtd unknown /proc/57519/root (readlink: Permission denied) 2024-06-03T03:27:58.673 INFO:tasks.workunit.client.0.smithi117.stdout:conmon 57519 root txt unknown /proc/57519/exe (readlink: Permission denied) 2024-06-03T03:27:58.673 INFO:tasks.workunit.client.0.smithi117.stdout:conmon 57519 root NOFD /proc/57519/fd (opendir: Permission denied) 2024-06-03T03:27:58.673 INFO:tasks.workunit.client.0.smithi117.stdout:podman-in 57521 root cwd unknown /proc/57521/cwd (readlink: Permission denied) 2024-06-03T03:27:58.673 INFO:tasks.workunit.client.0.smithi117.stdout:podman-in 57521 root rtd unknown /proc/57521/root (readlink: Permission denied) 2024-06-03T03:27:58.673 INFO:tasks.workunit.client.0.smithi117.stdout:podman-in 57521 root txt unknown /proc/57521/exe (readlink: Permission denied) 2024-06-03T03:27:58.673 INFO:tasks.workunit.client.0.smithi117.stdout:podman-in 57521 root NOFD /proc/57521/fd (opendir: Permission denied) 2024-06-03T03:27:58.673 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 ceph cwd unknown /proc/57525/cwd (readlink: Permission denied) 2024-06-03T03:27:58.673 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 ceph rtd unknown /proc/57525/root (readlink: Permission denied) 2024-06-03T03:27:58.673 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 ceph txt unknown /proc/57525/exe (readlink: Permission denied) 2024-06-03T03:27:58.673 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 ceph NOFD /proc/57525/fd (opendir: Permission denied) 2024-06-03T03:27:58.673 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 57527 log ceph cwd unknown /proc/57525/task/57527/cwd (readlink: Permission denied) 2024-06-03T03:27:58.673 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 57527 log ceph rtd unknown /proc/57525/task/57527/root (readlink: Permission denied) 2024-06-03T03:27:58.673 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 57527 log ceph txt unknown /proc/57525/task/57527/exe (readlink: Permission denied) 2024-06-03T03:27:58.674 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 57527 log ceph NOFD /proc/57525/task/57527/fd (opendir: Permission denied) 2024-06-03T03:27:58.674 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 57529 msgr-work ceph cwd unknown /proc/57525/task/57529/cwd (readlink: Permission denied) 2024-06-03T03:27:58.674 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 57529 msgr-work ceph rtd unknown /proc/57525/task/57529/root (readlink: Permission denied) 2024-06-03T03:27:58.674 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 57529 msgr-work ceph txt unknown /proc/57525/task/57529/exe (readlink: Permission denied) 2024-06-03T03:27:58.674 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 57529 msgr-work ceph NOFD /proc/57525/task/57529/fd (opendir: Permission denied) 2024-06-03T03:27:58.674 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 57530 msgr-work ceph cwd unknown /proc/57525/task/57530/cwd (readlink: Permission denied) 2024-06-03T03:27:58.674 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 57530 msgr-work ceph rtd unknown /proc/57525/task/57530/root (readlink: Permission denied) 2024-06-03T03:27:58.674 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 57530 msgr-work ceph txt unknown /proc/57525/task/57530/exe (readlink: Permission denied) 2024-06-03T03:27:58.674 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 57530 msgr-work ceph NOFD /proc/57525/task/57530/fd (opendir: Permission denied) 2024-06-03T03:27:58.674 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 57531 msgr-work ceph cwd unknown /proc/57525/task/57531/cwd (readlink: Permission denied) 2024-06-03T03:27:58.674 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 57531 msgr-work ceph rtd unknown /proc/57525/task/57531/root (readlink: Permission denied) 2024-06-03T03:27:58.675 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 57531 msgr-work ceph txt unknown /proc/57525/task/57531/exe (readlink: Permission denied) 2024-06-03T03:27:58.675 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 57531 msgr-work ceph NOFD /proc/57525/task/57531/fd (opendir: Permission denied) 2024-06-03T03:27:58.675 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 57535 service ceph cwd unknown /proc/57525/task/57535/cwd (readlink: Permission denied) 2024-06-03T03:27:58.675 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 57535 service ceph rtd unknown /proc/57525/task/57535/root (readlink: Permission denied) 2024-06-03T03:27:58.675 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 57535 service ceph txt unknown /proc/57525/task/57535/exe (readlink: Permission denied) 2024-06-03T03:27:58.675 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 57535 service ceph NOFD /proc/57525/task/57535/fd (opendir: Permission denied) 2024-06-03T03:27:58.675 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 57536 admin_soc ceph cwd unknown /proc/57525/task/57536/cwd (readlink: Permission denied) 2024-06-03T03:27:58.675 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 57536 admin_soc ceph rtd unknown /proc/57525/task/57536/root (readlink: Permission denied) 2024-06-03T03:27:58.675 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 57536 admin_soc ceph txt unknown /proc/57525/task/57536/exe (readlink: Permission denied) 2024-06-03T03:27:58.675 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 57536 admin_soc ceph NOFD /proc/57525/task/57536/fd (opendir: Permission denied) 2024-06-03T03:27:58.675 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 57687 signal_ha ceph cwd unknown /proc/57525/task/57687/cwd (readlink: Permission denied) 2024-06-03T03:27:58.675 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 57687 signal_ha ceph rtd unknown /proc/57525/task/57687/root (readlink: Permission denied) 2024-06-03T03:27:58.676 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 57687 signal_ha ceph txt unknown /proc/57525/task/57687/exe (readlink: Permission denied) 2024-06-03T03:27:58.676 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 57687 signal_ha ceph NOFD /proc/57525/task/57687/fd (opendir: Permission denied) 2024-06-03T03:27:58.676 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 57688 io_contex ceph cwd unknown /proc/57525/task/57688/cwd (readlink: Permission denied) 2024-06-03T03:27:58.676 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 57688 io_contex ceph rtd unknown /proc/57525/task/57688/root (readlink: Permission denied) 2024-06-03T03:27:58.676 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 57688 io_contex ceph txt unknown /proc/57525/task/57688/exe (readlink: Permission denied) 2024-06-03T03:27:58.676 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 57688 io_contex ceph NOFD /proc/57525/task/57688/fd (opendir: Permission denied) 2024-06-03T03:27:58.676 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 57689 io_contex ceph cwd unknown /proc/57525/task/57689/cwd (readlink: Permission denied) 2024-06-03T03:27:58.676 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 57689 io_contex ceph rtd unknown /proc/57525/task/57689/root (readlink: Permission denied) 2024-06-03T03:27:58.676 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 57689 io_contex ceph txt unknown /proc/57525/task/57689/exe (readlink: Permission denied) 2024-06-03T03:27:58.676 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 57689 io_contex ceph NOFD /proc/57525/task/57689/fd (opendir: Permission denied) 2024-06-03T03:27:58.676 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 57714 OpHistory ceph cwd unknown /proc/57525/task/57714/cwd (readlink: Permission denied) 2024-06-03T03:27:58.676 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 57714 OpHistory ceph rtd unknown /proc/57525/task/57714/root (readlink: Permission denied) 2024-06-03T03:27:58.676 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 57714 OpHistory ceph txt unknown /proc/57525/task/57714/exe (readlink: Permission denied) 2024-06-03T03:27:58.677 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 57714 OpHistory ceph NOFD /proc/57525/task/57714/fd (opendir: Permission denied) 2024-06-03T03:27:58.677 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 57716 ceph_time ceph cwd unknown /proc/57525/task/57716/cwd (readlink: Permission denied) 2024-06-03T03:27:58.677 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 57716 ceph_time ceph rtd unknown /proc/57525/task/57716/root (readlink: Permission denied) 2024-06-03T03:27:58.677 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 57716 ceph_time ceph txt unknown /proc/57525/task/57716/exe (readlink: Permission denied) 2024-06-03T03:27:58.677 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 57716 ceph_time ceph NOFD /proc/57525/task/57716/fd (opendir: Permission denied) 2024-06-03T03:27:58.677 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 57909 ceph-osd ceph cwd unknown /proc/57525/task/57909/cwd (readlink: Permission denied) 2024-06-03T03:27:58.677 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 57909 ceph-osd ceph rtd unknown /proc/57525/task/57909/root (readlink: Permission denied) 2024-06-03T03:27:58.677 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 57909 ceph-osd ceph txt unknown /proc/57525/task/57909/exe (readlink: Permission denied) 2024-06-03T03:27:58.677 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 57909 ceph-osd ceph NOFD /proc/57525/task/57909/fd (opendir: Permission denied) 2024-06-03T03:27:58.677 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 57910 safe_time ceph cwd unknown /proc/57525/task/57910/cwd (readlink: Permission denied) 2024-06-03T03:27:58.677 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 57910 safe_time ceph rtd unknown /proc/57525/task/57910/root (readlink: Permission denied) 2024-06-03T03:27:58.677 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 57910 safe_time ceph txt unknown /proc/57525/task/57910/exe (readlink: Permission denied) 2024-06-03T03:27:58.677 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 57910 safe_time ceph NOFD /proc/57525/task/57910/fd (opendir: Permission denied) 2024-06-03T03:27:58.677 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 57911 safe_time ceph cwd unknown /proc/57525/task/57911/cwd (readlink: Permission denied) 2024-06-03T03:27:58.678 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 57911 safe_time ceph rtd unknown /proc/57525/task/57911/root (readlink: Permission denied) 2024-06-03T03:27:58.678 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 57911 safe_time ceph txt unknown /proc/57525/task/57911/exe (readlink: Permission denied) 2024-06-03T03:27:58.678 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 57911 safe_time ceph NOFD /proc/57525/task/57911/fd (opendir: Permission denied) 2024-06-03T03:27:58.678 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 57912 safe_time ceph cwd unknown /proc/57525/task/57912/cwd (readlink: Permission denied) 2024-06-03T03:27:58.678 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 57912 safe_time ceph rtd unknown /proc/57525/task/57912/root (readlink: Permission denied) 2024-06-03T03:27:58.678 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 57912 safe_time ceph txt unknown /proc/57525/task/57912/exe (readlink: Permission denied) 2024-06-03T03:27:58.678 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 57912 safe_time ceph NOFD /proc/57525/task/57912/fd (opendir: Permission denied) 2024-06-03T03:27:58.678 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 57913 safe_time ceph cwd unknown /proc/57525/task/57913/cwd (readlink: Permission denied) 2024-06-03T03:27:58.678 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 57913 safe_time ceph rtd unknown /proc/57525/task/57913/root (readlink: Permission denied) 2024-06-03T03:27:58.678 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 57913 safe_time ceph txt unknown /proc/57525/task/57913/exe (readlink: Permission denied) 2024-06-03T03:27:58.678 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 57913 safe_time ceph NOFD /proc/57525/task/57913/fd (opendir: Permission denied) 2024-06-03T03:27:58.678 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 57914 fn_anonym ceph cwd unknown /proc/57525/task/57914/cwd (readlink: Permission denied) 2024-06-03T03:27:58.678 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 57914 fn_anonym ceph rtd unknown /proc/57525/task/57914/root (readlink: Permission denied) 2024-06-03T03:27:58.679 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 57914 fn_anonym ceph txt unknown /proc/57525/task/57914/exe (readlink: Permission denied) 2024-06-03T03:27:58.679 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 57914 fn_anonym ceph NOFD /proc/57525/task/57914/fd (opendir: Permission denied) 2024-06-03T03:27:58.679 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 57919 rocksdb:l ceph cwd unknown /proc/57525/task/57919/cwd (readlink: Permission denied) 2024-06-03T03:27:58.679 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 57919 rocksdb:l ceph rtd unknown /proc/57525/task/57919/root (readlink: Permission denied) 2024-06-03T03:27:58.679 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 57919 rocksdb:l ceph txt unknown /proc/57525/task/57919/exe (readlink: Permission denied) 2024-06-03T03:27:58.679 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 57919 rocksdb:l ceph NOFD /proc/57525/task/57919/fd (opendir: Permission denied) 2024-06-03T03:27:58.679 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 57920 rocksdb:l ceph cwd unknown /proc/57525/task/57920/cwd (readlink: Permission denied) 2024-06-03T03:27:58.679 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 57920 rocksdb:l ceph rtd unknown /proc/57525/task/57920/root (readlink: Permission denied) 2024-06-03T03:27:58.679 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 57920 rocksdb:l ceph txt unknown /proc/57525/task/57920/exe (readlink: Permission denied) 2024-06-03T03:27:58.679 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 57920 rocksdb:l ceph NOFD /proc/57525/task/57920/fd (opendir: Permission denied) 2024-06-03T03:27:58.679 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 57921 rocksdb:l ceph cwd unknown /proc/57525/task/57921/cwd (readlink: Permission denied) 2024-06-03T03:27:58.679 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 57921 rocksdb:l ceph rtd unknown /proc/57525/task/57921/root (readlink: Permission denied) 2024-06-03T03:27:58.679 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 57921 rocksdb:l ceph txt unknown /proc/57525/task/57921/exe (readlink: Permission denied) 2024-06-03T03:27:58.679 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 57921 rocksdb:l ceph NOFD /proc/57525/task/57921/fd (opendir: Permission denied) 2024-06-03T03:27:58.680 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 57922 rocksdb:h ceph cwd unknown /proc/57525/task/57922/cwd (readlink: Permission denied) 2024-06-03T03:27:58.680 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 57922 rocksdb:h ceph rtd unknown /proc/57525/task/57922/root (readlink: Permission denied) 2024-06-03T03:27:58.680 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 57922 rocksdb:h ceph txt unknown /proc/57525/task/57922/exe (readlink: Permission denied) 2024-06-03T03:27:58.680 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 57922 rocksdb:h ceph NOFD /proc/57525/task/57922/fd (opendir: Permission denied) 2024-06-03T03:27:58.680 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58374 bstore_ai ceph cwd unknown /proc/57525/task/58374/cwd (readlink: Permission denied) 2024-06-03T03:27:58.680 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58374 bstore_ai ceph rtd unknown /proc/57525/task/58374/root (readlink: Permission denied) 2024-06-03T03:27:58.680 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58374 bstore_ai ceph txt unknown /proc/57525/task/58374/exe (readlink: Permission denied) 2024-06-03T03:27:58.680 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58374 bstore_ai ceph NOFD /proc/57525/task/58374/fd (opendir: Permission denied) 2024-06-03T03:27:58.680 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58628 bstore_ai ceph cwd unknown /proc/57525/task/58628/cwd (readlink: Permission denied) 2024-06-03T03:27:58.680 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58628 bstore_ai ceph rtd unknown /proc/57525/task/58628/root (readlink: Permission denied) 2024-06-03T03:27:58.680 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58628 bstore_ai ceph txt unknown /proc/57525/task/58628/exe (readlink: Permission denied) 2024-06-03T03:27:58.680 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58628 bstore_ai ceph NOFD /proc/57525/task/58628/fd (opendir: Permission denied) 2024-06-03T03:27:58.680 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58811 ceph-osd ceph cwd unknown /proc/57525/task/58811/cwd (readlink: Permission denied) 2024-06-03T03:27:58.681 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58811 ceph-osd ceph rtd unknown /proc/57525/task/58811/root (readlink: Permission denied) 2024-06-03T03:27:58.681 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58811 ceph-osd ceph txt unknown /proc/57525/task/58811/exe (readlink: Permission denied) 2024-06-03T03:27:58.681 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58811 ceph-osd ceph NOFD /proc/57525/task/58811/fd (opendir: Permission denied) 2024-06-03T03:27:58.681 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58812 cfin ceph cwd unknown /proc/57525/task/58812/cwd (readlink: Permission denied) 2024-06-03T03:27:58.681 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58812 cfin ceph rtd unknown /proc/57525/task/58812/root (readlink: Permission denied) 2024-06-03T03:27:58.681 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58812 cfin ceph txt unknown /proc/57525/task/58812/exe (readlink: Permission denied) 2024-06-03T03:27:58.681 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58812 cfin ceph NOFD /proc/57525/task/58812/fd (opendir: Permission denied) 2024-06-03T03:27:58.681 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58813 bstore_kv ceph cwd unknown /proc/57525/task/58813/cwd (readlink: Permission denied) 2024-06-03T03:27:58.681 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58813 bstore_kv ceph rtd unknown /proc/57525/task/58813/root (readlink: Permission denied) 2024-06-03T03:27:58.681 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58813 bstore_kv ceph txt unknown /proc/57525/task/58813/exe (readlink: Permission denied) 2024-06-03T03:27:58.681 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58813 bstore_kv ceph NOFD /proc/57525/task/58813/fd (opendir: Permission denied) 2024-06-03T03:27:58.681 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58814 bstore_kv ceph cwd unknown /proc/57525/task/58814/cwd (readlink: Permission denied) 2024-06-03T03:27:58.681 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58814 bstore_kv ceph rtd unknown /proc/57525/task/58814/root (readlink: Permission denied) 2024-06-03T03:27:58.682 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58814 bstore_kv ceph txt unknown /proc/57525/task/58814/exe (readlink: Permission denied) 2024-06-03T03:27:58.682 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58814 bstore_kv ceph NOFD /proc/57525/task/58814/fd (opendir: Permission denied) 2024-06-03T03:27:58.682 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58815 bstore_me ceph cwd unknown /proc/57525/task/58815/cwd (readlink: Permission denied) 2024-06-03T03:27:58.682 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58815 bstore_me ceph rtd unknown /proc/57525/task/58815/root (readlink: Permission denied) 2024-06-03T03:27:58.682 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58815 bstore_me ceph txt unknown /proc/57525/task/58815/exe (readlink: Permission denied) 2024-06-03T03:27:58.682 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58815 bstore_me ceph NOFD /proc/57525/task/58815/fd (opendir: Permission denied) 2024-06-03T03:27:58.682 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58819 ms_dispat ceph cwd unknown /proc/57525/task/58819/cwd (readlink: Permission denied) 2024-06-03T03:27:58.682 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58819 ms_dispat ceph rtd unknown /proc/57525/task/58819/root (readlink: Permission denied) 2024-06-03T03:27:58.682 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58819 ms_dispat ceph txt unknown /proc/57525/task/58819/exe (readlink: Permission denied) 2024-06-03T03:27:58.682 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58819 ms_dispat ceph NOFD /proc/57525/task/58819/fd (opendir: Permission denied) 2024-06-03T03:27:58.682 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58820 ms_local ceph cwd unknown /proc/57525/task/58820/cwd (readlink: Permission denied) 2024-06-03T03:27:58.682 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58820 ms_local ceph rtd unknown /proc/57525/task/58820/root (readlink: Permission denied) 2024-06-03T03:27:58.682 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58820 ms_local ceph txt unknown /proc/57525/task/58820/exe (readlink: Permission denied) 2024-06-03T03:27:58.683 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58820 ms_local ceph NOFD /proc/57525/task/58820/fd (opendir: Permission denied) 2024-06-03T03:27:58.683 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58821 safe_time ceph cwd unknown /proc/57525/task/58821/cwd (readlink: Permission denied) 2024-06-03T03:27:58.683 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58821 safe_time ceph rtd unknown /proc/57525/task/58821/root (readlink: Permission denied) 2024-06-03T03:27:58.683 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58821 safe_time ceph txt unknown /proc/57525/task/58821/exe (readlink: Permission denied) 2024-06-03T03:27:58.683 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58821 safe_time ceph NOFD /proc/57525/task/58821/fd (opendir: Permission denied) 2024-06-03T03:27:58.683 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58822 safe_time ceph cwd unknown /proc/57525/task/58822/cwd (readlink: Permission denied) 2024-06-03T03:27:58.683 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58822 safe_time ceph rtd unknown /proc/57525/task/58822/root (readlink: Permission denied) 2024-06-03T03:27:58.683 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58822 safe_time ceph txt unknown /proc/57525/task/58822/exe (readlink: Permission denied) 2024-06-03T03:27:58.683 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58822 safe_time ceph NOFD /proc/57525/task/58822/fd (opendir: Permission denied) 2024-06-03T03:27:58.683 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58823 ms_dispat ceph cwd unknown /proc/57525/task/58823/cwd (readlink: Permission denied) 2024-06-03T03:27:58.683 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58823 ms_dispat ceph rtd unknown /proc/57525/task/58823/root (readlink: Permission denied) 2024-06-03T03:27:58.684 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58823 ms_dispat ceph txt unknown /proc/57525/task/58823/exe (readlink: Permission denied) 2024-06-03T03:27:58.684 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58823 ms_dispat ceph NOFD /proc/57525/task/58823/fd (opendir: Permission denied) 2024-06-03T03:27:58.684 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58824 ms_local ceph cwd unknown /proc/57525/task/58824/cwd (readlink: Permission denied) 2024-06-03T03:27:58.684 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58824 ms_local ceph rtd unknown /proc/57525/task/58824/root (readlink: Permission denied) 2024-06-03T03:27:58.684 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58824 ms_local ceph txt unknown /proc/57525/task/58824/exe (readlink: Permission denied) 2024-06-03T03:27:58.684 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58824 ms_local ceph NOFD /proc/57525/task/58824/fd (opendir: Permission denied) 2024-06-03T03:27:58.685 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58825 ms_dispat ceph cwd unknown /proc/57525/task/58825/cwd (readlink: Permission denied) 2024-06-03T03:27:58.685 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58825 ms_dispat ceph rtd unknown /proc/57525/task/58825/root (readlink: Permission denied) 2024-06-03T03:27:58.685 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58825 ms_dispat ceph txt unknown /proc/57525/task/58825/exe (readlink: Permission denied) 2024-06-03T03:27:58.685 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58825 ms_dispat ceph NOFD /proc/57525/task/58825/fd (opendir: Permission denied) 2024-06-03T03:27:58.685 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58826 ms_local ceph cwd unknown /proc/57525/task/58826/cwd (readlink: Permission denied) 2024-06-03T03:27:58.685 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58826 ms_local ceph rtd unknown /proc/57525/task/58826/root (readlink: Permission denied) 2024-06-03T03:27:58.685 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58826 ms_local ceph txt unknown /proc/57525/task/58826/exe (readlink: Permission denied) 2024-06-03T03:27:58.686 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58826 ms_local ceph NOFD /proc/57525/task/58826/fd (opendir: Permission denied) 2024-06-03T03:27:58.686 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58827 ms_dispat ceph cwd unknown /proc/57525/task/58827/cwd (readlink: Permission denied) 2024-06-03T03:27:58.686 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58827 ms_dispat ceph rtd unknown /proc/57525/task/58827/root (readlink: Permission denied) 2024-06-03T03:27:58.686 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58827 ms_dispat ceph txt unknown /proc/57525/task/58827/exe (readlink: Permission denied) 2024-06-03T03:27:58.686 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58827 ms_dispat ceph NOFD /proc/57525/task/58827/fd (opendir: Permission denied) 2024-06-03T03:27:58.686 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58828 ms_local ceph cwd unknown /proc/57525/task/58828/cwd (readlink: Permission denied) 2024-06-03T03:27:58.687 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58828 ms_local ceph rtd unknown /proc/57525/task/58828/root (readlink: Permission denied) 2024-06-03T03:27:58.687 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58828 ms_local ceph txt unknown /proc/57525/task/58828/exe (readlink: Permission denied) 2024-06-03T03:27:58.687 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58828 ms_local ceph NOFD /proc/57525/task/58828/fd (opendir: Permission denied) 2024-06-03T03:27:58.687 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58829 ms_dispat ceph cwd unknown /proc/57525/task/58829/cwd (readlink: Permission denied) 2024-06-03T03:27:58.687 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58829 ms_dispat ceph rtd unknown /proc/57525/task/58829/root (readlink: Permission denied) 2024-06-03T03:27:58.687 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58829 ms_dispat ceph txt unknown /proc/57525/task/58829/exe (readlink: Permission denied) 2024-06-03T03:27:58.687 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58829 ms_dispat ceph NOFD /proc/57525/task/58829/fd (opendir: Permission denied) 2024-06-03T03:27:58.687 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58830 ms_local ceph cwd unknown /proc/57525/task/58830/cwd (readlink: Permission denied) 2024-06-03T03:27:58.688 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58830 ms_local ceph rtd unknown /proc/57525/task/58830/root (readlink: Permission denied) 2024-06-03T03:27:58.688 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58830 ms_local ceph txt unknown /proc/57525/task/58830/exe (readlink: Permission denied) 2024-06-03T03:27:58.688 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58830 ms_local ceph NOFD /proc/57525/task/58830/fd (opendir: Permission denied) 2024-06-03T03:27:58.688 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58831 ms_dispat ceph cwd unknown /proc/57525/task/58831/cwd (readlink: Permission denied) 2024-06-03T03:27:58.688 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58831 ms_dispat ceph rtd unknown /proc/57525/task/58831/root (readlink: Permission denied) 2024-06-03T03:27:58.688 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58831 ms_dispat ceph txt unknown /proc/57525/task/58831/exe (readlink: Permission denied) 2024-06-03T03:27:58.688 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58831 ms_dispat ceph NOFD /proc/57525/task/58831/fd (opendir: Permission denied) 2024-06-03T03:27:58.689 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58832 ms_local ceph cwd unknown /proc/57525/task/58832/cwd (readlink: Permission denied) 2024-06-03T03:27:58.689 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58832 ms_local ceph rtd unknown /proc/57525/task/58832/root (readlink: Permission denied) 2024-06-03T03:27:58.689 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58832 ms_local ceph txt unknown /proc/57525/task/58832/exe (readlink: Permission denied) 2024-06-03T03:27:58.689 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58832 ms_local ceph NOFD /proc/57525/task/58832/fd (opendir: Permission denied) 2024-06-03T03:27:58.689 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58833 ms_dispat ceph cwd unknown /proc/57525/task/58833/cwd (readlink: Permission denied) 2024-06-03T03:27:58.689 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58833 ms_dispat ceph rtd unknown /proc/57525/task/58833/root (readlink: Permission denied) 2024-06-03T03:27:58.690 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58833 ms_dispat ceph txt unknown /proc/57525/task/58833/exe (readlink: Permission denied) 2024-06-03T03:27:58.690 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58833 ms_dispat ceph NOFD /proc/57525/task/58833/fd (opendir: Permission denied) 2024-06-03T03:27:58.690 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58834 ms_local ceph cwd unknown /proc/57525/task/58834/cwd (readlink: Permission denied) 2024-06-03T03:27:58.690 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58834 ms_local ceph rtd unknown /proc/57525/task/58834/root (readlink: Permission denied) 2024-06-03T03:27:58.690 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58834 ms_local ceph txt unknown /proc/57525/task/58834/exe (readlink: Permission denied) 2024-06-03T03:27:58.690 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58834 ms_local ceph NOFD /proc/57525/task/58834/fd (opendir: Permission denied) 2024-06-03T03:27:58.691 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58835 fn_anonym ceph cwd unknown /proc/57525/task/58835/cwd (readlink: Permission denied) 2024-06-03T03:27:58.691 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58835 fn_anonym ceph rtd unknown /proc/57525/task/58835/root (readlink: Permission denied) 2024-06-03T03:27:58.691 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58835 fn_anonym ceph txt unknown /proc/57525/task/58835/exe (readlink: Permission denied) 2024-06-03T03:27:58.691 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58835 fn_anonym ceph NOFD /proc/57525/task/58835/fd (opendir: Permission denied) 2024-06-03T03:27:58.691 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58836 finisher ceph cwd unknown /proc/57525/task/58836/cwd (readlink: Permission denied) 2024-06-03T03:27:58.691 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58836 finisher ceph rtd unknown /proc/57525/task/58836/root (readlink: Permission denied) 2024-06-03T03:27:58.691 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58836 finisher ceph txt unknown /proc/57525/task/58836/exe (readlink: Permission denied) 2024-06-03T03:27:58.691 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58836 finisher ceph NOFD /proc/57525/task/58836/fd (opendir: Permission denied) 2024-06-03T03:27:58.691 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58837 safe_time ceph cwd unknown /proc/57525/task/58837/cwd (readlink: Permission denied) 2024-06-03T03:27:58.691 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58837 safe_time ceph rtd unknown /proc/57525/task/58837/root (readlink: Permission denied) 2024-06-03T03:27:58.691 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58837 safe_time ceph txt unknown /proc/57525/task/58837/exe (readlink: Permission denied) 2024-06-03T03:27:58.691 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58837 safe_time ceph NOFD /proc/57525/task/58837/fd (opendir: Permission denied) 2024-06-03T03:27:58.692 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58838 safe_time ceph cwd unknown /proc/57525/task/58838/cwd (readlink: Permission denied) 2024-06-03T03:27:58.692 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58838 safe_time ceph rtd unknown /proc/57525/task/58838/root (readlink: Permission denied) 2024-06-03T03:27:58.692 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58838 safe_time ceph txt unknown /proc/57525/task/58838/exe (readlink: Permission denied) 2024-06-03T03:27:58.692 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58838 safe_time ceph NOFD /proc/57525/task/58838/fd (opendir: Permission denied) 2024-06-03T03:27:58.692 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58839 ceph-osd ceph cwd unknown /proc/57525/task/58839/cwd (readlink: Permission denied) 2024-06-03T03:27:58.692 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58839 ceph-osd ceph rtd unknown /proc/57525/task/58839/root (readlink: Permission denied) 2024-06-03T03:27:58.692 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58839 ceph-osd ceph txt unknown /proc/57525/task/58839/exe (readlink: Permission denied) 2024-06-03T03:27:58.692 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58839 ceph-osd ceph NOFD /proc/57525/task/58839/fd (opendir: Permission denied) 2024-06-03T03:27:58.692 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58840 osd_srv_a ceph cwd unknown /proc/57525/task/58840/cwd (readlink: Permission denied) 2024-06-03T03:27:58.692 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58840 osd_srv_a ceph rtd unknown /proc/57525/task/58840/root (readlink: Permission denied) 2024-06-03T03:27:58.692 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58840 osd_srv_a ceph txt unknown /proc/57525/task/58840/exe (readlink: Permission denied) 2024-06-03T03:27:58.692 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58840 osd_srv_a ceph NOFD /proc/57525/task/58840/fd (opendir: Permission denied) 2024-06-03T03:27:58.693 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58841 tp_osd_tp ceph cwd unknown /proc/57525/task/58841/cwd (readlink: Permission denied) 2024-06-03T03:27:58.693 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58841 tp_osd_tp ceph rtd unknown /proc/57525/task/58841/root (readlink: Permission denied) 2024-06-03T03:27:58.693 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58841 tp_osd_tp ceph txt unknown /proc/57525/task/58841/exe (readlink: Permission denied) 2024-06-03T03:27:58.693 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58841 tp_osd_tp ceph NOFD /proc/57525/task/58841/fd (opendir: Permission denied) 2024-06-03T03:27:58.693 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58842 tp_osd_tp ceph cwd unknown /proc/57525/task/58842/cwd (readlink: Permission denied) 2024-06-03T03:27:58.693 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58842 tp_osd_tp ceph rtd unknown /proc/57525/task/58842/root (readlink: Permission denied) 2024-06-03T03:27:58.693 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58842 tp_osd_tp ceph txt unknown /proc/57525/task/58842/exe (readlink: Permission denied) 2024-06-03T03:27:58.693 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58842 tp_osd_tp ceph NOFD /proc/57525/task/58842/fd (opendir: Permission denied) 2024-06-03T03:27:58.693 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58843 tp_osd_tp ceph cwd unknown /proc/57525/task/58843/cwd (readlink: Permission denied) 2024-06-03T03:27:58.693 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58843 tp_osd_tp ceph rtd unknown /proc/57525/task/58843/root (readlink: Permission denied) 2024-06-03T03:27:58.693 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58843 tp_osd_tp ceph txt unknown /proc/57525/task/58843/exe (readlink: Permission denied) 2024-06-03T03:27:58.693 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58843 tp_osd_tp ceph NOFD /proc/57525/task/58843/fd (opendir: Permission denied) 2024-06-03T03:27:58.693 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58844 tp_osd_tp ceph cwd unknown /proc/57525/task/58844/cwd (readlink: Permission denied) 2024-06-03T03:27:58.693 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58844 tp_osd_tp ceph rtd unknown /proc/57525/task/58844/root (readlink: Permission denied) 2024-06-03T03:27:58.694 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58844 tp_osd_tp ceph txt unknown /proc/57525/task/58844/exe (readlink: Permission denied) 2024-06-03T03:27:58.694 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58844 tp_osd_tp ceph NOFD /proc/57525/task/58844/fd (opendir: Permission denied) 2024-06-03T03:27:58.694 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58845 tp_osd_tp ceph cwd unknown /proc/57525/task/58845/cwd (readlink: Permission denied) 2024-06-03T03:27:58.694 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58845 tp_osd_tp ceph rtd unknown /proc/57525/task/58845/root (readlink: Permission denied) 2024-06-03T03:27:58.694 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58845 tp_osd_tp ceph txt unknown /proc/57525/task/58845/exe (readlink: Permission denied) 2024-06-03T03:27:58.694 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58845 tp_osd_tp ceph NOFD /proc/57525/task/58845/fd (opendir: Permission denied) 2024-06-03T03:27:58.694 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58846 tp_osd_tp ceph cwd unknown /proc/57525/task/58846/cwd (readlink: Permission denied) 2024-06-03T03:27:58.694 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58846 tp_osd_tp ceph rtd unknown /proc/57525/task/58846/root (readlink: Permission denied) 2024-06-03T03:27:58.694 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58846 tp_osd_tp ceph txt unknown /proc/57525/task/58846/exe (readlink: Permission denied) 2024-06-03T03:27:58.694 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58846 tp_osd_tp ceph NOFD /proc/57525/task/58846/fd (opendir: Permission denied) 2024-06-03T03:27:58.694 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58847 tp_osd_tp ceph cwd unknown /proc/57525/task/58847/cwd (readlink: Permission denied) 2024-06-03T03:27:58.694 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58847 tp_osd_tp ceph rtd unknown /proc/57525/task/58847/root (readlink: Permission denied) 2024-06-03T03:27:58.695 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58847 tp_osd_tp ceph txt unknown /proc/57525/task/58847/exe (readlink: Permission denied) 2024-06-03T03:27:58.695 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58847 tp_osd_tp ceph NOFD /proc/57525/task/58847/fd (opendir: Permission denied) 2024-06-03T03:27:58.695 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58848 tp_osd_tp ceph cwd unknown /proc/57525/task/58848/cwd (readlink: Permission denied) 2024-06-03T03:27:58.695 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58848 tp_osd_tp ceph rtd unknown /proc/57525/task/58848/root (readlink: Permission denied) 2024-06-03T03:27:58.695 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58848 tp_osd_tp ceph txt unknown /proc/57525/task/58848/exe (readlink: Permission denied) 2024-06-03T03:27:58.695 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58848 tp_osd_tp ceph NOFD /proc/57525/task/58848/fd (opendir: Permission denied) 2024-06-03T03:27:58.695 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58849 tp_osd_tp ceph cwd unknown /proc/57525/task/58849/cwd (readlink: Permission denied) 2024-06-03T03:27:58.695 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58849 tp_osd_tp ceph rtd unknown /proc/57525/task/58849/root (readlink: Permission denied) 2024-06-03T03:27:58.695 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58849 tp_osd_tp ceph txt unknown /proc/57525/task/58849/exe (readlink: Permission denied) 2024-06-03T03:27:58.695 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58849 tp_osd_tp ceph NOFD /proc/57525/task/58849/fd (opendir: Permission denied) 2024-06-03T03:27:58.695 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58850 tp_osd_tp ceph cwd unknown /proc/57525/task/58850/cwd (readlink: Permission denied) 2024-06-03T03:27:58.695 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58850 tp_osd_tp ceph rtd unknown /proc/57525/task/58850/root (readlink: Permission denied) 2024-06-03T03:27:58.695 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58850 tp_osd_tp ceph txt unknown /proc/57525/task/58850/exe (readlink: Permission denied) 2024-06-03T03:27:58.695 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58850 tp_osd_tp ceph NOFD /proc/57525/task/58850/fd (opendir: Permission denied) 2024-06-03T03:27:58.696 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58851 tp_osd_tp ceph cwd unknown /proc/57525/task/58851/cwd (readlink: Permission denied) 2024-06-03T03:27:58.696 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58851 tp_osd_tp ceph rtd unknown /proc/57525/task/58851/root (readlink: Permission denied) 2024-06-03T03:27:58.696 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58851 tp_osd_tp ceph txt unknown /proc/57525/task/58851/exe (readlink: Permission denied) 2024-06-03T03:27:58.696 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58851 tp_osd_tp ceph NOFD /proc/57525/task/58851/fd (opendir: Permission denied) 2024-06-03T03:27:58.696 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58852 tp_osd_tp ceph cwd unknown /proc/57525/task/58852/cwd (readlink: Permission denied) 2024-06-03T03:27:58.696 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58852 tp_osd_tp ceph rtd unknown /proc/57525/task/58852/root (readlink: Permission denied) 2024-06-03T03:27:58.696 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58852 tp_osd_tp ceph txt unknown /proc/57525/task/58852/exe (readlink: Permission denied) 2024-06-03T03:27:58.696 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58852 tp_osd_tp ceph NOFD /proc/57525/task/58852/fd (opendir: Permission denied) 2024-06-03T03:27:58.696 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58853 tp_osd_tp ceph cwd unknown /proc/57525/task/58853/cwd (readlink: Permission denied) 2024-06-03T03:27:58.696 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58853 tp_osd_tp ceph rtd unknown /proc/57525/task/58853/root (readlink: Permission denied) 2024-06-03T03:27:58.696 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58853 tp_osd_tp ceph txt unknown /proc/57525/task/58853/exe (readlink: Permission denied) 2024-06-03T03:27:58.696 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58853 tp_osd_tp ceph NOFD /proc/57525/task/58853/fd (opendir: Permission denied) 2024-06-03T03:27:58.696 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58854 tp_osd_tp ceph cwd unknown /proc/57525/task/58854/cwd (readlink: Permission denied) 2024-06-03T03:27:58.697 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58854 tp_osd_tp ceph rtd unknown /proc/57525/task/58854/root (readlink: Permission denied) 2024-06-03T03:27:58.697 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58854 tp_osd_tp ceph txt unknown /proc/57525/task/58854/exe (readlink: Permission denied) 2024-06-03T03:27:58.697 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58854 tp_osd_tp ceph NOFD /proc/57525/task/58854/fd (opendir: Permission denied) 2024-06-03T03:27:58.697 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58855 tp_osd_tp ceph cwd unknown /proc/57525/task/58855/cwd (readlink: Permission denied) 2024-06-03T03:27:58.697 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58855 tp_osd_tp ceph rtd unknown /proc/57525/task/58855/root (readlink: Permission denied) 2024-06-03T03:27:58.697 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58855 tp_osd_tp ceph txt unknown /proc/57525/task/58855/exe (readlink: Permission denied) 2024-06-03T03:27:58.697 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58855 tp_osd_tp ceph NOFD /proc/57525/task/58855/fd (opendir: Permission denied) 2024-06-03T03:27:58.697 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58856 tp_osd_tp ceph cwd unknown /proc/57525/task/58856/cwd (readlink: Permission denied) 2024-06-03T03:27:58.697 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58856 tp_osd_tp ceph rtd unknown /proc/57525/task/58856/root (readlink: Permission denied) 2024-06-03T03:27:58.697 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58856 tp_osd_tp ceph txt unknown /proc/57525/task/58856/exe (readlink: Permission denied) 2024-06-03T03:27:58.697 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58856 tp_osd_tp ceph NOFD /proc/57525/task/58856/fd (opendir: Permission denied) 2024-06-03T03:27:58.697 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58857 osd_srv_h ceph cwd unknown /proc/57525/task/58857/cwd (readlink: Permission denied) 2024-06-03T03:27:58.697 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58857 osd_srv_h ceph rtd unknown /proc/57525/task/58857/root (readlink: Permission denied) 2024-06-03T03:27:58.697 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58857 osd_srv_h ceph txt unknown /proc/57525/task/58857/exe (readlink: Permission denied) 2024-06-03T03:27:58.698 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 57525 58857 osd_srv_h ceph NOFD /proc/57525/task/58857/fd (opendir: Permission denied) 2024-06-03T03:27:58.698 INFO:tasks.workunit.client.0.smithi117.stdout:sudo 59050 root cwd unknown /proc/59050/cwd (readlink: Permission denied) 2024-06-03T03:27:58.698 INFO:tasks.workunit.client.0.smithi117.stdout:sudo 59050 root rtd unknown /proc/59050/root (readlink: Permission denied) 2024-06-03T03:27:58.698 INFO:tasks.workunit.client.0.smithi117.stdout:sudo 59050 root txt unknown /proc/59050/exe (readlink: Permission denied) 2024-06-03T03:27:58.698 INFO:tasks.workunit.client.0.smithi117.stdout:sudo 59050 root NOFD /proc/59050/fd (opendir: Permission denied) 2024-06-03T03:27:58.698 INFO:tasks.workunit.client.0.smithi117.stdout:journalct 59081 root cwd unknown /proc/59081/cwd (readlink: Permission denied) 2024-06-03T03:27:58.698 INFO:tasks.workunit.client.0.smithi117.stdout:journalct 59081 root rtd unknown /proc/59081/root (readlink: Permission denied) 2024-06-03T03:27:58.698 INFO:tasks.workunit.client.0.smithi117.stdout:journalct 59081 root txt unknown /proc/59081/exe (readlink: Permission denied) 2024-06-03T03:27:58.699 INFO:tasks.workunit.client.0.smithi117.stdout:journalct 59081 root NOFD /proc/59081/fd (opendir: Permission denied) 2024-06-03T03:27:58.699 INFO:tasks.workunit.client.0.smithi117.stdout:conmon 64376 root cwd unknown /proc/64376/cwd (readlink: Permission denied) 2024-06-03T03:27:58.699 INFO:tasks.workunit.client.0.smithi117.stdout:conmon 64376 root rtd unknown /proc/64376/root (readlink: Permission denied) 2024-06-03T03:27:58.699 INFO:tasks.workunit.client.0.smithi117.stdout:conmon 64376 root txt unknown /proc/64376/exe (readlink: Permission denied) 2024-06-03T03:27:58.699 INFO:tasks.workunit.client.0.smithi117.stdout:conmon 64376 root NOFD /proc/64376/fd (opendir: Permission denied) 2024-06-03T03:27:58.699 INFO:tasks.workunit.client.0.smithi117.stdout:podman-in 64378 root cwd unknown /proc/64378/cwd (readlink: Permission denied) 2024-06-03T03:27:58.700 INFO:tasks.workunit.client.0.smithi117.stdout:podman-in 64378 root rtd unknown /proc/64378/root (readlink: Permission denied) 2024-06-03T03:27:58.700 INFO:tasks.workunit.client.0.smithi117.stdout:podman-in 64378 root txt unknown /proc/64378/exe (readlink: Permission denied) 2024-06-03T03:27:58.700 INFO:tasks.workunit.client.0.smithi117.stdout:podman-in 64378 root NOFD /proc/64378/fd (opendir: Permission denied) 2024-06-03T03:27:58.700 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 ceph cwd unknown /proc/64380/cwd (readlink: Permission denied) 2024-06-03T03:27:58.700 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 ceph rtd unknown /proc/64380/root (readlink: Permission denied) 2024-06-03T03:27:58.700 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 ceph txt unknown /proc/64380/exe (readlink: Permission denied) 2024-06-03T03:27:58.700 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 ceph NOFD /proc/64380/fd (opendir: Permission denied) 2024-06-03T03:27:58.701 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64381 log ceph cwd unknown /proc/64380/task/64381/cwd (readlink: Permission denied) 2024-06-03T03:27:58.701 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64381 log ceph rtd unknown /proc/64380/task/64381/root (readlink: Permission denied) 2024-06-03T03:27:58.701 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64381 log ceph txt unknown /proc/64380/task/64381/exe (readlink: Permission denied) 2024-06-03T03:27:58.701 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64381 log ceph NOFD /proc/64380/task/64381/fd (opendir: Permission denied) 2024-06-03T03:27:58.701 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64383 msgr-work ceph cwd unknown /proc/64380/task/64383/cwd (readlink: Permission denied) 2024-06-03T03:27:58.701 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64383 msgr-work ceph rtd unknown /proc/64380/task/64383/root (readlink: Permission denied) 2024-06-03T03:27:58.701 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64383 msgr-work ceph txt unknown /proc/64380/task/64383/exe (readlink: Permission denied) 2024-06-03T03:27:58.702 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64383 msgr-work ceph NOFD /proc/64380/task/64383/fd (opendir: Permission denied) 2024-06-03T03:27:58.702 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64384 msgr-work ceph cwd unknown /proc/64380/task/64384/cwd (readlink: Permission denied) 2024-06-03T03:27:58.702 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64384 msgr-work ceph rtd unknown /proc/64380/task/64384/root (readlink: Permission denied) 2024-06-03T03:27:58.702 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64384 msgr-work ceph txt unknown /proc/64380/task/64384/exe (readlink: Permission denied) 2024-06-03T03:27:58.702 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64384 msgr-work ceph NOFD /proc/64380/task/64384/fd (opendir: Permission denied) 2024-06-03T03:27:58.702 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64385 msgr-work ceph cwd unknown /proc/64380/task/64385/cwd (readlink: Permission denied) 2024-06-03T03:27:58.702 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64385 msgr-work ceph rtd unknown /proc/64380/task/64385/root (readlink: Permission denied) 2024-06-03T03:27:58.702 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64385 msgr-work ceph txt unknown /proc/64380/task/64385/exe (readlink: Permission denied) 2024-06-03T03:27:58.703 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64385 msgr-work ceph NOFD /proc/64380/task/64385/fd (opendir: Permission denied) 2024-06-03T03:27:58.703 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64389 service ceph cwd unknown /proc/64380/task/64389/cwd (readlink: Permission denied) 2024-06-03T03:27:58.703 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64389 service ceph rtd unknown /proc/64380/task/64389/root (readlink: Permission denied) 2024-06-03T03:27:58.703 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64389 service ceph txt unknown /proc/64380/task/64389/exe (readlink: Permission denied) 2024-06-03T03:27:58.703 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64389 service ceph NOFD /proc/64380/task/64389/fd (opendir: Permission denied) 2024-06-03T03:27:58.703 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64390 admin_soc ceph cwd unknown /proc/64380/task/64390/cwd (readlink: Permission denied) 2024-06-03T03:27:58.704 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64390 admin_soc ceph rtd unknown /proc/64380/task/64390/root (readlink: Permission denied) 2024-06-03T03:27:58.704 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64390 admin_soc ceph txt unknown /proc/64380/task/64390/exe (readlink: Permission denied) 2024-06-03T03:27:58.704 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64390 admin_soc ceph NOFD /proc/64380/task/64390/fd (opendir: Permission denied) 2024-06-03T03:27:58.704 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64535 signal_ha ceph cwd unknown /proc/64380/task/64535/cwd (readlink: Permission denied) 2024-06-03T03:27:58.704 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64535 signal_ha ceph rtd unknown /proc/64380/task/64535/root (readlink: Permission denied) 2024-06-03T03:27:58.704 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64535 signal_ha ceph txt unknown /proc/64380/task/64535/exe (readlink: Permission denied) 2024-06-03T03:27:58.705 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64535 signal_ha ceph NOFD /proc/64380/task/64535/fd (opendir: Permission denied) 2024-06-03T03:27:58.705 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64536 io_contex ceph cwd unknown /proc/64380/task/64536/cwd (readlink: Permission denied) 2024-06-03T03:27:58.705 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64536 io_contex ceph rtd unknown /proc/64380/task/64536/root (readlink: Permission denied) 2024-06-03T03:27:58.705 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64536 io_contex ceph txt unknown /proc/64380/task/64536/exe (readlink: Permission denied) 2024-06-03T03:27:58.705 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64536 io_contex ceph NOFD /proc/64380/task/64536/fd (opendir: Permission denied) 2024-06-03T03:27:58.705 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64537 io_contex ceph cwd unknown /proc/64380/task/64537/cwd (readlink: Permission denied) 2024-06-03T03:27:58.706 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64537 io_contex ceph rtd unknown /proc/64380/task/64537/root (readlink: Permission denied) 2024-06-03T03:27:58.706 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64537 io_contex ceph txt unknown /proc/64380/task/64537/exe (readlink: Permission denied) 2024-06-03T03:27:58.706 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64537 io_contex ceph NOFD /proc/64380/task/64537/fd (opendir: Permission denied) 2024-06-03T03:27:58.706 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64541 OpHistory ceph cwd unknown /proc/64380/task/64541/cwd (readlink: Permission denied) 2024-06-03T03:27:58.706 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64541 OpHistory ceph rtd unknown /proc/64380/task/64541/root (readlink: Permission denied) 2024-06-03T03:27:58.706 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64541 OpHistory ceph txt unknown /proc/64380/task/64541/exe (readlink: Permission denied) 2024-06-03T03:27:58.706 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64541 OpHistory ceph NOFD /proc/64380/task/64541/fd (opendir: Permission denied) 2024-06-03T03:27:58.706 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64542 ceph_time ceph cwd unknown /proc/64380/task/64542/cwd (readlink: Permission denied) 2024-06-03T03:27:58.706 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64542 ceph_time ceph rtd unknown /proc/64380/task/64542/root (readlink: Permission denied) 2024-06-03T03:27:58.706 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64542 ceph_time ceph txt unknown /proc/64380/task/64542/exe (readlink: Permission denied) 2024-06-03T03:27:58.706 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64542 ceph_time ceph NOFD /proc/64380/task/64542/fd (opendir: Permission denied) 2024-06-03T03:27:58.707 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64657 ceph-osd ceph cwd unknown /proc/64380/task/64657/cwd (readlink: Permission denied) 2024-06-03T03:27:58.707 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64657 ceph-osd ceph rtd unknown /proc/64380/task/64657/root (readlink: Permission denied) 2024-06-03T03:27:58.707 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64657 ceph-osd ceph txt unknown /proc/64380/task/64657/exe (readlink: Permission denied) 2024-06-03T03:27:58.707 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64657 ceph-osd ceph NOFD /proc/64380/task/64657/fd (opendir: Permission denied) 2024-06-03T03:27:58.707 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64667 ceph-osd ceph cwd unknown /proc/64380/task/64667/cwd (readlink: Permission denied) 2024-06-03T03:27:58.707 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64667 ceph-osd ceph rtd unknown /proc/64380/task/64667/root (readlink: Permission denied) 2024-06-03T03:27:58.707 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64667 ceph-osd ceph txt unknown /proc/64380/task/64667/exe (readlink: Permission denied) 2024-06-03T03:27:58.707 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64667 ceph-osd ceph NOFD /proc/64380/task/64667/fd (opendir: Permission denied) 2024-06-03T03:27:58.707 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64698 ceph-osd ceph cwd unknown /proc/64380/task/64698/cwd (readlink: Permission denied) 2024-06-03T03:27:58.707 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64698 ceph-osd ceph rtd unknown /proc/64380/task/64698/root (readlink: Permission denied) 2024-06-03T03:27:58.707 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64698 ceph-osd ceph txt unknown /proc/64380/task/64698/exe (readlink: Permission denied) 2024-06-03T03:27:58.707 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64698 ceph-osd ceph NOFD /proc/64380/task/64698/fd (opendir: Permission denied) 2024-06-03T03:27:58.707 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64705 ceph-osd ceph cwd unknown /proc/64380/task/64705/cwd (readlink: Permission denied) 2024-06-03T03:27:58.708 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64705 ceph-osd ceph rtd unknown /proc/64380/task/64705/root (readlink: Permission denied) 2024-06-03T03:27:58.708 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64705 ceph-osd ceph txt unknown /proc/64380/task/64705/exe (readlink: Permission denied) 2024-06-03T03:27:58.708 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64705 ceph-osd ceph NOFD /proc/64380/task/64705/fd (opendir: Permission denied) 2024-06-03T03:27:58.708 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64716 ceph-osd ceph cwd unknown /proc/64380/task/64716/cwd (readlink: Permission denied) 2024-06-03T03:27:58.708 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64716 ceph-osd ceph rtd unknown /proc/64380/task/64716/root (readlink: Permission denied) 2024-06-03T03:27:58.708 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64716 ceph-osd ceph txt unknown /proc/64380/task/64716/exe (readlink: Permission denied) 2024-06-03T03:27:58.708 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64716 ceph-osd ceph NOFD /proc/64380/task/64716/fd (opendir: Permission denied) 2024-06-03T03:27:58.708 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64726 ceph-osd ceph cwd unknown /proc/64380/task/64726/cwd (readlink: Permission denied) 2024-06-03T03:27:58.708 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64726 ceph-osd ceph rtd unknown /proc/64380/task/64726/root (readlink: Permission denied) 2024-06-03T03:27:58.708 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64726 ceph-osd ceph txt unknown /proc/64380/task/64726/exe (readlink: Permission denied) 2024-06-03T03:27:58.708 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64726 ceph-osd ceph NOFD /proc/64380/task/64726/fd (opendir: Permission denied) 2024-06-03T03:27:58.708 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64752 ceph-osd ceph cwd unknown /proc/64380/task/64752/cwd (readlink: Permission denied) 2024-06-03T03:27:58.708 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64752 ceph-osd ceph rtd unknown /proc/64380/task/64752/root (readlink: Permission denied) 2024-06-03T03:27:58.708 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64752 ceph-osd ceph txt unknown /proc/64380/task/64752/exe (readlink: Permission denied) 2024-06-03T03:27:58.709 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64752 ceph-osd ceph NOFD /proc/64380/task/64752/fd (opendir: Permission denied) 2024-06-03T03:27:58.709 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64772 ceph-osd ceph cwd unknown /proc/64380/task/64772/cwd (readlink: Permission denied) 2024-06-03T03:27:58.709 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64772 ceph-osd ceph rtd unknown /proc/64380/task/64772/root (readlink: Permission denied) 2024-06-03T03:27:58.709 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64772 ceph-osd ceph txt unknown /proc/64380/task/64772/exe (readlink: Permission denied) 2024-06-03T03:27:58.709 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64772 ceph-osd ceph NOFD /proc/64380/task/64772/fd (opendir: Permission denied) 2024-06-03T03:27:58.709 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64773 ceph-osd ceph cwd unknown /proc/64380/task/64773/cwd (readlink: Permission denied) 2024-06-03T03:27:58.709 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64773 ceph-osd ceph rtd unknown /proc/64380/task/64773/root (readlink: Permission denied) 2024-06-03T03:27:58.709 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64773 ceph-osd ceph txt unknown /proc/64380/task/64773/exe (readlink: Permission denied) 2024-06-03T03:27:58.709 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64773 ceph-osd ceph NOFD /proc/64380/task/64773/fd (opendir: Permission denied) 2024-06-03T03:27:58.709 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64774 safe_time ceph cwd unknown /proc/64380/task/64774/cwd (readlink: Permission denied) 2024-06-03T03:27:58.709 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64774 safe_time ceph rtd unknown /proc/64380/task/64774/root (readlink: Permission denied) 2024-06-03T03:27:58.709 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64774 safe_time ceph txt unknown /proc/64380/task/64774/exe (readlink: Permission denied) 2024-06-03T03:27:58.709 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64774 safe_time ceph NOFD /proc/64380/task/64774/fd (opendir: Permission denied) 2024-06-03T03:27:58.710 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64775 safe_time ceph cwd unknown /proc/64380/task/64775/cwd (readlink: Permission denied) 2024-06-03T03:27:58.710 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64775 safe_time ceph rtd unknown /proc/64380/task/64775/root (readlink: Permission denied) 2024-06-03T03:27:58.710 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64775 safe_time ceph txt unknown /proc/64380/task/64775/exe (readlink: Permission denied) 2024-06-03T03:27:58.710 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64775 safe_time ceph NOFD /proc/64380/task/64775/fd (opendir: Permission denied) 2024-06-03T03:27:58.710 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64776 safe_time ceph cwd unknown /proc/64380/task/64776/cwd (readlink: Permission denied) 2024-06-03T03:27:58.710 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64776 safe_time ceph rtd unknown /proc/64380/task/64776/root (readlink: Permission denied) 2024-06-03T03:27:58.710 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64776 safe_time ceph txt unknown /proc/64380/task/64776/exe (readlink: Permission denied) 2024-06-03T03:27:58.710 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64776 safe_time ceph NOFD /proc/64380/task/64776/fd (opendir: Permission denied) 2024-06-03T03:27:58.710 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64777 safe_time ceph cwd unknown /proc/64380/task/64777/cwd (readlink: Permission denied) 2024-06-03T03:27:58.710 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64777 safe_time ceph rtd unknown /proc/64380/task/64777/root (readlink: Permission denied) 2024-06-03T03:27:58.710 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64777 safe_time ceph txt unknown /proc/64380/task/64777/exe (readlink: Permission denied) 2024-06-03T03:27:58.710 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64777 safe_time ceph NOFD /proc/64380/task/64777/fd (opendir: Permission denied) 2024-06-03T03:27:58.711 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64778 fn_anonym ceph cwd unknown /proc/64380/task/64778/cwd (readlink: Permission denied) 2024-06-03T03:27:58.711 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64778 fn_anonym ceph rtd unknown /proc/64380/task/64778/root (readlink: Permission denied) 2024-06-03T03:27:58.711 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64778 fn_anonym ceph txt unknown /proc/64380/task/64778/exe (readlink: Permission denied) 2024-06-03T03:27:58.711 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64778 fn_anonym ceph NOFD /proc/64380/task/64778/fd (opendir: Permission denied) 2024-06-03T03:27:58.711 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64783 rocksdb:l ceph cwd unknown /proc/64380/task/64783/cwd (readlink: Permission denied) 2024-06-03T03:27:58.711 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64783 rocksdb:l ceph rtd unknown /proc/64380/task/64783/root (readlink: Permission denied) 2024-06-03T03:27:58.711 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64783 rocksdb:l ceph txt unknown /proc/64380/task/64783/exe (readlink: Permission denied) 2024-06-03T03:27:58.711 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64783 rocksdb:l ceph NOFD /proc/64380/task/64783/fd (opendir: Permission denied) 2024-06-03T03:27:58.711 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64784 rocksdb:l ceph cwd unknown /proc/64380/task/64784/cwd (readlink: Permission denied) 2024-06-03T03:27:58.711 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64784 rocksdb:l ceph rtd unknown /proc/64380/task/64784/root (readlink: Permission denied) 2024-06-03T03:27:58.711 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64784 rocksdb:l ceph txt unknown /proc/64380/task/64784/exe (readlink: Permission denied) 2024-06-03T03:27:58.711 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64784 rocksdb:l ceph NOFD /proc/64380/task/64784/fd (opendir: Permission denied) 2024-06-03T03:27:58.711 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64785 rocksdb:l ceph cwd unknown /proc/64380/task/64785/cwd (readlink: Permission denied) 2024-06-03T03:27:58.711 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64785 rocksdb:l ceph rtd unknown /proc/64380/task/64785/root (readlink: Permission denied) 2024-06-03T03:27:58.712 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64785 rocksdb:l ceph txt unknown /proc/64380/task/64785/exe (readlink: Permission denied) 2024-06-03T03:27:58.712 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64785 rocksdb:l ceph NOFD /proc/64380/task/64785/fd (opendir: Permission denied) 2024-06-03T03:27:58.712 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64786 rocksdb:h ceph cwd unknown /proc/64380/task/64786/cwd (readlink: Permission denied) 2024-06-03T03:27:58.712 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64786 rocksdb:h ceph rtd unknown /proc/64380/task/64786/root (readlink: Permission denied) 2024-06-03T03:27:58.712 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64786 rocksdb:h ceph txt unknown /proc/64380/task/64786/exe (readlink: Permission denied) 2024-06-03T03:27:58.712 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 64786 rocksdb:h ceph NOFD /proc/64380/task/64786/fd (opendir: Permission denied) 2024-06-03T03:27:58.712 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65196 bstore_ai ceph cwd unknown /proc/64380/task/65196/cwd (readlink: Permission denied) 2024-06-03T03:27:58.712 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65196 bstore_ai ceph rtd unknown /proc/64380/task/65196/root (readlink: Permission denied) 2024-06-03T03:27:58.712 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65196 bstore_ai ceph txt unknown /proc/64380/task/65196/exe (readlink: Permission denied) 2024-06-03T03:27:58.712 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65196 bstore_ai ceph NOFD /proc/64380/task/65196/fd (opendir: Permission denied) 2024-06-03T03:27:58.712 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65395 bstore_ai ceph cwd unknown /proc/64380/task/65395/cwd (readlink: Permission denied) 2024-06-03T03:27:58.712 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65395 bstore_ai ceph rtd unknown /proc/64380/task/65395/root (readlink: Permission denied) 2024-06-03T03:27:58.712 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65395 bstore_ai ceph txt unknown /proc/64380/task/65395/exe (readlink: Permission denied) 2024-06-03T03:27:58.713 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65395 bstore_ai ceph NOFD /proc/64380/task/65395/fd (opendir: Permission denied) 2024-06-03T03:27:58.713 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65582 ceph-osd ceph cwd unknown /proc/64380/task/65582/cwd (readlink: Permission denied) 2024-06-03T03:27:58.713 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65582 ceph-osd ceph rtd unknown /proc/64380/task/65582/root (readlink: Permission denied) 2024-06-03T03:27:58.713 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65582 ceph-osd ceph txt unknown /proc/64380/task/65582/exe (readlink: Permission denied) 2024-06-03T03:27:58.713 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65582 ceph-osd ceph NOFD /proc/64380/task/65582/fd (opendir: Permission denied) 2024-06-03T03:27:58.713 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65583 cfin ceph cwd unknown /proc/64380/task/65583/cwd (readlink: Permission denied) 2024-06-03T03:27:58.713 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65583 cfin ceph rtd unknown /proc/64380/task/65583/root (readlink: Permission denied) 2024-06-03T03:27:58.713 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65583 cfin ceph txt unknown /proc/64380/task/65583/exe (readlink: Permission denied) 2024-06-03T03:27:58.713 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65583 cfin ceph NOFD /proc/64380/task/65583/fd (opendir: Permission denied) 2024-06-03T03:27:58.713 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65584 bstore_kv ceph cwd unknown /proc/64380/task/65584/cwd (readlink: Permission denied) 2024-06-03T03:27:58.713 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65584 bstore_kv ceph rtd unknown /proc/64380/task/65584/root (readlink: Permission denied) 2024-06-03T03:27:58.714 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65584 bstore_kv ceph txt unknown /proc/64380/task/65584/exe (readlink: Permission denied) 2024-06-03T03:27:58.714 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65584 bstore_kv ceph NOFD /proc/64380/task/65584/fd (opendir: Permission denied) 2024-06-03T03:27:58.714 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65585 bstore_kv ceph cwd unknown /proc/64380/task/65585/cwd (readlink: Permission denied) 2024-06-03T03:27:58.714 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65585 bstore_kv ceph rtd unknown /proc/64380/task/65585/root (readlink: Permission denied) 2024-06-03T03:27:58.714 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65585 bstore_kv ceph txt unknown /proc/64380/task/65585/exe (readlink: Permission denied) 2024-06-03T03:27:58.714 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65585 bstore_kv ceph NOFD /proc/64380/task/65585/fd (opendir: Permission denied) 2024-06-03T03:27:58.714 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65586 bstore_me ceph cwd unknown /proc/64380/task/65586/cwd (readlink: Permission denied) 2024-06-03T03:27:58.714 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65586 bstore_me ceph rtd unknown /proc/64380/task/65586/root (readlink: Permission denied) 2024-06-03T03:27:58.714 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65586 bstore_me ceph txt unknown /proc/64380/task/65586/exe (readlink: Permission denied) 2024-06-03T03:27:58.714 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65586 bstore_me ceph NOFD /proc/64380/task/65586/fd (opendir: Permission denied) 2024-06-03T03:27:58.714 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65589 ms_dispat ceph cwd unknown /proc/64380/task/65589/cwd (readlink: Permission denied) 2024-06-03T03:27:58.714 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65589 ms_dispat ceph rtd unknown /proc/64380/task/65589/root (readlink: Permission denied) 2024-06-03T03:27:58.715 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65589 ms_dispat ceph txt unknown /proc/64380/task/65589/exe (readlink: Permission denied) 2024-06-03T03:27:58.715 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65589 ms_dispat ceph NOFD /proc/64380/task/65589/fd (opendir: Permission denied) 2024-06-03T03:27:58.715 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65590 ms_local ceph cwd unknown /proc/64380/task/65590/cwd (readlink: Permission denied) 2024-06-03T03:27:58.715 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65590 ms_local ceph rtd unknown /proc/64380/task/65590/root (readlink: Permission denied) 2024-06-03T03:27:58.715 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65590 ms_local ceph txt unknown /proc/64380/task/65590/exe (readlink: Permission denied) 2024-06-03T03:27:58.715 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65590 ms_local ceph NOFD /proc/64380/task/65590/fd (opendir: Permission denied) 2024-06-03T03:27:58.715 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65591 safe_time ceph cwd unknown /proc/64380/task/65591/cwd (readlink: Permission denied) 2024-06-03T03:27:58.715 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65591 safe_time ceph rtd unknown /proc/64380/task/65591/root (readlink: Permission denied) 2024-06-03T03:27:58.715 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65591 safe_time ceph txt unknown /proc/64380/task/65591/exe (readlink: Permission denied) 2024-06-03T03:27:58.715 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65591 safe_time ceph NOFD /proc/64380/task/65591/fd (opendir: Permission denied) 2024-06-03T03:27:58.715 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65592 safe_time ceph cwd unknown /proc/64380/task/65592/cwd (readlink: Permission denied) 2024-06-03T03:27:58.715 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65592 safe_time ceph rtd unknown /proc/64380/task/65592/root (readlink: Permission denied) 2024-06-03T03:27:58.715 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65592 safe_time ceph txt unknown /proc/64380/task/65592/exe (readlink: Permission denied) 2024-06-03T03:27:58.716 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65592 safe_time ceph NOFD /proc/64380/task/65592/fd (opendir: Permission denied) 2024-06-03T03:27:58.716 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65593 ms_dispat ceph cwd unknown /proc/64380/task/65593/cwd (readlink: Permission denied) 2024-06-03T03:27:58.716 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65593 ms_dispat ceph rtd unknown /proc/64380/task/65593/root (readlink: Permission denied) 2024-06-03T03:27:58.716 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65593 ms_dispat ceph txt unknown /proc/64380/task/65593/exe (readlink: Permission denied) 2024-06-03T03:27:58.716 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65593 ms_dispat ceph NOFD /proc/64380/task/65593/fd (opendir: Permission denied) 2024-06-03T03:27:58.716 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65594 ms_local ceph cwd unknown /proc/64380/task/65594/cwd (readlink: Permission denied) 2024-06-03T03:27:58.716 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65594 ms_local ceph rtd unknown /proc/64380/task/65594/root (readlink: Permission denied) 2024-06-03T03:27:58.716 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65594 ms_local ceph txt unknown /proc/64380/task/65594/exe (readlink: Permission denied) 2024-06-03T03:27:58.716 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65594 ms_local ceph NOFD /proc/64380/task/65594/fd (opendir: Permission denied) 2024-06-03T03:27:58.716 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65595 ms_dispat ceph cwd unknown /proc/64380/task/65595/cwd (readlink: Permission denied) 2024-06-03T03:27:58.716 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65595 ms_dispat ceph rtd unknown /proc/64380/task/65595/root (readlink: Permission denied) 2024-06-03T03:27:58.716 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65595 ms_dispat ceph txt unknown /proc/64380/task/65595/exe (readlink: Permission denied) 2024-06-03T03:27:58.716 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65595 ms_dispat ceph NOFD /proc/64380/task/65595/fd (opendir: Permission denied) 2024-06-03T03:27:58.716 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65596 ms_local ceph cwd unknown /proc/64380/task/65596/cwd (readlink: Permission denied) 2024-06-03T03:27:58.717 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65596 ms_local ceph rtd unknown /proc/64380/task/65596/root (readlink: Permission denied) 2024-06-03T03:27:58.717 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65596 ms_local ceph txt unknown /proc/64380/task/65596/exe (readlink: Permission denied) 2024-06-03T03:27:58.717 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65596 ms_local ceph NOFD /proc/64380/task/65596/fd (opendir: Permission denied) 2024-06-03T03:27:58.717 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65597 ms_dispat ceph cwd unknown /proc/64380/task/65597/cwd (readlink: Permission denied) 2024-06-03T03:27:58.717 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65597 ms_dispat ceph rtd unknown /proc/64380/task/65597/root (readlink: Permission denied) 2024-06-03T03:27:58.717 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65597 ms_dispat ceph txt unknown /proc/64380/task/65597/exe (readlink: Permission denied) 2024-06-03T03:27:58.717 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65597 ms_dispat ceph NOFD /proc/64380/task/65597/fd (opendir: Permission denied) 2024-06-03T03:27:58.717 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65598 ms_local ceph cwd unknown /proc/64380/task/65598/cwd (readlink: Permission denied) 2024-06-03T03:27:58.717 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65598 ms_local ceph rtd unknown /proc/64380/task/65598/root (readlink: Permission denied) 2024-06-03T03:27:58.717 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65598 ms_local ceph txt unknown /proc/64380/task/65598/exe (readlink: Permission denied) 2024-06-03T03:27:58.717 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65598 ms_local ceph NOFD /proc/64380/task/65598/fd (opendir: Permission denied) 2024-06-03T03:27:58.717 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65599 ms_dispat ceph cwd unknown /proc/64380/task/65599/cwd (readlink: Permission denied) 2024-06-03T03:27:58.717 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65599 ms_dispat ceph rtd unknown /proc/64380/task/65599/root (readlink: Permission denied) 2024-06-03T03:27:58.718 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65599 ms_dispat ceph txt unknown /proc/64380/task/65599/exe (readlink: Permission denied) 2024-06-03T03:27:58.718 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65599 ms_dispat ceph NOFD /proc/64380/task/65599/fd (opendir: Permission denied) 2024-06-03T03:27:58.718 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65600 ms_local ceph cwd unknown /proc/64380/task/65600/cwd (readlink: Permission denied) 2024-06-03T03:27:58.718 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65600 ms_local ceph rtd unknown /proc/64380/task/65600/root (readlink: Permission denied) 2024-06-03T03:27:58.718 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65600 ms_local ceph txt unknown /proc/64380/task/65600/exe (readlink: Permission denied) 2024-06-03T03:27:58.718 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65600 ms_local ceph NOFD /proc/64380/task/65600/fd (opendir: Permission denied) 2024-06-03T03:27:58.718 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65601 ms_dispat ceph cwd unknown /proc/64380/task/65601/cwd (readlink: Permission denied) 2024-06-03T03:27:58.718 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65601 ms_dispat ceph rtd unknown /proc/64380/task/65601/root (readlink: Permission denied) 2024-06-03T03:27:58.718 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65601 ms_dispat ceph txt unknown /proc/64380/task/65601/exe (readlink: Permission denied) 2024-06-03T03:27:58.719 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65601 ms_dispat ceph NOFD /proc/64380/task/65601/fd (opendir: Permission denied) 2024-06-03T03:27:58.719 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65602 ms_local ceph cwd unknown /proc/64380/task/65602/cwd (readlink: Permission denied) 2024-06-03T03:27:58.719 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65602 ms_local ceph rtd unknown /proc/64380/task/65602/root (readlink: Permission denied) 2024-06-03T03:27:58.719 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65602 ms_local ceph txt unknown /proc/64380/task/65602/exe (readlink: Permission denied) 2024-06-03T03:27:58.719 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65602 ms_local ceph NOFD /proc/64380/task/65602/fd (opendir: Permission denied) 2024-06-03T03:27:58.719 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65603 ms_dispat ceph cwd unknown /proc/64380/task/65603/cwd (readlink: Permission denied) 2024-06-03T03:27:58.719 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65603 ms_dispat ceph rtd unknown /proc/64380/task/65603/root (readlink: Permission denied) 2024-06-03T03:27:58.719 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65603 ms_dispat ceph txt unknown /proc/64380/task/65603/exe (readlink: Permission denied) 2024-06-03T03:27:58.719 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65603 ms_dispat ceph NOFD /proc/64380/task/65603/fd (opendir: Permission denied) 2024-06-03T03:27:58.719 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65604 ms_local ceph cwd unknown /proc/64380/task/65604/cwd (readlink: Permission denied) 2024-06-03T03:27:58.719 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65604 ms_local ceph rtd unknown /proc/64380/task/65604/root (readlink: Permission denied) 2024-06-03T03:27:58.719 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65604 ms_local ceph txt unknown /proc/64380/task/65604/exe (readlink: Permission denied) 2024-06-03T03:27:58.720 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65604 ms_local ceph NOFD /proc/64380/task/65604/fd (opendir: Permission denied) 2024-06-03T03:27:58.720 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65605 fn_anonym ceph cwd unknown /proc/64380/task/65605/cwd (readlink: Permission denied) 2024-06-03T03:27:58.720 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65605 fn_anonym ceph rtd unknown /proc/64380/task/65605/root (readlink: Permission denied) 2024-06-03T03:27:58.720 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65605 fn_anonym ceph txt unknown /proc/64380/task/65605/exe (readlink: Permission denied) 2024-06-03T03:27:58.720 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65605 fn_anonym ceph NOFD /proc/64380/task/65605/fd (opendir: Permission denied) 2024-06-03T03:27:58.720 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65606 finisher ceph cwd unknown /proc/64380/task/65606/cwd (readlink: Permission denied) 2024-06-03T03:27:58.720 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65606 finisher ceph rtd unknown /proc/64380/task/65606/root (readlink: Permission denied) 2024-06-03T03:27:58.720 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65606 finisher ceph txt unknown /proc/64380/task/65606/exe (readlink: Permission denied) 2024-06-03T03:27:58.720 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65606 finisher ceph NOFD /proc/64380/task/65606/fd (opendir: Permission denied) 2024-06-03T03:27:58.720 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65607 safe_time ceph cwd unknown /proc/64380/task/65607/cwd (readlink: Permission denied) 2024-06-03T03:27:58.720 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65607 safe_time ceph rtd unknown /proc/64380/task/65607/root (readlink: Permission denied) 2024-06-03T03:27:58.720 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65607 safe_time ceph txt unknown /proc/64380/task/65607/exe (readlink: Permission denied) 2024-06-03T03:27:58.720 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65607 safe_time ceph NOFD /proc/64380/task/65607/fd (opendir: Permission denied) 2024-06-03T03:27:58.720 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65608 safe_time ceph cwd unknown /proc/64380/task/65608/cwd (readlink: Permission denied) 2024-06-03T03:27:58.721 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65608 safe_time ceph rtd unknown /proc/64380/task/65608/root (readlink: Permission denied) 2024-06-03T03:27:58.721 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65608 safe_time ceph txt unknown /proc/64380/task/65608/exe (readlink: Permission denied) 2024-06-03T03:27:58.721 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65608 safe_time ceph NOFD /proc/64380/task/65608/fd (opendir: Permission denied) 2024-06-03T03:27:58.721 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65609 ceph-osd ceph cwd unknown /proc/64380/task/65609/cwd (readlink: Permission denied) 2024-06-03T03:27:58.721 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65609 ceph-osd ceph rtd unknown /proc/64380/task/65609/root (readlink: Permission denied) 2024-06-03T03:27:58.721 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65609 ceph-osd ceph txt unknown /proc/64380/task/65609/exe (readlink: Permission denied) 2024-06-03T03:27:58.721 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65609 ceph-osd ceph NOFD /proc/64380/task/65609/fd (opendir: Permission denied) 2024-06-03T03:27:58.721 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65610 osd_srv_a ceph cwd unknown /proc/64380/task/65610/cwd (readlink: Permission denied) 2024-06-03T03:27:58.721 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65610 osd_srv_a ceph rtd unknown /proc/64380/task/65610/root (readlink: Permission denied) 2024-06-03T03:27:58.721 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65610 osd_srv_a ceph txt unknown /proc/64380/task/65610/exe (readlink: Permission denied) 2024-06-03T03:27:58.721 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65610 osd_srv_a ceph NOFD /proc/64380/task/65610/fd (opendir: Permission denied) 2024-06-03T03:27:58.721 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65611 tp_osd_tp ceph cwd unknown /proc/64380/task/65611/cwd (readlink: Permission denied) 2024-06-03T03:27:58.721 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65611 tp_osd_tp ceph rtd unknown /proc/64380/task/65611/root (readlink: Permission denied) 2024-06-03T03:27:58.721 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65611 tp_osd_tp ceph txt unknown /proc/64380/task/65611/exe (readlink: Permission denied) 2024-06-03T03:27:58.722 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65611 tp_osd_tp ceph NOFD /proc/64380/task/65611/fd (opendir: Permission denied) 2024-06-03T03:27:58.722 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65612 tp_osd_tp ceph cwd unknown /proc/64380/task/65612/cwd (readlink: Permission denied) 2024-06-03T03:27:58.722 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65612 tp_osd_tp ceph rtd unknown /proc/64380/task/65612/root (readlink: Permission denied) 2024-06-03T03:27:58.722 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65612 tp_osd_tp ceph txt unknown /proc/64380/task/65612/exe (readlink: Permission denied) 2024-06-03T03:27:58.722 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65612 tp_osd_tp ceph NOFD /proc/64380/task/65612/fd (opendir: Permission denied) 2024-06-03T03:27:58.722 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65613 tp_osd_tp ceph cwd unknown /proc/64380/task/65613/cwd (readlink: Permission denied) 2024-06-03T03:27:58.722 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65613 tp_osd_tp ceph rtd unknown /proc/64380/task/65613/root (readlink: Permission denied) 2024-06-03T03:27:58.722 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65613 tp_osd_tp ceph txt unknown /proc/64380/task/65613/exe (readlink: Permission denied) 2024-06-03T03:27:58.722 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65613 tp_osd_tp ceph NOFD /proc/64380/task/65613/fd (opendir: Permission denied) 2024-06-03T03:27:58.722 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65614 tp_osd_tp ceph cwd unknown /proc/64380/task/65614/cwd (readlink: Permission denied) 2024-06-03T03:27:58.722 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65614 tp_osd_tp ceph rtd unknown /proc/64380/task/65614/root (readlink: Permission denied) 2024-06-03T03:27:58.722 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65614 tp_osd_tp ceph txt unknown /proc/64380/task/65614/exe (readlink: Permission denied) 2024-06-03T03:27:58.722 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65614 tp_osd_tp ceph NOFD /proc/64380/task/65614/fd (opendir: Permission denied) 2024-06-03T03:27:58.722 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65615 tp_osd_tp ceph cwd unknown /proc/64380/task/65615/cwd (readlink: Permission denied) 2024-06-03T03:27:58.723 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65615 tp_osd_tp ceph rtd unknown /proc/64380/task/65615/root (readlink: Permission denied) 2024-06-03T03:27:58.723 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65615 tp_osd_tp ceph txt unknown /proc/64380/task/65615/exe (readlink: Permission denied) 2024-06-03T03:27:58.723 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65615 tp_osd_tp ceph NOFD /proc/64380/task/65615/fd (opendir: Permission denied) 2024-06-03T03:27:58.723 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65616 tp_osd_tp ceph cwd unknown /proc/64380/task/65616/cwd (readlink: Permission denied) 2024-06-03T03:27:58.723 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65616 tp_osd_tp ceph rtd unknown /proc/64380/task/65616/root (readlink: Permission denied) 2024-06-03T03:27:58.723 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65616 tp_osd_tp ceph txt unknown /proc/64380/task/65616/exe (readlink: Permission denied) 2024-06-03T03:27:58.723 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65616 tp_osd_tp ceph NOFD /proc/64380/task/65616/fd (opendir: Permission denied) 2024-06-03T03:27:58.723 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65617 tp_osd_tp ceph cwd unknown /proc/64380/task/65617/cwd (readlink: Permission denied) 2024-06-03T03:27:58.723 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65617 tp_osd_tp ceph rtd unknown /proc/64380/task/65617/root (readlink: Permission denied) 2024-06-03T03:27:58.723 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65617 tp_osd_tp ceph txt unknown /proc/64380/task/65617/exe (readlink: Permission denied) 2024-06-03T03:27:58.723 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65617 tp_osd_tp ceph NOFD /proc/64380/task/65617/fd (opendir: Permission denied) 2024-06-03T03:27:58.723 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65618 tp_osd_tp ceph cwd unknown /proc/64380/task/65618/cwd (readlink: Permission denied) 2024-06-03T03:27:58.724 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65618 tp_osd_tp ceph rtd unknown /proc/64380/task/65618/root (readlink: Permission denied) 2024-06-03T03:27:58.724 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65618 tp_osd_tp ceph txt unknown /proc/64380/task/65618/exe (readlink: Permission denied) 2024-06-03T03:27:58.724 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65618 tp_osd_tp ceph NOFD /proc/64380/task/65618/fd (opendir: Permission denied) 2024-06-03T03:27:58.724 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65619 tp_osd_tp ceph cwd unknown /proc/64380/task/65619/cwd (readlink: Permission denied) 2024-06-03T03:27:58.724 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65619 tp_osd_tp ceph rtd unknown /proc/64380/task/65619/root (readlink: Permission denied) 2024-06-03T03:27:58.724 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65619 tp_osd_tp ceph txt unknown /proc/64380/task/65619/exe (readlink: Permission denied) 2024-06-03T03:27:58.724 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65619 tp_osd_tp ceph NOFD /proc/64380/task/65619/fd (opendir: Permission denied) 2024-06-03T03:27:58.724 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65620 tp_osd_tp ceph cwd unknown /proc/64380/task/65620/cwd (readlink: Permission denied) 2024-06-03T03:27:58.724 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65620 tp_osd_tp ceph rtd unknown /proc/64380/task/65620/root (readlink: Permission denied) 2024-06-03T03:27:58.724 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65620 tp_osd_tp ceph txt unknown /proc/64380/task/65620/exe (readlink: Permission denied) 2024-06-03T03:27:58.724 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65620 tp_osd_tp ceph NOFD /proc/64380/task/65620/fd (opendir: Permission denied) 2024-06-03T03:27:58.724 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65621 tp_osd_tp ceph cwd unknown /proc/64380/task/65621/cwd (readlink: Permission denied) 2024-06-03T03:27:58.724 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65621 tp_osd_tp ceph rtd unknown /proc/64380/task/65621/root (readlink: Permission denied) 2024-06-03T03:27:58.725 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65621 tp_osd_tp ceph txt unknown /proc/64380/task/65621/exe (readlink: Permission denied) 2024-06-03T03:27:58.725 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65621 tp_osd_tp ceph NOFD /proc/64380/task/65621/fd (opendir: Permission denied) 2024-06-03T03:27:58.725 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65622 tp_osd_tp ceph cwd unknown /proc/64380/task/65622/cwd (readlink: Permission denied) 2024-06-03T03:27:58.725 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65622 tp_osd_tp ceph rtd unknown /proc/64380/task/65622/root (readlink: Permission denied) 2024-06-03T03:27:58.725 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65622 tp_osd_tp ceph txt unknown /proc/64380/task/65622/exe (readlink: Permission denied) 2024-06-03T03:27:58.725 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65622 tp_osd_tp ceph NOFD /proc/64380/task/65622/fd (opendir: Permission denied) 2024-06-03T03:27:58.725 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65623 tp_osd_tp ceph cwd unknown /proc/64380/task/65623/cwd (readlink: Permission denied) 2024-06-03T03:27:58.725 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65623 tp_osd_tp ceph rtd unknown /proc/64380/task/65623/root (readlink: Permission denied) 2024-06-03T03:27:58.725 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65623 tp_osd_tp ceph txt unknown /proc/64380/task/65623/exe (readlink: Permission denied) 2024-06-03T03:27:58.725 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65623 tp_osd_tp ceph NOFD /proc/64380/task/65623/fd (opendir: Permission denied) 2024-06-03T03:27:58.725 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65624 tp_osd_tp ceph cwd unknown /proc/64380/task/65624/cwd (readlink: Permission denied) 2024-06-03T03:27:58.725 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65624 tp_osd_tp ceph rtd unknown /proc/64380/task/65624/root (readlink: Permission denied) 2024-06-03T03:27:58.725 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65624 tp_osd_tp ceph txt unknown /proc/64380/task/65624/exe (readlink: Permission denied) 2024-06-03T03:27:58.725 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65624 tp_osd_tp ceph NOFD /proc/64380/task/65624/fd (opendir: Permission denied) 2024-06-03T03:27:58.726 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65625 tp_osd_tp ceph cwd unknown /proc/64380/task/65625/cwd (readlink: Permission denied) 2024-06-03T03:27:58.726 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65625 tp_osd_tp ceph rtd unknown /proc/64380/task/65625/root (readlink: Permission denied) 2024-06-03T03:27:58.726 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65625 tp_osd_tp ceph txt unknown /proc/64380/task/65625/exe (readlink: Permission denied) 2024-06-03T03:27:58.726 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65625 tp_osd_tp ceph NOFD /proc/64380/task/65625/fd (opendir: Permission denied) 2024-06-03T03:27:58.726 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65626 tp_osd_tp ceph cwd unknown /proc/64380/task/65626/cwd (readlink: Permission denied) 2024-06-03T03:27:58.726 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65626 tp_osd_tp ceph rtd unknown /proc/64380/task/65626/root (readlink: Permission denied) 2024-06-03T03:27:58.726 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65626 tp_osd_tp ceph txt unknown /proc/64380/task/65626/exe (readlink: Permission denied) 2024-06-03T03:27:58.726 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65626 tp_osd_tp ceph NOFD /proc/64380/task/65626/fd (opendir: Permission denied) 2024-06-03T03:27:58.726 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65627 osd_srv_h ceph cwd unknown /proc/64380/task/65627/cwd (readlink: Permission denied) 2024-06-03T03:27:58.726 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65627 osd_srv_h ceph rtd unknown /proc/64380/task/65627/root (readlink: Permission denied) 2024-06-03T03:27:58.726 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65627 osd_srv_h ceph txt unknown /proc/64380/task/65627/exe (readlink: Permission denied) 2024-06-03T03:27:58.726 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-osd 64380 65627 osd_srv_h ceph NOFD /proc/64380/task/65627/fd (opendir: Permission denied) 2024-06-03T03:27:58.726 INFO:tasks.workunit.client.0.smithi117.stdout:sudo 65917 root cwd unknown /proc/65917/cwd (readlink: Permission denied) 2024-06-03T03:27:58.727 INFO:tasks.workunit.client.0.smithi117.stdout:sudo 65917 root rtd unknown /proc/65917/root (readlink: Permission denied) 2024-06-03T03:27:58.727 INFO:tasks.workunit.client.0.smithi117.stdout:sudo 65917 root txt unknown /proc/65917/exe (readlink: Permission denied) 2024-06-03T03:27:58.727 INFO:tasks.workunit.client.0.smithi117.stdout:sudo 65917 root NOFD /proc/65917/fd (opendir: Permission denied) 2024-06-03T03:27:58.727 INFO:tasks.workunit.client.0.smithi117.stdout:journalct 65943 root cwd unknown /proc/65943/cwd (readlink: Permission denied) 2024-06-03T03:27:58.727 INFO:tasks.workunit.client.0.smithi117.stdout:journalct 65943 root rtd unknown /proc/65943/root (readlink: Permission denied) 2024-06-03T03:27:58.727 INFO:tasks.workunit.client.0.smithi117.stdout:journalct 65943 root txt unknown /proc/65943/exe (readlink: Permission denied) 2024-06-03T03:27:58.727 INFO:tasks.workunit.client.0.smithi117.stdout:journalct 65943 root NOFD /proc/65943/fd (opendir: Permission denied) 2024-06-03T03:27:58.727 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/0 68243 root cwd unknown /proc/68243/cwd (readlink: Permission denied) 2024-06-03T03:27:58.727 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/0 68243 root rtd unknown /proc/68243/root (readlink: Permission denied) 2024-06-03T03:27:58.727 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/0 68243 root txt unknown /proc/68243/exe (readlink: Permission denied) 2024-06-03T03:27:58.727 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/0 68243 root NOFD /proc/68243/fd (opendir: Permission denied) 2024-06-03T03:27:58.727 INFO:tasks.workunit.client.0.smithi117.stdout:sudo 68934 root cwd unknown /proc/68934/cwd (readlink: Permission denied) 2024-06-03T03:27:58.727 INFO:tasks.workunit.client.0.smithi117.stdout:sudo 68934 root rtd unknown /proc/68934/root (readlink: Permission denied) 2024-06-03T03:27:58.727 INFO:tasks.workunit.client.0.smithi117.stdout:sudo 68934 root txt unknown /proc/68934/exe (readlink: Permission denied) 2024-06-03T03:27:58.728 INFO:tasks.workunit.client.0.smithi117.stdout:sudo 68934 root NOFD /proc/68934/fd (opendir: Permission denied) 2024-06-03T03:27:58.728 INFO:tasks.workunit.client.0.smithi117.stdout:sudo 68954 root cwd unknown /proc/68954/cwd (readlink: Permission denied) 2024-06-03T03:27:58.728 INFO:tasks.workunit.client.0.smithi117.stdout:sudo 68954 root rtd unknown /proc/68954/root (readlink: Permission denied) 2024-06-03T03:27:58.728 INFO:tasks.workunit.client.0.smithi117.stdout:sudo 68954 root txt unknown /proc/68954/exe (readlink: Permission denied) 2024-06-03T03:27:58.728 INFO:tasks.workunit.client.0.smithi117.stdout:sudo 68954 root NOFD /proc/68954/fd (opendir: Permission denied) 2024-06-03T03:27:58.728 INFO:tasks.workunit.client.0.smithi117.stdout:journalct 68959 root cwd unknown /proc/68959/cwd (readlink: Permission denied) 2024-06-03T03:27:58.728 INFO:tasks.workunit.client.0.smithi117.stdout:journalct 68959 root rtd unknown /proc/68959/root (readlink: Permission denied) 2024-06-03T03:27:58.728 INFO:tasks.workunit.client.0.smithi117.stdout:journalct 68959 root txt unknown /proc/68959/exe (readlink: Permission denied) 2024-06-03T03:27:58.728 INFO:tasks.workunit.client.0.smithi117.stdout:journalct 68959 root NOFD /proc/68959/fd (opendir: Permission denied) 2024-06-03T03:27:58.728 INFO:tasks.workunit.client.0.smithi117.stdout:sudo 68973 root cwd unknown /proc/68973/cwd (readlink: Permission denied) 2024-06-03T03:27:58.728 INFO:tasks.workunit.client.0.smithi117.stdout:sudo 68973 root rtd unknown /proc/68973/root (readlink: Permission denied) 2024-06-03T03:27:58.728 INFO:tasks.workunit.client.0.smithi117.stdout:sudo 68973 root txt unknown /proc/68973/exe (readlink: Permission denied) 2024-06-03T03:27:58.728 INFO:tasks.workunit.client.0.smithi117.stdout:sudo 68973 root NOFD /proc/68973/fd (opendir: Permission denied) 2024-06-03T03:27:58.728 INFO:tasks.workunit.client.0.smithi117.stdout:journalct 68977 root cwd unknown /proc/68977/cwd (readlink: Permission denied) 2024-06-03T03:27:58.729 INFO:tasks.workunit.client.0.smithi117.stdout:journalct 68977 root rtd unknown /proc/68977/root (readlink: Permission denied) 2024-06-03T03:27:58.729 INFO:tasks.workunit.client.0.smithi117.stdout:journalct 68977 root txt unknown /proc/68977/exe (readlink: Permission denied) 2024-06-03T03:27:58.729 INFO:tasks.workunit.client.0.smithi117.stdout:journalct 68977 root NOFD /proc/68977/fd (opendir: Permission denied) 2024-06-03T03:27:58.729 INFO:tasks.workunit.client.0.smithi117.stdout:sudo 68993 root cwd unknown /proc/68993/cwd (readlink: Permission denied) 2024-06-03T03:27:58.729 INFO:tasks.workunit.client.0.smithi117.stdout:sudo 68993 root rtd unknown /proc/68993/root (readlink: Permission denied) 2024-06-03T03:27:58.729 INFO:tasks.workunit.client.0.smithi117.stdout:sudo 68993 root txt unknown /proc/68993/exe (readlink: Permission denied) 2024-06-03T03:27:58.729 INFO:tasks.workunit.client.0.smithi117.stdout:sudo 68993 root NOFD /proc/68993/fd (opendir: Permission denied) 2024-06-03T03:27:58.729 INFO:tasks.workunit.client.0.smithi117.stdout:journalct 68998 root cwd unknown /proc/68998/cwd (readlink: Permission denied) 2024-06-03T03:27:58.729 INFO:tasks.workunit.client.0.smithi117.stdout:journalct 68998 root rtd unknown /proc/68998/root (readlink: Permission denied) 2024-06-03T03:27:58.729 INFO:tasks.workunit.client.0.smithi117.stdout:journalct 68998 root txt unknown /proc/68998/exe (readlink: Permission denied) 2024-06-03T03:27:58.729 INFO:tasks.workunit.client.0.smithi117.stdout:journalct 68998 root NOFD /proc/68998/fd (opendir: Permission denied) 2024-06-03T03:27:58.729 INFO:tasks.workunit.client.0.smithi117.stdout:journalct 69014 root cwd unknown /proc/69014/cwd (readlink: Permission denied) 2024-06-03T03:27:58.729 INFO:tasks.workunit.client.0.smithi117.stdout:journalct 69014 root rtd unknown /proc/69014/root (readlink: Permission denied) 2024-06-03T03:27:58.730 INFO:tasks.workunit.client.0.smithi117.stdout:journalct 69014 root txt unknown /proc/69014/exe (readlink: Permission denied) 2024-06-03T03:27:58.730 INFO:tasks.workunit.client.0.smithi117.stdout:journalct 69014 root NOFD /proc/69014/fd (opendir: Permission denied) 2024-06-03T03:27:58.730 INFO:tasks.workunit.client.0.smithi117.stdout:conmon 69608 root cwd unknown /proc/69608/cwd (readlink: Permission denied) 2024-06-03T03:27:58.730 INFO:tasks.workunit.client.0.smithi117.stdout:conmon 69608 root rtd unknown /proc/69608/root (readlink: Permission denied) 2024-06-03T03:27:58.730 INFO:tasks.workunit.client.0.smithi117.stdout:conmon 69608 root txt unknown /proc/69608/exe (readlink: Permission denied) 2024-06-03T03:27:58.730 INFO:tasks.workunit.client.0.smithi117.stdout:conmon 69608 root NOFD /proc/69608/fd (opendir: Permission denied) 2024-06-03T03:27:58.730 INFO:tasks.workunit.client.0.smithi117.stdout:podman-in 69610 root cwd unknown /proc/69610/cwd (readlink: Permission denied) 2024-06-03T03:27:58.730 INFO:tasks.workunit.client.0.smithi117.stdout:podman-in 69610 root rtd unknown /proc/69610/root (readlink: Permission denied) 2024-06-03T03:27:58.730 INFO:tasks.workunit.client.0.smithi117.stdout:podman-in 69610 root txt unknown /proc/69610/exe (readlink: Permission denied) 2024-06-03T03:27:58.730 INFO:tasks.workunit.client.0.smithi117.stdout:podman-in 69610 root NOFD /proc/69610/fd (opendir: Permission denied) 2024-06-03T03:27:58.730 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 ceph cwd unknown /proc/69612/cwd (readlink: Permission denied) 2024-06-03T03:27:58.730 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 ceph rtd unknown /proc/69612/root (readlink: Permission denied) 2024-06-03T03:27:58.730 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 ceph txt unknown /proc/69612/exe (readlink: Permission denied) 2024-06-03T03:27:58.731 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 ceph NOFD /proc/69612/fd (opendir: Permission denied) 2024-06-03T03:27:58.731 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 69613 log ceph cwd unknown /proc/69612/task/69613/cwd (readlink: Permission denied) 2024-06-03T03:27:58.731 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 69613 log ceph rtd unknown /proc/69612/task/69613/root (readlink: Permission denied) 2024-06-03T03:27:58.731 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 69613 log ceph txt unknown /proc/69612/task/69613/exe (readlink: Permission denied) 2024-06-03T03:27:58.731 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 69613 log ceph NOFD /proc/69612/task/69613/fd (opendir: Permission denied) 2024-06-03T03:27:58.731 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 69615 msgr-work ceph cwd unknown /proc/69612/task/69615/cwd (readlink: Permission denied) 2024-06-03T03:27:58.731 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 69615 msgr-work ceph rtd unknown /proc/69612/task/69615/root (readlink: Permission denied) 2024-06-03T03:27:58.731 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 69615 msgr-work ceph txt unknown /proc/69612/task/69615/exe (readlink: Permission denied) 2024-06-03T03:27:58.731 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 69615 msgr-work ceph NOFD /proc/69612/task/69615/fd (opendir: Permission denied) 2024-06-03T03:27:58.731 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 69616 msgr-work ceph cwd unknown /proc/69612/task/69616/cwd (readlink: Permission denied) 2024-06-03T03:27:58.731 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 69616 msgr-work ceph rtd unknown /proc/69612/task/69616/root (readlink: Permission denied) 2024-06-03T03:27:58.731 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 69616 msgr-work ceph txt unknown /proc/69612/task/69616/exe (readlink: Permission denied) 2024-06-03T03:27:58.731 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 69616 msgr-work ceph NOFD /proc/69612/task/69616/fd (opendir: Permission denied) 2024-06-03T03:27:58.732 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 69617 msgr-work ceph cwd unknown /proc/69612/task/69617/cwd (readlink: Permission denied) 2024-06-03T03:27:58.732 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 69617 msgr-work ceph rtd unknown /proc/69612/task/69617/root (readlink: Permission denied) 2024-06-03T03:27:58.732 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 69617 msgr-work ceph txt unknown /proc/69612/task/69617/exe (readlink: Permission denied) 2024-06-03T03:27:58.732 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 69617 msgr-work ceph NOFD /proc/69612/task/69617/fd (opendir: Permission denied) 2024-06-03T03:27:58.732 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 69621 service ceph cwd unknown /proc/69612/task/69621/cwd (readlink: Permission denied) 2024-06-03T03:27:58.732 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 69621 service ceph rtd unknown /proc/69612/task/69621/root (readlink: Permission denied) 2024-06-03T03:27:58.732 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 69621 service ceph txt unknown /proc/69612/task/69621/exe (readlink: Permission denied) 2024-06-03T03:27:58.732 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 69621 service ceph NOFD /proc/69612/task/69621/fd (opendir: Permission denied) 2024-06-03T03:27:58.732 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 69622 admin_soc ceph cwd unknown /proc/69612/task/69622/cwd (readlink: Permission denied) 2024-06-03T03:27:58.732 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 69622 admin_soc ceph rtd unknown /proc/69612/task/69622/root (readlink: Permission denied) 2024-06-03T03:27:58.732 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 69622 admin_soc ceph txt unknown /proc/69612/task/69622/exe (readlink: Permission denied) 2024-06-03T03:27:58.732 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 69622 admin_soc ceph NOFD /proc/69612/task/69622/fd (opendir: Permission denied) 2024-06-03T03:27:58.732 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 69623 signal_ha ceph cwd unknown /proc/69612/task/69623/cwd (readlink: Permission denied) 2024-06-03T03:27:58.733 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 69623 signal_ha ceph rtd unknown /proc/69612/task/69623/root (readlink: Permission denied) 2024-06-03T03:27:58.733 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 69623 signal_ha ceph txt unknown /proc/69612/task/69623/exe (readlink: Permission denied) 2024-06-03T03:27:58.733 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 69623 signal_ha ceph NOFD /proc/69612/task/69623/fd (opendir: Permission denied) 2024-06-03T03:27:58.733 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 69624 io_contex ceph cwd unknown /proc/69612/task/69624/cwd (readlink: Permission denied) 2024-06-03T03:27:58.733 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 69624 io_contex ceph rtd unknown /proc/69612/task/69624/root (readlink: Permission denied) 2024-06-03T03:27:58.733 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 69624 io_contex ceph txt unknown /proc/69612/task/69624/exe (readlink: Permission denied) 2024-06-03T03:27:58.733 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 69624 io_contex ceph NOFD /proc/69612/task/69624/fd (opendir: Permission denied) 2024-06-03T03:27:58.733 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 69625 io_contex ceph cwd unknown /proc/69612/task/69625/cwd (readlink: Permission denied) 2024-06-03T03:27:58.733 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 69625 io_contex ceph rtd unknown /proc/69612/task/69625/root (readlink: Permission denied) 2024-06-03T03:27:58.733 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 69625 io_contex ceph txt unknown /proc/69612/task/69625/exe (readlink: Permission denied) 2024-06-03T03:27:58.733 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 69625 io_contex ceph NOFD /proc/69612/task/69625/fd (opendir: Permission denied) 2024-06-03T03:27:58.733 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 69626 ms_dispat ceph cwd unknown /proc/69612/task/69626/cwd (readlink: Permission denied) 2024-06-03T03:27:58.733 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 69626 ms_dispat ceph rtd unknown /proc/69612/task/69626/root (readlink: Permission denied) 2024-06-03T03:27:58.733 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 69626 ms_dispat ceph txt unknown /proc/69612/task/69626/exe (readlink: Permission denied) 2024-06-03T03:27:58.734 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 69626 ms_dispat ceph NOFD /proc/69612/task/69626/fd (opendir: Permission denied) 2024-06-03T03:27:58.734 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 69627 ms_local ceph cwd unknown /proc/69612/task/69627/cwd (readlink: Permission denied) 2024-06-03T03:27:58.734 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 69627 ms_local ceph rtd unknown /proc/69612/task/69627/root (readlink: Permission denied) 2024-06-03T03:27:58.734 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 69627 ms_local ceph txt unknown /proc/69612/task/69627/exe (readlink: Permission denied) 2024-06-03T03:27:58.734 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 69627 ms_local ceph NOFD /proc/69612/task/69627/fd (opendir: Permission denied) 2024-06-03T03:27:58.734 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 69628 safe_time ceph cwd unknown /proc/69612/task/69628/cwd (readlink: Permission denied) 2024-06-03T03:27:58.734 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 69628 safe_time ceph rtd unknown /proc/69612/task/69628/root (readlink: Permission denied) 2024-06-03T03:27:58.734 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 69628 safe_time ceph txt unknown /proc/69612/task/69628/exe (readlink: Permission denied) 2024-06-03T03:27:58.734 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 69628 safe_time ceph NOFD /proc/69612/task/69628/fd (opendir: Permission denied) 2024-06-03T03:27:58.734 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 69629 safe_time ceph cwd unknown /proc/69612/task/69629/cwd (readlink: Permission denied) 2024-06-03T03:27:58.734 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 69629 safe_time ceph rtd unknown /proc/69612/task/69629/root (readlink: Permission denied) 2024-06-03T03:27:58.734 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 69629 safe_time ceph txt unknown /proc/69612/task/69629/exe (readlink: Permission denied) 2024-06-03T03:27:58.734 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 69629 safe_time ceph NOFD /proc/69612/task/69629/fd (opendir: Permission denied) 2024-06-03T03:27:58.735 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 69630 ceph-mds ceph cwd unknown /proc/69612/task/69630/cwd (readlink: Permission denied) 2024-06-03T03:27:58.735 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 69630 ceph-mds ceph rtd unknown /proc/69612/task/69630/root (readlink: Permission denied) 2024-06-03T03:27:58.735 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 69630 ceph-mds ceph txt unknown /proc/69612/task/69630/exe (readlink: Permission denied) 2024-06-03T03:27:58.735 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 69630 ceph-mds ceph NOFD /proc/69612/task/69630/fd (opendir: Permission denied) 2024-06-03T03:27:58.735 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 69631 safe_time ceph cwd unknown /proc/69612/task/69631/cwd (readlink: Permission denied) 2024-06-03T03:27:58.735 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 69631 safe_time ceph rtd unknown /proc/69612/task/69631/root (readlink: Permission denied) 2024-06-03T03:27:58.735 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 69631 safe_time ceph txt unknown /proc/69612/task/69631/exe (readlink: Permission denied) 2024-06-03T03:27:58.735 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 69631 safe_time ceph NOFD /proc/69612/task/69631/fd (opendir: Permission denied) 2024-06-03T03:27:58.735 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 76712 ceph_time ceph cwd unknown /proc/69612/task/76712/cwd (readlink: Permission denied) 2024-06-03T03:27:58.735 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 76712 ceph_time ceph rtd unknown /proc/69612/task/76712/root (readlink: Permission denied) 2024-06-03T03:27:58.735 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 76712 ceph_time ceph txt unknown /proc/69612/task/76712/exe (readlink: Permission denied) 2024-06-03T03:27:58.735 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 76712 ceph_time ceph NOFD /proc/69612/task/76712/fd (opendir: Permission denied) 2024-06-03T03:27:58.735 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 76713 OpHistory ceph cwd unknown /proc/69612/task/76713/cwd (readlink: Permission denied) 2024-06-03T03:27:58.736 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 76713 OpHistory ceph rtd unknown /proc/69612/task/76713/root (readlink: Permission denied) 2024-06-03T03:27:58.736 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 76713 OpHistory ceph txt unknown /proc/69612/task/76713/exe (readlink: Permission denied) 2024-06-03T03:27:58.736 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 76713 OpHistory ceph NOFD /proc/69612/task/76713/fd (opendir: Permission denied) 2024-06-03T03:27:58.736 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 76714 ms_dispat ceph cwd unknown /proc/69612/task/76714/cwd (readlink: Permission denied) 2024-06-03T03:27:58.736 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 76714 ms_dispat ceph rtd unknown /proc/69612/task/76714/root (readlink: Permission denied) 2024-06-03T03:27:58.736 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 76714 ms_dispat ceph txt unknown /proc/69612/task/76714/exe (readlink: Permission denied) 2024-06-03T03:27:58.736 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 76714 ms_dispat ceph NOFD /proc/69612/task/76714/fd (opendir: Permission denied) 2024-06-03T03:27:58.736 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 76715 ms_dispat ceph cwd unknown /proc/69612/task/76715/cwd (readlink: Permission denied) 2024-06-03T03:27:58.736 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 76715 ms_dispat ceph rtd unknown /proc/69612/task/76715/root (readlink: Permission denied) 2024-06-03T03:27:58.736 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 76715 ms_dispat ceph txt unknown /proc/69612/task/76715/exe (readlink: Permission denied) 2024-06-03T03:27:58.736 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 76715 ms_dispat ceph NOFD /proc/69612/task/76715/fd (opendir: Permission denied) 2024-06-03T03:27:58.736 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 76716 mds_rank_ ceph cwd unknown /proc/69612/task/76716/cwd (readlink: Permission denied) 2024-06-03T03:27:58.736 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 76716 mds_rank_ ceph rtd unknown /proc/69612/task/76716/root (readlink: Permission denied) 2024-06-03T03:27:58.736 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 76716 mds_rank_ ceph txt unknown /proc/69612/task/76716/exe (readlink: Permission denied) 2024-06-03T03:27:58.737 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 76716 mds_rank_ ceph NOFD /proc/69612/task/76716/fd (opendir: Permission denied) 2024-06-03T03:27:58.737 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 76717 PQ_Finish ceph cwd unknown /proc/69612/task/76717/cwd (readlink: Permission denied) 2024-06-03T03:27:58.737 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 76717 PQ_Finish ceph rtd unknown /proc/69612/task/76717/root (readlink: Permission denied) 2024-06-03T03:27:58.737 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 76717 PQ_Finish ceph txt unknown /proc/69612/task/76717/exe (readlink: Permission denied) 2024-06-03T03:27:58.737 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 76717 PQ_Finish ceph NOFD /proc/69612/task/76717/fd (opendir: Permission denied) 2024-06-03T03:27:58.737 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 76718 safe_time ceph cwd unknown /proc/69612/task/76718/cwd (readlink: Permission denied) 2024-06-03T03:27:58.737 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 76718 safe_time ceph rtd unknown /proc/69612/task/76718/root (readlink: Permission denied) 2024-06-03T03:27:58.737 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 76718 safe_time ceph txt unknown /proc/69612/task/76718/exe (readlink: Permission denied) 2024-06-03T03:27:58.737 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 76718 safe_time ceph NOFD /proc/69612/task/76718/fd (opendir: Permission denied) 2024-06-03T03:27:58.737 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 76719 MR_Finish ceph cwd unknown /proc/69612/task/76719/cwd (readlink: Permission denied) 2024-06-03T03:27:58.737 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 76719 MR_Finish ceph rtd unknown /proc/69612/task/76719/root (readlink: Permission denied) 2024-06-03T03:27:58.737 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 76719 MR_Finish ceph txt unknown /proc/69612/task/76719/exe (readlink: Permission denied) 2024-06-03T03:27:58.737 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 76719 MR_Finish ceph NOFD /proc/69612/task/76719/fd (opendir: Permission denied) 2024-06-03T03:27:58.737 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 76721 md_submit ceph cwd unknown /proc/69612/task/76721/cwd (readlink: Permission denied) 2024-06-03T03:27:58.737 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 76721 md_submit ceph rtd unknown /proc/69612/task/76721/root (readlink: Permission denied) 2024-06-03T03:27:58.738 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 76721 md_submit ceph txt unknown /proc/69612/task/76721/exe (readlink: Permission denied) 2024-06-03T03:27:58.738 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 69612 76721 md_submit ceph NOFD /proc/69612/task/76721/fd (opendir: Permission denied) 2024-06-03T03:27:58.738 INFO:tasks.workunit.client.0.smithi117.stdout:conmon 70018 root cwd unknown /proc/70018/cwd (readlink: Permission denied) 2024-06-03T03:27:58.738 INFO:tasks.workunit.client.0.smithi117.stdout:conmon 70018 root rtd unknown /proc/70018/root (readlink: Permission denied) 2024-06-03T03:27:58.738 INFO:tasks.workunit.client.0.smithi117.stdout:conmon 70018 root txt unknown /proc/70018/exe (readlink: Permission denied) 2024-06-03T03:27:58.738 INFO:tasks.workunit.client.0.smithi117.stdout:conmon 70018 root NOFD /proc/70018/fd (opendir: Permission denied) 2024-06-03T03:27:58.738 INFO:tasks.workunit.client.0.smithi117.stdout:podman-in 70021 root cwd unknown /proc/70021/cwd (readlink: Permission denied) 2024-06-03T03:27:58.738 INFO:tasks.workunit.client.0.smithi117.stdout:podman-in 70021 root rtd unknown /proc/70021/root (readlink: Permission denied) 2024-06-03T03:27:58.738 INFO:tasks.workunit.client.0.smithi117.stdout:podman-in 70021 root txt unknown /proc/70021/exe (readlink: Permission denied) 2024-06-03T03:27:58.738 INFO:tasks.workunit.client.0.smithi117.stdout:podman-in 70021 root NOFD /proc/70021/fd (opendir: Permission denied) 2024-06-03T03:27:58.738 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70027 ceph cwd unknown /proc/70027/cwd (readlink: Permission denied) 2024-06-03T03:27:58.738 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70027 ceph rtd unknown /proc/70027/root (readlink: Permission denied) 2024-06-03T03:27:58.738 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70027 ceph txt unknown /proc/70027/exe (readlink: Permission denied) 2024-06-03T03:27:58.739 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70027 ceph NOFD /proc/70027/fd (opendir: Permission denied) 2024-06-03T03:27:58.739 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70027 70028 log ceph cwd unknown /proc/70027/task/70028/cwd (readlink: Permission denied) 2024-06-03T03:27:58.739 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70027 70028 log ceph rtd unknown /proc/70027/task/70028/root (readlink: Permission denied) 2024-06-03T03:27:58.739 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70027 70028 log ceph txt unknown /proc/70027/task/70028/exe (readlink: Permission denied) 2024-06-03T03:27:58.739 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70027 70028 log ceph NOFD /proc/70027/task/70028/fd (opendir: Permission denied) 2024-06-03T03:27:58.739 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70027 70030 msgr-work ceph cwd unknown /proc/70027/task/70030/cwd (readlink: Permission denied) 2024-06-03T03:27:58.739 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70027 70030 msgr-work ceph rtd unknown /proc/70027/task/70030/root (readlink: Permission denied) 2024-06-03T03:27:58.739 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70027 70030 msgr-work ceph txt unknown /proc/70027/task/70030/exe (readlink: Permission denied) 2024-06-03T03:27:58.739 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70027 70030 msgr-work ceph NOFD /proc/70027/task/70030/fd (opendir: Permission denied) 2024-06-03T03:27:58.739 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70027 70031 msgr-work ceph cwd unknown /proc/70027/task/70031/cwd (readlink: Permission denied) 2024-06-03T03:27:58.739 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70027 70031 msgr-work ceph rtd unknown /proc/70027/task/70031/root (readlink: Permission denied) 2024-06-03T03:27:58.739 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70027 70031 msgr-work ceph txt unknown /proc/70027/task/70031/exe (readlink: Permission denied) 2024-06-03T03:27:58.740 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70027 70031 msgr-work ceph NOFD /proc/70027/task/70031/fd (opendir: Permission denied) 2024-06-03T03:27:58.740 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70027 70032 msgr-work ceph cwd unknown /proc/70027/task/70032/cwd (readlink: Permission denied) 2024-06-03T03:27:58.740 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70027 70032 msgr-work ceph rtd unknown /proc/70027/task/70032/root (readlink: Permission denied) 2024-06-03T03:27:58.740 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70027 70032 msgr-work ceph txt unknown /proc/70027/task/70032/exe (readlink: Permission denied) 2024-06-03T03:27:58.740 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70027 70032 msgr-work ceph NOFD /proc/70027/task/70032/fd (opendir: Permission denied) 2024-06-03T03:27:58.740 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70027 70036 service ceph cwd unknown /proc/70027/task/70036/cwd (readlink: Permission denied) 2024-06-03T03:27:58.740 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70027 70036 service ceph rtd unknown /proc/70027/task/70036/root (readlink: Permission denied) 2024-06-03T03:27:58.740 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70027 70036 service ceph txt unknown /proc/70027/task/70036/exe (readlink: Permission denied) 2024-06-03T03:27:58.740 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70027 70036 service ceph NOFD /proc/70027/task/70036/fd (opendir: Permission denied) 2024-06-03T03:27:58.740 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70027 70037 admin_soc ceph cwd unknown /proc/70027/task/70037/cwd (readlink: Permission denied) 2024-06-03T03:27:58.740 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70027 70037 admin_soc ceph rtd unknown /proc/70027/task/70037/root (readlink: Permission denied) 2024-06-03T03:27:58.740 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70027 70037 admin_soc ceph txt unknown /proc/70027/task/70037/exe (readlink: Permission denied) 2024-06-03T03:27:58.740 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70027 70037 admin_soc ceph NOFD /proc/70027/task/70037/fd (opendir: Permission denied) 2024-06-03T03:27:58.741 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70027 70038 signal_ha ceph cwd unknown /proc/70027/task/70038/cwd (readlink: Permission denied) 2024-06-03T03:27:58.741 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70027 70038 signal_ha ceph rtd unknown /proc/70027/task/70038/root (readlink: Permission denied) 2024-06-03T03:27:58.741 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70027 70038 signal_ha ceph txt unknown /proc/70027/task/70038/exe (readlink: Permission denied) 2024-06-03T03:27:58.741 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70027 70038 signal_ha ceph NOFD /proc/70027/task/70038/fd (opendir: Permission denied) 2024-06-03T03:27:58.741 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70027 70039 io_contex ceph cwd unknown /proc/70027/task/70039/cwd (readlink: Permission denied) 2024-06-03T03:27:58.741 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70027 70039 io_contex ceph rtd unknown /proc/70027/task/70039/root (readlink: Permission denied) 2024-06-03T03:27:58.741 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70027 70039 io_contex ceph txt unknown /proc/70027/task/70039/exe (readlink: Permission denied) 2024-06-03T03:27:58.741 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70027 70039 io_contex ceph NOFD /proc/70027/task/70039/fd (opendir: Permission denied) 2024-06-03T03:27:58.741 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70027 70040 io_contex ceph cwd unknown /proc/70027/task/70040/cwd (readlink: Permission denied) 2024-06-03T03:27:58.742 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70027 70040 io_contex ceph rtd unknown /proc/70027/task/70040/root (readlink: Permission denied) 2024-06-03T03:27:58.742 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70027 70040 io_contex ceph txt unknown /proc/70027/task/70040/exe (readlink: Permission denied) 2024-06-03T03:27:58.742 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70027 70040 io_contex ceph NOFD /proc/70027/task/70040/fd (opendir: Permission denied) 2024-06-03T03:27:58.742 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70027 70041 ms_dispat ceph cwd unknown /proc/70027/task/70041/cwd (readlink: Permission denied) 2024-06-03T03:27:58.742 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70027 70041 ms_dispat ceph rtd unknown /proc/70027/task/70041/root (readlink: Permission denied) 2024-06-03T03:27:58.742 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70027 70041 ms_dispat ceph txt unknown /proc/70027/task/70041/exe (readlink: Permission denied) 2024-06-03T03:27:58.742 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70027 70041 ms_dispat ceph NOFD /proc/70027/task/70041/fd (opendir: Permission denied) 2024-06-03T03:27:58.742 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70027 70042 ms_local ceph cwd unknown /proc/70027/task/70042/cwd (readlink: Permission denied) 2024-06-03T03:27:58.743 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70027 70042 ms_local ceph rtd unknown /proc/70027/task/70042/root (readlink: Permission denied) 2024-06-03T03:27:58.743 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70027 70042 ms_local ceph txt unknown /proc/70027/task/70042/exe (readlink: Permission denied) 2024-06-03T03:27:58.743 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70027 70042 ms_local ceph NOFD /proc/70027/task/70042/fd (opendir: Permission denied) 2024-06-03T03:27:58.743 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70027 70043 safe_time ceph cwd unknown /proc/70027/task/70043/cwd (readlink: Permission denied) 2024-06-03T03:27:58.743 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70027 70043 safe_time ceph rtd unknown /proc/70027/task/70043/root (readlink: Permission denied) 2024-06-03T03:27:58.743 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70027 70043 safe_time ceph txt unknown /proc/70027/task/70043/exe (readlink: Permission denied) 2024-06-03T03:27:58.743 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70027 70043 safe_time ceph NOFD /proc/70027/task/70043/fd (opendir: Permission denied) 2024-06-03T03:27:58.743 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70027 70044 safe_time ceph cwd unknown /proc/70027/task/70044/cwd (readlink: Permission denied) 2024-06-03T03:27:58.744 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70027 70044 safe_time ceph rtd unknown /proc/70027/task/70044/root (readlink: Permission denied) 2024-06-03T03:27:58.744 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70027 70044 safe_time ceph txt unknown /proc/70027/task/70044/exe (readlink: Permission denied) 2024-06-03T03:27:58.744 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70027 70044 safe_time ceph NOFD /proc/70027/task/70044/fd (opendir: Permission denied) 2024-06-03T03:27:58.744 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70027 70045 ceph-mds ceph cwd unknown /proc/70027/task/70045/cwd (readlink: Permission denied) 2024-06-03T03:27:58.744 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70027 70045 ceph-mds ceph rtd unknown /proc/70027/task/70045/root (readlink: Permission denied) 2024-06-03T03:27:58.744 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70027 70045 ceph-mds ceph txt unknown /proc/70027/task/70045/exe (readlink: Permission denied) 2024-06-03T03:27:58.744 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70027 70045 ceph-mds ceph NOFD /proc/70027/task/70045/fd (opendir: Permission denied) 2024-06-03T03:27:58.744 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70027 70046 safe_time ceph cwd unknown /proc/70027/task/70046/cwd (readlink: Permission denied) 2024-06-03T03:27:58.744 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70027 70046 safe_time ceph rtd unknown /proc/70027/task/70046/root (readlink: Permission denied) 2024-06-03T03:27:58.745 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70027 70046 safe_time ceph txt unknown /proc/70027/task/70046/exe (readlink: Permission denied) 2024-06-03T03:27:58.745 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70027 70046 safe_time ceph NOFD /proc/70027/task/70046/fd (opendir: Permission denied) 2024-06-03T03:27:58.745 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/2 70715 root cwd unknown /proc/70715/cwd (readlink: Permission denied) 2024-06-03T03:27:58.745 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/2 70715 root rtd unknown /proc/70715/root (readlink: Permission denied) 2024-06-03T03:27:58.745 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/2 70715 root txt unknown /proc/70715/exe (readlink: Permission denied) 2024-06-03T03:27:58.745 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/2 70715 root NOFD /proc/70715/fd (opendir: Permission denied) 2024-06-03T03:27:58.745 INFO:tasks.workunit.client.0.smithi117.stdout:conmon 70802 root cwd unknown /proc/70802/cwd (readlink: Permission denied) 2024-06-03T03:27:58.745 INFO:tasks.workunit.client.0.smithi117.stdout:conmon 70802 root rtd unknown /proc/70802/root (readlink: Permission denied) 2024-06-03T03:27:58.746 INFO:tasks.workunit.client.0.smithi117.stdout:conmon 70802 root txt unknown /proc/70802/exe (readlink: Permission denied) 2024-06-03T03:27:58.746 INFO:tasks.workunit.client.0.smithi117.stdout:conmon 70802 root NOFD /proc/70802/fd (opendir: Permission denied) 2024-06-03T03:27:58.746 INFO:tasks.workunit.client.0.smithi117.stdout:podman-in 70804 root cwd unknown /proc/70804/cwd (readlink: Permission denied) 2024-06-03T03:27:58.746 INFO:tasks.workunit.client.0.smithi117.stdout:podman-in 70804 root rtd unknown /proc/70804/root (readlink: Permission denied) 2024-06-03T03:27:58.746 INFO:tasks.workunit.client.0.smithi117.stdout:podman-in 70804 root txt unknown /proc/70804/exe (readlink: Permission denied) 2024-06-03T03:27:58.746 INFO:tasks.workunit.client.0.smithi117.stdout:podman-in 70804 root NOFD /proc/70804/fd (opendir: Permission denied) 2024-06-03T03:27:58.746 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70811 ceph cwd unknown /proc/70811/cwd (readlink: Permission denied) 2024-06-03T03:27:58.746 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70811 ceph rtd unknown /proc/70811/root (readlink: Permission denied) 2024-06-03T03:27:58.747 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70811 ceph txt unknown /proc/70811/exe (readlink: Permission denied) 2024-06-03T03:27:58.747 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70811 ceph NOFD /proc/70811/fd (opendir: Permission denied) 2024-06-03T03:27:58.747 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70811 70812 log ceph cwd unknown /proc/70811/task/70812/cwd (readlink: Permission denied) 2024-06-03T03:27:58.747 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70811 70812 log ceph rtd unknown /proc/70811/task/70812/root (readlink: Permission denied) 2024-06-03T03:27:58.747 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70811 70812 log ceph txt unknown /proc/70811/task/70812/exe (readlink: Permission denied) 2024-06-03T03:27:58.747 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70811 70812 log ceph NOFD /proc/70811/task/70812/fd (opendir: Permission denied) 2024-06-03T03:27:58.747 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70811 70814 msgr-work ceph cwd unknown /proc/70811/task/70814/cwd (readlink: Permission denied) 2024-06-03T03:27:58.747 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70811 70814 msgr-work ceph rtd unknown /proc/70811/task/70814/root (readlink: Permission denied) 2024-06-03T03:27:58.748 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70811 70814 msgr-work ceph txt unknown /proc/70811/task/70814/exe (readlink: Permission denied) 2024-06-03T03:27:58.748 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70811 70814 msgr-work ceph NOFD /proc/70811/task/70814/fd (opendir: Permission denied) 2024-06-03T03:27:58.748 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70811 70815 msgr-work ceph cwd unknown /proc/70811/task/70815/cwd (readlink: Permission denied) 2024-06-03T03:27:58.748 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70811 70815 msgr-work ceph rtd unknown /proc/70811/task/70815/root (readlink: Permission denied) 2024-06-03T03:27:58.748 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70811 70815 msgr-work ceph txt unknown /proc/70811/task/70815/exe (readlink: Permission denied) 2024-06-03T03:27:58.748 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70811 70815 msgr-work ceph NOFD /proc/70811/task/70815/fd (opendir: Permission denied) 2024-06-03T03:27:58.748 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70811 70816 msgr-work ceph cwd unknown /proc/70811/task/70816/cwd (readlink: Permission denied) 2024-06-03T03:27:58.748 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70811 70816 msgr-work ceph rtd unknown /proc/70811/task/70816/root (readlink: Permission denied) 2024-06-03T03:27:58.748 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70811 70816 msgr-work ceph txt unknown /proc/70811/task/70816/exe (readlink: Permission denied) 2024-06-03T03:27:58.749 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70811 70816 msgr-work ceph NOFD /proc/70811/task/70816/fd (opendir: Permission denied) 2024-06-03T03:27:58.749 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70811 70820 service ceph cwd unknown /proc/70811/task/70820/cwd (readlink: Permission denied) 2024-06-03T03:27:58.749 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70811 70820 service ceph rtd unknown /proc/70811/task/70820/root (readlink: Permission denied) 2024-06-03T03:27:58.749 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70811 70820 service ceph txt unknown /proc/70811/task/70820/exe (readlink: Permission denied) 2024-06-03T03:27:58.749 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70811 70820 service ceph NOFD /proc/70811/task/70820/fd (opendir: Permission denied) 2024-06-03T03:27:58.749 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70811 70821 admin_soc ceph cwd unknown /proc/70811/task/70821/cwd (readlink: Permission denied) 2024-06-03T03:27:58.749 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70811 70821 admin_soc ceph rtd unknown /proc/70811/task/70821/root (readlink: Permission denied) 2024-06-03T03:27:58.749 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70811 70821 admin_soc ceph txt unknown /proc/70811/task/70821/exe (readlink: Permission denied) 2024-06-03T03:27:58.750 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70811 70821 admin_soc ceph NOFD /proc/70811/task/70821/fd (opendir: Permission denied) 2024-06-03T03:27:58.750 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70811 70822 signal_ha ceph cwd unknown /proc/70811/task/70822/cwd (readlink: Permission denied) 2024-06-03T03:27:58.750 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70811 70822 signal_ha ceph rtd unknown /proc/70811/task/70822/root (readlink: Permission denied) 2024-06-03T03:27:58.750 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70811 70822 signal_ha ceph txt unknown /proc/70811/task/70822/exe (readlink: Permission denied) 2024-06-03T03:27:58.750 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70811 70822 signal_ha ceph NOFD /proc/70811/task/70822/fd (opendir: Permission denied) 2024-06-03T03:27:58.750 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70811 70823 io_contex ceph cwd unknown /proc/70811/task/70823/cwd (readlink: Permission denied) 2024-06-03T03:27:58.750 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70811 70823 io_contex ceph rtd unknown /proc/70811/task/70823/root (readlink: Permission denied) 2024-06-03T03:27:58.750 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70811 70823 io_contex ceph txt unknown /proc/70811/task/70823/exe (readlink: Permission denied) 2024-06-03T03:27:58.750 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70811 70823 io_contex ceph NOFD /proc/70811/task/70823/fd (opendir: Permission denied) 2024-06-03T03:27:58.751 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70811 70824 io_contex ceph cwd unknown /proc/70811/task/70824/cwd (readlink: Permission denied) 2024-06-03T03:27:58.751 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70811 70824 io_contex ceph rtd unknown /proc/70811/task/70824/root (readlink: Permission denied) 2024-06-03T03:27:58.751 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70811 70824 io_contex ceph txt unknown /proc/70811/task/70824/exe (readlink: Permission denied) 2024-06-03T03:27:58.751 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70811 70824 io_contex ceph NOFD /proc/70811/task/70824/fd (opendir: Permission denied) 2024-06-03T03:27:58.751 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70811 70825 ms_dispat ceph cwd unknown /proc/70811/task/70825/cwd (readlink: Permission denied) 2024-06-03T03:27:58.751 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70811 70825 ms_dispat ceph rtd unknown /proc/70811/task/70825/root (readlink: Permission denied) 2024-06-03T03:27:58.751 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70811 70825 ms_dispat ceph txt unknown /proc/70811/task/70825/exe (readlink: Permission denied) 2024-06-03T03:27:58.751 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70811 70825 ms_dispat ceph NOFD /proc/70811/task/70825/fd (opendir: Permission denied) 2024-06-03T03:27:58.752 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70811 70826 ms_local ceph cwd unknown /proc/70811/task/70826/cwd (readlink: Permission denied) 2024-06-03T03:27:58.752 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70811 70826 ms_local ceph rtd unknown /proc/70811/task/70826/root (readlink: Permission denied) 2024-06-03T03:27:58.752 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70811 70826 ms_local ceph txt unknown /proc/70811/task/70826/exe (readlink: Permission denied) 2024-06-03T03:27:58.752 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70811 70826 ms_local ceph NOFD /proc/70811/task/70826/fd (opendir: Permission denied) 2024-06-03T03:27:58.752 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70811 70827 safe_time ceph cwd unknown /proc/70811/task/70827/cwd (readlink: Permission denied) 2024-06-03T03:27:58.752 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70811 70827 safe_time ceph rtd unknown /proc/70811/task/70827/root (readlink: Permission denied) 2024-06-03T03:27:58.752 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70811 70827 safe_time ceph txt unknown /proc/70811/task/70827/exe (readlink: Permission denied) 2024-06-03T03:27:58.752 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70811 70827 safe_time ceph NOFD /proc/70811/task/70827/fd (opendir: Permission denied) 2024-06-03T03:27:58.752 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70811 70828 safe_time ceph cwd unknown /proc/70811/task/70828/cwd (readlink: Permission denied) 2024-06-03T03:27:58.753 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70811 70828 safe_time ceph rtd unknown /proc/70811/task/70828/root (readlink: Permission denied) 2024-06-03T03:27:58.753 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70811 70828 safe_time ceph txt unknown /proc/70811/task/70828/exe (readlink: Permission denied) 2024-06-03T03:27:58.753 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70811 70828 safe_time ceph NOFD /proc/70811/task/70828/fd (opendir: Permission denied) 2024-06-03T03:27:58.753 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70811 70829 ceph-mds ceph cwd unknown /proc/70811/task/70829/cwd (readlink: Permission denied) 2024-06-03T03:27:58.753 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70811 70829 ceph-mds ceph rtd unknown /proc/70811/task/70829/root (readlink: Permission denied) 2024-06-03T03:27:58.753 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70811 70829 ceph-mds ceph txt unknown /proc/70811/task/70829/exe (readlink: Permission denied) 2024-06-03T03:27:58.753 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70811 70829 ceph-mds ceph NOFD /proc/70811/task/70829/fd (opendir: Permission denied) 2024-06-03T03:27:58.753 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70811 70877 safe_time ceph cwd unknown /proc/70811/task/70877/cwd (readlink: Permission denied) 2024-06-03T03:27:58.754 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70811 70877 safe_time ceph rtd unknown /proc/70811/task/70877/root (readlink: Permission denied) 2024-06-03T03:27:58.754 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70811 70877 safe_time ceph txt unknown /proc/70811/task/70877/exe (readlink: Permission denied) 2024-06-03T03:27:58.754 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 70811 70877 safe_time ceph NOFD /proc/70811/task/70877/fd (opendir: Permission denied) 2024-06-03T03:27:58.754 INFO:tasks.workunit.client.0.smithi117.stdout:conmon 71198 root cwd unknown /proc/71198/cwd (readlink: Permission denied) 2024-06-03T03:27:58.754 INFO:tasks.workunit.client.0.smithi117.stdout:conmon 71198 root rtd unknown /proc/71198/root (readlink: Permission denied) 2024-06-03T03:27:58.754 INFO:tasks.workunit.client.0.smithi117.stdout:conmon 71198 root txt unknown /proc/71198/exe (readlink: Permission denied) 2024-06-03T03:27:58.754 INFO:tasks.workunit.client.0.smithi117.stdout:conmon 71198 root NOFD /proc/71198/fd (opendir: Permission denied) 2024-06-03T03:27:58.754 INFO:tasks.workunit.client.0.smithi117.stdout:podman-in 71200 root cwd unknown /proc/71200/cwd (readlink: Permission denied) 2024-06-03T03:27:58.754 INFO:tasks.workunit.client.0.smithi117.stdout:podman-in 71200 root rtd unknown /proc/71200/root (readlink: Permission denied) 2024-06-03T03:27:58.754 INFO:tasks.workunit.client.0.smithi117.stdout:podman-in 71200 root txt unknown /proc/71200/exe (readlink: Permission denied) 2024-06-03T03:27:58.754 INFO:tasks.workunit.client.0.smithi117.stdout:podman-in 71200 root NOFD /proc/71200/fd (opendir: Permission denied) 2024-06-03T03:27:58.754 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 ceph cwd unknown /proc/71202/cwd (readlink: Permission denied) 2024-06-03T03:27:58.754 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 ceph rtd unknown /proc/71202/root (readlink: Permission denied) 2024-06-03T03:27:58.755 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 ceph txt unknown /proc/71202/exe (readlink: Permission denied) 2024-06-03T03:27:58.755 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 ceph NOFD /proc/71202/fd (opendir: Permission denied) 2024-06-03T03:27:58.755 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 71205 log ceph cwd unknown /proc/71202/task/71205/cwd (readlink: Permission denied) 2024-06-03T03:27:58.755 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 71205 log ceph rtd unknown /proc/71202/task/71205/root (readlink: Permission denied) 2024-06-03T03:27:58.755 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 71205 log ceph txt unknown /proc/71202/task/71205/exe (readlink: Permission denied) 2024-06-03T03:27:58.755 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 71205 log ceph NOFD /proc/71202/task/71205/fd (opendir: Permission denied) 2024-06-03T03:27:58.755 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 71207 msgr-work ceph cwd unknown /proc/71202/task/71207/cwd (readlink: Permission denied) 2024-06-03T03:27:58.755 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 71207 msgr-work ceph rtd unknown /proc/71202/task/71207/root (readlink: Permission denied) 2024-06-03T03:27:58.755 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 71207 msgr-work ceph txt unknown /proc/71202/task/71207/exe (readlink: Permission denied) 2024-06-03T03:27:58.755 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 71207 msgr-work ceph NOFD /proc/71202/task/71207/fd (opendir: Permission denied) 2024-06-03T03:27:58.755 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 71208 msgr-work ceph cwd unknown /proc/71202/task/71208/cwd (readlink: Permission denied) 2024-06-03T03:27:58.755 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 71208 msgr-work ceph rtd unknown /proc/71202/task/71208/root (readlink: Permission denied) 2024-06-03T03:27:58.756 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 71208 msgr-work ceph txt unknown /proc/71202/task/71208/exe (readlink: Permission denied) 2024-06-03T03:27:58.756 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 71208 msgr-work ceph NOFD /proc/71202/task/71208/fd (opendir: Permission denied) 2024-06-03T03:27:58.756 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 71209 msgr-work ceph cwd unknown /proc/71202/task/71209/cwd (readlink: Permission denied) 2024-06-03T03:27:58.756 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 71209 msgr-work ceph rtd unknown /proc/71202/task/71209/root (readlink: Permission denied) 2024-06-03T03:27:58.756 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 71209 msgr-work ceph txt unknown /proc/71202/task/71209/exe (readlink: Permission denied) 2024-06-03T03:27:58.756 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 71209 msgr-work ceph NOFD /proc/71202/task/71209/fd (opendir: Permission denied) 2024-06-03T03:27:58.756 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 71213 service ceph cwd unknown /proc/71202/task/71213/cwd (readlink: Permission denied) 2024-06-03T03:27:58.756 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 71213 service ceph rtd unknown /proc/71202/task/71213/root (readlink: Permission denied) 2024-06-03T03:27:58.756 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 71213 service ceph txt unknown /proc/71202/task/71213/exe (readlink: Permission denied) 2024-06-03T03:27:58.756 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 71213 service ceph NOFD /proc/71202/task/71213/fd (opendir: Permission denied) 2024-06-03T03:27:58.756 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 71214 admin_soc ceph cwd unknown /proc/71202/task/71214/cwd (readlink: Permission denied) 2024-06-03T03:27:58.756 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 71214 admin_soc ceph rtd unknown /proc/71202/task/71214/root (readlink: Permission denied) 2024-06-03T03:27:58.756 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 71214 admin_soc ceph txt unknown /proc/71202/task/71214/exe (readlink: Permission denied) 2024-06-03T03:27:58.756 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 71214 admin_soc ceph NOFD /proc/71202/task/71214/fd (opendir: Permission denied) 2024-06-03T03:27:58.757 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 71215 signal_ha ceph cwd unknown /proc/71202/task/71215/cwd (readlink: Permission denied) 2024-06-03T03:27:58.757 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 71215 signal_ha ceph rtd unknown /proc/71202/task/71215/root (readlink: Permission denied) 2024-06-03T03:27:58.757 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 71215 signal_ha ceph txt unknown /proc/71202/task/71215/exe (readlink: Permission denied) 2024-06-03T03:27:58.757 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 71215 signal_ha ceph NOFD /proc/71202/task/71215/fd (opendir: Permission denied) 2024-06-03T03:27:58.757 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 71216 io_contex ceph cwd unknown /proc/71202/task/71216/cwd (readlink: Permission denied) 2024-06-03T03:27:58.757 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 71216 io_contex ceph rtd unknown /proc/71202/task/71216/root (readlink: Permission denied) 2024-06-03T03:27:58.757 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 71216 io_contex ceph txt unknown /proc/71202/task/71216/exe (readlink: Permission denied) 2024-06-03T03:27:58.757 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 71216 io_contex ceph NOFD /proc/71202/task/71216/fd (opendir: Permission denied) 2024-06-03T03:27:58.757 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 71217 io_contex ceph cwd unknown /proc/71202/task/71217/cwd (readlink: Permission denied) 2024-06-03T03:27:58.757 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 71217 io_contex ceph rtd unknown /proc/71202/task/71217/root (readlink: Permission denied) 2024-06-03T03:27:58.757 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 71217 io_contex ceph txt unknown /proc/71202/task/71217/exe (readlink: Permission denied) 2024-06-03T03:27:58.757 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 71217 io_contex ceph NOFD /proc/71202/task/71217/fd (opendir: Permission denied) 2024-06-03T03:27:58.757 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 71218 ms_dispat ceph cwd unknown /proc/71202/task/71218/cwd (readlink: Permission denied) 2024-06-03T03:27:58.757 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 71218 ms_dispat ceph rtd unknown /proc/71202/task/71218/root (readlink: Permission denied) 2024-06-03T03:27:58.758 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 71218 ms_dispat ceph txt unknown /proc/71202/task/71218/exe (readlink: Permission denied) 2024-06-03T03:27:58.758 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 71218 ms_dispat ceph NOFD /proc/71202/task/71218/fd (opendir: Permission denied) 2024-06-03T03:27:58.758 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 71219 ms_local ceph cwd unknown /proc/71202/task/71219/cwd (readlink: Permission denied) 2024-06-03T03:27:58.758 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 71219 ms_local ceph rtd unknown /proc/71202/task/71219/root (readlink: Permission denied) 2024-06-03T03:27:58.758 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 71219 ms_local ceph txt unknown /proc/71202/task/71219/exe (readlink: Permission denied) 2024-06-03T03:27:58.758 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 71219 ms_local ceph NOFD /proc/71202/task/71219/fd (opendir: Permission denied) 2024-06-03T03:27:58.758 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 71220 safe_time ceph cwd unknown /proc/71202/task/71220/cwd (readlink: Permission denied) 2024-06-03T03:27:58.758 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 71220 safe_time ceph rtd unknown /proc/71202/task/71220/root (readlink: Permission denied) 2024-06-03T03:27:58.758 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 71220 safe_time ceph txt unknown /proc/71202/task/71220/exe (readlink: Permission denied) 2024-06-03T03:27:58.758 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 71220 safe_time ceph NOFD /proc/71202/task/71220/fd (opendir: Permission denied) 2024-06-03T03:27:58.758 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 71221 safe_time ceph cwd unknown /proc/71202/task/71221/cwd (readlink: Permission denied) 2024-06-03T03:27:58.758 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 71221 safe_time ceph rtd unknown /proc/71202/task/71221/root (readlink: Permission denied) 2024-06-03T03:27:58.758 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 71221 safe_time ceph txt unknown /proc/71202/task/71221/exe (readlink: Permission denied) 2024-06-03T03:27:58.759 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 71221 safe_time ceph NOFD /proc/71202/task/71221/fd (opendir: Permission denied) 2024-06-03T03:27:58.759 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 71222 ceph-mds ceph cwd unknown /proc/71202/task/71222/cwd (readlink: Permission denied) 2024-06-03T03:27:58.759 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 71222 ceph-mds ceph rtd unknown /proc/71202/task/71222/root (readlink: Permission denied) 2024-06-03T03:27:58.759 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 71222 ceph-mds ceph txt unknown /proc/71202/task/71222/exe (readlink: Permission denied) 2024-06-03T03:27:58.759 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 71222 ceph-mds ceph NOFD /proc/71202/task/71222/fd (opendir: Permission denied) 2024-06-03T03:27:58.759 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 71258 safe_time ceph cwd unknown /proc/71202/task/71258/cwd (readlink: Permission denied) 2024-06-03T03:27:58.759 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 71258 safe_time ceph rtd unknown /proc/71202/task/71258/root (readlink: Permission denied) 2024-06-03T03:27:58.759 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 71258 safe_time ceph txt unknown /proc/71202/task/71258/exe (readlink: Permission denied) 2024-06-03T03:27:58.759 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 71258 safe_time ceph NOFD /proc/71202/task/71258/fd (opendir: Permission denied) 2024-06-03T03:27:58.759 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 76557 ceph_time ceph cwd unknown /proc/71202/task/76557/cwd (readlink: Permission denied) 2024-06-03T03:27:58.759 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 76557 ceph_time ceph rtd unknown /proc/71202/task/76557/root (readlink: Permission denied) 2024-06-03T03:27:58.759 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 76557 ceph_time ceph txt unknown /proc/71202/task/76557/exe (readlink: Permission denied) 2024-06-03T03:27:58.759 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 76557 ceph_time ceph NOFD /proc/71202/task/76557/fd (opendir: Permission denied) 2024-06-03T03:27:58.759 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 76558 OpHistory ceph cwd unknown /proc/71202/task/76558/cwd (readlink: Permission denied) 2024-06-03T03:27:58.760 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 76558 OpHistory ceph rtd unknown /proc/71202/task/76558/root (readlink: Permission denied) 2024-06-03T03:27:58.760 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 76558 OpHistory ceph txt unknown /proc/71202/task/76558/exe (readlink: Permission denied) 2024-06-03T03:27:58.760 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 76558 OpHistory ceph NOFD /proc/71202/task/76558/fd (opendir: Permission denied) 2024-06-03T03:27:58.760 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 76559 ms_dispat ceph cwd unknown /proc/71202/task/76559/cwd (readlink: Permission denied) 2024-06-03T03:27:58.760 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 76559 ms_dispat ceph rtd unknown /proc/71202/task/76559/root (readlink: Permission denied) 2024-06-03T03:27:58.760 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 76559 ms_dispat ceph txt unknown /proc/71202/task/76559/exe (readlink: Permission denied) 2024-06-03T03:27:58.760 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 76559 ms_dispat ceph NOFD /proc/71202/task/76559/fd (opendir: Permission denied) 2024-06-03T03:27:58.760 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 76560 ms_dispat ceph cwd unknown /proc/71202/task/76560/cwd (readlink: Permission denied) 2024-06-03T03:27:58.760 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 76560 ms_dispat ceph rtd unknown /proc/71202/task/76560/root (readlink: Permission denied) 2024-06-03T03:27:58.760 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 76560 ms_dispat ceph txt unknown /proc/71202/task/76560/exe (readlink: Permission denied) 2024-06-03T03:27:58.760 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 76560 ms_dispat ceph NOFD /proc/71202/task/76560/fd (opendir: Permission denied) 2024-06-03T03:27:58.760 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 76561 mds_rank_ ceph cwd unknown /proc/71202/task/76561/cwd (readlink: Permission denied) 2024-06-03T03:27:58.760 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 76561 mds_rank_ ceph rtd unknown /proc/71202/task/76561/root (readlink: Permission denied) 2024-06-03T03:27:58.760 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 76561 mds_rank_ ceph txt unknown /proc/71202/task/76561/exe (readlink: Permission denied) 2024-06-03T03:27:58.761 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 76561 mds_rank_ ceph NOFD /proc/71202/task/76561/fd (opendir: Permission denied) 2024-06-03T03:27:58.761 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 76562 PQ_Finish ceph cwd unknown /proc/71202/task/76562/cwd (readlink: Permission denied) 2024-06-03T03:27:58.761 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 76562 PQ_Finish ceph rtd unknown /proc/71202/task/76562/root (readlink: Permission denied) 2024-06-03T03:27:58.761 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 76562 PQ_Finish ceph txt unknown /proc/71202/task/76562/exe (readlink: Permission denied) 2024-06-03T03:27:58.761 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 76562 PQ_Finish ceph NOFD /proc/71202/task/76562/fd (opendir: Permission denied) 2024-06-03T03:27:58.761 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 76563 safe_time ceph cwd unknown /proc/71202/task/76563/cwd (readlink: Permission denied) 2024-06-03T03:27:58.761 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 76563 safe_time ceph rtd unknown /proc/71202/task/76563/root (readlink: Permission denied) 2024-06-03T03:27:58.761 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 76563 safe_time ceph txt unknown /proc/71202/task/76563/exe (readlink: Permission denied) 2024-06-03T03:27:58.762 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 76563 safe_time ceph NOFD /proc/71202/task/76563/fd (opendir: Permission denied) 2024-06-03T03:27:58.762 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 76564 MR_Finish ceph cwd unknown /proc/71202/task/76564/cwd (readlink: Permission denied) 2024-06-03T03:27:58.762 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 76564 MR_Finish ceph rtd unknown /proc/71202/task/76564/root (readlink: Permission denied) 2024-06-03T03:27:58.762 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 76564 MR_Finish ceph txt unknown /proc/71202/task/76564/exe (readlink: Permission denied) 2024-06-03T03:27:58.762 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 76564 MR_Finish ceph NOFD /proc/71202/task/76564/fd (opendir: Permission denied) 2024-06-03T03:27:58.762 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 76565 md_submit ceph cwd unknown /proc/71202/task/76565/cwd (readlink: Permission denied) 2024-06-03T03:27:58.762 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 76565 md_submit ceph rtd unknown /proc/71202/task/76565/root (readlink: Permission denied) 2024-06-03T03:27:58.762 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 76565 md_submit ceph txt unknown /proc/71202/task/76565/exe (readlink: Permission denied) 2024-06-03T03:27:58.762 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 76565 md_submit ceph NOFD /proc/71202/task/76565/fd (opendir: Permission denied) 2024-06-03T03:27:58.762 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 76652 ms_dispat ceph cwd unknown /proc/71202/task/76652/cwd (readlink: Permission denied) 2024-06-03T03:27:58.762 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 76652 ms_dispat ceph rtd unknown /proc/71202/task/76652/root (readlink: Permission denied) 2024-06-03T03:27:58.762 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 76652 ms_dispat ceph txt unknown /proc/71202/task/76652/exe (readlink: Permission denied) 2024-06-03T03:27:58.762 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 76652 ms_dispat ceph NOFD /proc/71202/task/76652/fd (opendir: Permission denied) 2024-06-03T03:27:58.763 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 76653 quiesce.a ceph cwd unknown /proc/71202/task/76653/cwd (readlink: Permission denied) 2024-06-03T03:27:58.763 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 76653 quiesce.a ceph rtd unknown /proc/71202/task/76653/root (readlink: Permission denied) 2024-06-03T03:27:58.763 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 76653 quiesce.a ceph txt unknown /proc/71202/task/76653/exe (readlink: Permission denied) 2024-06-03T03:27:58.763 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 76653 quiesce.a ceph NOFD /proc/71202/task/76653/fd (opendir: Permission denied) 2024-06-03T03:27:58.763 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 76654 quiesce_d ceph cwd unknown /proc/71202/task/76654/cwd (readlink: Permission denied) 2024-06-03T03:27:58.763 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 76654 quiesce_d ceph rtd unknown /proc/71202/task/76654/root (readlink: Permission denied) 2024-06-03T03:27:58.763 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 76654 quiesce_d ceph txt unknown /proc/71202/task/76654/exe (readlink: Permission denied) 2024-06-03T03:27:58.763 INFO:tasks.workunit.client.0.smithi117.stdout:ceph-mds 71202 76654 quiesce_d ceph NOFD /proc/71202/task/76654/fd (opendir: Permission denied) 2024-06-03T03:27:58.763 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/6 72557 root cwd unknown /proc/72557/cwd (readlink: Permission denied) 2024-06-03T03:27:58.763 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/6 72557 root rtd unknown /proc/72557/root (readlink: Permission denied) 2024-06-03T03:27:58.763 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/6 72557 root txt unknown /proc/72557/exe (readlink: Permission denied) 2024-06-03T03:27:58.763 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/6 72557 root NOFD /proc/72557/fd (opendir: Permission denied) 2024-06-03T03:27:58.763 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/3 72673 root cwd unknown /proc/72673/cwd (readlink: Permission denied) 2024-06-03T03:27:58.763 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/3 72673 root rtd unknown /proc/72673/root (readlink: Permission denied) 2024-06-03T03:27:58.764 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/3 72673 root txt unknown /proc/72673/exe (readlink: Permission denied) 2024-06-03T03:27:58.764 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/3 72673 root NOFD /proc/72673/fd (opendir: Permission denied) 2024-06-03T03:27:58.764 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/u 72761 root cwd unknown /proc/72761/cwd (readlink: Permission denied) 2024-06-03T03:27:58.764 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/u 72761 root rtd unknown /proc/72761/root (readlink: Permission denied) 2024-06-03T03:27:58.764 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/u 72761 root txt unknown /proc/72761/exe (readlink: Permission denied) 2024-06-03T03:27:58.764 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/u 72761 root NOFD /proc/72761/fd (opendir: Permission denied) 2024-06-03T03:27:58.764 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/1 73743 root cwd unknown /proc/73743/cwd (readlink: Permission denied) 2024-06-03T03:27:58.764 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/1 73743 root rtd unknown /proc/73743/root (readlink: Permission denied) 2024-06-03T03:27:58.764 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/1 73743 root txt unknown /proc/73743/exe (readlink: Permission denied) 2024-06-03T03:27:58.764 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/1 73743 root NOFD /proc/73743/fd (opendir: Permission denied) 2024-06-03T03:27:58.764 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/u 74623 root cwd unknown /proc/74623/cwd (readlink: Permission denied) 2024-06-03T03:27:58.764 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/u 74623 root rtd unknown /proc/74623/root (readlink: Permission denied) 2024-06-03T03:27:58.764 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/u 74623 root txt unknown /proc/74623/exe (readlink: Permission denied) 2024-06-03T03:27:58.764 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/u 74623 root NOFD /proc/74623/fd (opendir: Permission denied) 2024-06-03T03:27:58.765 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/7 74754 root cwd unknown /proc/74754/cwd (readlink: Permission denied) 2024-06-03T03:27:58.765 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/7 74754 root rtd unknown /proc/74754/root (readlink: Permission denied) 2024-06-03T03:27:58.765 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/7 74754 root txt unknown /proc/74754/exe (readlink: Permission denied) 2024-06-03T03:27:58.765 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/7 74754 root NOFD /proc/74754/fd (opendir: Permission denied) 2024-06-03T03:27:58.765 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/4 78256 root cwd unknown /proc/78256/cwd (readlink: Permission denied) 2024-06-03T03:27:58.765 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/4 78256 root rtd unknown /proc/78256/root (readlink: Permission denied) 2024-06-03T03:27:58.765 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/4 78256 root txt unknown /proc/78256/exe (readlink: Permission denied) 2024-06-03T03:27:58.765 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/4 78256 root NOFD /proc/78256/fd (opendir: Permission denied) 2024-06-03T03:27:58.765 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/5 83016 root cwd unknown /proc/83016/cwd (readlink: Permission denied) 2024-06-03T03:27:58.765 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/5 83016 root rtd unknown /proc/83016/root (readlink: Permission denied) 2024-06-03T03:27:58.765 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/5 83016 root txt unknown /proc/83016/exe (readlink: Permission denied) 2024-06-03T03:27:58.765 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/5 83016 root NOFD /proc/83016/fd (opendir: Permission denied) 2024-06-03T03:27:58.765 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/2 83293 root cwd unknown /proc/83293/cwd (readlink: Permission denied) 2024-06-03T03:27:58.766 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/2 83293 root rtd unknown /proc/83293/root (readlink: Permission denied) 2024-06-03T03:27:58.766 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/2 83293 root txt unknown /proc/83293/exe (readlink: Permission denied) 2024-06-03T03:27:58.766 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/2 83293 root NOFD /proc/83293/fd (opendir: Permission denied) 2024-06-03T03:27:58.766 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/0 85182 root cwd unknown /proc/85182/cwd (readlink: Permission denied) 2024-06-03T03:27:58.766 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/0 85182 root rtd unknown /proc/85182/root (readlink: Permission denied) 2024-06-03T03:27:58.766 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/0 85182 root txt unknown /proc/85182/exe (readlink: Permission denied) 2024-06-03T03:27:58.767 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/0 85182 root NOFD /proc/85182/fd (opendir: Permission denied) 2024-06-03T03:27:58.767 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 88571 root cwd unknown /proc/88571/cwd (readlink: Permission denied) 2024-06-03T03:27:58.767 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 88571 root rtd unknown /proc/88571/root (readlink: Permission denied) 2024-06-03T03:27:58.767 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 88571 root txt unknown /proc/88571/exe (readlink: Permission denied) 2024-06-03T03:27:58.767 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 88571 root NOFD /proc/88571/fd (opendir: Permission denied) 2024-06-03T03:27:58.767 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 88586 root cwd unknown /proc/88586/cwd (readlink: Permission denied) 2024-06-03T03:27:58.767 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 88586 root rtd unknown /proc/88586/root (readlink: Permission denied) 2024-06-03T03:27:58.767 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 88586 root txt unknown /proc/88586/exe (readlink: Permission denied) 2024-06-03T03:27:58.767 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 88586 root NOFD /proc/88586/fd (opendir: Permission denied) 2024-06-03T03:27:58.767 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 88587 root cwd unknown /proc/88587/cwd (readlink: Permission denied) 2024-06-03T03:27:58.767 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 88587 root rtd unknown /proc/88587/root (readlink: Permission denied) 2024-06-03T03:27:58.768 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 88587 root txt unknown /proc/88587/exe (readlink: Permission denied) 2024-06-03T03:27:58.768 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/R 88587 root NOFD /proc/88587/fd (opendir: Permission denied) 2024-06-03T03:27:58.768 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/0 89016 root cwd unknown /proc/89016/cwd (readlink: Permission denied) 2024-06-03T03:27:58.768 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/0 89016 root rtd unknown /proc/89016/root (readlink: Permission denied) 2024-06-03T03:27:58.768 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/0 89016 root txt unknown /proc/89016/exe (readlink: Permission denied) 2024-06-03T03:27:58.768 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/0 89016 root NOFD /proc/89016/fd (opendir: Permission denied) 2024-06-03T03:27:58.768 INFO:tasks.workunit.client.0.smithi117.stdout:sudo 89590 root cwd unknown /proc/89590/cwd (readlink: Permission denied) 2024-06-03T03:27:58.768 INFO:tasks.workunit.client.0.smithi117.stdout:sudo 89590 root rtd unknown /proc/89590/root (readlink: Permission denied) 2024-06-03T03:27:58.768 INFO:tasks.workunit.client.0.smithi117.stdout:sudo 89590 root txt unknown /proc/89590/exe (readlink: Permission denied) 2024-06-03T03:27:58.768 INFO:tasks.workunit.client.0.smithi117.stdout:sudo 89590 root NOFD /proc/89590/fd (opendir: Permission denied) 2024-06-03T03:27:58.768 INFO:tasks.workunit.client.0.smithi117.stdout:daemon-he 89640 root cwd unknown /proc/89640/cwd (readlink: Permission denied) 2024-06-03T03:27:58.768 INFO:tasks.workunit.client.0.smithi117.stdout:daemon-he 89640 root rtd unknown /proc/89640/root (readlink: Permission denied) 2024-06-03T03:27:58.769 INFO:tasks.workunit.client.0.smithi117.stdout:daemon-he 89640 root txt unknown /proc/89640/exe (readlink: Permission denied) 2024-06-03T03:27:58.769 INFO:tasks.workunit.client.0.smithi117.stdout:daemon-he 89640 root NOFD /proc/89640/fd (opendir: Permission denied) 2024-06-03T03:27:58.769 INFO:tasks.workunit.client.0.smithi117.stdout:bash 89642 root cwd unknown /proc/89642/cwd (readlink: Permission denied) 2024-06-03T03:27:58.769 INFO:tasks.workunit.client.0.smithi117.stdout:bash 89642 root rtd unknown /proc/89642/root (readlink: Permission denied) 2024-06-03T03:27:58.769 INFO:tasks.workunit.client.0.smithi117.stdout:bash 89642 root txt unknown /proc/89642/exe (readlink: Permission denied) 2024-06-03T03:27:58.769 INFO:tasks.workunit.client.0.smithi117.stdout:bash 89642 root NOFD /proc/89642/fd (opendir: Permission denied) 2024-06-03T03:27:58.769 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/6 89834 root cwd unknown /proc/89834/cwd (readlink: Permission denied) 2024-06-03T03:27:58.769 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/6 89834 root rtd unknown /proc/89834/root (readlink: Permission denied) 2024-06-03T03:27:58.769 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/6 89834 root txt unknown /proc/89834/exe (readlink: Permission denied) 2024-06-03T03:27:58.769 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/6 89834 root NOFD /proc/89834/fd (opendir: Permission denied) 2024-06-03T03:27:58.769 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/7 90776 root cwd unknown /proc/90776/cwd (readlink: Permission denied) 2024-06-03T03:27:58.769 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/7 90776 root rtd unknown /proc/90776/root (readlink: Permission denied) 2024-06-03T03:27:58.770 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/7 90776 root txt unknown /proc/90776/exe (readlink: Permission denied) 2024-06-03T03:27:58.770 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/7 90776 root NOFD /proc/90776/fd (opendir: Permission denied) 2024-06-03T03:27:58.770 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/7 90833 root cwd unknown /proc/90833/cwd (readlink: Permission denied) 2024-06-03T03:27:58.770 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/7 90833 root rtd unknown /proc/90833/root (readlink: Permission denied) 2024-06-03T03:27:58.770 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/7 90833 root txt unknown /proc/90833/exe (readlink: Permission denied) 2024-06-03T03:27:58.770 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/7 90833 root NOFD /proc/90833/fd (opendir: Permission denied) 2024-06-03T03:27:58.770 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/3 91780 root cwd unknown /proc/91780/cwd (readlink: Permission denied) 2024-06-03T03:27:58.770 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/3 91780 root rtd unknown /proc/91780/root (readlink: Permission denied) 2024-06-03T03:27:58.770 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/3 91780 root txt unknown /proc/91780/exe (readlink: Permission denied) 2024-06-03T03:27:58.770 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/3 91780 root NOFD /proc/91780/fd (opendir: Permission denied) 2024-06-03T03:27:58.770 INFO:tasks.workunit.client.0.smithi117.stdout:timeout 92497 ubuntu cwd DIR 0,119 4 1099511627782 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-06-03T03:27:58.771 INFO:tasks.workunit.client.0.smithi117.stdout:timeout 92497 ubuntu rtd DIR 8,1 4096 2 / 2024-06-03T03:27:58.771 INFO:tasks.workunit.client.0.smithi117.stdout:timeout 92497 ubuntu txt REG 8,1 41144 7580 /usr/bin/timeout 2024-06-03T03:27:58.771 INFO:tasks.workunit.client.0.smithi117.stdout:timeout 92497 ubuntu mem REG 8,1 2586930 16078 /usr/lib/locale/en_US.utf8/LC_COLLATE 2024-06-03T03:27:58.771 INFO:tasks.workunit.client.0.smithi117.stdout:timeout 92497 ubuntu mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-06-03T03:27:58.771 INFO:tasks.workunit.client.0.smithi117.stdout:timeout 92497 ubuntu mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-06-03T03:27:58.771 INFO:tasks.workunit.client.0.smithi117.stdout:timeout 92497 ubuntu mem REG 8,1 54 16081 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2024-06-03T03:27:58.771 INFO:tasks.workunit.client.0.smithi117.stdout:timeout 92497 ubuntu mem REG 8,1 3284 28786 /usr/lib/locale/en_US.utf8/LC_TIME 2024-06-03T03:27:58.771 INFO:tasks.workunit.client.0.smithi117.stdout:timeout 92497 ubuntu mem REG 8,1 286 28783 /usr/lib/locale/en_US.utf8/LC_MONETARY 2024-06-03T03:27:58.771 INFO:tasks.workunit.client.0.smithi117.stdout:timeout 92497 ubuntu mem REG 8,1 57 26536 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2024-06-03T03:27:58.771 INFO:tasks.workunit.client.0.smithi117.stdout:timeout 92497 ubuntu mem REG 8,1 34 28836 /usr/lib/locale/en_US.utf8/LC_PAPER 2024-06-03T03:27:58.771 INFO:tasks.workunit.client.0.smithi117.stdout:timeout 92497 ubuntu mem REG 8,1 77 16080 /usr/lib/locale/en_US.utf8/LC_NAME 2024-06-03T03:27:58.771 INFO:tasks.workunit.client.0.smithi117.stdout:timeout 92497 ubuntu mem REG 8,1 167 28778 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2024-06-03T03:27:58.772 INFO:tasks.workunit.client.0.smithi117.stdout:timeout 92497 ubuntu mem REG 8,1 59 28785 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2024-06-03T03:27:58.772 INFO:tasks.workunit.client.0.smithi117.stdout:timeout 92497 ubuntu mem REG 8,1 23 28782 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2024-06-03T03:27:58.772 INFO:tasks.workunit.client.0.smithi117.stdout:timeout 92497 ubuntu mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-06-03T03:27:58.772 INFO:tasks.workunit.client.0.smithi117.stdout:timeout 92497 ubuntu mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-06-03T03:27:58.772 INFO:tasks.workunit.client.0.smithi117.stdout:timeout 92497 ubuntu mem REG 8,1 369 28781 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2024-06-03T03:27:58.772 INFO:tasks.workunit.client.0.smithi117.stdout:timeout 92497 ubuntu 0r FIFO 0,14 0t0 254675 pipe 2024-06-03T03:27:58.772 INFO:tasks.workunit.client.0.smithi117.stdout:timeout 92497 ubuntu 1w FIFO 0,14 0t0 254676 pipe 2024-06-03T03:27:58.772 INFO:tasks.workunit.client.0.smithi117.stdout:timeout 92497 ubuntu 2w FIFO 0,14 0t0 254677 pipe 2024-06-03T03:27:58.772 INFO:tasks.workunit.client.0.smithi117.stdout:fsync-tes 92547 ubuntu cwd DIR 0,119 4 1099511627782 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-06-03T03:27:58.772 INFO:tasks.workunit.client.0.smithi117.stdout:fsync-tes 92547 ubuntu rtd DIR 8,1 4096 2 / 2024-06-03T03:27:58.772 INFO:tasks.workunit.client.0.smithi117.stdout:fsync-tes 92547 ubuntu txt REG 8,1 1389072 7149 /usr/bin/bash 2024-06-03T03:27:58.772 INFO:tasks.workunit.client.0.smithi117.stdout:fsync-tes 92547 ubuntu mem REG 8,1 2586930 16078 /usr/lib/locale/en_US.utf8/LC_COLLATE 2024-06-03T03:27:58.772 INFO:tasks.workunit.client.0.smithi117.stdout:fsync-tes 92547 ubuntu mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-06-03T03:27:58.772 INFO:tasks.workunit.client.0.smithi117.stdout:fsync-tes 92547 ubuntu mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-06-03T03:27:58.773 INFO:tasks.workunit.client.0.smithi117.stdout:fsync-tes 92547 ubuntu mem REG 8,1 54 16081 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2024-06-03T03:27:58.773 INFO:tasks.workunit.client.0.smithi117.stdout:fsync-tes 92547 ubuntu mem REG 8,1 3284 28786 /usr/lib/locale/en_US.utf8/LC_TIME 2024-06-03T03:27:58.773 INFO:tasks.workunit.client.0.smithi117.stdout:fsync-tes 92547 ubuntu mem REG 8,1 286 28783 /usr/lib/locale/en_US.utf8/LC_MONETARY 2024-06-03T03:27:58.773 INFO:tasks.workunit.client.0.smithi117.stdout:fsync-tes 92547 ubuntu mem REG 8,1 57 26536 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2024-06-03T03:27:58.773 INFO:tasks.workunit.client.0.smithi117.stdout:fsync-tes 92547 ubuntu mem REG 8,1 34 28836 /usr/lib/locale/en_US.utf8/LC_PAPER 2024-06-03T03:27:58.773 INFO:tasks.workunit.client.0.smithi117.stdout:fsync-tes 92547 ubuntu mem REG 8,1 77 16080 /usr/lib/locale/en_US.utf8/LC_NAME 2024-06-03T03:27:58.773 INFO:tasks.workunit.client.0.smithi117.stdout:fsync-tes 92547 ubuntu mem REG 8,1 167 28778 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2024-06-03T03:27:58.773 INFO:tasks.workunit.client.0.smithi117.stdout:fsync-tes 92547 ubuntu mem REG 8,1 59 28785 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2024-06-03T03:27:58.773 INFO:tasks.workunit.client.0.smithi117.stdout:fsync-tes 92547 ubuntu mem REG 8,1 195080 7147 /usr/lib64/libtinfo.so.6.2 2024-06-03T03:27:58.773 INFO:tasks.workunit.client.0.smithi117.stdout:fsync-tes 92547 ubuntu mem REG 8,1 23 28782 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2024-06-03T03:27:58.773 INFO:tasks.workunit.client.0.smithi117.stdout:fsync-tes 92547 ubuntu mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-06-03T03:27:58.773 INFO:tasks.workunit.client.0.smithi117.stdout:fsync-tes 92547 ubuntu mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-06-03T03:27:58.773 INFO:tasks.workunit.client.0.smithi117.stdout:fsync-tes 92547 ubuntu mem REG 8,1 369 28781 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2024-06-03T03:27:58.774 INFO:tasks.workunit.client.0.smithi117.stdout:fsync-tes 92547 ubuntu 0r FIFO 0,14 0t0 254675 pipe 2024-06-03T03:27:58.774 INFO:tasks.workunit.client.0.smithi117.stdout:fsync-tes 92547 ubuntu 1w FIFO 0,14 0t0 254676 pipe 2024-06-03T03:27:58.774 INFO:tasks.workunit.client.0.smithi117.stdout:fsync-tes 92547 ubuntu 2w FIFO 0,14 0t0 254677 pipe 2024-06-03T03:27:58.774 INFO:tasks.workunit.client.0.smithi117.stdout:fsync-tes 92547 ubuntu 255r REG 8,1 207 662005 /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsync-tester.sh 2024-06-03T03:27:58.774 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/3 93776 root cwd unknown /proc/93776/cwd (readlink: Permission denied) 2024-06-03T03:27:58.774 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/3 93776 root rtd unknown /proc/93776/root (readlink: Permission denied) 2024-06-03T03:27:58.774 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/3 93776 root txt unknown /proc/93776/exe (readlink: Permission denied) 2024-06-03T03:27:58.774 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/3 93776 root NOFD /proc/93776/fd (opendir: Permission denied) 2024-06-03T03:27:58.774 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/1 93822 root cwd unknown /proc/93822/cwd (readlink: Permission denied) 2024-06-03T03:27:58.774 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/1 93822 root rtd unknown /proc/93822/root (readlink: Permission denied) 2024-06-03T03:27:58.774 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/1 93822 root txt unknown /proc/93822/exe (readlink: Permission denied) 2024-06-03T03:27:58.774 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/1 93822 root NOFD /proc/93822/fd (opendir: Permission denied) 2024-06-03T03:27:58.774 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/6 93825 root cwd unknown /proc/93825/cwd (readlink: Permission denied) 2024-06-03T03:27:58.775 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/6 93825 root rtd unknown /proc/93825/root (readlink: Permission denied) 2024-06-03T03:27:58.775 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/6 93825 root txt unknown /proc/93825/exe (readlink: Permission denied) 2024-06-03T03:27:58.775 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/6 93825 root NOFD /proc/93825/fd (opendir: Permission denied) 2024-06-03T03:27:58.775 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/u 93838 root cwd unknown /proc/93838/cwd (readlink: Permission denied) 2024-06-03T03:27:58.775 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/u 93838 root rtd unknown /proc/93838/root (readlink: Permission denied) 2024-06-03T03:27:58.775 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/u 93838 root txt unknown /proc/93838/exe (readlink: Permission denied) 2024-06-03T03:27:58.775 INFO:tasks.workunit.client.0.smithi117.stdout:kworker/u 93838 root NOFD /proc/93838/fd (opendir: Permission denied) 2024-06-03T03:27:58.775 INFO:tasks.workunit.client.0.smithi117.stdout:sleep 94024 root cwd unknown /proc/94024/cwd (readlink: Permission denied) 2024-06-03T03:27:58.775 INFO:tasks.workunit.client.0.smithi117.stdout:sleep 94024 root rtd unknown /proc/94024/root (readlink: Permission denied) 2024-06-03T03:27:58.775 INFO:tasks.workunit.client.0.smithi117.stdout:sleep 94024 root txt unknown /proc/94024/exe (readlink: Permission denied) 2024-06-03T03:27:58.775 INFO:tasks.workunit.client.0.smithi117.stdout:sleep 94024 root NOFD /proc/94024/fd (opendir: Permission denied) 2024-06-03T03:27:58.775 INFO:tasks.workunit.client.0.smithi117.stdout:sudo 94239 root cwd unknown /proc/94239/cwd (readlink: Permission denied) 2024-06-03T03:27:58.775 INFO:tasks.workunit.client.0.smithi117.stdout:sudo 94239 root rtd unknown /proc/94239/root (readlink: Permission denied) 2024-06-03T03:27:58.775 INFO:tasks.workunit.client.0.smithi117.stdout:sudo 94239 root txt unknown /proc/94239/exe (readlink: Permission denied) 2024-06-03T03:27:58.775 INFO:tasks.workunit.client.0.smithi117.stdout:sudo 94239 root NOFD /proc/94239/fd (opendir: Permission denied) 2024-06-03T03:27:58.776 INFO:tasks.workunit.client.0.smithi117.stdout:cephadm 94256 root cwd unknown /proc/94256/cwd (readlink: Permission denied) 2024-06-03T03:27:58.776 INFO:tasks.workunit.client.0.smithi117.stdout:cephadm 94256 root rtd unknown /proc/94256/root (readlink: Permission denied) 2024-06-03T03:27:58.776 INFO:tasks.workunit.client.0.smithi117.stdout:cephadm 94256 root txt unknown /proc/94256/exe (readlink: Permission denied) 2024-06-03T03:27:58.776 INFO:tasks.workunit.client.0.smithi117.stdout:cephadm 94256 root NOFD /proc/94256/fd (opendir: Permission denied) 2024-06-03T03:27:58.776 INFO:tasks.workunit.client.0.smithi117.stdout:cephadm 94256 94301 cephadm root cwd unknown /proc/94256/task/94301/cwd (readlink: Permission denied) 2024-06-03T03:27:58.776 INFO:tasks.workunit.client.0.smithi117.stdout:cephadm 94256 94301 cephadm root rtd unknown /proc/94256/task/94301/root (readlink: Permission denied) 2024-06-03T03:27:58.776 INFO:tasks.workunit.client.0.smithi117.stdout:cephadm 94256 94301 cephadm root txt unknown /proc/94256/task/94301/exe (readlink: Permission denied) 2024-06-03T03:27:58.776 INFO:tasks.workunit.client.0.smithi117.stdout:cephadm 94256 94301 cephadm root NOFD /proc/94256/task/94301/fd (opendir: Permission denied) 2024-06-03T03:27:58.776 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94300 root cwd unknown /proc/94300/cwd (readlink: Permission denied) 2024-06-03T03:27:58.776 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94300 root rtd unknown /proc/94300/root (readlink: Permission denied) 2024-06-03T03:27:58.776 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94300 root txt unknown /proc/94300/exe (readlink: Permission denied) 2024-06-03T03:27:58.776 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94300 root NOFD /proc/94300/fd (opendir: Permission denied) 2024-06-03T03:27:58.776 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94300 94302 podman root cwd unknown /proc/94300/task/94302/cwd (readlink: Permission denied) 2024-06-03T03:27:58.777 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94300 94302 podman root rtd unknown /proc/94300/task/94302/root (readlink: Permission denied) 2024-06-03T03:27:58.777 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94300 94302 podman root txt unknown /proc/94300/task/94302/exe (readlink: Permission denied) 2024-06-03T03:27:58.777 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94300 94302 podman root NOFD /proc/94300/task/94302/fd (opendir: Permission denied) 2024-06-03T03:27:58.777 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94300 94303 podman root cwd unknown /proc/94300/task/94303/cwd (readlink: Permission denied) 2024-06-03T03:27:58.777 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94300 94303 podman root rtd unknown /proc/94300/task/94303/root (readlink: Permission denied) 2024-06-03T03:27:58.777 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94300 94303 podman root txt unknown /proc/94300/task/94303/exe (readlink: Permission denied) 2024-06-03T03:27:58.777 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94300 94303 podman root NOFD /proc/94300/task/94303/fd (opendir: Permission denied) 2024-06-03T03:27:58.777 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94300 94304 podman root cwd unknown /proc/94300/task/94304/cwd (readlink: Permission denied) 2024-06-03T03:27:58.777 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94300 94304 podman root rtd unknown /proc/94300/task/94304/root (readlink: Permission denied) 2024-06-03T03:27:58.777 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94300 94304 podman root txt unknown /proc/94300/task/94304/exe (readlink: Permission denied) 2024-06-03T03:27:58.777 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94300 94304 podman root NOFD /proc/94300/task/94304/fd (opendir: Permission denied) 2024-06-03T03:27:58.777 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94300 94305 podman root cwd unknown /proc/94300/task/94305/cwd (readlink: Permission denied) 2024-06-03T03:27:58.778 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94300 94305 podman root rtd unknown /proc/94300/task/94305/root (readlink: Permission denied) 2024-06-03T03:27:58.778 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94300 94305 podman root txt unknown /proc/94300/task/94305/exe (readlink: Permission denied) 2024-06-03T03:27:58.778 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94300 94305 podman root NOFD /proc/94300/task/94305/fd (opendir: Permission denied) 2024-06-03T03:27:58.778 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94300 94306 podman root cwd unknown /proc/94300/task/94306/cwd (readlink: Permission denied) 2024-06-03T03:27:58.778 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94300 94306 podman root rtd unknown /proc/94300/task/94306/root (readlink: Permission denied) 2024-06-03T03:27:58.778 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94300 94306 podman root txt unknown /proc/94300/task/94306/exe (readlink: Permission denied) 2024-06-03T03:27:58.778 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94300 94306 podman root NOFD /proc/94300/task/94306/fd (opendir: Permission denied) 2024-06-03T03:27:58.778 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94300 94307 podman root cwd unknown /proc/94300/task/94307/cwd (readlink: Permission denied) 2024-06-03T03:27:58.778 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94300 94307 podman root rtd unknown /proc/94300/task/94307/root (readlink: Permission denied) 2024-06-03T03:27:58.779 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94300 94307 podman root txt unknown /proc/94300/task/94307/exe (readlink: Permission denied) 2024-06-03T03:27:58.779 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94300 94307 podman root NOFD /proc/94300/task/94307/fd (opendir: Permission denied) 2024-06-03T03:27:58.779 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94300 94308 podman root cwd unknown /proc/94300/task/94308/cwd (readlink: Permission denied) 2024-06-03T03:27:58.779 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94300 94308 podman root rtd unknown /proc/94300/task/94308/root (readlink: Permission denied) 2024-06-03T03:27:58.779 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94300 94308 podman root txt unknown /proc/94300/task/94308/exe (readlink: Permission denied) 2024-06-03T03:27:58.779 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94300 94308 podman root NOFD /proc/94300/task/94308/fd (opendir: Permission denied) 2024-06-03T03:27:58.779 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94300 94309 podman root cwd unknown /proc/94300/task/94309/cwd (readlink: Permission denied) 2024-06-03T03:27:58.779 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94300 94309 podman root rtd unknown /proc/94300/task/94309/root (readlink: Permission denied) 2024-06-03T03:27:58.779 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94300 94309 podman root txt unknown /proc/94300/task/94309/exe (readlink: Permission denied) 2024-06-03T03:27:58.779 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94300 94309 podman root NOFD /proc/94300/task/94309/fd (opendir: Permission denied) 2024-06-03T03:27:58.779 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94300 94310 podman root cwd unknown /proc/94300/task/94310/cwd (readlink: Permission denied) 2024-06-03T03:27:58.779 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94300 94310 podman root rtd unknown /proc/94300/task/94310/root (readlink: Permission denied) 2024-06-03T03:27:58.779 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94300 94310 podman root txt unknown /proc/94300/task/94310/exe (readlink: Permission denied) 2024-06-03T03:27:58.779 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94300 94310 podman root NOFD /proc/94300/task/94310/fd (opendir: Permission denied) 2024-06-03T03:27:58.780 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94300 94311 podman root cwd unknown /proc/94300/task/94311/cwd (readlink: Permission denied) 2024-06-03T03:27:58.780 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94300 94311 podman root rtd unknown /proc/94300/task/94311/root (readlink: Permission denied) 2024-06-03T03:27:58.780 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94300 94311 podman root txt unknown /proc/94300/task/94311/exe (readlink: Permission denied) 2024-06-03T03:27:58.780 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94300 94311 podman root NOFD /proc/94300/task/94311/fd (opendir: Permission denied) 2024-06-03T03:27:58.780 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94300 94312 podman root cwd unknown /proc/94300/task/94312/cwd (readlink: Permission denied) 2024-06-03T03:27:58.780 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94300 94312 podman root rtd unknown /proc/94300/task/94312/root (readlink: Permission denied) 2024-06-03T03:27:58.780 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94300 94312 podman root txt unknown /proc/94300/task/94312/exe (readlink: Permission denied) 2024-06-03T03:27:58.780 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94300 94312 podman root NOFD /proc/94300/task/94312/fd (opendir: Permission denied) 2024-06-03T03:27:58.780 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94300 94313 podman root cwd unknown /proc/94300/task/94313/cwd (readlink: Permission denied) 2024-06-03T03:27:58.780 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94300 94313 podman root rtd unknown /proc/94300/task/94313/root (readlink: Permission denied) 2024-06-03T03:27:58.780 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94300 94313 podman root txt unknown /proc/94300/task/94313/exe (readlink: Permission denied) 2024-06-03T03:27:58.780 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94300 94313 podman root NOFD /proc/94300/task/94313/fd (opendir: Permission denied) 2024-06-03T03:27:58.780 INFO:tasks.workunit.client.0.smithi117.stdout:conmon 94315 root cwd unknown /proc/94315/cwd (readlink: Permission denied) 2024-06-03T03:27:58.780 INFO:tasks.workunit.client.0.smithi117.stdout:conmon 94315 root rtd unknown /proc/94315/root (readlink: Permission denied) 2024-06-03T03:27:58.781 INFO:tasks.workunit.client.0.smithi117.stdout:conmon 94315 root txt unknown /proc/94315/exe (readlink: Permission denied) 2024-06-03T03:27:58.781 INFO:tasks.workunit.client.0.smithi117.stdout:conmon 94315 root NOFD /proc/94315/fd (opendir: Permission denied) 2024-06-03T03:27:58.781 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94320 root cwd unknown /proc/94320/cwd (readlink: Permission denied) 2024-06-03T03:27:58.781 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94320 root rtd unknown /proc/94320/root (readlink: Permission denied) 2024-06-03T03:27:58.781 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94320 root txt unknown /proc/94320/exe (readlink: Permission denied) 2024-06-03T03:27:58.781 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94320 root NOFD /proc/94320/fd (opendir: Permission denied) 2024-06-03T03:27:58.781 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94320 94321 podman root cwd unknown /proc/94320/task/94321/cwd (readlink: Permission denied) 2024-06-03T03:27:58.781 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94320 94321 podman root rtd unknown /proc/94320/task/94321/root (readlink: Permission denied) 2024-06-03T03:27:58.781 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94320 94321 podman root txt unknown /proc/94320/task/94321/exe (readlink: Permission denied) 2024-06-03T03:27:58.781 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94320 94321 podman root NOFD /proc/94320/task/94321/fd (opendir: Permission denied) 2024-06-03T03:27:58.781 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94320 94322 podman root cwd unknown /proc/94320/task/94322/cwd (readlink: Permission denied) 2024-06-03T03:27:58.781 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94320 94322 podman root rtd unknown /proc/94320/task/94322/root (readlink: Permission denied) 2024-06-03T03:27:58.781 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94320 94322 podman root txt unknown /proc/94320/task/94322/exe (readlink: Permission denied) 2024-06-03T03:27:58.781 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94320 94322 podman root NOFD /proc/94320/task/94322/fd (opendir: Permission denied) 2024-06-03T03:27:58.782 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94320 94323 podman root cwd unknown /proc/94320/task/94323/cwd (readlink: Permission denied) 2024-06-03T03:27:58.782 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94320 94323 podman root rtd unknown /proc/94320/task/94323/root (readlink: Permission denied) 2024-06-03T03:27:58.782 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94320 94323 podman root txt unknown /proc/94320/task/94323/exe (readlink: Permission denied) 2024-06-03T03:27:58.782 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94320 94323 podman root NOFD /proc/94320/task/94323/fd (opendir: Permission denied) 2024-06-03T03:27:58.782 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94320 94324 podman root cwd unknown /proc/94320/task/94324/cwd (readlink: Permission denied) 2024-06-03T03:27:58.782 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94320 94324 podman root rtd unknown /proc/94320/task/94324/root (readlink: Permission denied) 2024-06-03T03:27:58.782 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94320 94324 podman root txt unknown /proc/94320/task/94324/exe (readlink: Permission denied) 2024-06-03T03:27:58.782 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94320 94324 podman root NOFD /proc/94320/task/94324/fd (opendir: Permission denied) 2024-06-03T03:27:58.782 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94320 94325 podman root cwd unknown /proc/94320/task/94325/cwd (readlink: Permission denied) 2024-06-03T03:27:58.782 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94320 94325 podman root rtd unknown /proc/94320/task/94325/root (readlink: Permission denied) 2024-06-03T03:27:58.782 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94320 94325 podman root txt unknown /proc/94320/task/94325/exe (readlink: Permission denied) 2024-06-03T03:27:58.782 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94320 94325 podman root NOFD /proc/94320/task/94325/fd (opendir: Permission denied) 2024-06-03T03:27:58.782 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94320 94326 podman root cwd unknown /proc/94320/task/94326/cwd (readlink: Permission denied) 2024-06-03T03:27:58.783 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94320 94326 podman root rtd unknown /proc/94320/task/94326/root (readlink: Permission denied) 2024-06-03T03:27:58.783 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94320 94326 podman root txt unknown /proc/94320/task/94326/exe (readlink: Permission denied) 2024-06-03T03:27:58.783 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94320 94326 podman root NOFD /proc/94320/task/94326/fd (opendir: Permission denied) 2024-06-03T03:27:58.783 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94320 94327 podman root cwd unknown /proc/94320/task/94327/cwd (readlink: Permission denied) 2024-06-03T03:27:58.783 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94320 94327 podman root rtd unknown /proc/94320/task/94327/root (readlink: Permission denied) 2024-06-03T03:27:58.783 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94320 94327 podman root txt unknown /proc/94320/task/94327/exe (readlink: Permission denied) 2024-06-03T03:27:58.783 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94320 94327 podman root NOFD /proc/94320/task/94327/fd (opendir: Permission denied) 2024-06-03T03:27:58.783 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94320 94328 podman root cwd unknown /proc/94320/task/94328/cwd (readlink: Permission denied) 2024-06-03T03:27:58.783 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94320 94328 podman root rtd unknown /proc/94320/task/94328/root (readlink: Permission denied) 2024-06-03T03:27:58.783 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94320 94328 podman root txt unknown /proc/94320/task/94328/exe (readlink: Permission denied) 2024-06-03T03:27:58.783 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94320 94328 podman root NOFD /proc/94320/task/94328/fd (opendir: Permission denied) 2024-06-03T03:27:58.783 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94320 94329 podman root cwd unknown /proc/94320/task/94329/cwd (readlink: Permission denied) 2024-06-03T03:27:58.783 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94320 94329 podman root rtd unknown /proc/94320/task/94329/root (readlink: Permission denied) 2024-06-03T03:27:58.783 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94320 94329 podman root txt unknown /proc/94320/task/94329/exe (readlink: Permission denied) 2024-06-03T03:27:58.784 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94320 94329 podman root NOFD /proc/94320/task/94329/fd (opendir: Permission denied) 2024-06-03T03:27:58.784 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94320 94331 podman root cwd unknown /proc/94320/task/94331/cwd (readlink: Permission denied) 2024-06-03T03:27:58.784 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94320 94331 podman root rtd unknown /proc/94320/task/94331/root (readlink: Permission denied) 2024-06-03T03:27:58.784 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94320 94331 podman root txt unknown /proc/94320/task/94331/exe (readlink: Permission denied) 2024-06-03T03:27:58.784 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94320 94331 podman root NOFD /proc/94320/task/94331/fd (opendir: Permission denied) 2024-06-03T03:27:58.784 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94320 94332 podman root cwd unknown /proc/94320/task/94332/cwd (readlink: Permission denied) 2024-06-03T03:27:58.784 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94320 94332 podman root rtd unknown /proc/94320/task/94332/root (readlink: Permission denied) 2024-06-03T03:27:58.784 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94320 94332 podman root txt unknown /proc/94320/task/94332/exe (readlink: Permission denied) 2024-06-03T03:27:58.784 INFO:tasks.workunit.client.0.smithi117.stdout:podman 94320 94332 podman root NOFD /proc/94320/task/94332/fd (opendir: Permission denied) 2024-06-03T03:27:58.784 INFO:tasks.workunit.client.0.smithi117.stdout:lsof 94334 ubuntu cwd DIR 0,119 4 1099511627782 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-06-03T03:27:58.784 INFO:tasks.workunit.client.0.smithi117.stdout:lsof 94334 ubuntu rtd DIR 8,1 4096 2 / 2024-06-03T03:27:58.784 INFO:tasks.workunit.client.0.smithi117.stdout:lsof 94334 ubuntu txt REG 8,1 179232 9797 /usr/bin/lsof 2024-06-03T03:27:58.784 INFO:tasks.workunit.client.0.smithi117.stdout:lsof 94334 ubuntu mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-06-03T03:27:58.784 INFO:tasks.workunit.client.0.smithi117.stdout:lsof 94334 ubuntu mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-06-03T03:27:58.785 INFO:tasks.workunit.client.0.smithi117.stdout:lsof 94334 ubuntu mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-06-03T03:27:58.785 INFO:tasks.workunit.client.0.smithi117.stdout:lsof 94334 ubuntu mem REG 8,1 5431656 24524 /usr/lib64/libcrypto.so.3.2.1 2024-06-03T03:27:58.785 INFO:tasks.workunit.client.0.smithi117.stdout:lsof 94334 ubuntu mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-06-03T03:27:58.785 INFO:tasks.workunit.client.0.smithi117.stdout:lsof 94334 ubuntu mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-06-03T03:27:58.785 INFO:tasks.workunit.client.0.smithi117.stdout:lsof 94334 ubuntu mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-06-03T03:27:58.785 INFO:tasks.workunit.client.0.smithi117.stdout:lsof 94334 ubuntu mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-06-03T03:27:58.785 INFO:tasks.workunit.client.0.smithi117.stdout:lsof 94334 ubuntu mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-06-03T03:27:58.785 INFO:tasks.workunit.client.0.smithi117.stdout:lsof 94334 ubuntu mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-06-03T03:27:58.785 INFO:tasks.workunit.client.0.smithi117.stdout:lsof 94334 ubuntu mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-06-03T03:27:58.785 INFO:tasks.workunit.client.0.smithi117.stdout:lsof 94334 ubuntu mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-06-03T03:27:58.785 INFO:tasks.workunit.client.0.smithi117.stdout:lsof 94334 ubuntu mem REG 8,1 196224 8914 /usr/lib64/libtirpc.so.3.0.0 2024-06-03T03:27:58.785 INFO:tasks.workunit.client.0.smithi117.stdout:lsof 94334 ubuntu mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-06-03T03:27:58.786 INFO:tasks.workunit.client.0.smithi117.stdout:lsof 94334 ubuntu mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-06-03T03:27:58.786 INFO:tasks.workunit.client.0.smithi117.stdout:lsof 94334 ubuntu mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-06-03T03:27:58.786 INFO:tasks.workunit.client.0.smithi117.stdout:lsof 94334 ubuntu 0r FIFO 0,14 0t0 254675 pipe 2024-06-03T03:27:58.786 INFO:tasks.workunit.client.0.smithi117.stdout:lsof 94334 ubuntu 1w FIFO 0,14 0t0 254676 pipe 2024-06-03T03:27:58.786 INFO:tasks.workunit.client.0.smithi117.stdout:lsof 94334 ubuntu 2w FIFO 0,14 0t0 254677 pipe 2024-06-03T03:27:58.786 INFO:tasks.workunit.client.0.smithi117.stdout:lsof 94334 ubuntu 3r DIR 0,21 0 1 /proc 2024-06-03T03:27:58.786 INFO:tasks.workunit.client.0.smithi117.stdout:lsof 94334 ubuntu 4r DIR 0,21 7 264432 /proc/94334/fd 2024-06-03T03:27:58.786 INFO:tasks.workunit.client.0.smithi117.stdout:lsof 94334 ubuntu 5w FIFO 0,14 0t0 264437 pipe 2024-06-03T03:27:58.786 INFO:tasks.workunit.client.0.smithi117.stdout:lsof 94334 ubuntu 6r FIFO 0,14 0t0 264438 pipe 2024-06-03T03:27:58.786 INFO:tasks.workunit.client.0.smithi117.stdout:lsof 94335 ubuntu cwd DIR 0,119 4 1099511627782 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-06-03T03:27:58.786 INFO:tasks.workunit.client.0.smithi117.stdout:lsof 94335 ubuntu rtd DIR 8,1 4096 2 / 2024-06-03T03:27:58.786 INFO:tasks.workunit.client.0.smithi117.stdout:lsof 94335 ubuntu txt REG 8,1 179232 9797 /usr/bin/lsof 2024-06-03T03:27:58.786 INFO:tasks.workunit.client.0.smithi117.stdout:lsof 94335 ubuntu mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-06-03T03:27:58.787 INFO:tasks.workunit.client.0.smithi117.stdout:lsof 94335 ubuntu mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-06-03T03:27:58.787 INFO:tasks.workunit.client.0.smithi117.stdout:lsof 94335 ubuntu mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-06-03T03:27:58.787 INFO:tasks.workunit.client.0.smithi117.stdout:lsof 94335 ubuntu mem REG 8,1 5431656 24524 /usr/lib64/libcrypto.so.3.2.1 2024-06-03T03:27:58.787 INFO:tasks.workunit.client.0.smithi117.stdout:lsof 94335 ubuntu mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-06-03T03:27:58.787 INFO:tasks.workunit.client.0.smithi117.stdout:lsof 94335 ubuntu mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-06-03T03:27:58.787 INFO:tasks.workunit.client.0.smithi117.stdout:lsof 94335 ubuntu mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-06-03T03:27:58.787 INFO:tasks.workunit.client.0.smithi117.stdout:lsof 94335 ubuntu mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-06-03T03:27:58.787 INFO:tasks.workunit.client.0.smithi117.stdout:lsof 94335 ubuntu mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-06-03T03:27:58.787 INFO:tasks.workunit.client.0.smithi117.stdout:lsof 94335 ubuntu mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-06-03T03:27:58.787 INFO:tasks.workunit.client.0.smithi117.stdout:lsof 94335 ubuntu mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-06-03T03:27:58.787 INFO:tasks.workunit.client.0.smithi117.stdout:lsof 94335 ubuntu mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-06-03T03:27:58.788 INFO:tasks.workunit.client.0.smithi117.stdout:lsof 94335 ubuntu mem REG 8,1 196224 8914 /usr/lib64/libtirpc.so.3.0.0 2024-06-03T03:27:58.788 INFO:tasks.workunit.client.0.smithi117.stdout:lsof 94335 ubuntu mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-06-03T03:27:58.788 INFO:tasks.workunit.client.0.smithi117.stdout:lsof 94335 ubuntu mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-06-03T03:27:58.788 INFO:tasks.workunit.client.0.smithi117.stdout:lsof 94335 ubuntu mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-06-03T03:27:58.788 INFO:tasks.workunit.client.0.smithi117.stdout:lsof 94335 ubuntu 4r FIFO 0,14 0t0 264437 pipe 2024-06-03T03:27:58.788 INFO:tasks.workunit.client.0.smithi117.stdout:lsof 94335 ubuntu 7w FIFO 0,14 0t0 264438 pipe 2024-06-03T03:27:58.788 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-06-03T03:27:58.788 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-06-03T03:27:58.899 INFO:tasks.workunit:Stopping ['suites/fsync-tester.sh'] on client.0... 2024-06-03T03:27:58.899 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-06-03T03:27:59.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:27:59 smithi151 ceph-mon[37043]: pgmap v586: 129 pgs: 129 active+clean; 258 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 863 KiB/s wr, 17 op/s 2024-06-03T03:27:59.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:27:59 smithi117 ceph-mon[31649]: pgmap v586: 129 pgs: 129 active+clean; 258 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 863 KiB/s wr, 17 op/s 2024-06-03T03:27:59.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:27:59 smithi183 ceph-mon[37326]: pgmap v586: 129 pgs: 129 active+clean; 258 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 863 KiB/s wr, 17 op/s 2024-06-03T03:27:59.970 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:59.969+0000 7feee5000700 1 -- 172.21.15.117:0/3490370711 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feee01073c0 msgr2=0x7feee006a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:59.970 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:59.969+0000 7feee5000700 1 --2- 172.21.15.117:0/3490370711 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feee01073c0 0x7feee006a1a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7feed0009990 tx=0x7feed0009ca0 comp rx=0 tx=0).stop 2024-06-03T03:27:59.970 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:59.969+0000 7feee5000700 1 -- 172.21.15.117:0/3490370711 shutdown_connections 2024-06-03T03:27:59.970 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:59.969+0000 7feee5000700 1 --2- 172.21.15.117:0/3490370711 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feee0104ed0 0x7feee0105360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:59.970 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:59.969+0000 7feee5000700 1 --2- 172.21.15.117:0/3490370711 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feee01073c0 0x7feee006a1a0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:59.971 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:59.969+0000 7feee5000700 1 --2- 172.21.15.117:0/3490370711 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feee006a6e0 0x7feee006ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:59.971 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:59.969+0000 7feee5000700 1 -- 172.21.15.117:0/3490370711 >> 172.21.15.117:0/3490370711 conn(0x7feee0078340 msgr2=0x7feee0078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:27:59.971 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:59.969+0000 7feee5000700 1 -- 172.21.15.117:0/3490370711 shutdown_connections 2024-06-03T03:27:59.971 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:59.969+0000 7feee5000700 1 -- 172.21.15.117:0/3490370711 wait complete. 2024-06-03T03:27:59.971 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:59.971+0000 7feee5000700 1 Processor -- start 2024-06-03T03:27:59.971 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:59.971+0000 7feee5000700 1 -- start start 2024-06-03T03:27:59.972 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:59.971+0000 7feee5000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feee006a6e0 0x7feee019f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:59.972 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:59.971+0000 7feee5000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feee0104ed0 0x7feee019f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:59.972 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:59.971+0000 7feee5000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feee01073c0 0x7feee01a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:59.972 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:59.971+0000 7feee5000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7feee0118d50 con 0x7feee006a6e0 2024-06-03T03:27:59.972 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:59.972+0000 7feee5000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7feee0118bd0 con 0x7feee0104ed0 2024-06-03T03:27:59.972 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:59.972+0000 7feee5000700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7feee0118ed0 con 0x7feee01073c0 2024-06-03T03:27:59.973 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:59.972+0000 7feedd400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feee0104ed0 0x7feee019f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:59.973 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:59.972+0000 7feedd400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feee0104ed0 0x7feee019f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:57138/0 (socket says 172.21.15.117:57138) 2024-06-03T03:27:59.973 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:59.972+0000 7feedd400700 1 -- 172.21.15.117:0/2430157512 learned_addr learned my addr 172.21.15.117:0/2430157512 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:27:59.973 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:59.972+0000 7feedde00700 1 --2- 172.21.15.117:0/2430157512 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feee006a6e0 0x7feee019f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:59.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:59.972+0000 7feede800700 1 --2- 172.21.15.117:0/2430157512 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feee01073c0 0x7feee01a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:59.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:59.972+0000 7feedd400700 1 -- 172.21.15.117:0/2430157512 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feee01073c0 msgr2=0x7feee01a3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:59.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:59.972+0000 7feedd400700 1 --2- 172.21.15.117:0/2430157512 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feee01073c0 0x7feee01a3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:59.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:59.972+0000 7feedd400700 1 -- 172.21.15.117:0/2430157512 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feee006a6e0 msgr2=0x7feee019f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:27:59.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:59.972+0000 7feedd400700 1 --2- 172.21.15.117:0/2430157512 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feee006a6e0 0x7feee019f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:27:59.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:59.972+0000 7feedd400700 1 -- 172.21.15.117:0/2430157512 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feee01a4230 con 0x7feee0104ed0 2024-06-03T03:27:59.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:59.973+0000 7feede800700 1 --2- 172.21.15.117:0/2430157512 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feee01073c0 0x7feee01a3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T03:27:59.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:59.973+0000 7feedde00700 1 --2- 172.21.15.117:0/2430157512 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feee006a6e0 0x7feee019f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T03:27:59.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:59.973+0000 7feedd400700 1 --2- 172.21.15.117:0/2430157512 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feee0104ed0 0x7feee019f760 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7feecc00e9a0 tx=0x7feecc00ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:27:59.975 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:59.974+0000 7feed6a00700 1 -- 172.21.15.117:0/2430157512 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feecc00cbd0 con 0x7feee0104ed0 2024-06-03T03:27:59.975 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:59.974+0000 7feed6a00700 1 -- 172.21.15.117:0/2430157512 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feecc00cd70 con 0x7feee0104ed0 2024-06-03T03:27:59.975 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:59.974+0000 7feed6a00700 1 -- 172.21.15.117:0/2430157512 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feecc0187c0 con 0x7feee0104ed0 2024-06-03T03:27:59.975 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:59.974+0000 7feee5000700 1 -- 172.21.15.117:0/2430157512 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feee01a4520 con 0x7feee0104ed0 2024-06-03T03:27:59.976 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:59.974+0000 7feee5000700 1 -- 172.21.15.117:0/2430157512 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feee01abda0 con 0x7feee0104ed0 2024-06-03T03:27:59.976 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:59.975+0000 7feee5000700 1 -- 172.21.15.117:0/2430157512 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feee0069180 con 0x7feee0104ed0 2024-06-03T03:27:59.980 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:59.980+0000 7feed6a00700 1 -- 172.21.15.117:0/2430157512 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7feecc010430 con 0x7feee0104ed0 2024-06-03T03:27:59.981 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:59.981+0000 7feed6a00700 1 --2- 172.21.15.117:0/2430157512 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7feec40a2ca0 0x7feec40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:27:59.981 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:59.981+0000 7feed6a00700 1 -- 172.21.15.117:0/2430157512 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7feecc0cf6e0 con 0x7feee0104ed0 2024-06-03T03:27:59.982 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:59.981+0000 7feedde00700 1 --2- 172.21.15.117:0/2430157512 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7feec40a2ca0 0x7feec40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:27:59.982 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:59.982+0000 7feedde00700 1 --2- 172.21.15.117:0/2430157512 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7feec40a2ca0 0x7feec40a5160 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7feed000b2e0 tx=0x7feed0005e80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:27:59.988 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:27:59.987+0000 7feed6a00700 1 -- 172.21.15.117:0/2430157512 <== mon.2 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7feecc09c250 con 0x7feee0104ed0 2024-06-03T03:28:00.315 DEBUG:teuthology.parallel:result is None 2024-06-03T03:28:00.315 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-06-03T03:28:00.417 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-06-03T03:28:00.417 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-06-03T03:28:00.427 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi117.front.sepia.ceph.com 2024-06-03T03:28:00.458 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-06-03T03:28:00.467 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-06-03T03:28:00.491 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:00.490+0000 7feee5000700 1 -- 172.21.15.117:0/2430157512 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feee01abf70 con 0x7feee0104ed0 2024-06-03T03:28:00.494 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:00.493+0000 7feed6a00700 1 -- 172.21.15.117:0/2430157512 <== mon.2 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11955 (secure 0 0 0) 0x7feecc09bb50 con 0x7feee0104ed0 2024-06-03T03:28:00.495 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T03:28:00.495 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3353001119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3353001119},{"type":"v1","addr":"172.21.15.117:6837","nonce":3353001119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14388,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6839/405782304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":405782304},{"type":"v1","addr":"172.21.15.117:6839","nonce":405782304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-03T03:19:44.629822+0000","modified":"2024-06-03T03:25:11.576498+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24517,"mds_1":24512,"mds_2":24503,"mds_3":24499,"mds_4":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24431":{"gid":24431,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6833/3355662416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3355662416},{"type":"v1","addr":"172.21.15.151:6833","nonce":3355662416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/2429725085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2429725085},{"type":"v1","addr":"172.21.15.183:6833","nonce":2429725085}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/31153579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":31153579},{"type":"v1","addr":"172.21.15.151:6835","nonce":31153579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"j","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/2884460088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2884460088},{"type":"v1","addr":"172.21.15.117:6835","nonce":2884460088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/2392407848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":2392407848},{"type":"v1","addr":"172.21.15.151:6837","nonce":2392407848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":32,"addr":"172.21.15.183:6835/1009004320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1009004320},{"type":"v1","addr":"172.21.15.183:6835","nonce":1009004320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.183:6837/2697518526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":2697518526},{"type":"v1","addr":"172.21.15.183:6837","nonce":2697518526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"a","rank":2,"incarnation":22,"state":"up:active","state_seq":27,"addr":"172.21.15.117:6841/1212242967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6840","nonce":1212242967},{"type":"v1","addr":"172.21.15.117:6841","nonce":1212242967}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.151:6839/3623444616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":3623444616},{"type":"v1","addr":"172.21.15.151:6839","nonce":3623444616}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":10,"addr":"172.21.15.183:6839/1439615083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1439615083},{"type":"v1","addr":"172.21.15.183:6839","nonce":1439615083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24517,"qdb_cluster":[24499,24487,24512,24503,24517]},"id":1}]} 2024-06-03T03:28:00.498 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:00.497+0000 7feee5000700 1 -- 172.21.15.117:0/2430157512 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7feec40a2ca0 msgr2=0x7feec40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:28:00.498 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:00.497+0000 7feee5000700 1 --2- 172.21.15.117:0/2430157512 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7feec40a2ca0 0x7feec40a5160 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7feed000b2e0 tx=0x7feed0005e80 comp rx=0 tx=0).stop 2024-06-03T03:28:00.498 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:00.497+0000 7feee5000700 1 -- 172.21.15.117:0/2430157512 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feee0104ed0 msgr2=0x7feee019f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:28:00.498 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:00.497+0000 7feee5000700 1 --2- 172.21.15.117:0/2430157512 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feee0104ed0 0x7feee019f760 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7feecc00e9a0 tx=0x7feecc00ed60 comp rx=0 tx=0).stop 2024-06-03T03:28:00.498 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:00.497+0000 7feee5000700 1 -- 172.21.15.117:0/2430157512 shutdown_connections 2024-06-03T03:28:00.499 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:00.497+0000 7feee5000700 1 --2- 172.21.15.117:0/2430157512 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7feec40a2ca0 0x7feec40a5160 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:00.499 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:00.497+0000 7feee5000700 1 --2- 172.21.15.117:0/2430157512 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feee01073c0 0x7feee01a3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:00.499 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:00.497+0000 7feee5000700 1 --2- 172.21.15.117:0/2430157512 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feee006a6e0 0x7feee019f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:00.499 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:00.497+0000 7feee5000700 1 --2- 172.21.15.117:0/2430157512 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feee0104ed0 0x7feee019f760 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:00.499 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:00.497+0000 7feee5000700 1 -- 172.21.15.117:0/2430157512 >> 172.21.15.117:0/2430157512 conn(0x7feee0078340 msgr2=0x7feee0100380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:28:00.499 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:00.498+0000 7feee5000700 1 -- 172.21.15.117:0/2430157512 shutdown_connections 2024-06-03T03:28:00.499 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:00.498+0000 7feee5000700 1 -- 172.21.15.117:0/2430157512 wait complete. 2024-06-03T03:28:00.502 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 31 2024-06-03T03:28:00.588 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:28:00 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/2430157512' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T03:28:00.906 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:28:00 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/2430157512' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T03:28:00.981 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:28:00 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/2430157512' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T03:28:01.651 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-06-03T03:28:01.768 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:28:01 smithi117 ceph-mon[31649]: pgmap v587: 129 pgs: 129 active+clean; 258 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.5 MiB/s wr, 16 op/s 2024-06-03T03:28:01.906 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:28:01 smithi151 ceph-mon[37043]: pgmap v587: 129 pgs: 129 active+clean; 258 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.5 MiB/s wr, 16 op/s 2024-06-03T03:28:01.981 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:28:01 smithi183 ceph-mon[37326]: pgmap v587: 129 pgs: 129 active+clean; 258 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.5 MiB/s wr, 16 op/s 2024-06-03T03:28:02.169 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:28:03.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:28:03 smithi151 ceph-mon[37043]: pgmap v588: 129 pgs: 129 active+clean; 182 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 23 op/s 2024-06-03T03:28:03.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:28:03 smithi117 ceph-mon[31649]: pgmap v588: 129 pgs: 129 active+clean; 182 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 23 op/s 2024-06-03T03:28:03.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:28:03 smithi183 ceph-mon[37326]: pgmap v588: 129 pgs: 129 active+clean; 182 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 23 op/s 2024-06-03T03:28:04.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.670+0000 7fb4bea00700 1 -- 172.21.15.117:0/997651627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb4b81024e0 msgr2=0x7fb4b8102960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:28:04.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.670+0000 7fb4bea00700 1 --2- 172.21.15.117:0/997651627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb4b81024e0 0x7fb4b8102960 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb4a40098f0 tx=0x7fb4a4009c00 comp rx=0 tx=0).stop 2024-06-03T03:28:04.672 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.670+0000 7fb4bea00700 1 -- 172.21.15.117:0/997651627 shutdown_connections 2024-06-03T03:28:04.672 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.670+0000 7fb4bea00700 1 --2- 172.21.15.117:0/997651627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4b8102ea0 0x7fb4b810f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:04.672 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.670+0000 7fb4bea00700 1 --2- 172.21.15.117:0/997651627 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4b81085f0 0x7fb4b81089d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:04.672 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.670+0000 7fb4bea00700 1 --2- 172.21.15.117:0/997651627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb4b81024e0 0x7fb4b8102960 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:04.672 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.670+0000 7fb4bea00700 1 -- 172.21.15.117:0/997651627 >> 172.21.15.117:0/997651627 conn(0x7fb4b8079640 msgr2=0x7fb4b8079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:28:04.672 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.670+0000 7fb4bea00700 1 -- 172.21.15.117:0/997651627 shutdown_connections 2024-06-03T03:28:04.672 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.670+0000 7fb4bea00700 1 -- 172.21.15.117:0/997651627 wait complete. 2024-06-03T03:28:04.674 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.671+0000 7fb4bea00700 1 Processor -- start 2024-06-03T03:28:04.674 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.672+0000 7fb4bea00700 1 -- start start 2024-06-03T03:28:04.674 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.672+0000 7fb4bea00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb4b81024e0 0x7fb4b8074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:04.674 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.672+0000 7fb4bea00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4b8102ea0 0x7fb4b8074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:04.674 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.672+0000 7fb4bea00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4b81085f0 0x7fb4b8075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:04.674 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.672+0000 7fb4bea00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b8112790 con 0x7fb4b8102ea0 2024-06-03T03:28:04.674 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.672+0000 7fb4bea00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b8112610 con 0x7fb4b81085f0 2024-06-03T03:28:04.674 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.672+0000 7fb4bea00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b8112910 con 0x7fb4b81024e0 2024-06-03T03:28:04.674 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.672+0000 7fb4b7e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb4b81024e0 0x7fb4b8074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:04.674 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.672+0000 7fb4b7e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb4b81024e0 0x7fb4b8074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:44778/0 (socket says 172.21.15.117:44778) 2024-06-03T03:28:04.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.672+0000 7fb4b7e00700 1 -- 172.21.15.117:0/2700001580 learned_addr learned my addr 172.21.15.117:0/2700001580 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:28:04.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.672+0000 7fb4b7400700 1 --2- 172.21.15.117:0/2700001580 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4b8102ea0 0x7fb4b8074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:04.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.672+0000 7fb4bca00700 1 --2- 172.21.15.117:0/2700001580 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4b81085f0 0x7fb4b8075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:04.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.673+0000 7fb4b7e00700 1 -- 172.21.15.117:0/2700001580 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4b81085f0 msgr2=0x7fb4b8075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:28:04.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.673+0000 7fb4b7e00700 1 --2- 172.21.15.117:0/2700001580 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4b81085f0 0x7fb4b8075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:04.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.673+0000 7fb4b7e00700 1 -- 172.21.15.117:0/2700001580 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4b8102ea0 msgr2=0x7fb4b8074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:28:04.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.673+0000 7fb4b7e00700 1 --2- 172.21.15.117:0/2700001580 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4b8102ea0 0x7fb4b8074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:04.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.673+0000 7fb4b7e00700 1 -- 172.21.15.117:0/2700001580 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4b81a7130 con 0x7fb4b81024e0 2024-06-03T03:28:04.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.673+0000 7fb4b7e00700 1 --2- 172.21.15.117:0/2700001580 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb4b81024e0 0x7fb4b8074790 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb4a800e9a0 tx=0x7fb4a800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:28:04.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.673+0000 7fb4b4c00700 1 -- 172.21.15.117:0/2700001580 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a800ca10 con 0x7fb4b81024e0 2024-06-03T03:28:04.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.673+0000 7fb4b4c00700 1 -- 172.21.15.117:0/2700001580 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4a800cbb0 con 0x7fb4b81024e0 2024-06-03T03:28:04.676 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.673+0000 7fb4bea00700 1 -- 172.21.15.117:0/2700001580 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4b81a7420 con 0x7fb4b81024e0 2024-06-03T03:28:04.676 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.673+0000 7fb4b4c00700 1 -- 172.21.15.117:0/2700001580 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a8018680 con 0x7fb4b81024e0 2024-06-03T03:28:04.676 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.674+0000 7fb4bea00700 1 -- 172.21.15.117:0/2700001580 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4b81a7930 con 0x7fb4b81024e0 2024-06-03T03:28:04.677 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.677+0000 7fb4b4c00700 1 -- 172.21.15.117:0/2700001580 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fb4a8010430 con 0x7fb4b81024e0 2024-06-03T03:28:04.678 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.678+0000 7fb4c93ecb80 1 Processor -- start 2024-06-03T03:28:04.679 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.678+0000 7fb4b4c00700 1 --2- 172.21.15.117:0/2700001580 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fb49c0a2c20 0x7fb49c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:04.679 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.678+0000 7fb4c93ecb80 1 -- start start 2024-06-03T03:28:04.679 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.678+0000 7fb4b4c00700 1 -- 172.21.15.117:0/2700001580 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7fb4a80cf5e0 con 0x7fb4b81024e0 2024-06-03T03:28:04.680 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.679+0000 7fb4bca00700 1 --2- 172.21.15.117:0/2700001580 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fb49c0a2c20 0x7fb49c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:04.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.679+0000 7fb4c93ecb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557840b2fb90 0x557840b2ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:04.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.679+0000 7fb4c93ecb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557840b304b0 0x557840b29d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:04.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.679+0000 7fb4c93ecb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557840b2a240 0x557840b2a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:04.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.679+0000 7fb4c93ecb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557840a38a10 con 0x557840b304b0 2024-06-03T03:28:04.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.679+0000 7fb4c93ecb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x557840a7bb00 con 0x557840b2fb90 2024-06-03T03:28:04.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.679+0000 7fb4c93ecb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557840a368f0 con 0x557840b2a240 2024-06-03T03:28:04.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.679+0000 7fb4b7400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557840b304b0 0x557840b29d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:04.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.679+0000 7fb4b7400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557840b304b0 0x557840b29d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:43094/0 (socket says 172.21.15.117:43094) 2024-06-03T03:28:04.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.679+0000 7fb4b7400700 1 -- 172.21.15.117:0/4075840296 learned_addr learned my addr 172.21.15.117:0/4075840296 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:28:04.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.679+0000 7fb4bca00700 1 --2- 172.21.15.117:0/4075840296 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557840b2a240 0x557840b2a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:04.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.680+0000 7fb4bca00700 1 --2- 172.21.15.117:0/2700001580 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fb49c0a2c20 0x7fb49c0a50e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb4b0009fd0 tx=0x7fb4b000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:28:04.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.680+0000 7fb4b7400700 1 -- 172.21.15.117:0/4075840296 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557840b2a240 msgr2=0x557840b2a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:28:04.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.680+0000 7fb4b7400700 1 --2- 172.21.15.117:0/4075840296 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557840b2a240 0x557840b2a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:04.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.680+0000 7fb4b7e00700 1 --2- 172.21.15.117:0/4075840296 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557840b2fb90 0x557840b2ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:04.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.680+0000 7fb4b7400700 1 -- 172.21.15.117:0/4075840296 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557840b2fb90 msgr2=0x557840b2ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:28:04.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.680+0000 7fb4b7400700 1 --2- 172.21.15.117:0/4075840296 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557840b2fb90 0x557840b2ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:04.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.680+0000 7fb4b7400700 1 -- 172.21.15.117:0/4075840296 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557840a23f90 con 0x557840b304b0 2024-06-03T03:28:04.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.680+0000 7fb4b7e00700 1 --2- 172.21.15.117:0/4075840296 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557840b2fb90 0x557840b2ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T03:28:04.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.680+0000 7fb4b7400700 1 --2- 172.21.15.117:0/4075840296 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557840b304b0 0x557840b29d00 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb4a4009230 tx=0x7fb4a4005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=902ff42bb7d13684 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:28:04.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.681+0000 7fb4ad600700 1 -- 172.21.15.117:0/4075840296 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a400f7e0 con 0x557840b304b0 2024-06-03T03:28:04.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.681+0000 7fb4ad600700 1 -- 172.21.15.117:0/4075840296 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4a400f980 con 0x557840b304b0 2024-06-03T03:28:04.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.681+0000 7fb4ad600700 1 -- 172.21.15.117:0/4075840296 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a4003d00 con 0x557840b304b0 2024-06-03T03:28:04.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.681+0000 7fb4c93ecb80 1 -- 172.21.15.117:0/4075840296 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557840b304b0 msgr2=0x557840b29d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:28:04.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.681+0000 7fb4c93ecb80 1 --2- 172.21.15.117:0/4075840296 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557840b304b0 0x557840b29d00 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb4a4009230 tx=0x7fb4a4005dc0 comp rx=0 tx=0).stop 2024-06-03T03:28:04.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.682+0000 7fb4c93ecb80 1 -- 172.21.15.117:0/4075840296 shutdown_connections 2024-06-03T03:28:04.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.682+0000 7fb4c93ecb80 1 --2- 172.21.15.117:0/4075840296 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557840b2a240 0x557840b2a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:04.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.682+0000 7fb4c93ecb80 1 --2- 172.21.15.117:0/4075840296 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557840b2fb90 0x557840b2ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:04.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.682+0000 7fb4c93ecb80 1 --2- 172.21.15.117:0/4075840296 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557840b304b0 0x557840b29d00 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:04.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.682+0000 7fb4c93ecb80 1 -- 172.21.15.117:0/4075840296 >> 172.21.15.117:0/4075840296 conn(0x557840a56170 msgr2=0x557840b24ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:28:04.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.683+0000 7fb4c93ecb80 1 -- 172.21.15.117:0/4075840296 shutdown_connections 2024-06-03T03:28:04.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.683+0000 7fb4c93ecb80 1 -- 172.21.15.117:0/4075840296 wait complete. 2024-06-03T03:28:04.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.683+0000 7fb4c93ecb80 1 -- start start 2024-06-03T03:28:04.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.683+0000 7fb4c93ecb80 1 Processor -- start 2024-06-03T03:28:04.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.684+0000 7fb4c93ecb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557840b2a240 0x557840b361b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:04.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.684+0000 7fb4c93ecb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557840b2fb90 0x557840b334b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:04.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.684+0000 7fb4c93ecb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557840b304b0 0x557840b339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:04.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.684+0000 7fb4c93ecb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557840a17f60 con 0x557840b2a240 2024-06-03T03:28:04.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.684+0000 7fb4c93ecb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x557840a77ac0 con 0x557840b304b0 2024-06-03T03:28:04.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.684+0000 7fb4c93ecb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557840b36e80 con 0x557840b2fb90 2024-06-03T03:28:04.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.684+0000 7fb4c93ecb80 10 client.0 _ll_register_callbacks cb 0x557840a11380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:28:04.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.684+0000 7fb4b7e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557840b2a240 0x557840b361b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:04.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.684+0000 7fb4b7e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557840b2a240 0x557840b361b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:43106/0 (socket says 172.21.15.117:43106) 2024-06-03T03:28:04.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.684+0000 7fb4b7e00700 1 -- 172.21.15.117:0/74767565 learned_addr learned my addr 172.21.15.117:0/74767565 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:28:04.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.685+0000 7fb4bca00700 1 --2- 172.21.15.117:0/74767565 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557840b304b0 0x557840b339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:04.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.685+0000 7fb4b7400700 1 --2- 172.21.15.117:0/74767565 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557840b2fb90 0x557840b334b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:04.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.685+0000 7fb4b7e00700 1 -- 172.21.15.117:0/74767565 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557840b2fb90 msgr2=0x557840b334b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:28:04.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.685+0000 7fb4b7e00700 1 --2- 172.21.15.117:0/74767565 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557840b2fb90 0x557840b334b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:04.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.685+0000 7fb4b7e00700 1 -- 172.21.15.117:0/74767565 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557840b304b0 msgr2=0x557840b339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:28:04.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.685+0000 7fb4b7e00700 1 --2- 172.21.15.117:0/74767565 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557840b304b0 0x557840b339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:04.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.685+0000 7fb4b7e00700 1 -- 172.21.15.117:0/74767565 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578409ca8d0 con 0x557840b2a240 2024-06-03T03:28:04.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.685+0000 7fb4b7e00700 1 --2- 172.21.15.117:0/74767565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557840b2a240 0x557840b361b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fb4a8009230 tx=0x7fb4a809bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe0e12daa3bb197 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:28:04.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.685+0000 7fb4a2000700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-06-03T03:28:04.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.686+0000 7fb4a2000700 1 -- 172.21.15.117:0/74767565 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a809fe00 con 0x557840b2a240 2024-06-03T03:28:04.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.686+0000 7fb4a2000700 1 -- 172.21.15.117:0/74767565 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7fb4a80cf290 con 0x557840b2a240 2024-06-03T03:28:04.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.686+0000 7fb4c93ecb80 1 -- 172.21.15.117:0/74767565 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4a400f980 con 0x557840b2a240 2024-06-03T03:28:04.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.686+0000 7fb4a2000700 1 -- 172.21.15.117:0/74767565 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fb4a80a46c0 con 0x557840b2a240 2024-06-03T03:28:04.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.686+0000 7fb4c93ecb80 10 client.15663 fetch_fsmap learned FSMap version 31 2024-06-03T03:28:04.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.686+0000 7fb4c93ecb80 1 -- 172.21.15.117:0/74767565 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x557840b41550 con 0x557840b2a240 2024-06-03T03:28:04.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.686+0000 7fb4a2000700 1 -- 172.21.15.117:0/74767565 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4a400f980 con 0x557840b2a240 2024-06-03T03:28:04.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.686+0000 7fb4a2000700 1 -- 172.21.15.117:0/74767565 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fb4a809aea0 con 0x557840b2a240 2024-06-03T03:28:04.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.687+0000 7fb4c93ecb80 10 client.15663 fetch_fsmap finished waiting for FSMap version 31 2024-06-03T03:28:04.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.687+0000 7fb4c93ecb80 10 client.15663 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-06-03T03:28:04.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.687+0000 7fb4c93ecb80 20 client.15663 populate_metadata read hostname 'smithi117' 2024-06-03T03:28:04.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.687+0000 7fb4c93ecb80 1 --2- 172.21.15.117:0/74767565 >> [v2:172.21.15.183:6834/1009004320,v1:172.21.15.183:6835/1009004320] conn(0x557840b43b90 0x557840b45f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:04.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.687+0000 7fb4c93ecb80 4 client.15663 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T03:28:04.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.687+0000 7fb4c93ecb80 1 -- 172.21.15.117:0/74767565 --> [v2:172.21.15.183:6834/1009004320,v1:172.21.15.183:6835/1009004320] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557840a89000 con 0x557840b43b90 2024-06-03T03:28:04.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.687+0000 7fb4b7400700 1 --2- 172.21.15.117:0/74767565 >> [v2:172.21.15.183:6834/1009004320,v1:172.21.15.183:6835/1009004320] conn(0x557840b43b90 0x557840b45f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:04.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.688+0000 7fb4b7400700 1 --2- 172.21.15.117:0/74767565 >> [v2:172.21.15.183:6834/1009004320,v1:172.21.15.183:6835/1009004320] conn(0x557840b43b90 0x557840b45f80 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ccfc3ee0551bb97 server_cookie=f199b77f7f5a9d4 in_seq=0 out_seq=0 2024-06-03T03:28:04.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.688+0000 7fb4a2000700 10 client.15663 ms_handle_connect on v2:172.21.15.183:6834/1009004320 2024-06-03T03:28:04.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.688+0000 7fb4a2000700 1 -- 172.21.15.117:0/74767565 <== mds.4 v2:172.21.15.183:6834/1009004320 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557840a89000 con 0x557840b43b90 2024-06-03T03:28:04.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.688+0000 7fb4a2000700 10 client.15663 handle_command_reply: tid=0 2024-06-03T03:28:04.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.689+0000 7fb4c93ecb80 1 client.15663 shutdown 2024-06-03T03:28:04.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.689+0000 7fb4c93ecb80 1 -- 172.21.15.117:0/74767565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557840b2a240 msgr2=0x557840b361b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:28:04.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.689+0000 7fb4c93ecb80 1 --2- 172.21.15.117:0/74767565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557840b2a240 0x557840b361b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fb4a8009230 tx=0x7fb4a809bf90 comp rx=0 tx=0).stop 2024-06-03T03:28:04.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.689+0000 7fb4c93ecb80 1 -- 172.21.15.117:0/74767565 shutdown_connections 2024-06-03T03:28:04.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.689+0000 7fb4c93ecb80 1 --2- 172.21.15.117:0/74767565 >> [v2:172.21.15.183:6834/1009004320,v1:172.21.15.183:6835/1009004320] conn(0x557840b43b90 0x557840b45f80 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:04.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.689+0000 7fb4c93ecb80 1 --2- 172.21.15.117:0/74767565 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557840b304b0 0x557840b339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:04.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.689+0000 7fb4c93ecb80 1 --2- 172.21.15.117:0/74767565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557840b2a240 0x557840b361b0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:04.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.689+0000 7fb4c93ecb80 1 --2- 172.21.15.117:0/74767565 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557840b2fb90 0x557840b334b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:04.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.689+0000 7fb4c93ecb80 1 -- 172.21.15.117:0/74767565 >> 172.21.15.117:0/74767565 conn(0x557840a56170 msgr2=0x557840b1e790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:28:04.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.689+0000 7fb4a2000700 0 client.15663 ms_handle_reset on v2:172.21.15.183:6834/1009004320 2024-06-03T03:28:04.697 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.689+0000 7fb4c93ecb80 1 -- 172.21.15.117:0/74767565 shutdown_connections 2024-06-03T03:28:04.697 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.689+0000 7fb4c93ecb80 1 -- 172.21.15.117:0/74767565 wait complete. 2024-06-03T03:28:04.697 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.689+0000 7fb4c93ecb80 20 client.15663 trim_cache size 0 max 16384 2024-06-03T03:28:04.707 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.706+0000 7fb4c93ecb80 1 Processor -- start 2024-06-03T03:28:04.707 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.706+0000 7fb4c93ecb80 1 -- start start 2024-06-03T03:28:04.707 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.706+0000 7fb4c93ecb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557840b2a240 0x557840b29d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:04.707 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.706+0000 7fb4c93ecb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557840b2fb90 0x557840b33ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:04.707 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.706+0000 7fb4c93ecb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557840b304b0 0x557840b1f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:04.707 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.706+0000 7fb4c93ecb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b81b0490 con 0x557840b2a240 2024-06-03T03:28:04.707 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.706+0000 7fb4c93ecb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b81b1fd0 con 0x557840b2fb90 2024-06-03T03:28:04.707 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.706+0000 7fb4c93ecb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b81b0b60 con 0x557840b304b0 2024-06-03T03:28:04.708 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.707+0000 7fb4bca00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557840b304b0 0x557840b1f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:04.708 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.707+0000 7fb4b7e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557840b2a240 0x557840b29d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:04.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.707+0000 7fb4bca00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557840b304b0 0x557840b1f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:44798/0 (socket says 172.21.15.117:44798) 2024-06-03T03:28:04.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.707+0000 7fb4bca00700 1 -- 172.21.15.117:0/1090773867 learned_addr learned my addr 172.21.15.117:0/1090773867 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:28:04.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.707+0000 7fb4b7400700 1 --2- 172.21.15.117:0/1090773867 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557840b2fb90 0x557840b33ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:04.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.707+0000 7fb4bca00700 1 -- 172.21.15.117:0/1090773867 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557840b2fb90 msgr2=0x557840b33ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:28:04.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.707+0000 7fb4bca00700 1 --2- 172.21.15.117:0/1090773867 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557840b2fb90 0x557840b33ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:04.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.707+0000 7fb4bca00700 1 -- 172.21.15.117:0/1090773867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557840b2a240 msgr2=0x557840b29d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:28:04.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.707+0000 7fb4bca00700 1 --2- 172.21.15.117:0/1090773867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557840b2a240 0x557840b29d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:04.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.707+0000 7fb4bca00700 1 -- 172.21.15.117:0/1090773867 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557840b2f580 con 0x557840b304b0 2024-06-03T03:28:04.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.708+0000 7fb4bca00700 1 --2- 172.21.15.117:0/1090773867 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557840b304b0 0x557840b1f130 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb4b0017630 tx=0x7fb4b0014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1882bcac7c572a28 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:28:04.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.708+0000 7fb4acc00700 1 -- 172.21.15.117:0/1090773867 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b0023850 con 0x557840b304b0 2024-06-03T03:28:04.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.708+0000 7fb4acc00700 1 -- 172.21.15.117:0/1090773867 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4b001b050 con 0x557840b304b0 2024-06-03T03:28:04.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.708+0000 7fb4acc00700 1 -- 172.21.15.117:0/1090773867 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b002cdb0 con 0x557840b304b0 2024-06-03T03:28:04.711 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.709+0000 7fb4c93ecb80 1 -- 172.21.15.117:0/1090773867 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557840b304b0 msgr2=0x557840b1f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:28:04.711 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.709+0000 7fb4c93ecb80 1 --2- 172.21.15.117:0/1090773867 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557840b304b0 0x557840b1f130 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb4b0017630 tx=0x7fb4b0014000 comp rx=0 tx=0).stop 2024-06-03T03:28:04.711 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.709+0000 7fb4c93ecb80 1 -- 172.21.15.117:0/1090773867 shutdown_connections 2024-06-03T03:28:04.711 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.709+0000 7fb4c93ecb80 1 --2- 172.21.15.117:0/1090773867 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557840b304b0 0x557840b1f130 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:04.712 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.709+0000 7fb4c93ecb80 1 --2- 172.21.15.117:0/1090773867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557840b2a240 0x557840b29d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:04.712 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.709+0000 7fb4c93ecb80 1 --2- 172.21.15.117:0/1090773867 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557840b2fb90 0x557840b33ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:04.712 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.709+0000 7fb4c93ecb80 1 -- 172.21.15.117:0/1090773867 >> 172.21.15.117:0/1090773867 conn(0x557840a56170 msgr2=0x557840b30890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:28:04.712 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.709+0000 7fb4c93ecb80 1 -- 172.21.15.117:0/1090773867 shutdown_connections 2024-06-03T03:28:04.712 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.709+0000 7fb4c93ecb80 1 -- 172.21.15.117:0/1090773867 wait complete. 2024-06-03T03:28:04.712 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.710+0000 7fb4c93ecb80 1 -- start start 2024-06-03T03:28:04.712 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.710+0000 7fb4c93ecb80 1 Processor -- start 2024-06-03T03:28:04.712 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.710+0000 7fb4c93ecb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557840b2a240 0x557840b3c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:04.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.710+0000 7fb4c93ecb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557840b2fb90 0x557840b3c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:04.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.710+0000 7fb4c93ecb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557840b304b0 0x557840b4a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:04.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.710+0000 7fb4c93ecb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b81b1900 con 0x557840b2a240 2024-06-03T03:28:04.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.710+0000 7fb4c93ecb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b81b1230 con 0x557840b304b0 2024-06-03T03:28:04.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.710+0000 7fb4c93ecb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b81ae430 con 0x557840b2fb90 2024-06-03T03:28:04.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.710+0000 7fb4c93ecb80 10 client.0 _ll_register_callbacks cb 0x557840a11380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:28:04.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.711+0000 7fb4b7400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557840b2fb90 0x557840b3c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:04.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.711+0000 7fb4b7400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557840b2fb90 0x557840b3c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:44804/0 (socket says 172.21.15.117:44804) 2024-06-03T03:28:04.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.711+0000 7fb4b7400700 1 -- 172.21.15.117:0/4042374876 learned_addr learned my addr 172.21.15.117:0/4042374876 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:28:04.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.711+0000 7fb4b7e00700 1 --2- 172.21.15.117:0/4042374876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557840b2a240 0x557840b3c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:04.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.711+0000 7fb4bca00700 1 --2- 172.21.15.117:0/4042374876 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557840b304b0 0x557840b4a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:04.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.711+0000 7fb4b7400700 1 -- 172.21.15.117:0/4042374876 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557840b304b0 msgr2=0x557840b4a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:28:04.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.711+0000 7fb4b7400700 1 --2- 172.21.15.117:0/4042374876 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557840b304b0 0x557840b4a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:04.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.711+0000 7fb4b7400700 1 -- 172.21.15.117:0/4042374876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557840b2a240 msgr2=0x557840b3c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:28:04.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.711+0000 7fb4b7400700 1 --2- 172.21.15.117:0/4042374876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557840b2a240 0x557840b3c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:04.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.711+0000 7fb4b7400700 1 -- 172.21.15.117:0/4042374876 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557840b4ad80 con 0x557840b2fb90 2024-06-03T03:28:04.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.711+0000 7fb4b7400700 1 --2- 172.21.15.117:0/4042374876 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557840b2fb90 0x557840b3c680 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb4a4003890 tx=0x7fb4a4009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df8f4dcc743f986b server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:28:04.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.711+0000 7fb4a2000700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-06-03T03:28:04.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.712+0000 7fb4a2000700 1 -- 172.21.15.117:0/4042374876 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a4033060 con 0x557840b2fb90 2024-06-03T03:28:04.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.712+0000 7fb4c93ecb80 1 -- 172.21.15.117:0/4042374876 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4b001b050 con 0x557840b2fb90 2024-06-03T03:28:04.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.712+0000 7fb4a2000700 1 -- 172.21.15.117:0/4042374876 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7fb4a4003930 con 0x557840b2fb90 2024-06-03T03:28:04.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.712+0000 7fb4a2000700 1 -- 172.21.15.117:0/4042374876 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fb4a4003b30 con 0x557840b2fb90 2024-06-03T03:28:04.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.712+0000 7fb4c93ecb80 10 client.25816 fetch_fsmap learned FSMap version 31 2024-06-03T03:28:04.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.712+0000 7fb4c93ecb80 1 -- 172.21.15.117:0/4042374876 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x557840badee0 con 0x557840b2fb90 2024-06-03T03:28:04.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.712+0000 7fb4a2000700 1 -- 172.21.15.117:0/4042374876 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4b001b050 con 0x557840b2fb90 2024-06-03T03:28:04.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.713+0000 7fb4a2000700 1 -- 172.21.15.117:0/4042374876 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fb4a400b7d0 con 0x557840b2fb90 2024-06-03T03:28:04.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.713+0000 7fb4c93ecb80 10 client.25816 fetch_fsmap finished waiting for FSMap version 31 2024-06-03T03:28:04.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.713+0000 7fb4c93ecb80 10 client.25816 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-06-03T03:28:04.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.713+0000 7fb4c93ecb80 20 client.25816 populate_metadata read hostname 'smithi117' 2024-06-03T03:28:04.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.713+0000 7fb4c93ecb80 1 --2- 172.21.15.117:0/4042374876 >> [v2:172.21.15.183:6834/1009004320,v1:172.21.15.183:6835/1009004320] conn(0x557840bae170 0x557840bb0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:04.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.713+0000 7fb4c93ecb80 4 client.25816 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T03:28:04.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.713+0000 7fb4c93ecb80 1 -- 172.21.15.117:0/4042374876 --> [v2:172.21.15.183:6834/1009004320,v1:172.21.15.183:6835/1009004320] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557840a83580 con 0x557840bae170 2024-06-03T03:28:04.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.713+0000 7fb4b7e00700 1 --2- 172.21.15.117:0/4042374876 >> [v2:172.21.15.183:6834/1009004320,v1:172.21.15.183:6835/1009004320] conn(0x557840bae170 0x557840bb0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:04.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.713+0000 7fb4b7e00700 1 --2- 172.21.15.117:0/4042374876 >> [v2:172.21.15.183:6834/1009004320,v1:172.21.15.183:6835/1009004320] conn(0x557840bae170 0x557840bb0560 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6ea5a31a3e592d41 server_cookie=1fb61ebe5c60c0bf in_seq=0 out_seq=0 2024-06-03T03:28:04.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.713+0000 7fb4a2000700 10 client.25816 ms_handle_connect on v2:172.21.15.183:6834/1009004320 2024-06-03T03:28:04.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.714+0000 7fb4a2000700 1 -- 172.21.15.117:0/4042374876 <== mds.4 v2:172.21.15.183:6834/1009004320 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557840a83580 con 0x557840bae170 2024-06-03T03:28:04.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.714+0000 7fb4a2000700 10 client.25816 handle_command_reply: tid=0 2024-06-03T03:28:04.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.714+0000 7fb4c93ecb80 1 client.25816 shutdown 2024-06-03T03:28:04.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.714+0000 7fb4c93ecb80 1 -- 172.21.15.117:0/4042374876 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557840b2fb90 msgr2=0x557840b3c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:28:04.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.714+0000 7fb4c93ecb80 1 --2- 172.21.15.117:0/4042374876 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557840b2fb90 0x557840b3c680 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb4a4003890 tx=0x7fb4a4009840 comp rx=0 tx=0).stop 2024-06-03T03:28:04.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.714+0000 7fb4c93ecb80 1 -- 172.21.15.117:0/4042374876 shutdown_connections 2024-06-03T03:28:04.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.714+0000 7fb4c93ecb80 1 --2- 172.21.15.117:0/4042374876 >> [v2:172.21.15.183:6834/1009004320,v1:172.21.15.183:6835/1009004320] conn(0x557840bae170 0x557840bb0560 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:04.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.714+0000 7fb4c93ecb80 1 --2- 172.21.15.117:0/4042374876 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557840b304b0 0x557840b4a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:04.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.714+0000 7fb4c93ecb80 1 --2- 172.21.15.117:0/4042374876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557840b2a240 0x557840b3c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:04.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.714+0000 7fb4c93ecb80 1 --2- 172.21.15.117:0/4042374876 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557840b2fb90 0x557840b3c680 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:04.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.714+0000 7fb4a2000700 0 client.25816 ms_handle_reset on v2:172.21.15.183:6834/1009004320 2024-06-03T03:28:04.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.714+0000 7fb4c93ecb80 1 -- 172.21.15.117:0/4042374876 >> 172.21.15.117:0/4042374876 conn(0x557840a56170 msgr2=0x557840b31da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:28:04.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.714+0000 7fb4c93ecb80 1 -- 172.21.15.117:0/4042374876 shutdown_connections 2024-06-03T03:28:04.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.714+0000 7fb4c93ecb80 1 -- 172.21.15.117:0/4042374876 wait complete. 2024-06-03T03:28:04.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.714+0000 7fb4c93ecb80 20 client.25816 trim_cache size 0 max 16384 2024-06-03T03:28:04.719 INFO:teuthology.orchestra.run.smithi117.stdout:[] 2024-06-03T03:28:04.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.715+0000 7fb4bea00700 1 -- 172.21.15.117:0/2700001580 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fb49c0a2c20 msgr2=0x7fb49c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:28:04.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.715+0000 7fb4bea00700 1 --2- 172.21.15.117:0/2700001580 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fb49c0a2c20 0x7fb49c0a50e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb4b0009fd0 tx=0x7fb4b000b040 comp rx=0 tx=0).stop 2024-06-03T03:28:04.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.715+0000 7fb4bea00700 1 -- 172.21.15.117:0/2700001580 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb4b81024e0 msgr2=0x7fb4b8074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:28:04.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.715+0000 7fb4bea00700 1 --2- 172.21.15.117:0/2700001580 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb4b81024e0 0x7fb4b8074790 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb4a800e9a0 tx=0x7fb4a800ecb0 comp rx=0 tx=0).stop 2024-06-03T03:28:04.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.715+0000 7fb4bea00700 1 -- 172.21.15.117:0/2700001580 shutdown_connections 2024-06-03T03:28:04.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.715+0000 7fb4bea00700 1 --2- 172.21.15.117:0/2700001580 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fb49c0a2c20 0x7fb49c0a50e0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:04.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.715+0000 7fb4bea00700 1 --2- 172.21.15.117:0/2700001580 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4b81085f0 0x7fb4b8075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:04.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.715+0000 7fb4bea00700 1 --2- 172.21.15.117:0/2700001580 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb4b81024e0 0x7fb4b8074790 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:04.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.715+0000 7fb4bea00700 1 --2- 172.21.15.117:0/2700001580 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4b8102ea0 0x7fb4b8074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:04.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.715+0000 7fb4bea00700 1 -- 172.21.15.117:0/2700001580 >> 172.21.15.117:0/2700001580 conn(0x7fb4b8079640 msgr2=0x7fb4b8101030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:28:04.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.715+0000 7fb4bea00700 1 -- 172.21.15.117:0/2700001580 shutdown_connections 2024-06-03T03:28:04.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:04.716+0000 7fb4bea00700 1 -- 172.21.15.117:0/2700001580 wait complete. 2024-06-03T03:28:05.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:28:05 smithi151 ceph-mon[37043]: pgmap v589: 129 pgs: 129 active+clean; 89 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 MiB/s wr, 23 op/s 2024-06-03T03:28:05.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:28:05 smithi117 ceph-mon[31649]: pgmap v589: 129 pgs: 129 active+clean; 89 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 MiB/s wr, 23 op/s 2024-06-03T03:28:05.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:28:05 smithi183 ceph-mon[37326]: pgmap v589: 129 pgs: 129 active+clean; 89 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 MiB/s wr, 23 op/s 2024-06-03T03:28:05.778 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-06-03T03:28:06.207 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:28:07.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:28:07 smithi151 ceph-mon[37043]: pgmap v590: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 MiB/s wr, 24 op/s 2024-06-03T03:28:07.409 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:28:07 smithi117 ceph-mon[31649]: pgmap v590: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 MiB/s wr, 24 op/s 2024-06-03T03:28:07.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:28:07 smithi183 ceph-mon[37326]: pgmap v590: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 MiB/s wr, 24 op/s 2024-06-03T03:28:08.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.749+0000 7f2a42000700 1 -- 172.21.15.117:0/3207253933 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a3c1030f0 msgr2=0x7f2a3c107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:28:08.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.749+0000 7f2a42000700 1 --2- 172.21.15.117:0/3207253933 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a3c1030f0 0x7f2a3c107110 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2a28009990 tx=0x7f2a28009ca0 comp rx=0 tx=0).stop 2024-06-03T03:28:08.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.749+0000 7f2a42000700 1 -- 172.21.15.117:0/3207253933 shutdown_connections 2024-06-03T03:28:08.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.749+0000 7f2a42000700 1 --2- 172.21.15.117:0/3207253933 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2a3c107840 0x7f2a3c10f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:08.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.749+0000 7f2a42000700 1 --2- 172.21.15.117:0/3207253933 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a3c102740 0x7f2a3c102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:08.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.749+0000 7f2a42000700 1 --2- 172.21.15.117:0/3207253933 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a3c1030f0 0x7f2a3c107110 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:08.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.749+0000 7f2a42000700 1 -- 172.21.15.117:0/3207253933 >> 172.21.15.117:0/3207253933 conn(0x7f2a3c0795f0 msgr2=0x7f2a3c079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:28:08.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.749+0000 7f2a42000700 1 -- 172.21.15.117:0/3207253933 shutdown_connections 2024-06-03T03:28:08.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.749+0000 7f2a42000700 1 -- 172.21.15.117:0/3207253933 wait complete. 2024-06-03T03:28:08.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.750+0000 7f2a42000700 1 Processor -- start 2024-06-03T03:28:08.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.750+0000 7f2a42000700 1 -- start start 2024-06-03T03:28:08.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.751+0000 7f2a42000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a3c102740 0x7f2a3c19ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:08.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.751+0000 7f2a42000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2a3c1030f0 0x7f2a3c19b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:08.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.751+0000 7f2a42000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a3c107840 0x7f2a3c19f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:08.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.751+0000 7f2a42000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2a3c1127c0 con 0x7f2a3c102740 2024-06-03T03:28:08.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.751+0000 7f2a42000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2a3c112640 con 0x7f2a3c1030f0 2024-06-03T03:28:08.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.751+0000 7f2a42000700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2a3c112940 con 0x7f2a3c107840 2024-06-03T03:28:08.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.751+0000 7f2a3aa00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2a3c1030f0 0x7f2a3c19b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:08.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.751+0000 7f2a3aa00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2a3c1030f0 0x7f2a3c19b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:57538/0 (socket says 172.21.15.117:57538) 2024-06-03T03:28:08.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.751+0000 7f2a3aa00700 1 -- 172.21.15.117:0/775048182 learned_addr learned my addr 172.21.15.117:0/775048182 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:28:08.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.751+0000 7f2a3b400700 1 --2- 172.21.15.117:0/775048182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a3c102740 0x7f2a3c19ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:08.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.751+0000 7f2a3be00700 1 --2- 172.21.15.117:0/775048182 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a3c107840 0x7f2a3c19f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:08.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.751+0000 7f2a3aa00700 1 -- 172.21.15.117:0/775048182 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a3c107840 msgr2=0x7f2a3c19f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:28:08.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.751+0000 7f2a3aa00700 1 --2- 172.21.15.117:0/775048182 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a3c107840 0x7f2a3c19f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:08.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.751+0000 7f2a3aa00700 1 -- 172.21.15.117:0/775048182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a3c102740 msgr2=0x7f2a3c19ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:28:08.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.751+0000 7f2a3aa00700 1 --2- 172.21.15.117:0/775048182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a3c102740 0x7f2a3c19ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:08.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.751+0000 7f2a3aa00700 1 -- 172.21.15.117:0/775048182 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a3c19fe30 con 0x7f2a3c1030f0 2024-06-03T03:28:08.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.752+0000 7f2a3aa00700 1 --2- 172.21.15.117:0/775048182 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2a3c1030f0 0x7f2a3c19b3b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2a28005910 tx=0x7f2a280047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:28:08.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.752+0000 7f2a33e00700 1 -- 172.21.15.117:0/775048182 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a28027070 con 0x7f2a3c1030f0 2024-06-03T03:28:08.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.752+0000 7f2a42000700 1 -- 172.21.15.117:0/775048182 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a3c1a00c0 con 0x7f2a3c1030f0 2024-06-03T03:28:08.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.752+0000 7f2a33e00700 1 -- 172.21.15.117:0/775048182 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a2800fde0 con 0x7f2a3c1030f0 2024-06-03T03:28:08.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.752+0000 7f2a33e00700 1 -- 172.21.15.117:0/775048182 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a280173f0 con 0x7f2a3c1030f0 2024-06-03T03:28:08.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.752+0000 7f2a42000700 1 -- 172.21.15.117:0/775048182 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a3c1a79a0 con 0x7f2a3c1030f0 2024-06-03T03:28:08.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.755+0000 7f2a33e00700 1 -- 172.21.15.117:0/775048182 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f2a2802a070 con 0x7f2a3c1030f0 2024-06-03T03:28:08.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.757+0000 7f2a4ca1cb80 1 Processor -- start 2024-06-03T03:28:08.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.757+0000 7f2a4ca1cb80 1 -- start start 2024-06-03T03:28:08.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.757+0000 7f2a33e00700 1 --2- 172.21.15.117:0/775048182 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f2a200a2c20 0x7f2a200a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:08.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.757+0000 7f2a4ca1cb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56530de75b90 0x56530de75f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:08.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.757+0000 7f2a4ca1cb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56530de764b0 0x56530de6fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:08.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.757+0000 7f2a4ca1cb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56530de70240 0x56530de70680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:08.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.757+0000 7f2a4ca1cb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56530dd7ea10 con 0x56530de70240 2024-06-03T03:28:08.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.757+0000 7f2a4ca1cb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56530ddc1b00 con 0x56530de764b0 2024-06-03T03:28:08.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.757+0000 7f2a4ca1cb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56530dd7c8f0 con 0x56530de75b90 2024-06-03T03:28:08.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.757+0000 7f2a33e00700 1 -- 172.21.15.117:0/775048182 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f2a280cff50 con 0x7f2a3c1030f0 2024-06-03T03:28:08.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.758+0000 7f2a3be00700 1 --2- 172.21.15.117:0/775048182 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f2a200a2c20 0x7f2a200a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:08.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.758+0000 7f2a3aa00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56530de764b0 0x56530de6fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:08.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.758+0000 7f2a3b400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56530de75b90 0x56530de75f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:08.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.758+0000 7f2a3be00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56530de70240 0x56530de70680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:08.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.758+0000 7f2a3be00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56530de70240 0x56530de70680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:52294/0 (socket says 172.21.15.117:52294) 2024-06-03T03:28:08.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.758+0000 7f2a3be00700 1 -- 172.21.15.117:0/2842550134 learned_addr learned my addr 172.21.15.117:0/2842550134 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:28:08.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.758+0000 7f2a3aa00700 1 -- 172.21.15.117:0/2842550134 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56530de75b90 msgr2=0x56530de75f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:28:08.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.758+0000 7f2a3aa00700 1 --2- 172.21.15.117:0/2842550134 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56530de75b90 0x56530de75f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:08.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.758+0000 7f2a3aa00700 1 -- 172.21.15.117:0/2842550134 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56530de70240 msgr2=0x56530de70680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:28:08.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.758+0000 7f2a3aa00700 1 --2- 172.21.15.117:0/2842550134 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56530de70240 0x56530de70680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:08.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.758+0000 7f2a3aa00700 1 -- 172.21.15.117:0/2842550134 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56530dd69f90 con 0x56530de764b0 2024-06-03T03:28:08.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.759+0000 7f2a3be00700 1 --2- 172.21.15.117:0/775048182 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f2a200a2c20 0x7f2a200a50e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f2a34009550 tx=0x7f2a34009380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:28:08.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.759+0000 7f2a3aa00700 1 --2- 172.21.15.117:0/2842550134 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56530de764b0 0x56530de6fd00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2a28006010 tx=0x7f2a2800ff90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae20a276089417e9 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:28:08.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.759+0000 7f2a30c00700 1 -- 172.21.15.117:0/2842550134 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a2800a430 con 0x56530de764b0 2024-06-03T03:28:08.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.759+0000 7f2a30c00700 1 -- 172.21.15.117:0/2842550134 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a2809cb50 con 0x56530de764b0 2024-06-03T03:28:08.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.759+0000 7f2a30c00700 1 -- 172.21.15.117:0/2842550134 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a2800a250 con 0x56530de764b0 2024-06-03T03:28:08.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.760+0000 7f2a4ca1cb80 1 -- 172.21.15.117:0/2842550134 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56530de764b0 msgr2=0x56530de6fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:28:08.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.760+0000 7f2a4ca1cb80 1 --2- 172.21.15.117:0/2842550134 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56530de764b0 0x56530de6fd00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2a28006010 tx=0x7f2a2800ff90 comp rx=0 tx=0).stop 2024-06-03T03:28:08.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.760+0000 7f2a4ca1cb80 1 -- 172.21.15.117:0/2842550134 shutdown_connections 2024-06-03T03:28:08.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.760+0000 7f2a4ca1cb80 1 --2- 172.21.15.117:0/2842550134 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56530de70240 0x56530de70680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:08.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.760+0000 7f2a4ca1cb80 1 --2- 172.21.15.117:0/2842550134 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56530de75b90 0x56530de75f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:08.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.760+0000 7f2a4ca1cb80 1 --2- 172.21.15.117:0/2842550134 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56530de764b0 0x56530de6fd00 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:08.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.760+0000 7f2a4ca1cb80 1 -- 172.21.15.117:0/2842550134 >> 172.21.15.117:0/2842550134 conn(0x56530dd9c170 msgr2=0x56530de6aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:28:08.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.760+0000 7f2a4ca1cb80 1 -- 172.21.15.117:0/2842550134 shutdown_connections 2024-06-03T03:28:08.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.760+0000 7f2a4ca1cb80 1 -- 172.21.15.117:0/2842550134 wait complete. 2024-06-03T03:28:08.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.761+0000 7f2a4ca1cb80 1 -- start start 2024-06-03T03:28:08.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.761+0000 7f2a4ca1cb80 1 Processor -- start 2024-06-03T03:28:08.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.762+0000 7f2a4ca1cb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56530de70240 0x56530de7c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:08.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.762+0000 7f2a4ca1cb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56530de75b90 0x56530de79540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:08.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.762+0000 7f2a4ca1cb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56530de764b0 0x56530de79a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:08.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.762+0000 7f2a4ca1cb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56530dd5df60 con 0x56530de75b90 2024-06-03T03:28:08.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.762+0000 7f2a4ca1cb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56530ddbdac0 con 0x56530de764b0 2024-06-03T03:28:08.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.762+0000 7f2a4ca1cb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56530de7cc40 con 0x56530de70240 2024-06-03T03:28:08.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.762+0000 7f2a4ca1cb80 10 client.0 _ll_register_callbacks cb 0x56530dd57380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:28:08.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.762+0000 7f2a3aa00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56530de75b90 0x56530de79540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:08.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.762+0000 7f2a3b400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56530de70240 0x56530de7c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:08.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.762+0000 7f2a3b400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56530de70240 0x56530de7c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:42724/0 (socket says 172.21.15.117:42724) 2024-06-03T03:28:08.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.762+0000 7f2a3b400700 1 -- 172.21.15.117:0/2744764057 learned_addr learned my addr 172.21.15.117:0/2744764057 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:28:08.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.762+0000 7f2a3be00700 1 --2- 172.21.15.117:0/2744764057 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56530de764b0 0x56530de79a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:08.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.762+0000 7f2a3b400700 1 -- 172.21.15.117:0/2744764057 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56530de764b0 msgr2=0x56530de79a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:28:08.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.762+0000 7f2a3b400700 1 --2- 172.21.15.117:0/2744764057 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56530de764b0 0x56530de79a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:08.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.762+0000 7f2a3b400700 1 -- 172.21.15.117:0/2744764057 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56530de75b90 msgr2=0x56530de79540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:28:08.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.762+0000 7f2a3b400700 1 --2- 172.21.15.117:0/2744764057 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56530de75b90 0x56530de79540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:08.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.763+0000 7f2a3b400700 1 -- 172.21.15.117:0/2744764057 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56530dd108d0 con 0x56530de70240 2024-06-03T03:28:08.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.763+0000 7f2a3aa00700 1 --2- 172.21.15.117:0/2744764057 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56530de75b90 0x56530de79540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:28:08.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.763+0000 7f2a3b400700 1 --2- 172.21.15.117:0/2744764057 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56530de70240 0x56530de7c240 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c0116f0 tx=0x7f2a2c011a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c02d3dd6b3df1b0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:28:08.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.763+0000 7f2a25600700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-06-03T03:28:08.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.763+0000 7f2a25600700 1 -- 172.21.15.117:0/2744764057 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a2c00f5d0 con 0x56530de70240 2024-06-03T03:28:08.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.763+0000 7f2a25600700 1 -- 172.21.15.117:0/2744764057 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f2a2c00f770 con 0x56530de70240 2024-06-03T03:28:08.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.763+0000 7f2a4ca1cb80 1 -- 172.21.15.117:0/2744764057 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a2809cb50 con 0x56530de70240 2024-06-03T03:28:08.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.763+0000 7f2a25600700 1 -- 172.21.15.117:0/2744764057 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f2a2c027050 con 0x56530de70240 2024-06-03T03:28:08.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.764+0000 7f2a4ca1cb80 10 client.25828 fetch_fsmap learned FSMap version 31 2024-06-03T03:28:08.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.764+0000 7f2a4ca1cb80 1 -- 172.21.15.117:0/2744764057 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56530de872b0 con 0x56530de70240 2024-06-03T03:28:08.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.764+0000 7f2a25600700 1 -- 172.21.15.117:0/2744764057 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a2809cb50 con 0x56530de70240 2024-06-03T03:28:08.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.764+0000 7f2a25600700 1 -- 172.21.15.117:0/2744764057 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f2a2c022030 con 0x56530de70240 2024-06-03T03:28:08.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.764+0000 7f2a4ca1cb80 10 client.25828 fetch_fsmap finished waiting for FSMap version 31 2024-06-03T03:28:08.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.764+0000 7f2a4ca1cb80 10 client.25828 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-06-03T03:28:08.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.764+0000 7f2a4ca1cb80 20 client.25828 populate_metadata read hostname 'smithi117' 2024-06-03T03:28:08.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.764+0000 7f2a4ca1cb80 1 --2- 172.21.15.117:0/2744764057 >> [v2:172.21.15.183:6836/2697518526,v1:172.21.15.183:6837/2697518526] conn(0x56530de89990 0x56530de8be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:08.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.764+0000 7f2a4ca1cb80 4 client.25828 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T03:28:08.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.764+0000 7f2a4ca1cb80 1 -- 172.21.15.117:0/2744764057 --> [v2:172.21.15.183:6836/2697518526,v1:172.21.15.183:6837/2697518526] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56530ddcf000 con 0x56530de89990 2024-06-03T03:28:08.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.765+0000 7f2a3aa00700 1 --2- 172.21.15.117:0/2744764057 >> [v2:172.21.15.183:6836/2697518526,v1:172.21.15.183:6837/2697518526] conn(0x56530de89990 0x56530de8be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:08.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.765+0000 7f2a3aa00700 1 --2- 172.21.15.117:0/2744764057 >> [v2:172.21.15.183:6836/2697518526,v1:172.21.15.183:6837/2697518526] conn(0x56530de89990 0x56530de8be10 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a0d749fd291a8c4e server_cookie=4bfea940483a010c in_seq=0 out_seq=0 2024-06-03T03:28:08.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.765+0000 7f2a25600700 10 client.25828 ms_handle_connect on v2:172.21.15.183:6836/2697518526 2024-06-03T03:28:08.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.766+0000 7f2a25600700 1 -- 172.21.15.117:0/2744764057 <== mds.3 v2:172.21.15.183:6836/2697518526 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56530ddcf000 con 0x56530de89990 2024-06-03T03:28:08.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.766+0000 7f2a25600700 10 client.25828 handle_command_reply: tid=0 2024-06-03T03:28:08.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.766+0000 7f2a4ca1cb80 1 client.25828 shutdown 2024-06-03T03:28:08.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.766+0000 7f2a4ca1cb80 1 -- 172.21.15.117:0/2744764057 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56530de70240 msgr2=0x56530de7c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:28:08.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.766+0000 7f2a4ca1cb80 1 --2- 172.21.15.117:0/2744764057 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56530de70240 0x56530de7c240 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c0116f0 tx=0x7f2a2c011a00 comp rx=0 tx=0).stop 2024-06-03T03:28:08.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.766+0000 7f2a4ca1cb80 1 -- 172.21.15.117:0/2744764057 shutdown_connections 2024-06-03T03:28:08.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.766+0000 7f2a4ca1cb80 1 --2- 172.21.15.117:0/2744764057 >> [v2:172.21.15.183:6836/2697518526,v1:172.21.15.183:6837/2697518526] conn(0x56530de89990 0x56530de8be10 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:08.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.766+0000 7f2a4ca1cb80 1 --2- 172.21.15.117:0/2744764057 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56530de764b0 0x56530de79a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:08.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.766+0000 7f2a4ca1cb80 1 --2- 172.21.15.117:0/2744764057 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56530de70240 0x56530de7c240 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:08.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.766+0000 7f2a4ca1cb80 1 --2- 172.21.15.117:0/2744764057 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56530de75b90 0x56530de79540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:08.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.766+0000 7f2a25600700 0 client.25828 ms_handle_reset on v2:172.21.15.183:6836/2697518526 2024-06-03T03:28:08.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.766+0000 7f2a4ca1cb80 1 -- 172.21.15.117:0/2744764057 >> 172.21.15.117:0/2744764057 conn(0x56530dd9c170 msgr2=0x56530de64790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:28:08.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.766+0000 7f2a4ca1cb80 1 -- 172.21.15.117:0/2744764057 shutdown_connections 2024-06-03T03:28:08.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.766+0000 7f2a4ca1cb80 1 -- 172.21.15.117:0/2744764057 wait complete. 2024-06-03T03:28:08.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.767+0000 7f2a4ca1cb80 20 client.25828 trim_cache size 0 max 16384 2024-06-03T03:28:08.785 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.783+0000 7f2a4ca1cb80 1 Processor -- start 2024-06-03T03:28:08.785 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.783+0000 7f2a4ca1cb80 1 -- start start 2024-06-03T03:28:08.785 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.783+0000 7f2a4ca1cb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56530de70240 0x56530de6fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:08.785 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.784+0000 7f2a4ca1cb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56530de75b90 0x56530de6b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:08.785 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.784+0000 7f2a4ca1cb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56530de764b0 0x56530de65130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:08.785 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.784+0000 7f2a4ca1cb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2a1c00ead0 con 0x56530de70240 2024-06-03T03:28:08.785 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.784+0000 7f2a4ca1cb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2a1c010610 con 0x56530de764b0 2024-06-03T03:28:08.786 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.784+0000 7f2a4ca1cb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2a1c00f1a0 con 0x56530de75b90 2024-06-03T03:28:08.787 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.784+0000 7f2a3b400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56530de70240 0x56530de6fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:08.787 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.784+0000 7f2a3be00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56530de764b0 0x56530de65130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:08.787 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.784+0000 7f2a3be00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56530de764b0 0x56530de65130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:57558/0 (socket says 172.21.15.117:57558) 2024-06-03T03:28:08.787 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.784+0000 7f2a3b400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56530de70240 0x56530de6fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:52312/0 (socket says 172.21.15.117:52312) 2024-06-03T03:28:08.787 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.784+0000 7f2a3be00700 1 -- 172.21.15.117:0/2946381024 learned_addr learned my addr 172.21.15.117:0/2946381024 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:28:08.788 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.784+0000 7f2a3aa00700 1 --2- 172.21.15.117:0/2946381024 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56530de75b90 0x56530de6b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:08.788 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.784+0000 7f2a3aa00700 1 -- 172.21.15.117:0/2946381024 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56530de764b0 msgr2=0x56530de65130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:28:08.788 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.784+0000 7f2a3aa00700 1 --2- 172.21.15.117:0/2946381024 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56530de764b0 0x56530de65130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:08.788 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.784+0000 7f2a3aa00700 1 -- 172.21.15.117:0/2946381024 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56530de70240 msgr2=0x56530de6fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:28:08.788 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.784+0000 7f2a3aa00700 1 --2- 172.21.15.117:0/2946381024 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56530de70240 0x56530de6fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:08.788 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.784+0000 7f2a3aa00700 1 -- 172.21.15.117:0/2946381024 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56530de8c490 con 0x56530de75b90 2024-06-03T03:28:08.788 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.785+0000 7f2a3b400700 1 --2- 172.21.15.117:0/2946381024 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56530de70240 0x56530de6fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:28:08.788 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.785+0000 7f2a3aa00700 1 --2- 172.21.15.117:0/2946381024 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56530de75b90 0x56530de6b1a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2a2809cd00 tx=0x7f2a28009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d4ae96b4d1be876 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:28:08.788 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.785+0000 7f2a27e00700 1 -- 172.21.15.117:0/2946381024 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a28083070 con 0x56530de75b90 2024-06-03T03:28:08.788 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.785+0000 7f2a27e00700 1 -- 172.21.15.117:0/2946381024 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a2809c450 con 0x56530de75b90 2024-06-03T03:28:08.788 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.785+0000 7f2a27e00700 1 -- 172.21.15.117:0/2946381024 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a2809c6b0 con 0x56530de75b90 2024-06-03T03:28:08.788 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.786+0000 7f2a4ca1cb80 1 -- 172.21.15.117:0/2946381024 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56530de75b90 msgr2=0x56530de6b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:28:08.789 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.786+0000 7f2a4ca1cb80 1 --2- 172.21.15.117:0/2946381024 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56530de75b90 0x56530de6b1a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2a2809cd00 tx=0x7f2a28009ca0 comp rx=0 tx=0).stop 2024-06-03T03:28:08.789 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.786+0000 7f2a4ca1cb80 1 -- 172.21.15.117:0/2946381024 shutdown_connections 2024-06-03T03:28:08.789 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.786+0000 7f2a4ca1cb80 1 --2- 172.21.15.117:0/2946381024 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56530de764b0 0x56530de65130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:08.789 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.786+0000 7f2a4ca1cb80 1 --2- 172.21.15.117:0/2946381024 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56530de70240 0x56530de6fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:08.789 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.786+0000 7f2a4ca1cb80 1 --2- 172.21.15.117:0/2946381024 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56530de75b90 0x56530de6b1a0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:08.789 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.786+0000 7f2a4ca1cb80 1 -- 172.21.15.117:0/2946381024 >> 172.21.15.117:0/2946381024 conn(0x56530dd9c170 msgr2=0x56530de6aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:28:08.789 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.786+0000 7f2a4ca1cb80 1 -- 172.21.15.117:0/2946381024 shutdown_connections 2024-06-03T03:28:08.789 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.786+0000 7f2a4ca1cb80 1 -- 172.21.15.117:0/2946381024 wait complete. 2024-06-03T03:28:08.791 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.786+0000 7f2a4ca1cb80 1 -- start start 2024-06-03T03:28:08.791 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.787+0000 7f2a4ca1cb80 1 Processor -- start 2024-06-03T03:28:08.791 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.787+0000 7f2a4ca1cb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56530de70240 0x56530de81e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:08.791 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.787+0000 7f2a4ca1cb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56530de75b90 0x56530de823d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:08.791 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.787+0000 7f2a4ca1cb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56530de764b0 0x56530de90770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:08.791 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.787+0000 7f2a4ca1cb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2a1c00ff40 con 0x56530de70240 2024-06-03T03:28:08.791 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.787+0000 7f2a4ca1cb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2a1c00f870 con 0x56530de75b90 2024-06-03T03:28:08.791 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.787+0000 7f2a4ca1cb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2a1c00ca70 con 0x56530de764b0 2024-06-03T03:28:08.792 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.787+0000 7f2a4ca1cb80 10 client.0 _ll_register_callbacks cb 0x56530dd57380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:28:08.792 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.787+0000 7f2a3b400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56530de70240 0x56530de81e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:08.792 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.788+0000 7f2a3be00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56530de764b0 0x56530de90770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:08.792 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.788+0000 7f2a3b400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56530de70240 0x56530de81e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:52328/0 (socket says 172.21.15.117:52328) 2024-06-03T03:28:08.792 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.788+0000 7f2a3b400700 1 -- 172.21.15.117:0/4136978298 learned_addr learned my addr 172.21.15.117:0/4136978298 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:28:08.792 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.788+0000 7f2a3aa00700 1 --2- 172.21.15.117:0/4136978298 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56530de75b90 0x56530de823d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:08.792 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.788+0000 7f2a3b400700 1 -- 172.21.15.117:0/4136978298 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56530de764b0 msgr2=0x56530de90770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:28:08.792 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.788+0000 7f2a3b400700 1 --2- 172.21.15.117:0/4136978298 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56530de764b0 0x56530de90770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:08.792 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.788+0000 7f2a3b400700 1 -- 172.21.15.117:0/4136978298 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56530de75b90 msgr2=0x56530de823d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:28:08.792 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.788+0000 7f2a3b400700 1 --2- 172.21.15.117:0/4136978298 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56530de75b90 0x56530de823d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:08.792 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.788+0000 7f2a3b400700 1 -- 172.21.15.117:0/4136978298 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56530de90cb0 con 0x56530de70240 2024-06-03T03:28:08.792 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.788+0000 7f2a3b400700 1 --2- 172.21.15.117:0/4136978298 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56530de70240 0x56530de81e90 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c009da0 tx=0x7f2a2c019410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b86d6d7f4ec06eb server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:28:08.792 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.788+0000 7f2a25600700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-06-03T03:28:08.792 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.789+0000 7f2a25600700 1 -- 172.21.15.117:0/4136978298 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a2c01a600 con 0x56530de70240 2024-06-03T03:28:08.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.789+0000 7f2a25600700 1 -- 172.21.15.117:0/4136978298 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f2a2c01a7a0 con 0x56530de70240 2024-06-03T03:28:08.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.789+0000 7f2a4ca1cb80 1 -- 172.21.15.117:0/4136978298 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a2809c450 con 0x56530de70240 2024-06-03T03:28:08.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.789+0000 7f2a25600700 1 -- 172.21.15.117:0/4136978298 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f2a2c03c050 con 0x56530de70240 2024-06-03T03:28:08.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.789+0000 7f2a4ca1cb80 10 client.15687 fetch_fsmap learned FSMap version 31 2024-06-03T03:28:08.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.789+0000 7f2a4ca1cb80 1 -- 172.21.15.117:0/4136978298 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56530def3e10 con 0x56530de70240 2024-06-03T03:28:08.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.789+0000 7f2a25600700 1 -- 172.21.15.117:0/4136978298 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a2809c450 con 0x56530de70240 2024-06-03T03:28:08.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.789+0000 7f2a25600700 1 -- 172.21.15.117:0/4136978298 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f2a2c022050 con 0x56530de70240 2024-06-03T03:28:08.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.790+0000 7f2a4ca1cb80 10 client.15687 fetch_fsmap finished waiting for FSMap version 31 2024-06-03T03:28:08.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.790+0000 7f2a4ca1cb80 10 client.15687 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-06-03T03:28:08.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.790+0000 7f2a4ca1cb80 20 client.15687 populate_metadata read hostname 'smithi117' 2024-06-03T03:28:08.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.790+0000 7f2a4ca1cb80 1 --2- 172.21.15.117:0/4136978298 >> [v2:172.21.15.183:6836/2697518526,v1:172.21.15.183:6837/2697518526] conn(0x56530def4480 0x56530def68a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:08.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.790+0000 7f2a4ca1cb80 4 client.15687 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T03:28:08.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.790+0000 7f2a4ca1cb80 1 -- 172.21.15.117:0/4136978298 --> [v2:172.21.15.183:6836/2697518526,v1:172.21.15.183:6837/2697518526] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56530ddc9580 con 0x56530def4480 2024-06-03T03:28:08.795 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.790+0000 7f2a3aa00700 1 --2- 172.21.15.117:0/4136978298 >> [v2:172.21.15.183:6836/2697518526,v1:172.21.15.183:6837/2697518526] conn(0x56530def4480 0x56530def68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:08.795 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.791+0000 7f2a3aa00700 1 --2- 172.21.15.117:0/4136978298 >> [v2:172.21.15.183:6836/2697518526,v1:172.21.15.183:6837/2697518526] conn(0x56530def4480 0x56530def68a0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ca0cd0e7f69e7ace server_cookie=bc2fdc9363b80b5c in_seq=0 out_seq=0 2024-06-03T03:28:08.795 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.791+0000 7f2a25600700 10 client.15687 ms_handle_connect on v2:172.21.15.183:6836/2697518526 2024-06-03T03:28:08.795 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.791+0000 7f2a25600700 1 -- 172.21.15.117:0/4136978298 <== mds.3 v2:172.21.15.183:6836/2697518526 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56530ddc9580 con 0x56530def4480 2024-06-03T03:28:08.795 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.791+0000 7f2a25600700 10 client.15687 handle_command_reply: tid=0 2024-06-03T03:28:08.795 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.791+0000 7f2a4ca1cb80 1 client.15687 shutdown 2024-06-03T03:28:08.795 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.791+0000 7f2a4ca1cb80 1 -- 172.21.15.117:0/4136978298 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56530de70240 msgr2=0x56530de81e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:28:08.795 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.791+0000 7f2a4ca1cb80 1 --2- 172.21.15.117:0/4136978298 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56530de70240 0x56530de81e90 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c009da0 tx=0x7f2a2c019410 comp rx=0 tx=0).stop 2024-06-03T03:28:08.796 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.791+0000 7f2a4ca1cb80 1 -- 172.21.15.117:0/4136978298 shutdown_connections 2024-06-03T03:28:08.796 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.791+0000 7f2a4ca1cb80 1 --2- 172.21.15.117:0/4136978298 >> [v2:172.21.15.183:6836/2697518526,v1:172.21.15.183:6837/2697518526] conn(0x56530def4480 0x56530def68a0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:08.796 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.791+0000 7f2a4ca1cb80 1 --2- 172.21.15.117:0/4136978298 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56530de764b0 0x56530de90770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:08.796 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.791+0000 7f2a4ca1cb80 1 --2- 172.21.15.117:0/4136978298 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56530de70240 0x56530de81e90 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:08.796 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.791+0000 7f2a4ca1cb80 1 --2- 172.21.15.117:0/4136978298 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56530de75b90 0x56530de823d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:08.796 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.791+0000 7f2a25600700 0 client.15687 ms_handle_reset on v2:172.21.15.183:6836/2697518526 2024-06-03T03:28:08.796 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.791+0000 7f2a4ca1cb80 1 -- 172.21.15.117:0/4136978298 >> 172.21.15.117:0/4136978298 conn(0x56530dd9c170 msgr2=0x56530de76890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:28:08.796 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.791+0000 7f2a4ca1cb80 1 -- 172.21.15.117:0/4136978298 shutdown_connections 2024-06-03T03:28:08.796 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.791+0000 7f2a4ca1cb80 1 -- 172.21.15.117:0/4136978298 wait complete. 2024-06-03T03:28:08.796 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.792+0000 7f2a4ca1cb80 20 client.15687 trim_cache size 0 max 16384 2024-06-03T03:28:08.797 INFO:teuthology.orchestra.run.smithi117.stdout:[] 2024-06-03T03:28:08.797 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.792+0000 7f2a42000700 1 -- 172.21.15.117:0/775048182 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f2a200a2c20 msgr2=0x7f2a200a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:28:08.797 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.792+0000 7f2a42000700 1 --2- 172.21.15.117:0/775048182 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f2a200a2c20 0x7f2a200a50e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f2a34009550 tx=0x7f2a34009380 comp rx=0 tx=0).stop 2024-06-03T03:28:08.797 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.792+0000 7f2a42000700 1 -- 172.21.15.117:0/775048182 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2a3c1030f0 msgr2=0x7f2a3c19b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:28:08.797 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.792+0000 7f2a42000700 1 --2- 172.21.15.117:0/775048182 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2a3c1030f0 0x7f2a3c19b3b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2a28005910 tx=0x7f2a280047b0 comp rx=0 tx=0).stop 2024-06-03T03:28:08.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.792+0000 7f2a42000700 1 -- 172.21.15.117:0/775048182 shutdown_connections 2024-06-03T03:28:08.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.792+0000 7f2a42000700 1 --2- 172.21.15.117:0/775048182 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f2a200a2c20 0x7f2a200a50e0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:08.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.792+0000 7f2a42000700 1 --2- 172.21.15.117:0/775048182 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a3c107840 0x7f2a3c19f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:08.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.792+0000 7f2a42000700 1 --2- 172.21.15.117:0/775048182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a3c102740 0x7f2a3c19ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:08.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.792+0000 7f2a42000700 1 --2- 172.21.15.117:0/775048182 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2a3c1030f0 0x7f2a3c19b3b0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:08.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.792+0000 7f2a42000700 1 -- 172.21.15.117:0/775048182 >> 172.21.15.117:0/775048182 conn(0x7f2a3c0795f0 msgr2=0x7f2a3c100830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:28:08.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.792+0000 7f2a42000700 1 -- 172.21.15.117:0/775048182 shutdown_connections 2024-06-03T03:28:08.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:08.793+0000 7f2a42000700 1 -- 172.21.15.117:0/775048182 wait complete. 2024-06-03T03:28:09.387 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:28:09 smithi117 ceph-mon[31649]: pgmap v591: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 MiB/s wr, 27 op/s 2024-06-03T03:28:09.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:28:09 smithi151 ceph-mon[37043]: pgmap v591: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 MiB/s wr, 27 op/s 2024-06-03T03:28:09.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:28:09 smithi183 ceph-mon[37326]: pgmap v591: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 MiB/s wr, 27 op/s 2024-06-03T03:28:10.015 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-03T03:28:10.461 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:28:11.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:28:11 smithi151 ceph-mon[37043]: pgmap v592: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 22 MiB/s wr, 23 op/s 2024-06-03T03:28:11.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:28:11 smithi117 ceph-mon[31649]: pgmap v592: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 22 MiB/s wr, 23 op/s 2024-06-03T03:28:11.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:28:11 smithi183 ceph-mon[37326]: pgmap v592: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 22 MiB/s wr, 23 op/s 2024-06-03T03:28:12.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:12.990+0000 7fd85ac00700 1 -- 172.21.15.117:0/2858671179 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd854105050 msgr2=0x7fd854111560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:28:12.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:12.990+0000 7fd85ac00700 1 --2- 172.21.15.117:0/2858671179 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd854105050 0x7fd854111560 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd84c00b0a0 tx=0x7fd84c00b3b0 comp rx=0 tx=0).stop 2024-06-03T03:28:12.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:12.990+0000 7fd85ac00700 1 -- 172.21.15.117:0/2858671179 shutdown_connections 2024-06-03T03:28:12.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:12.990+0000 7fd85ac00700 1 --2- 172.21.15.117:0/2858671179 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd854105050 0x7fd854111560 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:12.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:12.990+0000 7fd85ac00700 1 --2- 172.21.15.117:0/2858671179 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd85410a690 0x7fd85410aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:12.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:12.990+0000 7fd85ac00700 1 --2- 172.21.15.117:0/2858671179 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd854104690 0x7fd854104b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:12.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:12.990+0000 7fd85ac00700 1 -- 172.21.15.117:0/2858671179 >> 172.21.15.117:0/2858671179 conn(0x7fd8541001a0 msgr2=0x7fd8541025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:28:12.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:12.990+0000 7fd85ac00700 1 -- 172.21.15.117:0/2858671179 shutdown_connections 2024-06-03T03:28:12.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:12.990+0000 7fd85ac00700 1 -- 172.21.15.117:0/2858671179 wait complete. 2024-06-03T03:28:12.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:12.992+0000 7fd85ac00700 1 Processor -- start 2024-06-03T03:28:12.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:12.992+0000 7fd85ac00700 1 -- start start 2024-06-03T03:28:12.994 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:12.993+0000 7fd85ac00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd854104690 0x7fd85419ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:12.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:12.993+0000 7fd85ac00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd854105050 0x7fd85419b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:12.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:12.993+0000 7fd85ac00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd85410a690 0x7fd85419f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:12.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:12.993+0000 7fd85ac00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd854114a70 con 0x7fd854105050 2024-06-03T03:28:12.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:12.993+0000 7fd85ac00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd8541148f0 con 0x7fd854104690 2024-06-03T03:28:12.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:12.993+0000 7fd85ac00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd854114bf0 con 0x7fd85410a690 2024-06-03T03:28:12.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:12.993+0000 7fd853400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd854105050 0x7fd85419b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:12.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:12.993+0000 7fd853e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd854104690 0x7fd85419ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:12.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:12.993+0000 7fd853400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd854105050 0x7fd85419b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:52342/0 (socket says 172.21.15.117:52342) 2024-06-03T03:28:12.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:12.993+0000 7fd853e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd854104690 0x7fd85419ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:57580/0 (socket says 172.21.15.117:57580) 2024-06-03T03:28:12.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:12.993+0000 7fd853400700 1 -- 172.21.15.117:0/1459163597 learned_addr learned my addr 172.21.15.117:0/1459163597 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:28:12.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:12.993+0000 7fd858c00700 1 --2- 172.21.15.117:0/1459163597 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd85410a690 0x7fd85419f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:12.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:12.993+0000 7fd853e00700 1 -- 172.21.15.117:0/1459163597 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd85410a690 msgr2=0x7fd85419f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:28:12.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:12.993+0000 7fd853e00700 1 --2- 172.21.15.117:0/1459163597 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd85410a690 0x7fd85419f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:12.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:12.993+0000 7fd853e00700 1 -- 172.21.15.117:0/1459163597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd854105050 msgr2=0x7fd85419b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:28:12.997 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:12.993+0000 7fd853e00700 1 --2- 172.21.15.117:0/1459163597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd854105050 0x7fd85419b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:12.997 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:12.994+0000 7fd853e00700 1 -- 172.21.15.117:0/1459163597 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd85419fe90 con 0x7fd854104690 2024-06-03T03:28:12.997 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:12.994+0000 7fd853e00700 1 --2- 172.21.15.117:0/1459163597 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd854104690 0x7fd85419ae80 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd84400e9f0 tx=0x7fd84400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:28:12.997 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:12.994+0000 7fd850c00700 1 -- 172.21.15.117:0/1459163597 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd844004220 con 0x7fd854104690 2024-06-03T03:28:12.997 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:12.994+0000 7fd850c00700 1 -- 172.21.15.117:0/1459163597 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd844004d10 con 0x7fd854104690 2024-06-03T03:28:12.997 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:12.994+0000 7fd85ac00700 1 -- 172.21.15.117:0/1459163597 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8541a0180 con 0x7fd854104690 2024-06-03T03:28:12.997 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:12.994+0000 7fd850c00700 1 -- 172.21.15.117:0/1459163597 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd844005020 con 0x7fd854104690 2024-06-03T03:28:12.997 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:12.994+0000 7fd85ac00700 1 -- 172.21.15.117:0/1459163597 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8541a7a00 con 0x7fd854104690 2024-06-03T03:28:12.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:12.998+0000 7fd850c00700 1 -- 172.21.15.117:0/1459163597 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fd844004750 con 0x7fd854104690 2024-06-03T03:28:13.000 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:12.999+0000 7fd86561fb80 1 Processor -- start 2024-06-03T03:28:13.000 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:12.999+0000 7fd86561fb80 1 -- start start 2024-06-03T03:28:13.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:12.999+0000 7fd850c00700 1 --2- 172.21.15.117:0/1459163597 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fd8380a2c20 0x7fd8380a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:13.002 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:12.999+0000 7fd86561fb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562cee831e40 0x562cee8b5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:13.002 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:12.999+0000 7fd86561fb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cee8b55b0 0x562cee8b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:13.002 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:12.999+0000 7fd86561fb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cee8c0300 0x562cee8c26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:13.002 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.000+0000 7fd86561fb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562cee7c4a10 con 0x562cee831e40 2024-06-03T03:28:13.002 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.000+0000 7fd86561fb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562cee807b00 con 0x562cee8b55b0 2024-06-03T03:28:13.002 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.000+0000 7fd86561fb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562cee7c28f0 con 0x562cee8c0300 2024-06-03T03:28:13.002 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.000+0000 7fd850c00700 1 -- 172.21.15.117:0/1459163597 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7fd8440cf270 con 0x7fd854104690 2024-06-03T03:28:13.002 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.000+0000 7fd853e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562cee831e40 0x562cee8b5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:13.002 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.000+0000 7fd853e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562cee831e40 0x562cee8b5070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:52348/0 (socket says 172.21.15.117:52348) 2024-06-03T03:28:13.002 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.000+0000 7fd853e00700 1 -- 172.21.15.117:0/763868774 learned_addr learned my addr 172.21.15.117:0/763868774 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:28:13.002 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.000+0000 7fd858c00700 1 --2- 172.21.15.117:0/1459163597 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fd8380a2c20 0x7fd8380a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:13.002 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.000+0000 7fd853400700 1 --2- 172.21.15.117:0/763868774 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cee8b55b0 0x562cee8b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:13.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.000+0000 7fd858c00700 1 --2- 172.21.15.117:0/763868774 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cee8c0300 0x562cee8c26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:13.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.000+0000 7fd853e00700 1 -- 172.21.15.117:0/763868774 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cee8c0300 msgr2=0x562cee8c26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:28:13.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.000+0000 7fd853e00700 1 --2- 172.21.15.117:0/763868774 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cee8c0300 0x562cee8c26f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:13.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.000+0000 7fd853e00700 1 -- 172.21.15.117:0/763868774 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cee8b55b0 msgr2=0x562cee8b5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:28:13.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.000+0000 7fd853e00700 1 --2- 172.21.15.117:0/763868774 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cee8b55b0 0x562cee8b5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:13.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.000+0000 7fd853e00700 1 -- 172.21.15.117:0/763868774 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cee7aff90 con 0x562cee831e40 2024-06-03T03:28:13.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.000+0000 7fd853400700 1 --2- 172.21.15.117:0/763868774 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cee8b55b0 0x562cee8b5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T03:28:13.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.001+0000 7fd853e00700 1 --2- 172.21.15.117:0/763868774 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562cee831e40 0x562cee8b5070 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd8440a3e20 tx=0x7fd84409b2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=315f955a2f11808 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:28:13.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.001+0000 7fd849600700 1 -- 172.21.15.117:0/763868774 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd84401c040 con 0x562cee831e40 2024-06-03T03:28:13.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.001+0000 7fd858c00700 1 --2- 172.21.15.117:0/1459163597 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fd8380a2c20 0x7fd8380a50e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd84c00b780 tx=0x7fd84c009d40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:28:13.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.001+0000 7fd849600700 1 -- 172.21.15.117:0/763868774 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd84409bde0 con 0x562cee831e40 2024-06-03T03:28:13.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.001+0000 7fd849600700 1 -- 172.21.15.117:0/763868774 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd84400c760 con 0x562cee831e40 2024-06-03T03:28:13.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.002+0000 7fd86561fb80 1 -- 172.21.15.117:0/763868774 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562cee831e40 msgr2=0x562cee8b5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:28:13.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.002+0000 7fd86561fb80 1 --2- 172.21.15.117:0/763868774 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562cee831e40 0x562cee8b5070 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd8440a3e20 tx=0x7fd84409b2a0 comp rx=0 tx=0).stop 2024-06-03T03:28:13.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.002+0000 7fd86561fb80 1 -- 172.21.15.117:0/763868774 shutdown_connections 2024-06-03T03:28:13.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.002+0000 7fd86561fb80 1 --2- 172.21.15.117:0/763868774 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cee8c0300 0x562cee8c26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:13.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.002+0000 7fd86561fb80 1 --2- 172.21.15.117:0/763868774 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562cee831e40 0x562cee8b5070 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:13.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.002+0000 7fd86561fb80 1 --2- 172.21.15.117:0/763868774 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cee8b55b0 0x562cee8b5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:13.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.002+0000 7fd86561fb80 1 -- 172.21.15.117:0/763868774 >> 172.21.15.117:0/763868774 conn(0x562cee7e2170 msgr2=0x562cee8b14b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:28:13.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.002+0000 7fd86561fb80 1 -- 172.21.15.117:0/763868774 shutdown_connections 2024-06-03T03:28:13.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.002+0000 7fd86561fb80 1 -- 172.21.15.117:0/763868774 wait complete. 2024-06-03T03:28:13.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.003+0000 7fd86561fb80 1 -- start start 2024-06-03T03:28:13.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.003+0000 7fd86561fb80 1 Processor -- start 2024-06-03T03:28:13.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.003+0000 7fd86561fb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cee831e40 0x562cee8bfa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:13.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.003+0000 7fd86561fb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cee8b55b0 0x562cee8bce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:13.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.003+0000 7fd86561fb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562cee8c0300 0x562cee8bd340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:13.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.003+0000 7fd86561fb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562cee7a3f60 con 0x562cee8c0300 2024-06-03T03:28:13.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.003+0000 7fd86561fb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562cee803ac0 con 0x562cee8b55b0 2024-06-03T03:28:13.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.003+0000 7fd86561fb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562cee8c2fc0 con 0x562cee831e40 2024-06-03T03:28:13.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.003+0000 7fd86561fb80 10 client.0 _ll_register_callbacks cb 0x562cee7a1720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:28:13.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.004+0000 7fd853400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cee8b55b0 0x562cee8bce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:13.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.004+0000 7fd858c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562cee8c0300 0x562cee8bd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:13.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.004+0000 7fd858c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562cee8c0300 0x562cee8bd340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:52364/0 (socket says 172.21.15.117:52364) 2024-06-03T03:28:13.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.004+0000 7fd858c00700 1 -- 172.21.15.117:0/3825390963 learned_addr learned my addr 172.21.15.117:0/3825390963 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:28:13.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.004+0000 7fd853e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cee831e40 0x562cee8bfa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:13.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.004+0000 7fd853e00700 1 -- 172.21.15.117:0/3825390963 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cee8b55b0 msgr2=0x562cee8bce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:28:13.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.004+0000 7fd853e00700 1 --2- 172.21.15.117:0/3825390963 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cee8b55b0 0x562cee8bce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:13.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.004+0000 7fd853e00700 1 -- 172.21.15.117:0/3825390963 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562cee8c0300 msgr2=0x562cee8bd340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:28:13.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.004+0000 7fd853e00700 1 --2- 172.21.15.117:0/3825390963 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562cee8c0300 0x562cee8bd340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:13.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.004+0000 7fd853e00700 1 -- 172.21.15.117:0/3825390963 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cee7568d0 con 0x562cee831e40 2024-06-03T03:28:13.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.004+0000 7fd853e00700 1 --2- 172.21.15.117:0/3825390963 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cee831e40 0x562cee8bfa90 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x562cee8b6000 tx=0x7fd84409b750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=602f0005da9a7cfe server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:28:13.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.004+0000 7fd83e000700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-06-03T03:28:13.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.005+0000 7fd83e000700 1 -- 172.21.15.117:0/3825390963 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd84401c070 con 0x562cee831e40 2024-06-03T03:28:13.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.005+0000 7fd83e000700 1 -- 172.21.15.117:0/3825390963 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7fd84401ede0 con 0x562cee831e40 2024-06-03T03:28:13.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.005+0000 7fd86561fb80 1 -- 172.21.15.117:0/3825390963 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd84409bde0 con 0x562cee831e40 2024-06-03T03:28:13.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.005+0000 7fd83e000700 1 -- 172.21.15.117:0/3825390963 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fd8440a4350 con 0x562cee831e40 2024-06-03T03:28:13.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.005+0000 7fd86561fb80 10 client.25843 fetch_fsmap learned FSMap version 31 2024-06-03T03:28:13.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.005+0000 7fd86561fb80 1 -- 172.21.15.117:0/3825390963 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562cee8cd780 con 0x562cee831e40 2024-06-03T03:28:13.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.005+0000 7fd83e000700 1 -- 172.21.15.117:0/3825390963 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd84409bde0 con 0x562cee831e40 2024-06-03T03:28:13.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.005+0000 7fd83e000700 1 -- 172.21.15.117:0/3825390963 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fd8440a3ec0 con 0x562cee831e40 2024-06-03T03:28:13.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.005+0000 7fd86561fb80 10 client.25843 fetch_fsmap finished waiting for FSMap version 31 2024-06-03T03:28:13.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.005+0000 7fd86561fb80 10 client.25843 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.a 2024-06-03T03:28:13.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.006+0000 7fd86561fb80 20 client.25843 populate_metadata read hostname 'smithi117' 2024-06-03T03:28:13.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.006+0000 7fd86561fb80 1 --2- 172.21.15.117:0/3825390963 >> [v2:172.21.15.117:6840/1212242967,v1:172.21.15.117:6841/1212242967] conn(0x562cee8cfc50 0x562cee8d2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:13.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.006+0000 7fd86561fb80 4 client.25843 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T03:28:13.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.006+0000 7fd86561fb80 1 -- 172.21.15.117:0/3825390963 --> [v2:172.21.15.117:6840/1212242967,v1:172.21.15.117:6841/1212242967] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562cee815000 con 0x562cee8cfc50 2024-06-03T03:28:13.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.006+0000 7fd853400700 1 --2- 172.21.15.117:0/3825390963 >> [v2:172.21.15.117:6840/1212242967,v1:172.21.15.117:6841/1212242967] conn(0x562cee8cfc50 0x562cee8d2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:13.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.007+0000 7fd853400700 1 --2- 172.21.15.117:0/3825390963 >> [v2:172.21.15.117:6840/1212242967,v1:172.21.15.117:6841/1212242967] conn(0x562cee8cfc50 0x562cee8d2040 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f9d3c9711750e00f server_cookie=e28a4d78c1a2c8d7 in_seq=0 out_seq=0 2024-06-03T03:28:13.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.007+0000 7fd83e000700 10 client.25843 ms_handle_connect on v2:172.21.15.117:6840/1212242967 2024-06-03T03:28:13.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.007+0000 7fd83e000700 1 -- 172.21.15.117:0/3825390963 <== mds.2 v2:172.21.15.117:6840/1212242967 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562cee815000 con 0x562cee8cfc50 2024-06-03T03:28:13.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.007+0000 7fd83e000700 10 client.25843 handle_command_reply: tid=0 2024-06-03T03:28:13.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.007+0000 7fd86561fb80 1 client.25843 shutdown 2024-06-03T03:28:13.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.008+0000 7fd86561fb80 1 -- 172.21.15.117:0/3825390963 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cee831e40 msgr2=0x562cee8bfa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:28:13.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.008+0000 7fd86561fb80 1 --2- 172.21.15.117:0/3825390963 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cee831e40 0x562cee8bfa90 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x562cee8b6000 tx=0x7fd84409b750 comp rx=0 tx=0).stop 2024-06-03T03:28:13.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.008+0000 7fd86561fb80 1 -- 172.21.15.117:0/3825390963 shutdown_connections 2024-06-03T03:28:13.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.008+0000 7fd86561fb80 1 --2- 172.21.15.117:0/3825390963 >> [v2:172.21.15.117:6840/1212242967,v1:172.21.15.117:6841/1212242967] conn(0x562cee8cfc50 0x562cee8d2040 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:13.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.008+0000 7fd86561fb80 1 --2- 172.21.15.117:0/3825390963 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562cee8c0300 0x562cee8bd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:13.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.008+0000 7fd86561fb80 1 --2- 172.21.15.117:0/3825390963 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cee831e40 0x562cee8bfa90 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:13.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.008+0000 7fd83e000700 0 client.25843 ms_handle_reset on v2:172.21.15.117:6840/1212242967 2024-06-03T03:28:13.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.008+0000 7fd86561fb80 1 --2- 172.21.15.117:0/3825390963 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cee8b55b0 0x562cee8bce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:13.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.008+0000 7fd86561fb80 1 -- 172.21.15.117:0/3825390963 >> 172.21.15.117:0/3825390963 conn(0x562cee7e2170 msgr2=0x562cee8b0740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:28:13.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.008+0000 7fd86561fb80 1 -- 172.21.15.117:0/3825390963 shutdown_connections 2024-06-03T03:28:13.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.008+0000 7fd86561fb80 1 -- 172.21.15.117:0/3825390963 wait complete. 2024-06-03T03:28:13.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.008+0000 7fd86561fb80 20 client.25843 trim_cache size 0 max 16384 2024-06-03T03:28:13.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.024+0000 7fd86561fb80 1 Processor -- start 2024-06-03T03:28:13.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.025+0000 7fd86561fb80 1 -- start start 2024-06-03T03:28:13.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.025+0000 7fd86561fb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cee831e40 0x562cee8b7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:13.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.025+0000 7fd86561fb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562cee8b55b0 0x562cee8b8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:13.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.025+0000 7fd86561fb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cee8c0300 0x562cee8b0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:13.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.025+0000 7fd86561fb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd8541b0550 con 0x562cee8b55b0 2024-06-03T03:28:13.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.025+0000 7fd86561fb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd8541b2090 con 0x562cee831e40 2024-06-03T03:28:13.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.025+0000 7fd86561fb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd8541b0c20 con 0x562cee8c0300 2024-06-03T03:28:13.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.025+0000 7fd853400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562cee8b55b0 0x562cee8b8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:13.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.025+0000 7fd853e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cee831e40 0x562cee8b7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:13.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.025+0000 7fd853400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562cee8b55b0 0x562cee8b8510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:52366/0 (socket says 172.21.15.117:52366) 2024-06-03T03:28:13.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.025+0000 7fd853400700 1 -- 172.21.15.117:0/3882489136 learned_addr learned my addr 172.21.15.117:0/3882489136 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:28:13.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.025+0000 7fd858c00700 1 --2- 172.21.15.117:0/3882489136 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cee8c0300 0x562cee8b0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:13.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.026+0000 7fd853400700 1 -- 172.21.15.117:0/3882489136 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cee8c0300 msgr2=0x562cee8b0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:28:13.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.026+0000 7fd853400700 1 --2- 172.21.15.117:0/3882489136 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cee8c0300 0x562cee8b0ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:13.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.026+0000 7fd853400700 1 -- 172.21.15.117:0/3882489136 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cee831e40 msgr2=0x562cee8b7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:28:13.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.026+0000 7fd853400700 1 --2- 172.21.15.117:0/3882489136 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cee831e40 0x562cee8b7fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:13.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.026+0000 7fd853400700 1 -- 172.21.15.117:0/3882489136 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cee8b61c0 con 0x562cee8b55b0 2024-06-03T03:28:13.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.026+0000 7fd853400700 1 --2- 172.21.15.117:0/3882489136 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562cee8b55b0 0x562cee8b8510 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd840005d90 tx=0x7fd840009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c7f02935b09f0ac server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:28:13.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.026+0000 7fd848c00700 1 -- 172.21.15.117:0/3882489136 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd840011dc0 con 0x562cee8b55b0 2024-06-03T03:28:13.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.026+0000 7fd848c00700 1 -- 172.21.15.117:0/3882489136 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd840004d10 con 0x562cee8b55b0 2024-06-03T03:28:13.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.026+0000 7fd848c00700 1 -- 172.21.15.117:0/3882489136 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd84001e680 con 0x562cee8b55b0 2024-06-03T03:28:13.030 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.027+0000 7fd86561fb80 1 -- 172.21.15.117:0/3882489136 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562cee8b55b0 msgr2=0x562cee8b8510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:28:13.030 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.027+0000 7fd86561fb80 1 --2- 172.21.15.117:0/3882489136 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562cee8b55b0 0x562cee8b8510 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd840005d90 tx=0x7fd840009260 comp rx=0 tx=0).stop 2024-06-03T03:28:13.030 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.027+0000 7fd86561fb80 1 -- 172.21.15.117:0/3882489136 shutdown_connections 2024-06-03T03:28:13.030 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.027+0000 7fd86561fb80 1 --2- 172.21.15.117:0/3882489136 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cee8c0300 0x562cee8b0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:13.030 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.027+0000 7fd86561fb80 1 --2- 172.21.15.117:0/3882489136 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cee831e40 0x562cee8b7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:13.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.027+0000 7fd86561fb80 1 --2- 172.21.15.117:0/3882489136 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562cee8b55b0 0x562cee8b8510 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:13.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.027+0000 7fd86561fb80 1 -- 172.21.15.117:0/3882489136 >> 172.21.15.117:0/3882489136 conn(0x562cee7e2170 msgr2=0x562cee8be630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:28:13.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.027+0000 7fd86561fb80 1 -- 172.21.15.117:0/3882489136 shutdown_connections 2024-06-03T03:28:13.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.027+0000 7fd86561fb80 1 -- 172.21.15.117:0/3882489136 wait complete. 2024-06-03T03:28:13.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.028+0000 7fd86561fb80 1 -- start start 2024-06-03T03:28:13.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.028+0000 7fd86561fb80 1 Processor -- start 2024-06-03T03:28:13.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.028+0000 7fd86561fb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cee831e40 0x562cee8d2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:13.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.028+0000 7fd86561fb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562cee8b55b0 0x562cee8d26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:13.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.028+0000 7fd86561fb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cee8c0300 0x562cee938580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:13.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.028+0000 7fd86561fb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd8541b19c0 con 0x562cee8b55b0 2024-06-03T03:28:13.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.028+0000 7fd86561fb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd8541b12f0 con 0x562cee8c0300 2024-06-03T03:28:13.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.028+0000 7fd86561fb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd8541ae4f0 con 0x562cee831e40 2024-06-03T03:28:13.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.029+0000 7fd86561fb80 10 client.0 _ll_register_callbacks cb 0x562cee7a1720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:28:13.033 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.029+0000 7fd853400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562cee8b55b0 0x562cee8d26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:13.033 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.029+0000 7fd853e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cee831e40 0x562cee8d2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:13.033 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.029+0000 7fd858c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cee8c0300 0x562cee938580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:13.033 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.029+0000 7fd853e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cee831e40 0x562cee8d2160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:42794/0 (socket says 172.21.15.117:42794) 2024-06-03T03:28:13.033 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.029+0000 7fd853400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562cee8b55b0 0x562cee8d26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:52374/0 (socket says 172.21.15.117:52374) 2024-06-03T03:28:13.033 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.029+0000 7fd858c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cee8c0300 0x562cee938580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:57626/0 (socket says 172.21.15.117:57626) 2024-06-03T03:28:13.034 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.029+0000 7fd853e00700 1 -- 172.21.15.117:0/3307359091 learned_addr learned my addr 172.21.15.117:0/3307359091 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:28:13.034 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.029+0000 7fd853e00700 1 -- 172.21.15.117:0/3307359091 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cee8c0300 msgr2=0x562cee938580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:28:13.034 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.029+0000 7fd853e00700 1 --2- 172.21.15.117:0/3307359091 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cee8c0300 0x562cee938580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:13.034 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.029+0000 7fd853e00700 1 -- 172.21.15.117:0/3307359091 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562cee8b55b0 msgr2=0x562cee8d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:28:13.034 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.029+0000 7fd853e00700 1 --2- 172.21.15.117:0/3307359091 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562cee8b55b0 0x562cee8d26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:13.034 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.029+0000 7fd853e00700 1 -- 172.21.15.117:0/3307359091 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cee8d2f60 con 0x562cee831e40 2024-06-03T03:28:13.034 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.030+0000 7fd853e00700 1 --2- 172.21.15.117:0/3307359091 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cee831e40 0x562cee8d2160 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd84400c5b0 tx=0x7fd84409b220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=463ccad8e960fc5f server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:28:13.034 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.030+0000 7fd83e000700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-06-03T03:28:13.034 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.030+0000 7fd83e000700 1 -- 172.21.15.117:0/3307359091 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd84401c070 con 0x562cee831e40 2024-06-03T03:28:13.034 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.030+0000 7fd83e000700 1 -- 172.21.15.117:0/3307359091 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7fd84401ede0 con 0x562cee831e40 2024-06-03T03:28:13.034 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.030+0000 7fd86561fb80 1 -- 172.21.15.117:0/3307359091 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd840004d10 con 0x562cee831e40 2024-06-03T03:28:13.034 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.030+0000 7fd83e000700 1 -- 172.21.15.117:0/3307359091 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fd84409b040 con 0x562cee831e40 2024-06-03T03:28:13.034 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.030+0000 7fd86561fb80 10 client.25849 fetch_fsmap learned FSMap version 31 2024-06-03T03:28:13.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.030+0000 7fd86561fb80 1 -- 172.21.15.117:0/3307359091 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562cee939bc0 con 0x562cee831e40 2024-06-03T03:28:13.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.030+0000 7fd83e000700 1 -- 172.21.15.117:0/3307359091 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd840004d10 con 0x562cee831e40 2024-06-03T03:28:13.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.031+0000 7fd83e000700 1 -- 172.21.15.117:0/3307359091 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fd84409ab30 con 0x562cee831e40 2024-06-03T03:28:13.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.031+0000 7fd86561fb80 10 client.25849 fetch_fsmap finished waiting for FSMap version 31 2024-06-03T03:28:13.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.031+0000 7fd86561fb80 10 client.25849 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.a 2024-06-03T03:28:13.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.031+0000 7fd86561fb80 20 client.25849 populate_metadata read hostname 'smithi117' 2024-06-03T03:28:13.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.031+0000 7fd86561fb80 1 --2- 172.21.15.117:0/3307359091 >> [v2:172.21.15.117:6840/1212242967,v1:172.21.15.117:6841/1212242967] conn(0x562cee939f30 0x562cee93c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:13.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.031+0000 7fd86561fb80 4 client.25849 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T03:28:13.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.031+0000 7fd86561fb80 1 -- 172.21.15.117:0/3307359091 --> [v2:172.21.15.117:6840/1212242967,v1:172.21.15.117:6841/1212242967] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562cee80f580 con 0x562cee939f30 2024-06-03T03:28:13.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.031+0000 7fd853400700 1 --2- 172.21.15.117:0/3307359091 >> [v2:172.21.15.117:6840/1212242967,v1:172.21.15.117:6841/1212242967] conn(0x562cee939f30 0x562cee93c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:13.036 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.032+0000 7fd853400700 1 --2- 172.21.15.117:0/3307359091 >> [v2:172.21.15.117:6840/1212242967,v1:172.21.15.117:6841/1212242967] conn(0x562cee939f30 0x562cee93c320 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5af1504145d41b51 server_cookie=838b258e3a72d785 in_seq=0 out_seq=0 2024-06-03T03:28:13.036 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.032+0000 7fd83e000700 10 client.25849 ms_handle_connect on v2:172.21.15.117:6840/1212242967 2024-06-03T03:28:13.036 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.032+0000 7fd83e000700 1 -- 172.21.15.117:0/3307359091 <== mds.2 v2:172.21.15.117:6840/1212242967 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562cee80f580 con 0x562cee939f30 2024-06-03T03:28:13.036 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.032+0000 7fd83e000700 10 client.25849 handle_command_reply: tid=0 2024-06-03T03:28:13.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.032+0000 7fd86561fb80 1 client.25849 shutdown 2024-06-03T03:28:13.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.032+0000 7fd86561fb80 1 -- 172.21.15.117:0/3307359091 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cee831e40 msgr2=0x562cee8d2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:28:13.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.032+0000 7fd86561fb80 1 --2- 172.21.15.117:0/3307359091 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cee831e40 0x562cee8d2160 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd84400c5b0 tx=0x7fd84409b220 comp rx=0 tx=0).stop 2024-06-03T03:28:13.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.032+0000 7fd86561fb80 1 -- 172.21.15.117:0/3307359091 shutdown_connections 2024-06-03T03:28:13.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.032+0000 7fd86561fb80 1 --2- 172.21.15.117:0/3307359091 >> [v2:172.21.15.117:6840/1212242967,v1:172.21.15.117:6841/1212242967] conn(0x562cee939f30 0x562cee93c320 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:13.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.032+0000 7fd86561fb80 1 --2- 172.21.15.117:0/3307359091 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cee8c0300 0x562cee938580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:13.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.032+0000 7fd86561fb80 1 --2- 172.21.15.117:0/3307359091 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cee831e40 0x562cee8d2160 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:13.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.032+0000 7fd83e000700 0 client.25849 ms_handle_reset on v2:172.21.15.117:6840/1212242967 2024-06-03T03:28:13.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.032+0000 7fd86561fb80 1 --2- 172.21.15.117:0/3307359091 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562cee8b55b0 0x562cee8d26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:13.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.032+0000 7fd86561fb80 1 -- 172.21.15.117:0/3307359091 >> 172.21.15.117:0/3307359091 conn(0x562cee7e2170 msgr2=0x562cee8beec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:28:13.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.033+0000 7fd86561fb80 1 -- 172.21.15.117:0/3307359091 shutdown_connections 2024-06-03T03:28:13.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.033+0000 7fd86561fb80 1 -- 172.21.15.117:0/3307359091 wait complete. 2024-06-03T03:28:13.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.033+0000 7fd86561fb80 20 client.25849 trim_cache size 0 max 16384 2024-06-03T03:28:13.038 INFO:teuthology.orchestra.run.smithi117.stdout:[] 2024-06-03T03:28:13.038 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.033+0000 7fd85ac00700 1 -- 172.21.15.117:0/1459163597 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fd8380a2c20 msgr2=0x7fd8380a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:28:13.038 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.034+0000 7fd85ac00700 1 --2- 172.21.15.117:0/1459163597 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fd8380a2c20 0x7fd8380a50e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd84c00b780 tx=0x7fd84c009d40 comp rx=0 tx=0).stop 2024-06-03T03:28:13.038 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.034+0000 7fd85ac00700 1 -- 172.21.15.117:0/1459163597 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd854104690 msgr2=0x7fd85419ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:28:13.038 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.034+0000 7fd85ac00700 1 --2- 172.21.15.117:0/1459163597 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd854104690 0x7fd85419ae80 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd84400e9f0 tx=0x7fd84400ed00 comp rx=0 tx=0).stop 2024-06-03T03:28:13.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.034+0000 7fd85ac00700 1 -- 172.21.15.117:0/1459163597 shutdown_connections 2024-06-03T03:28:13.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.034+0000 7fd85ac00700 1 --2- 172.21.15.117:0/1459163597 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fd8380a2c20 0x7fd8380a50e0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:13.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.034+0000 7fd85ac00700 1 --2- 172.21.15.117:0/1459163597 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd85410a690 0x7fd85419f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:13.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.034+0000 7fd85ac00700 1 --2- 172.21.15.117:0/1459163597 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd854104690 0x7fd85419ae80 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:13.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.034+0000 7fd85ac00700 1 --2- 172.21.15.117:0/1459163597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd854105050 0x7fd85419b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:13.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.034+0000 7fd85ac00700 1 -- 172.21.15.117:0/1459163597 >> 172.21.15.117:0/1459163597 conn(0x7fd8541001a0 msgr2=0x7fd8541006c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:28:13.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.034+0000 7fd85ac00700 1 -- 172.21.15.117:0/1459163597 shutdown_connections 2024-06-03T03:28:13.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:13.034+0000 7fd85ac00700 1 -- 172.21.15.117:0/1459163597 wait complete. 2024-06-03T03:28:13.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:28:13 smithi151 ceph-mon[37043]: pgmap v593: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 16 MiB/s wr, 23 op/s 2024-06-03T03:28:13.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:28:13 smithi117 ceph-mon[31649]: pgmap v593: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 16 MiB/s wr, 23 op/s 2024-06-03T03:28:13.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:28:13 smithi183 ceph-mon[37326]: pgmap v593: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 16 MiB/s wr, 23 op/s 2024-06-03T03:28:13.941 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-03T03:28:14.374 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:28:15.409 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:28:15 smithi151 ceph-mon[37043]: pgmap v594: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 6.7 MiB/s wr, 16 op/s 2024-06-03T03:28:15.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:28:15 smithi117 ceph-mon[31649]: pgmap v594: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 6.7 MiB/s wr, 16 op/s 2024-06-03T03:28:15.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:28:15 smithi183 ceph-mon[37326]: pgmap v594: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 6.7 MiB/s wr, 16 op/s 2024-06-03T03:28:17.065 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.063+0000 7f2b2f800700 1 -- 172.21.15.117:0/312231384 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b2810eac0 msgr2=0x7f2b2810eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:28:17.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.063+0000 7f2b2f800700 1 --2- 172.21.15.117:0/312231384 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b2810eac0 0x7f2b2810eea0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c009990 tx=0x7f2b1c009ca0 comp rx=0 tx=0).stop 2024-06-03T03:28:17.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.063+0000 7f2b2f800700 1 -- 172.21.15.117:0/312231384 shutdown_connections 2024-06-03T03:28:17.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.063+0000 7f2b2f800700 1 --2- 172.21.15.117:0/312231384 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b28109480 0x7f2b28115990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:17.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.063+0000 7f2b2f800700 1 --2- 172.21.15.117:0/312231384 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b2810eac0 0x7f2b2810eea0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:17.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.063+0000 7f2b2f800700 1 --2- 172.21.15.117:0/312231384 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2b28108ac0 0x7f2b28108f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:17.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.063+0000 7f2b2f800700 1 -- 172.21.15.117:0/312231384 >> 172.21.15.117:0/312231384 conn(0x7f2b28075c60 msgr2=0x7f2b281072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:28:17.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.064+0000 7f2b2f800700 1 -- 172.21.15.117:0/312231384 shutdown_connections 2024-06-03T03:28:17.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.064+0000 7f2b2f800700 1 -- 172.21.15.117:0/312231384 wait complete. 2024-06-03T03:28:17.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.065+0000 7f2b2f800700 1 Processor -- start 2024-06-03T03:28:17.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.065+0000 7f2b2f800700 1 -- start start 2024-06-03T03:28:17.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.066+0000 7f2b2f800700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b28108ac0 0x7f2b2819f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:17.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.066+0000 7f2b2f800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b28109480 0x7f2b2819f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:17.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.066+0000 7f2b2f800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2b2810eac0 0x7f2b281a3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:17.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.066+0000 7f2b2f800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2b28118e60 con 0x7f2b28109480 2024-06-03T03:28:17.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.066+0000 7f2b2f800700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2b28118ce0 con 0x7f2b2810eac0 2024-06-03T03:28:17.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.066+0000 7f2b2f800700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2b28118fe0 con 0x7f2b28108ac0 2024-06-03T03:28:17.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.066+0000 7f2b27e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b28109480 0x7f2b2819f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:17.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.066+0000 7f2b27e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b28109480 0x7f2b2819f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:52302/0 (socket says 172.21.15.117:52302) 2024-06-03T03:28:17.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.066+0000 7f2b27e00700 1 -- 172.21.15.117:0/2137842521 learned_addr learned my addr 172.21.15.117:0/2137842521 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:28:17.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.066+0000 7f2b2d800700 1 --2- 172.21.15.117:0/2137842521 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2b2810eac0 0x7f2b281a3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:17.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.066+0000 7f2b2ce00700 1 --2- 172.21.15.117:0/2137842521 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b28108ac0 0x7f2b2819f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:17.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.067+0000 7f2b27e00700 1 -- 172.21.15.117:0/2137842521 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b28108ac0 msgr2=0x7f2b2819f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:28:17.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.067+0000 7f2b27e00700 1 --2- 172.21.15.117:0/2137842521 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b28108ac0 0x7f2b2819f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:17.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.067+0000 7f2b27e00700 1 -- 172.21.15.117:0/2137842521 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2b2810eac0 msgr2=0x7f2b281a3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:28:17.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.067+0000 7f2b27e00700 1 --2- 172.21.15.117:0/2137842521 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2b2810eac0 0x7f2b281a3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:17.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.067+0000 7f2b27e00700 1 -- 172.21.15.117:0/2137842521 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b281a4290 con 0x7f2b28109480 2024-06-03T03:28:17.070 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.067+0000 7f2b27e00700 1 --2- 172.21.15.117:0/2137842521 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b28109480 0x7f2b2819f810 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f2b1400e9a0 tx=0x7f2b1400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:28:17.070 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.067+0000 7f2b25600700 1 -- 172.21.15.117:0/2137842521 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b1400ca10 con 0x7f2b28109480 2024-06-03T03:28:17.070 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.067+0000 7f2b25600700 1 -- 172.21.15.117:0/2137842521 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b1400cbb0 con 0x7f2b28109480 2024-06-03T03:28:17.070 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.067+0000 7f2b2f800700 1 -- 172.21.15.117:0/2137842521 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b281a4580 con 0x7f2b28109480 2024-06-03T03:28:17.070 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.067+0000 7f2b25600700 1 -- 172.21.15.117:0/2137842521 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b14018680 con 0x7f2b28109480 2024-06-03T03:28:17.070 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.067+0000 7f2b2f800700 1 -- 172.21.15.117:0/2137842521 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b281abe00 con 0x7f2b28109480 2024-06-03T03:28:17.072 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.071+0000 7f2b25600700 1 -- 172.21.15.117:0/2137842521 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f2b14010430 con 0x7f2b28109480 2024-06-03T03:28:17.074 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.072+0000 7f2b25600700 1 --2- 172.21.15.117:0/2137842521 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f2b100a2c20 0x7f2b100a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:17.074 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.072+0000 7f2b3a1f1b80 1 Processor -- start 2024-06-03T03:28:17.074 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.072+0000 7f2b25600700 1 -- 172.21.15.117:0/2137842521 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f2b140cf670 con 0x7f2b28109480 2024-06-03T03:28:17.074 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.072+0000 7f2b2ce00700 1 --2- 172.21.15.117:0/2137842521 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f2b100a2c20 0x7f2b100a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:17.074 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.072+0000 7f2b3a1f1b80 1 -- start start 2024-06-03T03:28:17.075 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.073+0000 7f2b3a1f1b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a6c61dbe40 0x55a6c625f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:17.075 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.073+0000 7f2b3a1f1b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6c625f5b0 0x55a6c625fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:17.075 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.073+0000 7f2b3a1f1b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6c626a300 0x55a6c626c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:17.075 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.073+0000 7f2b3a1f1b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a6c616ea10 con 0x55a6c626a300 2024-06-03T03:28:17.075 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.073+0000 7f2b3a1f1b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a6c61b1b00 con 0x55a6c61dbe40 2024-06-03T03:28:17.075 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.073+0000 7f2b3a1f1b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a6c616c8f0 con 0x55a6c625f5b0 2024-06-03T03:28:17.075 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.073+0000 7f2b2ce00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a6c61dbe40 0x55a6c625f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:17.075 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.073+0000 7f2b27e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6c625f5b0 0x55a6c625fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:17.075 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.073+0000 7f2b27e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6c625f5b0 0x55a6c625fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:34080/0 (socket says 172.21.15.117:34080) 2024-06-03T03:28:17.075 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.073+0000 7f2b2ce00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a6c61dbe40 0x55a6c625f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:42604/0 (socket says 172.21.15.117:42604) 2024-06-03T03:28:17.075 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.073+0000 7f2b27e00700 1 -- 172.21.15.117:0/1779047736 learned_addr learned my addr 172.21.15.117:0/1779047736 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:28:17.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.073+0000 7f2b2d800700 1 --2- 172.21.15.117:0/1779047736 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6c626a300 0x55a6c626c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:17.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.074+0000 7f2b2ce00700 1 --2- 172.21.15.117:0/2137842521 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f2b100a2c20 0x7f2b100a50e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c006010 tx=0x7f2b1c00b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:28:17.077 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.074+0000 7f2b27e00700 1 -- 172.21.15.117:0/1779047736 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a6c61dbe40 msgr2=0x55a6c625f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:28:17.077 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.074+0000 7f2b27e00700 1 --2- 172.21.15.117:0/1779047736 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a6c61dbe40 0x55a6c625f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:17.077 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.074+0000 7f2b27e00700 1 -- 172.21.15.117:0/1779047736 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6c626a300 msgr2=0x55a6c626c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:28:17.077 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.074+0000 7f2b27e00700 1 --2- 172.21.15.117:0/1779047736 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6c626a300 0x55a6c626c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:17.077 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.074+0000 7f2b27e00700 1 -- 172.21.15.117:0/1779047736 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6c6159f90 con 0x55a6c625f5b0 2024-06-03T03:28:17.077 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.074+0000 7f2b2ce00700 1 --2- 172.21.15.117:0/1779047736 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a6c61dbe40 0x55a6c625f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:28:17.077 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.074+0000 7f2b27e00700 1 --2- 172.21.15.117:0/1779047736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6c625f5b0 0x55a6c625fa30 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2b140a4220 tx=0x7f2b1409c1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7e64d0101fc8f1d server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:28:17.077 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.074+0000 7f2b1a000700 1 -- 172.21.15.117:0/1779047736 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b1409fe90 con 0x55a6c625f5b0 2024-06-03T03:28:17.077 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.074+0000 7f2b1a000700 1 -- 172.21.15.117:0/1779047736 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b140a4750 con 0x55a6c625f5b0 2024-06-03T03:28:17.078 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.074+0000 7f2b1a000700 1 -- 172.21.15.117:0/1779047736 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b140a4550 con 0x55a6c625f5b0 2024-06-03T03:28:17.078 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.075+0000 7f2b3a1f1b80 1 -- 172.21.15.117:0/1779047736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6c625f5b0 msgr2=0x55a6c625fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:28:17.078 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.075+0000 7f2b3a1f1b80 1 --2- 172.21.15.117:0/1779047736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6c625f5b0 0x55a6c625fa30 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2b140a4220 tx=0x7f2b1409c1e0 comp rx=0 tx=0).stop 2024-06-03T03:28:17.078 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.075+0000 7f2b3a1f1b80 1 -- 172.21.15.117:0/1779047736 shutdown_connections 2024-06-03T03:28:17.078 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.075+0000 7f2b3a1f1b80 1 --2- 172.21.15.117:0/1779047736 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6c626a300 0x55a6c626c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:17.078 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.075+0000 7f2b3a1f1b80 1 --2- 172.21.15.117:0/1779047736 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a6c61dbe40 0x55a6c625f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:17.078 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.075+0000 7f2b3a1f1b80 1 --2- 172.21.15.117:0/1779047736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6c625f5b0 0x55a6c625fa30 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:17.078 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.075+0000 7f2b3a1f1b80 1 -- 172.21.15.117:0/1779047736 >> 172.21.15.117:0/1779047736 conn(0x55a6c618c170 msgr2=0x55a6c625b4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:28:17.078 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.075+0000 7f2b3a1f1b80 1 -- 172.21.15.117:0/1779047736 shutdown_connections 2024-06-03T03:28:17.078 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.075+0000 7f2b3a1f1b80 1 -- 172.21.15.117:0/1779047736 wait complete. 2024-06-03T03:28:17.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.076+0000 7f2b3a1f1b80 1 -- start start 2024-06-03T03:28:17.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.076+0000 7f2b3a1f1b80 1 Processor -- start 2024-06-03T03:28:17.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.077+0000 7f2b3a1f1b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a6c61dbe40 0x55a6c6269aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:17.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.077+0000 7f2b3a1f1b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6c625f5b0 0x55a6c6266da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:17.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.077+0000 7f2b3a1f1b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6c626a300 0x55a6c62672e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:17.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.077+0000 7f2b3a1f1b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a6c614df60 con 0x55a6c626a300 2024-06-03T03:28:17.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.077+0000 7f2b3a1f1b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a6c61adac0 con 0x55a6c61dbe40 2024-06-03T03:28:17.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.077+0000 7f2b3a1f1b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a6c626cfc0 con 0x55a6c625f5b0 2024-06-03T03:28:17.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.077+0000 7f2b3a1f1b80 10 client.0 _ll_register_callbacks cb 0x55a6c614b720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:28:17.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.077+0000 7f2b2ce00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a6c61dbe40 0x55a6c6269aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:17.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.077+0000 7f2b2ce00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a6c61dbe40 0x55a6c6269aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:42606/0 (socket says 172.21.15.117:42606) 2024-06-03T03:28:17.081 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.077+0000 7f2b2ce00700 1 -- 172.21.15.117:0/915997525 learned_addr learned my addr 172.21.15.117:0/915997525 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:28:17.081 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.077+0000 7f2b27e00700 1 --2- 172.21.15.117:0/915997525 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6c625f5b0 0x55a6c6266da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:17.081 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.077+0000 7f2b2d800700 1 --2- 172.21.15.117:0/915997525 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6c626a300 0x55a6c62672e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:17.081 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.078+0000 7f2b2ce00700 1 -- 172.21.15.117:0/915997525 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6c625f5b0 msgr2=0x55a6c6266da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:28:17.081 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.078+0000 7f2b2ce00700 1 --2- 172.21.15.117:0/915997525 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6c625f5b0 0x55a6c6266da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:17.081 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.078+0000 7f2b2ce00700 1 -- 172.21.15.117:0/915997525 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6c626a300 msgr2=0x55a6c62672e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:28:17.081 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.078+0000 7f2b2ce00700 1 --2- 172.21.15.117:0/915997525 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6c626a300 0x55a6c62672e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:17.081 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.078+0000 7f2b2ce00700 1 -- 172.21.15.117:0/915997525 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6c61008d0 con 0x55a6c61dbe40 2024-06-03T03:28:17.081 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.078+0000 7f2b2d800700 1 --2- 172.21.15.117:0/915997525 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6c626a300 0x55a6c62672e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:28:17.081 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.078+0000 7f2b27e00700 1 --2- 172.21.15.117:0/915997525 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6c625f5b0 0x55a6c6266da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:28:17.081 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.078+0000 7f2b2ce00700 1 --2- 172.21.15.117:0/915997525 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a6c61dbe40 0x55a6c6269aa0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c000c00 tx=0x7f2b1c009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b047286d5e5f674 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:28:17.081 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.078+0000 7f2b0ea00700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-06-03T03:28:17.081 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.078+0000 7f2b0ea00700 1 -- 172.21.15.117:0/915997525 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b1c033020 con 0x55a6c61dbe40 2024-06-03T03:28:17.082 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.078+0000 7f2b3a1f1b80 1 -- 172.21.15.117:0/915997525 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b140a4750 con 0x55a6c61dbe40 2024-06-03T03:28:17.082 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.078+0000 7f2b0ea00700 1 -- 172.21.15.117:0/915997525 <== mon.2 v2:172.21.15.151:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f2b1c00f460 con 0x55a6c61dbe40 2024-06-03T03:28:17.083 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.079+0000 7f2b0ea00700 1 -- 172.21.15.117:0/915997525 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f2b1c03e050 con 0x55a6c61dbe40 2024-06-03T03:28:17.084 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.079+0000 7f2b3a1f1b80 10 client.25826 fetch_fsmap learned FSMap version 31 2024-06-03T03:28:17.084 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.079+0000 7f2b3a1f1b80 1 -- 172.21.15.117:0/915997525 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a6c6277760 con 0x55a6c61dbe40 2024-06-03T03:28:17.084 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.079+0000 7f2b0ea00700 1 -- 172.21.15.117:0/915997525 <== mon.2 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b140a4750 con 0x55a6c61dbe40 2024-06-03T03:28:17.084 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.079+0000 7f2b0ea00700 1 -- 172.21.15.117:0/915997525 <== mon.2 v2:172.21.15.151:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f2b1c02f700 con 0x55a6c61dbe40 2024-06-03T03:28:17.084 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.079+0000 7f2b3a1f1b80 10 client.25826 fetch_fsmap finished waiting for FSMap version 31 2024-06-03T03:28:17.084 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.079+0000 7f2b3a1f1b80 10 client.25826 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-06-03T03:28:17.084 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.079+0000 7f2b3a1f1b80 20 client.25826 populate_metadata read hostname 'smithi117' 2024-06-03T03:28:17.084 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.079+0000 7f2b3a1f1b80 1 --2- 172.21.15.117:0/915997525 >> [v2:172.21.15.151:6838/3623444616,v1:172.21.15.151:6839/3623444616] conn(0x55a6c6279cf0 0x55a6c627c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:17.084 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.079+0000 7f2b3a1f1b80 4 client.25826 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T03:28:17.084 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.079+0000 7f2b3a1f1b80 1 -- 172.21.15.117:0/915997525 --> [v2:172.21.15.151:6838/3623444616,v1:172.21.15.151:6839/3623444616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a6c61bf000 con 0x55a6c6279cf0 2024-06-03T03:28:17.084 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.080+0000 7f2b27e00700 1 --2- 172.21.15.117:0/915997525 >> [v2:172.21.15.151:6838/3623444616,v1:172.21.15.151:6839/3623444616] conn(0x55a6c6279cf0 0x55a6c627c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:17.084 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.080+0000 7f2b27e00700 1 --2- 172.21.15.117:0/915997525 >> [v2:172.21.15.151:6838/3623444616,v1:172.21.15.151:6839/3623444616] conn(0x55a6c6279cf0 0x55a6c627c160 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5f8664146210d402 server_cookie=e1b3bb0a36ade88a in_seq=0 out_seq=0 2024-06-03T03:28:17.084 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.080+0000 7f2b0ea00700 10 client.25826 ms_handle_connect on v2:172.21.15.151:6838/3623444616 2024-06-03T03:28:17.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.081+0000 7f2b0ea00700 1 -- 172.21.15.117:0/915997525 <== mds.1 v2:172.21.15.151:6838/3623444616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a6c61bf000 con 0x55a6c6279cf0 2024-06-03T03:28:17.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.081+0000 7f2b0ea00700 10 client.25826 handle_command_reply: tid=0 2024-06-03T03:28:17.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.081+0000 7f2b3a1f1b80 1 client.25826 shutdown 2024-06-03T03:28:17.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.081+0000 7f2b3a1f1b80 1 -- 172.21.15.117:0/915997525 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a6c61dbe40 msgr2=0x55a6c6269aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:28:17.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.081+0000 7f2b3a1f1b80 1 --2- 172.21.15.117:0/915997525 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a6c61dbe40 0x55a6c6269aa0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c000c00 tx=0x7f2b1c009310 comp rx=0 tx=0).stop 2024-06-03T03:28:17.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.081+0000 7f2b3a1f1b80 1 -- 172.21.15.117:0/915997525 shutdown_connections 2024-06-03T03:28:17.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.081+0000 7f2b3a1f1b80 1 --2- 172.21.15.117:0/915997525 >> [v2:172.21.15.151:6838/3623444616,v1:172.21.15.151:6839/3623444616] conn(0x55a6c6279cf0 0x55a6c627c160 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:17.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.081+0000 7f2b3a1f1b80 1 --2- 172.21.15.117:0/915997525 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6c626a300 0x55a6c62672e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:17.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.082+0000 7f2b0ea00700 0 client.25826 ms_handle_reset on v2:172.21.15.151:6838/3623444616 2024-06-03T03:28:17.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.082+0000 7f2b3a1f1b80 1 --2- 172.21.15.117:0/915997525 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a6c61dbe40 0x55a6c6269aa0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:17.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.082+0000 7f2b3a1f1b80 1 --2- 172.21.15.117:0/915997525 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6c625f5b0 0x55a6c6266da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:17.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.082+0000 7f2b3a1f1b80 1 -- 172.21.15.117:0/915997525 >> 172.21.15.117:0/915997525 conn(0x55a6c618c170 msgr2=0x55a6c625a740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:28:17.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.082+0000 7f2b3a1f1b80 1 -- 172.21.15.117:0/915997525 shutdown_connections 2024-06-03T03:28:17.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.082+0000 7f2b3a1f1b80 1 -- 172.21.15.117:0/915997525 wait complete. 2024-06-03T03:28:17.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.082+0000 7f2b3a1f1b80 20 client.25826 trim_cache size 0 max 16384 2024-06-03T03:28:17.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.101+0000 7f2b3a1f1b80 1 Processor -- start 2024-06-03T03:28:17.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.101+0000 7f2b3a1f1b80 1 -- start start 2024-06-03T03:28:17.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.101+0000 7f2b3a1f1b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6c625f5b0 0x55a6c625a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:17.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.101+0000 7f2b3a1f1b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6c626a300 0x55a6c62695e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:17.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.101+0000 7f2b3a1f1b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a6c625ab00 0x55a6c626b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:17.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.101+0000 7f2b3a1f1b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2b281b49a0 con 0x55a6c626a300 2024-06-03T03:28:17.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.101+0000 7f2b3a1f1b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2b281b64e0 con 0x55a6c625ab00 2024-06-03T03:28:17.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.101+0000 7f2b3a1f1b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2b281b5070 con 0x55a6c625f5b0 2024-06-03T03:28:17.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.102+0000 7f2b27e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6c626a300 0x55a6c62695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:17.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.102+0000 7f2b27e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6c626a300 0x55a6c62695e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:52334/0 (socket says 172.21.15.117:52334) 2024-06-03T03:28:17.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.102+0000 7f2b27e00700 1 -- 172.21.15.117:0/921277995 learned_addr learned my addr 172.21.15.117:0/921277995 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:28:17.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.102+0000 7f2b2d800700 1 --2- 172.21.15.117:0/921277995 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a6c625ab00 0x55a6c626b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:17.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.102+0000 7f2b2ce00700 1 --2- 172.21.15.117:0/921277995 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6c625f5b0 0x55a6c625a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:17.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.102+0000 7f2b27e00700 1 -- 172.21.15.117:0/921277995 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6c625f5b0 msgr2=0x55a6c625a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:28:17.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.102+0000 7f2b27e00700 1 --2- 172.21.15.117:0/921277995 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6c625f5b0 0x55a6c625a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:17.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.102+0000 7f2b27e00700 1 -- 172.21.15.117:0/921277995 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a6c625ab00 msgr2=0x55a6c626b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:28:17.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.102+0000 7f2b27e00700 1 --2- 172.21.15.117:0/921277995 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a6c625ab00 0x55a6c626b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:17.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.102+0000 7f2b27e00700 1 -- 172.21.15.117:0/921277995 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6c6267ca0 con 0x55a6c626a300 2024-06-03T03:28:17.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.102+0000 7f2b2d800700 1 --2- 172.21.15.117:0/921277995 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a6c625ab00 0x55a6c626b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:28:17.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.102+0000 7f2b27e00700 1 --2- 172.21.15.117:0/921277995 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6c626a300 0x55a6c62695e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2b1400cfd0 tx=0x7f2b1409bc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98d5d1f2183fee44 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:28:17.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.103+0000 7f2b19600700 1 -- 172.21.15.117:0/921277995 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b140a4750 con 0x55a6c626a300 2024-06-03T03:28:17.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.103+0000 7f2b19600700 1 -- 172.21.15.117:0/921277995 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b1407f070 con 0x55a6c626a300 2024-06-03T03:28:17.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.103+0000 7f2b19600700 1 -- 172.21.15.117:0/921277995 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b14010720 con 0x55a6c626a300 2024-06-03T03:28:17.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.103+0000 7f2b3a1f1b80 1 -- 172.21.15.117:0/921277995 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6c626a300 msgr2=0x55a6c62695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:28:17.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.103+0000 7f2b3a1f1b80 1 --2- 172.21.15.117:0/921277995 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6c626a300 0x55a6c62695e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2b1400cfd0 tx=0x7f2b1409bc60 comp rx=0 tx=0).stop 2024-06-03T03:28:17.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.103+0000 7f2b3a1f1b80 1 -- 172.21.15.117:0/921277995 shutdown_connections 2024-06-03T03:28:17.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.103+0000 7f2b3a1f1b80 1 --2- 172.21.15.117:0/921277995 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a6c625ab00 0x55a6c626b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:17.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.103+0000 7f2b3a1f1b80 1 --2- 172.21.15.117:0/921277995 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6c625f5b0 0x55a6c625a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:17.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.103+0000 7f2b3a1f1b80 1 --2- 172.21.15.117:0/921277995 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6c626a300 0x55a6c62695e0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:17.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.103+0000 7f2b3a1f1b80 1 -- 172.21.15.117:0/921277995 >> 172.21.15.117:0/921277995 conn(0x55a6c618c170 msgr2=0x55a6c6259eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:28:17.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.104+0000 7f2b3a1f1b80 1 -- 172.21.15.117:0/921277995 shutdown_connections 2024-06-03T03:28:17.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.104+0000 7f2b3a1f1b80 1 -- 172.21.15.117:0/921277995 wait complete. 2024-06-03T03:28:17.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.104+0000 7f2b3a1f1b80 1 -- start start 2024-06-03T03:28:17.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.104+0000 7f2b3a1f1b80 1 Processor -- start 2024-06-03T03:28:17.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.105+0000 7f2b3a1f1b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6c625ab00 0x55a6c62647b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:17.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.105+0000 7f2b3a1f1b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a6c625f5b0 0x55a6c6272250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:17.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.105+0000 7f2b3a1f1b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6c626a300 0x55a6c6272790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:17.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.105+0000 7f2b3a1f1b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2b281b5e10 con 0x55a6c626a300 2024-06-03T03:28:17.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.105+0000 7f2b3a1f1b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2b281b5740 con 0x55a6c625f5b0 2024-06-03T03:28:17.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.105+0000 7f2b3a1f1b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2b281b2940 con 0x55a6c625ab00 2024-06-03T03:28:17.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.105+0000 7f2b3a1f1b80 10 client.0 _ll_register_callbacks cb 0x55a6c614b720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:28:17.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.105+0000 7f2b2d800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6c626a300 0x55a6c6272790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:17.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.105+0000 7f2b2d800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6c626a300 0x55a6c6272790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:52350/0 (socket says 172.21.15.117:52350) 2024-06-03T03:28:17.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.105+0000 7f2b2ce00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6c625ab00 0x55a6c62647b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:17.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.105+0000 7f2b2d800700 1 -- 172.21.15.117:0/3623053999 learned_addr learned my addr 172.21.15.117:0/3623053999 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:28:17.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.105+0000 7f2b27e00700 1 --2- 172.21.15.117:0/3623053999 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a6c625f5b0 0x55a6c6272250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:17.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.106+0000 7f2b2ce00700 1 -- 172.21.15.117:0/3623053999 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a6c625f5b0 msgr2=0x55a6c6272250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:28:17.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.106+0000 7f2b2ce00700 1 --2- 172.21.15.117:0/3623053999 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a6c625f5b0 0x55a6c6272250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:17.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.106+0000 7f2b2ce00700 1 -- 172.21.15.117:0/3623053999 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6c626a300 msgr2=0x55a6c6272790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:28:17.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.106+0000 7f2b2ce00700 1 --2- 172.21.15.117:0/3623053999 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6c626a300 0x55a6c6272790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:17.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.106+0000 7f2b2ce00700 1 -- 172.21.15.117:0/3623053999 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6c6272dd0 con 0x55a6c625ab00 2024-06-03T03:28:17.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.106+0000 7f2b2d800700 1 --2- 172.21.15.117:0/3623053999 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6c626a300 0x55a6c6272790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:28:17.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.106+0000 7f2b2ce00700 1 --2- 172.21.15.117:0/3623053999 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6c625ab00 0x55a6c62647b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c005ac0 tx=0x7f2b1c005950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a404176181285c1d server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:28:17.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.106+0000 7f2b0ea00700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-06-03T03:28:17.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.106+0000 7f2b0ea00700 1 -- 172.21.15.117:0/3623053999 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b1c033040 con 0x55a6c625ab00 2024-06-03T03:28:17.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.106+0000 7f2b3a1f1b80 1 -- 172.21.15.117:0/3623053999 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b1407f070 con 0x55a6c625ab00 2024-06-03T03:28:17.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.106+0000 7f2b0ea00700 1 -- 172.21.15.117:0/3623053999 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f2b1c02f620 con 0x55a6c625ab00 2024-06-03T03:28:17.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.107+0000 7f2b0ea00700 1 -- 172.21.15.117:0/3623053999 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f2b1c02f820 con 0x55a6c625ab00 2024-06-03T03:28:17.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.107+0000 7f2b3a1f1b80 10 client.25867 fetch_fsmap learned FSMap version 31 2024-06-03T03:28:17.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.107+0000 7f2b3a1f1b80 1 -- 172.21.15.117:0/3623053999 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a6c62e4a60 con 0x55a6c625ab00 2024-06-03T03:28:17.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.107+0000 7f2b0ea00700 1 -- 172.21.15.117:0/3623053999 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b1407f070 con 0x55a6c625ab00 2024-06-03T03:28:17.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.107+0000 7f2b0ea00700 1 -- 172.21.15.117:0/3623053999 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f2b1c04c410 con 0x55a6c625ab00 2024-06-03T03:28:17.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.107+0000 7f2b3a1f1b80 10 client.25867 fetch_fsmap finished waiting for FSMap version 31 2024-06-03T03:28:17.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.107+0000 7f2b3a1f1b80 10 client.25867 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-06-03T03:28:17.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.107+0000 7f2b3a1f1b80 20 client.25867 populate_metadata read hostname 'smithi117' 2024-06-03T03:28:17.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.107+0000 7f2b3a1f1b80 1 --2- 172.21.15.117:0/3623053999 >> [v2:172.21.15.151:6838/3623444616,v1:172.21.15.151:6839/3623444616] conn(0x55a6c62e5000 0x55a6c62e73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:17.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.107+0000 7f2b3a1f1b80 4 client.25867 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T03:28:17.113 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.107+0000 7f2b3a1f1b80 1 -- 172.21.15.117:0/3623053999 --> [v2:172.21.15.151:6838/3623444616,v1:172.21.15.151:6839/3623444616] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a6c61b9580 con 0x55a6c62e5000 2024-06-03T03:28:17.113 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.108+0000 7f2b27e00700 1 --2- 172.21.15.117:0/3623053999 >> [v2:172.21.15.151:6838/3623444616,v1:172.21.15.151:6839/3623444616] conn(0x55a6c62e5000 0x55a6c62e73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:17.113 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.108+0000 7f2b27e00700 1 --2- 172.21.15.117:0/3623053999 >> [v2:172.21.15.151:6838/3623444616,v1:172.21.15.151:6839/3623444616] conn(0x55a6c62e5000 0x55a6c62e73f0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e30f0bf3d57ac9ed server_cookie=c5a72627aada9926 in_seq=0 out_seq=0 2024-06-03T03:28:17.113 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.108+0000 7f2b0ea00700 10 client.25867 ms_handle_connect on v2:172.21.15.151:6838/3623444616 2024-06-03T03:28:17.113 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.108+0000 7f2b0ea00700 1 -- 172.21.15.117:0/3623053999 <== mds.1 v2:172.21.15.151:6838/3623444616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a6c61b9580 con 0x55a6c62e5000 2024-06-03T03:28:17.113 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.108+0000 7f2b0ea00700 10 client.25867 handle_command_reply: tid=0 2024-06-03T03:28:17.113 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.108+0000 7f2b3a1f1b80 1 client.25867 shutdown 2024-06-03T03:28:17.113 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.108+0000 7f2b3a1f1b80 1 -- 172.21.15.117:0/3623053999 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6c625ab00 msgr2=0x55a6c62647b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:28:17.113 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.108+0000 7f2b3a1f1b80 1 --2- 172.21.15.117:0/3623053999 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6c625ab00 0x55a6c62647b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c005ac0 tx=0x7f2b1c005950 comp rx=0 tx=0).stop 2024-06-03T03:28:17.113 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.108+0000 7f2b3a1f1b80 1 -- 172.21.15.117:0/3623053999 shutdown_connections 2024-06-03T03:28:17.113 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.108+0000 7f2b3a1f1b80 1 --2- 172.21.15.117:0/3623053999 >> [v2:172.21.15.151:6838/3623444616,v1:172.21.15.151:6839/3623444616] conn(0x55a6c62e5000 0x55a6c62e73f0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:17.113 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.108+0000 7f2b3a1f1b80 1 --2- 172.21.15.117:0/3623053999 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6c626a300 0x55a6c6272790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:17.113 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.108+0000 7f2b3a1f1b80 1 --2- 172.21.15.117:0/3623053999 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6c625ab00 0x55a6c62647b0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:17.113 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.108+0000 7f2b3a1f1b80 1 --2- 172.21.15.117:0/3623053999 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a6c625f5b0 0x55a6c6272250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:17.114 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.108+0000 7f2b0ea00700 0 client.25867 ms_handle_reset on v2:172.21.15.151:6838/3623444616 2024-06-03T03:28:17.114 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.108+0000 7f2b3a1f1b80 1 -- 172.21.15.117:0/3623053999 >> 172.21.15.117:0/3623053999 conn(0x55a6c618c170 msgr2=0x55a6c6260fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:28:17.114 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.109+0000 7f2b3a1f1b80 1 -- 172.21.15.117:0/3623053999 shutdown_connections 2024-06-03T03:28:17.114 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.109+0000 7f2b3a1f1b80 1 -- 172.21.15.117:0/3623053999 wait complete. 2024-06-03T03:28:17.114 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.109+0000 7f2b3a1f1b80 20 client.25867 trim_cache size 0 max 16384 2024-06-03T03:28:17.114 INFO:teuthology.orchestra.run.smithi117.stdout:[] 2024-06-03T03:28:17.115 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.109+0000 7f2b2f800700 1 -- 172.21.15.117:0/2137842521 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f2b100a2c20 msgr2=0x7f2b100a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:28:17.115 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.109+0000 7f2b2f800700 1 --2- 172.21.15.117:0/2137842521 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f2b100a2c20 0x7f2b100a50e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c006010 tx=0x7f2b1c00b260 comp rx=0 tx=0).stop 2024-06-03T03:28:17.115 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.110+0000 7f2b2f800700 1 -- 172.21.15.117:0/2137842521 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b28109480 msgr2=0x7f2b2819f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:28:17.115 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.110+0000 7f2b2f800700 1 --2- 172.21.15.117:0/2137842521 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b28109480 0x7f2b2819f810 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f2b1400e9a0 tx=0x7f2b1400ecb0 comp rx=0 tx=0).stop 2024-06-03T03:28:17.115 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.110+0000 7f2b2f800700 1 -- 172.21.15.117:0/2137842521 shutdown_connections 2024-06-03T03:28:17.115 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.110+0000 7f2b2f800700 1 --2- 172.21.15.117:0/2137842521 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f2b100a2c20 0x7f2b100a50e0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:17.115 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.110+0000 7f2b2f800700 1 --2- 172.21.15.117:0/2137842521 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2b2810eac0 0x7f2b281a3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:17.115 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.110+0000 7f2b2f800700 1 --2- 172.21.15.117:0/2137842521 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b28108ac0 0x7f2b2819f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:17.115 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.110+0000 7f2b2f800700 1 --2- 172.21.15.117:0/2137842521 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b28109480 0x7f2b2819f810 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:17.116 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.110+0000 7f2b2f800700 1 -- 172.21.15.117:0/2137842521 >> 172.21.15.117:0/2137842521 conn(0x7f2b28075c60 msgr2=0x7f2b28076220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:28:17.116 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.110+0000 7f2b2f800700 1 -- 172.21.15.117:0/2137842521 shutdown_connections 2024-06-03T03:28:17.116 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:17.110+0000 7f2b2f800700 1 -- 172.21.15.117:0/2137842521 wait complete. 2024-06-03T03:28:17.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:28:17 smithi151 ceph-mon[37043]: pgmap v595: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.9 KiB/s wr, 12 op/s 2024-06-03T03:28:17.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:28:17 smithi117 ceph-mon[31649]: pgmap v595: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.9 KiB/s wr, 12 op/s 2024-06-03T03:28:17.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:28:17 smithi183 ceph-mon[37326]: pgmap v595: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.9 KiB/s wr, 12 op/s 2024-06-03T03:28:18.146 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-03T03:28:18.590 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:28:19.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:28:19 smithi151 ceph-mon[37043]: pgmap v596: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 1.9 KiB/s wr, 12 op/s 2024-06-03T03:28:19.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:28:19 smithi117 ceph-mon[31649]: pgmap v596: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 1.9 KiB/s wr, 12 op/s 2024-06-03T03:28:19.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:28:19 smithi183 ceph-mon[37326]: pgmap v596: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 1.9 KiB/s wr, 12 op/s 2024-06-03T03:28:21.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.202+0000 7fa6b2200700 1 -- 172.21.15.117:0/1291928686 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6ac104f80 msgr2=0x7fa6ac111490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:28:21.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.202+0000 7fa6b2200700 1 --2- 172.21.15.117:0/1291928686 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6ac104f80 0x7fa6ac111490 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa6a4009f90 tx=0x7fa6a400b330 comp rx=0 tx=0).stop 2024-06-03T03:28:21.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.203+0000 7fa6b2200700 1 -- 172.21.15.117:0/1291928686 shutdown_connections 2024-06-03T03:28:21.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.203+0000 7fa6b2200700 1 --2- 172.21.15.117:0/1291928686 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6ac104f80 0x7fa6ac111490 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:21.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.203+0000 7fa6b2200700 1 --2- 172.21.15.117:0/1291928686 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6ac10a650 0x7fa6ac10aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:21.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.203+0000 7fa6b2200700 1 --2- 172.21.15.117:0/1291928686 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6ac1045c0 0x7fa6ac104a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:21.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.203+0000 7fa6b2200700 1 -- 172.21.15.117:0/1291928686 >> 172.21.15.117:0/1291928686 conn(0x7fa6ac1000b0 msgr2=0x7fa6ac1024d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:28:21.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.203+0000 7fa6b2200700 1 -- 172.21.15.117:0/1291928686 shutdown_connections 2024-06-03T03:28:21.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.203+0000 7fa6b2200700 1 -- 172.21.15.117:0/1291928686 wait complete. 2024-06-03T03:28:21.206 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.204+0000 7fa6b2200700 1 Processor -- start 2024-06-03T03:28:21.206 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.204+0000 7fa6b2200700 1 -- start start 2024-06-03T03:28:21.206 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.205+0000 7fa6b2200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6ac1045c0 0x7fa6ac1a08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:21.206 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.205+0000 7fa6b2200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6ac104f80 0x7fa6ac1a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:21.206 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.205+0000 7fa6b2200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6ac10a650 0x7fa6ac19aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:21.206 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.205+0000 7fa6b2200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ac114910 con 0x7fa6ac10a650 2024-06-03T03:28:21.207 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.205+0000 7fa6b2200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ac114790 con 0x7fa6ac1045c0 2024-06-03T03:28:21.207 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.205+0000 7fa6b2200700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ac114a90 con 0x7fa6ac104f80 2024-06-03T03:28:21.207 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.205+0000 7fa6b0e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6ac1045c0 0x7fa6ac1a08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:21.207 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.205+0000 7fa6abe00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6ac104f80 0x7fa6ac1a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:21.207 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.205+0000 7fa6abe00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6ac104f80 0x7fa6ac1a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:34116/0 (socket says 172.21.15.117:34116) 2024-06-03T03:28:21.207 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.205+0000 7fa6abe00700 1 -- 172.21.15.117:0/1035049952 learned_addr learned my addr 172.21.15.117:0/1035049952 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:28:21.207 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.205+0000 7fa6b1800700 1 --2- 172.21.15.117:0/1035049952 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6ac10a650 0x7fa6ac19aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:21.207 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.206+0000 7fa6b0e00700 1 -- 172.21.15.117:0/1035049952 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6ac104f80 msgr2=0x7fa6ac1a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:28:21.207 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.206+0000 7fa6b0e00700 1 --2- 172.21.15.117:0/1035049952 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6ac104f80 0x7fa6ac1a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:21.207 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.206+0000 7fa6b0e00700 1 -- 172.21.15.117:0/1035049952 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6ac10a650 msgr2=0x7fa6ac19aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:28:21.207 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.206+0000 7fa6b0e00700 1 --2- 172.21.15.117:0/1035049952 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6ac10a650 0x7fa6ac19aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:21.207 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.206+0000 7fa6b0e00700 1 -- 172.21.15.117:0/1035049952 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6ac19b230 con 0x7fa6ac1045c0 2024-06-03T03:28:21.207 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.206+0000 7fa6abe00700 1 --2- 172.21.15.117:0/1035049952 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6ac104f80 0x7fa6ac1a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:28:21.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.206+0000 7fa6b0e00700 1 --2- 172.21.15.117:0/1035049952 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6ac1045c0 0x7fa6ac1a08c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa6a0005fd0 tx=0x7fa6a000ee70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:28:21.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.206+0000 7fa6a9600700 1 -- 172.21.15.117:0/1035049952 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a000cbd0 con 0x7fa6ac1045c0 2024-06-03T03:28:21.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.206+0000 7fa6a9600700 1 -- 172.21.15.117:0/1035049952 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6a000cd70 con 0x7fa6ac1045c0 2024-06-03T03:28:21.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.206+0000 7fa6b2200700 1 -- 172.21.15.117:0/1035049952 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6ac19b520 con 0x7fa6ac1045c0 2024-06-03T03:28:21.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.206+0000 7fa6a9600700 1 -- 172.21.15.117:0/1035049952 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a0018820 con 0x7fa6ac1045c0 2024-06-03T03:28:21.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.207+0000 7fa6b2200700 1 -- 172.21.15.117:0/1035049952 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6ac1a7790 con 0x7fa6ac1045c0 2024-06-03T03:28:21.210 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.210+0000 7fa6a9600700 1 -- 172.21.15.117:0/1035049952 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fa6a001f020 con 0x7fa6ac1045c0 2024-06-03T03:28:21.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.211+0000 7fa6a9600700 1 --2- 172.21.15.117:0/1035049952 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fa6940a2ca0 0x7fa6940a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:21.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.211+0000 7fa6be0adb80 1 Processor -- start 2024-06-03T03:28:21.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.211+0000 7fa6a9600700 1 -- 172.21.15.117:0/1035049952 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7fa6a00cf360 con 0x7fa6ac1045c0 2024-06-03T03:28:21.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.211+0000 7fa6abe00700 1 --2- 172.21.15.117:0/1035049952 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fa6940a2ca0 0x7fa6940a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:21.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.211+0000 7fa6be0adb80 1 -- start start 2024-06-03T03:28:21.214 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.212+0000 7fa6be0adb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d2b2dfd60 0x563d2b2e0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:21.214 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.212+0000 7fa6be0adb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d2b2e0680 0x563d2b2eaff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:21.214 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.212+0000 7fa6be0adb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d2b2eb530 0x563d2b2ed920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:21.214 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.212+0000 7fa6be0adb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563d2b1efa10 con 0x563d2b2eb530 2024-06-03T03:28:21.214 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.212+0000 7fa6be0adb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x563d2b232b00 con 0x563d2b2dfd60 2024-06-03T03:28:21.214 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.212+0000 7fa6be0adb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563d2b1ed8f0 con 0x563d2b2e0680 2024-06-03T03:28:21.214 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.212+0000 7fa6b0e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d2b2dfd60 0x563d2b2e0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:21.214 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.212+0000 7fa6b0e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d2b2dfd60 0x563d2b2e0140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:42664/0 (socket says 172.21.15.117:42664) 2024-06-03T03:28:21.214 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.212+0000 7fa6b0e00700 1 -- 172.21.15.117:0/785331510 learned_addr learned my addr 172.21.15.117:0/785331510 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:28:21.215 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.212+0000 7fa6b1800700 1 --2- 172.21.15.117:0/785331510 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d2b2eb530 0x563d2b2ed920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:21.215 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.213+0000 7fa6abe00700 1 --2- 172.21.15.117:0/1035049952 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fa6940a2ca0 0x7fa6940a5160 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac109c90 tx=0x7fa698009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:28:21.215 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.213+0000 7fa6b0e00700 1 -- 172.21.15.117:0/785331510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d2b2e0680 msgr2=0x563d2b2eaff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:28:21.215 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.213+0000 7fa6abe00700 1 --2- 172.21.15.117:0/785331510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d2b2e0680 0x563d2b2eaff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:21.215 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.213+0000 7fa6b0e00700 1 --2- 172.21.15.117:0/785331510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d2b2e0680 0x563d2b2eaff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:21.215 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.213+0000 7fa6b0e00700 1 -- 172.21.15.117:0/785331510 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d2b2eb530 msgr2=0x563d2b2ed920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:28:21.215 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.213+0000 7fa6b0e00700 1 --2- 172.21.15.117:0/785331510 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d2b2eb530 0x563d2b2ed920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:21.215 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.213+0000 7fa6b0e00700 1 -- 172.21.15.117:0/785331510 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d2b1daf90 con 0x563d2b2dfd60 2024-06-03T03:28:21.215 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.213+0000 7fa6b1800700 1 --2- 172.21.15.117:0/785331510 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d2b2eb530 0x563d2b2ed920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T03:28:21.215 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.213+0000 7fa6abe00700 1 --2- 172.21.15.117:0/785331510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d2b2e0680 0x563d2b2eaff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T03:28:21.215 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.213+0000 7fa6b0e00700 1 --2- 172.21.15.117:0/785331510 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d2b2dfd60 0x563d2b2e0140 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa6a00981a0 tx=0x7fa6a009b390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a467aa909241f519 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:28:21.215 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.213+0000 7fa69e000700 1 -- 172.21.15.117:0/785331510 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a00b4070 con 0x563d2b2dfd60 2024-06-03T03:28:21.215 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.213+0000 7fa69e000700 1 -- 172.21.15.117:0/785331510 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6a009bed0 con 0x563d2b2dfd60 2024-06-03T03:28:21.215 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.213+0000 7fa69e000700 1 -- 172.21.15.117:0/785331510 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a009ba30 con 0x563d2b2dfd60 2024-06-03T03:28:21.218 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.214+0000 7fa6be0adb80 1 -- 172.21.15.117:0/785331510 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d2b2dfd60 msgr2=0x563d2b2e0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:28:21.218 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.214+0000 7fa6be0adb80 1 --2- 172.21.15.117:0/785331510 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d2b2dfd60 0x563d2b2e0140 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa6a00981a0 tx=0x7fa6a009b390 comp rx=0 tx=0).stop 2024-06-03T03:28:21.218 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.214+0000 7fa6be0adb80 1 -- 172.21.15.117:0/785331510 shutdown_connections 2024-06-03T03:28:21.218 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.214+0000 7fa6be0adb80 1 --2- 172.21.15.117:0/785331510 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d2b2eb530 0x563d2b2ed920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:21.218 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.214+0000 7fa6be0adb80 1 --2- 172.21.15.117:0/785331510 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d2b2dfd60 0x563d2b2e0140 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:21.218 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.214+0000 7fa6be0adb80 1 --2- 172.21.15.117:0/785331510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d2b2e0680 0x563d2b2eaff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:21.218 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.214+0000 7fa6be0adb80 1 -- 172.21.15.117:0/785331510 >> 172.21.15.117:0/785331510 conn(0x563d2b20d170 msgr2=0x563d2b2dc280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:28:21.218 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.214+0000 7fa6be0adb80 1 -- 172.21.15.117:0/785331510 shutdown_connections 2024-06-03T03:28:21.218 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.214+0000 7fa6be0adb80 1 -- 172.21.15.117:0/785331510 wait complete. 2024-06-03T03:28:21.218 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.215+0000 7fa6be0adb80 1 -- start start 2024-06-03T03:28:21.218 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.215+0000 7fa6be0adb80 1 Processor -- start 2024-06-03T03:28:21.218 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.215+0000 7fa6be0adb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d2b2dfd60 0x563d2b2eadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:21.219 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.215+0000 7fa6be0adb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d2b2e0680 0x563d2b2e80b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:21.219 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.216+0000 7fa6be0adb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d2b2eb530 0x563d2b2e85f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:21.219 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.216+0000 7fa6be0adb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563d2b1cef60 con 0x563d2b2dfd60 2024-06-03T03:28:21.219 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.216+0000 7fa6be0adb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x563d2b22eac0 con 0x563d2b2e0680 2024-06-03T03:28:21.219 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.216+0000 7fa6be0adb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563d2b2edf60 con 0x563d2b2eb530 2024-06-03T03:28:21.219 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.216+0000 7fa6be0adb80 10 client.0 _ll_register_callbacks cb 0x563d2b1c8380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:28:21.219 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.216+0000 7fa6b0e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d2b2dfd60 0x563d2b2eadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:21.219 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.216+0000 7fa6b1800700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d2b2eb530 0x563d2b2e85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:21.219 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.216+0000 7fa6b0e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d2b2dfd60 0x563d2b2eadb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:52396/0 (socket says 172.21.15.117:52396) 2024-06-03T03:28:21.219 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.216+0000 7fa6b1800700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d2b2eb530 0x563d2b2e85f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:34134/0 (socket says 172.21.15.117:34134) 2024-06-03T03:28:21.219 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.216+0000 7fa6b0e00700 1 -- 172.21.15.117:0/2811876710 learned_addr learned my addr 172.21.15.117:0/2811876710 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:28:21.219 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.216+0000 7fa6abe00700 1 --2- 172.21.15.117:0/2811876710 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d2b2e0680 0x563d2b2e80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:21.219 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.216+0000 7fa6b1800700 1 -- 172.21.15.117:0/2811876710 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d2b2e0680 msgr2=0x563d2b2e80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:28:21.220 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.216+0000 7fa6b1800700 1 --2- 172.21.15.117:0/2811876710 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d2b2e0680 0x563d2b2e80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:21.220 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.216+0000 7fa6b1800700 1 -- 172.21.15.117:0/2811876710 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d2b2dfd60 msgr2=0x563d2b2eadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:28:21.220 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.216+0000 7fa6b1800700 1 --2- 172.21.15.117:0/2811876710 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d2b2dfd60 0x563d2b2eadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:21.220 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.216+0000 7fa6b1800700 1 -- 172.21.15.117:0/2811876710 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d2b1818d0 con 0x563d2b2eb530 2024-06-03T03:28:21.220 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.216+0000 7fa6abe00700 1 --2- 172.21.15.117:0/2811876710 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d2b2e0680 0x563d2b2e80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T03:28:21.220 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.217+0000 7fa6b1800700 1 --2- 172.21.15.117:0/2811876710 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d2b2eb530 0x563d2b2e85f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa6a4009f90 tx=0x7fa6a4003d20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2887b611299a512 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:28:21.220 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.217+0000 7fa692a00700 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-06-03T03:28:21.220 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.217+0000 7fa692a00700 1 -- 172.21.15.117:0/2811876710 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a400bbf0 con 0x563d2b2eb530 2024-06-03T03:28:21.220 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.217+0000 7fa6be0adb80 1 -- 172.21.15.117:0/2811876710 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6a009bed0 con 0x563d2b2eb530 2024-06-03T03:28:21.220 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.217+0000 7fa692a00700 1 -- 172.21.15.117:0/2811876710 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7fa6a400bd90 con 0x563d2b2eb530 2024-06-03T03:28:21.220 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.217+0000 7fa692a00700 1 -- 172.21.15.117:0/2811876710 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa6a402e050 con 0x563d2b2eb530 2024-06-03T03:28:21.220 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.217+0000 7fa6be0adb80 10 client.25876 fetch_fsmap learned FSMap version 31 2024-06-03T03:28:21.220 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.217+0000 7fa6be0adb80 1 -- 172.21.15.117:0/2811876710 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x563d2b2f8720 con 0x563d2b2eb530 2024-06-03T03:28:21.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.218+0000 7fa692a00700 1 -- 172.21.15.117:0/2811876710 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6a009bed0 con 0x563d2b2eb530 2024-06-03T03:28:21.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.218+0000 7fa692a00700 1 -- 172.21.15.117:0/2811876710 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fa6a4011070 con 0x563d2b2eb530 2024-06-03T03:28:21.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.218+0000 7fa6be0adb80 10 client.25876 fetch_fsmap finished waiting for FSMap version 31 2024-06-03T03:28:21.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.218+0000 7fa6be0adb80 10 client.25876 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-03T03:28:21.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.218+0000 7fa6be0adb80 20 client.25876 populate_metadata read hostname 'smithi117' 2024-06-03T03:28:21.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.218+0000 7fa6be0adb80 1 --2- 172.21.15.117:0/2811876710 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x563d2b2facb0 0x563d2b2fd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:21.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.218+0000 7fa6be0adb80 4 client.25876 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T03:28:21.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.218+0000 7fa6be0adb80 1 -- 172.21.15.117:0/2811876710 --> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d2b240000 con 0x563d2b2facb0 2024-06-03T03:28:21.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.218+0000 7fa6b0e00700 1 --2- 172.21.15.117:0/2811876710 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x563d2b2facb0 0x563d2b2fd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:21.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.219+0000 7fa6b0e00700 1 --2- 172.21.15.117:0/2811876710 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x563d2b2facb0 0x563d2b2fd0d0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d4cccbce4f99dfe server_cookie=8313d14687d3e3e0 in_seq=0 out_seq=0 2024-06-03T03:28:21.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.219+0000 7fa692a00700 10 client.25876 ms_handle_connect on v2:172.21.15.183:6838/1439615083 2024-06-03T03:28:21.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.220+0000 7fa692a00700 1 -- 172.21.15.117:0/2811876710 <== mds.0 v2:172.21.15.183:6838/1439615083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563d2b240000 con 0x563d2b2facb0 2024-06-03T03:28:21.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.220+0000 7fa692a00700 10 client.25876 handle_command_reply: tid=0 2024-06-03T03:28:21.224 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.220+0000 7fa6be0adb80 1 client.25876 shutdown 2024-06-03T03:28:21.224 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.220+0000 7fa6be0adb80 1 -- 172.21.15.117:0/2811876710 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d2b2eb530 msgr2=0x563d2b2e85f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:28:21.224 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.220+0000 7fa6be0adb80 1 --2- 172.21.15.117:0/2811876710 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d2b2eb530 0x563d2b2e85f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa6a4009f90 tx=0x7fa6a4003d20 comp rx=0 tx=0).stop 2024-06-03T03:28:21.224 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.220+0000 7fa6be0adb80 1 -- 172.21.15.117:0/2811876710 shutdown_connections 2024-06-03T03:28:21.224 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.220+0000 7fa6be0adb80 1 --2- 172.21.15.117:0/2811876710 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x563d2b2facb0 0x563d2b2fd0d0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:21.224 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.220+0000 7fa6be0adb80 1 --2- 172.21.15.117:0/2811876710 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d2b2eb530 0x563d2b2e85f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:21.224 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.220+0000 7fa6be0adb80 1 --2- 172.21.15.117:0/2811876710 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d2b2dfd60 0x563d2b2eadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:21.224 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.220+0000 7fa6be0adb80 1 --2- 172.21.15.117:0/2811876710 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d2b2e0680 0x563d2b2e80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:21.224 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.220+0000 7fa692a00700 0 client.25876 ms_handle_reset on v2:172.21.15.183:6838/1439615083 2024-06-03T03:28:21.224 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.220+0000 7fa6be0adb80 1 -- 172.21.15.117:0/2811876710 >> 172.21.15.117:0/2811876710 conn(0x563d2b20d170 msgr2=0x563d2b2dbe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:28:21.224 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.221+0000 7fa6be0adb80 1 -- 172.21.15.117:0/2811876710 shutdown_connections 2024-06-03T03:28:21.224 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.221+0000 7fa6be0adb80 1 -- 172.21.15.117:0/2811876710 wait complete. 2024-06-03T03:28:21.224 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.221+0000 7fa6be0adb80 20 client.25876 trim_cache size 0 max 16384 2024-06-03T03:28:21.238 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.237+0000 7fa6be0adb80 1 Processor -- start 2024-06-03T03:28:21.238 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.237+0000 7fa6be0adb80 1 -- start start 2024-06-03T03:28:21.238 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.237+0000 7fa6be0adb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d2b2dfd60 0x563d2b2f7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:21.238 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.237+0000 7fa6be0adb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d2b2e0680 0x563d2b2dbac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:21.238 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.238+0000 7fa6be0adb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d2b2eb530 0x563d2b3029d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:21.238 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.238+0000 7fa6be0adb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ac1b0350 con 0x563d2b2dfd60 2024-06-03T03:28:21.239 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.238+0000 7fa6be0adb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ac1b1e90 con 0x563d2b2e0680 2024-06-03T03:28:21.239 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.238+0000 7fa6be0adb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ac1b0a20 con 0x563d2b2eb530 2024-06-03T03:28:21.239 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.238+0000 7fa6b0e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d2b2dfd60 0x563d2b2f7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:21.240 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.238+0000 7fa6b0e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d2b2dfd60 0x563d2b2f7040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:52398/0 (socket says 172.21.15.117:52398) 2024-06-03T03:28:21.240 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.238+0000 7fa6b0e00700 1 -- 172.21.15.117:0/2454622679 learned_addr learned my addr 172.21.15.117:0/2454622679 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:28:21.240 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.238+0000 7fa6abe00700 1 --2- 172.21.15.117:0/2454622679 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d2b2e0680 0x563d2b2dbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:21.240 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.238+0000 7fa6b1800700 1 --2- 172.21.15.117:0/2454622679 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d2b2eb530 0x563d2b3029d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:21.240 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.238+0000 7fa6b1800700 1 -- 172.21.15.117:0/2454622679 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d2b2e0680 msgr2=0x563d2b2dbac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:28:21.240 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.238+0000 7fa6b1800700 1 --2- 172.21.15.117:0/2454622679 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d2b2e0680 0x563d2b2dbac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:21.240 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.238+0000 7fa6b1800700 1 -- 172.21.15.117:0/2454622679 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d2b2dfd60 msgr2=0x563d2b2f7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:28:21.240 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.238+0000 7fa6b1800700 1 --2- 172.21.15.117:0/2454622679 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d2b2dfd60 0x563d2b2f7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:21.240 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.238+0000 7fa6b1800700 1 -- 172.21.15.117:0/2454622679 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d2b2e8920 con 0x563d2b2eb530 2024-06-03T03:28:21.241 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.238+0000 7fa6b0e00700 1 --2- 172.21.15.117:0/2454622679 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d2b2dfd60 0x563d2b2f7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:28:21.241 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.239+0000 7fa6b1800700 1 --2- 172.21.15.117:0/2454622679 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d2b2eb530 0x563d2b3029d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa6a4005390 tx=0x7fa6a4005b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa0ecf725c73613b server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:28:21.241 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.239+0000 7fa69d600700 1 -- 172.21.15.117:0/2454622679 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a40056c0 con 0x563d2b2eb530 2024-06-03T03:28:21.241 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.239+0000 7fa69d600700 1 -- 172.21.15.117:0/2454622679 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6a400b9f0 con 0x563d2b2eb530 2024-06-03T03:28:21.241 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.239+0000 7fa69d600700 1 -- 172.21.15.117:0/2454622679 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a4025a10 con 0x563d2b2eb530 2024-06-03T03:28:21.241 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.240+0000 7fa6be0adb80 1 -- 172.21.15.117:0/2454622679 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d2b2eb530 msgr2=0x563d2b3029d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:28:21.241 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.240+0000 7fa6be0adb80 1 --2- 172.21.15.117:0/2454622679 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d2b2eb530 0x563d2b3029d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa6a4005390 tx=0x7fa6a4005b40 comp rx=0 tx=0).stop 2024-06-03T03:28:21.243 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.240+0000 7fa6be0adb80 1 -- 172.21.15.117:0/2454622679 shutdown_connections 2024-06-03T03:28:21.243 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.240+0000 7fa6be0adb80 1 --2- 172.21.15.117:0/2454622679 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d2b2eb530 0x563d2b3029d0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:21.244 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.240+0000 7fa6be0adb80 1 --2- 172.21.15.117:0/2454622679 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d2b2dfd60 0x563d2b2f7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:21.244 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.240+0000 7fa6be0adb80 1 --2- 172.21.15.117:0/2454622679 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d2b2e0680 0x563d2b2dbac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:21.244 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.240+0000 7fa6be0adb80 1 -- 172.21.15.117:0/2454622679 >> 172.21.15.117:0/2454622679 conn(0x563d2b20d170 msgr2=0x563d2b2db3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:28:21.244 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.240+0000 7fa6be0adb80 1 -- 172.21.15.117:0/2454622679 shutdown_connections 2024-06-03T03:28:21.244 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.240+0000 7fa6be0adb80 1 -- 172.21.15.117:0/2454622679 wait complete. 2024-06-03T03:28:21.244 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.240+0000 7fa6be0adb80 1 -- start start 2024-06-03T03:28:21.244 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.240+0000 7fa6be0adb80 1 Processor -- start 2024-06-03T03:28:21.244 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.241+0000 7fa6be0adb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d2b2dfd60 0x563d2b2f36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:21.244 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.241+0000 7fa6be0adb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d2b2e0680 0x563d2b2f0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:21.244 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.241+0000 7fa6be0adb80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d2b2eb530 0x563d2b2f0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:21.244 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.241+0000 7fa6be0adb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ac1b17c0 con 0x563d2b2e0680 2024-06-03T03:28:21.244 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.241+0000 7fa6be0adb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ac1b10f0 con 0x563d2b2dfd60 2024-06-03T03:28:21.245 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.241+0000 7fa6be0adb80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ac1ae2f0 con 0x563d2b2eb530 2024-06-03T03:28:21.245 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.241+0000 7fa6be0adb80 10 client.0 _ll_register_callbacks cb 0x563d2b1c8380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T03:28:21.245 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.241+0000 7fa6b0e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d2b2dfd60 0x563d2b2f36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:21.245 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.241+0000 7fa6abe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d2b2e0680 0x563d2b2f0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:21.245 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.241+0000 7fa6b0e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d2b2dfd60 0x563d2b2f36d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:42700/0 (socket says 172.21.15.117:42700) 2024-06-03T03:28:21.245 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.241+0000 7fa6b0e00700 1 -- 172.21.15.117:0/2026780832 learned_addr learned my addr 172.21.15.117:0/2026780832 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:28:21.245 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.241+0000 7fa6b1800700 1 --2- 172.21.15.117:0/2026780832 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d2b2eb530 0x563d2b2f0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:21.245 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.242+0000 7fa6b0e00700 1 -- 172.21.15.117:0/2026780832 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d2b2eb530 msgr2=0x563d2b2f0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:28:21.245 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.242+0000 7fa6b0e00700 1 --2- 172.21.15.117:0/2026780832 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d2b2eb530 0x563d2b2f0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:21.246 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.242+0000 7fa6b0e00700 1 -- 172.21.15.117:0/2026780832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d2b2e0680 msgr2=0x563d2b2f0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T03:28:21.246 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.242+0000 7fa6b0e00700 1 --2- 172.21.15.117:0/2026780832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d2b2e0680 0x563d2b2f0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:21.246 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.242+0000 7fa6b0e00700 1 -- 172.21.15.117:0/2026780832 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d2b2f15a0 con 0x563d2b2dfd60 2024-06-03T03:28:21.246 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.242+0000 7fa6b0e00700 1 --2- 172.21.15.117:0/2026780832 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d2b2dfd60 0x563d2b2f36d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa6a00973b0 tx=0x7fa6a00973e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e90af78e79dd3bdf server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:28:21.246 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.242+0000 7fa692a00700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-06-03T03:28:21.246 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.242+0000 7fa692a00700 1 -- 172.21.15.117:0/2026780832 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a00b4040 con 0x563d2b2dfd60 2024-06-03T03:28:21.246 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.242+0000 7fa6be0adb80 1 -- 172.21.15.117:0/2026780832 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6a400b9f0 con 0x563d2b2dfd60 2024-06-03T03:28:21.246 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.242+0000 7fa692a00700 1 -- 172.21.15.117:0/2026780832 <== mon.2 v2:172.21.15.151:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7fa6a0099ad0 con 0x563d2b2dfd60 2024-06-03T03:28:21.246 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.243+0000 7fa692a00700 1 -- 172.21.15.117:0/2026780832 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa6a009bed0 con 0x563d2b2dfd60 2024-06-03T03:28:21.246 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.243+0000 7fa6be0adb80 10 client.25850 fetch_fsmap learned FSMap version 31 2024-06-03T03:28:21.246 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.243+0000 7fa6be0adb80 1 -- 172.21.15.117:0/2026780832 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x563d2b3657b0 con 0x563d2b2dfd60 2024-06-03T03:28:21.247 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.243+0000 7fa692a00700 1 -- 172.21.15.117:0/2026780832 <== mon.2 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6a400b9f0 con 0x563d2b2dfd60 2024-06-03T03:28:21.247 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.243+0000 7fa692a00700 1 -- 172.21.15.117:0/2026780832 <== mon.2 v2:172.21.15.151:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fa6a009ba30 con 0x563d2b2dfd60 2024-06-03T03:28:21.247 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.243+0000 7fa6be0adb80 10 client.25850 fetch_fsmap finished waiting for FSMap version 31 2024-06-03T03:28:21.247 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.243+0000 7fa6be0adb80 10 client.25850 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-03T03:28:21.247 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.243+0000 7fa6be0adb80 20 client.25850 populate_metadata read hostname 'smithi117' 2024-06-03T03:28:21.247 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.243+0000 7fa6be0adb80 1 --2- 172.21.15.117:0/2026780832 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x563d2b365a40 0x563d2b367e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:21.247 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.243+0000 7fa6be0adb80 4 client.25850 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-03T03:28:21.247 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.243+0000 7fa6be0adb80 1 -- 172.21.15.117:0/2026780832 --> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563d2b23a580 con 0x563d2b365a40 2024-06-03T03:28:21.248 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.244+0000 7fa6abe00700 1 --2- 172.21.15.117:0/2026780832 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x563d2b365a40 0x563d2b367e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:21.248 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.244+0000 7fa6abe00700 1 --2- 172.21.15.117:0/2026780832 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x563d2b365a40 0x563d2b367e30 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9b8ae79b0aef14 server_cookie=2b5b140723940ecb in_seq=0 out_seq=0 2024-06-03T03:28:21.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.244+0000 7fa692a00700 10 client.25850 ms_handle_connect on v2:172.21.15.183:6838/1439615083 2024-06-03T03:28:21.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.244+0000 7fa692a00700 1 -- 172.21.15.117:0/2026780832 <== mds.0 v2:172.21.15.183:6838/1439615083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563d2b23a580 con 0x563d2b365a40 2024-06-03T03:28:21.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.244+0000 7fa692a00700 10 client.25850 handle_command_reply: tid=0 2024-06-03T03:28:21.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.244+0000 7fa6be0adb80 1 client.25850 shutdown 2024-06-03T03:28:21.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.245+0000 7fa6be0adb80 1 -- 172.21.15.117:0/2026780832 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d2b2dfd60 msgr2=0x563d2b2f36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:28:21.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.245+0000 7fa6be0adb80 1 --2- 172.21.15.117:0/2026780832 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d2b2dfd60 0x563d2b2f36d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa6a00973b0 tx=0x7fa6a00973e0 comp rx=0 tx=0).stop 2024-06-03T03:28:21.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.245+0000 7fa6be0adb80 1 -- 172.21.15.117:0/2026780832 shutdown_connections 2024-06-03T03:28:21.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.245+0000 7fa6be0adb80 1 --2- 172.21.15.117:0/2026780832 >> [v2:172.21.15.183:6838/1439615083,v1:172.21.15.183:6839/1439615083] conn(0x563d2b365a40 0x563d2b367e30 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:21.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.245+0000 7fa6be0adb80 1 --2- 172.21.15.117:0/2026780832 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d2b2eb530 0x563d2b2f0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:21.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.245+0000 7fa692a00700 0 client.25850 ms_handle_reset on v2:172.21.15.183:6838/1439615083 2024-06-03T03:28:21.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.245+0000 7fa6be0adb80 1 --2- 172.21.15.117:0/2026780832 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d2b2dfd60 0x563d2b2f36d0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:21.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.245+0000 7fa6be0adb80 1 --2- 172.21.15.117:0/2026780832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d2b2e0680 0x563d2b2f0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:21.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.245+0000 7fa6be0adb80 1 -- 172.21.15.117:0/2026780832 >> 172.21.15.117:0/2026780832 conn(0x563d2b20d170 msgr2=0x563d2b2dbc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:28:21.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.245+0000 7fa6be0adb80 1 -- 172.21.15.117:0/2026780832 shutdown_connections 2024-06-03T03:28:21.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.245+0000 7fa6be0adb80 1 -- 172.21.15.117:0/2026780832 wait complete. 2024-06-03T03:28:21.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.245+0000 7fa6be0adb80 20 client.25850 trim_cache size 0 max 16384 2024-06-03T03:28:21.250 INFO:teuthology.orchestra.run.smithi117.stdout:[] 2024-06-03T03:28:21.251 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.246+0000 7fa6b2200700 1 -- 172.21.15.117:0/1035049952 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fa6940a2ca0 msgr2=0x7fa6940a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:28:21.251 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.246+0000 7fa6b2200700 1 --2- 172.21.15.117:0/1035049952 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fa6940a2ca0 0x7fa6940a5160 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac109c90 tx=0x7fa698009260 comp rx=0 tx=0).stop 2024-06-03T03:28:21.251 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.246+0000 7fa6b2200700 1 -- 172.21.15.117:0/1035049952 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6ac1045c0 msgr2=0x7fa6ac1a08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:28:21.251 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.246+0000 7fa6b2200700 1 --2- 172.21.15.117:0/1035049952 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6ac1045c0 0x7fa6ac1a08c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa6a0005fd0 tx=0x7fa6a000ee70 comp rx=0 tx=0).stop 2024-06-03T03:28:21.251 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.246+0000 7fa6b2200700 1 -- 172.21.15.117:0/1035049952 shutdown_connections 2024-06-03T03:28:21.251 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.246+0000 7fa6b2200700 1 --2- 172.21.15.117:0/1035049952 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fa6940a2ca0 0x7fa6940a5160 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:21.252 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.246+0000 7fa6b2200700 1 --2- 172.21.15.117:0/1035049952 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6ac10a650 0x7fa6ac19aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:21.252 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.246+0000 7fa6b2200700 1 --2- 172.21.15.117:0/1035049952 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6ac1045c0 0x7fa6ac1a08c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:21.252 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.246+0000 7fa6b2200700 1 --2- 172.21.15.117:0/1035049952 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6ac104f80 0x7fa6ac1a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:21.252 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.246+0000 7fa6b2200700 1 -- 172.21.15.117:0/1035049952 >> 172.21.15.117:0/1035049952 conn(0x7fa6ac1000b0 msgr2=0x7fa6ac101690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:28:21.252 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.246+0000 7fa6b2200700 1 -- 172.21.15.117:0/1035049952 shutdown_connections 2024-06-03T03:28:21.252 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:21.246+0000 7fa6b2200700 1 -- 172.21.15.117:0/1035049952 wait complete. 2024-06-03T03:28:21.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:28:21 smithi151 ceph-mon[37043]: pgmap v597: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 852 B/s wr, 9 op/s 2024-06-03T03:28:21.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:28:21 smithi117 ceph-mon[31649]: pgmap v597: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 852 B/s wr, 9 op/s 2024-06-03T03:28:21.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:28:21 smithi183 ceph-mon[37326]: pgmap v597: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 852 B/s wr, 9 op/s 2024-06-03T03:28:22.534 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph_dfcc4b3c2c55532c4c04fa47551c9df1dffdf746/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph_dfcc4b3c2c55532c4c04fa47551c9df1dffdf746/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph_dfcc4b3c2c55532c4c04fa47551c9df1dffdf746/qa/tasks/thrasher.py", line 47, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-06-03T03:28:22.535 INFO:tasks.fwd_scrub:done joining 2024-06-03T03:28:22.535 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-06-03T03:28:22.552 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-06-03T03:28:22.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:22.758+0000 7f9fab600640 1 -- 172.21.15.117:0/265484830 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9fa4108000 msgr2=0x7f9fa4108420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:28:22.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:22.758+0000 7f9fab600640 1 --2- 172.21.15.117:0/265484830 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9fa4108000 0x7f9fa4108420 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f9f94009900 tx=0x7f9f9402a3f0 comp rx=0 tx=0).stop 2024-06-03T03:28:22.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:22.758+0000 7f9fab600640 1 -- 172.21.15.117:0/265484830 shutdown_connections 2024-06-03T03:28:22.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:22.758+0000 7f9fab600640 1 --2- 172.21.15.117:0/265484830 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9fa4109bb0 0x7f9fa4110440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:22.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:22.758+0000 7f9fab600640 1 --2- 172.21.15.117:0/265484830 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9fa41091f0 0x7f9fa4109670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:22.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:22.758+0000 7f9fab600640 1 --2- 172.21.15.117:0/265484830 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9fa4108000 0x7f9fa4108420 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:22.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:22.758+0000 7f9fab600640 1 -- 172.21.15.117:0/265484830 >> 172.21.15.117:0/265484830 conn(0x7f9fa4076fc0 msgr2=0x7f9fa40793e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:28:22.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:22.758+0000 7f9fab600640 1 -- 172.21.15.117:0/265484830 shutdown_connections 2024-06-03T03:28:22.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:22.758+0000 7f9fab600640 1 -- 172.21.15.117:0/265484830 wait complete. 2024-06-03T03:28:22.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:22.759+0000 7f9fab600640 1 Processor -- start 2024-06-03T03:28:22.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:22.759+0000 7f9fab600640 1 -- start start 2024-06-03T03:28:22.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:22.760+0000 7f9fab600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9fa4108000 0x7f9fa419e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:22.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:22.760+0000 7f9fab600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9fa41091f0 0x7f9fa419e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:22.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:22.760+0000 7f9fab600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9fa4109bb0 0x7f9fa41a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:22.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:22.760+0000 7f9fab600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa4113ca0 con 0x7f9fa4108000 2024-06-03T03:28:22.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:22.760+0000 7f9fab600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa4113b20 con 0x7f9fa41091f0 2024-06-03T03:28:22.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:22.760+0000 7f9fab600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa4113e20 con 0x7f9fa4109bb0 2024-06-03T03:28:22.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:22.760+0000 7f9fa8c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9fa4108000 0x7f9fa419e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:22.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:22.760+0000 7f9f9be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9fa41091f0 0x7f9fa419e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:22.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:22.760+0000 7f9fa8c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9fa4108000 0x7f9fa419e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:52438/0 (socket says 172.21.15.117:52438) 2024-06-03T03:28:22.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:22.760+0000 7f9f9be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9fa41091f0 0x7f9fa419e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:42712/0 (socket says 172.21.15.117:42712) 2024-06-03T03:28:22.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:22.760+0000 7f9fa8c00640 1 -- 172.21.15.117:0/2981576844 learned_addr learned my addr 172.21.15.117:0/2981576844 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:28:22.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:22.760+0000 7f9fa9600640 1 --2- 172.21.15.117:0/2981576844 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9fa4109bb0 0x7f9fa41a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:22.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:22.761+0000 7f9fa8c00640 1 -- 172.21.15.117:0/2981576844 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9fa4109bb0 msgr2=0x7f9fa41a58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:28:22.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:22.761+0000 7f9fa8c00640 1 --2- 172.21.15.117:0/2981576844 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9fa4109bb0 0x7f9fa41a58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:22.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:22.761+0000 7f9fa8c00640 1 -- 172.21.15.117:0/2981576844 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9fa41091f0 msgr2=0x7f9fa419e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:28:22.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:22.761+0000 7f9fa8c00640 1 --2- 172.21.15.117:0/2981576844 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9fa41091f0 0x7f9fa419e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:22.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:22.761+0000 7f9fa8c00640 1 -- 172.21.15.117:0/2981576844 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fa41a5f60 con 0x7f9fa4108000 2024-06-03T03:28:22.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:22.761+0000 7f9f9be00640 1 --2- 172.21.15.117:0/2981576844 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9fa41091f0 0x7f9fa419e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:28:22.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:22.761+0000 7f9fa8c00640 1 --2- 172.21.15.117:0/2981576844 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9fa4108000 0x7f9fa419e340 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9f94031440 tx=0x7f9f94032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:28:22.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:22.761+0000 7f9fa2000640 1 -- 172.21.15.117:0/2981576844 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f9403f030 con 0x7f9fa4108000 2024-06-03T03:28:22.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:22.761+0000 7f9fa2000640 1 -- 172.21.15.117:0/2981576844 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f94003ce0 con 0x7f9fa4108000 2024-06-03T03:28:22.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:22.761+0000 7f9fab600640 1 -- 172.21.15.117:0/2981576844 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fa41a61f0 con 0x7f9fa4108000 2024-06-03T03:28:22.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:22.761+0000 7f9fa2000640 1 -- 172.21.15.117:0/2981576844 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f94035070 con 0x7f9fa4108000 2024-06-03T03:28:22.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:22.761+0000 7f9fab600640 1 -- 172.21.15.117:0/2981576844 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fa41a66e0 con 0x7f9fa4108000 2024-06-03T03:28:22.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:22.762+0000 7f9fab600640 1 -- 172.21.15.117:0/2981576844 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fa410f320 con 0x7f9fa4108000 2024-06-03T03:28:22.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:22.764+0000 7f9fa2000640 1 -- 172.21.15.117:0/2981576844 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f9f94042030 con 0x7f9fa4108000 2024-06-03T03:28:22.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:22.766+0000 7f9fa2000640 1 --2- 172.21.15.117:0/2981576844 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f9f880a2b90 0x7f9f880a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:22.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:22.766+0000 7f9fa2000640 1 -- 172.21.15.117:0/2981576844 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f9f940ef900 con 0x7f9fa4108000 2024-06-03T03:28:22.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:22.766+0000 7f9f9be00640 1 --2- 172.21.15.117:0/2981576844 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f9f880a2b90 0x7f9f880a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:22.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:22.767+0000 7f9f9be00640 1 --2- 172.21.15.117:0/2981576844 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f9f880a2b90 0x7f9f880a5050 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c0095f0 tx=0x7f9f8c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:28:22.771 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:22.771+0000 7f9fa2000640 1 -- 172.21.15.117:0/2981576844 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9f940bc470 con 0x7f9fa4108000 2024-06-03T03:28:22.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:22.974+0000 7f9fab600640 1 -- 172.21.15.117:0/2981576844 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9fa40625a0 con 0x7f9fa4108000 2024-06-03T03:28:22.977 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:22.977+0000 7f9fa2000640 1 -- 172.21.15.117:0/2981576844 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+24807 (secure 0 0 0) 0x7f9f940bbd70 con 0x7f9fa4108000 2024-06-03T03:28:22.978 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T03:28:22.978 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":99,"fsid":"a0e8473a-2156-11ef-bc9b-c7b262605968","created":"2024-06-03T03:10:08.446859+0000","modified":"2024-06-03T03:21:25.966405+0000","last_up_change":"2024-06-03T03:18:28.736904+0000","last_in_change":"2024-06-03T03:18:08.485711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-03T03:14:09.370986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-03T03:19:35.044683+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-03T03:19:40.097569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-03T03:19:55.394480+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8fefa089-9401-4a1c-b9fb-fe30c63e6a1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":4286644492},{"type":"v1","addr":"172.21.15.117:6803","nonce":4286644492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":4286644492},{"type":"v1","addr":"172.21.15.117:6805","nonce":4286644492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":4286644492},{"type":"v1","addr":"172.21.15.117:6809","nonce":4286644492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":4286644492},{"type":"v1","addr":"172.21.15.117:6807","nonce":4286644492}]},"public_addr":"172.21.15.117:6803/4286644492","cluster_addr":"172.21.15.117:6805/4286644492","heartbeat_back_addr":"172.21.15.117:6809/4286644492","heartbeat_front_addr":"172.21.15.117:6807/4286644492","state":["exists","up"]},{"osd":1,"uuid":"5b12b371-f155-4b67-a470-25cbe59a2999","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":262491158},{"type":"v1","addr":"172.21.15.151:6801","nonce":262491158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":262491158},{"type":"v1","addr":"172.21.15.151:6803","nonce":262491158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":262491158},{"type":"v1","addr":"172.21.15.151:6807","nonce":262491158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":262491158},{"type":"v1","addr":"172.21.15.151:6805","nonce":262491158}]},"public_addr":"172.21.15.151:6801/262491158","cluster_addr":"172.21.15.151:6803/262491158","heartbeat_back_addr":"172.21.15.151:6807/262491158","heartbeat_front_addr":"172.21.15.151:6805/262491158","state":["exists","up"]},{"osd":2,"uuid":"dbc17818-c54c-4ecf-b656-03578b95d8c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":3542453623},{"type":"v1","addr":"172.21.15.183:6801","nonce":3542453623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":3542453623},{"type":"v1","addr":"172.21.15.183:6803","nonce":3542453623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":3542453623},{"type":"v1","addr":"172.21.15.183:6807","nonce":3542453623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":3542453623},{"type":"v1","addr":"172.21.15.183:6805","nonce":3542453623}]},"public_addr":"172.21.15.183:6801/3542453623","cluster_addr":"172.21.15.183:6803/3542453623","heartbeat_back_addr":"172.21.15.183:6807/3542453623","heartbeat_front_addr":"172.21.15.183:6805/3542453623","state":["exists","up"]},{"osd":3,"uuid":"ca08d2ac-b1d6-4a49-9ae8-e3ec146899a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":589012315},{"type":"v1","addr":"172.21.15.117:6811","nonce":589012315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":589012315},{"type":"v1","addr":"172.21.15.117:6813","nonce":589012315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":589012315},{"type":"v1","addr":"172.21.15.117:6817","nonce":589012315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":589012315},{"type":"v1","addr":"172.21.15.117:6815","nonce":589012315}]},"public_addr":"172.21.15.117:6811/589012315","cluster_addr":"172.21.15.117:6813/589012315","heartbeat_back_addr":"172.21.15.117:6817/589012315","heartbeat_front_addr":"172.21.15.117:6815/589012315","state":["exists","up"]},{"osd":4,"uuid":"810d6c7a-023f-4930-82af-bc2bbfd647da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":1239048647},{"type":"v1","addr":"172.21.15.151:6809","nonce":1239048647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":1239048647},{"type":"v1","addr":"172.21.15.151:6811","nonce":1239048647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":1239048647},{"type":"v1","addr":"172.21.15.151:6815","nonce":1239048647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":1239048647},{"type":"v1","addr":"172.21.15.151:6813","nonce":1239048647}]},"public_addr":"172.21.15.151:6809/1239048647","cluster_addr":"172.21.15.151:6811/1239048647","heartbeat_back_addr":"172.21.15.151:6815/1239048647","heartbeat_front_addr":"172.21.15.151:6813/1239048647","state":["exists","up"]},{"osd":5,"uuid":"8ab6a5f0-34bb-48d8-9471-a7dc836e486e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":2985874686},{"type":"v1","addr":"172.21.15.183:6809","nonce":2985874686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":2985874686},{"type":"v1","addr":"172.21.15.183:6811","nonce":2985874686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":2985874686},{"type":"v1","addr":"172.21.15.183:6815","nonce":2985874686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":2985874686},{"type":"v1","addr":"172.21.15.183:6813","nonce":2985874686}]},"public_addr":"172.21.15.183:6809/2985874686","cluster_addr":"172.21.15.183:6811/2985874686","heartbeat_back_addr":"172.21.15.183:6815/2985874686","heartbeat_front_addr":"172.21.15.183:6813/2985874686","state":["exists","up"]},{"osd":6,"uuid":"78d15739-e9e0-4f19-ae5f-ee8527a5b527","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4250583193},{"type":"v1","addr":"172.21.15.117:6819","nonce":4250583193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":4250583193},{"type":"v1","addr":"172.21.15.117:6821","nonce":4250583193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":4250583193},{"type":"v1","addr":"172.21.15.117:6825","nonce":4250583193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4250583193},{"type":"v1","addr":"172.21.15.117:6823","nonce":4250583193}]},"public_addr":"172.21.15.117:6819/4250583193","cluster_addr":"172.21.15.117:6821/4250583193","heartbeat_back_addr":"172.21.15.117:6825/4250583193","heartbeat_front_addr":"172.21.15.117:6823/4250583193","state":["exists","up"]},{"osd":7,"uuid":"3724fe1f-f4d7-443b-aea8-ff3ab02bfdc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":386094645},{"type":"v1","addr":"172.21.15.151:6817","nonce":386094645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":386094645},{"type":"v1","addr":"172.21.15.151:6819","nonce":386094645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":386094645},{"type":"v1","addr":"172.21.15.151:6823","nonce":386094645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":386094645},{"type":"v1","addr":"172.21.15.151:6821","nonce":386094645}]},"public_addr":"172.21.15.151:6817/386094645","cluster_addr":"172.21.15.151:6819/386094645","heartbeat_back_addr":"172.21.15.151:6823/386094645","heartbeat_front_addr":"172.21.15.151:6821/386094645","state":["exists","up"]},{"osd":8,"uuid":"006ac092-f015-448e-a0c0-f7b0c4597a18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1829748222},{"type":"v1","addr":"172.21.15.183:6817","nonce":1829748222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1829748222},{"type":"v1","addr":"172.21.15.183:6819","nonce":1829748222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1829748222},{"type":"v1","addr":"172.21.15.183:6823","nonce":1829748222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1829748222},{"type":"v1","addr":"172.21.15.183:6821","nonce":1829748222}]},"public_addr":"172.21.15.183:6817/1829748222","cluster_addr":"172.21.15.183:6819/1829748222","heartbeat_back_addr":"172.21.15.183:6823/1829748222","heartbeat_front_addr":"172.21.15.183:6821/1829748222","state":["exists","up"]},{"osd":9,"uuid":"00a970e6-2502-459e-975c-badac93c9bd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2538315141},{"type":"v1","addr":"172.21.15.117:6827","nonce":2538315141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2538315141},{"type":"v1","addr":"172.21.15.117:6829","nonce":2538315141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2538315141},{"type":"v1","addr":"172.21.15.117:6833","nonce":2538315141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2538315141},{"type":"v1","addr":"172.21.15.117:6831","nonce":2538315141}]},"public_addr":"172.21.15.117:6827/2538315141","cluster_addr":"172.21.15.117:6829/2538315141","heartbeat_back_addr":"172.21.15.117:6833/2538315141","heartbeat_front_addr":"172.21.15.117:6831/2538315141","state":["exists","up"]},{"osd":10,"uuid":"ba2333cd-32a3-41ef-9ea5-287c707f512b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":3801045133},{"type":"v1","addr":"172.21.15.151:6825","nonce":3801045133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3801045133},{"type":"v1","addr":"172.21.15.151:6827","nonce":3801045133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3801045133},{"type":"v1","addr":"172.21.15.151:6831","nonce":3801045133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3801045133},{"type":"v1","addr":"172.21.15.151:6829","nonce":3801045133}]},"public_addr":"172.21.15.151:6825/3801045133","cluster_addr":"172.21.15.151:6827/3801045133","heartbeat_back_addr":"172.21.15.151:6831/3801045133","heartbeat_front_addr":"172.21.15.151:6829/3801045133","state":["exists","up"]},{"osd":11,"uuid":"1556d654-e6fa-42c1-b399-aa53f6837a1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3513899696},{"type":"v1","addr":"172.21.15.183:6825","nonce":3513899696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3513899696},{"type":"v1","addr":"172.21.15.183:6827","nonce":3513899696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3513899696},{"type":"v1","addr":"172.21.15.183:6831","nonce":3513899696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3513899696},{"type":"v1","addr":"172.21.15.183:6829","nonce":3513899696}]},"public_addr":"172.21.15.183:6825/3513899696","cluster_addr":"172.21.15.183:6827/3513899696","heartbeat_back_addr":"172.21.15.183:6831/3513899696","heartbeat_front_addr":"172.21.15.183:6829/3513899696","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:13:17.174041+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:13:42.237001+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:14:06.273315+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:14:39.912564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:15:06.063202+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:15:30.424818+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:16:05.623554+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:16:32.926393+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:16:58.165899+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:17:32.897477+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:18:00.720866+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:18:26.474441+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.117:0/186309008":"2024-06-04T03:11:24.655516+0000","172.21.15.117:6801/2142313590":"2024-06-04T03:11:24.655516+0000","172.21.15.117:6800/2142313590":"2024-06-04T03:11:24.655516+0000","172.21.15.117:0/2399725156":"2024-06-04T03:11:24.655516+0000","172.21.15.117:0/1453232061":"2024-06-04T03:10:50.552899+0000","172.21.15.117:0/4281674974":"2024-06-04T03:10:50.552899+0000","172.21.15.117:6800/2424554985":"2024-06-04T03:10:50.552899+0000","172.21.15.117:0/3946365225":"2024-06-04T03:11:24.655516+0000","172.21.15.117:0/1146162354":"2024-06-04T03:10:50.552899+0000","172.21.15.117:6801/2424554985":"2024-06-04T03:10:50.552899+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-03T03:28:22.980 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:22.980+0000 7f9fab600640 1 -- 172.21.15.117:0/2981576844 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f9f880a2b90 msgr2=0x7f9f880a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:28:22.981 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:22.980+0000 7f9fab600640 1 --2- 172.21.15.117:0/2981576844 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f9f880a2b90 0x7f9f880a5050 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c0095f0 tx=0x7f9f8c005f90 comp rx=0 tx=0).stop 2024-06-03T03:28:22.981 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:22.980+0000 7f9fab600640 1 -- 172.21.15.117:0/2981576844 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9fa4108000 msgr2=0x7f9fa419e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:28:22.981 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:22.980+0000 7f9fab600640 1 --2- 172.21.15.117:0/2981576844 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9fa4108000 0x7f9fa419e340 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9f94031440 tx=0x7f9f94032ce0 comp rx=0 tx=0).stop 2024-06-03T03:28:22.981 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:22.980+0000 7f9fab600640 1 -- 172.21.15.117:0/2981576844 shutdown_connections 2024-06-03T03:28:22.981 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:22.980+0000 7f9fab600640 1 --2- 172.21.15.117:0/2981576844 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9fa4109bb0 0x7f9fa41a58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:22.982 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:22.980+0000 7f9fab600640 1 --2- 172.21.15.117:0/2981576844 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f9f880a2b90 0x7f9f880a5050 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:22.982 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:22.980+0000 7f9fab600640 1 --2- 172.21.15.117:0/2981576844 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9fa41091f0 0x7f9fa419e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:22.982 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:22.980+0000 7f9fab600640 1 --2- 172.21.15.117:0/2981576844 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9fa4108000 0x7f9fa419e340 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:22.982 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:22.980+0000 7f9fab600640 1 -- 172.21.15.117:0/2981576844 >> 172.21.15.117:0/2981576844 conn(0x7f9fa4076fc0 msgr2=0x7f9fa410c4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:28:22.982 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:22.980+0000 7f9fab600640 1 -- 172.21.15.117:0/2981576844 shutdown_connections 2024-06-03T03:28:22.983 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:22.980+0000 7f9fab600640 1 -- 172.21.15.117:0/2981576844 wait complete. 2024-06-03T03:28:23.001 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-06-03T03:14:09.370986+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-06-03T03:19:35.044683+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-06-03T03:19:40.097569+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-06-03T03:19:55.394480+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-06-03T03:28:23.002 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-06-03T03:28:23.207 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.207+0000 7f89fbe00640 1 -- 172.21.15.117:0/4055349735 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89fc107e90 msgr2=0x7f89fc1082b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:28:23.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.207+0000 7f89fbe00640 1 --2- 172.21.15.117:0/4055349735 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89fc107e90 0x7f89fc1082b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f89ec009900 tx=0x7f89ec02a3f0 comp rx=0 tx=0).stop 2024-06-03T03:28:23.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.207+0000 7f89fbe00640 1 -- 172.21.15.117:0/4055349735 shutdown_connections 2024-06-03T03:28:23.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.207+0000 7f89fbe00640 1 --2- 172.21.15.117:0/4055349735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89fc109a30 0x7f89fc110350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:23.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.207+0000 7f89fbe00640 1 --2- 172.21.15.117:0/4055349735 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89fc109070 0x7f89fc1094f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:23.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.207+0000 7f89fbe00640 1 --2- 172.21.15.117:0/4055349735 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89fc107e90 0x7f89fc1082b0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:23.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.207+0000 7f89fbe00640 1 -- 172.21.15.117:0/4055349735 >> 172.21.15.117:0/4055349735 conn(0x7f89fc076e70 msgr2=0x7f89fc0792b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:28:23.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.207+0000 7f89fbe00640 1 -- 172.21.15.117:0/4055349735 shutdown_connections 2024-06-03T03:28:23.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.207+0000 7f89fbe00640 1 -- 172.21.15.117:0/4055349735 wait complete. 2024-06-03T03:28:23.210 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.208+0000 7f89fbe00640 1 Processor -- start 2024-06-03T03:28:23.210 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.208+0000 7f89fbe00640 1 -- start start 2024-06-03T03:28:23.210 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.208+0000 7f89fbe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89fc107e90 0x7f89fc19e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:23.210 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.208+0000 7f89fbe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89fc109070 0x7f89fc19e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:23.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.208+0000 7f89fbe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89fc109a30 0x7f89fc1a5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:23.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.208+0000 7f89fbe00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f89fc113b60 con 0x7f89fc109a30 2024-06-03T03:28:23.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.208+0000 7f89fbe00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f89fc1139e0 con 0x7f89fc107e90 2024-06-03T03:28:23.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.208+0000 7f89fbe00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f89fc113ce0 con 0x7f89fc109070 2024-06-03T03:28:23.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.209+0000 7f89fb400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89fc109a30 0x7f89fc1a5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:23.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.209+0000 7f89faa00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89fc107e90 0x7f89fc19e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:23.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.209+0000 7f89faa00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89fc107e90 0x7f89fc19e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:42730/0 (socket says 172.21.15.117:42730) 2024-06-03T03:28:23.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.209+0000 7f89faa00640 1 -- 172.21.15.117:0/2230231484 learned_addr learned my addr 172.21.15.117:0/2230231484 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:28:23.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.209+0000 7f89f2000640 1 --2- 172.21.15.117:0/2230231484 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89fc109070 0x7f89fc19e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:23.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.209+0000 7f89fb400640 1 -- 172.21.15.117:0/2230231484 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89fc109070 msgr2=0x7f89fc19e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:28:23.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.209+0000 7f89fb400640 1 --2- 172.21.15.117:0/2230231484 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89fc109070 0x7f89fc19e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:23.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.209+0000 7f89fb400640 1 -- 172.21.15.117:0/2230231484 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89fc107e90 msgr2=0x7f89fc19e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:28:23.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.209+0000 7f89fb400640 1 --2- 172.21.15.117:0/2230231484 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89fc107e90 0x7f89fc19e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:23.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.209+0000 7f89fb400640 1 -- 172.21.15.117:0/2230231484 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89fc1a5dc0 con 0x7f89fc109a30 2024-06-03T03:28:23.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.209+0000 7f89f2000640 1 --2- 172.21.15.117:0/2230231484 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89fc109070 0x7f89fc19e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T03:28:23.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.209+0000 7f89fb400640 1 --2- 172.21.15.117:0/2230231484 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89fc109a30 0x7f89fc1a5750 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f89f4009fd0 tx=0x7f89f400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:28:23.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.209+0000 7f89f3e00640 1 -- 172.21.15.117:0/2230231484 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f400ce20 con 0x7f89fc109a30 2024-06-03T03:28:23.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.210+0000 7f89f3e00640 1 -- 172.21.15.117:0/2230231484 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89f4005020 con 0x7f89fc109a30 2024-06-03T03:28:23.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.210+0000 7f89fbe00640 1 -- 172.21.15.117:0/2230231484 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89fc1a60b0 con 0x7f89fc109a30 2024-06-03T03:28:23.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.210+0000 7f89f3e00640 1 -- 172.21.15.117:0/2230231484 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f4010720 con 0x7f89fc109a30 2024-06-03T03:28:23.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.210+0000 7f89fbe00640 1 -- 172.21.15.117:0/2230231484 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89fc1a65b0 con 0x7f89fc109a30 2024-06-03T03:28:23.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.211+0000 7f89fbe00640 1 -- 172.21.15.117:0/2230231484 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89fc10f230 con 0x7f89fc109a30 2024-06-03T03:28:23.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.213+0000 7f89f3e00640 1 -- 172.21.15.117:0/2230231484 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f89f4005540 con 0x7f89fc109a30 2024-06-03T03:28:23.215 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.214+0000 7f89f3e00640 1 --2- 172.21.15.117:0/2230231484 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f89e00a2b90 0x7f89e00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:23.215 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.214+0000 7f89f3e00640 1 -- 172.21.15.117:0/2230231484 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f89f40cfb60 con 0x7f89fc109a30 2024-06-03T03:28:23.215 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.215+0000 7f89faa00640 1 --2- 172.21.15.117:0/2230231484 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f89e00a2b90 0x7f89e00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:23.216 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.216+0000 7f89faa00640 1 --2- 172.21.15.117:0/2230231484 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f89e00a2b90 0x7f89e00a5050 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f89ec000c00 tx=0x7f89ec0046c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:28:23.219 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.219+0000 7f89f3e00640 1 -- 172.21.15.117:0/2230231484 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f89f409c6d0 con 0x7f89fc109a30 2024-06-03T03:28:23.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:28:23 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/2981576844' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T03:28:23.418 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.418+0000 7f89fbe00640 1 -- 172.21.15.117:0/2230231484 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f89fc0625a0 con 0x7f89fc109a30 2024-06-03T03:28:23.419 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.419+0000 7f89f3e00640 1 -- 172.21.15.117:0/2230231484 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v99) ==== 93+0+10 (secure 0 0 0) 0x7f89f409bfd0 con 0x7f89fc109a30 2024-06-03T03:28:23.419 INFO:teuthology.orchestra.run.smithi117.stdout:pg_num: 1 2024-06-03T03:28:23.422 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.421+0000 7f89fbe00640 1 -- 172.21.15.117:0/2230231484 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f89e00a2b90 msgr2=0x7f89e00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:28:23.422 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.421+0000 7f89fbe00640 1 --2- 172.21.15.117:0/2230231484 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f89e00a2b90 0x7f89e00a5050 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f89ec000c00 tx=0x7f89ec0046c0 comp rx=0 tx=0).stop 2024-06-03T03:28:23.422 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.421+0000 7f89fbe00640 1 -- 172.21.15.117:0/2230231484 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89fc109a30 msgr2=0x7f89fc1a5750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:28:23.422 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.421+0000 7f89fbe00640 1 --2- 172.21.15.117:0/2230231484 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89fc109a30 0x7f89fc1a5750 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f89f4009fd0 tx=0x7f89f400ef10 comp rx=0 tx=0).stop 2024-06-03T03:28:23.422 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.421+0000 7f89fbe00640 1 -- 172.21.15.117:0/2230231484 shutdown_connections 2024-06-03T03:28:23.422 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.421+0000 7f89fbe00640 1 --2- 172.21.15.117:0/2230231484 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89fc109a30 0x7f89fc1a5750 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:23.422 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.421+0000 7f89fbe00640 1 --2- 172.21.15.117:0/2230231484 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89fc109070 0x7f89fc19e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:23.422 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.421+0000 7f89fbe00640 1 --2- 172.21.15.117:0/2230231484 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f89e00a2b90 0x7f89e00a5050 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:23.422 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.421+0000 7f89fbe00640 1 --2- 172.21.15.117:0/2230231484 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89fc107e90 0x7f89fc19e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:23.423 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.421+0000 7f89fbe00640 1 -- 172.21.15.117:0/2230231484 >> 172.21.15.117:0/2230231484 conn(0x7f89fc076e70 msgr2=0x7f89fc10c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:28:23.423 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.422+0000 7f89fbe00640 1 -- 172.21.15.117:0/2230231484 shutdown_connections 2024-06-03T03:28:23.423 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.422+0000 7f89fbe00640 1 -- 172.21.15.117:0/2230231484 wait complete. 2024-06-03T03:28:23.438 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:28:23 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/2981576844' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T03:28:23.442 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-06-03T03:28:23.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:28:23 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/2981576844' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T03:28:23.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.650+0000 7f10d4a00640 1 -- 172.21.15.117:0/2875184134 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10d01023b0 msgr2=0x7f10d01047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:28:23.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.650+0000 7f10d4a00640 1 --2- 172.21.15.117:0/2875184134 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10d01023b0 0x7f10d01047a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f10c0009900 tx=0x7f10c002a3f0 comp rx=0 tx=0).stop 2024-06-03T03:28:23.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.650+0000 7f10d4a00640 1 -- 172.21.15.117:0/2875184134 shutdown_connections 2024-06-03T03:28:23.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.650+0000 7f10d4a00640 1 --2- 172.21.15.117:0/2875184134 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10d0107610 0x7f10d0109a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:23.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.650+0000 7f10d4a00640 1 --2- 172.21.15.117:0/2875184134 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10d0104ce0 0x7f10d01070d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:23.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.650+0000 7f10d4a00640 1 --2- 172.21.15.117:0/2875184134 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10d01023b0 0x7f10d01047a0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:23.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.650+0000 7f10d4a00640 1 -- 172.21.15.117:0/2875184134 >> 172.21.15.117:0/2875184134 conn(0x7f10d00fbd40 msgr2=0x7f10d00fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:28:23.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.650+0000 7f10d4a00640 1 -- 172.21.15.117:0/2875184134 shutdown_connections 2024-06-03T03:28:23.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.650+0000 7f10d4a00640 1 -- 172.21.15.117:0/2875184134 wait complete. 2024-06-03T03:28:23.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.651+0000 7f10d4a00640 1 Processor -- start 2024-06-03T03:28:23.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.651+0000 7f10d4a00640 1 -- start start 2024-06-03T03:28:23.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.652+0000 7f10d4a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10d01023b0 0x7f10d0199df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:23.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.652+0000 7f10d4a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10d0104ce0 0x7f10d019a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:23.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.652+0000 7f10d4a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10d0107610 0x7f10d01a13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:23.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.652+0000 7f10d4a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f10d010d520 con 0x7f10d0104ce0 2024-06-03T03:28:23.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.652+0000 7f10d4a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f10d010d3a0 con 0x7f10d0107610 2024-06-03T03:28:23.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.652+0000 7f10d4a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f10d010d6a0 con 0x7f10d01023b0 2024-06-03T03:28:23.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.652+0000 7f10cea00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10d0104ce0 0x7f10d019a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:23.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.652+0000 7f10cea00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10d0104ce0 0x7f10d019a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:52484/0 (socket says 172.21.15.117:52484) 2024-06-03T03:28:23.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.652+0000 7f10cea00640 1 -- 172.21.15.117:0/122289945 learned_addr learned my addr 172.21.15.117:0/122289945 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:28:23.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.652+0000 7f10cfe00640 1 --2- 172.21.15.117:0/122289945 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10d0107610 0x7f10d01a13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:23.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.652+0000 7f10cf400640 1 --2- 172.21.15.117:0/122289945 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10d01023b0 0x7f10d0199df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:23.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.652+0000 7f10cfe00640 1 -- 172.21.15.117:0/122289945 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10d01023b0 msgr2=0x7f10d0199df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:28:23.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.652+0000 7f10cfe00640 1 --2- 172.21.15.117:0/122289945 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10d01023b0 0x7f10d0199df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:23.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.652+0000 7f10cfe00640 1 -- 172.21.15.117:0/122289945 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10d0104ce0 msgr2=0x7f10d019a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:28:23.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.652+0000 7f10cfe00640 1 --2- 172.21.15.117:0/122289945 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10d0104ce0 0x7f10d019a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:23.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.652+0000 7f10cfe00640 1 -- 172.21.15.117:0/122289945 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10d01a1a10 con 0x7f10d0107610 2024-06-03T03:28:23.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.653+0000 7f10cfe00640 1 --2- 172.21.15.117:0/122289945 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10d0107610 0x7f10d01a13a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f10c800e940 tx=0x7f10c800ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:28:23.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.653+0000 7f10c7e00640 1 -- 172.21.15.117:0/122289945 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c8004d20 con 0x7f10d0107610 2024-06-03T03:28:23.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.653+0000 7f10c7e00640 1 -- 172.21.15.117:0/122289945 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10c8005020 con 0x7f10d0107610 2024-06-03T03:28:23.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.653+0000 7f10d4a00640 1 -- 172.21.15.117:0/122289945 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10d01a1d00 con 0x7f10d0107610 2024-06-03T03:28:23.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.653+0000 7f10c7e00640 1 -- 172.21.15.117:0/122289945 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c80116b0 con 0x7f10d0107610 2024-06-03T03:28:23.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.653+0000 7f10d4a00640 1 -- 172.21.15.117:0/122289945 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10d01a21f0 con 0x7f10d0107610 2024-06-03T03:28:23.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.654+0000 7f10d4a00640 1 -- 172.21.15.117:0/122289945 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10d0109100 con 0x7f10d0107610 2024-06-03T03:28:23.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.656+0000 7f10c7e00640 1 -- 172.21.15.117:0/122289945 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f10c80054d0 con 0x7f10d0107610 2024-06-03T03:28:23.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.657+0000 7f10c7e00640 1 --2- 172.21.15.117:0/122289945 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f10b40a2c10 0x7f10b40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:23.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.658+0000 7f10c7e00640 1 -- 172.21.15.117:0/122289945 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f10c80cf390 con 0x7f10d0107610 2024-06-03T03:28:23.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.658+0000 7f10cf400640 1 --2- 172.21.15.117:0/122289945 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f10b40a2c10 0x7f10b40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:23.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.659+0000 7f10cf400640 1 --2- 172.21.15.117:0/122289945 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f10b40a2c10 0x7f10b40a50d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f10c00314e0 tx=0x7f10c0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:28:23.663 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.662+0000 7f10c7e00640 1 -- 172.21.15.117:0/122289945 <== mon.2 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f10c8016030 con 0x7f10d0107610 2024-06-03T03:28:23.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.861+0000 7f10d4a00640 1 -- 172.21.15.117:0/122289945 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f10d00625a0 con 0x7f10d0107610 2024-06-03T03:28:23.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.862+0000 7f10c7e00640 1 -- 172.21.15.117:0/122289945 <== mon.2 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v99) ==== 104+0+11 (secure 0 0 0) 0x7f10c809b8b0 con 0x7f10d0107610 2024-06-03T03:28:23.863 INFO:teuthology.orchestra.run.smithi117.stdout:pg_num: 32 2024-06-03T03:28:23.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.865+0000 7f10d4a00640 1 -- 172.21.15.117:0/122289945 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f10b40a2c10 msgr2=0x7f10b40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:28:23.866 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.865+0000 7f10d4a00640 1 --2- 172.21.15.117:0/122289945 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f10b40a2c10 0x7f10b40a50d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f10c00314e0 tx=0x7f10c0000bc0 comp rx=0 tx=0).stop 2024-06-03T03:28:23.866 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.865+0000 7f10d4a00640 1 -- 172.21.15.117:0/122289945 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10d0107610 msgr2=0x7f10d01a13a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:28:23.866 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.865+0000 7f10d4a00640 1 --2- 172.21.15.117:0/122289945 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10d0107610 0x7f10d01a13a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f10c800e940 tx=0x7f10c800ee00 comp rx=0 tx=0).stop 2024-06-03T03:28:23.866 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.865+0000 7f10d4a00640 1 -- 172.21.15.117:0/122289945 shutdown_connections 2024-06-03T03:28:23.866 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.865+0000 7f10d4a00640 1 --2- 172.21.15.117:0/122289945 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f10b40a2c10 0x7f10b40a50d0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:23.866 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.865+0000 7f10d4a00640 1 --2- 172.21.15.117:0/122289945 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10d0107610 0x7f10d01a13a0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:23.866 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.865+0000 7f10d4a00640 1 --2- 172.21.15.117:0/122289945 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10d0104ce0 0x7f10d019a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:23.866 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.865+0000 7f10d4a00640 1 --2- 172.21.15.117:0/122289945 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10d01023b0 0x7f10d0199df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:23.867 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.865+0000 7f10d4a00640 1 -- 172.21.15.117:0/122289945 >> 172.21.15.117:0/122289945 conn(0x7f10d00fbd40 msgr2=0x7f10d0101b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:28:23.867 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.865+0000 7f10d4a00640 1 -- 172.21.15.117:0/122289945 shutdown_connections 2024-06-03T03:28:23.867 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:23.865+0000 7f10d4a00640 1 -- 172.21.15.117:0/122289945 wait complete. 2024-06-03T03:28:23.886 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-06-03T03:28:24.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.096+0000 7f3e73000640 1 -- 172.21.15.117:0/82400540 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e6c105690 msgr2=0x7f3e6c10bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:28:24.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.096+0000 7f3e73000640 1 --2- 172.21.15.117:0/82400540 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e6c105690 0x7f3e6c10bf20 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f3e64009ca0 tx=0x7f3e6402a8c0 comp rx=0 tx=0).stop 2024-06-03T03:28:24.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.096+0000 7f3e73000640 1 -- 172.21.15.117:0/82400540 shutdown_connections 2024-06-03T03:28:24.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.096+0000 7f3e73000640 1 --2- 172.21.15.117:0/82400540 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e6c105690 0x7f3e6c10bf20 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:24.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.096+0000 7f3e73000640 1 --2- 172.21.15.117:0/82400540 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3e6c104cd0 0x7f3e6c105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:24.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.096+0000 7f3e73000640 1 --2- 172.21.15.117:0/82400540 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e6c103ae0 0x7f3e6c103f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:24.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.096+0000 7f3e73000640 1 -- 172.21.15.117:0/82400540 >> 172.21.15.117:0/82400540 conn(0x7f3e6c0ff050 msgr2=0x7f3e6c101490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:28:24.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.096+0000 7f3e73000640 1 -- 172.21.15.117:0/82400540 shutdown_connections 2024-06-03T03:28:24.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.096+0000 7f3e73000640 1 -- 172.21.15.117:0/82400540 wait complete. 2024-06-03T03:28:24.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.097+0000 7f3e73000640 1 Processor -- start 2024-06-03T03:28:24.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.097+0000 7f3e73000640 1 -- start start 2024-06-03T03:28:24.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.097+0000 7f3e73000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e6c103ae0 0x7f3e6c199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:24.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.098+0000 7f3e73000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3e6c104cd0 0x7f3e6c19a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:24.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.098+0000 7f3e73000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e6c105690 0x7f3e6c1a12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:24.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.098+0000 7f3e73000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3e6c10f790 con 0x7f3e6c105690 2024-06-03T03:28:24.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.098+0000 7f3e73000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3e6c10f610 con 0x7f3e6c104cd0 2024-06-03T03:28:24.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.098+0000 7f3e73000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3e6c10f910 con 0x7f3e6c103ae0 2024-06-03T03:28:24.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.098+0000 7f3e71c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e6c103ae0 0x7f3e6c199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:24.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.098+0000 7f3e72600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e6c105690 0x7f3e6c1a12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:24.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.098+0000 7f3e71c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e6c103ae0 0x7f3e6c199d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:34228/0 (socket says 172.21.15.117:34228) 2024-06-03T03:28:24.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.098+0000 7f3e71200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3e6c104cd0 0x7f3e6c19a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:24.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.098+0000 7f3e71c00640 1 -- 172.21.15.117:0/2189770831 learned_addr learned my addr 172.21.15.117:0/2189770831 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:28:24.100 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.098+0000 7f3e71200640 1 -- 172.21.15.117:0/2189770831 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e6c103ae0 msgr2=0x7f3e6c199d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:28:24.100 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.098+0000 7f3e71200640 1 --2- 172.21.15.117:0/2189770831 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e6c103ae0 0x7f3e6c199d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:24.100 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.098+0000 7f3e71200640 1 -- 172.21.15.117:0/2189770831 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e6c105690 msgr2=0x7f3e6c1a12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:28:24.100 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.098+0000 7f3e71200640 1 --2- 172.21.15.117:0/2189770831 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e6c105690 0x7f3e6c1a12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:24.100 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.098+0000 7f3e71200640 1 -- 172.21.15.117:0/2189770831 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e6c1a1960 con 0x7f3e6c104cd0 2024-06-03T03:28:24.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.098+0000 7f3e71c00640 1 --2- 172.21.15.117:0/2189770831 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e6c103ae0 0x7f3e6c199d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:28:24.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.098+0000 7f3e72600640 1 --2- 172.21.15.117:0/2189770831 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e6c105690 0x7f3e6c1a12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:28:24.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.099+0000 7f3e71200640 1 --2- 172.21.15.117:0/2189770831 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3e6c104cd0 0x7f3e6c19a280 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3e5800e9f0 tx=0x7f3e5800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:28:24.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.099+0000 7f3e6a000640 1 -- 172.21.15.117:0/2189770831 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e5800ccc0 con 0x7f3e6c104cd0 2024-06-03T03:28:24.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.099+0000 7f3e73000640 1 -- 172.21.15.117:0/2189770831 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e6c1a1c50 con 0x7f3e6c104cd0 2024-06-03T03:28:24.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.099+0000 7f3e6a000640 1 -- 172.21.15.117:0/2189770831 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e58010910 con 0x7f3e6c104cd0 2024-06-03T03:28:24.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.099+0000 7f3e6a000640 1 -- 172.21.15.117:0/2189770831 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e58018960 con 0x7f3e6c104cd0 2024-06-03T03:28:24.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.099+0000 7f3e73000640 1 -- 172.21.15.117:0/2189770831 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e6c1a21a0 con 0x7f3e6c104cd0 2024-06-03T03:28:24.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.100+0000 7f3e73000640 1 -- 172.21.15.117:0/2189770831 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e6c10ada0 con 0x7f3e6c104cd0 2024-06-03T03:28:24.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.103+0000 7f3e6a000640 1 -- 172.21.15.117:0/2189770831 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f3e58010ac0 con 0x7f3e6c104cd0 2024-06-03T03:28:24.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.104+0000 7f3e6a000640 1 --2- 172.21.15.117:0/2189770831 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f3e540a2c10 0x7f3e540a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:24.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.104+0000 7f3e6a000640 1 -- 172.21.15.117:0/2189770831 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f3e580cf740 con 0x7f3e6c104cd0 2024-06-03T03:28:24.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.104+0000 7f3e71c00640 1 --2- 172.21.15.117:0/2189770831 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f3e540a2c10 0x7f3e540a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:24.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.105+0000 7f3e71c00640 1 --2- 172.21.15.117:0/2189770831 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f3e540a2c10 0x7f3e540a50d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3e600058c0 tx=0x7f3e60005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:28:24.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.109+0000 7f3e6a000640 1 -- 172.21.15.117:0/2189770831 <== mon.2 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3e5809bc60 con 0x7f3e6c104cd0 2024-06-03T03:28:24.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:28:24 smithi117 ceph-mon[31649]: pgmap v598: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 852 B/s wr, 11 op/s 2024-06-03T03:28:24.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:28:24 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/2230231484' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-03T03:28:24.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:28:24 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/122289945' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-03T03:28:24.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.311+0000 7f3e73000640 1 -- 172.21.15.117:0/2189770831 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3e6c02d2e0 con 0x7f3e6c104cd0 2024-06-03T03:28:24.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.312+0000 7f3e6a000640 1 -- 172.21.15.117:0/2189770831 <== mon.2 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v99) ==== 100+0+11 (secure 0 0 0) 0x7f3e5809ff60 con 0x7f3e6c104cd0 2024-06-03T03:28:24.313 INFO:teuthology.orchestra.run.smithi117.stdout:pg_num: 64 2024-06-03T03:28:24.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.315+0000 7f3e73000640 1 -- 172.21.15.117:0/2189770831 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f3e540a2c10 msgr2=0x7f3e540a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:28:24.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.315+0000 7f3e73000640 1 --2- 172.21.15.117:0/2189770831 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f3e540a2c10 0x7f3e540a50d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3e600058c0 tx=0x7f3e60005f90 comp rx=0 tx=0).stop 2024-06-03T03:28:24.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.315+0000 7f3e73000640 1 -- 172.21.15.117:0/2189770831 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3e6c104cd0 msgr2=0x7f3e6c19a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:28:24.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.315+0000 7f3e73000640 1 --2- 172.21.15.117:0/2189770831 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3e6c104cd0 0x7f3e6c19a280 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3e5800e9f0 tx=0x7f3e5800eeb0 comp rx=0 tx=0).stop 2024-06-03T03:28:24.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.316+0000 7f3e73000640 1 -- 172.21.15.117:0/2189770831 shutdown_connections 2024-06-03T03:28:24.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.316+0000 7f3e73000640 1 --2- 172.21.15.117:0/2189770831 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e6c105690 0x7f3e6c1a12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:24.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.316+0000 7f3e73000640 1 --2- 172.21.15.117:0/2189770831 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f3e540a2c10 0x7f3e540a50d0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:24.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.316+0000 7f3e73000640 1 --2- 172.21.15.117:0/2189770831 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3e6c104cd0 0x7f3e6c19a280 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:24.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.316+0000 7f3e73000640 1 --2- 172.21.15.117:0/2189770831 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e6c103ae0 0x7f3e6c199d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:24.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.316+0000 7f3e73000640 1 -- 172.21.15.117:0/2189770831 >> 172.21.15.117:0/2189770831 conn(0x7f3e6c0ff050 msgr2=0x7f3e6c107f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:28:24.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.316+0000 7f3e73000640 1 -- 172.21.15.117:0/2189770831 shutdown_connections 2024-06-03T03:28:24.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.316+0000 7f3e73000640 1 -- 172.21.15.117:0/2189770831 wait complete. 2024-06-03T03:28:24.337 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-06-03T03:28:24.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:28:24 smithi151 ceph-mon[37043]: pgmap v598: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 852 B/s wr, 11 op/s 2024-06-03T03:28:24.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:28:24 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/2230231484' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-03T03:28:24.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:28:24 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/122289945' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-03T03:28:24.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:28:24 smithi183 ceph-mon[37326]: pgmap v598: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 852 B/s wr, 11 op/s 2024-06-03T03:28:24.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:28:24 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/2230231484' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-03T03:28:24.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:28:24 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/122289945' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-03T03:28:24.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.543+0000 7fe2ebe00640 1 -- 172.21.15.117:0/3503872540 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe2e4104df0 msgr2=0x7fe2e4105270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:28:24.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.543+0000 7fe2ebe00640 1 --2- 172.21.15.117:0/3503872540 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe2e4104df0 0x7fe2e4105270 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe2d0009860 tx=0x7fe2d002a340 comp rx=0 tx=0).stop 2024-06-03T03:28:24.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.543+0000 7fe2ebe00640 1 -- 172.21.15.117:0/3503872540 shutdown_connections 2024-06-03T03:28:24.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.543+0000 7fe2ebe00640 1 --2- 172.21.15.117:0/3503872540 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe2e41057b0 0x7fe2e410c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:24.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.543+0000 7fe2ebe00640 1 --2- 172.21.15.117:0/3503872540 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe2e4104df0 0x7fe2e4105270 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:24.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.543+0000 7fe2ebe00640 1 --2- 172.21.15.117:0/3503872540 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2e4103c00 0x7fe2e4104020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:24.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.543+0000 7fe2ebe00640 1 -- 172.21.15.117:0/3503872540 >> 172.21.15.117:0/3503872540 conn(0x7fe2e40ff190 msgr2=0x7fe2e41015b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:28:24.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.544+0000 7fe2ebe00640 1 -- 172.21.15.117:0/3503872540 shutdown_connections 2024-06-03T03:28:24.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.544+0000 7fe2ebe00640 1 -- 172.21.15.117:0/3503872540 wait complete. 2024-06-03T03:28:24.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.545+0000 7fe2ebe00640 1 Processor -- start 2024-06-03T03:28:24.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.545+0000 7fe2ebe00640 1 -- start start 2024-06-03T03:28:24.546 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.545+0000 7fe2ebe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2e4103c00 0x7fe2e4199f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:24.546 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.545+0000 7fe2ebe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe2e4104df0 0x7fe2e419a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:24.546 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.545+0000 7fe2ebe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe2e41057b0 0x7fe2e41a14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:24.546 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.545+0000 7fe2ebe00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe2e410f860 con 0x7fe2e4103c00 2024-06-03T03:28:24.546 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.545+0000 7fe2ebe00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe2e410f6e0 con 0x7fe2e4104df0 2024-06-03T03:28:24.546 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.545+0000 7fe2ebe00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe2e410f9e0 con 0x7fe2e41057b0 2024-06-03T03:28:24.546 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.546+0000 7fe2e9400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2e4103c00 0x7fe2e4199f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:24.546 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.546+0000 7fe2e9400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2e4103c00 0x7fe2e4199f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:52530/0 (socket says 172.21.15.117:52530) 2024-06-03T03:28:24.547 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.546+0000 7fe2e9400640 1 -- 172.21.15.117:0/2684813020 learned_addr learned my addr 172.21.15.117:0/2684813020 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:28:24.547 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.546+0000 7fe2e9e00640 1 --2- 172.21.15.117:0/2684813020 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe2e41057b0 0x7fe2e41a14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:24.547 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.546+0000 7fe2e8a00640 1 --2- 172.21.15.117:0/2684813020 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe2e4104df0 0x7fe2e419a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:24.548 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.546+0000 7fe2e8a00640 1 -- 172.21.15.117:0/2684813020 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe2e41057b0 msgr2=0x7fe2e41a14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:28:24.548 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.546+0000 7fe2e8a00640 1 --2- 172.21.15.117:0/2684813020 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe2e41057b0 0x7fe2e41a14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:24.548 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.546+0000 7fe2e8a00640 1 -- 172.21.15.117:0/2684813020 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2e4103c00 msgr2=0x7fe2e4199f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:28:24.548 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.546+0000 7fe2e8a00640 1 --2- 172.21.15.117:0/2684813020 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2e4103c00 0x7fe2e4199f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:24.548 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.546+0000 7fe2e8a00640 1 -- 172.21.15.117:0/2684813020 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2e41a1b30 con 0x7fe2e4104df0 2024-06-03T03:28:24.548 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.546+0000 7fe2e9e00640 1 --2- 172.21.15.117:0/2684813020 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe2e41057b0 0x7fe2e41a14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:28:24.548 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.547+0000 7fe2e8a00640 1 --2- 172.21.15.117:0/2684813020 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe2e4104df0 0x7fe2e419a450 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe2d0035040 tx=0x7fe2d0033d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:28:24.548 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.547+0000 7fe2e2000640 1 -- 172.21.15.117:0/2684813020 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2d0037040 con 0x7fe2e4104df0 2024-06-03T03:28:24.548 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.547+0000 7fe2ebe00640 1 -- 172.21.15.117:0/2684813020 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2e41a1dc0 con 0x7fe2e4104df0 2024-06-03T03:28:24.548 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.547+0000 7fe2e2000640 1 -- 172.21.15.117:0/2684813020 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2d0004040 con 0x7fe2e4104df0 2024-06-03T03:28:24.548 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.547+0000 7fe2e2000640 1 -- 172.21.15.117:0/2684813020 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2d0005650 con 0x7fe2e4104df0 2024-06-03T03:28:24.549 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.547+0000 7fe2ebe00640 1 -- 172.21.15.117:0/2684813020 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2e41a2270 con 0x7fe2e4104df0 2024-06-03T03:28:24.549 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.548+0000 7fe2ebe00640 1 -- 172.21.15.117:0/2684813020 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2e410af20 con 0x7fe2e4104df0 2024-06-03T03:28:24.551 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.551+0000 7fe2e2000640 1 -- 172.21.15.117:0/2684813020 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fe2d0042020 con 0x7fe2e4104df0 2024-06-03T03:28:24.552 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.552+0000 7fe2e2000640 1 --2- 172.21.15.117:0/2684813020 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fe2cc0a2c10 0x7fe2cc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:24.552 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.552+0000 7fe2e2000640 1 -- 172.21.15.117:0/2684813020 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7fe2d00eebd0 con 0x7fe2e4104df0 2024-06-03T03:28:24.553 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.553+0000 7fe2e9400640 1 --2- 172.21.15.117:0/2684813020 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fe2cc0a2c10 0x7fe2cc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:24.553 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.553+0000 7fe2e9400640 1 --2- 172.21.15.117:0/2684813020 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fe2cc0a2c10 0x7fe2cc0a50d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe2d80096c0 tx=0x7fe2d8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:28:24.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.557+0000 7fe2e2000640 1 -- 172.21.15.117:0/2684813020 <== mon.2 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe2d0049020 con 0x7fe2e4104df0 2024-06-03T03:28:24.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.763+0000 7fe2ebe00640 1 -- 172.21.15.117:0/2684813020 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fe2e4062670 con 0x7fe2e4104df0 2024-06-03T03:28:24.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.765+0000 7fe2e2000640 1 -- 172.21.15.117:0/2684813020 <== mon.2 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v99) ==== 103+0+11 (secure 0 0 0) 0x7fe2d00bb740 con 0x7fe2e4104df0 2024-06-03T03:28:24.765 INFO:teuthology.orchestra.run.smithi117.stdout:pg_num: 32 2024-06-03T03:28:24.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.767+0000 7fe2ebe00640 1 -- 172.21.15.117:0/2684813020 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fe2cc0a2c10 msgr2=0x7fe2cc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:28:24.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.767+0000 7fe2ebe00640 1 --2- 172.21.15.117:0/2684813020 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fe2cc0a2c10 0x7fe2cc0a50d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe2d80096c0 tx=0x7fe2d8009260 comp rx=0 tx=0).stop 2024-06-03T03:28:24.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.767+0000 7fe2ebe00640 1 -- 172.21.15.117:0/2684813020 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe2e4104df0 msgr2=0x7fe2e419a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:28:24.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.767+0000 7fe2ebe00640 1 --2- 172.21.15.117:0/2684813020 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe2e4104df0 0x7fe2e419a450 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe2d0035040 tx=0x7fe2d0033d90 comp rx=0 tx=0).stop 2024-06-03T03:28:24.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.768+0000 7fe2ebe00640 1 -- 172.21.15.117:0/2684813020 shutdown_connections 2024-06-03T03:28:24.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.768+0000 7fe2ebe00640 1 --2- 172.21.15.117:0/2684813020 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe2e41057b0 0x7fe2e41a14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:24.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.768+0000 7fe2ebe00640 1 --2- 172.21.15.117:0/2684813020 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fe2cc0a2c10 0x7fe2cc0a50d0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:24.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.768+0000 7fe2ebe00640 1 --2- 172.21.15.117:0/2684813020 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe2e4104df0 0x7fe2e419a450 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:24.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.768+0000 7fe2ebe00640 1 --2- 172.21.15.117:0/2684813020 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2e4103c00 0x7fe2e4199f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:24.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.768+0000 7fe2ebe00640 1 -- 172.21.15.117:0/2684813020 >> 172.21.15.117:0/2684813020 conn(0x7fe2e40ff190 msgr2=0x7fe2e41080f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:28:24.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.768+0000 7fe2ebe00640 1 -- 172.21.15.117:0/2684813020 shutdown_connections 2024-06-03T03:28:24.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.768+0000 7fe2ebe00640 1 -- 172.21.15.117:0/2684813020 wait complete. 2024-06-03T03:28:24.789 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-06-03T03:28:24.991 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.991+0000 7f5a8a200640 1 -- 172.21.15.117:0/826884079 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a84101570 msgr2=0x7f5a841019f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:28:24.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.991+0000 7f5a8a200640 1 --2- 172.21.15.117:0/826884079 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a84101570 0x7f5a841019f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c009ca0 tx=0x7f5a7c02a8c0 comp rx=0 tx=0).stop 2024-06-03T03:28:24.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.991+0000 7f5a8a200640 1 -- 172.21.15.117:0/826884079 shutdown_connections 2024-06-03T03:28:24.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.991+0000 7f5a8a200640 1 --2- 172.21.15.117:0/826884079 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a84101570 0x7f5a841019f0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:24.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.991+0000 7f5a8a200640 1 --2- 172.21.15.117:0/826884079 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a84100bd0 0x7f5a84101030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:24.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.991+0000 7f5a8a200640 1 --2- 172.21.15.117:0/826884079 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a84100270 0x7f5a84100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:24.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.991+0000 7f5a8a200640 1 -- 172.21.15.117:0/826884079 >> 172.21.15.117:0/826884079 conn(0x7f5a840fbe90 msgr2=0x7f5a840fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:28:24.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.991+0000 7f5a8a200640 1 -- 172.21.15.117:0/826884079 shutdown_connections 2024-06-03T03:28:24.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.991+0000 7f5a8a200640 1 -- 172.21.15.117:0/826884079 wait complete. 2024-06-03T03:28:24.994 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.992+0000 7f5a8a200640 1 Processor -- start 2024-06-03T03:28:24.994 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.992+0000 7f5a8a200640 1 -- start start 2024-06-03T03:28:24.994 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.992+0000 7f5a8a200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a84100270 0x7f5a84199e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:24.994 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.992+0000 7f5a8a200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a84100bd0 0x7f5a8419a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:24.994 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.992+0000 7f5a8a200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a84101570 0x7f5a841a1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:24.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.992+0000 7f5a8a200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5a8410f8a0 con 0x7f5a84100bd0 2024-06-03T03:28:24.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.992+0000 7f5a8a200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5a8410f720 con 0x7f5a84100270 2024-06-03T03:28:24.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.992+0000 7f5a8a200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5a8410fa20 con 0x7f5a84101570 2024-06-03T03:28:24.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.992+0000 7f5a82a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a84100bd0 0x7f5a8419a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:24.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.993+0000 7f5a82a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a84100bd0 0x7f5a8419a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:52540/0 (socket says 172.21.15.117:52540) 2024-06-03T03:28:24.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.993+0000 7f5a82a00640 1 -- 172.21.15.117:0/3573246030 learned_addr learned my addr 172.21.15.117:0/3573246030 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:28:24.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.993+0000 7f5a83400640 1 --2- 172.21.15.117:0/3573246030 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a84100270 0x7f5a84199e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:24.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.993+0000 7f5a83e00640 1 --2- 172.21.15.117:0/3573246030 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a84101570 0x7f5a841a1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:24.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.993+0000 7f5a82a00640 1 -- 172.21.15.117:0/3573246030 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a84101570 msgr2=0x7f5a841a1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:28:24.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.993+0000 7f5a82a00640 1 --2- 172.21.15.117:0/3573246030 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a84101570 0x7f5a841a1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:24.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.993+0000 7f5a82a00640 1 -- 172.21.15.117:0/3573246030 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a84100270 msgr2=0x7f5a84199e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:28:24.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.993+0000 7f5a82a00640 1 --2- 172.21.15.117:0/3573246030 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a84100270 0x7f5a84199e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:24.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.993+0000 7f5a82a00640 1 -- 172.21.15.117:0/3573246030 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a841a1aa0 con 0x7f5a84100bd0 2024-06-03T03:28:24.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.993+0000 7f5a82a00640 1 --2- 172.21.15.117:0/3573246030 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a84100bd0 0x7f5a8419a3c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5a7000ea60 tx=0x7f5a7000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:28:24.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.994+0000 7f5a7be00640 1 -- 172.21.15.117:0/3573246030 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a7000cde0 con 0x7f5a84100bd0 2024-06-03T03:28:24.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.994+0000 7f5a8a200640 1 -- 172.21.15.117:0/3573246030 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a841a1d90 con 0x7f5a84100bd0 2024-06-03T03:28:24.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.994+0000 7f5a7be00640 1 -- 172.21.15.117:0/3573246030 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a700035f0 con 0x7f5a84100bd0 2024-06-03T03:28:24.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.994+0000 7f5a7be00640 1 -- 172.21.15.117:0/3573246030 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a70010670 con 0x7f5a84100bd0 2024-06-03T03:28:24.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.994+0000 7f5a8a200640 1 -- 172.21.15.117:0/3573246030 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a841a2280 con 0x7f5a84100bd0 2024-06-03T03:28:24.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.995+0000 7f5a8a200640 1 -- 172.21.15.117:0/3573246030 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a8410afc0 con 0x7f5a84100bd0 2024-06-03T03:28:24.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.997+0000 7f5a7be00640 1 -- 172.21.15.117:0/3573246030 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f5a700037a0 con 0x7f5a84100bd0 2024-06-03T03:28:24.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.999+0000 7f5a7be00640 1 --2- 172.21.15.117:0/3573246030 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f5a680a2b90 0x7f5a680a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:28:24.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.999+0000 7f5a7be00640 1 -- 172.21.15.117:0/3573246030 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f5a700cf6e0 con 0x7f5a84100bd0 2024-06-03T03:28:24.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:24.999+0000 7f5a83400640 1 --2- 172.21.15.117:0/3573246030 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f5a680a2b90 0x7f5a680a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:28:25.000 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:25.000+0000 7f5a83400640 1 --2- 172.21.15.117:0/3573246030 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f5a680a2b90 0x7f5a680a5050 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f5a74009bd0 tx=0x7f5a74005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:28:25.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:25.003+0000 7f5a7be00640 1 -- 172.21.15.117:0/3573246030 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5a7009c250 con 0x7f5a84100bd0 2024-06-03T03:28:25.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:25.255+0000 7f5a8a200640 1 -- 172.21.15.117:0/3573246030 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f5a84062640 con 0x7f5a84100bd0 2024-06-03T03:28:25.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:25.286+0000 7f5a7be00640 1 -- 172.21.15.117:0/3573246030 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v20) ==== 81+0+740379 (secure 0 0 0) 0x7f5a7009bb50 con 0x7f5a84100bd0 2024-06-03T03:28:25.294 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T03:28:25.294 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-06-03T03:28:25.294 INFO:teuthology.orchestra.run.smithi117.stdout: "epoch": 20, 2024-06-03T03:28:25.294 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.294 INFO:teuthology.orchestra.run.smithi117.stdout: "active_gid": 14152, 2024-06-03T03:28:25.294 INFO:teuthology.orchestra.run.smithi117.stdout: "active_name": "x", 2024-06-03T03:28:25.294 INFO:teuthology.orchestra.run.smithi117.stdout: "active_addrs": { 2024-06-03T03:28:25.294 INFO:teuthology.orchestra.run.smithi117.stdout: "addrvec": [ 2024-06-03T03:28:25.294 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:25.295 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v2", 2024-06-03T03:28:25.295 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.117:6800", 2024-06-03T03:28:25.295 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 1398160887 2024-06-03T03:28:25.295 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.295 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:25.295 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v1", 2024-06-03T03:28:25.295 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.117:6801", 2024-06-03T03:28:25.295 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 1398160887 2024-06-03T03:28:25.295 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.295 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-06-03T03:28:25.295 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.295 INFO:teuthology.orchestra.run.smithi117.stdout: "active_addr": "172.21.15.117:6801/1398160887", 2024-06-03T03:28:25.295 INFO:teuthology.orchestra.run.smithi117.stdout: "active_change": "2024-06-03T03:11:24.655863+0000", 2024-06-03T03:28:25.295 INFO:teuthology.orchestra.run.smithi117.stdout: "active_mgr_features": 4540701547738038271, 2024-06-03T03:28:25.296 INFO:teuthology.orchestra.run.smithi117.stdout: "available": true, 2024-06-03T03:28:25.296 INFO:teuthology.orchestra.run.smithi117.stdout: "standbys": [ 2024-06-03T03:28:25.296 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:25.296 INFO:teuthology.orchestra.run.smithi117.stdout: "gid": 24109, 2024-06-03T03:28:25.296 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "z", 2024-06-03T03:28:25.296 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr_features": 4540701547738038271, 2024-06-03T03:28:25.296 INFO:teuthology.orchestra.run.smithi117.stdout: "available_modules": [ 2024-06-03T03:28:25.296 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:25.296 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "alerts", 2024-06-03T03:28:25.296 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:25.296 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:25.296 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:25.296 INFO:teuthology.orchestra.run.smithi117.stdout: "interval": { 2024-06-03T03:28:25.297 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "interval", 2024-06-03T03:28:25.297 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-06-03T03:28:25.297 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.297 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.297 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "60", 2024-06-03T03:28:25.297 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.297 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.297 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.297 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "How frequently to reexamine health status", 2024-06-03T03:28:25.297 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.297 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.297 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.297 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.297 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:25.298 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:25.298 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.298 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.298 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.298 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.298 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.298 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.298 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.298 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.298 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.298 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.298 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.298 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.299 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.299 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.299 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.299 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.299 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.299 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.299 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.299 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:25.299 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:25.299 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.299 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.299 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.299 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.299 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.300 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.300 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.300 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.300 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.300 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.300 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.300 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.300 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:25.300 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:25.300 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.300 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.300 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.300 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:25.300 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.301 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.301 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.301 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.301 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.301 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.301 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.301 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.301 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.301 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.301 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.301 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.301 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.301 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.301 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.302 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:25.302 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:25.302 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.302 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.302 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.302 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.302 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.302 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.302 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.302 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.302 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.302 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.302 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.303 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.303 INFO:teuthology.orchestra.run.smithi117.stdout: "smtp_destination": { 2024-06-03T03:28:25.303 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "smtp_destination", 2024-06-03T03:28:25.303 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.303 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.303 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.303 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.303 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.303 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.303 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.303 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Email address to send alerts to", 2024-06-03T03:28:25.303 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.303 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.303 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.303 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.304 INFO:teuthology.orchestra.run.smithi117.stdout: "smtp_from_name": { 2024-06-03T03:28:25.304 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "smtp_from_name", 2024-06-03T03:28:25.304 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.304 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.304 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.304 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "Ceph", 2024-06-03T03:28:25.304 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.304 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.304 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.304 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Email From: name", 2024-06-03T03:28:25.304 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.304 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.304 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.304 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.305 INFO:teuthology.orchestra.run.smithi117.stdout: "smtp_host": { 2024-06-03T03:28:25.305 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "smtp_host", 2024-06-03T03:28:25.305 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.305 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.305 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.305 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.305 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.305 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.305 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.305 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "SMTP server", 2024-06-03T03:28:25.305 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.305 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.305 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.305 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.306 INFO:teuthology.orchestra.run.smithi117.stdout: "smtp_password": { 2024-06-03T03:28:25.306 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "smtp_password", 2024-06-03T03:28:25.306 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.306 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.306 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.306 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.306 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.306 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.306 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.306 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Password to authenticate with", 2024-06-03T03:28:25.306 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.306 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.306 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.306 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.307 INFO:teuthology.orchestra.run.smithi117.stdout: "smtp_port": { 2024-06-03T03:28:25.307 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "smtp_port", 2024-06-03T03:28:25.307 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:25.307 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.307 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.307 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "465", 2024-06-03T03:28:25.307 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.307 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.307 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.307 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "SMTP port", 2024-06-03T03:28:25.307 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.307 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.307 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.307 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.308 INFO:teuthology.orchestra.run.smithi117.stdout: "smtp_sender": { 2024-06-03T03:28:25.308 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "smtp_sender", 2024-06-03T03:28:25.308 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.308 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.308 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.308 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.308 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.308 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.308 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.308 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "SMTP envelope sender", 2024-06-03T03:28:25.308 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.308 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.308 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.309 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.309 INFO:teuthology.orchestra.run.smithi117.stdout: "smtp_ssl": { 2024-06-03T03:28:25.309 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "smtp_ssl", 2024-06-03T03:28:25.309 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.309 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.309 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.309 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:25.309 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.309 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.309 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.309 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Use SSL to connect to SMTP server", 2024-06-03T03:28:25.309 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.309 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.309 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.310 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.310 INFO:teuthology.orchestra.run.smithi117.stdout: "smtp_user": { 2024-06-03T03:28:25.310 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "smtp_user", 2024-06-03T03:28:25.310 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.310 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.310 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.310 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.310 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.310 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.310 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.310 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "User to authenticate as", 2024-06-03T03:28:25.310 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.311 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.311 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.311 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.311 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.311 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.311 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:25.311 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "balancer", 2024-06-03T03:28:25.311 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:25.311 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:25.311 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:25.311 INFO:teuthology.orchestra.run.smithi117.stdout: "active": { 2024-06-03T03:28:25.311 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "active", 2024-06-03T03:28:25.311 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.311 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.312 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.312 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:25.312 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.312 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.312 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.312 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "automatically balance PGs across cluster", 2024-06-03T03:28:25.312 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.312 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.312 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.312 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.312 INFO:teuthology.orchestra.run.smithi117.stdout: "begin_time": { 2024-06-03T03:28:25.312 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "begin_time", 2024-06-03T03:28:25.312 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.312 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.313 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.313 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0000", 2024-06-03T03:28:25.313 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.313 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.313 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.313 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "beginning time of day to automatically balance", 2024-06-03T03:28:25.313 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-03T03:28:25.313 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.313 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.313 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.313 INFO:teuthology.orchestra.run.smithi117.stdout: "begin_weekday": { 2024-06-03T03:28:25.313 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "begin_weekday", 2024-06-03T03:28:25.313 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "uint", 2024-06-03T03:28:25.313 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.314 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.314 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-06-03T03:28:25.314 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "0", 2024-06-03T03:28:25.314 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "6", 2024-06-03T03:28:25.314 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.314 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-06-03T03:28:25.314 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-03T03:28:25.314 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.314 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.314 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.314 INFO:teuthology.orchestra.run.smithi117.stdout: "crush_compat_max_iterations": { 2024-06-03T03:28:25.314 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crush_compat_max_iterations", 2024-06-03T03:28:25.314 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "uint", 2024-06-03T03:28:25.315 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.315 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.315 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "25", 2024-06-03T03:28:25.315 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "1", 2024-06-03T03:28:25.315 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "250", 2024-06-03T03:28:25.315 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.315 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-06-03T03:28:25.315 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.315 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.315 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.315 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.315 INFO:teuthology.orchestra.run.smithi117.stdout: "crush_compat_metrics": { 2024-06-03T03:28:25.315 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crush_compat_metrics", 2024-06-03T03:28:25.316 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.316 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.316 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.316 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "pgs,objects,bytes", 2024-06-03T03:28:25.316 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.316 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.316 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.316 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-06-03T03:28:25.316 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-06-03T03:28:25.316 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.316 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.316 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.316 INFO:teuthology.orchestra.run.smithi117.stdout: "crush_compat_step": { 2024-06-03T03:28:25.316 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crush_compat_step", 2024-06-03T03:28:25.317 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "float", 2024-06-03T03:28:25.317 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.317 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.317 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0.5", 2024-06-03T03:28:25.317 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "0.001", 2024-06-03T03:28:25.317 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "0.999", 2024-06-03T03:28:25.317 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.317 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "aggressiveness of optimization", 2024-06-03T03:28:25.317 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-06-03T03:28:25.317 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.317 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.317 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.317 INFO:teuthology.orchestra.run.smithi117.stdout: "end_time": { 2024-06-03T03:28:25.317 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "end_time", 2024-06-03T03:28:25.318 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.318 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.318 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.318 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "2359", 2024-06-03T03:28:25.318 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.318 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.318 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.318 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "ending time of day to automatically balance", 2024-06-03T03:28:25.318 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-03T03:28:25.318 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.318 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.318 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.318 INFO:teuthology.orchestra.run.smithi117.stdout: "end_weekday": { 2024-06-03T03:28:25.319 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "end_weekday", 2024-06-03T03:28:25.319 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "uint", 2024-06-03T03:28:25.319 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.319 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.319 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-06-03T03:28:25.319 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "0", 2024-06-03T03:28:25.319 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "6", 2024-06-03T03:28:25.319 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.320 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-06-03T03:28:25.320 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-03T03:28:25.320 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.320 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.320 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.320 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:25.320 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:25.320 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.320 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.320 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.320 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.320 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.320 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.321 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.321 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.321 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.321 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.321 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.321 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.321 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.321 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.321 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.321 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.321 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.321 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.321 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.321 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:25.322 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:25.322 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.322 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.322 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.322 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.322 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.322 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.322 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.322 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.322 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.322 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.322 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.323 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.323 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:25.323 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:25.323 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.323 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.323 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.323 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:25.323 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.323 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.323 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.323 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.323 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.323 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.323 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.324 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.324 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.324 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.324 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.324 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.324 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.324 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.324 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.324 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:25.324 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:25.324 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.324 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.324 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.324 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.325 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.325 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.325 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.325 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.325 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.325 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.325 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.325 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.325 INFO:teuthology.orchestra.run.smithi117.stdout: "min_score": { 2024-06-03T03:28:25.325 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "min_score", 2024-06-03T03:28:25.325 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "float", 2024-06-03T03:28:25.325 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.325 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.325 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-06-03T03:28:25.326 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.326 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.326 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.326 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-06-03T03:28:25.326 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.326 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.326 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.326 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.326 INFO:teuthology.orchestra.run.smithi117.stdout: "mode": { 2024-06-03T03:28:25.326 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "mode", 2024-06-03T03:28:25.326 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.326 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.327 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.327 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "upmap", 2024-06-03T03:28:25.327 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.327 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.327 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.327 INFO:teuthology.orchestra.run.smithi117.stdout: "crush-compat", 2024-06-03T03:28:25.327 INFO:teuthology.orchestra.run.smithi117.stdout: "none", 2024-06-03T03:28:25.327 INFO:teuthology.orchestra.run.smithi117.stdout: "read", 2024-06-03T03:28:25.327 INFO:teuthology.orchestra.run.smithi117.stdout: "upmap", 2024-06-03T03:28:25.327 INFO:teuthology.orchestra.run.smithi117.stdout: "upmap-read" 2024-06-03T03:28:25.327 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.327 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Balancer mode", 2024-06-03T03:28:25.327 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.327 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.328 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.328 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.328 INFO:teuthology.orchestra.run.smithi117.stdout: "pool_ids": { 2024-06-03T03:28:25.328 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "pool_ids", 2024-06-03T03:28:25.328 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.328 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.328 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.328 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.328 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.328 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.328 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.328 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-06-03T03:28:25.328 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.328 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.329 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.329 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.329 INFO:teuthology.orchestra.run.smithi117.stdout: "sleep_interval": { 2024-06-03T03:28:25.329 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sleep_interval", 2024-06-03T03:28:25.329 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-06-03T03:28:25.329 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.329 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.329 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "60", 2024-06-03T03:28:25.329 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.329 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.329 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.329 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-06-03T03:28:25.329 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.329 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.330 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.330 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.330 INFO:teuthology.orchestra.run.smithi117.stdout: "upmap_max_deviation": { 2024-06-03T03:28:25.330 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "upmap_max_deviation", 2024-06-03T03:28:25.330 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:25.330 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.330 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.330 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "5", 2024-06-03T03:28:25.330 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "1", 2024-06-03T03:28:25.330 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.330 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.330 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "deviation below which no optimization is attempted", 2024-06-03T03:28:25.330 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-06-03T03:28:25.330 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.331 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.331 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.331 INFO:teuthology.orchestra.run.smithi117.stdout: "upmap_max_optimizations": { 2024-06-03T03:28:25.331 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "upmap_max_optimizations", 2024-06-03T03:28:25.331 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "uint", 2024-06-03T03:28:25.331 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.331 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.331 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "10", 2024-06-03T03:28:25.331 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.331 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.331 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.331 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-06-03T03:28:25.331 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.332 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.332 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.332 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.332 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.332 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.332 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:25.332 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "cephadm", 2024-06-03T03:28:25.332 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:25.332 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:25.332 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:25.332 INFO:teuthology.orchestra.run.smithi117.stdout: "agent_down_multiplier": { 2024-06-03T03:28:25.332 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "agent_down_multiplier", 2024-06-03T03:28:25.332 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "float", 2024-06-03T03:28:25.332 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.333 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.333 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "3.0", 2024-06-03T03:28:25.333 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.333 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.333 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.333 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-06-03T03:28:25.333 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.333 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.333 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.333 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.333 INFO:teuthology.orchestra.run.smithi117.stdout: "agent_refresh_rate": { 2024-06-03T03:28:25.333 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "agent_refresh_rate", 2024-06-03T03:28:25.333 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-06-03T03:28:25.333 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.334 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.334 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "20", 2024-06-03T03:28:25.334 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.334 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.334 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.334 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-06-03T03:28:25.334 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.334 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.334 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.334 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.334 INFO:teuthology.orchestra.run.smithi117.stdout: "agent_starting_port": { 2024-06-03T03:28:25.334 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "agent_starting_port", 2024-06-03T03:28:25.335 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:25.335 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.335 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.335 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "4721", 2024-06-03T03:28:25.335 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.335 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.335 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.335 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-06-03T03:28:25.335 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.335 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.335 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.335 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.335 INFO:teuthology.orchestra.run.smithi117.stdout: "allow_ptrace": { 2024-06-03T03:28:25.336 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "allow_ptrace", 2024-06-03T03:28:25.336 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.336 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.336 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.336 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.336 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.336 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.336 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.336 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-06-03T03:28:25.336 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-06-03T03:28:25.336 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.336 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.336 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.337 INFO:teuthology.orchestra.run.smithi117.stdout: "autotune_interval": { 2024-06-03T03:28:25.337 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "autotune_interval", 2024-06-03T03:28:25.337 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-06-03T03:28:25.337 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.337 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.337 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "600", 2024-06-03T03:28:25.337 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.337 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.337 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.337 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "how frequently to autotune daemon memory", 2024-06-03T03:28:25.337 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.337 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.337 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.337 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.338 INFO:teuthology.orchestra.run.smithi117.stdout: "autotune_memory_target_ratio": { 2024-06-03T03:28:25.338 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "autotune_memory_target_ratio", 2024-06-03T03:28:25.338 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "float", 2024-06-03T03:28:25.338 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.338 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.338 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0.7", 2024-06-03T03:28:25.338 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.338 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.338 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.338 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-06-03T03:28:25.338 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.338 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.339 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.339 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.339 INFO:teuthology.orchestra.run.smithi117.stdout: "cephadm_log_destination": { 2024-06-03T03:28:25.339 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "cephadm_log_destination", 2024-06-03T03:28:25.339 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.339 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.339 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.339 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.339 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.339 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.339 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.339 INFO:teuthology.orchestra.run.smithi117.stdout: "file", 2024-06-03T03:28:25.339 INFO:teuthology.orchestra.run.smithi117.stdout: "file,syslog", 2024-06-03T03:28:25.340 INFO:teuthology.orchestra.run.smithi117.stdout: "syslog" 2024-06-03T03:28:25.340 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.340 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-06-03T03:28:25.340 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.340 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.340 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.340 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.340 INFO:teuthology.orchestra.run.smithi117.stdout: "cgroups_split": { 2024-06-03T03:28:25.340 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "cgroups_split", 2024-06-03T03:28:25.340 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.340 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.340 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.340 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:25.340 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.341 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.341 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.341 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-06-03T03:28:25.341 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.341 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.341 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.341 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.342 INFO:teuthology.orchestra.run.smithi117.stdout: "config_checks_enabled": { 2024-06-03T03:28:25.342 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "config_checks_enabled", 2024-06-03T03:28:25.342 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.342 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.342 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.342 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.342 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.342 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.342 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.343 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-06-03T03:28:25.343 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.343 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.343 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.343 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.343 INFO:teuthology.orchestra.run.smithi117.stdout: "config_dashboard": { 2024-06-03T03:28:25.343 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "config_dashboard", 2024-06-03T03:28:25.343 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.343 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.343 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.343 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:25.343 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.343 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.344 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.344 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-06-03T03:28:25.344 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.344 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.344 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.344 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.344 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_alertmanager": { 2024-06-03T03:28:25.344 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_alertmanager", 2024-06-03T03:28:25.344 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.344 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.344 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.344 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-06-03T03:28:25.344 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.344 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.345 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.345 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Prometheus container image", 2024-06-03T03:28:25.345 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.345 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.345 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.345 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.345 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_base": { 2024-06-03T03:28:25.345 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_base", 2024-06-03T03:28:25.345 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.345 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.345 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.345 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/ceph/ceph", 2024-06-03T03:28:25.345 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.346 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.346 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.346 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Container image name, without the tag", 2024-06-03T03:28:25.346 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.346 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.346 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.346 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.346 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_elasticsearch": { 2024-06-03T03:28:25.346 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_elasticsearch", 2024-06-03T03:28:25.346 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.346 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.346 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.346 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-06-03T03:28:25.347 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.347 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.347 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.347 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "elasticsearch container image", 2024-06-03T03:28:25.347 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.347 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.347 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.347 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.347 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_grafana": { 2024-06-03T03:28:25.347 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_grafana", 2024-06-03T03:28:25.347 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.347 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.347 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.347 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-06-03T03:28:25.348 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.348 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.348 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.348 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Prometheus container image", 2024-06-03T03:28:25.348 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.348 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.348 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.348 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.348 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_haproxy": { 2024-06-03T03:28:25.348 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_haproxy", 2024-06-03T03:28:25.348 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.348 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.348 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.348 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-06-03T03:28:25.349 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.349 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.349 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.349 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "HAproxy container image", 2024-06-03T03:28:25.349 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.349 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.349 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.349 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.349 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_jaeger_agent": { 2024-06-03T03:28:25.349 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_jaeger_agent", 2024-06-03T03:28:25.349 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.349 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.350 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.350 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-06-03T03:28:25.350 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.350 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.350 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.350 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Jaeger agent container image", 2024-06-03T03:28:25.350 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.350 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.350 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.350 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.350 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_jaeger_collector": { 2024-06-03T03:28:25.350 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_jaeger_collector", 2024-06-03T03:28:25.350 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.351 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.351 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.351 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-06-03T03:28:25.351 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.351 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.351 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.351 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Jaeger collector container image", 2024-06-03T03:28:25.351 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.351 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.351 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.351 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.351 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_jaeger_query": { 2024-06-03T03:28:25.351 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_jaeger_query", 2024-06-03T03:28:25.352 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.352 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.352 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.352 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-06-03T03:28:25.352 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.352 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.352 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.352 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Jaeger query container image", 2024-06-03T03:28:25.352 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.352 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.352 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.352 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.352 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_keepalived": { 2024-06-03T03:28:25.353 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_keepalived", 2024-06-03T03:28:25.353 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.353 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.353 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.353 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-06-03T03:28:25.353 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.353 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.353 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.353 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Keepalived container image", 2024-06-03T03:28:25.353 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.353 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.353 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.353 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.353 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_loki": { 2024-06-03T03:28:25.354 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_loki", 2024-06-03T03:28:25.354 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.354 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.354 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.354 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-06-03T03:28:25.354 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.354 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.354 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.354 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Loki container image", 2024-06-03T03:28:25.354 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.354 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.354 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.355 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.355 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_node_exporter": { 2024-06-03T03:28:25.355 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_node_exporter", 2024-06-03T03:28:25.355 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.355 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.355 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.355 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-06-03T03:28:25.355 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.355 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.355 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.355 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Prometheus container image", 2024-06-03T03:28:25.355 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.355 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.355 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.356 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.356 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_nvmeof": { 2024-06-03T03:28:25.356 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_nvmeof", 2024-06-03T03:28:25.356 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.356 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.356 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.356 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-06-03T03:28:25.356 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.356 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.356 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.356 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Nvme-of container image", 2024-06-03T03:28:25.356 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.356 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.356 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.357 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.357 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_prometheus": { 2024-06-03T03:28:25.357 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_prometheus", 2024-06-03T03:28:25.357 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.357 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.357 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.357 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-06-03T03:28:25.357 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.357 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.357 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.357 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Prometheus container image", 2024-06-03T03:28:25.357 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.357 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.358 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.358 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.358 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_promtail": { 2024-06-03T03:28:25.358 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_promtail", 2024-06-03T03:28:25.358 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.358 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.358 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.358 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-06-03T03:28:25.358 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.358 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.358 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.358 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Promtail container image", 2024-06-03T03:28:25.358 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.359 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.359 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.359 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.359 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_samba": { 2024-06-03T03:28:25.359 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_samba", 2024-06-03T03:28:25.359 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.359 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.359 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.359 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-06-03T03:28:25.359 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.359 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.359 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.359 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Samba/SMB container image", 2024-06-03T03:28:25.359 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.360 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.360 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.360 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.360 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_snmp_gateway": { 2024-06-03T03:28:25.360 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_snmp_gateway", 2024-06-03T03:28:25.360 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.360 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.360 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.360 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-06-03T03:28:25.360 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.360 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.361 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.361 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "SNMP Gateway container image", 2024-06-03T03:28:25.361 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.361 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.361 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.361 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.361 INFO:teuthology.orchestra.run.smithi117.stdout: "container_init": { 2024-06-03T03:28:25.361 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_init", 2024-06-03T03:28:25.361 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.361 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.361 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.361 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:25.362 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.362 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.362 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.362 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Run podman/docker with `--init`", 2024-06-03T03:28:25.362 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.362 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.362 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.362 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.362 INFO:teuthology.orchestra.run.smithi117.stdout: "daemon_cache_timeout": { 2024-06-03T03:28:25.362 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "daemon_cache_timeout", 2024-06-03T03:28:25.362 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-06-03T03:28:25.362 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.363 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.363 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "600", 2024-06-03T03:28:25.363 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.363 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.363 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.363 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-06-03T03:28:25.363 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.363 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.363 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.363 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.363 INFO:teuthology.orchestra.run.smithi117.stdout: "default_cephadm_command_timeout": { 2024-06-03T03:28:25.363 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "default_cephadm_command_timeout", 2024-06-03T03:28:25.363 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:25.364 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.364 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.364 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "900", 2024-06-03T03:28:25.364 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.364 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.364 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.364 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-06-03T03:28:25.364 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.364 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.364 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.364 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.364 INFO:teuthology.orchestra.run.smithi117.stdout: "default_registry": { 2024-06-03T03:28:25.364 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "default_registry", 2024-06-03T03:28:25.365 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.365 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.365 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.365 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "docker.io", 2024-06-03T03:28:25.365 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.365 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.365 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.365 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-06-03T03:28:25.365 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.365 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.365 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.365 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.365 INFO:teuthology.orchestra.run.smithi117.stdout: "device_cache_timeout": { 2024-06-03T03:28:25.366 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "device_cache_timeout", 2024-06-03T03:28:25.366 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-06-03T03:28:25.366 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.366 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.366 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "1800", 2024-06-03T03:28:25.366 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.366 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.366 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.366 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "seconds to cache device inventory", 2024-06-03T03:28:25.366 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.366 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.366 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.366 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.367 INFO:teuthology.orchestra.run.smithi117.stdout: "device_enhanced_scan": { 2024-06-03T03:28:25.367 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "device_enhanced_scan", 2024-06-03T03:28:25.367 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.367 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.367 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.367 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.367 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.367 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.367 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.367 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Use libstoragemgmt during device scans", 2024-06-03T03:28:25.367 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.367 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.368 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.368 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.368 INFO:teuthology.orchestra.run.smithi117.stdout: "facts_cache_timeout": { 2024-06-03T03:28:25.368 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "facts_cache_timeout", 2024-06-03T03:28:25.368 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-06-03T03:28:25.368 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.368 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.368 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "60", 2024-06-03T03:28:25.368 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.368 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.368 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.368 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "seconds to cache host facts data", 2024-06-03T03:28:25.368 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.369 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.369 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.369 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.369 INFO:teuthology.orchestra.run.smithi117.stdout: "grafana_dashboards_path": { 2024-06-03T03:28:25.369 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "grafana_dashboards_path", 2024-06-03T03:28:25.369 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.369 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.369 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.369 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-06-03T03:28:25.369 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.369 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.369 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.369 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-06-03T03:28:25.369 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.370 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.370 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.370 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.370 INFO:teuthology.orchestra.run.smithi117.stdout: "host_check_interval": { 2024-06-03T03:28:25.370 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "host_check_interval", 2024-06-03T03:28:25.370 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-06-03T03:28:25.370 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.370 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.370 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "600", 2024-06-03T03:28:25.370 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.370 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.370 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.370 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "how frequently to perform a host check", 2024-06-03T03:28:25.371 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.371 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.371 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.371 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.371 INFO:teuthology.orchestra.run.smithi117.stdout: "hw_monitoring": { 2024-06-03T03:28:25.371 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "hw_monitoring", 2024-06-03T03:28:25.371 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.371 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.371 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.371 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.371 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.371 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.371 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.371 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-06-03T03:28:25.372 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.372 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.372 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.372 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.372 INFO:teuthology.orchestra.run.smithi117.stdout: "inventory_list_all": { 2024-06-03T03:28:25.372 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "inventory_list_all", 2024-06-03T03:28:25.372 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.372 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.372 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.372 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.372 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.372 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.372 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.372 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-06-03T03:28:25.373 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.373 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.373 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.373 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.373 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:25.373 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:25.373 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.373 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.373 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.373 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.373 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.373 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.373 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.374 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.374 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.374 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.374 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.374 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.374 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.374 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.374 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.374 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.374 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.374 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.374 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.374 INFO:teuthology.orchestra.run.smithi117.stdout: "log_refresh_metadata": { 2024-06-03T03:28:25.375 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_refresh_metadata", 2024-06-03T03:28:25.375 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.375 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.375 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.375 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.375 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.375 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.375 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.375 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-06-03T03:28:25.375 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.375 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.375 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.376 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.376 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:25.376 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:25.376 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.376 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.376 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.376 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:25.376 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.376 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.376 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.376 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-06-03T03:28:25.376 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.376 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.376 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.377 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.377 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:25.377 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:25.377 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.377 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.377 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.377 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:25.377 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.377 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.377 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.377 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.377 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.377 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.377 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.378 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.378 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.378 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.378 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.378 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.378 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.378 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.378 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.378 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:25.378 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:25.378 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.378 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.378 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.379 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.379 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.379 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.379 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.379 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.379 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.379 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.379 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.379 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.379 INFO:teuthology.orchestra.run.smithi117.stdout: "manage_etc_ceph_ceph_conf": { 2024-06-03T03:28:25.379 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-06-03T03:28:25.379 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.379 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.380 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.380 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.380 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.380 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.380 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.380 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-06-03T03:28:25.380 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.380 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.380 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.380 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.380 INFO:teuthology.orchestra.run.smithi117.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-06-03T03:28:25.380 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-06-03T03:28:25.380 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.380 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.381 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.381 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "*", 2024-06-03T03:28:25.381 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.381 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.381 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.381 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-06-03T03:28:25.381 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.381 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.381 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.381 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.381 INFO:teuthology.orchestra.run.smithi117.stdout: "max_count_per_host": { 2024-06-03T03:28:25.381 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "max_count_per_host", 2024-06-03T03:28:25.381 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:25.382 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.382 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.382 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "10", 2024-06-03T03:28:25.382 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.382 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.382 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.382 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "max number of daemons per service per host", 2024-06-03T03:28:25.382 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.382 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.382 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.382 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.382 INFO:teuthology.orchestra.run.smithi117.stdout: "max_osd_draining_count": { 2024-06-03T03:28:25.382 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "max_osd_draining_count", 2024-06-03T03:28:25.382 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:25.383 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.383 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.383 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "10", 2024-06-03T03:28:25.383 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.383 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.383 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.383 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-06-03T03:28:25.383 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.383 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.383 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.383 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.383 INFO:teuthology.orchestra.run.smithi117.stdout: "migration_current": { 2024-06-03T03:28:25.383 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "migration_current", 2024-06-03T03:28:25.384 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:25.384 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.384 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.384 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.384 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.384 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.384 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.384 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "internal - do not modify", 2024-06-03T03:28:25.384 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.384 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.384 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.384 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.384 INFO:teuthology.orchestra.run.smithi117.stdout: "mode": { 2024-06-03T03:28:25.384 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "mode", 2024-06-03T03:28:25.385 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.385 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.385 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.385 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "root", 2024-06-03T03:28:25.385 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.385 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.385 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.385 INFO:teuthology.orchestra.run.smithi117.stdout: "cephadm-package", 2024-06-03T03:28:25.385 INFO:teuthology.orchestra.run.smithi117.stdout: "root" 2024-06-03T03:28:25.385 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.385 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "mode for remote execution of cephadm", 2024-06-03T03:28:25.385 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.385 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.386 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.386 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.386 INFO:teuthology.orchestra.run.smithi117.stdout: "oob_default_addr": { 2024-06-03T03:28:25.386 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "oob_default_addr", 2024-06-03T03:28:25.386 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.386 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.386 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.386 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "169.254.1.1", 2024-06-03T03:28:25.386 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.386 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.386 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.386 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Default address for RedFish API (oob management).", 2024-06-03T03:28:25.386 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.387 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.387 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.387 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.387 INFO:teuthology.orchestra.run.smithi117.stdout: "prometheus_alerts_path": { 2024-06-03T03:28:25.387 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "prometheus_alerts_path", 2024-06-03T03:28:25.387 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.387 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.387 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.387 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-06-03T03:28:25.387 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.387 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.387 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.387 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-06-03T03:28:25.387 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.388 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.388 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.388 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.388 INFO:teuthology.orchestra.run.smithi117.stdout: "registry_insecure": { 2024-06-03T03:28:25.388 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "registry_insecure", 2024-06-03T03:28:25.388 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.388 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.388 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.388 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.388 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.388 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.388 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.388 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-06-03T03:28:25.389 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.389 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.389 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.389 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.389 INFO:teuthology.orchestra.run.smithi117.stdout: "registry_password": { 2024-06-03T03:28:25.389 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "registry_password", 2024-06-03T03:28:25.389 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.389 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.389 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.389 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.389 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.389 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.389 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.390 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-06-03T03:28:25.390 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.390 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.390 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.390 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.390 INFO:teuthology.orchestra.run.smithi117.stdout: "registry_url": { 2024-06-03T03:28:25.390 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "registry_url", 2024-06-03T03:28:25.390 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.390 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.390 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.390 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.390 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.391 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.391 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.391 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-06-03T03:28:25.391 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.391 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.391 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.391 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.391 INFO:teuthology.orchestra.run.smithi117.stdout: "registry_username": { 2024-06-03T03:28:25.391 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "registry_username", 2024-06-03T03:28:25.391 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.391 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.391 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.391 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.391 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.392 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.392 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.392 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-06-03T03:28:25.392 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.392 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.392 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.392 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.392 INFO:teuthology.orchestra.run.smithi117.stdout: "secure_monitoring_stack": { 2024-06-03T03:28:25.392 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "secure_monitoring_stack", 2024-06-03T03:28:25.392 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.392 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.392 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.393 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.393 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.393 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.393 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.393 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-06-03T03:28:25.393 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.393 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.393 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.393 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.393 INFO:teuthology.orchestra.run.smithi117.stdout: "service_discovery_port": { 2024-06-03T03:28:25.393 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "service_discovery_port", 2024-06-03T03:28:25.393 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:25.393 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.393 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.394 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "8765", 2024-06-03T03:28:25.394 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.394 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.394 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.394 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "cephadm service discovery port", 2024-06-03T03:28:25.394 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.394 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.394 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.394 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.394 INFO:teuthology.orchestra.run.smithi117.stdout: "ssh_config_file": { 2024-06-03T03:28:25.394 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "ssh_config_file", 2024-06-03T03:28:25.394 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.394 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.395 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.395 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.395 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.395 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.395 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.395 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-06-03T03:28:25.395 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.395 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.395 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.395 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.395 INFO:teuthology.orchestra.run.smithi117.stdout: "use_agent": { 2024-06-03T03:28:25.395 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "use_agent", 2024-06-03T03:28:25.395 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.395 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.396 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.396 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.396 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.396 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.396 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.396 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-06-03T03:28:25.396 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.396 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.396 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.396 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.396 INFO:teuthology.orchestra.run.smithi117.stdout: "use_repo_digest": { 2024-06-03T03:28:25.396 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "use_repo_digest", 2024-06-03T03:28:25.396 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.397 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.397 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.397 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:25.397 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.397 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.397 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.397 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-06-03T03:28:25.397 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.397 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.397 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.397 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.397 INFO:teuthology.orchestra.run.smithi117.stdout: "warn_on_failed_host_check": { 2024-06-03T03:28:25.397 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "warn_on_failed_host_check", 2024-06-03T03:28:25.398 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.398 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.398 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.398 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:25.398 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.398 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.398 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.398 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "raise a health warning if the host check fails", 2024-06-03T03:28:25.398 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.398 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.398 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.398 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.398 INFO:teuthology.orchestra.run.smithi117.stdout: "warn_on_stray_daemons": { 2024-06-03T03:28:25.399 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "warn_on_stray_daemons", 2024-06-03T03:28:25.399 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.399 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.399 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.399 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:25.399 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.399 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.399 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.399 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-06-03T03:28:25.399 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.399 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.399 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.399 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.399 INFO:teuthology.orchestra.run.smithi117.stdout: "warn_on_stray_hosts": { 2024-06-03T03:28:25.400 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "warn_on_stray_hosts", 2024-06-03T03:28:25.400 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.400 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.400 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.400 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:25.400 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.400 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.400 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.400 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-06-03T03:28:25.400 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.400 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.400 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.400 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.401 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.401 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.401 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:25.401 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crash", 2024-06-03T03:28:25.401 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:25.401 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:25.401 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:25.401 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:25.401 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:25.401 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.401 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.401 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.401 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.402 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.402 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.402 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.402 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.402 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.402 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.402 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.402 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.402 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.402 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.402 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.402 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.402 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.403 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.403 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.403 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:25.406 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:25.406 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.406 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.406 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.406 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.406 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.406 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.406 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.406 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.406 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.406 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.406 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.406 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.407 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:25.407 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:25.407 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.407 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.407 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.407 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:25.407 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.407 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.407 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.407 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.407 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.407 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.408 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.408 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.408 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.408 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.408 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.408 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.408 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.408 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.408 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.408 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:25.408 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:25.408 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.408 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.408 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.409 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.409 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.409 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.409 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.409 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.409 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.409 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.409 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.409 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.409 INFO:teuthology.orchestra.run.smithi117.stdout: "retain_interval": { 2024-06-03T03:28:25.409 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "retain_interval", 2024-06-03T03:28:25.409 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-06-03T03:28:25.409 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.410 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.410 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "31536000", 2024-06-03T03:28:25.410 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.410 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.410 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.410 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "how long to retain crashes before pruning them", 2024-06-03T03:28:25.410 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.410 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.410 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.410 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.410 INFO:teuthology.orchestra.run.smithi117.stdout: "warn_recent_interval": { 2024-06-03T03:28:25.410 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "warn_recent_interval", 2024-06-03T03:28:25.410 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-06-03T03:28:25.411 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.411 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.411 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "1209600", 2024-06-03T03:28:25.411 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.411 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.411 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.411 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "time interval in which to warn about recent crashes", 2024-06-03T03:28:25.411 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.411 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.411 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.411 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.411 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.411 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.412 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:25.412 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "dashboard", 2024-06-03T03:28:25.412 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:25.412 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:25.412 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:25.412 INFO:teuthology.orchestra.run.smithi117.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-06-03T03:28:25.412 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-06-03T03:28:25.412 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:25.412 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.412 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.412 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "10", 2024-06-03T03:28:25.412 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.412 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.412 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.413 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.413 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.413 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.413 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.413 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.413 INFO:teuthology.orchestra.run.smithi117.stdout: "ALERTMANAGER_API_HOST": { 2024-06-03T03:28:25.413 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "ALERTMANAGER_API_HOST", 2024-06-03T03:28:25.413 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.413 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.413 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.413 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.413 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.413 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.413 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.414 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.414 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.414 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.414 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.414 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.414 INFO:teuthology.orchestra.run.smithi117.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-06-03T03:28:25.414 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-06-03T03:28:25.414 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.414 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.414 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.414 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:25.414 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.414 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.415 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.415 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.415 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.415 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.415 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.415 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.415 INFO:teuthology.orchestra.run.smithi117.stdout: "AUDIT_API_ENABLED": { 2024-06-03T03:28:25.415 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "AUDIT_API_ENABLED", 2024-06-03T03:28:25.415 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.415 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.415 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.415 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.415 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.416 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.416 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.416 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.416 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.416 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.416 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.416 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.416 INFO:teuthology.orchestra.run.smithi117.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-06-03T03:28:25.416 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-06-03T03:28:25.416 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.416 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.416 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.416 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:25.417 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.417 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.417 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.417 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.417 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.417 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.417 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.417 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.417 INFO:teuthology.orchestra.run.smithi117.stdout: "ENABLE_BROWSABLE_API": { 2024-06-03T03:28:25.417 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "ENABLE_BROWSABLE_API", 2024-06-03T03:28:25.417 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.417 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.417 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.417 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:25.418 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.418 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.418 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.418 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.418 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.418 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.418 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.418 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.418 INFO:teuthology.orchestra.run.smithi117.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-06-03T03:28:25.418 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-06-03T03:28:25.418 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.418 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.419 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.419 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:25.419 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.419 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.419 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.419 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.419 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.419 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.419 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.419 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.419 INFO:teuthology.orchestra.run.smithi117.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-06-03T03:28:25.419 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-06-03T03:28:25.419 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.420 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.420 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.420 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:25.420 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.420 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.420 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.420 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.420 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.420 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.420 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.420 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.420 INFO:teuthology.orchestra.run.smithi117.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-06-03T03:28:25.420 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-06-03T03:28:25.420 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.421 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.421 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.421 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:25.421 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.421 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.421 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.421 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.421 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.421 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.421 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.421 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.421 INFO:teuthology.orchestra.run.smithi117.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-06-03T03:28:25.421 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-06-03T03:28:25.421 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.422 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.422 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.422 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:25.422 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.422 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.422 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.422 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.422 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.422 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.422 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.422 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.422 INFO:teuthology.orchestra.run.smithi117.stdout: "FEATURE_TOGGLE_NFS": { 2024-06-03T03:28:25.422 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-06-03T03:28:25.423 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.423 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.423 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.423 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:25.423 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.423 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.423 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.423 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.423 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.423 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.423 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.423 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.423 INFO:teuthology.orchestra.run.smithi117.stdout: "FEATURE_TOGGLE_RBD": { 2024-06-03T03:28:25.424 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-06-03T03:28:25.424 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.424 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.424 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.424 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:25.424 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.424 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.424 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.424 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.424 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.424 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.424 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.424 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.424 INFO:teuthology.orchestra.run.smithi117.stdout: "FEATURE_TOGGLE_RGW": { 2024-06-03T03:28:25.425 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-06-03T03:28:25.425 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.425 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.425 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.425 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:25.425 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.425 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.425 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.425 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.425 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.425 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.425 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.425 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.425 INFO:teuthology.orchestra.run.smithi117.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-06-03T03:28:25.426 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-06-03T03:28:25.426 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.426 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.426 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.426 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.426 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.426 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.426 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.426 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.426 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.426 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.426 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.426 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.427 INFO:teuthology.orchestra.run.smithi117.stdout: "GRAFANA_API_PASSWORD": { 2024-06-03T03:28:25.427 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "GRAFANA_API_PASSWORD", 2024-06-03T03:28:25.427 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.427 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.427 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.427 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "admin", 2024-06-03T03:28:25.427 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.427 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.427 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.427 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.427 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.427 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.427 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.427 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.428 INFO:teuthology.orchestra.run.smithi117.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-06-03T03:28:25.428 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-06-03T03:28:25.428 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.428 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.428 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.428 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:25.428 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.428 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.428 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.428 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.428 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.428 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.428 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.429 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.429 INFO:teuthology.orchestra.run.smithi117.stdout: "GRAFANA_API_URL": { 2024-06-03T03:28:25.429 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "GRAFANA_API_URL", 2024-06-03T03:28:25.429 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.429 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.429 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.429 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.429 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.429 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.429 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.429 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.429 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.429 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.430 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.430 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.430 INFO:teuthology.orchestra.run.smithi117.stdout: "GRAFANA_API_USERNAME": { 2024-06-03T03:28:25.430 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "GRAFANA_API_USERNAME", 2024-06-03T03:28:25.430 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.430 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.430 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.430 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "admin", 2024-06-03T03:28:25.430 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.430 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.430 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.430 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.430 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.431 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.431 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.431 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.431 INFO:teuthology.orchestra.run.smithi117.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-06-03T03:28:25.431 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-06-03T03:28:25.431 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.431 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.431 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.431 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.431 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.431 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.431 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.431 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.431 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.432 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.432 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.432 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.432 INFO:teuthology.orchestra.run.smithi117.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-06-03T03:28:25.432 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-06-03T03:28:25.432 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.432 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.432 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.432 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.432 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.432 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.432 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.432 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.433 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.433 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.433 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.433 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.433 INFO:teuthology.orchestra.run.smithi117.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-06-03T03:28:25.433 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-06-03T03:28:25.433 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.433 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.433 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.433 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:25.433 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.433 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.433 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.433 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.434 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.434 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.434 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.434 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.434 INFO:teuthology.orchestra.run.smithi117.stdout: "ISSUE_TRACKER_API_KEY": { 2024-06-03T03:28:25.434 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-06-03T03:28:25.434 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.434 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.434 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.434 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.434 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.434 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.434 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.435 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.435 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.435 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.435 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.435 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.435 INFO:teuthology.orchestra.run.smithi117.stdout: "PROMETHEUS_API_HOST": { 2024-06-03T03:28:25.435 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "PROMETHEUS_API_HOST", 2024-06-03T03:28:25.435 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.435 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.435 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.435 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.435 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.435 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.436 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.436 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.436 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.436 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.436 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.436 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.436 INFO:teuthology.orchestra.run.smithi117.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-06-03T03:28:25.436 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-06-03T03:28:25.436 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.436 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.436 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.436 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:25.436 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.437 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.437 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.437 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.437 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.437 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.437 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.437 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.437 INFO:teuthology.orchestra.run.smithi117.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-06-03T03:28:25.437 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-06-03T03:28:25.437 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.437 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.437 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.437 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.438 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.438 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.438 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.438 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.438 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.438 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.438 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.438 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.438 INFO:teuthology.orchestra.run.smithi117.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-06-03T03:28:25.438 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-06-03T03:28:25.438 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.438 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.438 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.439 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.439 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.439 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.439 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.439 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.439 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.439 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.439 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.439 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.439 INFO:teuthology.orchestra.run.smithi117.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-06-03T03:28:25.439 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-06-03T03:28:25.439 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.439 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.440 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.440 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:25.440 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.440 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.440 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.440 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.440 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.440 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.440 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.440 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.440 INFO:teuthology.orchestra.run.smithi117.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-06-03T03:28:25.440 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-06-03T03:28:25.440 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.440 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.441 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.441 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:25.441 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.441 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.441 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.441 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.441 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.441 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.441 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.441 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.441 INFO:teuthology.orchestra.run.smithi117.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-06-03T03:28:25.441 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-06-03T03:28:25.441 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.441 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.442 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.442 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.442 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.442 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.442 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.442 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.442 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.442 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.442 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.442 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.442 INFO:teuthology.orchestra.run.smithi117.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-06-03T03:28:25.442 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-06-03T03:28:25.442 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.443 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.443 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.443 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.443 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.443 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.443 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.443 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.443 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.443 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.443 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.443 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.443 INFO:teuthology.orchestra.run.smithi117.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-06-03T03:28:25.443 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-06-03T03:28:25.444 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.444 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.444 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.444 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.444 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.444 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.444 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.444 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.444 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.444 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.444 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.444 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.444 INFO:teuthology.orchestra.run.smithi117.stdout: "PWD_POLICY_ENABLED": { 2024-06-03T03:28:25.444 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "PWD_POLICY_ENABLED", 2024-06-03T03:28:25.445 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.445 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.445 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.445 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:25.445 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.445 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.445 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.445 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.445 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.445 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.445 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.445 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.445 INFO:teuthology.orchestra.run.smithi117.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-06-03T03:28:25.446 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-06-03T03:28:25.446 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.446 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.446 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.446 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-06-03T03:28:25.446 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.446 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.446 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.446 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.446 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.446 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.446 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.446 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.447 INFO:teuthology.orchestra.run.smithi117.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-06-03T03:28:25.447 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-06-03T03:28:25.447 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:25.447 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.447 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.447 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "10", 2024-06-03T03:28:25.447 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.447 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.447 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.447 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.447 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.447 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.447 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.448 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.448 INFO:teuthology.orchestra.run.smithi117.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-06-03T03:28:25.448 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-06-03T03:28:25.448 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:25.448 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.448 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.448 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "8", 2024-06-03T03:28:25.448 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.448 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.448 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.448 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.448 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.448 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.448 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.449 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.449 INFO:teuthology.orchestra.run.smithi117.stdout: "REST_REQUESTS_TIMEOUT": { 2024-06-03T03:28:25.449 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-06-03T03:28:25.449 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:25.449 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.449 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.449 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "45", 2024-06-03T03:28:25.449 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.449 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.449 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.449 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.449 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.449 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.450 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.450 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.450 INFO:teuthology.orchestra.run.smithi117.stdout: "RGW_API_ACCESS_KEY": { 2024-06-03T03:28:25.450 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "RGW_API_ACCESS_KEY", 2024-06-03T03:28:25.450 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.450 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.450 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.450 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.450 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.450 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.450 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.450 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.450 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.450 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.451 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.451 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.451 INFO:teuthology.orchestra.run.smithi117.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-06-03T03:28:25.451 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-06-03T03:28:25.451 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.451 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.451 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.451 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "admin", 2024-06-03T03:28:25.451 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.451 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.451 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.451 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.451 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.451 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.452 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.452 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.452 INFO:teuthology.orchestra.run.smithi117.stdout: "RGW_API_SECRET_KEY": { 2024-06-03T03:28:25.452 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "RGW_API_SECRET_KEY", 2024-06-03T03:28:25.452 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.452 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.452 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.452 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.452 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.452 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.452 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.452 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.452 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.453 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.453 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.453 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.453 INFO:teuthology.orchestra.run.smithi117.stdout: "RGW_API_SSL_VERIFY": { 2024-06-03T03:28:25.453 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "RGW_API_SSL_VERIFY", 2024-06-03T03:28:25.453 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.453 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.453 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.453 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:25.453 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.453 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.453 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.453 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.453 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.454 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.454 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.454 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.454 INFO:teuthology.orchestra.run.smithi117.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-06-03T03:28:25.454 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-06-03T03:28:25.454 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:25.454 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.454 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.454 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-06-03T03:28:25.454 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.454 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.454 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.454 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.454 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.455 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.455 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.455 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.455 INFO:teuthology.orchestra.run.smithi117.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-06-03T03:28:25.455 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-06-03T03:28:25.455 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:25.455 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.455 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.455 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "10", 2024-06-03T03:28:25.455 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.455 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.455 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.455 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.455 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.456 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.456 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.456 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.456 INFO:teuthology.orchestra.run.smithi117.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-06-03T03:28:25.456 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-06-03T03:28:25.456 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:25.456 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.456 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.456 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "5", 2024-06-03T03:28:25.456 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.456 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.456 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.456 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.456 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.457 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.457 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.457 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.457 INFO:teuthology.orchestra.run.smithi117.stdout: "cross_origin_url": { 2024-06-03T03:28:25.457 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "cross_origin_url", 2024-06-03T03:28:25.457 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.457 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.457 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.457 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.457 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.457 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.457 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.457 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.457 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.458 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.458 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.458 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.458 INFO:teuthology.orchestra.run.smithi117.stdout: "crt_file": { 2024-06-03T03:28:25.458 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crt_file", 2024-06-03T03:28:25.458 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.458 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.458 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.458 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.458 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.458 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.458 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.458 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.459 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.459 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.459 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.459 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.459 INFO:teuthology.orchestra.run.smithi117.stdout: "debug": { 2024-06-03T03:28:25.459 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "debug", 2024-06-03T03:28:25.459 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.459 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.459 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.459 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.459 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.459 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.459 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.459 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Enable/disable debug options", 2024-06-03T03:28:25.460 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.460 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.460 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.460 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.460 INFO:teuthology.orchestra.run.smithi117.stdout: "jwt_token_ttl": { 2024-06-03T03:28:25.460 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "jwt_token_ttl", 2024-06-03T03:28:25.460 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:25.460 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.460 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.460 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "28800", 2024-06-03T03:28:25.460 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.461 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.461 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.461 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.461 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.461 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.461 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.461 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.461 INFO:teuthology.orchestra.run.smithi117.stdout: "key_file": { 2024-06-03T03:28:25.461 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "key_file", 2024-06-03T03:28:25.461 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.461 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.461 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.461 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.462 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.462 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.462 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.462 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.462 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.462 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.462 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.462 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.462 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:25.462 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:25.462 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.462 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.462 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.463 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.463 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.463 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.463 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.463 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.463 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.463 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.463 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.463 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.463 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.463 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.463 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.463 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.464 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.464 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.464 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.464 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:25.464 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:25.464 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.464 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.464 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.464 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.464 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.464 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.464 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.464 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.464 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.465 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.465 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.465 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.465 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:25.465 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:25.465 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.465 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.465 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.465 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:25.465 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.465 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.465 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.465 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.466 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.466 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.466 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.466 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.466 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.466 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.466 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.466 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.466 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.466 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.466 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.466 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:25.466 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:25.466 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.467 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.467 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.467 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.467 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.467 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.467 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.467 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.467 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.467 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.467 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.467 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.467 INFO:teuthology.orchestra.run.smithi117.stdout: "motd": { 2024-06-03T03:28:25.467 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "motd", 2024-06-03T03:28:25.468 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.468 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.468 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.468 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.468 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.468 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.468 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.468 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "The message of the day", 2024-06-03T03:28:25.468 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.468 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.468 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.468 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.468 INFO:teuthology.orchestra.run.smithi117.stdout: "redirect_resolve_ip_addr": { 2024-06-03T03:28:25.468 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "redirect_resolve_ip_addr", 2024-06-03T03:28:25.469 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.469 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.469 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.469 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.469 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.469 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.469 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.469 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.469 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.469 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.469 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.469 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.469 INFO:teuthology.orchestra.run.smithi117.stdout: "server_addr": { 2024-06-03T03:28:25.469 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "server_addr", 2024-06-03T03:28:25.470 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.470 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.470 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.470 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "::", 2024-06-03T03:28:25.470 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.470 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.470 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.470 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.470 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.470 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.470 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.470 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.470 INFO:teuthology.orchestra.run.smithi117.stdout: "server_port": { 2024-06-03T03:28:25.471 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "server_port", 2024-06-03T03:28:25.471 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:25.471 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.471 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.471 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "8080", 2024-06-03T03:28:25.471 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.471 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.471 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.471 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.471 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.471 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.471 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.471 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.471 INFO:teuthology.orchestra.run.smithi117.stdout: "ssl": { 2024-06-03T03:28:25.472 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "ssl", 2024-06-03T03:28:25.472 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.472 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.472 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.472 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:25.472 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.472 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.472 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.472 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.472 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.472 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.472 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.472 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.472 INFO:teuthology.orchestra.run.smithi117.stdout: "ssl_server_port": { 2024-06-03T03:28:25.473 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "ssl_server_port", 2024-06-03T03:28:25.473 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:25.473 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.473 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.473 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "8443", 2024-06-03T03:28:25.473 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.473 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.473 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.473 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.473 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.473 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.473 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.473 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.474 INFO:teuthology.orchestra.run.smithi117.stdout: "standby_behaviour": { 2024-06-03T03:28:25.474 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "standby_behaviour", 2024-06-03T03:28:25.474 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.474 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.474 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.474 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "redirect", 2024-06-03T03:28:25.474 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.474 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.474 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.474 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.474 INFO:teuthology.orchestra.run.smithi117.stdout: "redirect" 2024-06-03T03:28:25.474 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.474 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.475 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.475 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.475 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.475 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.475 INFO:teuthology.orchestra.run.smithi117.stdout: "standby_error_status_code": { 2024-06-03T03:28:25.475 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "standby_error_status_code", 2024-06-03T03:28:25.475 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:25.475 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.475 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.475 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "500", 2024-06-03T03:28:25.475 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "400", 2024-06-03T03:28:25.475 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "599", 2024-06-03T03:28:25.475 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.475 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.476 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.476 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.476 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.476 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.476 INFO:teuthology.orchestra.run.smithi117.stdout: "url_prefix": { 2024-06-03T03:28:25.476 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "url_prefix", 2024-06-03T03:28:25.476 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.476 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.476 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.476 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.476 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.476 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.476 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.476 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.477 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.477 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.477 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.477 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.477 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.477 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.477 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:25.477 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "devicehealth", 2024-06-03T03:28:25.477 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:25.477 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:25.477 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:25.477 INFO:teuthology.orchestra.run.smithi117.stdout: "enable_monitoring": { 2024-06-03T03:28:25.477 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "enable_monitoring", 2024-06-03T03:28:25.478 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.478 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.478 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.478 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:25.478 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.478 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.478 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.478 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "monitor device health metrics", 2024-06-03T03:28:25.478 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.478 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.478 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.478 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.478 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:25.479 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:25.479 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.479 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.479 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.479 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.479 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.479 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.479 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.479 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.479 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.479 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.479 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.479 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.480 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.480 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.480 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.480 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.480 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.480 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.480 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.480 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:25.480 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:25.480 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.480 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.480 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.480 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.480 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.481 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.481 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.481 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.481 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.481 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.481 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.481 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.481 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:25.481 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:25.481 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.481 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.481 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.481 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:25.481 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.482 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.482 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.482 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.482 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.482 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.482 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.482 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.482 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.482 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.482 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.482 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.482 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.482 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.483 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.483 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:25.483 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:25.483 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.483 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.483 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.483 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.483 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.483 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.483 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.483 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.483 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.483 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.483 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.484 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.484 INFO:teuthology.orchestra.run.smithi117.stdout: "mark_out_threshold": { 2024-06-03T03:28:25.484 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "mark_out_threshold", 2024-06-03T03:28:25.484 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-06-03T03:28:25.484 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.484 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.484 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "2419200", 2024-06-03T03:28:25.484 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.484 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.484 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.484 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-06-03T03:28:25.484 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.484 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.484 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.485 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.485 INFO:teuthology.orchestra.run.smithi117.stdout: "pool_name": { 2024-06-03T03:28:25.485 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "pool_name", 2024-06-03T03:28:25.485 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.485 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.485 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.485 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "device_health_metrics", 2024-06-03T03:28:25.485 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.485 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.485 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.485 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "name of pool in which to store device health metrics", 2024-06-03T03:28:25.485 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.485 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.485 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.486 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.486 INFO:teuthology.orchestra.run.smithi117.stdout: "retention_period": { 2024-06-03T03:28:25.486 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "retention_period", 2024-06-03T03:28:25.486 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-06-03T03:28:25.486 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.486 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.486 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "15552000", 2024-06-03T03:28:25.486 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.486 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.486 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.486 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "how long to retain device health metrics", 2024-06-03T03:28:25.486 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.486 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.487 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.487 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.487 INFO:teuthology.orchestra.run.smithi117.stdout: "scrape_frequency": { 2024-06-03T03:28:25.487 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "scrape_frequency", 2024-06-03T03:28:25.487 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-06-03T03:28:25.487 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.487 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.487 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "86400", 2024-06-03T03:28:25.487 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.487 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.487 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.487 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "how frequently to scrape device health metrics", 2024-06-03T03:28:25.488 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.488 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.488 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.488 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.488 INFO:teuthology.orchestra.run.smithi117.stdout: "self_heal": { 2024-06-03T03:28:25.488 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "self_heal", 2024-06-03T03:28:25.488 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.488 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.488 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.488 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:25.488 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.488 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.488 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.489 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-06-03T03:28:25.489 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.489 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.489 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.489 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.489 INFO:teuthology.orchestra.run.smithi117.stdout: "sleep_interval": { 2024-06-03T03:28:25.489 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sleep_interval", 2024-06-03T03:28:25.489 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-06-03T03:28:25.489 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.489 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.489 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "600", 2024-06-03T03:28:25.489 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.489 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.489 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.490 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "how frequently to wake up and check device health", 2024-06-03T03:28:25.490 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.490 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.490 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.490 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.490 INFO:teuthology.orchestra.run.smithi117.stdout: "warn_threshold": { 2024-06-03T03:28:25.490 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "warn_threshold", 2024-06-03T03:28:25.490 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-06-03T03:28:25.490 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.490 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.490 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "7257600", 2024-06-03T03:28:25.490 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.491 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.491 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.491 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-06-03T03:28:25.491 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.491 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.491 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.491 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.491 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.491 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.491 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:25.491 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "diskprediction_local", 2024-06-03T03:28:25.491 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:25.491 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:25.492 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:25.492 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:25.492 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:25.492 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.492 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.492 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.492 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.492 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.492 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.492 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.492 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.492 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.492 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.493 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.493 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.493 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.493 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.493 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.493 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.493 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.493 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.493 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.493 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:25.493 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:25.493 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.493 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.494 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.494 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.494 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.494 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.494 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.494 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.494 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.494 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.494 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.494 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.494 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:25.494 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:25.495 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.495 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.495 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.495 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:25.495 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.495 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.495 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.495 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.495 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.495 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.495 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.495 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.495 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.495 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.495 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.496 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.496 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.496 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.496 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.496 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:25.496 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:25.496 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.496 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.496 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.496 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.496 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.496 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.496 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.496 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.497 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.497 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.497 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.497 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.497 INFO:teuthology.orchestra.run.smithi117.stdout: "predict_interval": { 2024-06-03T03:28:25.497 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "predict_interval", 2024-06-03T03:28:25.497 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.497 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.497 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.497 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "86400", 2024-06-03T03:28:25.497 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.497 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.497 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.498 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.498 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.498 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.498 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.498 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.498 INFO:teuthology.orchestra.run.smithi117.stdout: "predictor_model": { 2024-06-03T03:28:25.498 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "predictor_model", 2024-06-03T03:28:25.498 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.498 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.498 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.498 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "prophetstor", 2024-06-03T03:28:25.498 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.498 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.499 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.499 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.499 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.499 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.499 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.499 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.499 INFO:teuthology.orchestra.run.smithi117.stdout: "sleep_interval": { 2024-06-03T03:28:25.499 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sleep_interval", 2024-06-03T03:28:25.499 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.499 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.499 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.499 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "600", 2024-06-03T03:28:25.499 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.500 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.500 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.500 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.500 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.500 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.500 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.500 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.500 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.500 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.500 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:25.500 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "influx", 2024-06-03T03:28:25.500 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": false, 2024-06-03T03:28:25.500 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "influxdb python module not found", 2024-06-03T03:28:25.500 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:25.501 INFO:teuthology.orchestra.run.smithi117.stdout: "batch_size": { 2024-06-03T03:28:25.501 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "batch_size", 2024-06-03T03:28:25.501 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:25.501 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.501 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.501 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "5000", 2024-06-03T03:28:25.501 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.501 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.501 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.501 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-06-03T03:28:25.501 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.501 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.501 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.501 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.502 INFO:teuthology.orchestra.run.smithi117.stdout: "database": { 2024-06-03T03:28:25.502 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "database", 2024-06-03T03:28:25.502 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.502 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.502 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.502 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "ceph", 2024-06-03T03:28:25.502 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.502 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.502 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.502 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-06-03T03:28:25.502 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.503 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.503 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.503 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.503 INFO:teuthology.orchestra.run.smithi117.stdout: "hostname": { 2024-06-03T03:28:25.503 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "hostname", 2024-06-03T03:28:25.503 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.503 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.503 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.503 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.503 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.503 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.503 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.503 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "InfluxDB server hostname", 2024-06-03T03:28:25.504 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.504 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.504 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.504 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.504 INFO:teuthology.orchestra.run.smithi117.stdout: "interval": { 2024-06-03T03:28:25.504 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "interval", 2024-06-03T03:28:25.504 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-06-03T03:28:25.504 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.504 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.504 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "30", 2024-06-03T03:28:25.504 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "5", 2024-06-03T03:28:25.504 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.504 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.504 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-06-03T03:28:25.505 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.505 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.505 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.505 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.505 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:25.505 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:25.505 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.505 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.505 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.505 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.505 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.505 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.506 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.506 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.506 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.506 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.506 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.506 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.506 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.506 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.506 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.506 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.506 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.506 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.506 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.507 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:25.507 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:25.507 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.507 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.507 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.507 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.507 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.507 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.507 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.507 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.507 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.507 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.507 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.507 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.508 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:25.508 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:25.508 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.508 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.508 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.508 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:25.508 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.508 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.508 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.508 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.508 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.508 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.508 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.509 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.509 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.509 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.509 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.509 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.509 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.509 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.509 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.509 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:25.509 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:25.509 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.509 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.509 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.509 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.510 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.510 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.510 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.510 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.510 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.510 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.510 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.510 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.510 INFO:teuthology.orchestra.run.smithi117.stdout: "password": { 2024-06-03T03:28:25.510 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "password", 2024-06-03T03:28:25.510 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.510 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.510 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.511 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.511 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.511 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.511 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.511 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "password of InfluxDB server user", 2024-06-03T03:28:25.511 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.511 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.511 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.511 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.511 INFO:teuthology.orchestra.run.smithi117.stdout: "port": { 2024-06-03T03:28:25.511 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "port", 2024-06-03T03:28:25.511 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:25.511 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.511 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.512 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "8086", 2024-06-03T03:28:25.512 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.512 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.512 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.512 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "InfluxDB server port", 2024-06-03T03:28:25.512 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.512 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.512 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.512 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.512 INFO:teuthology.orchestra.run.smithi117.stdout: "ssl": { 2024-06-03T03:28:25.512 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "ssl", 2024-06-03T03:28:25.512 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.512 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.513 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.513 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "false", 2024-06-03T03:28:25.513 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.513 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.513 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.513 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-06-03T03:28:25.513 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.513 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.513 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.513 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.513 INFO:teuthology.orchestra.run.smithi117.stdout: "threads": { 2024-06-03T03:28:25.513 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "threads", 2024-06-03T03:28:25.513 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:25.513 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.514 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.514 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "5", 2024-06-03T03:28:25.514 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "1", 2024-06-03T03:28:25.514 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "32", 2024-06-03T03:28:25.514 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.514 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-06-03T03:28:25.514 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.514 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.514 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.514 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.514 INFO:teuthology.orchestra.run.smithi117.stdout: "username": { 2024-06-03T03:28:25.514 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "username", 2024-06-03T03:28:25.514 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.514 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.515 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.515 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.515 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.515 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.515 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.515 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "username of InfluxDB server user", 2024-06-03T03:28:25.515 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.515 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.515 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.515 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.515 INFO:teuthology.orchestra.run.smithi117.stdout: "verify_ssl": { 2024-06-03T03:28:25.515 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "verify_ssl", 2024-06-03T03:28:25.515 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.516 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.516 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.516 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "true", 2024-06-03T03:28:25.516 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.516 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.516 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.516 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-06-03T03:28:25.516 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.516 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.516 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.516 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.516 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.516 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.517 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:25.517 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "insights", 2024-06-03T03:28:25.517 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:25.517 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:25.517 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:25.517 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:25.517 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:25.517 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.517 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.517 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.517 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.517 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.517 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.517 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.518 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.518 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.518 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.518 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.518 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.518 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.518 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.518 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.518 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.518 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.518 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.518 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.518 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:25.519 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:25.519 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.519 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.519 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.519 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.519 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.519 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.519 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.519 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.519 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.519 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.519 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.519 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.520 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:25.520 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:25.520 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.520 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.520 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.520 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:25.520 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.520 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.520 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.520 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.520 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.520 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.520 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.521 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.521 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.521 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.521 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.521 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.521 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.521 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.521 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.521 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:25.521 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:25.521 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.521 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.521 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.522 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.522 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.522 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.522 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.522 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.522 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.522 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.522 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.522 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.522 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.522 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.522 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:25.522 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "iostat", 2024-06-03T03:28:25.522 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:25.523 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:25.523 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:25.523 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:25.523 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:25.523 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.523 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.523 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.523 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.523 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.523 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.523 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.523 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.523 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.524 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.524 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.524 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.524 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.524 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.524 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.524 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.524 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.524 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.524 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.524 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:25.524 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:25.524 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.524 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.525 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.525 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.525 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.525 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.525 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.525 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.525 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.525 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.525 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.525 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.525 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:25.525 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:25.525 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.525 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.526 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.526 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:25.526 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.526 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.526 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.526 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.526 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.526 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.526 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.526 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.526 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.526 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.526 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.527 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.527 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.527 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.527 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.527 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:25.527 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:25.527 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.527 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.527 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.527 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.527 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.527 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.527 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.528 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.528 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.528 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.528 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.528 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.528 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.528 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.528 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:25.528 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "k8sevents", 2024-06-03T03:28:25.528 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:25.528 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:25.528 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:25.528 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_event_retention_days": { 2024-06-03T03:28:25.528 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "ceph_event_retention_days", 2024-06-03T03:28:25.529 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:25.529 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.529 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.529 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "7", 2024-06-03T03:28:25.529 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.529 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.529 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.529 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Days to hold ceph event information within local cache", 2024-06-03T03:28:25.529 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.529 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.529 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.529 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.529 INFO:teuthology.orchestra.run.smithi117.stdout: "config_check_secs": { 2024-06-03T03:28:25.529 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "config_check_secs", 2024-06-03T03:28:25.530 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:25.530 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.530 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.530 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "10", 2024-06-03T03:28:25.530 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "10", 2024-06-03T03:28:25.530 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.530 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.530 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-06-03T03:28:25.530 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.530 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.530 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.530 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.530 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:25.531 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:25.531 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.531 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.531 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.531 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.531 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.531 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.531 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.531 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.531 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.531 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.531 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.531 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.532 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.532 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.532 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.532 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.532 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.532 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.532 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.532 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:25.532 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:25.532 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.532 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.532 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.532 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.532 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.533 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.533 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.533 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.533 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.533 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.533 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.533 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.533 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:25.533 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:25.533 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.533 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.533 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.533 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:25.534 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.534 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.534 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.534 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.534 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.534 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.534 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.534 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.534 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.534 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.534 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.534 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.534 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.535 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.535 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.535 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:25.535 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:25.535 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.535 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.535 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.535 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.535 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.535 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.535 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.535 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.535 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.535 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.536 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.536 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.536 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.536 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.536 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:25.536 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "localpool", 2024-06-03T03:28:25.536 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:25.536 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:25.536 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:25.536 INFO:teuthology.orchestra.run.smithi117.stdout: "failure_domain": { 2024-06-03T03:28:25.536 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "failure_domain", 2024-06-03T03:28:25.536 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.536 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.536 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.537 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "host", 2024-06-03T03:28:25.537 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.537 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.537 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.537 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "failure domain for any created local pool", 2024-06-03T03:28:25.537 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-06-03T03:28:25.537 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.537 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.537 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.537 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:25.537 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:25.537 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.537 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.537 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.538 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.538 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.538 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.538 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.538 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.538 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.538 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.538 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.538 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.538 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.538 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.538 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.539 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.539 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.539 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.539 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.539 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:25.539 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:25.539 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.539 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.539 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.539 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.539 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.539 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.539 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.539 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.540 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.540 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.540 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.540 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.540 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:25.540 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:25.540 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.540 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.540 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.540 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:25.540 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.540 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.540 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.540 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.541 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.541 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.541 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.541 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.541 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.541 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.541 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.541 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.541 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.541 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.541 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.541 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:25.541 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:25.541 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.542 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.542 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.542 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.542 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.542 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.542 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.542 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.542 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.542 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.542 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.542 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.542 INFO:teuthology.orchestra.run.smithi117.stdout: "min_size": { 2024-06-03T03:28:25.543 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "min_size", 2024-06-03T03:28:25.543 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:25.543 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.543 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.543 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.543 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.543 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.543 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.543 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "default min_size for any created local pool", 2024-06-03T03:28:25.543 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-06-03T03:28:25.543 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.543 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.543 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.543 INFO:teuthology.orchestra.run.smithi117.stdout: "num_rep": { 2024-06-03T03:28:25.544 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "num_rep", 2024-06-03T03:28:25.544 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:25.544 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.544 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.544 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "3", 2024-06-03T03:28:25.544 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.544 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.544 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.544 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "default replica count for any created local pool", 2024-06-03T03:28:25.544 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.544 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.544 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.545 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.545 INFO:teuthology.orchestra.run.smithi117.stdout: "pg_num": { 2024-06-03T03:28:25.545 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "pg_num", 2024-06-03T03:28:25.545 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:25.545 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.545 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.545 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "128", 2024-06-03T03:28:25.545 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.545 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.545 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.545 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "default pg_num for any created local pool", 2024-06-03T03:28:25.545 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.545 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.546 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.546 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.546 INFO:teuthology.orchestra.run.smithi117.stdout: "prefix": { 2024-06-03T03:28:25.546 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "prefix", 2024-06-03T03:28:25.546 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.546 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.546 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.546 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.546 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.546 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.546 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.546 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "name prefix for any created local pool", 2024-06-03T03:28:25.547 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.547 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.547 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.547 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.547 INFO:teuthology.orchestra.run.smithi117.stdout: "subtree": { 2024-06-03T03:28:25.547 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "subtree", 2024-06-03T03:28:25.547 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.547 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.547 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.547 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "rack", 2024-06-03T03:28:25.547 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.547 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.547 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.547 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "CRUSH level for which to create a local pool", 2024-06-03T03:28:25.548 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-06-03T03:28:25.548 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.548 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.548 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.548 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.548 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.548 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:25.548 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "mds_autoscaler", 2024-06-03T03:28:25.548 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:25.548 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:25.548 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:25.548 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:25.548 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:25.549 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.549 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.549 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.549 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.549 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.549 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.549 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.549 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.549 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.549 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.549 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.549 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.549 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.549 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.550 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.550 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.550 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.550 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.550 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.550 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:25.550 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:25.550 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.550 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.550 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.550 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.550 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.551 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.551 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.551 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.551 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.551 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.551 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.551 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.551 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:25.551 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:25.551 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.551 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.551 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.551 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:25.551 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.552 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.552 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.552 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.552 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.552 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.552 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.552 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.552 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.552 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.552 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.552 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.552 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.552 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.552 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.553 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:25.553 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:25.553 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.553 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.553 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.553 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.553 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.553 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.553 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.553 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.553 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.553 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.553 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.553 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.554 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.554 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.554 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:25.554 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "mirroring", 2024-06-03T03:28:25.554 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:25.554 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:25.554 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:25.554 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:25.554 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:25.554 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.554 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.554 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.555 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.555 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.555 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.555 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.555 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.555 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.555 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.555 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.555 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.555 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.555 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.555 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.555 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.555 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.555 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.556 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.556 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:25.556 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:25.556 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.556 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.556 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.556 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.556 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.556 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.556 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.556 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.556 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.556 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.556 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.557 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.557 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:25.557 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:25.557 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.557 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.557 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.557 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:25.557 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.557 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.557 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.557 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.557 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.557 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.558 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.558 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.558 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.558 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.558 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.558 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.558 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.558 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.558 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.558 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:25.558 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:25.558 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.558 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.559 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.559 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.559 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.559 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.559 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.559 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.559 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.559 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.559 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.559 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.559 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.559 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.559 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:25.560 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "nfs", 2024-06-03T03:28:25.560 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:25.560 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:25.560 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:25.560 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:25.560 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:25.560 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.560 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.560 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.560 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.560 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.560 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.561 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.561 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.561 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.561 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.561 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.561 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.561 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.561 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.561 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.561 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.561 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.561 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.562 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.562 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:25.562 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:25.562 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.562 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.562 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.562 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.562 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.562 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.562 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.562 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.562 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.562 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.563 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.563 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.563 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:25.563 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:25.563 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.563 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.563 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.563 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:25.563 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.563 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.563 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.563 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.563 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.564 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.564 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.564 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.564 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.564 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.564 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.564 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.564 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.564 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.564 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.564 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:25.564 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:25.565 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.565 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.565 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.565 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.565 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.565 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.565 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.565 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.565 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.565 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.565 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.565 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.565 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.566 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.566 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:25.566 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "orchestrator", 2024-06-03T03:28:25.566 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:25.566 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:25.566 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:25.566 INFO:teuthology.orchestra.run.smithi117.stdout: "fail_fs": { 2024-06-03T03:28:25.566 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "fail_fs", 2024-06-03T03:28:25.566 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.566 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.566 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.566 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.566 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.567 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.567 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.567 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-06-03T03:28:25.567 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.567 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.567 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.567 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.567 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:25.567 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:25.567 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.567 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.567 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.567 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.567 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.568 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.568 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.568 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.568 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.568 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.568 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.568 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.568 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.568 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.568 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.568 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.568 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.568 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.568 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.569 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:25.569 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:25.569 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.569 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.569 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.569 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.569 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.569 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.569 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.569 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.569 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.569 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.569 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.570 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.570 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:25.570 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:25.570 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.570 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.570 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.570 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:25.570 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.570 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.570 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.570 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.570 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.570 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.570 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.571 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.571 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.571 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.571 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.571 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.571 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.571 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.571 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.571 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:25.571 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:25.571 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.571 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.571 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.571 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.572 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.572 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.572 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.572 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.572 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.572 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.572 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.572 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.572 INFO:teuthology.orchestra.run.smithi117.stdout: "orchestrator": { 2024-06-03T03:28:25.572 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "orchestrator", 2024-06-03T03:28:25.572 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.572 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.572 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.573 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.573 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.573 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.573 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.573 INFO:teuthology.orchestra.run.smithi117.stdout: "cephadm", 2024-06-03T03:28:25.573 INFO:teuthology.orchestra.run.smithi117.stdout: "rook", 2024-06-03T03:28:25.573 INFO:teuthology.orchestra.run.smithi117.stdout: "test_orchestrator" 2024-06-03T03:28:25.573 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.573 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Orchestrator backend", 2024-06-03T03:28:25.573 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.573 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.573 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.573 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.573 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.574 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.574 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:25.574 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "osd_perf_query", 2024-06-03T03:28:25.574 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:25.574 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:25.574 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:25.574 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:25.574 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:25.574 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.574 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.574 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.574 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.574 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.575 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.575 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.575 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.575 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.575 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.575 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.575 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.575 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.575 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.575 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.575 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.575 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.575 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.575 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.576 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:25.576 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:25.576 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.576 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.576 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.576 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.576 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.576 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.576 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.576 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.576 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.576 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.576 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.577 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.577 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:25.577 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:25.577 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.577 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.577 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.577 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:25.577 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.577 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.577 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.577 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.577 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.577 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.577 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.578 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.578 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.578 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.578 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.578 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.578 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.578 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.578 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.578 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:25.578 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:25.578 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.578 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.578 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.579 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.579 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.579 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.579 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.579 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.579 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.579 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.579 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.579 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.579 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.579 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.579 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:25.579 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "osd_support", 2024-06-03T03:28:25.580 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:25.580 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:25.580 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:25.580 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:25.580 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:25.580 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.580 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.580 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.580 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.580 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.580 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.580 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.581 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.581 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.581 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.581 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.581 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.581 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.581 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.581 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.581 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.581 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.581 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.581 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.581 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:25.581 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:25.582 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.582 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.582 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.582 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.582 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.582 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.582 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.582 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.582 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.582 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.582 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.582 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.582 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:25.582 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:25.583 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.583 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.583 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.583 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:25.583 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.583 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.583 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.583 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.583 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.583 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.583 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.583 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.583 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.584 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.584 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.584 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.584 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.584 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.584 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.584 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:25.584 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:25.584 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.584 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.584 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.584 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.584 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.584 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.585 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.585 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.585 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.585 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.585 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.585 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.585 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.585 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.585 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:25.585 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "pg_autoscaler", 2024-06-03T03:28:25.585 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:25.585 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:25.585 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:25.585 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:25.586 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:25.586 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.586 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.586 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.586 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.586 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.586 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.586 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.586 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.586 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.586 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.586 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.587 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.587 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.587 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.587 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.587 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.587 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.587 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.587 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.587 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:25.587 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:25.587 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.587 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.587 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.587 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.588 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.588 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.588 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.588 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.588 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.588 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.588 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.588 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.588 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:25.588 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:25.588 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.588 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.588 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.589 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:25.589 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.589 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.589 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.589 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.589 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.589 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.589 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.589 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.589 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.589 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.589 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.589 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.590 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.590 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.590 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.590 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:25.590 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:25.590 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.590 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.590 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.590 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.590 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.590 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.590 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.590 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.591 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.591 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.591 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.591 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.591 INFO:teuthology.orchestra.run.smithi117.stdout: "sleep_interval": { 2024-06-03T03:28:25.591 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sleep_interval", 2024-06-03T03:28:25.591 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-06-03T03:28:25.591 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.591 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.591 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "60", 2024-06-03T03:28:25.591 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.591 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.591 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.592 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.592 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.592 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.592 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.592 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.592 INFO:teuthology.orchestra.run.smithi117.stdout: "threshold": { 2024-06-03T03:28:25.592 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "threshold", 2024-06-03T03:28:25.592 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "float", 2024-06-03T03:28:25.592 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.592 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.592 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "3.0", 2024-06-03T03:28:25.592 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "1.0", 2024-06-03T03:28:25.592 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.593 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.593 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "scaling threshold", 2024-06-03T03:28:25.593 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-06-03T03:28:25.593 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.593 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.593 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.593 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.593 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.593 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:25.593 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "progress", 2024-06-03T03:28:25.593 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:25.593 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:25.593 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:25.593 INFO:teuthology.orchestra.run.smithi117.stdout: "allow_pg_recovery_event": { 2024-06-03T03:28:25.594 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "allow_pg_recovery_event", 2024-06-03T03:28:25.594 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.594 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.594 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.594 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.594 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.594 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.594 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.594 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "allow the module to show pg recovery progress", 2024-06-03T03:28:25.594 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.594 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.594 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.594 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.595 INFO:teuthology.orchestra.run.smithi117.stdout: "enabled": { 2024-06-03T03:28:25.595 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "enabled", 2024-06-03T03:28:25.595 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.595 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.595 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.595 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:25.595 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.595 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.595 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.595 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.595 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.595 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.595 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.595 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.596 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:25.596 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:25.596 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.596 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.596 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.596 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.596 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.596 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.596 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.596 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.596 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.596 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.596 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.597 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.597 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.597 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.597 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.597 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.597 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.597 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.597 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.597 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:25.597 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:25.597 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.597 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.597 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.597 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.598 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.598 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.598 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.598 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.598 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.598 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.598 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.598 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.598 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:25.598 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:25.598 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.598 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.598 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.599 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:25.599 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.599 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.599 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.599 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.599 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.599 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.599 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.599 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.599 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.599 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.599 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.599 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.600 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.600 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.600 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.600 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:25.600 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:25.600 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.600 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.600 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.600 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.600 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.600 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.600 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.600 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.600 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.601 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.601 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.601 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.601 INFO:teuthology.orchestra.run.smithi117.stdout: "max_completed_events": { 2024-06-03T03:28:25.601 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "max_completed_events", 2024-06-03T03:28:25.601 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:25.601 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.601 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.601 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "50", 2024-06-03T03:28:25.601 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.601 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.601 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.601 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "number of past completed events to remember", 2024-06-03T03:28:25.601 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.602 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.602 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.602 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.602 INFO:teuthology.orchestra.run.smithi117.stdout: "sleep_interval": { 2024-06-03T03:28:25.602 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sleep_interval", 2024-06-03T03:28:25.602 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-06-03T03:28:25.602 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.602 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.602 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "5", 2024-06-03T03:28:25.602 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.602 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.602 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.603 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "how long the module is going to sleep", 2024-06-03T03:28:25.603 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.603 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.603 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.603 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.603 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.603 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.603 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:25.603 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "prometheus", 2024-06-03T03:28:25.603 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:25.603 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:25.603 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:25.603 INFO:teuthology.orchestra.run.smithi117.stdout: "cache": { 2024-06-03T03:28:25.604 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "cache", 2024-06-03T03:28:25.604 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.604 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.604 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.604 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:25.604 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.604 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.604 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.604 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.604 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.604 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.604 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.604 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.604 INFO:teuthology.orchestra.run.smithi117.stdout: "exclude_perf_counters": { 2024-06-03T03:28:25.605 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "exclude_perf_counters", 2024-06-03T03:28:25.605 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.605 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.605 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.605 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:25.605 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.605 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.605 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.605 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-06-03T03:28:25.605 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-06-03T03:28:25.605 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.605 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.605 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.606 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:25.606 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:25.606 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.606 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.606 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.606 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.606 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.606 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.606 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.609 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.610 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.610 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.610 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.610 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.610 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.610 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.610 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.610 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.610 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.610 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.610 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.610 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:25.611 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:25.611 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.611 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.611 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.611 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.611 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.611 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.611 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.611 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.611 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.611 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.611 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.611 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.611 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:25.612 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:25.612 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.612 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.612 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.612 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:25.612 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.612 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.612 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.612 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.612 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.612 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.612 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.612 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.613 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.613 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.613 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.613 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.613 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.613 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.613 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.613 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:25.613 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:25.613 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.613 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.613 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.613 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.613 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.614 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.614 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.614 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.614 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.614 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.614 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.614 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.614 INFO:teuthology.orchestra.run.smithi117.stdout: "rbd_stats_pools": { 2024-06-03T03:28:25.614 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "rbd_stats_pools", 2024-06-03T03:28:25.614 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.614 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.614 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.614 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.615 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.615 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.615 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.615 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.615 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.615 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.615 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.615 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.615 INFO:teuthology.orchestra.run.smithi117.stdout: "rbd_stats_pools_refresh_interval": { 2024-06-03T03:28:25.615 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-06-03T03:28:25.615 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:25.615 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.615 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.616 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "300", 2024-06-03T03:28:25.616 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.616 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.616 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.616 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.616 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.616 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.616 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.616 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.616 INFO:teuthology.orchestra.run.smithi117.stdout: "scrape_interval": { 2024-06-03T03:28:25.616 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "scrape_interval", 2024-06-03T03:28:25.616 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "float", 2024-06-03T03:28:25.616 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.617 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.617 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "15.0", 2024-06-03T03:28:25.617 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.617 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.617 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.617 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.617 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.617 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.617 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.617 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.617 INFO:teuthology.orchestra.run.smithi117.stdout: "server_addr": { 2024-06-03T03:28:25.617 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "server_addr", 2024-06-03T03:28:25.617 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.617 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.618 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.618 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "::", 2024-06-03T03:28:25.618 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.618 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.618 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.618 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-06-03T03:28:25.618 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.618 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.618 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.618 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.618 INFO:teuthology.orchestra.run.smithi117.stdout: "server_port": { 2024-06-03T03:28:25.618 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "server_port", 2024-06-03T03:28:25.618 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:25.619 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.619 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.619 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "9283", 2024-06-03T03:28:25.619 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.619 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.619 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.619 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-06-03T03:28:25.619 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.619 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.619 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.619 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.619 INFO:teuthology.orchestra.run.smithi117.stdout: "stale_cache_strategy": { 2024-06-03T03:28:25.619 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "stale_cache_strategy", 2024-06-03T03:28:25.620 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.620 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.620 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.620 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "log", 2024-06-03T03:28:25.620 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.620 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.620 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.620 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.620 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.620 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.620 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.620 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.620 INFO:teuthology.orchestra.run.smithi117.stdout: "standby_behaviour": { 2024-06-03T03:28:25.621 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "standby_behaviour", 2024-06-03T03:28:25.621 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.621 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.621 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.621 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "default", 2024-06-03T03:28:25.621 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.621 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.621 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.621 INFO:teuthology.orchestra.run.smithi117.stdout: "default", 2024-06-03T03:28:25.621 INFO:teuthology.orchestra.run.smithi117.stdout: "error" 2024-06-03T03:28:25.621 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.621 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.621 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.622 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.622 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.622 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.622 INFO:teuthology.orchestra.run.smithi117.stdout: "standby_error_status_code": { 2024-06-03T03:28:25.622 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "standby_error_status_code", 2024-06-03T03:28:25.622 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:25.622 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.622 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.622 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "500", 2024-06-03T03:28:25.622 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "400", 2024-06-03T03:28:25.622 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "599", 2024-06-03T03:28:25.622 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.623 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.623 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.623 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.623 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.623 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.623 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.623 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.623 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:25.623 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "rbd_support", 2024-06-03T03:28:25.623 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:25.623 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:25.623 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:25.623 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:25.623 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:25.624 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.624 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.624 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.624 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.624 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.624 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.624 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.624 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.624 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.624 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.624 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.624 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.624 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.625 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.625 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.625 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.625 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.625 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.625 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.625 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:25.625 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:25.625 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.625 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.625 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.625 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.625 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.625 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.626 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.626 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.626 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.626 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.626 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.626 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.626 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:25.626 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:25.626 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.626 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.626 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.626 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:25.626 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.627 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.627 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.627 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.627 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.627 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.627 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.627 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.627 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.627 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.627 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.627 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.627 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.627 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.628 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.628 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:25.628 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:25.628 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.628 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.628 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.628 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.628 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.628 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.628 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.628 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.628 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.628 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.628 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.629 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.629 INFO:teuthology.orchestra.run.smithi117.stdout: "max_concurrent_snap_create": { 2024-06-03T03:28:25.629 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "max_concurrent_snap_create", 2024-06-03T03:28:25.629 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:25.629 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.629 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.629 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "10", 2024-06-03T03:28:25.629 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.629 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.629 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.629 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.629 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.629 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.630 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.630 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.630 INFO:teuthology.orchestra.run.smithi117.stdout: "mirror_snapshot_schedule": { 2024-06-03T03:28:25.630 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "mirror_snapshot_schedule", 2024-06-03T03:28:25.630 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.630 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.630 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.630 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.630 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.630 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.630 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.630 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.630 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.631 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.631 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.631 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.631 INFO:teuthology.orchestra.run.smithi117.stdout: "trash_purge_schedule": { 2024-06-03T03:28:25.631 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "trash_purge_schedule", 2024-06-03T03:28:25.631 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.631 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.631 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.631 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.631 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.631 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.631 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.631 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.631 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.632 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.632 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.632 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.632 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.632 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.632 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:25.632 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "restful", 2024-06-03T03:28:25.632 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:25.632 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:25.632 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:25.632 INFO:teuthology.orchestra.run.smithi117.stdout: "enable_auth": { 2024-06-03T03:28:25.632 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "enable_auth", 2024-06-03T03:28:25.632 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.632 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.633 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.633 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:25.633 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.633 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.633 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.633 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.633 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.633 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.633 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.633 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.633 INFO:teuthology.orchestra.run.smithi117.stdout: "key_file": { 2024-06-03T03:28:25.633 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "key_file", 2024-06-03T03:28:25.633 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.634 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.634 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.634 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.634 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.634 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.634 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.634 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.634 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.634 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.635 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.635 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.635 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:25.635 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:25.635 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.635 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.635 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.635 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.635 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.635 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.635 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.635 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.635 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.635 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.636 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.636 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.636 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.636 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.636 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.636 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.636 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.636 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.636 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.636 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:25.636 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:25.636 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.636 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.637 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.637 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.637 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.637 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.637 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.637 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.637 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.637 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.637 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.637 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.637 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:25.637 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:25.637 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.637 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.638 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.638 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:25.638 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.638 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.638 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.638 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.638 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.638 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.638 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.638 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.638 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.638 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.638 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.639 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.639 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.639 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.639 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.639 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:25.639 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:25.639 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.639 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.639 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.639 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.639 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.639 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.639 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.640 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.640 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.640 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.640 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.640 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.640 INFO:teuthology.orchestra.run.smithi117.stdout: "server_addr": { 2024-06-03T03:28:25.640 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "server_addr", 2024-06-03T03:28:25.640 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.640 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.640 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.640 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.640 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.640 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.641 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.641 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.641 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.641 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.641 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.641 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.641 INFO:teuthology.orchestra.run.smithi117.stdout: "server_port": { 2024-06-03T03:28:25.641 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "server_port", 2024-06-03T03:28:25.641 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.641 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.641 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.641 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.641 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.641 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.642 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.642 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.642 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.642 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.642 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.642 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.642 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.642 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.642 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:25.642 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "rgw", 2024-06-03T03:28:25.642 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:25.642 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:25.642 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:25.643 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:25.643 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:25.643 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.643 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.643 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.643 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.643 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.643 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.643 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.643 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.643 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.643 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.643 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.644 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.644 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.644 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.644 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.644 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.644 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.644 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.644 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.644 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:25.644 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:25.644 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.644 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.644 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.644 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.645 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.645 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.645 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.645 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.645 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.645 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.645 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.645 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.645 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:25.645 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:25.645 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.645 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.646 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.646 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:25.646 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.646 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.646 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.646 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.646 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.646 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.646 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.646 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.646 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.646 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.646 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.647 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.647 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.647 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.647 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.647 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:25.647 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:25.647 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.647 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.647 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.647 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.647 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.647 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.647 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.647 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.648 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.648 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.648 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.648 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.648 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.648 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.648 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:25.648 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "rook", 2024-06-03T03:28:25.648 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:25.648 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:25.648 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:25.648 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:25.648 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:25.648 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.649 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.649 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.649 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.649 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.649 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.649 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.649 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.649 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.649 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.649 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.649 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.649 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.649 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.650 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.650 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.650 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.650 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.650 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.650 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:25.650 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:25.650 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.650 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.650 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.650 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.650 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.651 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.651 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.651 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.651 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.651 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.651 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.651 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.651 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:25.651 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:25.651 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.651 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.651 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.652 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:25.652 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.652 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.652 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.652 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.652 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.652 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.652 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.652 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.652 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.652 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.652 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.652 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.652 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.653 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.653 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.653 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:25.653 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:25.653 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.653 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.653 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.653 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.653 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.653 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.653 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.653 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.653 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.654 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.654 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.654 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.654 INFO:teuthology.orchestra.run.smithi117.stdout: "storage_class": { 2024-06-03T03:28:25.654 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "storage_class", 2024-06-03T03:28:25.654 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.654 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.654 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.654 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "local", 2024-06-03T03:28:25.654 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.654 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.654 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.655 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-06-03T03:28:25.655 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.655 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.655 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.655 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.655 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.655 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.655 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:25.655 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "selftest", 2024-06-03T03:28:25.655 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:25.655 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:25.655 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:25.655 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:25.656 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:25.656 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.656 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.656 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.656 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.656 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.656 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.656 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.656 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.656 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.656 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.656 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.656 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.657 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.657 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.657 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.657 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.657 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.657 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.657 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.657 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:25.657 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:25.657 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.657 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.657 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.657 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.657 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.658 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.658 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.658 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.658 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.658 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.658 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.658 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.658 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:25.658 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:25.658 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.658 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.658 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.658 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:25.658 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.659 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.659 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.659 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.659 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.659 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.659 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.659 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.659 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.659 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.659 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.659 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.659 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.659 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.660 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.660 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:25.660 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:25.660 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.660 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.660 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.660 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.660 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.660 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.660 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.660 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.660 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.660 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.661 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.661 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.661 INFO:teuthology.orchestra.run.smithi117.stdout: "roption1": { 2024-06-03T03:28:25.661 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "roption1", 2024-06-03T03:28:25.661 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.661 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.661 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.661 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.661 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.661 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.662 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.662 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.662 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.662 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.662 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.662 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.662 INFO:teuthology.orchestra.run.smithi117.stdout: "roption2": { 2024-06-03T03:28:25.662 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "roption2", 2024-06-03T03:28:25.662 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.662 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.662 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.662 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "xyz", 2024-06-03T03:28:25.662 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.663 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.663 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.663 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.663 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.663 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.663 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.663 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.663 INFO:teuthology.orchestra.run.smithi117.stdout: "rwoption1": { 2024-06-03T03:28:25.663 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "rwoption1", 2024-06-03T03:28:25.663 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.663 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.663 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.663 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.663 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.664 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.664 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.664 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.664 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.664 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.664 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.664 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.664 INFO:teuthology.orchestra.run.smithi117.stdout: "rwoption2": { 2024-06-03T03:28:25.664 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "rwoption2", 2024-06-03T03:28:25.664 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:25.664 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.664 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.664 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.665 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.665 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.665 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.665 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.665 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.665 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.665 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.665 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.665 INFO:teuthology.orchestra.run.smithi117.stdout: "rwoption3": { 2024-06-03T03:28:25.665 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "rwoption3", 2024-06-03T03:28:25.665 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "float", 2024-06-03T03:28:25.665 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.665 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.666 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.666 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.666 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.666 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.666 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.666 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.666 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.666 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.666 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.666 INFO:teuthology.orchestra.run.smithi117.stdout: "rwoption4": { 2024-06-03T03:28:25.666 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "rwoption4", 2024-06-03T03:28:25.666 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.666 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.667 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.667 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.667 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.667 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.667 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.667 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.667 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.667 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.667 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.667 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.667 INFO:teuthology.orchestra.run.smithi117.stdout: "rwoption5": { 2024-06-03T03:28:25.667 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "rwoption5", 2024-06-03T03:28:25.667 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.667 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.668 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.668 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.668 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.668 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.668 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.668 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.668 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.668 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.668 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.668 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.668 INFO:teuthology.orchestra.run.smithi117.stdout: "rwoption6": { 2024-06-03T03:28:25.668 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "rwoption6", 2024-06-03T03:28:25.668 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.668 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.669 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.669 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:25.669 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.669 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.669 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.669 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.669 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.669 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.669 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.669 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.669 INFO:teuthology.orchestra.run.smithi117.stdout: "rwoption7": { 2024-06-03T03:28:25.669 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "rwoption7", 2024-06-03T03:28:25.669 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:25.670 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.670 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.670 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.670 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "1", 2024-06-03T03:28:25.670 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "42", 2024-06-03T03:28:25.670 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.670 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.670 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.670 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.670 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.670 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.670 INFO:teuthology.orchestra.run.smithi117.stdout: "testkey": { 2024-06-03T03:28:25.670 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "testkey", 2024-06-03T03:28:25.671 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.671 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.671 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.671 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.671 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.671 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.671 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.671 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.671 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.671 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.671 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.671 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.671 INFO:teuthology.orchestra.run.smithi117.stdout: "testlkey": { 2024-06-03T03:28:25.671 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "testlkey", 2024-06-03T03:28:25.672 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.672 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.672 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.672 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.672 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.672 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.672 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.672 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.672 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.672 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.672 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.672 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.672 INFO:teuthology.orchestra.run.smithi117.stdout: "testnewline": { 2024-06-03T03:28:25.673 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "testnewline", 2024-06-03T03:28:25.673 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.673 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.673 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.673 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.673 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.673 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.673 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.673 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.673 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.673 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.673 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.673 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.673 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.674 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.674 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:25.674 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "snap_schedule", 2024-06-03T03:28:25.674 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:25.674 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:25.674 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:25.674 INFO:teuthology.orchestra.run.smithi117.stdout: "allow_m_granularity": { 2024-06-03T03:28:25.674 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "allow_m_granularity", 2024-06-03T03:28:25.674 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.674 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.674 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.674 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.674 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.675 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.675 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.675 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "allow minute scheduled snapshots", 2024-06-03T03:28:25.675 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.675 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.675 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.675 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.675 INFO:teuthology.orchestra.run.smithi117.stdout: "dump_on_update": { 2024-06-03T03:28:25.675 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "dump_on_update", 2024-06-03T03:28:25.675 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.675 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.675 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.675 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.676 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.676 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.676 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.676 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "dump database to debug log on update", 2024-06-03T03:28:25.676 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.676 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.676 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.676 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.676 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:25.676 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:25.676 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.676 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.676 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.676 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.677 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.677 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.677 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.677 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.677 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.677 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.677 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.677 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.677 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.677 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.677 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.677 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.677 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.677 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.678 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.678 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:25.678 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:25.678 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.678 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.678 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.678 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.678 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.678 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.678 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.678 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.678 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.678 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.678 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.679 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.679 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:25.679 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:25.679 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.679 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.679 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.679 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:25.679 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.679 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.679 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.679 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.679 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.679 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.680 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.680 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.680 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.680 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.680 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.680 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.680 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.680 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.680 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.680 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:25.680 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:25.680 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.680 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.681 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.681 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.681 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.681 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.681 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.681 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.681 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.681 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.681 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.681 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.681 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.681 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.681 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:25.681 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "stats", 2024-06-03T03:28:25.682 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:25.682 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:25.682 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:25.682 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:25.682 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:25.682 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.682 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.682 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.682 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.682 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.682 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.682 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.682 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.683 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.683 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.683 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.683 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.683 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.683 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.683 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.683 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.683 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.683 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.683 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.683 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:25.683 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:25.684 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.684 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.684 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.684 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.684 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.684 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.684 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.684 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.684 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.684 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.684 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.684 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.684 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:25.685 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:25.685 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.685 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.685 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.685 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:25.685 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.685 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.685 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.685 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.685 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.685 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.685 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.685 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.685 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.686 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.686 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.686 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.686 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.686 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.686 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.686 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:25.686 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:25.686 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.686 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.686 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.686 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.686 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.686 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.687 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.687 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.687 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.687 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.687 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.687 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.687 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.687 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.687 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:25.687 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "status", 2024-06-03T03:28:25.687 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:25.687 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:25.687 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:25.687 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:25.688 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:25.688 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.688 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.688 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.688 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.688 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.688 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.688 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.688 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.688 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.688 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.688 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.688 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.689 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.689 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.689 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.689 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.689 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.689 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.689 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.689 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:25.689 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:25.689 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.689 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.689 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.689 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.689 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.690 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.690 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.690 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.690 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.690 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.690 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.690 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.690 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:25.690 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:25.690 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.690 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.690 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.690 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:25.691 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.691 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.691 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.691 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.691 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.691 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.691 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.691 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.691 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.691 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.691 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.691 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.691 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.692 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.692 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.692 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:25.692 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:25.692 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.692 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.692 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.692 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.692 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.692 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.692 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.692 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.692 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.692 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.693 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.693 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.693 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.693 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.693 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:25.693 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "telegraf", 2024-06-03T03:28:25.693 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:25.693 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:25.693 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:25.693 INFO:teuthology.orchestra.run.smithi117.stdout: "address": { 2024-06-03T03:28:25.693 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "address", 2024-06-03T03:28:25.693 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.693 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.694 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.694 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-06-03T03:28:25.694 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.694 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.694 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.694 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.694 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.694 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.694 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.694 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.694 INFO:teuthology.orchestra.run.smithi117.stdout: "interval": { 2024-06-03T03:28:25.694 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "interval", 2024-06-03T03:28:25.695 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-06-03T03:28:25.695 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.695 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.695 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "15", 2024-06-03T03:28:25.695 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.695 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.695 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.695 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.695 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.695 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.695 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.695 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.695 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:25.696 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:25.696 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.696 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.696 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.696 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.696 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.696 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.696 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.696 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.696 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.696 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.696 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.696 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.696 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.697 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.697 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.697 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.697 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.697 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.697 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.697 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:25.697 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:25.697 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.697 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.697 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.697 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.697 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.698 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.698 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.698 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.698 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.698 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.698 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.698 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.698 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:25.698 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:25.698 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.698 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.698 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.698 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:25.699 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.699 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.699 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.699 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.699 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.699 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.699 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.699 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.699 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.699 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.699 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.699 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.699 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.700 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.700 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.700 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:25.700 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:25.700 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.700 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.700 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.700 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.700 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.700 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.700 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.700 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.700 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.700 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.701 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.701 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.701 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.701 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.701 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:25.701 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "telemetry", 2024-06-03T03:28:25.701 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:25.701 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:25.701 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:25.701 INFO:teuthology.orchestra.run.smithi117.stdout: "channel_basic": { 2024-06-03T03:28:25.701 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "channel_basic", 2024-06-03T03:28:25.701 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.701 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.701 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.702 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:25.702 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.702 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.702 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.702 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Share basic cluster information (size, version)", 2024-06-03T03:28:25.702 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.702 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.702 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.702 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.702 INFO:teuthology.orchestra.run.smithi117.stdout: "channel_crash": { 2024-06-03T03:28:25.702 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "channel_crash", 2024-06-03T03:28:25.702 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.702 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.703 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.703 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:25.703 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.703 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.703 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.703 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-06-03T03:28:25.703 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.703 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.703 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.703 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.703 INFO:teuthology.orchestra.run.smithi117.stdout: "channel_device": { 2024-06-03T03:28:25.703 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "channel_device", 2024-06-03T03:28:25.703 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.703 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.704 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.704 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:25.704 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.704 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.704 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.704 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-06-03T03:28:25.704 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.704 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.704 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.704 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.704 INFO:teuthology.orchestra.run.smithi117.stdout: "channel_ident": { 2024-06-03T03:28:25.704 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "channel_ident", 2024-06-03T03:28:25.704 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.705 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.705 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.705 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.705 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.705 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.705 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.705 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-06-03T03:28:25.705 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.705 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.705 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.705 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.705 INFO:teuthology.orchestra.run.smithi117.stdout: "channel_perf": { 2024-06-03T03:28:25.705 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "channel_perf", 2024-06-03T03:28:25.705 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.706 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.706 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.706 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.706 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.706 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.706 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.706 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Share various performance metrics of a cluster", 2024-06-03T03:28:25.706 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.706 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.706 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.706 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.706 INFO:teuthology.orchestra.run.smithi117.stdout: "contact": { 2024-06-03T03:28:25.706 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "contact", 2024-06-03T03:28:25.706 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.707 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.707 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.707 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.707 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.707 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.707 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.707 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.707 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.707 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.707 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.707 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.707 INFO:teuthology.orchestra.run.smithi117.stdout: "description": { 2024-06-03T03:28:25.707 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "description", 2024-06-03T03:28:25.707 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.708 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.708 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.708 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.708 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.708 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.708 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.708 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.708 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.708 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.708 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.708 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.708 INFO:teuthology.orchestra.run.smithi117.stdout: "device_url": { 2024-06-03T03:28:25.708 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "device_url", 2024-06-03T03:28:25.709 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.709 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.709 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.709 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-06-03T03:28:25.709 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.709 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.709 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.709 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.709 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.709 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.709 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.709 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.709 INFO:teuthology.orchestra.run.smithi117.stdout: "enabled": { 2024-06-03T03:28:25.710 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "enabled", 2024-06-03T03:28:25.710 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.710 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.710 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.710 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.710 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.710 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.710 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.710 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.710 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.710 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.710 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.710 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.711 INFO:teuthology.orchestra.run.smithi117.stdout: "interval": { 2024-06-03T03:28:25.711 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "interval", 2024-06-03T03:28:25.711 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:25.711 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.711 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.711 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "24", 2024-06-03T03:28:25.711 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "8", 2024-06-03T03:28:25.711 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.711 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.711 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.711 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.711 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.711 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.711 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.712 INFO:teuthology.orchestra.run.smithi117.stdout: "last_opt_revision": { 2024-06-03T03:28:25.712 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "last_opt_revision", 2024-06-03T03:28:25.712 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:25.712 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.712 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.712 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "1", 2024-06-03T03:28:25.712 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.712 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.712 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.712 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.712 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.712 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.712 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.713 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.713 INFO:teuthology.orchestra.run.smithi117.stdout: "leaderboard": { 2024-06-03T03:28:25.713 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "leaderboard", 2024-06-03T03:28:25.713 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.713 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.713 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.713 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.713 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.713 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.713 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.713 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.713 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.713 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.713 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.714 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.714 INFO:teuthology.orchestra.run.smithi117.stdout: "leaderboard_description": { 2024-06-03T03:28:25.714 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "leaderboard_description", 2024-06-03T03:28:25.714 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.714 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.714 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.714 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.714 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.714 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.714 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.714 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.714 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.714 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.715 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.715 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.715 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:25.715 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:25.715 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.715 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.715 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.715 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.715 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.715 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.715 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.715 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.715 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.716 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.716 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.716 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.716 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.716 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.716 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.716 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.716 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.716 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.716 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.716 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:25.716 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:25.716 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.716 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.716 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.717 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.717 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.717 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.717 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.717 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.717 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.717 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.717 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.717 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.717 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:25.717 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:25.717 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.717 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.717 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.718 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:25.718 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.718 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.718 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.718 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.718 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.718 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.718 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.718 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.718 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.718 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.718 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.718 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.719 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.719 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.719 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.719 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:25.719 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:25.719 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.719 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.719 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.719 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.719 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.719 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.719 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.719 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.719 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.720 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.720 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.720 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.720 INFO:teuthology.orchestra.run.smithi117.stdout: "organization": { 2024-06-03T03:28:25.720 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "organization", 2024-06-03T03:28:25.720 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.720 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.720 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.720 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.720 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.720 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.720 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.720 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.721 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.721 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.721 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.721 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.721 INFO:teuthology.orchestra.run.smithi117.stdout: "proxy": { 2024-06-03T03:28:25.721 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "proxy", 2024-06-03T03:28:25.721 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.721 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.721 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.721 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.721 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.721 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.721 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.721 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.721 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.722 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.722 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.722 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.722 INFO:teuthology.orchestra.run.smithi117.stdout: "url": { 2024-06-03T03:28:25.722 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "url", 2024-06-03T03:28:25.722 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.722 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.722 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.722 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-06-03T03:28:25.722 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.722 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.723 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.723 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.723 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.723 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.723 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.723 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.723 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.723 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.723 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:25.723 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "test_orchestrator", 2024-06-03T03:28:25.723 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:25.723 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:25.723 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:25.723 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:25.724 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:25.724 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.724 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.724 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.724 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.724 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.724 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.724 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.724 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.724 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.724 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.724 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.724 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.725 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.725 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.725 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.725 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.725 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.725 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.725 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.725 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:25.725 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:25.725 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.725 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.725 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.725 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.725 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.726 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.726 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.726 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.726 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.726 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.726 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.726 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.726 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:25.726 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:25.726 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.726 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.726 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.726 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:25.727 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.727 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.727 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.727 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.727 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.727 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.727 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.727 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.727 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.727 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.727 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.727 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.727 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.727 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.728 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.728 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:25.728 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:25.728 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.728 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.728 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.728 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.728 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.728 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.728 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.728 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.728 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.728 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.728 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.729 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.729 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.729 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.729 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:25.729 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "volumes", 2024-06-03T03:28:25.729 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:25.729 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:25.729 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:25.729 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:25.729 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:25.729 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.729 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.729 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.729 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.730 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.730 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.730 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.730 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.730 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.730 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.730 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.730 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.730 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.730 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.730 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.730 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.730 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.731 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.731 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.731 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:25.731 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:25.731 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.731 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.731 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.731 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.731 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.731 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.731 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.731 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.731 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.731 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.732 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.732 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.732 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:25.732 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:25.732 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.732 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.732 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.732 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:25.732 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.732 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.732 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.732 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.732 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.732 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.733 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.733 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.733 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.733 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.733 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.733 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.733 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.733 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.733 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.733 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:25.733 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:25.733 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.733 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.733 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.734 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.734 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.734 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.734 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.734 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.734 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.734 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.734 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.734 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.734 INFO:teuthology.orchestra.run.smithi117.stdout: "max_concurrent_clones": { 2024-06-03T03:28:25.734 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "max_concurrent_clones", 2024-06-03T03:28:25.734 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:25.734 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.734 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.735 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "4", 2024-06-03T03:28:25.735 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.735 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.735 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.735 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Number of asynchronous cloner threads", 2024-06-03T03:28:25.735 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.735 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.735 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.735 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.735 INFO:teuthology.orchestra.run.smithi117.stdout: "periodic_async_work": { 2024-06-03T03:28:25.735 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "periodic_async_work", 2024-06-03T03:28:25.735 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.735 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.735 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.736 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.736 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.736 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.736 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.736 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Periodically check for async work", 2024-06-03T03:28:25.736 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.736 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.736 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.736 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.736 INFO:teuthology.orchestra.run.smithi117.stdout: "snapshot_clone_delay": { 2024-06-03T03:28:25.736 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "snapshot_clone_delay", 2024-06-03T03:28:25.736 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:25.736 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.737 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.737 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-06-03T03:28:25.737 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.737 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.737 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.737 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-06-03T03:28:25.737 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.737 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.737 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.737 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.737 INFO:teuthology.orchestra.run.smithi117.stdout: "snapshot_clone_no_wait": { 2024-06-03T03:28:25.737 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "snapshot_clone_no_wait", 2024-06-03T03:28:25.737 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.737 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.738 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.738 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:25.738 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.738 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.738 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.738 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-06-03T03:28:25.738 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.738 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.738 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.738 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.738 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.738 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.738 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:25.739 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "zabbix", 2024-06-03T03:28:25.739 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:25.739 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:25.739 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:25.739 INFO:teuthology.orchestra.run.smithi117.stdout: "discovery_interval": { 2024-06-03T03:28:25.739 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "discovery_interval", 2024-06-03T03:28:25.739 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "uint", 2024-06-03T03:28:25.739 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.739 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.739 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "100", 2024-06-03T03:28:25.739 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.739 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.739 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.739 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.740 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.740 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.740 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.740 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.740 INFO:teuthology.orchestra.run.smithi117.stdout: "identifier": { 2024-06-03T03:28:25.740 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "identifier", 2024-06-03T03:28:25.740 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.740 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.740 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.740 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.740 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.740 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.740 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.740 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.741 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.741 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.741 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.741 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.741 INFO:teuthology.orchestra.run.smithi117.stdout: "interval": { 2024-06-03T03:28:25.741 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "interval", 2024-06-03T03:28:25.741 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-06-03T03:28:25.741 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.741 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.741 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "60", 2024-06-03T03:28:25.741 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.741 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.741 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.742 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.742 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.742 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.742 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.742 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.742 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:25.742 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:25.742 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.742 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.742 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.742 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.742 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.742 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.742 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.743 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.743 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.743 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.743 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.743 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.743 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.743 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.743 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.743 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.743 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.743 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.743 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.743 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:25.743 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:25.744 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.744 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.744 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.744 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.744 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.744 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.744 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.744 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.744 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.744 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.744 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.744 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.744 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:25.744 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:25.745 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.745 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.745 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.745 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:25.745 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.745 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.745 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.745 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.745 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.745 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.745 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.745 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.745 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.745 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.746 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.746 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.746 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.746 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.746 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.746 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:25.746 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:25.746 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.746 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.746 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.746 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.746 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.746 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.746 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.747 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.747 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.747 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.747 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.747 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.747 INFO:teuthology.orchestra.run.smithi117.stdout: "zabbix_host": { 2024-06-03T03:28:25.747 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "zabbix_host", 2024-06-03T03:28:25.747 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.747 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.747 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.747 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.747 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.747 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.747 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.748 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.748 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.748 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.748 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.748 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.748 INFO:teuthology.orchestra.run.smithi117.stdout: "zabbix_port": { 2024-06-03T03:28:25.748 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "zabbix_port", 2024-06-03T03:28:25.748 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:25.748 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.748 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.748 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "10051", 2024-06-03T03:28:25.748 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.748 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.748 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.749 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.749 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.749 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.749 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.749 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.749 INFO:teuthology.orchestra.run.smithi117.stdout: "zabbix_sender": { 2024-06-03T03:28:25.749 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "zabbix_sender", 2024-06-03T03:28:25.749 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.749 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.749 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.749 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-06-03T03:28:25.749 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.749 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.749 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.750 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.750 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.750 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.750 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.750 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.750 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.750 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.750 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-06-03T03:28:25.750 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.750 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:25.750 INFO:teuthology.orchestra.run.smithi117.stdout: "gid": 24113, 2024-06-03T03:28:25.750 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "y", 2024-06-03T03:28:25.750 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr_features": 4540701547738038271, 2024-06-03T03:28:25.751 INFO:teuthology.orchestra.run.smithi117.stdout: "available_modules": [ 2024-06-03T03:28:25.751 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:25.751 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "alerts", 2024-06-03T03:28:25.751 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:25.751 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:25.751 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:25.751 INFO:teuthology.orchestra.run.smithi117.stdout: "interval": { 2024-06-03T03:28:25.751 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "interval", 2024-06-03T03:28:25.751 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-06-03T03:28:25.751 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.751 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.751 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "60", 2024-06-03T03:28:25.751 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.751 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.752 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.752 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "How frequently to reexamine health status", 2024-06-03T03:28:25.752 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.752 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.752 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.752 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.752 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:25.752 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:25.752 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.752 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.752 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.752 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.752 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.752 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.753 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.753 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.753 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.753 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.753 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.753 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.753 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.753 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.753 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.753 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.753 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.753 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.753 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.753 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:25.754 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:25.754 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.754 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.754 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.754 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.754 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.754 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.754 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.754 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.754 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.754 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.754 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.754 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.755 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:25.755 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:25.755 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.755 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.755 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.755 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:25.755 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.755 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.755 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.755 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.755 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.755 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.755 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.756 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.756 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.756 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.756 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.756 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.756 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.756 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.756 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.756 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:25.756 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:25.756 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.756 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.756 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.756 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.757 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.757 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.757 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.757 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.757 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.757 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.757 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.757 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.757 INFO:teuthology.orchestra.run.smithi117.stdout: "smtp_destination": { 2024-06-03T03:28:25.757 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "smtp_destination", 2024-06-03T03:28:25.757 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.757 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.757 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.757 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.758 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.758 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.758 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.758 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Email address to send alerts to", 2024-06-03T03:28:25.758 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.758 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.758 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.758 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.758 INFO:teuthology.orchestra.run.smithi117.stdout: "smtp_from_name": { 2024-06-03T03:28:25.758 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "smtp_from_name", 2024-06-03T03:28:25.758 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.758 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.758 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.759 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "Ceph", 2024-06-03T03:28:25.759 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.759 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.759 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.759 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Email From: name", 2024-06-03T03:28:25.759 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.759 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.759 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.759 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.759 INFO:teuthology.orchestra.run.smithi117.stdout: "smtp_host": { 2024-06-03T03:28:25.759 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "smtp_host", 2024-06-03T03:28:25.759 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.759 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.759 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.760 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.760 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.760 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.760 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.760 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "SMTP server", 2024-06-03T03:28:25.760 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.760 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.760 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.760 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.760 INFO:teuthology.orchestra.run.smithi117.stdout: "smtp_password": { 2024-06-03T03:28:25.760 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "smtp_password", 2024-06-03T03:28:25.760 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.760 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.760 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.761 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.761 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.761 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.761 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.761 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Password to authenticate with", 2024-06-03T03:28:25.761 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.761 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.761 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.761 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.761 INFO:teuthology.orchestra.run.smithi117.stdout: "smtp_port": { 2024-06-03T03:28:25.761 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "smtp_port", 2024-06-03T03:28:25.762 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:25.762 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.762 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.762 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "465", 2024-06-03T03:28:25.762 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.762 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.762 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.762 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "SMTP port", 2024-06-03T03:28:25.762 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.762 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.762 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.762 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.763 INFO:teuthology.orchestra.run.smithi117.stdout: "smtp_sender": { 2024-06-03T03:28:25.763 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "smtp_sender", 2024-06-03T03:28:25.763 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.763 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.763 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.763 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.763 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.763 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.763 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.763 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "SMTP envelope sender", 2024-06-03T03:28:25.763 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.763 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.763 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.763 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.764 INFO:teuthology.orchestra.run.smithi117.stdout: "smtp_ssl": { 2024-06-03T03:28:25.764 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "smtp_ssl", 2024-06-03T03:28:25.764 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.764 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.764 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.764 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:25.764 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.764 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.764 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.764 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Use SSL to connect to SMTP server", 2024-06-03T03:28:25.764 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.764 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.764 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.765 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.765 INFO:teuthology.orchestra.run.smithi117.stdout: "smtp_user": { 2024-06-03T03:28:25.765 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "smtp_user", 2024-06-03T03:28:25.765 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.765 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.765 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.765 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.765 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.765 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.765 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.765 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "User to authenticate as", 2024-06-03T03:28:25.765 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.765 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.766 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.766 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.766 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.766 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.766 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:25.766 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "balancer", 2024-06-03T03:28:25.766 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:25.766 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:25.766 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:25.766 INFO:teuthology.orchestra.run.smithi117.stdout: "active": { 2024-06-03T03:28:25.766 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "active", 2024-06-03T03:28:25.766 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.766 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.767 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.767 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:25.767 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.767 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.767 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.767 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "automatically balance PGs across cluster", 2024-06-03T03:28:25.767 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.767 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.767 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.767 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.767 INFO:teuthology.orchestra.run.smithi117.stdout: "begin_time": { 2024-06-03T03:28:25.767 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "begin_time", 2024-06-03T03:28:25.767 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.767 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.768 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.768 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0000", 2024-06-03T03:28:25.768 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.768 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.768 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.768 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "beginning time of day to automatically balance", 2024-06-03T03:28:25.768 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-03T03:28:25.768 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.768 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.768 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.768 INFO:teuthology.orchestra.run.smithi117.stdout: "begin_weekday": { 2024-06-03T03:28:25.768 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "begin_weekday", 2024-06-03T03:28:25.768 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "uint", 2024-06-03T03:28:25.769 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.769 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.769 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-06-03T03:28:25.769 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "0", 2024-06-03T03:28:25.769 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "6", 2024-06-03T03:28:25.769 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.769 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-06-03T03:28:25.769 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-03T03:28:25.769 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.769 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.769 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.769 INFO:teuthology.orchestra.run.smithi117.stdout: "crush_compat_max_iterations": { 2024-06-03T03:28:25.769 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crush_compat_max_iterations", 2024-06-03T03:28:25.770 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "uint", 2024-06-03T03:28:25.770 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.770 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.770 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "25", 2024-06-03T03:28:25.770 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "1", 2024-06-03T03:28:25.770 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "250", 2024-06-03T03:28:25.770 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.770 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-06-03T03:28:25.770 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.770 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.770 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.770 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.770 INFO:teuthology.orchestra.run.smithi117.stdout: "crush_compat_metrics": { 2024-06-03T03:28:25.771 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crush_compat_metrics", 2024-06-03T03:28:25.771 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.771 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.771 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.771 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "pgs,objects,bytes", 2024-06-03T03:28:25.771 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.771 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.771 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.771 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-06-03T03:28:25.771 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-06-03T03:28:25.771 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.771 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.771 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.771 INFO:teuthology.orchestra.run.smithi117.stdout: "crush_compat_step": { 2024-06-03T03:28:25.772 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crush_compat_step", 2024-06-03T03:28:25.772 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "float", 2024-06-03T03:28:25.772 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.772 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.772 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0.5", 2024-06-03T03:28:25.772 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "0.001", 2024-06-03T03:28:25.772 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "0.999", 2024-06-03T03:28:25.772 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.772 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "aggressiveness of optimization", 2024-06-03T03:28:25.772 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-06-03T03:28:25.772 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.772 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.772 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.772 INFO:teuthology.orchestra.run.smithi117.stdout: "end_time": { 2024-06-03T03:28:25.773 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "end_time", 2024-06-03T03:28:25.773 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.773 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.773 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.773 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "2359", 2024-06-03T03:28:25.773 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.773 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.773 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.773 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "ending time of day to automatically balance", 2024-06-03T03:28:25.773 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-03T03:28:25.773 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.773 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.773 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.773 INFO:teuthology.orchestra.run.smithi117.stdout: "end_weekday": { 2024-06-03T03:28:25.774 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "end_weekday", 2024-06-03T03:28:25.774 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "uint", 2024-06-03T03:28:25.774 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.774 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.774 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-06-03T03:28:25.774 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "0", 2024-06-03T03:28:25.774 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "6", 2024-06-03T03:28:25.774 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.774 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-06-03T03:28:25.774 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-03T03:28:25.774 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.774 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.774 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.774 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:25.775 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:25.775 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.775 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.775 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.775 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.775 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.775 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.775 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.775 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.775 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.775 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.775 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.775 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.776 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.776 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.776 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.776 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.776 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.776 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.776 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.776 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:25.776 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:25.776 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.776 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.776 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.776 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.776 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.777 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.777 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.777 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.777 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.777 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.777 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.777 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.777 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:25.777 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:25.777 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.777 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.777 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.777 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:25.778 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.778 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.778 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.778 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.778 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.778 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.778 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.778 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.778 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.778 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.778 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.778 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.778 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.779 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.779 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.779 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:25.779 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:25.779 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.779 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.779 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.779 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.779 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.779 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.779 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.779 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.779 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.779 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.780 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.780 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.780 INFO:teuthology.orchestra.run.smithi117.stdout: "min_score": { 2024-06-03T03:28:25.780 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "min_score", 2024-06-03T03:28:25.780 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "float", 2024-06-03T03:28:25.780 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.780 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.780 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-06-03T03:28:25.780 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.780 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.780 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.780 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-06-03T03:28:25.780 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.781 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.781 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.781 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.781 INFO:teuthology.orchestra.run.smithi117.stdout: "mode": { 2024-06-03T03:28:25.781 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "mode", 2024-06-03T03:28:25.781 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.781 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.781 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.781 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "upmap", 2024-06-03T03:28:25.781 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.781 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.781 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.781 INFO:teuthology.orchestra.run.smithi117.stdout: "crush-compat", 2024-06-03T03:28:25.781 INFO:teuthology.orchestra.run.smithi117.stdout: "none", 2024-06-03T03:28:25.782 INFO:teuthology.orchestra.run.smithi117.stdout: "read", 2024-06-03T03:28:25.782 INFO:teuthology.orchestra.run.smithi117.stdout: "upmap", 2024-06-03T03:28:25.782 INFO:teuthology.orchestra.run.smithi117.stdout: "upmap-read" 2024-06-03T03:28:25.782 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.782 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Balancer mode", 2024-06-03T03:28:25.782 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.782 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.782 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.782 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.782 INFO:teuthology.orchestra.run.smithi117.stdout: "pool_ids": { 2024-06-03T03:28:25.782 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "pool_ids", 2024-06-03T03:28:25.782 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.782 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.783 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.783 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.783 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.783 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.783 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.783 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-06-03T03:28:25.783 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.783 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.783 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.783 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.783 INFO:teuthology.orchestra.run.smithi117.stdout: "sleep_interval": { 2024-06-03T03:28:25.783 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sleep_interval", 2024-06-03T03:28:25.783 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-06-03T03:28:25.784 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.784 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.784 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "60", 2024-06-03T03:28:25.784 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.784 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.784 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.784 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-06-03T03:28:25.784 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.784 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.784 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.784 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.784 INFO:teuthology.orchestra.run.smithi117.stdout: "upmap_max_deviation": { 2024-06-03T03:28:25.784 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "upmap_max_deviation", 2024-06-03T03:28:25.785 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:25.785 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.785 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.785 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "5", 2024-06-03T03:28:25.785 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "1", 2024-06-03T03:28:25.785 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.785 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.785 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "deviation below which no optimization is attempted", 2024-06-03T03:28:25.785 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-06-03T03:28:25.785 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.785 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.785 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.785 INFO:teuthology.orchestra.run.smithi117.stdout: "upmap_max_optimizations": { 2024-06-03T03:28:25.785 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "upmap_max_optimizations", 2024-06-03T03:28:25.786 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "uint", 2024-06-03T03:28:25.786 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.786 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.786 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "10", 2024-06-03T03:28:25.786 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.786 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.786 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.786 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-06-03T03:28:25.786 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.786 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.786 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.786 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.786 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.786 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.787 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:25.787 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "cephadm", 2024-06-03T03:28:25.787 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:25.787 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:25.787 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:25.787 INFO:teuthology.orchestra.run.smithi117.stdout: "agent_down_multiplier": { 2024-06-03T03:28:25.787 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "agent_down_multiplier", 2024-06-03T03:28:25.787 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "float", 2024-06-03T03:28:25.787 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.787 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.787 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "3.0", 2024-06-03T03:28:25.787 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.787 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.788 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.788 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-06-03T03:28:25.788 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.788 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.788 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.788 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.788 INFO:teuthology.orchestra.run.smithi117.stdout: "agent_refresh_rate": { 2024-06-03T03:28:25.788 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "agent_refresh_rate", 2024-06-03T03:28:25.788 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-06-03T03:28:25.788 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.788 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.788 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "20", 2024-06-03T03:28:25.788 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.788 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.789 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.789 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-06-03T03:28:25.789 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.789 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.789 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.789 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.789 INFO:teuthology.orchestra.run.smithi117.stdout: "agent_starting_port": { 2024-06-03T03:28:25.789 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "agent_starting_port", 2024-06-03T03:28:25.789 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:25.789 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.789 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.789 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "4721", 2024-06-03T03:28:25.789 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.789 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.790 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.790 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-06-03T03:28:25.790 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.790 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.790 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.790 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.790 INFO:teuthology.orchestra.run.smithi117.stdout: "allow_ptrace": { 2024-06-03T03:28:25.790 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "allow_ptrace", 2024-06-03T03:28:25.790 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.790 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.790 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.790 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.790 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.790 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.791 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.791 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-06-03T03:28:25.791 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-06-03T03:28:25.791 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.791 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.791 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.791 INFO:teuthology.orchestra.run.smithi117.stdout: "autotune_interval": { 2024-06-03T03:28:25.791 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "autotune_interval", 2024-06-03T03:28:25.791 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-06-03T03:28:25.791 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.791 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.791 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "600", 2024-06-03T03:28:25.791 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.792 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.792 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.792 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "how frequently to autotune daemon memory", 2024-06-03T03:28:25.792 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.792 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.792 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.792 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.792 INFO:teuthology.orchestra.run.smithi117.stdout: "autotune_memory_target_ratio": { 2024-06-03T03:28:25.792 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "autotune_memory_target_ratio", 2024-06-03T03:28:25.792 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "float", 2024-06-03T03:28:25.792 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.792 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.792 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0.7", 2024-06-03T03:28:25.793 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.793 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.793 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.793 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-06-03T03:28:25.793 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.793 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.793 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.793 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.793 INFO:teuthology.orchestra.run.smithi117.stdout: "cephadm_log_destination": { 2024-06-03T03:28:25.793 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "cephadm_log_destination", 2024-06-03T03:28:25.793 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.793 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.793 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.793 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.794 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.794 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.794 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.794 INFO:teuthology.orchestra.run.smithi117.stdout: "file", 2024-06-03T03:28:25.794 INFO:teuthology.orchestra.run.smithi117.stdout: "file,syslog", 2024-06-03T03:28:25.794 INFO:teuthology.orchestra.run.smithi117.stdout: "syslog" 2024-06-03T03:28:25.794 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.794 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-06-03T03:28:25.794 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.794 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.794 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.794 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.794 INFO:teuthology.orchestra.run.smithi117.stdout: "cgroups_split": { 2024-06-03T03:28:25.795 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "cgroups_split", 2024-06-03T03:28:25.795 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.795 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.795 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.795 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:25.795 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.795 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.795 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.795 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-06-03T03:28:25.795 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.795 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.795 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.795 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.795 INFO:teuthology.orchestra.run.smithi117.stdout: "config_checks_enabled": { 2024-06-03T03:28:25.796 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "config_checks_enabled", 2024-06-03T03:28:25.796 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.796 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.796 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.796 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.796 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.796 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.796 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.796 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-06-03T03:28:25.796 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.796 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.796 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.796 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.797 INFO:teuthology.orchestra.run.smithi117.stdout: "config_dashboard": { 2024-06-03T03:28:25.797 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "config_dashboard", 2024-06-03T03:28:25.797 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.797 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.797 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.797 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:25.797 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.797 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.797 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.797 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-06-03T03:28:25.797 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.797 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.797 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.797 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.798 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_alertmanager": { 2024-06-03T03:28:25.798 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_alertmanager", 2024-06-03T03:28:25.798 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.798 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.798 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.798 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-06-03T03:28:25.798 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.798 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.798 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.798 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Prometheus container image", 2024-06-03T03:28:25.798 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.798 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.798 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.799 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.799 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_base": { 2024-06-03T03:28:25.799 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_base", 2024-06-03T03:28:25.799 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.799 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.799 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.799 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/ceph/ceph", 2024-06-03T03:28:25.799 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.799 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.799 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.799 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Container image name, without the tag", 2024-06-03T03:28:25.799 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.799 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.799 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.800 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.800 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_elasticsearch": { 2024-06-03T03:28:25.800 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_elasticsearch", 2024-06-03T03:28:25.800 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.800 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.800 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.800 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-06-03T03:28:25.800 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.800 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.800 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.800 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "elasticsearch container image", 2024-06-03T03:28:25.800 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.800 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.800 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.801 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.801 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_grafana": { 2024-06-03T03:28:25.801 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_grafana", 2024-06-03T03:28:25.801 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.801 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.801 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.801 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-06-03T03:28:25.801 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.801 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.801 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.801 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Prometheus container image", 2024-06-03T03:28:25.801 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.801 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.801 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.802 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.802 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_haproxy": { 2024-06-03T03:28:25.802 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_haproxy", 2024-06-03T03:28:25.802 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.802 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.802 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.802 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-06-03T03:28:25.802 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.802 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.802 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.802 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "HAproxy container image", 2024-06-03T03:28:25.802 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.802 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.802 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.803 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.803 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_jaeger_agent": { 2024-06-03T03:28:25.803 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_jaeger_agent", 2024-06-03T03:28:25.803 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.803 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.803 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.803 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-06-03T03:28:25.803 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.803 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.803 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.803 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Jaeger agent container image", 2024-06-03T03:28:25.803 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.804 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.804 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.804 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.804 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_jaeger_collector": { 2024-06-03T03:28:25.804 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_jaeger_collector", 2024-06-03T03:28:25.804 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.804 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.804 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.804 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-06-03T03:28:25.804 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.804 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.804 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.804 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Jaeger collector container image", 2024-06-03T03:28:25.804 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.805 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.805 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.805 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.805 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_jaeger_query": { 2024-06-03T03:28:25.805 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_jaeger_query", 2024-06-03T03:28:25.805 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.805 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.805 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.805 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-06-03T03:28:25.805 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.805 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.805 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.805 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Jaeger query container image", 2024-06-03T03:28:25.806 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.806 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.806 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.806 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.806 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_keepalived": { 2024-06-03T03:28:25.806 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_keepalived", 2024-06-03T03:28:25.806 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.806 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.806 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.806 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-06-03T03:28:25.806 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.806 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.806 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.806 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Keepalived container image", 2024-06-03T03:28:25.807 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.807 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.807 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.807 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.807 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_loki": { 2024-06-03T03:28:25.807 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_loki", 2024-06-03T03:28:25.807 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.807 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.807 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.807 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-06-03T03:28:25.807 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.807 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.807 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.808 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Loki container image", 2024-06-03T03:28:25.808 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.808 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.808 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.808 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.808 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_node_exporter": { 2024-06-03T03:28:25.808 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_node_exporter", 2024-06-03T03:28:25.808 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.808 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.808 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.808 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-06-03T03:28:25.808 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.808 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.808 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.809 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Prometheus container image", 2024-06-03T03:28:25.809 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.809 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.809 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.809 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.809 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_nvmeof": { 2024-06-03T03:28:25.809 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_nvmeof", 2024-06-03T03:28:25.809 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.809 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.809 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.809 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-06-03T03:28:25.809 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.809 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.809 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.810 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Nvme-of container image", 2024-06-03T03:28:25.810 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.810 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.810 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.810 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.810 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_prometheus": { 2024-06-03T03:28:25.810 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_prometheus", 2024-06-03T03:28:25.810 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.810 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.810 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.810 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-06-03T03:28:25.810 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.810 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.811 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.811 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Prometheus container image", 2024-06-03T03:28:25.811 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.811 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.811 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.811 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.811 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_promtail": { 2024-06-03T03:28:25.811 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_promtail", 2024-06-03T03:28:25.811 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.811 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.811 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.811 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-06-03T03:28:25.811 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.811 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.812 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.812 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Promtail container image", 2024-06-03T03:28:25.812 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.812 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.812 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.812 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.812 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_samba": { 2024-06-03T03:28:25.812 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_samba", 2024-06-03T03:28:25.812 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.812 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.812 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.812 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-06-03T03:28:25.812 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.812 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.813 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.813 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Samba/SMB container image", 2024-06-03T03:28:25.813 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.813 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.813 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.813 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.813 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_snmp_gateway": { 2024-06-03T03:28:25.813 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_snmp_gateway", 2024-06-03T03:28:25.813 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.813 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.813 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.813 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-06-03T03:28:25.813 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.813 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.814 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.814 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "SNMP Gateway container image", 2024-06-03T03:28:25.814 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.814 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.814 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.814 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.814 INFO:teuthology.orchestra.run.smithi117.stdout: "container_init": { 2024-06-03T03:28:25.814 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_init", 2024-06-03T03:28:25.814 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.814 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.814 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.814 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:25.814 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.815 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.815 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.815 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Run podman/docker with `--init`", 2024-06-03T03:28:25.815 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.815 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.815 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.815 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.815 INFO:teuthology.orchestra.run.smithi117.stdout: "daemon_cache_timeout": { 2024-06-03T03:28:25.815 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "daemon_cache_timeout", 2024-06-03T03:28:25.815 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-06-03T03:28:25.815 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.815 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.815 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "600", 2024-06-03T03:28:25.816 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.816 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.816 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.816 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-06-03T03:28:25.816 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.816 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.816 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.816 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.816 INFO:teuthology.orchestra.run.smithi117.stdout: "default_cephadm_command_timeout": { 2024-06-03T03:28:25.816 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "default_cephadm_command_timeout", 2024-06-03T03:28:25.816 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:25.816 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.816 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.816 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "900", 2024-06-03T03:28:25.817 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.817 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.817 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.817 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-06-03T03:28:25.817 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.817 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.817 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.817 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.817 INFO:teuthology.orchestra.run.smithi117.stdout: "default_registry": { 2024-06-03T03:28:25.817 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "default_registry", 2024-06-03T03:28:25.817 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.817 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.817 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.818 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "docker.io", 2024-06-03T03:28:25.818 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.818 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.818 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.818 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-06-03T03:28:25.818 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.818 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.818 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.818 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.818 INFO:teuthology.orchestra.run.smithi117.stdout: "device_cache_timeout": { 2024-06-03T03:28:25.818 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "device_cache_timeout", 2024-06-03T03:28:25.818 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-06-03T03:28:25.818 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.819 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.819 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "1800", 2024-06-03T03:28:25.819 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.819 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.819 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.819 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "seconds to cache device inventory", 2024-06-03T03:28:25.819 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.819 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.819 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.819 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.819 INFO:teuthology.orchestra.run.smithi117.stdout: "device_enhanced_scan": { 2024-06-03T03:28:25.819 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "device_enhanced_scan", 2024-06-03T03:28:25.819 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.819 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.820 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.820 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.820 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.820 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.820 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.820 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Use libstoragemgmt during device scans", 2024-06-03T03:28:25.820 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.820 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.820 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.820 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.820 INFO:teuthology.orchestra.run.smithi117.stdout: "facts_cache_timeout": { 2024-06-03T03:28:25.820 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "facts_cache_timeout", 2024-06-03T03:28:25.820 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-06-03T03:28:25.821 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.821 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.821 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "60", 2024-06-03T03:28:25.821 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.821 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.821 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.821 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "seconds to cache host facts data", 2024-06-03T03:28:25.821 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.821 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.821 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.821 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.821 INFO:teuthology.orchestra.run.smithi117.stdout: "grafana_dashboards_path": { 2024-06-03T03:28:25.821 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "grafana_dashboards_path", 2024-06-03T03:28:25.821 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.822 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.822 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.822 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-06-03T03:28:25.822 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.822 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.822 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.822 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-06-03T03:28:25.822 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.822 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.822 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.822 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.822 INFO:teuthology.orchestra.run.smithi117.stdout: "host_check_interval": { 2024-06-03T03:28:25.822 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "host_check_interval", 2024-06-03T03:28:25.823 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-06-03T03:28:25.823 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.823 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.823 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "600", 2024-06-03T03:28:25.823 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.823 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.823 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.823 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "how frequently to perform a host check", 2024-06-03T03:28:25.823 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.823 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.823 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.823 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.823 INFO:teuthology.orchestra.run.smithi117.stdout: "hw_monitoring": { 2024-06-03T03:28:25.823 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "hw_monitoring", 2024-06-03T03:28:25.824 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.824 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.824 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.824 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.824 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.824 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.824 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.824 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-06-03T03:28:25.824 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.824 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.824 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.824 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.824 INFO:teuthology.orchestra.run.smithi117.stdout: "inventory_list_all": { 2024-06-03T03:28:25.824 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "inventory_list_all", 2024-06-03T03:28:25.825 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.825 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.825 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.825 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.825 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.825 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.825 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.825 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-06-03T03:28:25.825 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.825 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.825 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.825 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.825 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:25.825 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:25.826 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.826 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.826 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.826 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.826 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.826 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.826 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.826 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.826 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.826 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.826 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.826 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.827 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.827 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.827 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.827 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.827 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.827 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.827 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.827 INFO:teuthology.orchestra.run.smithi117.stdout: "log_refresh_metadata": { 2024-06-03T03:28:25.827 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_refresh_metadata", 2024-06-03T03:28:25.827 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.827 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.827 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.827 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.827 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.828 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.828 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.828 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-06-03T03:28:25.828 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.828 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.828 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.828 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.828 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:25.828 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:25.828 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.828 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.828 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.828 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:25.829 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.829 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.829 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.829 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-06-03T03:28:25.829 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.829 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.829 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.829 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.829 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:25.829 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:25.829 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.829 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.829 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.830 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:25.830 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.830 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.830 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.830 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.830 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.830 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.830 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.830 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.830 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.830 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.830 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.830 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.830 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.831 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.831 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.831 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:25.831 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:25.831 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.831 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.831 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.831 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.831 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.831 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.831 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.831 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.831 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.832 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.832 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.832 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.832 INFO:teuthology.orchestra.run.smithi117.stdout: "manage_etc_ceph_ceph_conf": { 2024-06-03T03:28:25.832 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-06-03T03:28:25.832 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.832 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.832 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.832 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.832 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.832 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.832 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.832 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-06-03T03:28:25.832 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.833 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.833 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.833 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.833 INFO:teuthology.orchestra.run.smithi117.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-06-03T03:28:25.833 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-06-03T03:28:25.833 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.833 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.833 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.833 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "*", 2024-06-03T03:28:25.833 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.833 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.833 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.833 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-06-03T03:28:25.834 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.834 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.834 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.834 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.834 INFO:teuthology.orchestra.run.smithi117.stdout: "max_count_per_host": { 2024-06-03T03:28:25.834 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "max_count_per_host", 2024-06-03T03:28:25.834 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:25.834 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.834 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.834 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "10", 2024-06-03T03:28:25.834 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.834 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.834 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.835 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "max number of daemons per service per host", 2024-06-03T03:28:25.835 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.835 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.835 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.835 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.835 INFO:teuthology.orchestra.run.smithi117.stdout: "max_osd_draining_count": { 2024-06-03T03:28:25.835 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "max_osd_draining_count", 2024-06-03T03:28:25.835 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:25.835 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.835 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.835 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "10", 2024-06-03T03:28:25.835 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.835 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.835 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.835 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-06-03T03:28:25.836 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.836 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.836 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.836 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.836 INFO:teuthology.orchestra.run.smithi117.stdout: "migration_current": { 2024-06-03T03:28:25.836 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "migration_current", 2024-06-03T03:28:25.836 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:25.836 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.836 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.836 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.836 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.836 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.836 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.837 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "internal - do not modify", 2024-06-03T03:28:25.837 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.837 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.837 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.837 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.837 INFO:teuthology.orchestra.run.smithi117.stdout: "mode": { 2024-06-03T03:28:25.837 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "mode", 2024-06-03T03:28:25.837 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.837 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.837 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.837 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "root", 2024-06-03T03:28:25.837 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.837 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.837 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.838 INFO:teuthology.orchestra.run.smithi117.stdout: "cephadm-package", 2024-06-03T03:28:25.838 INFO:teuthology.orchestra.run.smithi117.stdout: "root" 2024-06-03T03:28:25.838 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.838 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "mode for remote execution of cephadm", 2024-06-03T03:28:25.838 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.838 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.838 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.838 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.838 INFO:teuthology.orchestra.run.smithi117.stdout: "oob_default_addr": { 2024-06-03T03:28:25.838 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "oob_default_addr", 2024-06-03T03:28:25.838 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.838 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.838 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.839 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "169.254.1.1", 2024-06-03T03:28:25.839 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.839 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.839 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.839 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Default address for RedFish API (oob management).", 2024-06-03T03:28:25.839 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.839 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.839 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.839 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.839 INFO:teuthology.orchestra.run.smithi117.stdout: "prometheus_alerts_path": { 2024-06-03T03:28:25.839 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "prometheus_alerts_path", 2024-06-03T03:28:25.839 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.839 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.840 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.840 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-06-03T03:28:25.840 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.840 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.840 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.840 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-06-03T03:28:25.840 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.840 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.840 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.840 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.840 INFO:teuthology.orchestra.run.smithi117.stdout: "registry_insecure": { 2024-06-03T03:28:25.840 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "registry_insecure", 2024-06-03T03:28:25.840 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.840 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.841 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.841 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.841 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.841 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.841 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.841 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-06-03T03:28:25.841 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.841 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.841 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.841 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.841 INFO:teuthology.orchestra.run.smithi117.stdout: "registry_password": { 2024-06-03T03:28:25.841 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "registry_password", 2024-06-03T03:28:25.841 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.842 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.842 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.842 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.842 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.842 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.842 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.842 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-06-03T03:28:25.842 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.842 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.842 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.842 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.842 INFO:teuthology.orchestra.run.smithi117.stdout: "registry_url": { 2024-06-03T03:28:25.842 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "registry_url", 2024-06-03T03:28:25.843 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.843 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.843 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.843 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.843 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.843 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.843 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.843 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-06-03T03:28:25.843 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.843 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.843 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.843 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.843 INFO:teuthology.orchestra.run.smithi117.stdout: "registry_username": { 2024-06-03T03:28:25.843 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "registry_username", 2024-06-03T03:28:25.844 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.844 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.844 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.844 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.844 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.844 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.844 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.844 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-06-03T03:28:25.844 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.844 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.844 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.844 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.844 INFO:teuthology.orchestra.run.smithi117.stdout: "secure_monitoring_stack": { 2024-06-03T03:28:25.844 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "secure_monitoring_stack", 2024-06-03T03:28:25.845 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.845 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.845 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.845 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.845 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.845 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.845 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.845 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-06-03T03:28:25.845 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.845 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.845 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.845 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.845 INFO:teuthology.orchestra.run.smithi117.stdout: "service_discovery_port": { 2024-06-03T03:28:25.846 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "service_discovery_port", 2024-06-03T03:28:25.846 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:25.846 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.846 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.846 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "8765", 2024-06-03T03:28:25.846 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.846 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.846 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.846 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "cephadm service discovery port", 2024-06-03T03:28:25.846 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.846 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.846 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.846 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.847 INFO:teuthology.orchestra.run.smithi117.stdout: "ssh_config_file": { 2024-06-03T03:28:25.847 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "ssh_config_file", 2024-06-03T03:28:25.847 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.847 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.847 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.847 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.847 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.847 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.847 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.847 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-06-03T03:28:25.847 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.847 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.847 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.847 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.848 INFO:teuthology.orchestra.run.smithi117.stdout: "use_agent": { 2024-06-03T03:28:25.848 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "use_agent", 2024-06-03T03:28:25.848 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.848 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.848 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.848 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.848 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.848 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.848 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.848 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-06-03T03:28:25.848 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.848 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.848 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.848 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.848 INFO:teuthology.orchestra.run.smithi117.stdout: "use_repo_digest": { 2024-06-03T03:28:25.849 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "use_repo_digest", 2024-06-03T03:28:25.849 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.849 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.849 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.849 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:25.849 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.849 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.849 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.849 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-06-03T03:28:25.849 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.849 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.849 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.849 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.849 INFO:teuthology.orchestra.run.smithi117.stdout: "warn_on_failed_host_check": { 2024-06-03T03:28:25.850 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "warn_on_failed_host_check", 2024-06-03T03:28:25.850 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.850 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.850 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.850 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:25.850 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.850 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.850 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.850 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "raise a health warning if the host check fails", 2024-06-03T03:28:25.850 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.850 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.850 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.850 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.851 INFO:teuthology.orchestra.run.smithi117.stdout: "warn_on_stray_daemons": { 2024-06-03T03:28:25.851 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "warn_on_stray_daemons", 2024-06-03T03:28:25.851 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.851 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.851 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.851 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:25.851 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.851 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.851 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.851 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-06-03T03:28:25.851 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.851 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.851 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.851 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.852 INFO:teuthology.orchestra.run.smithi117.stdout: "warn_on_stray_hosts": { 2024-06-03T03:28:25.852 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "warn_on_stray_hosts", 2024-06-03T03:28:25.852 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.852 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.852 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.852 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:25.852 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.852 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.852 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.852 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-06-03T03:28:25.852 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.852 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.852 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.852 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.853 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.853 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.853 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:25.853 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crash", 2024-06-03T03:28:25.853 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:25.853 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:25.853 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:25.853 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:25.853 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:25.853 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.853 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.853 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.853 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.854 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.854 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.854 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.854 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.854 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.854 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.854 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.854 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.854 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.854 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.854 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.854 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.854 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.855 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.855 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.855 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:25.855 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:25.855 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.855 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.855 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.855 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.855 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.855 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.855 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.855 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.855 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.856 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.856 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.856 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.856 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:25.856 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:25.856 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.856 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.856 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.856 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:25.856 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.856 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.856 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.856 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.856 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.857 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.857 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.857 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.857 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.857 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.857 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.857 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.857 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.857 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.857 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.857 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:25.857 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:25.857 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.858 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.858 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.858 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.858 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.858 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.858 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.858 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.858 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.858 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.858 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.858 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.858 INFO:teuthology.orchestra.run.smithi117.stdout: "retain_interval": { 2024-06-03T03:28:25.858 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "retain_interval", 2024-06-03T03:28:25.858 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-06-03T03:28:25.859 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.859 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.859 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "31536000", 2024-06-03T03:28:25.859 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.859 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.859 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.859 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "how long to retain crashes before pruning them", 2024-06-03T03:28:25.859 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.859 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.859 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.859 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.859 INFO:teuthology.orchestra.run.smithi117.stdout: "warn_recent_interval": { 2024-06-03T03:28:25.859 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "warn_recent_interval", 2024-06-03T03:28:25.860 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-06-03T03:28:25.860 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.860 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.860 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "1209600", 2024-06-03T03:28:25.860 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.860 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.860 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.860 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "time interval in which to warn about recent crashes", 2024-06-03T03:28:25.860 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.860 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.860 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.860 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.860 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.860 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.860 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:25.861 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "dashboard", 2024-06-03T03:28:25.861 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:25.861 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:25.861 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:25.861 INFO:teuthology.orchestra.run.smithi117.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-06-03T03:28:25.861 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-06-03T03:28:25.861 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:25.861 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.861 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.861 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "10", 2024-06-03T03:28:25.861 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.862 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.862 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.862 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.862 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.862 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.862 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.862 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.862 INFO:teuthology.orchestra.run.smithi117.stdout: "ALERTMANAGER_API_HOST": { 2024-06-03T03:28:25.862 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "ALERTMANAGER_API_HOST", 2024-06-03T03:28:25.862 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.862 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.862 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.863 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.863 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.863 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.863 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.863 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.863 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.863 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.863 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.863 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.863 INFO:teuthology.orchestra.run.smithi117.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-06-03T03:28:25.863 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-06-03T03:28:25.863 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.863 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.864 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.864 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:25.864 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.864 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.864 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.864 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.864 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.864 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.864 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.864 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.864 INFO:teuthology.orchestra.run.smithi117.stdout: "AUDIT_API_ENABLED": { 2024-06-03T03:28:25.864 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "AUDIT_API_ENABLED", 2024-06-03T03:28:25.864 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.864 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.865 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.865 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.865 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.865 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.865 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.865 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.865 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.865 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.865 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.865 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.865 INFO:teuthology.orchestra.run.smithi117.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-06-03T03:28:25.865 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-06-03T03:28:25.865 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.865 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.866 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.866 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:25.866 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.866 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.866 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.866 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.866 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.866 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.866 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.866 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.866 INFO:teuthology.orchestra.run.smithi117.stdout: "ENABLE_BROWSABLE_API": { 2024-06-03T03:28:25.866 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "ENABLE_BROWSABLE_API", 2024-06-03T03:28:25.867 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.867 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.867 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.867 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:25.867 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.867 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.867 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.867 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.867 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.867 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.867 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.867 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.867 INFO:teuthology.orchestra.run.smithi117.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-06-03T03:28:25.867 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-06-03T03:28:25.868 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.868 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.868 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.868 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:25.868 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.868 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.868 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.868 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.868 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.868 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.868 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.868 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.869 INFO:teuthology.orchestra.run.smithi117.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-06-03T03:28:25.869 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-06-03T03:28:25.869 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.869 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.869 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.869 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:25.869 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.869 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.869 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.869 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.869 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.869 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.869 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.869 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.870 INFO:teuthology.orchestra.run.smithi117.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-06-03T03:28:25.870 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-06-03T03:28:25.870 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.870 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.870 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.870 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:25.870 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.870 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.870 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.870 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.870 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.870 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.870 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.871 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.871 INFO:teuthology.orchestra.run.smithi117.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-06-03T03:28:25.871 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-06-03T03:28:25.871 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.871 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.871 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.871 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:25.871 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.871 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.871 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.871 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.871 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.871 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.872 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.872 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.872 INFO:teuthology.orchestra.run.smithi117.stdout: "FEATURE_TOGGLE_NFS": { 2024-06-03T03:28:25.872 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-06-03T03:28:25.872 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.872 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.872 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.872 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:25.872 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.872 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.872 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.872 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.872 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.873 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.873 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.873 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.873 INFO:teuthology.orchestra.run.smithi117.stdout: "FEATURE_TOGGLE_RBD": { 2024-06-03T03:28:25.873 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-06-03T03:28:25.873 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.873 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.873 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.873 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:25.873 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.873 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.873 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.873 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.873 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.874 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.874 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.874 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.874 INFO:teuthology.orchestra.run.smithi117.stdout: "FEATURE_TOGGLE_RGW": { 2024-06-03T03:28:25.874 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-06-03T03:28:25.874 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.874 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.874 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.874 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:25.874 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.874 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.874 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.874 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.875 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.875 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.875 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.875 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.875 INFO:teuthology.orchestra.run.smithi117.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-06-03T03:28:25.875 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-06-03T03:28:25.875 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.875 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.875 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.875 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.875 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.875 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.875 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.876 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.876 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.876 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.876 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.876 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.876 INFO:teuthology.orchestra.run.smithi117.stdout: "GRAFANA_API_PASSWORD": { 2024-06-03T03:28:25.876 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "GRAFANA_API_PASSWORD", 2024-06-03T03:28:25.876 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.876 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.876 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.876 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "admin", 2024-06-03T03:28:25.876 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.876 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.876 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.877 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.877 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.877 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.877 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.877 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.877 INFO:teuthology.orchestra.run.smithi117.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-06-03T03:28:25.877 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-06-03T03:28:25.877 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.877 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.877 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.877 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:25.877 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.877 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.878 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.878 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.878 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.878 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.878 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.878 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.878 INFO:teuthology.orchestra.run.smithi117.stdout: "GRAFANA_API_URL": { 2024-06-03T03:28:25.878 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "GRAFANA_API_URL", 2024-06-03T03:28:25.878 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.878 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.878 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.878 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.878 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.879 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.879 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.879 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.879 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.879 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.879 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.879 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.879 INFO:teuthology.orchestra.run.smithi117.stdout: "GRAFANA_API_USERNAME": { 2024-06-03T03:28:25.879 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "GRAFANA_API_USERNAME", 2024-06-03T03:28:25.879 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.879 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.879 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.879 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "admin", 2024-06-03T03:28:25.879 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.880 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.880 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.880 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.880 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.880 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.880 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.880 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.880 INFO:teuthology.orchestra.run.smithi117.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-06-03T03:28:25.880 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-06-03T03:28:25.880 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.880 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.880 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.880 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.880 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.881 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.881 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.881 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.881 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.881 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.881 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.881 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.881 INFO:teuthology.orchestra.run.smithi117.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-06-03T03:28:25.881 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-06-03T03:28:25.881 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.881 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.881 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.881 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.882 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.882 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.882 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.882 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.882 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.882 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.883 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:28:25 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/2189770831' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-03T03:28:25.883 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:28:25 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/2684813020' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-06-03T03:28:25.883 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:28:25 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/2189770831' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-03T03:28:25.883 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:28:25 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/2684813020' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-06-03T03:28:25.886 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.887 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.887 INFO:teuthology.orchestra.run.smithi117.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-06-03T03:28:25.887 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-06-03T03:28:25.887 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.887 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.887 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.887 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:25.887 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.887 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.887 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.887 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.887 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.888 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.888 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.888 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.888 INFO:teuthology.orchestra.run.smithi117.stdout: "ISSUE_TRACKER_API_KEY": { 2024-06-03T03:28:25.888 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-06-03T03:28:25.888 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.888 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.888 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.888 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.888 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.888 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.888 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.889 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.889 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.889 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.889 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.889 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.889 INFO:teuthology.orchestra.run.smithi117.stdout: "PROMETHEUS_API_HOST": { 2024-06-03T03:28:25.889 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "PROMETHEUS_API_HOST", 2024-06-03T03:28:25.889 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.889 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.889 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.889 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.889 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.889 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.890 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.890 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.890 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.890 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.890 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.890 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.890 INFO:teuthology.orchestra.run.smithi117.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-06-03T03:28:25.890 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-06-03T03:28:25.890 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.890 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.890 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.891 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:25.891 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.891 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.891 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.891 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.891 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.891 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.891 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.891 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.891 INFO:teuthology.orchestra.run.smithi117.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-06-03T03:28:25.891 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-06-03T03:28:25.891 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.891 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.892 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.892 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.892 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.892 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.892 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.892 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.892 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.892 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.892 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.892 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.892 INFO:teuthology.orchestra.run.smithi117.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-06-03T03:28:25.892 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-06-03T03:28:25.892 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.892 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.893 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.893 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.893 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.893 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.893 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.893 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.893 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.893 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.893 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.893 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.893 INFO:teuthology.orchestra.run.smithi117.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-06-03T03:28:25.893 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-06-03T03:28:25.893 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.894 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.894 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.894 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:25.894 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.894 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.894 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.894 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.894 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.894 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.894 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.894 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.894 INFO:teuthology.orchestra.run.smithi117.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-06-03T03:28:25.895 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-06-03T03:28:25.895 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.895 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.895 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.895 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:25.895 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.895 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.895 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.895 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.895 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.895 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.895 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.895 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.895 INFO:teuthology.orchestra.run.smithi117.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-06-03T03:28:25.896 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-06-03T03:28:25.896 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.896 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.896 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.896 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.896 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.896 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.896 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.896 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.896 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.896 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.896 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.896 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.897 INFO:teuthology.orchestra.run.smithi117.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-06-03T03:28:25.897 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-06-03T03:28:25.897 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.897 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.897 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.897 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.897 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.897 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.897 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.897 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.897 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.897 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.897 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.897 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.898 INFO:teuthology.orchestra.run.smithi117.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-06-03T03:28:25.898 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-06-03T03:28:25.898 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.898 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.898 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.898 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.898 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.898 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.898 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.898 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.898 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.898 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.899 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.899 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.899 INFO:teuthology.orchestra.run.smithi117.stdout: "PWD_POLICY_ENABLED": { 2024-06-03T03:28:25.899 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "PWD_POLICY_ENABLED", 2024-06-03T03:28:25.899 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.899 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.899 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.899 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:25.899 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.899 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.899 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.899 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.899 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.899 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.900 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.900 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.900 INFO:teuthology.orchestra.run.smithi117.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-06-03T03:28:25.900 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-06-03T03:28:25.900 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.900 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.900 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.900 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-06-03T03:28:25.900 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.900 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.900 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.900 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.900 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.901 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.901 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.901 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.901 INFO:teuthology.orchestra.run.smithi117.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-06-03T03:28:25.901 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-06-03T03:28:25.901 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:25.901 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.901 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.901 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "10", 2024-06-03T03:28:25.901 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.901 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.901 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.901 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.902 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.902 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.902 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.902 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.902 INFO:teuthology.orchestra.run.smithi117.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-06-03T03:28:25.902 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-06-03T03:28:25.902 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:25.902 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.902 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.902 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "8", 2024-06-03T03:28:25.902 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.902 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.902 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.903 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.903 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.903 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.903 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.903 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.903 INFO:teuthology.orchestra.run.smithi117.stdout: "REST_REQUESTS_TIMEOUT": { 2024-06-03T03:28:25.903 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-06-03T03:28:25.903 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:25.903 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.903 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.903 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "45", 2024-06-03T03:28:25.903 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.903 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.904 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.904 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.904 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.904 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.904 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.904 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.904 INFO:teuthology.orchestra.run.smithi117.stdout: "RGW_API_ACCESS_KEY": { 2024-06-03T03:28:25.904 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "RGW_API_ACCESS_KEY", 2024-06-03T03:28:25.904 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.904 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.904 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.904 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.904 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.905 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.905 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.905 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.905 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.905 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.905 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.905 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.905 INFO:teuthology.orchestra.run.smithi117.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-06-03T03:28:25.905 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-06-03T03:28:25.905 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.905 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.905 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.905 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "admin", 2024-06-03T03:28:25.905 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.906 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.906 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.906 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.906 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.906 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.906 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.906 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.906 INFO:teuthology.orchestra.run.smithi117.stdout: "RGW_API_SECRET_KEY": { 2024-06-03T03:28:25.906 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "RGW_API_SECRET_KEY", 2024-06-03T03:28:25.906 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.906 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.906 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.906 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.907 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.907 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.907 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.907 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.907 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.907 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.907 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.907 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.907 INFO:teuthology.orchestra.run.smithi117.stdout: "RGW_API_SSL_VERIFY": { 2024-06-03T03:28:25.907 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "RGW_API_SSL_VERIFY", 2024-06-03T03:28:25.907 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.907 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.907 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.908 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:25.908 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.908 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.908 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.908 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.908 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.908 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.908 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.908 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.908 INFO:teuthology.orchestra.run.smithi117.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-06-03T03:28:25.908 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-06-03T03:28:25.908 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:25.908 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.908 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.909 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-06-03T03:28:25.909 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.909 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.909 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.909 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.909 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.909 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.909 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.909 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.909 INFO:teuthology.orchestra.run.smithi117.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-06-03T03:28:25.909 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-06-03T03:28:25.909 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:25.909 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.909 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.910 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "10", 2024-06-03T03:28:25.910 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.910 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.910 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.910 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.910 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.910 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.910 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.910 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.910 INFO:teuthology.orchestra.run.smithi117.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-06-03T03:28:25.910 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-06-03T03:28:25.910 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:25.910 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.911 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.911 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "5", 2024-06-03T03:28:25.911 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.911 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.911 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.911 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.911 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.911 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.911 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.911 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.911 INFO:teuthology.orchestra.run.smithi117.stdout: "cross_origin_url": { 2024-06-03T03:28:25.911 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "cross_origin_url", 2024-06-03T03:28:25.911 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.911 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.912 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.912 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.912 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.912 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.912 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.912 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.912 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.912 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.912 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.912 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.912 INFO:teuthology.orchestra.run.smithi117.stdout: "crt_file": { 2024-06-03T03:28:25.912 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crt_file", 2024-06-03T03:28:25.912 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.913 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.913 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.913 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.913 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.913 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.913 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.913 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.913 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.913 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.913 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.913 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.913 INFO:teuthology.orchestra.run.smithi117.stdout: "debug": { 2024-06-03T03:28:25.913 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "debug", 2024-06-03T03:28:25.913 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.914 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.914 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.914 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.914 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.914 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.914 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.914 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Enable/disable debug options", 2024-06-03T03:28:25.914 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.914 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.914 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.914 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.914 INFO:teuthology.orchestra.run.smithi117.stdout: "jwt_token_ttl": { 2024-06-03T03:28:25.915 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "jwt_token_ttl", 2024-06-03T03:28:25.915 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:25.915 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.915 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.915 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "28800", 2024-06-03T03:28:25.915 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.915 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.915 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.915 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.915 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.915 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.915 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.915 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.915 INFO:teuthology.orchestra.run.smithi117.stdout: "key_file": { 2024-06-03T03:28:25.916 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "key_file", 2024-06-03T03:28:25.916 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.916 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.916 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.916 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.916 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.916 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.916 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.916 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.916 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.916 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.916 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.916 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.916 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:25.917 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:25.917 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.917 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.917 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.917 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.917 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.917 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.917 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.917 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.917 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.917 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.917 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.918 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.918 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.918 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.918 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.918 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.918 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.918 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.918 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.918 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:25.918 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:25.918 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.918 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.918 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.919 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.919 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.919 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.919 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.919 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.919 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.919 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.919 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.919 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.919 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:25.919 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:25.919 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.919 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.919 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.920 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:25.920 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.920 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.920 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.920 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.920 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.920 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.920 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.920 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.920 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.920 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.920 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.920 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.921 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.921 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.921 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.921 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:25.921 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:25.921 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.921 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.921 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.921 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.921 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.921 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.921 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.921 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.922 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.922 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.922 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.922 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.922 INFO:teuthology.orchestra.run.smithi117.stdout: "motd": { 2024-06-03T03:28:25.922 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "motd", 2024-06-03T03:28:25.922 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.922 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.922 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.922 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.922 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.922 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.922 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.923 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "The message of the day", 2024-06-03T03:28:25.923 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.923 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.923 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.923 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.923 INFO:teuthology.orchestra.run.smithi117.stdout: "redirect_resolve_ip_addr": { 2024-06-03T03:28:25.923 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "redirect_resolve_ip_addr", 2024-06-03T03:28:25.923 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.923 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.923 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.923 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.923 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.923 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.924 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.924 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.924 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.924 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.924 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.924 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.924 INFO:teuthology.orchestra.run.smithi117.stdout: "server_addr": { 2024-06-03T03:28:25.924 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "server_addr", 2024-06-03T03:28:25.924 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.924 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.924 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.924 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "::", 2024-06-03T03:28:25.924 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.925 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.925 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.925 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.925 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.925 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.925 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.925 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.925 INFO:teuthology.orchestra.run.smithi117.stdout: "server_port": { 2024-06-03T03:28:25.925 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "server_port", 2024-06-03T03:28:25.925 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:25.925 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.925 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.925 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "8080", 2024-06-03T03:28:25.926 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.926 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.926 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.926 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.926 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.926 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.926 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.926 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.926 INFO:teuthology.orchestra.run.smithi117.stdout: "ssl": { 2024-06-03T03:28:25.926 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "ssl", 2024-06-03T03:28:25.926 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.926 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.926 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.927 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:25.927 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.927 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.927 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.927 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.927 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.927 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.927 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.927 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.927 INFO:teuthology.orchestra.run.smithi117.stdout: "ssl_server_port": { 2024-06-03T03:28:25.927 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "ssl_server_port", 2024-06-03T03:28:25.927 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:25.927 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.927 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.928 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "8443", 2024-06-03T03:28:25.928 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.928 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.928 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.928 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.928 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.928 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.928 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.928 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.928 INFO:teuthology.orchestra.run.smithi117.stdout: "standby_behaviour": { 2024-06-03T03:28:25.928 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "standby_behaviour", 2024-06-03T03:28:25.928 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.928 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.929 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.929 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "redirect", 2024-06-03T03:28:25.929 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.929 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.929 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.929 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.929 INFO:teuthology.orchestra.run.smithi117.stdout: "redirect" 2024-06-03T03:28:25.929 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.929 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.929 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.929 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.929 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.930 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.930 INFO:teuthology.orchestra.run.smithi117.stdout: "standby_error_status_code": { 2024-06-03T03:28:25.930 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "standby_error_status_code", 2024-06-03T03:28:25.930 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:25.930 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.930 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.930 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "500", 2024-06-03T03:28:25.930 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "400", 2024-06-03T03:28:25.930 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "599", 2024-06-03T03:28:25.930 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.930 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.930 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.930 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.931 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.931 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.931 INFO:teuthology.orchestra.run.smithi117.stdout: "url_prefix": { 2024-06-03T03:28:25.931 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "url_prefix", 2024-06-03T03:28:25.931 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.931 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.931 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.931 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.931 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.931 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.931 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.931 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.931 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.932 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.932 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.932 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.932 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.932 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.932 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:25.932 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "devicehealth", 2024-06-03T03:28:25.932 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:25.932 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:25.932 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:25.932 INFO:teuthology.orchestra.run.smithi117.stdout: "enable_monitoring": { 2024-06-03T03:28:25.932 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "enable_monitoring", 2024-06-03T03:28:25.932 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.933 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.933 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.933 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:25.933 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.933 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.933 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.933 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "monitor device health metrics", 2024-06-03T03:28:25.933 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.933 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.933 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.933 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.933 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:25.933 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:25.934 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.934 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.934 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.934 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.934 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.934 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.934 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.934 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.934 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.934 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.934 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.934 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.935 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.935 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.935 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.935 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.935 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.935 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.935 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.935 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:25.935 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:25.935 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.935 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.935 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.935 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.936 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.936 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.936 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.936 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.936 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.936 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.936 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.936 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.936 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:25.936 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:25.936 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.936 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.936 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.937 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:25.937 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.937 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.937 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.937 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.937 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.937 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.937 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.937 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.937 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.937 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.937 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.937 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.937 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.938 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.938 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.938 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:25.938 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:25.938 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.938 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.938 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.938 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.938 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.938 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.938 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.938 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.938 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.939 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.939 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.939 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.939 INFO:teuthology.orchestra.run.smithi117.stdout: "mark_out_threshold": { 2024-06-03T03:28:25.939 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "mark_out_threshold", 2024-06-03T03:28:25.939 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-06-03T03:28:25.939 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.939 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.939 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "2419200", 2024-06-03T03:28:25.939 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.939 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.939 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.939 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-06-03T03:28:25.940 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.940 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.940 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.940 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.940 INFO:teuthology.orchestra.run.smithi117.stdout: "pool_name": { 2024-06-03T03:28:25.940 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "pool_name", 2024-06-03T03:28:25.940 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.940 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.940 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.940 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "device_health_metrics", 2024-06-03T03:28:25.940 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.940 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.940 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.940 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "name of pool in which to store device health metrics", 2024-06-03T03:28:25.941 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.941 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.941 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.941 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.941 INFO:teuthology.orchestra.run.smithi117.stdout: "retention_period": { 2024-06-03T03:28:25.941 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "retention_period", 2024-06-03T03:28:25.941 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-06-03T03:28:25.941 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.941 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.941 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "15552000", 2024-06-03T03:28:25.941 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.941 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.941 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.941 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "how long to retain device health metrics", 2024-06-03T03:28:25.941 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.942 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.942 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.942 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.942 INFO:teuthology.orchestra.run.smithi117.stdout: "scrape_frequency": { 2024-06-03T03:28:25.942 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "scrape_frequency", 2024-06-03T03:28:25.942 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-06-03T03:28:25.942 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.942 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.942 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "86400", 2024-06-03T03:28:25.942 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.942 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.942 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.942 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "how frequently to scrape device health metrics", 2024-06-03T03:28:25.943 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.943 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.943 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.943 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.943 INFO:teuthology.orchestra.run.smithi117.stdout: "self_heal": { 2024-06-03T03:28:25.943 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "self_heal", 2024-06-03T03:28:25.943 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.943 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.943 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.943 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:25.943 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.943 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.943 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.944 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-06-03T03:28:25.944 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.944 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.944 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.944 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.944 INFO:teuthology.orchestra.run.smithi117.stdout: "sleep_interval": { 2024-06-03T03:28:25.944 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sleep_interval", 2024-06-03T03:28:25.944 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-06-03T03:28:25.944 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.944 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.944 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "600", 2024-06-03T03:28:25.944 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.944 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.944 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.945 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "how frequently to wake up and check device health", 2024-06-03T03:28:25.945 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.945 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.945 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.945 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.945 INFO:teuthology.orchestra.run.smithi117.stdout: "warn_threshold": { 2024-06-03T03:28:25.945 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "warn_threshold", 2024-06-03T03:28:25.945 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-06-03T03:28:25.945 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.945 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.945 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "7257600", 2024-06-03T03:28:25.945 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.945 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.946 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.946 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-06-03T03:28:25.946 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.946 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.946 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.946 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.946 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.946 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.946 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:25.946 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "diskprediction_local", 2024-06-03T03:28:25.946 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:25.946 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:25.946 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:25.947 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:25.947 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:25.947 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.947 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.947 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.947 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.947 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.947 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.947 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.947 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.947 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.947 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.947 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.948 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.948 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.948 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.948 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.948 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.948 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.948 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.948 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.948 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:25.948 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:25.948 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.948 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.948 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.949 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.949 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.949 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.949 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.949 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.949 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.949 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.949 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.949 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.949 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:25.949 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:25.949 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.949 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.949 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.950 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:25.950 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.950 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.950 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.950 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.950 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.950 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.950 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.950 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.950 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.950 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.950 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.951 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.951 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.951 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.951 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.951 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:25.951 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:25.951 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.951 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.951 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.951 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.951 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.951 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.951 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.952 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.952 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.952 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.952 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.952 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.952 INFO:teuthology.orchestra.run.smithi117.stdout: "predict_interval": { 2024-06-03T03:28:25.952 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "predict_interval", 2024-06-03T03:28:25.952 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.952 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.952 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.952 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "86400", 2024-06-03T03:28:25.952 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.952 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.952 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.953 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.953 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.953 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.953 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.953 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.953 INFO:teuthology.orchestra.run.smithi117.stdout: "predictor_model": { 2024-06-03T03:28:25.953 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "predictor_model", 2024-06-03T03:28:25.953 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.953 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.953 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.953 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "prophetstor", 2024-06-03T03:28:25.953 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.953 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.954 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.954 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.954 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.954 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.954 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.954 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.954 INFO:teuthology.orchestra.run.smithi117.stdout: "sleep_interval": { 2024-06-03T03:28:25.954 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sleep_interval", 2024-06-03T03:28:25.954 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.954 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.954 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.954 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "600", 2024-06-03T03:28:25.954 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.955 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.955 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.955 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.955 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.955 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.955 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.955 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.955 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.955 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.955 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:25.955 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "influx", 2024-06-03T03:28:25.955 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": false, 2024-06-03T03:28:25.955 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "influxdb python module not found", 2024-06-03T03:28:25.956 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:25.956 INFO:teuthology.orchestra.run.smithi117.stdout: "batch_size": { 2024-06-03T03:28:25.956 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "batch_size", 2024-06-03T03:28:25.956 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:25.956 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.956 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.956 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "5000", 2024-06-03T03:28:25.956 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.956 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.956 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.956 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-06-03T03:28:25.956 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.956 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.956 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.957 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.957 INFO:teuthology.orchestra.run.smithi117.stdout: "database": { 2024-06-03T03:28:25.957 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "database", 2024-06-03T03:28:25.957 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.957 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.957 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.957 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "ceph", 2024-06-03T03:28:25.957 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.957 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.957 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.957 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-06-03T03:28:25.957 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.957 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.958 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.958 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.958 INFO:teuthology.orchestra.run.smithi117.stdout: "hostname": { 2024-06-03T03:28:25.958 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "hostname", 2024-06-03T03:28:25.958 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.958 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.958 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.958 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.958 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.958 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.958 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.958 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "InfluxDB server hostname", 2024-06-03T03:28:25.958 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.959 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.959 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.959 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.959 INFO:teuthology.orchestra.run.smithi117.stdout: "interval": { 2024-06-03T03:28:25.959 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "interval", 2024-06-03T03:28:25.959 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-06-03T03:28:25.959 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.959 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.959 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "30", 2024-06-03T03:28:25.959 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "5", 2024-06-03T03:28:25.959 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.959 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.959 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-06-03T03:28:25.959 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.960 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.960 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.960 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.960 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:25.960 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:25.960 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.960 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.960 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.960 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.960 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.960 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.960 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.960 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.961 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.961 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.961 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.961 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.961 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.961 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.961 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.961 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.961 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.961 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.961 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.961 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:25.961 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:25.962 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.962 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.962 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.962 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.962 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.962 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.962 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.962 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.962 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.962 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.962 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.962 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.962 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:25.963 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:25.963 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.963 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.963 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.963 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:25.963 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.963 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.963 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.963 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.963 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.963 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.963 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.963 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.963 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.964 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.964 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.964 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.964 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.964 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.964 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.964 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:25.964 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:25.964 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.964 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.964 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.964 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.964 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.964 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.965 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.965 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.965 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.965 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.965 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.965 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.965 INFO:teuthology.orchestra.run.smithi117.stdout: "password": { 2024-06-03T03:28:25.965 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "password", 2024-06-03T03:28:25.965 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.965 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.965 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.965 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.965 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.966 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.966 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.966 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "password of InfluxDB server user", 2024-06-03T03:28:25.966 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.966 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.966 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.966 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.966 INFO:teuthology.orchestra.run.smithi117.stdout: "port": { 2024-06-03T03:28:25.966 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "port", 2024-06-03T03:28:25.966 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:25.966 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.966 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.966 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "8086", 2024-06-03T03:28:25.966 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.967 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.967 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.967 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "InfluxDB server port", 2024-06-03T03:28:25.967 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.967 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.967 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.967 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.967 INFO:teuthology.orchestra.run.smithi117.stdout: "ssl": { 2024-06-03T03:28:25.967 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "ssl", 2024-06-03T03:28:25.967 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.967 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.967 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.967 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "false", 2024-06-03T03:28:25.968 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.968 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.968 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.968 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-06-03T03:28:25.968 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.968 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.968 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.968 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.968 INFO:teuthology.orchestra.run.smithi117.stdout: "threads": { 2024-06-03T03:28:25.968 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "threads", 2024-06-03T03:28:25.968 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:25.968 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.968 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.968 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "5", 2024-06-03T03:28:25.969 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "1", 2024-06-03T03:28:25.969 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "32", 2024-06-03T03:28:25.969 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.969 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-06-03T03:28:25.969 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.969 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.969 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.969 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.969 INFO:teuthology.orchestra.run.smithi117.stdout: "username": { 2024-06-03T03:28:25.969 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "username", 2024-06-03T03:28:25.969 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.969 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.969 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.970 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.970 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.970 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.970 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.970 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "username of InfluxDB server user", 2024-06-03T03:28:25.970 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.970 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.970 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.970 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.970 INFO:teuthology.orchestra.run.smithi117.stdout: "verify_ssl": { 2024-06-03T03:28:25.970 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "verify_ssl", 2024-06-03T03:28:25.970 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.970 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.971 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.971 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "true", 2024-06-03T03:28:25.971 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.971 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.971 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.971 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-06-03T03:28:25.971 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.971 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.971 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.971 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.971 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.971 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.971 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:25.972 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "insights", 2024-06-03T03:28:25.972 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:25.972 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:25.972 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:25.972 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:25.972 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:25.972 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.972 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.972 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.972 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.972 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.972 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.972 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.973 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.973 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.973 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.973 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.973 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.973 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.973 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.973 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.973 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.973 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.973 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.973 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.973 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:25.974 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:25.974 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.974 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.974 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.974 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.974 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.974 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.974 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.974 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.974 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.974 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.974 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.974 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.975 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:25.975 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:25.975 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.975 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.975 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.975 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:25.975 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.975 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.975 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.975 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.975 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.975 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.975 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.976 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.976 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.976 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.976 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.976 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.976 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.976 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.976 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.976 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:25.976 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:25.976 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.976 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.976 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.977 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.977 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.977 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.977 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.977 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.977 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.977 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.977 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.977 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.977 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.977 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.977 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:25.977 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "iostat", 2024-06-03T03:28:25.977 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:25.978 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:25.978 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:25.978 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:25.978 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:25.978 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.978 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.978 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.978 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.978 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.978 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.978 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.978 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.978 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.979 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.979 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.979 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.979 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.979 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.979 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.979 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.979 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.979 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.979 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.979 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:25.979 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:25.979 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.980 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.980 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.980 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.980 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.980 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.980 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.980 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.980 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.980 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.980 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.980 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.980 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:25.980 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:25.980 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.981 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.981 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.981 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:25.981 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.981 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.981 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.981 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.981 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.981 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.981 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.981 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.981 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.981 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.982 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.982 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.982 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.982 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.982 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.982 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:25.982 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:25.982 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.982 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.982 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.982 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.982 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.982 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.983 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.983 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.983 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.983 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.983 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.983 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.983 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.983 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.983 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:25.983 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "k8sevents", 2024-06-03T03:28:25.983 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:25.983 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:25.983 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:25.983 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_event_retention_days": { 2024-06-03T03:28:25.984 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "ceph_event_retention_days", 2024-06-03T03:28:25.984 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:25.984 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.984 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.984 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "7", 2024-06-03T03:28:25.984 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.984 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.984 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.984 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Days to hold ceph event information within local cache", 2024-06-03T03:28:25.984 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.984 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.984 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.984 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.985 INFO:teuthology.orchestra.run.smithi117.stdout: "config_check_secs": { 2024-06-03T03:28:25.985 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "config_check_secs", 2024-06-03T03:28:25.985 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:25.985 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.985 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:25.985 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "10", 2024-06-03T03:28:25.985 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "10", 2024-06-03T03:28:25.985 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.985 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.985 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-06-03T03:28:25.985 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.985 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.985 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.985 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.986 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:25.986 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:25.986 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.986 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.986 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.986 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.986 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.986 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.986 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.986 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.986 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.986 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.986 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.987 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.987 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.987 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.987 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.987 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.987 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.987 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.987 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.987 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:25.987 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:25.987 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.987 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.987 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.988 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.988 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.988 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.988 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.988 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.988 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.988 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.988 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.988 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.988 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:25.988 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:25.988 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.988 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.988 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.989 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:25.989 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.989 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.989 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.989 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.989 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.989 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.989 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.989 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.989 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.989 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.989 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.989 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.990 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.990 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.990 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.990 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:25.990 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:25.990 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.990 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.990 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.990 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.990 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.990 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.990 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.990 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.991 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.991 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.991 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.991 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.991 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:25.991 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.991 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:25.991 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "localpool", 2024-06-03T03:28:25.991 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:25.991 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:25.991 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:25.991 INFO:teuthology.orchestra.run.smithi117.stdout: "failure_domain": { 2024-06-03T03:28:25.991 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "failure_domain", 2024-06-03T03:28:25.992 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.992 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.992 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.992 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "host", 2024-06-03T03:28:25.992 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.992 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.992 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.992 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "failure domain for any created local pool", 2024-06-03T03:28:25.992 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-06-03T03:28:25.992 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.992 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.992 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.992 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:25.993 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:25.993 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.993 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.993 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.993 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.993 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.993 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.993 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.993 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.993 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.993 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.993 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.993 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.993 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.994 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.994 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.994 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.994 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.994 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.994 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.994 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:25.994 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:25.994 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.994 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.994 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.994 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.995 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.995 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.995 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.995 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.995 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.995 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.995 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.995 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.995 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:25.995 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:25.995 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:25.995 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.995 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.995 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:25.996 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.996 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.996 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:25.996 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:25.996 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:25.996 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:25.996 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:25.996 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:25.996 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:25.996 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:25.996 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.996 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.996 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.997 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.997 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.997 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:25.997 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:25.997 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:25.997 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.997 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.997 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:25.997 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.997 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.997 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.997 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:25.997 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:25.997 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.998 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.998 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.998 INFO:teuthology.orchestra.run.smithi117.stdout: "min_size": { 2024-06-03T03:28:25.998 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "min_size", 2024-06-03T03:28:25.998 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:25.998 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.998 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.998 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:25.998 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.998 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.998 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.998 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "default min_size for any created local pool", 2024-06-03T03:28:25.998 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-06-03T03:28:25.998 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:25.999 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:25.999 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:25.999 INFO:teuthology.orchestra.run.smithi117.stdout: "num_rep": { 2024-06-03T03:28:25.999 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "num_rep", 2024-06-03T03:28:25.999 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:25.999 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:25.999 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:25.999 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "3", 2024-06-03T03:28:25.999 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:25.999 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:25.999 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:25.999 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "default replica count for any created local pool", 2024-06-03T03:28:25.999 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.000 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.000 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.000 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.000 INFO:teuthology.orchestra.run.smithi117.stdout: "pg_num": { 2024-06-03T03:28:26.000 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "pg_num", 2024-06-03T03:28:26.000 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:26.000 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.000 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.000 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "128", 2024-06-03T03:28:26.000 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.000 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.001 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.001 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "default pg_num for any created local pool", 2024-06-03T03:28:26.001 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.001 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.001 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.001 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.001 INFO:teuthology.orchestra.run.smithi117.stdout: "prefix": { 2024-06-03T03:28:26.001 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "prefix", 2024-06-03T03:28:26.001 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.001 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.001 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.001 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.001 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.001 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.002 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.002 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "name prefix for any created local pool", 2024-06-03T03:28:26.002 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.002 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.002 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.002 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.002 INFO:teuthology.orchestra.run.smithi117.stdout: "subtree": { 2024-06-03T03:28:26.002 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "subtree", 2024-06-03T03:28:26.002 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.002 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.002 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.002 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "rack", 2024-06-03T03:28:26.003 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.003 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.003 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.003 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "CRUSH level for which to create a local pool", 2024-06-03T03:28:26.003 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-06-03T03:28:26.003 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.003 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.003 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.003 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.003 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.003 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:26.003 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "mds_autoscaler", 2024-06-03T03:28:26.003 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:26.004 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:26.004 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:26.004 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:26.004 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:26.004 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.004 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.004 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.004 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.004 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.004 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.004 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.004 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.004 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.004 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.005 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.005 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.005 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.005 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.005 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.005 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.005 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.005 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.005 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.005 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:26.005 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:26.005 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.005 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.006 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.006 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.006 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.006 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.006 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.006 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.006 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.006 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.006 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.006 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.006 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:26.006 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:26.006 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.007 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.007 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.007 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:26.007 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.007 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.007 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.007 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.007 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.007 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.007 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.007 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.007 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.008 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.008 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.008 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.008 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.008 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.008 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.008 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:26.008 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:26.008 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.008 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.008 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.008 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.008 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.008 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.009 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.009 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.009 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.009 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.009 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.009 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.009 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.009 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.009 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:26.009 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "mirroring", 2024-06-03T03:28:26.009 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:26.009 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:26.009 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:26.010 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:26.010 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:26.010 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.010 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.010 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.010 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.010 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.010 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.010 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.010 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.010 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.010 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.010 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.010 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.011 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.011 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.011 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.011 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.011 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.011 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.011 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.011 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:26.011 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:26.011 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.011 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.011 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.011 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.011 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.012 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.012 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.012 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.012 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.012 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.012 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.012 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.012 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:26.012 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:26.012 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.012 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.012 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.012 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:26.013 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.013 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.013 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.013 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.013 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.013 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.013 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.013 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.013 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.013 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.013 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.013 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.013 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.013 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.014 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.014 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:26.014 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:26.014 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.014 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.014 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.014 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.014 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.014 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.014 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.014 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.014 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.014 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.017 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.018 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.018 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.018 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.018 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:26.018 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "nfs", 2024-06-03T03:28:26.018 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:26.018 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:26.018 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:26.018 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:26.018 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:26.018 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.018 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.019 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.019 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.019 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.019 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.019 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.019 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.019 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.019 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.019 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.019 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.019 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.019 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.019 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.020 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.020 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.020 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.020 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.020 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:26.020 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:26.020 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.020 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.020 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.020 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.020 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.020 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.020 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.021 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.021 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.021 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.021 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.021 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.021 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:26.021 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:26.021 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.021 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.021 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.021 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:26.021 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.021 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.021 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.022 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.022 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.022 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.022 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.022 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.022 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.022 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.022 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.022 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.022 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.022 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.022 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.022 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:26.023 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:26.023 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.023 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.023 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.023 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.023 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.023 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.023 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.023 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.023 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.023 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.023 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.023 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.024 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.024 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.024 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:26.024 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "orchestrator", 2024-06-03T03:28:26.024 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:26.024 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:26.024 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:26.024 INFO:teuthology.orchestra.run.smithi117.stdout: "fail_fs": { 2024-06-03T03:28:26.024 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "fail_fs", 2024-06-03T03:28:26.024 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.024 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.024 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.024 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.024 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.025 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.025 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.025 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-06-03T03:28:26.025 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.025 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.025 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.025 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.025 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:26.025 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:26.025 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.025 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.025 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.025 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.026 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.026 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.026 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.026 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.026 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.026 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.026 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.026 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.026 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.026 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.026 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.026 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.026 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.027 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.027 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.027 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:26.027 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:26.027 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.027 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.027 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.027 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.027 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.027 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.027 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.027 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.027 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.027 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.028 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.028 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.028 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:26.028 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:26.028 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.028 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.028 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.028 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:26.028 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.028 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.028 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.028 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.028 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.028 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.029 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.029 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.029 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.029 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.029 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.029 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.029 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.029 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.029 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.029 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:26.029 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:26.029 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.029 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.030 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.030 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.030 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.030 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.030 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.030 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.030 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.030 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.030 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.030 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.030 INFO:teuthology.orchestra.run.smithi117.stdout: "orchestrator": { 2024-06-03T03:28:26.030 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "orchestrator", 2024-06-03T03:28:26.030 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.030 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.031 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.031 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.031 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.031 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.031 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.031 INFO:teuthology.orchestra.run.smithi117.stdout: "cephadm", 2024-06-03T03:28:26.031 INFO:teuthology.orchestra.run.smithi117.stdout: "rook", 2024-06-03T03:28:26.031 INFO:teuthology.orchestra.run.smithi117.stdout: "test_orchestrator" 2024-06-03T03:28:26.031 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.031 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Orchestrator backend", 2024-06-03T03:28:26.031 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.031 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.031 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.032 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.032 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.032 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.032 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:26.032 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "osd_perf_query", 2024-06-03T03:28:26.032 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:26.032 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:26.032 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:26.032 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:26.032 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:26.032 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.032 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.032 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.032 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.033 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.033 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.033 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.033 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.033 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.033 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.033 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.033 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.033 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.033 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.033 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.033 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.034 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.034 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.034 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.034 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:26.034 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:26.034 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.034 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.034 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.034 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.034 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.034 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.034 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.034 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.035 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.035 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.035 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.035 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.035 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:26.035 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:26.035 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.035 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.035 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.035 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:26.035 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.035 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.035 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.035 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.036 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.036 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.036 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.036 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.036 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.036 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.036 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.036 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.036 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.036 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.036 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.036 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:26.036 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:26.036 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.037 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.037 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.037 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.037 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.037 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.037 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.037 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.037 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.037 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.037 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.037 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.037 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.037 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.038 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:26.038 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "osd_support", 2024-06-03T03:28:26.038 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:26.038 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:26.038 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:26.038 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:26.038 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:26.038 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.038 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.038 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.038 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.038 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.038 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.039 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.039 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.039 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.039 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.039 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.039 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.039 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.039 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.039 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.039 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.039 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.039 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.039 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.039 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:26.040 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:26.040 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.040 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.040 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.040 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.040 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.040 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.040 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.040 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.040 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.040 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.040 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.040 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.040 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:26.041 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:26.041 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.041 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.041 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.041 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:26.041 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.041 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.041 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.041 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.041 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.041 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.041 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.041 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.042 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.042 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.042 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.042 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.042 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.042 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.042 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.042 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:26.042 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:26.042 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.042 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.042 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.042 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.043 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.043 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.043 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.043 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.043 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.043 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.043 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.043 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.043 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.043 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.043 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:26.043 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "pg_autoscaler", 2024-06-03T03:28:26.043 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:26.043 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:26.044 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:26.044 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:26.044 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:26.044 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.044 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.044 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.044 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.044 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.044 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.044 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.044 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.044 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.044 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.045 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.045 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.045 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.045 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.045 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.045 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.045 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.045 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.045 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.045 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:26.045 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:26.045 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.045 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.045 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.046 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.046 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.046 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.046 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.046 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.046 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.046 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.046 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.046 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.046 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:26.046 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:26.046 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.046 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.047 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.047 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:26.047 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.047 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.047 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.047 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.047 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.047 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.047 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.047 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.047 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.047 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.047 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.048 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.048 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.048 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.048 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.048 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:26.048 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:26.048 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.048 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.048 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.048 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.048 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.048 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.048 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.049 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.049 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.049 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.049 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.049 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.049 INFO:teuthology.orchestra.run.smithi117.stdout: "sleep_interval": { 2024-06-03T03:28:26.049 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sleep_interval", 2024-06-03T03:28:26.049 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-06-03T03:28:26.049 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.049 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.049 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "60", 2024-06-03T03:28:26.049 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.049 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.050 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.050 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.050 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.050 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.050 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.050 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.050 INFO:teuthology.orchestra.run.smithi117.stdout: "threshold": { 2024-06-03T03:28:26.050 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "threshold", 2024-06-03T03:28:26.050 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "float", 2024-06-03T03:28:26.050 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.050 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.050 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "3.0", 2024-06-03T03:28:26.050 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "1.0", 2024-06-03T03:28:26.051 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.051 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.051 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "scaling threshold", 2024-06-03T03:28:26.051 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-06-03T03:28:26.051 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.051 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.051 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.051 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.051 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.051 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:26.051 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "progress", 2024-06-03T03:28:26.051 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:26.051 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:26.052 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:26.052 INFO:teuthology.orchestra.run.smithi117.stdout: "allow_pg_recovery_event": { 2024-06-03T03:28:26.052 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "allow_pg_recovery_event", 2024-06-03T03:28:26.052 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.052 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.052 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.052 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.052 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.052 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.052 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.052 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "allow the module to show pg recovery progress", 2024-06-03T03:28:26.052 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.052 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.053 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.053 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.053 INFO:teuthology.orchestra.run.smithi117.stdout: "enabled": { 2024-06-03T03:28:26.053 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "enabled", 2024-06-03T03:28:26.053 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.053 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.053 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.053 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:26.053 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.053 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.053 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.053 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.053 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.053 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.054 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.054 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.054 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:26.054 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:26.054 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.054 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.054 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.054 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.054 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.054 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.054 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.054 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.054 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.055 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.055 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.055 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.055 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.055 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.055 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.055 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.055 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.055 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.055 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.055 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:26.055 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:26.055 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.055 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.056 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.056 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.056 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.056 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.056 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.056 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.056 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.056 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.056 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.056 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.056 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:26.056 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:26.056 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.057 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.057 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.057 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:26.057 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.057 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.057 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.057 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.057 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.057 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.057 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.057 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.057 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.057 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.057 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.058 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.058 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.058 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.058 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.058 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:26.058 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:26.058 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.058 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.058 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.058 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.058 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.058 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.058 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.059 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.059 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.059 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.059 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.059 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.059 INFO:teuthology.orchestra.run.smithi117.stdout: "max_completed_events": { 2024-06-03T03:28:26.059 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "max_completed_events", 2024-06-03T03:28:26.059 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:26.059 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.059 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.059 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "50", 2024-06-03T03:28:26.059 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.059 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.060 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.060 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "number of past completed events to remember", 2024-06-03T03:28:26.060 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.060 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.060 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.060 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.060 INFO:teuthology.orchestra.run.smithi117.stdout: "sleep_interval": { 2024-06-03T03:28:26.060 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sleep_interval", 2024-06-03T03:28:26.060 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-06-03T03:28:26.060 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.060 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.060 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "5", 2024-06-03T03:28:26.060 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.060 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.060 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.061 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "how long the module is going to sleep", 2024-06-03T03:28:26.061 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.061 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.061 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.061 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.061 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.061 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.061 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:26.061 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "prometheus", 2024-06-03T03:28:26.061 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:26.061 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:26.061 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:26.061 INFO:teuthology.orchestra.run.smithi117.stdout: "cache": { 2024-06-03T03:28:26.061 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "cache", 2024-06-03T03:28:26.062 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.062 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.062 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.062 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:26.062 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.062 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.062 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.062 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.062 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.062 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.062 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.062 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.063 INFO:teuthology.orchestra.run.smithi117.stdout: "exclude_perf_counters": { 2024-06-03T03:28:26.063 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "exclude_perf_counters", 2024-06-03T03:28:26.063 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.063 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.063 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.063 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:26.063 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.063 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.063 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.063 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-06-03T03:28:26.063 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-06-03T03:28:26.063 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.063 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.064 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.064 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:26.064 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:26.064 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.064 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.064 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.064 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.064 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.064 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.064 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.064 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.064 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.064 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.064 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.065 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.065 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.065 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.065 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.065 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.065 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.065 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.065 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.065 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:26.065 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:26.065 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.065 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.065 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.066 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.066 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.066 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.066 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.066 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.066 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.066 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.066 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.066 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.066 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:26.066 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:26.066 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.066 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.067 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.067 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:26.067 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.067 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.067 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.067 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.067 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.067 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.067 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.067 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.067 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.067 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.067 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.068 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.068 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.068 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.068 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.068 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:26.068 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:26.068 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.068 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.068 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.068 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.068 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.068 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.068 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.068 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.069 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.069 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.069 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.069 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.069 INFO:teuthology.orchestra.run.smithi117.stdout: "rbd_stats_pools": { 2024-06-03T03:28:26.069 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "rbd_stats_pools", 2024-06-03T03:28:26.069 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.069 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.069 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.069 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.069 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.069 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.069 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.070 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.070 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.070 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.070 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.070 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.070 INFO:teuthology.orchestra.run.smithi117.stdout: "rbd_stats_pools_refresh_interval": { 2024-06-03T03:28:26.070 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-06-03T03:28:26.070 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:26.070 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.070 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.070 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "300", 2024-06-03T03:28:26.070 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.070 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.071 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.071 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.071 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.071 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.071 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.071 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.071 INFO:teuthology.orchestra.run.smithi117.stdout: "scrape_interval": { 2024-06-03T03:28:26.071 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "scrape_interval", 2024-06-03T03:28:26.071 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "float", 2024-06-03T03:28:26.071 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.071 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.071 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "15.0", 2024-06-03T03:28:26.071 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.072 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.072 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.072 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.072 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.072 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.072 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.072 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.072 INFO:teuthology.orchestra.run.smithi117.stdout: "server_addr": { 2024-06-03T03:28:26.072 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "server_addr", 2024-06-03T03:28:26.072 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.072 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.072 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.072 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "::", 2024-06-03T03:28:26.072 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.073 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.073 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.073 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-06-03T03:28:26.073 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.073 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.073 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.073 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.073 INFO:teuthology.orchestra.run.smithi117.stdout: "server_port": { 2024-06-03T03:28:26.073 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "server_port", 2024-06-03T03:28:26.073 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:26.073 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.073 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.073 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "9283", 2024-06-03T03:28:26.074 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.074 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.074 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.074 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-06-03T03:28:26.074 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.074 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.074 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.074 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.074 INFO:teuthology.orchestra.run.smithi117.stdout: "stale_cache_strategy": { 2024-06-03T03:28:26.074 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "stale_cache_strategy", 2024-06-03T03:28:26.074 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.074 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.074 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.074 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "log", 2024-06-03T03:28:26.075 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.075 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.075 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.075 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.075 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.075 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.075 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.075 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.075 INFO:teuthology.orchestra.run.smithi117.stdout: "standby_behaviour": { 2024-06-03T03:28:26.075 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "standby_behaviour", 2024-06-03T03:28:26.075 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.075 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.076 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.076 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "default", 2024-06-03T03:28:26.076 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.076 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.076 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.076 INFO:teuthology.orchestra.run.smithi117.stdout: "default", 2024-06-03T03:28:26.076 INFO:teuthology.orchestra.run.smithi117.stdout: "error" 2024-06-03T03:28:26.076 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.076 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.076 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.076 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.076 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.076 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.076 INFO:teuthology.orchestra.run.smithi117.stdout: "standby_error_status_code": { 2024-06-03T03:28:26.077 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "standby_error_status_code", 2024-06-03T03:28:26.077 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:26.077 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.077 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.077 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "500", 2024-06-03T03:28:26.077 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "400", 2024-06-03T03:28:26.077 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "599", 2024-06-03T03:28:26.077 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.077 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.077 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.077 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.077 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.077 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.078 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.078 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.078 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:26.078 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "rbd_support", 2024-06-03T03:28:26.078 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:26.078 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:26.078 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:26.078 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:26.078 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:26.078 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.078 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.078 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.078 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.079 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.079 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.079 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.079 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.079 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.079 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.079 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.079 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.079 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.079 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.079 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.079 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.079 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.080 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.080 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.080 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:26.080 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:26.080 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.080 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.080 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.080 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.080 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.080 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.080 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.080 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.081 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.081 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.081 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.081 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.081 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:26.081 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:26.081 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.081 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.081 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.081 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:26.081 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.081 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.081 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.082 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.082 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.082 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.082 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.082 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.082 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.082 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.082 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.082 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.082 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.082 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.082 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.082 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:26.083 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:26.083 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.083 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.083 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.083 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.083 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.083 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.083 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.083 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.083 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.083 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.083 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.083 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.083 INFO:teuthology.orchestra.run.smithi117.stdout: "max_concurrent_snap_create": { 2024-06-03T03:28:26.084 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "max_concurrent_snap_create", 2024-06-03T03:28:26.084 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:26.084 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.084 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.084 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "10", 2024-06-03T03:28:26.084 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.084 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.084 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.084 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.084 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.084 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.084 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.084 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.084 INFO:teuthology.orchestra.run.smithi117.stdout: "mirror_snapshot_schedule": { 2024-06-03T03:28:26.085 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "mirror_snapshot_schedule", 2024-06-03T03:28:26.085 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.085 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.085 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.085 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.085 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.085 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.085 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.085 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.085 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.085 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.085 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.085 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.086 INFO:teuthology.orchestra.run.smithi117.stdout: "trash_purge_schedule": { 2024-06-03T03:28:26.086 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "trash_purge_schedule", 2024-06-03T03:28:26.086 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.086 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.086 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.086 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.086 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.086 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.086 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.086 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.086 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.086 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.086 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.087 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.087 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.087 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.087 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:26.087 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "restful", 2024-06-03T03:28:26.087 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:26.087 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:26.087 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:26.087 INFO:teuthology.orchestra.run.smithi117.stdout: "enable_auth": { 2024-06-03T03:28:26.087 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "enable_auth", 2024-06-03T03:28:26.087 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.087 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.087 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.087 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:26.088 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.088 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.088 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.088 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.088 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.088 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.088 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.088 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.088 INFO:teuthology.orchestra.run.smithi117.stdout: "key_file": { 2024-06-03T03:28:26.088 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "key_file", 2024-06-03T03:28:26.088 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.088 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.088 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.089 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.089 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.089 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.089 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.089 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.089 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.089 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.089 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.089 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.089 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:26.089 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:26.089 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.089 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.089 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.089 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.090 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.090 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.090 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.090 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.090 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.090 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.090 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.090 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.090 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.090 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.090 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.090 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.090 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.091 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.091 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.091 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:26.091 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:26.091 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.091 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.091 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.091 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.091 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.091 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.091 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.091 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.091 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.092 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.092 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.092 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.092 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:26.092 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:26.092 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.092 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.092 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.092 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:26.092 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.092 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.092 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.092 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.093 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.093 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.093 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.093 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.093 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.093 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.093 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.093 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.093 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.093 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.093 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.093 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:26.093 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:26.093 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.094 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.094 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.094 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.094 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.094 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.094 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.094 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.094 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.094 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.094 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.094 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.094 INFO:teuthology.orchestra.run.smithi117.stdout: "server_addr": { 2024-06-03T03:28:26.094 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "server_addr", 2024-06-03T03:28:26.095 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.095 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.095 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.095 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.095 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.095 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.095 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.095 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.095 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.095 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.095 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.095 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.095 INFO:teuthology.orchestra.run.smithi117.stdout: "server_port": { 2024-06-03T03:28:26.096 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "server_port", 2024-06-03T03:28:26.096 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.096 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.096 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.096 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.096 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.096 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.096 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.096 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.096 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.096 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.096 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.096 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.096 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.097 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.097 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:26.097 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "rgw", 2024-06-03T03:28:26.097 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:26.097 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:26.097 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:26.097 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:26.097 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:26.097 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.097 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.097 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.097 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.097 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.097 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.098 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.098 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.098 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.098 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.098 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.098 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.098 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.098 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.098 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.098 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.098 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.098 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.098 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.099 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:26.099 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:26.099 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.099 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.099 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.099 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.099 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.099 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.099 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.099 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.099 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.099 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.099 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.099 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.100 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:26.100 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:26.100 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.100 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.100 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.100 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:26.100 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.100 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.100 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.100 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.100 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.100 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.100 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.100 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.101 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.101 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.101 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.101 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.101 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.101 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.101 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.101 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:26.101 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:26.101 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.101 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.101 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.101 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.101 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.102 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.102 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.102 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.102 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.102 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.102 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.102 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.102 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.102 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.102 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:26.102 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "rook", 2024-06-03T03:28:26.102 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:26.102 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:26.103 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:26.103 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:26.103 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:26.103 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.103 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.103 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.103 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.103 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.103 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.103 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.103 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.103 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.103 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.103 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.103 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.104 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.104 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.104 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.104 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.104 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.104 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.104 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.104 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:26.104 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:26.104 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.104 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.104 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.105 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.105 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.105 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.105 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.105 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.105 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.105 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.105 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.105 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.105 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:26.105 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:26.105 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.105 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.106 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.106 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:26.106 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.106 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.106 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.106 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.106 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.106 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.106 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.106 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.106 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.106 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.106 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.107 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.107 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.107 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.107 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.107 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:26.107 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:26.107 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.107 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.107 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.107 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.107 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.107 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.108 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.108 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.108 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.108 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.108 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.108 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.108 INFO:teuthology.orchestra.run.smithi117.stdout: "storage_class": { 2024-06-03T03:28:26.108 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "storage_class", 2024-06-03T03:28:26.108 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.108 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.108 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.108 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "local", 2024-06-03T03:28:26.108 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.108 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.109 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.109 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-06-03T03:28:26.109 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.109 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.109 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.109 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.109 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.109 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.109 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:26.109 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "selftest", 2024-06-03T03:28:26.109 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:26.109 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:26.109 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:26.110 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:26.110 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:26.110 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.110 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.110 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.110 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.110 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.110 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.110 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.110 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.110 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.110 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.111 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.111 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.111 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.111 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.111 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.111 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.111 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.111 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.111 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.111 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:26.111 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:26.111 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.111 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.112 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.112 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.112 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.112 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.112 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.112 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.112 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.112 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.112 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.112 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.112 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:26.112 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:26.112 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.112 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.112 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.113 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:26.113 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.113 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.113 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.113 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.113 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.113 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.113 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.113 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.113 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.113 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.113 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.113 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.113 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.114 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.114 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.114 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:26.114 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:26.114 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.114 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.114 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.114 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.114 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.114 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.114 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.114 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.114 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.115 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.115 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.115 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.115 INFO:teuthology.orchestra.run.smithi117.stdout: "roption1": { 2024-06-03T03:28:26.115 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "roption1", 2024-06-03T03:28:26.115 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.115 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.115 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.115 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.115 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.115 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.115 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.115 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.115 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.116 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.116 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.116 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.116 INFO:teuthology.orchestra.run.smithi117.stdout: "roption2": { 2024-06-03T03:28:26.116 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "roption2", 2024-06-03T03:28:26.116 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.116 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.116 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.116 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "xyz", 2024-06-03T03:28:26.116 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.116 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.116 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.116 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.116 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.117 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.117 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.117 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.117 INFO:teuthology.orchestra.run.smithi117.stdout: "rwoption1": { 2024-06-03T03:28:26.117 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "rwoption1", 2024-06-03T03:28:26.117 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.117 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.117 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.117 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.117 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.117 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.117 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.117 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.117 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.118 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.118 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.118 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.118 INFO:teuthology.orchestra.run.smithi117.stdout: "rwoption2": { 2024-06-03T03:28:26.118 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "rwoption2", 2024-06-03T03:28:26.118 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:26.118 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.118 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.118 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.118 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.118 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.118 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.118 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.119 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.119 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.119 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.119 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.119 INFO:teuthology.orchestra.run.smithi117.stdout: "rwoption3": { 2024-06-03T03:28:26.119 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "rwoption3", 2024-06-03T03:28:26.119 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "float", 2024-06-03T03:28:26.119 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.119 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.119 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.119 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.119 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.119 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.120 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.120 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.120 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.120 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.120 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.120 INFO:teuthology.orchestra.run.smithi117.stdout: "rwoption4": { 2024-06-03T03:28:26.120 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "rwoption4", 2024-06-03T03:28:26.120 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.120 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.120 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.120 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.120 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.121 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.121 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.121 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.121 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.121 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.121 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.121 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.121 INFO:teuthology.orchestra.run.smithi117.stdout: "rwoption5": { 2024-06-03T03:28:26.121 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "rwoption5", 2024-06-03T03:28:26.121 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.121 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.121 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.121 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.121 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.122 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.122 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.122 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.122 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.122 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.122 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.122 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.122 INFO:teuthology.orchestra.run.smithi117.stdout: "rwoption6": { 2024-06-03T03:28:26.122 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "rwoption6", 2024-06-03T03:28:26.122 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.122 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.122 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.123 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:26.123 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.123 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.123 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.123 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.123 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.123 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.123 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.123 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.123 INFO:teuthology.orchestra.run.smithi117.stdout: "rwoption7": { 2024-06-03T03:28:26.123 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "rwoption7", 2024-06-03T03:28:26.123 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:26.123 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.123 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.124 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.124 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "1", 2024-06-03T03:28:26.124 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "42", 2024-06-03T03:28:26.124 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.124 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.124 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.124 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.124 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.124 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.124 INFO:teuthology.orchestra.run.smithi117.stdout: "testkey": { 2024-06-03T03:28:26.124 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "testkey", 2024-06-03T03:28:26.124 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.125 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.125 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.125 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.125 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.125 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.125 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.125 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.125 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.125 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.125 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.125 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.125 INFO:teuthology.orchestra.run.smithi117.stdout: "testlkey": { 2024-06-03T03:28:26.125 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "testlkey", 2024-06-03T03:28:26.125 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.126 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.126 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.126 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.126 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.126 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.126 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.126 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.126 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.126 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.126 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.126 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.126 INFO:teuthology.orchestra.run.smithi117.stdout: "testnewline": { 2024-06-03T03:28:26.127 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "testnewline", 2024-06-03T03:28:26.127 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.127 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.127 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.127 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.127 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.127 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.127 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.127 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.127 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.127 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.127 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.127 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.128 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.128 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.128 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:26.128 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "snap_schedule", 2024-06-03T03:28:26.128 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:26.128 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:26.128 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:26.128 INFO:teuthology.orchestra.run.smithi117.stdout: "allow_m_granularity": { 2024-06-03T03:28:26.128 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "allow_m_granularity", 2024-06-03T03:28:26.128 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.128 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.128 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.128 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.128 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.129 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.129 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.129 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "allow minute scheduled snapshots", 2024-06-03T03:28:26.129 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.129 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.129 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.129 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.129 INFO:teuthology.orchestra.run.smithi117.stdout: "dump_on_update": { 2024-06-03T03:28:26.129 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "dump_on_update", 2024-06-03T03:28:26.129 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.129 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.129 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.129 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.130 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.130 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.130 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.130 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "dump database to debug log on update", 2024-06-03T03:28:26.130 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.130 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.130 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.130 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.130 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:26.130 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:26.130 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.130 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.130 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.131 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.131 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.131 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.131 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.131 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.131 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.131 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.131 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.131 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.131 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.131 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.131 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.131 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.131 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.132 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.132 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.132 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:26.132 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:26.132 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.132 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.132 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.132 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.132 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.132 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.132 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.132 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.132 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.132 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.133 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.133 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.133 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:26.133 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:26.133 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.133 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.133 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.133 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:26.133 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.133 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.133 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.133 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.133 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.133 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.134 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.134 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.134 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.134 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.134 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.134 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.134 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.134 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.134 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.134 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:26.134 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:26.134 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.135 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.135 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.135 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.135 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.135 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.135 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.135 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.135 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.135 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.135 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.135 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.135 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.135 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.136 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:26.136 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "stats", 2024-06-03T03:28:26.136 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:26.136 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:26.136 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:26.136 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:26.136 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:26.136 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.136 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.136 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.136 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.136 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.136 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.137 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.137 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.137 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.137 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.137 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.137 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.137 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.137 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.137 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.137 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.137 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.137 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.137 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.137 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:26.138 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:26.138 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.138 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.138 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.138 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.138 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.138 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.138 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.138 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.138 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.138 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.138 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.138 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.139 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:26.139 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:26.139 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.139 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.139 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.139 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:26.139 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.139 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.139 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.139 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.139 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.139 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.139 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.140 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.140 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.140 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.140 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.140 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.140 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.140 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.140 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.140 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:26.140 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:26.140 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.140 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.140 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.140 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.141 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.141 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.141 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.141 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.141 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.141 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.141 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.141 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.141 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.141 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.141 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:26.141 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "status", 2024-06-03T03:28:26.141 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:26.142 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:26.142 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:26.142 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:26.142 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:26.142 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.142 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.142 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.142 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.142 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.142 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.142 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.142 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.142 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.142 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.143 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.143 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.143 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.143 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.143 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.143 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.143 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.143 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.143 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.143 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:26.143 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:26.143 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.143 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.144 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.144 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.144 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.144 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.144 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.144 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.144 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.144 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.144 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.144 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.144 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:26.144 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:26.144 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.145 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.145 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.145 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:26.145 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.145 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.145 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.145 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.145 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.145 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.145 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.145 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.145 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.145 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.145 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.146 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.146 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.146 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.146 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.146 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:26.146 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:26.146 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.146 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.146 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.146 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.146 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.146 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.146 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.147 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.147 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.147 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.147 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.147 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.147 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.147 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.147 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:26.147 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "telegraf", 2024-06-03T03:28:26.147 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:26.147 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:26.147 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:26.147 INFO:teuthology.orchestra.run.smithi117.stdout: "address": { 2024-06-03T03:28:26.147 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "address", 2024-06-03T03:28:26.148 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.148 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.148 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.148 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-06-03T03:28:26.148 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.148 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.148 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.148 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.148 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.148 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.148 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.148 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.148 INFO:teuthology.orchestra.run.smithi117.stdout: "interval": { 2024-06-03T03:28:26.149 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "interval", 2024-06-03T03:28:26.149 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-06-03T03:28:26.149 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.149 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.149 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "15", 2024-06-03T03:28:26.149 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.149 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.149 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.149 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.149 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.149 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.149 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.149 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.150 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:26.150 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:26.150 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.150 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.150 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.150 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.150 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.150 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.150 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.150 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.150 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.150 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.150 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.151 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.151 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.151 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.151 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.151 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.151 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.151 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.151 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.151 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:26.151 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:26.151 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.151 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.151 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.152 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.152 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.152 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.152 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.152 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.152 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.152 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.152 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.152 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.152 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:26.152 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:26.152 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.152 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.153 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.153 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:26.153 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.153 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.153 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.153 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.153 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.153 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.153 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.153 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.153 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.153 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.153 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.153 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.154 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.154 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.154 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.154 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:26.154 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:26.154 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.154 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.154 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.154 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.154 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.154 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.154 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.154 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.155 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.155 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.155 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.155 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.155 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.155 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.155 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:26.155 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "telemetry", 2024-06-03T03:28:26.155 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:26.155 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:26.155 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:26.155 INFO:teuthology.orchestra.run.smithi117.stdout: "channel_basic": { 2024-06-03T03:28:26.155 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "channel_basic", 2024-06-03T03:28:26.156 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.156 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.156 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.156 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:26.156 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.156 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.156 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.156 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Share basic cluster information (size, version)", 2024-06-03T03:28:26.156 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.156 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.156 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.156 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.156 INFO:teuthology.orchestra.run.smithi117.stdout: "channel_crash": { 2024-06-03T03:28:26.156 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "channel_crash", 2024-06-03T03:28:26.157 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.157 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.157 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.157 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:26.157 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.157 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.157 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.157 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-06-03T03:28:26.157 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.157 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.157 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.157 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.157 INFO:teuthology.orchestra.run.smithi117.stdout: "channel_device": { 2024-06-03T03:28:26.157 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "channel_device", 2024-06-03T03:28:26.158 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.158 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.158 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.158 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:26.158 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.158 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.158 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.158 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-06-03T03:28:26.158 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.158 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.158 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.158 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.158 INFO:teuthology.orchestra.run.smithi117.stdout: "channel_ident": { 2024-06-03T03:28:26.159 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "channel_ident", 2024-06-03T03:28:26.159 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.159 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.159 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.159 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.159 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.159 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.159 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.159 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-06-03T03:28:26.159 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.159 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.159 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.159 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.159 INFO:teuthology.orchestra.run.smithi117.stdout: "channel_perf": { 2024-06-03T03:28:26.160 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "channel_perf", 2024-06-03T03:28:26.160 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.160 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.160 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.160 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.160 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.160 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.160 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.160 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Share various performance metrics of a cluster", 2024-06-03T03:28:26.160 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.160 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.160 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.160 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.160 INFO:teuthology.orchestra.run.smithi117.stdout: "contact": { 2024-06-03T03:28:26.161 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "contact", 2024-06-03T03:28:26.161 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.161 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.161 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.161 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.161 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.161 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.161 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.161 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.161 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.161 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.161 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.161 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.161 INFO:teuthology.orchestra.run.smithi117.stdout: "description": { 2024-06-03T03:28:26.162 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "description", 2024-06-03T03:28:26.162 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.162 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.162 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.162 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.162 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.162 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.162 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.162 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.162 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.162 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.162 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.162 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.162 INFO:teuthology.orchestra.run.smithi117.stdout: "device_url": { 2024-06-03T03:28:26.163 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "device_url", 2024-06-03T03:28:26.163 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.163 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.163 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.163 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-06-03T03:28:26.163 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.163 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.163 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.163 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.163 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.163 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.163 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.163 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.164 INFO:teuthology.orchestra.run.smithi117.stdout: "enabled": { 2024-06-03T03:28:26.164 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "enabled", 2024-06-03T03:28:26.164 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.164 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.164 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.164 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.164 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.164 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.164 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.164 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.164 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.164 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.164 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.164 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.165 INFO:teuthology.orchestra.run.smithi117.stdout: "interval": { 2024-06-03T03:28:26.165 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "interval", 2024-06-03T03:28:26.165 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:26.165 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.165 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.165 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "24", 2024-06-03T03:28:26.165 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "8", 2024-06-03T03:28:26.165 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.165 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.165 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.165 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.165 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.165 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.165 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.166 INFO:teuthology.orchestra.run.smithi117.stdout: "last_opt_revision": { 2024-06-03T03:28:26.166 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "last_opt_revision", 2024-06-03T03:28:26.166 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:26.166 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.166 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.166 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "1", 2024-06-03T03:28:26.166 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.166 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.166 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.166 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.166 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.166 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.166 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.167 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.167 INFO:teuthology.orchestra.run.smithi117.stdout: "leaderboard": { 2024-06-03T03:28:26.167 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "leaderboard", 2024-06-03T03:28:26.167 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.167 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.167 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.167 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.167 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.167 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.167 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.167 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.167 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.167 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.168 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.168 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.168 INFO:teuthology.orchestra.run.smithi117.stdout: "leaderboard_description": { 2024-06-03T03:28:26.168 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "leaderboard_description", 2024-06-03T03:28:26.168 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.168 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.168 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.168 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.168 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.168 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.168 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.168 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.168 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.168 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.169 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.169 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.169 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:26.169 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:26.169 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.169 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.169 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.169 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.169 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.169 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.169 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.169 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.169 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.170 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.170 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.170 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.170 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.170 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.170 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.170 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.170 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.170 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.170 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.170 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:26.170 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:26.170 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.171 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.171 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.171 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.171 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.171 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.171 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.171 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.171 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.171 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.171 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.171 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.171 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:26.171 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:26.172 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.172 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.172 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.172 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:26.172 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.172 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.172 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.172 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.172 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.172 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.172 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.172 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.172 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.173 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.173 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.173 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.173 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.173 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.173 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.173 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:26.173 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:26.173 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.173 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.173 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.173 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.173 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.173 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.174 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.174 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.174 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.174 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.174 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.174 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.174 INFO:teuthology.orchestra.run.smithi117.stdout: "organization": { 2024-06-03T03:28:26.174 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "organization", 2024-06-03T03:28:26.174 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.174 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.174 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.174 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.174 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.175 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.175 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.175 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.175 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.175 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.175 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.175 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.175 INFO:teuthology.orchestra.run.smithi117.stdout: "proxy": { 2024-06-03T03:28:26.175 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "proxy", 2024-06-03T03:28:26.175 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.175 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.175 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.175 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.176 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.176 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.176 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.176 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.176 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.176 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.176 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.176 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.176 INFO:teuthology.orchestra.run.smithi117.stdout: "url": { 2024-06-03T03:28:26.176 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "url", 2024-06-03T03:28:26.176 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.176 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.176 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.176 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-06-03T03:28:26.177 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.177 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.177 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.177 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.177 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.177 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.177 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.177 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.177 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.177 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.177 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:26.177 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "test_orchestrator", 2024-06-03T03:28:26.177 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:26.178 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:26.178 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:26.178 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:26.178 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:26.178 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.178 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.178 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.178 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.178 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.178 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.178 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.178 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.178 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.179 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.179 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.179 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.179 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.179 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.179 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.179 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.179 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.179 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.179 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.179 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:26.179 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:26.179 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.180 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.180 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.180 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.180 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.180 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.180 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.180 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.180 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.180 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.180 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.180 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.180 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:26.180 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:26.181 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.181 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.181 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.181 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:26.181 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.181 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.181 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.181 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.181 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.181 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.181 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.181 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.181 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.181 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.182 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.182 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.182 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.182 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.182 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.182 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:26.182 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:26.182 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.182 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.182 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.182 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.182 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.183 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.183 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.183 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.183 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.183 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.183 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.183 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.183 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.183 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.183 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:26.183 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "volumes", 2024-06-03T03:28:26.183 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:26.183 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:26.184 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:26.184 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:26.184 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:26.184 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.184 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.184 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.184 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.184 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.184 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.184 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.184 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.184 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.185 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.185 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.185 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.185 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.185 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.185 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.185 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.185 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.185 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.185 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.185 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:26.185 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:26.185 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.185 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.186 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.186 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.186 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.186 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.186 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.186 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.186 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.186 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.186 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.186 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.186 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:26.186 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:26.186 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.187 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.187 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.187 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:26.187 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.187 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.187 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.187 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.187 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.187 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.187 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.187 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.187 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.187 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.187 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.188 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.188 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.188 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.188 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.188 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:26.188 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:26.188 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.188 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.188 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.188 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.188 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.188 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.188 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.188 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.189 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.189 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.189 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.189 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.189 INFO:teuthology.orchestra.run.smithi117.stdout: "max_concurrent_clones": { 2024-06-03T03:28:26.189 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "max_concurrent_clones", 2024-06-03T03:28:26.189 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:26.189 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.189 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.189 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "4", 2024-06-03T03:28:26.189 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.189 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.189 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.189 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Number of asynchronous cloner threads", 2024-06-03T03:28:26.190 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.190 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.190 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.190 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.190 INFO:teuthology.orchestra.run.smithi117.stdout: "periodic_async_work": { 2024-06-03T03:28:26.190 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "periodic_async_work", 2024-06-03T03:28:26.190 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.190 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.190 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.190 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.190 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.190 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.190 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.190 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Periodically check for async work", 2024-06-03T03:28:26.191 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.191 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.191 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.191 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.191 INFO:teuthology.orchestra.run.smithi117.stdout: "snapshot_clone_delay": { 2024-06-03T03:28:26.191 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "snapshot_clone_delay", 2024-06-03T03:28:26.191 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:26.191 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.191 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.191 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-06-03T03:28:26.191 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.191 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.191 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.192 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-06-03T03:28:26.192 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.192 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.192 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.192 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.192 INFO:teuthology.orchestra.run.smithi117.stdout: "snapshot_clone_no_wait": { 2024-06-03T03:28:26.192 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "snapshot_clone_no_wait", 2024-06-03T03:28:26.192 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.192 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.192 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.192 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:26.192 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.192 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.192 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.193 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-06-03T03:28:26.193 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.193 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.193 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.193 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.193 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.193 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.193 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:26.193 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "zabbix", 2024-06-03T03:28:26.193 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:26.193 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:26.193 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:26.193 INFO:teuthology.orchestra.run.smithi117.stdout: "discovery_interval": { 2024-06-03T03:28:26.194 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "discovery_interval", 2024-06-03T03:28:26.194 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "uint", 2024-06-03T03:28:26.194 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.194 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.194 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "100", 2024-06-03T03:28:26.194 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.194 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.194 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.194 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.194 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.194 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.194 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.194 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.195 INFO:teuthology.orchestra.run.smithi117.stdout: "identifier": { 2024-06-03T03:28:26.195 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "identifier", 2024-06-03T03:28:26.195 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.195 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.195 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.195 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.195 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.195 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.195 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.195 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.195 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.195 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.195 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.196 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.196 INFO:teuthology.orchestra.run.smithi117.stdout: "interval": { 2024-06-03T03:28:26.196 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "interval", 2024-06-03T03:28:26.196 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-06-03T03:28:26.196 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.196 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.196 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "60", 2024-06-03T03:28:26.196 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.196 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.196 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.196 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.196 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.196 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.197 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.197 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.197 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:26.197 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:26.197 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.197 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.197 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.197 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.197 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.197 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.197 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.197 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.197 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.197 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.198 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.198 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.198 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.198 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.198 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.198 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.198 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.198 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.198 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.198 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:26.198 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:26.198 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.198 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.199 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.199 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.199 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.199 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.199 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.199 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.199 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.199 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.199 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.199 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.199 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:26.199 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:26.199 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.199 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.200 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.200 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:26.200 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.200 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.200 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.200 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.200 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.200 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.200 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.200 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.200 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.200 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.200 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.200 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.201 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.201 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.201 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.201 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:26.201 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:26.201 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.201 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.201 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.201 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.201 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.201 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.201 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.201 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.201 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.201 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.202 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.202 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.202 INFO:teuthology.orchestra.run.smithi117.stdout: "zabbix_host": { 2024-06-03T03:28:26.202 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "zabbix_host", 2024-06-03T03:28:26.202 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.202 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.202 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.202 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.202 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.202 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.202 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.202 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.203 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.203 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.203 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.203 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.203 INFO:teuthology.orchestra.run.smithi117.stdout: "zabbix_port": { 2024-06-03T03:28:26.203 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "zabbix_port", 2024-06-03T03:28:26.203 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:26.203 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.203 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.203 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "10051", 2024-06-03T03:28:26.203 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.203 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.203 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.203 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.204 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.204 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.204 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.204 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.204 INFO:teuthology.orchestra.run.smithi117.stdout: "zabbix_sender": { 2024-06-03T03:28:26.204 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "zabbix_sender", 2024-06-03T03:28:26.204 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.204 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.204 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.204 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-06-03T03:28:26.204 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.204 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.204 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.204 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.205 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.205 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.205 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.205 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.205 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.205 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.205 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-06-03T03:28:26.205 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.205 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.205 INFO:teuthology.orchestra.run.smithi117.stdout: "modules": [ 2024-06-03T03:28:26.205 INFO:teuthology.orchestra.run.smithi117.stdout: "cephadm", 2024-06-03T03:28:26.205 INFO:teuthology.orchestra.run.smithi117.stdout: "dashboard", 2024-06-03T03:28:26.205 INFO:teuthology.orchestra.run.smithi117.stdout: "iostat", 2024-06-03T03:28:26.205 INFO:teuthology.orchestra.run.smithi117.stdout: "nfs", 2024-06-03T03:28:26.206 INFO:teuthology.orchestra.run.smithi117.stdout: "restful" 2024-06-03T03:28:26.206 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.206 INFO:teuthology.orchestra.run.smithi117.stdout: "available_modules": [ 2024-06-03T03:28:26.206 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:26.206 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "alerts", 2024-06-03T03:28:26.206 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:26.206 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:26.206 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:26.206 INFO:teuthology.orchestra.run.smithi117.stdout: "interval": { 2024-06-03T03:28:26.206 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "interval", 2024-06-03T03:28:26.206 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-06-03T03:28:26.206 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.206 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.207 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "60", 2024-06-03T03:28:26.207 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.207 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.207 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.207 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "How frequently to reexamine health status", 2024-06-03T03:28:26.207 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.207 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.207 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.207 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.207 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:26.207 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:26.207 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.207 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.207 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.208 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.208 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.208 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.208 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.208 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.208 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.208 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.208 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.208 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.208 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.208 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.208 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.208 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.209 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.209 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.209 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.209 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:26.209 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:26.209 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.209 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.209 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.209 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.209 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.209 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.209 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.209 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.209 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.210 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.210 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.210 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.210 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:26.210 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:26.210 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.210 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.210 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.210 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:26.210 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.210 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.210 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.210 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.211 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.211 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.211 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.211 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.211 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.211 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.211 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.211 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.211 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.211 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.211 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.211 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:26.211 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:26.211 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.212 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.212 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.212 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.212 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.212 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.212 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.212 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.212 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.212 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.212 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.212 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.212 INFO:teuthology.orchestra.run.smithi117.stdout: "smtp_destination": { 2024-06-03T03:28:26.212 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "smtp_destination", 2024-06-03T03:28:26.213 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.213 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.213 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.213 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.213 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.213 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.213 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.213 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Email address to send alerts to", 2024-06-03T03:28:26.213 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.213 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.213 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.213 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.213 INFO:teuthology.orchestra.run.smithi117.stdout: "smtp_from_name": { 2024-06-03T03:28:26.213 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "smtp_from_name", 2024-06-03T03:28:26.214 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.214 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.214 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.214 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "Ceph", 2024-06-03T03:28:26.214 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.214 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.214 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.214 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Email From: name", 2024-06-03T03:28:26.214 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.214 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.214 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.214 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.214 INFO:teuthology.orchestra.run.smithi117.stdout: "smtp_host": { 2024-06-03T03:28:26.215 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "smtp_host", 2024-06-03T03:28:26.215 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.215 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.215 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.215 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.215 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.215 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.215 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.215 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "SMTP server", 2024-06-03T03:28:26.215 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.215 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.215 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.215 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.215 INFO:teuthology.orchestra.run.smithi117.stdout: "smtp_password": { 2024-06-03T03:28:26.216 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "smtp_password", 2024-06-03T03:28:26.216 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.216 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.216 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.216 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.216 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.216 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.216 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.216 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Password to authenticate with", 2024-06-03T03:28:26.216 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.216 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.216 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.216 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.216 INFO:teuthology.orchestra.run.smithi117.stdout: "smtp_port": { 2024-06-03T03:28:26.217 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "smtp_port", 2024-06-03T03:28:26.217 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:26.217 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.217 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.217 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "465", 2024-06-03T03:28:26.217 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.217 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.217 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.217 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "SMTP port", 2024-06-03T03:28:26.217 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.217 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.217 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.217 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.217 INFO:teuthology.orchestra.run.smithi117.stdout: "smtp_sender": { 2024-06-03T03:28:26.218 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "smtp_sender", 2024-06-03T03:28:26.218 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.218 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.218 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.218 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.218 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.218 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.218 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.222 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "SMTP envelope sender", 2024-06-03T03:28:26.222 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.222 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.222 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.222 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.222 INFO:teuthology.orchestra.run.smithi117.stdout: "smtp_ssl": { 2024-06-03T03:28:26.222 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "smtp_ssl", 2024-06-03T03:28:26.222 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.222 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.222 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.222 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:26.222 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.223 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.223 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.223 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Use SSL to connect to SMTP server", 2024-06-03T03:28:26.223 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.223 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.223 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.223 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.223 INFO:teuthology.orchestra.run.smithi117.stdout: "smtp_user": { 2024-06-03T03:28:26.223 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "smtp_user", 2024-06-03T03:28:26.223 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.223 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.223 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.223 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.223 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.224 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.224 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.224 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "User to authenticate as", 2024-06-03T03:28:26.224 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.224 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.224 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.224 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.224 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.224 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.224 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:26.224 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "balancer", 2024-06-03T03:28:26.224 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:26.224 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:26.224 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:26.225 INFO:teuthology.orchestra.run.smithi117.stdout: "active": { 2024-06-03T03:28:26.225 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "active", 2024-06-03T03:28:26.225 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.225 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.225 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.225 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:26.225 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.225 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.225 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.225 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "automatically balance PGs across cluster", 2024-06-03T03:28:26.225 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.225 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.225 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.226 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.226 INFO:teuthology.orchestra.run.smithi117.stdout: "begin_time": { 2024-06-03T03:28:26.226 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "begin_time", 2024-06-03T03:28:26.226 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.226 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.226 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.226 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0000", 2024-06-03T03:28:26.226 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.226 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.226 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.226 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "beginning time of day to automatically balance", 2024-06-03T03:28:26.226 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-03T03:28:26.226 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.226 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.227 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.227 INFO:teuthology.orchestra.run.smithi117.stdout: "begin_weekday": { 2024-06-03T03:28:26.227 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "begin_weekday", 2024-06-03T03:28:26.227 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "uint", 2024-06-03T03:28:26.227 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.227 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.227 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-06-03T03:28:26.227 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "0", 2024-06-03T03:28:26.227 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "6", 2024-06-03T03:28:26.227 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.227 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-06-03T03:28:26.227 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-03T03:28:26.227 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.228 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.228 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.228 INFO:teuthology.orchestra.run.smithi117.stdout: "crush_compat_max_iterations": { 2024-06-03T03:28:26.228 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crush_compat_max_iterations", 2024-06-03T03:28:26.228 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "uint", 2024-06-03T03:28:26.228 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.228 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.228 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "25", 2024-06-03T03:28:26.228 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "1", 2024-06-03T03:28:26.228 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "250", 2024-06-03T03:28:26.228 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.228 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-06-03T03:28:26.228 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.229 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.229 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.229 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.229 INFO:teuthology.orchestra.run.smithi117.stdout: "crush_compat_metrics": { 2024-06-03T03:28:26.229 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crush_compat_metrics", 2024-06-03T03:28:26.229 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.229 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.229 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.229 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "pgs,objects,bytes", 2024-06-03T03:28:26.229 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.229 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.229 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.229 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-06-03T03:28:26.229 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-06-03T03:28:26.230 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.230 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.230 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.230 INFO:teuthology.orchestra.run.smithi117.stdout: "crush_compat_step": { 2024-06-03T03:28:26.230 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crush_compat_step", 2024-06-03T03:28:26.230 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "float", 2024-06-03T03:28:26.230 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.230 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.230 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0.5", 2024-06-03T03:28:26.230 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "0.001", 2024-06-03T03:28:26.230 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "0.999", 2024-06-03T03:28:26.230 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.231 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "aggressiveness of optimization", 2024-06-03T03:28:26.231 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-06-03T03:28:26.231 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.231 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.231 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.231 INFO:teuthology.orchestra.run.smithi117.stdout: "end_time": { 2024-06-03T03:28:26.231 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "end_time", 2024-06-03T03:28:26.231 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.231 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.231 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.231 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "2359", 2024-06-03T03:28:26.231 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.231 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.232 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.232 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "ending time of day to automatically balance", 2024-06-03T03:28:26.232 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-03T03:28:26.232 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.232 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.232 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.232 INFO:teuthology.orchestra.run.smithi117.stdout: "end_weekday": { 2024-06-03T03:28:26.232 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "end_weekday", 2024-06-03T03:28:26.232 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "uint", 2024-06-03T03:28:26.232 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.232 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.232 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-06-03T03:28:26.232 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "0", 2024-06-03T03:28:26.232 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "6", 2024-06-03T03:28:26.233 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.233 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-06-03T03:28:26.233 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-03T03:28:26.233 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.233 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.233 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.233 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:26.233 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:26.233 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.233 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.233 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.233 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.233 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.234 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.234 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.234 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.234 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.234 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.234 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.234 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.234 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.234 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.234 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.234 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.234 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.234 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.234 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.235 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:26.235 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:26.235 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.235 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.235 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.235 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.235 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.235 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.235 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.235 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.235 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.235 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.235 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.235 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.236 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:26.236 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:26.236 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.236 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.236 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.236 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:26.236 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.236 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.236 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.236 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.236 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.236 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.236 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.236 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.237 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.237 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.237 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.237 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.237 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.237 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.237 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.237 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:26.237 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:26.237 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.237 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.237 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.237 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.237 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.238 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.238 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.238 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.238 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.238 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.238 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.238 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.238 INFO:teuthology.orchestra.run.smithi117.stdout: "min_score": { 2024-06-03T03:28:26.238 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "min_score", 2024-06-03T03:28:26.238 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "float", 2024-06-03T03:28:26.238 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.238 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.238 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-06-03T03:28:26.239 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.239 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.239 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.239 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-06-03T03:28:26.239 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.239 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.239 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.239 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.239 INFO:teuthology.orchestra.run.smithi117.stdout: "mode": { 2024-06-03T03:28:26.239 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "mode", 2024-06-03T03:28:26.239 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.239 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.239 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.239 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "upmap", 2024-06-03T03:28:26.240 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.240 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.240 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.240 INFO:teuthology.orchestra.run.smithi117.stdout: "crush-compat", 2024-06-03T03:28:26.240 INFO:teuthology.orchestra.run.smithi117.stdout: "none", 2024-06-03T03:28:26.240 INFO:teuthology.orchestra.run.smithi117.stdout: "read", 2024-06-03T03:28:26.240 INFO:teuthology.orchestra.run.smithi117.stdout: "upmap", 2024-06-03T03:28:26.240 INFO:teuthology.orchestra.run.smithi117.stdout: "upmap-read" 2024-06-03T03:28:26.240 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.240 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Balancer mode", 2024-06-03T03:28:26.240 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.240 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.240 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.240 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.240 INFO:teuthology.orchestra.run.smithi117.stdout: "pool_ids": { 2024-06-03T03:28:26.241 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "pool_ids", 2024-06-03T03:28:26.241 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.241 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.241 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.241 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.241 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.241 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.241 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.241 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-06-03T03:28:26.241 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.241 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.241 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.241 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.241 INFO:teuthology.orchestra.run.smithi117.stdout: "sleep_interval": { 2024-06-03T03:28:26.242 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sleep_interval", 2024-06-03T03:28:26.242 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-06-03T03:28:26.242 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.242 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.242 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "60", 2024-06-03T03:28:26.242 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.242 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.242 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.242 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-06-03T03:28:26.242 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.242 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.242 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.242 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.243 INFO:teuthology.orchestra.run.smithi117.stdout: "upmap_max_deviation": { 2024-06-03T03:28:26.243 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "upmap_max_deviation", 2024-06-03T03:28:26.243 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:26.243 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.243 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.243 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "5", 2024-06-03T03:28:26.243 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "1", 2024-06-03T03:28:26.243 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.243 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.243 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "deviation below which no optimization is attempted", 2024-06-03T03:28:26.243 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-06-03T03:28:26.243 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.243 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.244 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.244 INFO:teuthology.orchestra.run.smithi117.stdout: "upmap_max_optimizations": { 2024-06-03T03:28:26.244 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "upmap_max_optimizations", 2024-06-03T03:28:26.244 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "uint", 2024-06-03T03:28:26.244 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.244 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.244 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "10", 2024-06-03T03:28:26.244 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.244 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.244 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.244 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-06-03T03:28:26.244 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.244 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.245 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.245 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.245 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.245 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.245 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:26.245 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "cephadm", 2024-06-03T03:28:26.245 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:26.245 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:26.245 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:26.245 INFO:teuthology.orchestra.run.smithi117.stdout: "agent_down_multiplier": { 2024-06-03T03:28:26.245 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "agent_down_multiplier", 2024-06-03T03:28:26.245 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "float", 2024-06-03T03:28:26.245 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.245 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.246 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "3.0", 2024-06-03T03:28:26.246 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.246 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.246 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.246 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-06-03T03:28:26.246 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.246 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.246 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.246 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.246 INFO:teuthology.orchestra.run.smithi117.stdout: "agent_refresh_rate": { 2024-06-03T03:28:26.246 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "agent_refresh_rate", 2024-06-03T03:28:26.246 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-06-03T03:28:26.246 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.247 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.247 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "20", 2024-06-03T03:28:26.247 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.247 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.247 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.247 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-06-03T03:28:26.247 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.247 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.247 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.247 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.247 INFO:teuthology.orchestra.run.smithi117.stdout: "agent_starting_port": { 2024-06-03T03:28:26.247 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "agent_starting_port", 2024-06-03T03:28:26.247 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:26.247 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.248 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.248 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "4721", 2024-06-03T03:28:26.248 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.248 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.248 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.248 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-06-03T03:28:26.248 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.248 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.248 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.248 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.248 INFO:teuthology.orchestra.run.smithi117.stdout: "allow_ptrace": { 2024-06-03T03:28:26.248 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "allow_ptrace", 2024-06-03T03:28:26.248 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.249 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.249 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.249 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.249 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.249 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.249 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.249 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-06-03T03:28:26.249 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-06-03T03:28:26.249 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.249 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.249 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.249 INFO:teuthology.orchestra.run.smithi117.stdout: "autotune_interval": { 2024-06-03T03:28:26.249 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "autotune_interval", 2024-06-03T03:28:26.249 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-06-03T03:28:26.250 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.250 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.250 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "600", 2024-06-03T03:28:26.250 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.250 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.250 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.250 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "how frequently to autotune daemon memory", 2024-06-03T03:28:26.250 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.250 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.250 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.250 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.250 INFO:teuthology.orchestra.run.smithi117.stdout: "autotune_memory_target_ratio": { 2024-06-03T03:28:26.250 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "autotune_memory_target_ratio", 2024-06-03T03:28:26.250 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "float", 2024-06-03T03:28:26.251 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.251 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.251 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0.7", 2024-06-03T03:28:26.251 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.251 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.251 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.251 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-06-03T03:28:26.251 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.251 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.251 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.251 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.251 INFO:teuthology.orchestra.run.smithi117.stdout: "cephadm_log_destination": { 2024-06-03T03:28:26.251 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "cephadm_log_destination", 2024-06-03T03:28:26.251 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.252 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.252 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.252 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.252 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.252 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.252 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.252 INFO:teuthology.orchestra.run.smithi117.stdout: "file", 2024-06-03T03:28:26.252 INFO:teuthology.orchestra.run.smithi117.stdout: "file,syslog", 2024-06-03T03:28:26.252 INFO:teuthology.orchestra.run.smithi117.stdout: "syslog" 2024-06-03T03:28:26.252 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.252 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-06-03T03:28:26.252 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.252 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.252 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.253 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.253 INFO:teuthology.orchestra.run.smithi117.stdout: "cgroups_split": { 2024-06-03T03:28:26.253 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "cgroups_split", 2024-06-03T03:28:26.253 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.253 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.253 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.253 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:26.253 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.253 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.253 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.253 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-06-03T03:28:26.253 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.253 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.253 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.254 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.254 INFO:teuthology.orchestra.run.smithi117.stdout: "config_checks_enabled": { 2024-06-03T03:28:26.254 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "config_checks_enabled", 2024-06-03T03:28:26.254 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.254 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.254 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.254 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.254 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.254 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.254 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.254 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-06-03T03:28:26.254 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.254 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.254 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.255 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.255 INFO:teuthology.orchestra.run.smithi117.stdout: "config_dashboard": { 2024-06-03T03:28:26.255 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "config_dashboard", 2024-06-03T03:28:26.255 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.255 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.255 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.255 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:26.255 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.255 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.255 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.255 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-06-03T03:28:26.255 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.255 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.256 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.256 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.256 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_alertmanager": { 2024-06-03T03:28:26.256 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_alertmanager", 2024-06-03T03:28:26.256 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.256 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.256 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.256 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-06-03T03:28:26.256 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.256 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.256 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.256 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Prometheus container image", 2024-06-03T03:28:26.256 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.256 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.256 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.257 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.257 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_base": { 2024-06-03T03:28:26.257 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_base", 2024-06-03T03:28:26.257 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.257 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.257 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.257 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/ceph/ceph", 2024-06-03T03:28:26.257 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.257 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.257 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.257 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Container image name, without the tag", 2024-06-03T03:28:26.257 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.257 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.258 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.258 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.258 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_elasticsearch": { 2024-06-03T03:28:26.258 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_elasticsearch", 2024-06-03T03:28:26.258 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.258 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.258 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.258 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-06-03T03:28:26.258 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.258 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.258 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.258 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "elasticsearch container image", 2024-06-03T03:28:26.258 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.258 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.259 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.259 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.259 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_grafana": { 2024-06-03T03:28:26.259 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_grafana", 2024-06-03T03:28:26.259 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.259 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.259 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.259 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-06-03T03:28:26.259 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.259 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.259 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.259 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Prometheus container image", 2024-06-03T03:28:26.259 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.260 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.260 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.260 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.260 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_haproxy": { 2024-06-03T03:28:26.260 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_haproxy", 2024-06-03T03:28:26.260 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.260 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.260 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.260 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-06-03T03:28:26.260 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.260 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.260 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.260 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "HAproxy container image", 2024-06-03T03:28:26.261 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.261 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.261 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.261 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.261 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_jaeger_agent": { 2024-06-03T03:28:26.261 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_jaeger_agent", 2024-06-03T03:28:26.261 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.261 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.261 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.261 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-06-03T03:28:26.261 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.261 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.261 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.261 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Jaeger agent container image", 2024-06-03T03:28:26.262 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.262 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.262 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.262 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.262 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_jaeger_collector": { 2024-06-03T03:28:26.262 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_jaeger_collector", 2024-06-03T03:28:26.262 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.262 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.262 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.262 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-06-03T03:28:26.262 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.262 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.262 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.263 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Jaeger collector container image", 2024-06-03T03:28:26.263 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.263 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.263 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.263 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.263 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_jaeger_query": { 2024-06-03T03:28:26.263 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_jaeger_query", 2024-06-03T03:28:26.263 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.263 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.263 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.263 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-06-03T03:28:26.263 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.263 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.263 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.264 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Jaeger query container image", 2024-06-03T03:28:26.264 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.264 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.264 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.264 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.264 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_keepalived": { 2024-06-03T03:28:26.264 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_keepalived", 2024-06-03T03:28:26.264 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.264 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.264 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.264 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-06-03T03:28:26.264 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.264 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.265 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.265 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Keepalived container image", 2024-06-03T03:28:26.265 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.265 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.265 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.265 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.265 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_loki": { 2024-06-03T03:28:26.265 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_loki", 2024-06-03T03:28:26.265 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.265 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.265 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.265 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-06-03T03:28:26.265 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.265 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.266 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.266 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Loki container image", 2024-06-03T03:28:26.266 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.266 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.266 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.266 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.266 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_node_exporter": { 2024-06-03T03:28:26.266 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_node_exporter", 2024-06-03T03:28:26.266 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.266 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.266 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.266 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-06-03T03:28:26.266 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.266 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.267 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.267 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Prometheus container image", 2024-06-03T03:28:26.267 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.267 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.267 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.267 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.267 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_nvmeof": { 2024-06-03T03:28:26.267 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_nvmeof", 2024-06-03T03:28:26.267 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.267 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.267 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.267 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-06-03T03:28:26.267 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.267 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.268 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.268 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Nvme-of container image", 2024-06-03T03:28:26.268 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.268 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.268 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.268 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.268 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_prometheus": { 2024-06-03T03:28:26.268 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_prometheus", 2024-06-03T03:28:26.268 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.268 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.268 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.268 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-06-03T03:28:26.268 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.268 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.269 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.269 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Prometheus container image", 2024-06-03T03:28:26.269 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.269 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.269 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.269 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.269 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_promtail": { 2024-06-03T03:28:26.269 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_promtail", 2024-06-03T03:28:26.269 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.269 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.269 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.269 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-06-03T03:28:26.269 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.269 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.270 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.270 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Promtail container image", 2024-06-03T03:28:26.270 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.270 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.270 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.270 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.270 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_samba": { 2024-06-03T03:28:26.270 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_samba", 2024-06-03T03:28:26.270 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.270 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.270 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.270 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-06-03T03:28:26.270 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.270 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.271 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.271 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Samba/SMB container image", 2024-06-03T03:28:26.271 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.271 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.271 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.271 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.271 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_snmp_gateway": { 2024-06-03T03:28:26.271 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_snmp_gateway", 2024-06-03T03:28:26.271 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.271 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.271 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.271 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-06-03T03:28:26.271 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.272 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.272 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.272 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "SNMP Gateway container image", 2024-06-03T03:28:26.272 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.272 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.272 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.272 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.272 INFO:teuthology.orchestra.run.smithi117.stdout: "container_init": { 2024-06-03T03:28:26.272 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_init", 2024-06-03T03:28:26.272 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.272 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.272 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.272 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:26.272 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.273 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.273 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.273 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Run podman/docker with `--init`", 2024-06-03T03:28:26.273 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.273 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.273 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.273 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.273 INFO:teuthology.orchestra.run.smithi117.stdout: "daemon_cache_timeout": { 2024-06-03T03:28:26.273 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "daemon_cache_timeout", 2024-06-03T03:28:26.273 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-06-03T03:28:26.273 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.273 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.273 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "600", 2024-06-03T03:28:26.274 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.274 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.274 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.274 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-06-03T03:28:26.274 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.274 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.274 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.274 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.274 INFO:teuthology.orchestra.run.smithi117.stdout: "default_cephadm_command_timeout": { 2024-06-03T03:28:26.274 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "default_cephadm_command_timeout", 2024-06-03T03:28:26.274 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:26.274 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.274 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.275 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "900", 2024-06-03T03:28:26.275 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.275 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.275 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.275 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-06-03T03:28:26.275 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.275 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.275 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.275 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.275 INFO:teuthology.orchestra.run.smithi117.stdout: "default_registry": { 2024-06-03T03:28:26.275 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "default_registry", 2024-06-03T03:28:26.275 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.275 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.276 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.276 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "docker.io", 2024-06-03T03:28:26.276 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.276 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.276 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.276 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-06-03T03:28:26.276 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.276 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.276 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.276 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.276 INFO:teuthology.orchestra.run.smithi117.stdout: "device_cache_timeout": { 2024-06-03T03:28:26.276 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "device_cache_timeout", 2024-06-03T03:28:26.276 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-06-03T03:28:26.277 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.277 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.277 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "1800", 2024-06-03T03:28:26.277 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.277 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.277 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.277 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "seconds to cache device inventory", 2024-06-03T03:28:26.277 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.277 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.277 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.277 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.277 INFO:teuthology.orchestra.run.smithi117.stdout: "device_enhanced_scan": { 2024-06-03T03:28:26.277 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "device_enhanced_scan", 2024-06-03T03:28:26.277 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.278 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.278 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.278 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.278 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.278 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.278 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.278 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Use libstoragemgmt during device scans", 2024-06-03T03:28:26.278 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.278 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.278 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.278 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.278 INFO:teuthology.orchestra.run.smithi117.stdout: "facts_cache_timeout": { 2024-06-03T03:28:26.278 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "facts_cache_timeout", 2024-06-03T03:28:26.279 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-06-03T03:28:26.279 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.279 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.279 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "60", 2024-06-03T03:28:26.279 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.279 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.279 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.279 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "seconds to cache host facts data", 2024-06-03T03:28:26.279 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.279 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.279 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.279 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.279 INFO:teuthology.orchestra.run.smithi117.stdout: "grafana_dashboards_path": { 2024-06-03T03:28:26.279 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "grafana_dashboards_path", 2024-06-03T03:28:26.280 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.280 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.280 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.280 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-06-03T03:28:26.280 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.280 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.280 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.280 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-06-03T03:28:26.280 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.280 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.280 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.280 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.280 INFO:teuthology.orchestra.run.smithi117.stdout: "host_check_interval": { 2024-06-03T03:28:26.280 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "host_check_interval", 2024-06-03T03:28:26.281 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-06-03T03:28:26.281 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.281 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.281 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "600", 2024-06-03T03:28:26.281 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.281 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.281 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.281 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "how frequently to perform a host check", 2024-06-03T03:28:26.281 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.281 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.281 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.281 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.281 INFO:teuthology.orchestra.run.smithi117.stdout: "hw_monitoring": { 2024-06-03T03:28:26.281 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "hw_monitoring", 2024-06-03T03:28:26.282 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.282 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.282 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.282 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.282 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.282 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.282 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.282 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-06-03T03:28:26.282 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.282 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.282 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.282 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.282 INFO:teuthology.orchestra.run.smithi117.stdout: "inventory_list_all": { 2024-06-03T03:28:26.283 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "inventory_list_all", 2024-06-03T03:28:26.283 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.283 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.283 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.283 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.283 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.283 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.283 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.283 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-06-03T03:28:26.283 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.283 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.283 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.283 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.283 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:26.284 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:26.284 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.284 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.284 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.284 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.284 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.284 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.284 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.284 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.284 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.284 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.284 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.284 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.284 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.285 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.285 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.285 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.285 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.285 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.285 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.285 INFO:teuthology.orchestra.run.smithi117.stdout: "log_refresh_metadata": { 2024-06-03T03:28:26.285 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_refresh_metadata", 2024-06-03T03:28:26.285 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.285 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.285 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.285 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.285 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.286 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.286 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.286 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-06-03T03:28:26.286 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.286 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.286 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.286 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.286 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:26.286 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:26.286 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.286 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.286 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.286 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:26.286 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.287 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.287 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.287 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-06-03T03:28:26.287 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.287 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.287 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.287 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.287 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:26.287 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:26.287 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.287 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.287 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.287 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:26.287 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.288 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.288 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.288 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.288 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.288 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.288 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.288 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.288 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.288 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.288 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.288 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.288 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.288 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.289 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.289 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:26.289 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:26.289 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.289 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.289 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.289 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.289 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.289 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.289 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.289 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.289 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.289 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.290 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.290 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.290 INFO:teuthology.orchestra.run.smithi117.stdout: "manage_etc_ceph_ceph_conf": { 2024-06-03T03:28:26.290 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-06-03T03:28:26.290 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.290 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.290 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.290 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.290 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.290 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.290 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.290 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-06-03T03:28:26.290 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.291 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.291 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.291 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.291 INFO:teuthology.orchestra.run.smithi117.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-06-03T03:28:26.291 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-06-03T03:28:26.291 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.291 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.291 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.291 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "*", 2024-06-03T03:28:26.291 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.291 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.291 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.291 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-06-03T03:28:26.291 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.292 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.292 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.292 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.292 INFO:teuthology.orchestra.run.smithi117.stdout: "max_count_per_host": { 2024-06-03T03:28:26.292 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "max_count_per_host", 2024-06-03T03:28:26.292 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:26.292 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.292 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.292 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "10", 2024-06-03T03:28:26.292 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.292 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.292 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.292 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "max number of daemons per service per host", 2024-06-03T03:28:26.293 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.293 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.293 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.293 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.293 INFO:teuthology.orchestra.run.smithi117.stdout: "max_osd_draining_count": { 2024-06-03T03:28:26.293 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "max_osd_draining_count", 2024-06-03T03:28:26.293 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:26.293 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.293 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.293 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "10", 2024-06-03T03:28:26.293 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.293 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.293 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.294 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-06-03T03:28:26.294 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.294 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.294 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.294 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.294 INFO:teuthology.orchestra.run.smithi117.stdout: "migration_current": { 2024-06-03T03:28:26.294 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "migration_current", 2024-06-03T03:28:26.294 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:26.294 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.294 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.294 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.294 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.294 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.295 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.295 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "internal - do not modify", 2024-06-03T03:28:26.295 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.295 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.295 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.295 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.295 INFO:teuthology.orchestra.run.smithi117.stdout: "mode": { 2024-06-03T03:28:26.295 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "mode", 2024-06-03T03:28:26.295 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.295 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.295 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.295 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "root", 2024-06-03T03:28:26.295 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.295 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.296 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.296 INFO:teuthology.orchestra.run.smithi117.stdout: "cephadm-package", 2024-06-03T03:28:26.296 INFO:teuthology.orchestra.run.smithi117.stdout: "root" 2024-06-03T03:28:26.296 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.296 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "mode for remote execution of cephadm", 2024-06-03T03:28:26.296 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.296 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.296 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.296 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.296 INFO:teuthology.orchestra.run.smithi117.stdout: "oob_default_addr": { 2024-06-03T03:28:26.296 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "oob_default_addr", 2024-06-03T03:28:26.296 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.296 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.296 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.297 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "169.254.1.1", 2024-06-03T03:28:26.297 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.297 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.297 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.297 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Default address for RedFish API (oob management).", 2024-06-03T03:28:26.297 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.297 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.297 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.297 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.297 INFO:teuthology.orchestra.run.smithi117.stdout: "prometheus_alerts_path": { 2024-06-03T03:28:26.297 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "prometheus_alerts_path", 2024-06-03T03:28:26.297 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.297 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.297 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.298 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-06-03T03:28:26.298 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.298 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.298 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.298 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-06-03T03:28:26.298 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.298 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.298 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.298 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.298 INFO:teuthology.orchestra.run.smithi117.stdout: "registry_insecure": { 2024-06-03T03:28:26.298 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "registry_insecure", 2024-06-03T03:28:26.298 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.298 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.298 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.299 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.299 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.299 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.299 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.299 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-06-03T03:28:26.299 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.299 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.299 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.299 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.299 INFO:teuthology.orchestra.run.smithi117.stdout: "registry_password": { 2024-06-03T03:28:26.299 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "registry_password", 2024-06-03T03:28:26.299 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.299 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.299 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.300 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.300 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.300 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.300 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.300 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-06-03T03:28:26.300 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.300 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.300 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.300 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.300 INFO:teuthology.orchestra.run.smithi117.stdout: "registry_url": { 2024-06-03T03:28:26.300 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "registry_url", 2024-06-03T03:28:26.300 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.300 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.300 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.301 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.301 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.301 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.301 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.301 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-06-03T03:28:26.301 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.301 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.301 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.301 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.301 INFO:teuthology.orchestra.run.smithi117.stdout: "registry_username": { 2024-06-03T03:28:26.301 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "registry_username", 2024-06-03T03:28:26.301 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.301 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.301 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.302 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.302 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.302 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.302 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.302 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-06-03T03:28:26.302 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.302 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.302 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.302 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.302 INFO:teuthology.orchestra.run.smithi117.stdout: "secure_monitoring_stack": { 2024-06-03T03:28:26.302 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "secure_monitoring_stack", 2024-06-03T03:28:26.302 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.302 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.302 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.303 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.303 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.303 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.303 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.303 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-06-03T03:28:26.303 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.303 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.303 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.303 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.303 INFO:teuthology.orchestra.run.smithi117.stdout: "service_discovery_port": { 2024-06-03T03:28:26.303 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "service_discovery_port", 2024-06-03T03:28:26.303 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:26.303 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.304 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.304 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "8765", 2024-06-03T03:28:26.304 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.304 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.304 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.304 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "cephadm service discovery port", 2024-06-03T03:28:26.304 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.304 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.304 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.304 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.304 INFO:teuthology.orchestra.run.smithi117.stdout: "ssh_config_file": { 2024-06-03T03:28:26.304 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "ssh_config_file", 2024-06-03T03:28:26.304 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.305 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.305 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.305 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.305 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.305 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.305 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.305 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-06-03T03:28:26.305 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.305 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.305 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.305 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.305 INFO:teuthology.orchestra.run.smithi117.stdout: "use_agent": { 2024-06-03T03:28:26.305 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "use_agent", 2024-06-03T03:28:26.305 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.306 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.306 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.306 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.306 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.306 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.306 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.306 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-06-03T03:28:26.306 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.306 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.306 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.306 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.306 INFO:teuthology.orchestra.run.smithi117.stdout: "use_repo_digest": { 2024-06-03T03:28:26.306 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "use_repo_digest", 2024-06-03T03:28:26.307 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.307 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.307 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.307 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:26.307 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.307 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.307 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.307 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-06-03T03:28:26.307 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.307 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.307 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.307 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.307 INFO:teuthology.orchestra.run.smithi117.stdout: "warn_on_failed_host_check": { 2024-06-03T03:28:26.307 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "warn_on_failed_host_check", 2024-06-03T03:28:26.308 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.308 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.308 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.308 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:26.308 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.308 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.308 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.308 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "raise a health warning if the host check fails", 2024-06-03T03:28:26.308 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.308 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.308 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.308 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.308 INFO:teuthology.orchestra.run.smithi117.stdout: "warn_on_stray_daemons": { 2024-06-03T03:28:26.309 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "warn_on_stray_daemons", 2024-06-03T03:28:26.309 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.309 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.309 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.309 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:26.309 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.309 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.309 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.309 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-06-03T03:28:26.309 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.309 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.309 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.309 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.309 INFO:teuthology.orchestra.run.smithi117.stdout: "warn_on_stray_hosts": { 2024-06-03T03:28:26.310 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "warn_on_stray_hosts", 2024-06-03T03:28:26.310 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.310 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.310 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.310 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:26.310 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.310 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.310 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.310 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-06-03T03:28:26.310 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.310 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.310 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.310 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.311 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.311 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.311 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:26.311 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crash", 2024-06-03T03:28:26.311 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:26.311 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:26.311 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:26.311 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:26.311 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:26.311 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.311 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.311 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.311 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.311 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.312 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.312 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.312 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.312 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.312 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.312 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.312 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.312 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.312 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.312 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.312 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.312 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.312 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.312 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.313 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:26.313 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:26.313 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.313 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.313 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.313 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.313 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.313 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.313 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.313 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.313 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.313 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.313 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.314 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.314 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:26.314 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:26.314 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.314 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.314 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.314 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:26.314 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.314 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.314 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.314 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.314 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.314 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.315 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.315 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.315 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.315 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.315 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.315 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.315 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.315 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.315 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.315 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:26.315 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:26.315 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.315 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.315 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.316 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.316 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.316 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.316 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.316 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.316 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.316 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.316 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.316 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.316 INFO:teuthology.orchestra.run.smithi117.stdout: "retain_interval": { 2024-06-03T03:28:26.316 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "retain_interval", 2024-06-03T03:28:26.316 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-06-03T03:28:26.316 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.316 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.317 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "31536000", 2024-06-03T03:28:26.317 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.317 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.317 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.317 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "how long to retain crashes before pruning them", 2024-06-03T03:28:26.317 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.317 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.317 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.317 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.317 INFO:teuthology.orchestra.run.smithi117.stdout: "warn_recent_interval": { 2024-06-03T03:28:26.317 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "warn_recent_interval", 2024-06-03T03:28:26.317 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-06-03T03:28:26.317 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.318 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.318 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "1209600", 2024-06-03T03:28:26.318 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.318 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.318 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.318 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "time interval in which to warn about recent crashes", 2024-06-03T03:28:26.318 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.318 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.318 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.318 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.318 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.318 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.318 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:26.318 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "dashboard", 2024-06-03T03:28:26.319 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:26.319 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:26.319 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:26.319 INFO:teuthology.orchestra.run.smithi117.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-06-03T03:28:26.319 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-06-03T03:28:26.319 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:26.319 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.319 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.319 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "10", 2024-06-03T03:28:26.319 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.319 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.319 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.319 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.320 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.320 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.320 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.320 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.320 INFO:teuthology.orchestra.run.smithi117.stdout: "ALERTMANAGER_API_HOST": { 2024-06-03T03:28:26.320 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "ALERTMANAGER_API_HOST", 2024-06-03T03:28:26.320 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.320 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.320 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.320 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.320 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.320 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.320 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.320 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.321 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.321 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.321 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.321 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.321 INFO:teuthology.orchestra.run.smithi117.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-06-03T03:28:26.321 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-06-03T03:28:26.321 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.321 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.321 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.321 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:26.321 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.321 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.321 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.322 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.322 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.322 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.322 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.322 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.322 INFO:teuthology.orchestra.run.smithi117.stdout: "AUDIT_API_ENABLED": { 2024-06-03T03:28:26.322 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "AUDIT_API_ENABLED", 2024-06-03T03:28:26.322 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.322 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.322 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.322 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.322 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.322 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.322 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.323 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.323 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.323 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.323 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.323 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.323 INFO:teuthology.orchestra.run.smithi117.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-06-03T03:28:26.323 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-06-03T03:28:26.323 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.323 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.323 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.323 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:26.323 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.323 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.324 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.324 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.324 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.324 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.324 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.324 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.324 INFO:teuthology.orchestra.run.smithi117.stdout: "ENABLE_BROWSABLE_API": { 2024-06-03T03:28:26.324 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "ENABLE_BROWSABLE_API", 2024-06-03T03:28:26.324 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.324 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.324 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.324 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:26.324 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.325 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.325 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.325 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.325 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.325 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.325 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.325 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.325 INFO:teuthology.orchestra.run.smithi117.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-06-03T03:28:26.325 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-06-03T03:28:26.325 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.325 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.325 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.325 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:26.325 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.326 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.326 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.326 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.326 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.326 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.326 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.326 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.326 INFO:teuthology.orchestra.run.smithi117.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-06-03T03:28:26.326 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-06-03T03:28:26.326 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.326 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.326 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.326 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:26.327 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.327 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.327 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.327 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.327 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.327 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.327 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.327 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.327 INFO:teuthology.orchestra.run.smithi117.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-06-03T03:28:26.327 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-06-03T03:28:26.327 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.327 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.327 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.327 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:26.328 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.328 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.328 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.328 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.328 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.328 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.328 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.328 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.328 INFO:teuthology.orchestra.run.smithi117.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-06-03T03:28:26.328 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-06-03T03:28:26.328 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.328 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.328 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.329 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:26.329 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.329 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.329 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.329 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.329 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.329 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.329 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.329 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.329 INFO:teuthology.orchestra.run.smithi117.stdout: "FEATURE_TOGGLE_NFS": { 2024-06-03T03:28:26.329 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-06-03T03:28:26.329 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.329 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.329 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.330 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:26.330 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.330 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.330 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.330 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.330 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.330 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.330 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.330 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.330 INFO:teuthology.orchestra.run.smithi117.stdout: "FEATURE_TOGGLE_RBD": { 2024-06-03T03:28:26.330 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-06-03T03:28:26.330 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.330 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.330 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.331 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:26.331 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.331 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.331 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.331 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.331 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.331 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.331 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.331 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.331 INFO:teuthology.orchestra.run.smithi117.stdout: "FEATURE_TOGGLE_RGW": { 2024-06-03T03:28:26.331 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-06-03T03:28:26.331 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.331 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.331 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.332 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:26.332 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.332 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.332 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.332 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.332 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.332 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.332 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.332 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.332 INFO:teuthology.orchestra.run.smithi117.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-06-03T03:28:26.332 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-06-03T03:28:26.332 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.332 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.332 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.333 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.333 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.333 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.333 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.333 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.333 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.333 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.333 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.333 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.333 INFO:teuthology.orchestra.run.smithi117.stdout: "GRAFANA_API_PASSWORD": { 2024-06-03T03:28:26.333 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "GRAFANA_API_PASSWORD", 2024-06-03T03:28:26.333 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.333 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.333 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.334 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "admin", 2024-06-03T03:28:26.334 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.334 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.334 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.334 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.334 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.334 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.334 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.334 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.334 INFO:teuthology.orchestra.run.smithi117.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-06-03T03:28:26.334 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-06-03T03:28:26.334 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.334 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.334 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.335 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:26.335 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.335 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.335 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.335 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.335 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.335 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.335 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.335 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.335 INFO:teuthology.orchestra.run.smithi117.stdout: "GRAFANA_API_URL": { 2024-06-03T03:28:26.335 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "GRAFANA_API_URL", 2024-06-03T03:28:26.335 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.335 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.335 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.336 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.336 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.336 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.336 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.336 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.336 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.336 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.336 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.336 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.336 INFO:teuthology.orchestra.run.smithi117.stdout: "GRAFANA_API_USERNAME": { 2024-06-03T03:28:26.336 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "GRAFANA_API_USERNAME", 2024-06-03T03:28:26.336 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.336 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.337 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.337 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "admin", 2024-06-03T03:28:26.337 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.337 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.337 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.337 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.337 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.337 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.337 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.337 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.337 INFO:teuthology.orchestra.run.smithi117.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-06-03T03:28:26.337 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-06-03T03:28:26.337 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.337 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.338 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.338 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.338 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.338 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.338 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.338 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.338 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.338 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.338 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.338 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.338 INFO:teuthology.orchestra.run.smithi117.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-06-03T03:28:26.338 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-06-03T03:28:26.338 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.339 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.339 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.339 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.339 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.339 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.339 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.339 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.339 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.339 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.339 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.339 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.339 INFO:teuthology.orchestra.run.smithi117.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-06-03T03:28:26.339 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-06-03T03:28:26.340 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.340 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.340 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.340 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:26.340 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.340 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.340 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.340 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.340 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.340 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.340 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.340 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.340 INFO:teuthology.orchestra.run.smithi117.stdout: "ISSUE_TRACKER_API_KEY": { 2024-06-03T03:28:26.340 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-06-03T03:28:26.341 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.341 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.341 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.341 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.341 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.341 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.341 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.341 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.341 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.341 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.341 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.341 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.341 INFO:teuthology.orchestra.run.smithi117.stdout: "PROMETHEUS_API_HOST": { 2024-06-03T03:28:26.342 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "PROMETHEUS_API_HOST", 2024-06-03T03:28:26.342 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.342 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.342 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.342 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.342 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.342 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.342 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.342 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.342 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.342 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.342 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.342 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.342 INFO:teuthology.orchestra.run.smithi117.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-06-03T03:28:26.343 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-06-03T03:28:26.343 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.343 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.343 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.343 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:26.343 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.343 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.343 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.343 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.343 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.343 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.343 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.343 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.344 INFO:teuthology.orchestra.run.smithi117.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-06-03T03:28:26.344 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-06-03T03:28:26.344 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.344 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.344 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.344 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.344 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.344 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.344 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.344 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.344 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.344 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.344 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.344 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.345 INFO:teuthology.orchestra.run.smithi117.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-06-03T03:28:26.345 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-06-03T03:28:26.345 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.345 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.345 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.345 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.345 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.345 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.345 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.345 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.345 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.345 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.346 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.346 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.346 INFO:teuthology.orchestra.run.smithi117.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-06-03T03:28:26.346 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-06-03T03:28:26.346 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.346 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.346 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.346 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:26.346 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.346 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.346 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.346 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.346 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.347 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.347 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.347 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.347 INFO:teuthology.orchestra.run.smithi117.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-06-03T03:28:26.347 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-06-03T03:28:26.347 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.347 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.347 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.347 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:26.347 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.347 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.347 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.347 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.347 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.347 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.348 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.348 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.348 INFO:teuthology.orchestra.run.smithi117.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-06-03T03:28:26.348 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-06-03T03:28:26.348 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.348 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.348 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.348 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.348 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.348 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.348 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.348 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.348 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.349 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.349 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.349 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.349 INFO:teuthology.orchestra.run.smithi117.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-06-03T03:28:26.349 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-06-03T03:28:26.349 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.349 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.349 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.349 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.349 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.349 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.349 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.349 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.349 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.350 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.350 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.350 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.350 INFO:teuthology.orchestra.run.smithi117.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-06-03T03:28:26.350 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-06-03T03:28:26.350 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.350 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.350 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.350 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.350 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.350 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.350 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.350 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.350 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.351 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.351 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.351 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.351 INFO:teuthology.orchestra.run.smithi117.stdout: "PWD_POLICY_ENABLED": { 2024-06-03T03:28:26.351 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "PWD_POLICY_ENABLED", 2024-06-03T03:28:26.351 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.351 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.351 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.351 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:26.351 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.351 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.351 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.351 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.351 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.352 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.352 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.352 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.352 INFO:teuthology.orchestra.run.smithi117.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-06-03T03:28:26.352 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-06-03T03:28:26.352 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.352 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.352 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.352 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-06-03T03:28:26.352 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.352 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.352 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.353 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.353 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.353 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.353 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.353 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.353 INFO:teuthology.orchestra.run.smithi117.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-06-03T03:28:26.353 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-06-03T03:28:26.353 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:26.353 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.353 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.353 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "10", 2024-06-03T03:28:26.353 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.353 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.353 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.354 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.354 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.354 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.354 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.354 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.354 INFO:teuthology.orchestra.run.smithi117.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-06-03T03:28:26.354 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-06-03T03:28:26.354 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:26.354 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.354 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.354 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "8", 2024-06-03T03:28:26.354 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.354 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.355 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.355 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.355 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.355 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.355 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.355 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.355 INFO:teuthology.orchestra.run.smithi117.stdout: "REST_REQUESTS_TIMEOUT": { 2024-06-03T03:28:26.355 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-06-03T03:28:26.355 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:26.355 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.355 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.355 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "45", 2024-06-03T03:28:26.355 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.356 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.356 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.356 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.356 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.356 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.356 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.356 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.356 INFO:teuthology.orchestra.run.smithi117.stdout: "RGW_API_ACCESS_KEY": { 2024-06-03T03:28:26.356 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "RGW_API_ACCESS_KEY", 2024-06-03T03:28:26.356 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.356 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.356 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.356 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.357 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.357 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.357 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.357 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.357 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.357 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.357 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.357 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.357 INFO:teuthology.orchestra.run.smithi117.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-06-03T03:28:26.357 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-06-03T03:28:26.357 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.357 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.357 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.357 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "admin", 2024-06-03T03:28:26.358 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.358 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.358 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.358 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.358 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.358 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.358 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.358 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.358 INFO:teuthology.orchestra.run.smithi117.stdout: "RGW_API_SECRET_KEY": { 2024-06-03T03:28:26.358 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "RGW_API_SECRET_KEY", 2024-06-03T03:28:26.358 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.358 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.358 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.359 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.359 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.359 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.359 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.359 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.359 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.359 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.359 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.359 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.359 INFO:teuthology.orchestra.run.smithi117.stdout: "RGW_API_SSL_VERIFY": { 2024-06-03T03:28:26.359 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "RGW_API_SSL_VERIFY", 2024-06-03T03:28:26.359 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.359 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.359 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.360 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:26.360 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.360 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.360 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.360 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.360 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.360 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.360 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.360 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.360 INFO:teuthology.orchestra.run.smithi117.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-06-03T03:28:26.360 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-06-03T03:28:26.360 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:26.360 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.361 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.361 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-06-03T03:28:26.361 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.361 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.361 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.361 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.361 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.361 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.361 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.361 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.361 INFO:teuthology.orchestra.run.smithi117.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-06-03T03:28:26.361 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-06-03T03:28:26.361 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:26.361 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.362 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.362 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "10", 2024-06-03T03:28:26.362 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.362 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.362 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.362 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.362 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.362 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.362 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.362 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.362 INFO:teuthology.orchestra.run.smithi117.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-06-03T03:28:26.362 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-06-03T03:28:26.362 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:26.362 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.363 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.363 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "5", 2024-06-03T03:28:26.363 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.363 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.363 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.363 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.363 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.363 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.363 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.363 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.363 INFO:teuthology.orchestra.run.smithi117.stdout: "cross_origin_url": { 2024-06-03T03:28:26.363 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "cross_origin_url", 2024-06-03T03:28:26.363 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.364 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.364 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.364 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.364 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.364 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.364 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.364 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.364 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.364 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.364 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.364 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.364 INFO:teuthology.orchestra.run.smithi117.stdout: "crt_file": { 2024-06-03T03:28:26.364 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crt_file", 2024-06-03T03:28:26.364 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.365 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.365 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.365 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.365 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.365 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.365 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.365 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.365 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.365 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.365 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.365 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.365 INFO:teuthology.orchestra.run.smithi117.stdout: "debug": { 2024-06-03T03:28:26.365 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "debug", 2024-06-03T03:28:26.365 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.366 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.366 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.366 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.366 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.366 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.366 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.366 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Enable/disable debug options", 2024-06-03T03:28:26.366 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.366 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.366 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.366 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.366 INFO:teuthology.orchestra.run.smithi117.stdout: "jwt_token_ttl": { 2024-06-03T03:28:26.366 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "jwt_token_ttl", 2024-06-03T03:28:26.367 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:26.367 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.367 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.367 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "28800", 2024-06-03T03:28:26.367 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.367 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.367 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.367 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.367 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.367 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.367 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.367 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.367 INFO:teuthology.orchestra.run.smithi117.stdout: "key_file": { 2024-06-03T03:28:26.367 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "key_file", 2024-06-03T03:28:26.368 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.368 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.368 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.368 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.368 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.368 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.368 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.368 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.368 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.368 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.368 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.368 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.368 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:26.368 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:26.369 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.369 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.369 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.369 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.369 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.369 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.369 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.369 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.369 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.369 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.369 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.369 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.369 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.370 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.370 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.370 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.370 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.370 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.370 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.370 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:26.370 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:26.370 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.370 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.370 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.370 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.370 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.371 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.371 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.371 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.371 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.371 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.371 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.371 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.371 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:26.371 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:26.371 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.371 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.371 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.371 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:26.372 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.372 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.372 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.372 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.372 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.372 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.372 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.372 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.372 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.372 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.372 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.372 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.372 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.372 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.373 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.373 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:26.373 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:26.373 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.373 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.373 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.373 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.373 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.373 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.373 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.373 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.373 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.373 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.374 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.374 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.374 INFO:teuthology.orchestra.run.smithi117.stdout: "motd": { 2024-06-03T03:28:26.374 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "motd", 2024-06-03T03:28:26.374 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.374 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.374 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.374 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.374 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.374 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.374 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.374 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "The message of the day", 2024-06-03T03:28:26.374 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.375 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.375 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.375 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.375 INFO:teuthology.orchestra.run.smithi117.stdout: "redirect_resolve_ip_addr": { 2024-06-03T03:28:26.375 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "redirect_resolve_ip_addr", 2024-06-03T03:28:26.375 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.375 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.375 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.375 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.375 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.375 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.375 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.375 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.375 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.376 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.376 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.376 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.376 INFO:teuthology.orchestra.run.smithi117.stdout: "server_addr": { 2024-06-03T03:28:26.376 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "server_addr", 2024-06-03T03:28:26.376 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.376 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.376 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.376 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "::", 2024-06-03T03:28:26.376 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.376 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.376 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.376 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.376 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.377 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.377 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.377 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.377 INFO:teuthology.orchestra.run.smithi117.stdout: "server_port": { 2024-06-03T03:28:26.377 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "server_port", 2024-06-03T03:28:26.377 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:26.377 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.377 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.377 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "8080", 2024-06-03T03:28:26.377 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.377 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.377 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.377 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.378 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.378 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.378 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.378 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.378 INFO:teuthology.orchestra.run.smithi117.stdout: "ssl": { 2024-06-03T03:28:26.378 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "ssl", 2024-06-03T03:28:26.378 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.378 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.378 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.378 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:26.378 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.378 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.378 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.379 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.379 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.379 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.379 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.379 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.379 INFO:teuthology.orchestra.run.smithi117.stdout: "ssl_server_port": { 2024-06-03T03:28:26.379 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "ssl_server_port", 2024-06-03T03:28:26.379 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:26.379 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.379 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.379 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "8443", 2024-06-03T03:28:26.379 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.379 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.379 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.380 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.380 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.380 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.380 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.380 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.380 INFO:teuthology.orchestra.run.smithi117.stdout: "standby_behaviour": { 2024-06-03T03:28:26.380 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "standby_behaviour", 2024-06-03T03:28:26.380 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.380 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.380 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.380 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "redirect", 2024-06-03T03:28:26.380 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.380 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.380 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.381 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.381 INFO:teuthology.orchestra.run.smithi117.stdout: "redirect" 2024-06-03T03:28:26.381 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.381 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.381 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.381 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.381 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.381 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.381 INFO:teuthology.orchestra.run.smithi117.stdout: "standby_error_status_code": { 2024-06-03T03:28:26.381 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "standby_error_status_code", 2024-06-03T03:28:26.381 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:26.381 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.381 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.382 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "500", 2024-06-03T03:28:26.382 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "400", 2024-06-03T03:28:26.382 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "599", 2024-06-03T03:28:26.382 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.382 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.382 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.382 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.382 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.382 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.382 INFO:teuthology.orchestra.run.smithi117.stdout: "url_prefix": { 2024-06-03T03:28:26.382 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "url_prefix", 2024-06-03T03:28:26.382 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.382 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.383 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.383 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.383 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.383 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.383 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.383 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.383 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.383 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.383 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.383 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.383 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.383 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.383 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:26.383 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "devicehealth", 2024-06-03T03:28:26.384 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:26.384 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:26.384 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:26.384 INFO:teuthology.orchestra.run.smithi117.stdout: "enable_monitoring": { 2024-06-03T03:28:26.384 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "enable_monitoring", 2024-06-03T03:28:26.384 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.384 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.384 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.384 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:26.384 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.384 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.384 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.384 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "monitor device health metrics", 2024-06-03T03:28:26.384 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.385 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.385 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.385 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.385 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:26.385 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:26.385 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.385 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.385 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.385 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.385 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.385 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.385 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.385 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.385 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.385 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.386 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.386 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.386 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.386 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.386 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.386 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.386 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.386 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.386 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.386 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:26.386 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:26.386 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.386 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.387 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.387 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.387 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.387 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.387 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.387 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.387 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.387 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.387 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.387 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.387 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:26.387 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:26.387 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.387 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.388 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.388 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:26.388 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.388 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.388 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.388 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.388 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.388 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.388 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.388 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.388 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.388 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.388 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.388 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.389 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.389 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.389 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.389 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:26.389 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:26.389 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.389 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.389 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.389 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.389 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.389 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.389 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.389 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.390 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.390 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.390 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.390 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.390 INFO:teuthology.orchestra.run.smithi117.stdout: "mark_out_threshold": { 2024-06-03T03:28:26.390 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "mark_out_threshold", 2024-06-03T03:28:26.390 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-06-03T03:28:26.390 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.390 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.390 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "2419200", 2024-06-03T03:28:26.390 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.390 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.390 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.391 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-06-03T03:28:26.391 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.391 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.391 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.391 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.391 INFO:teuthology.orchestra.run.smithi117.stdout: "pool_name": { 2024-06-03T03:28:26.391 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "pool_name", 2024-06-03T03:28:26.391 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.391 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.391 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.391 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "device_health_metrics", 2024-06-03T03:28:26.391 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.391 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.392 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.392 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "name of pool in which to store device health metrics", 2024-06-03T03:28:26.392 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.392 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.392 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.392 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.392 INFO:teuthology.orchestra.run.smithi117.stdout: "retention_period": { 2024-06-03T03:28:26.392 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "retention_period", 2024-06-03T03:28:26.392 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-06-03T03:28:26.392 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.392 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.392 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "15552000", 2024-06-03T03:28:26.392 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.393 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.393 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.393 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "how long to retain device health metrics", 2024-06-03T03:28:26.393 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.393 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.393 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.393 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.393 INFO:teuthology.orchestra.run.smithi117.stdout: "scrape_frequency": { 2024-06-03T03:28:26.393 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "scrape_frequency", 2024-06-03T03:28:26.393 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-06-03T03:28:26.393 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.393 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.393 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "86400", 2024-06-03T03:28:26.393 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.394 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.394 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.394 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "how frequently to scrape device health metrics", 2024-06-03T03:28:26.394 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.394 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.394 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.394 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.394 INFO:teuthology.orchestra.run.smithi117.stdout: "self_heal": { 2024-06-03T03:28:26.394 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "self_heal", 2024-06-03T03:28:26.394 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.394 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.394 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.394 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:26.394 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.395 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.395 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.395 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-06-03T03:28:26.395 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.395 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.395 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.395 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.395 INFO:teuthology.orchestra.run.smithi117.stdout: "sleep_interval": { 2024-06-03T03:28:26.395 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sleep_interval", 2024-06-03T03:28:26.395 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-06-03T03:28:26.395 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.395 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.395 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "600", 2024-06-03T03:28:26.396 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.396 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.396 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.396 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "how frequently to wake up and check device health", 2024-06-03T03:28:26.396 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.396 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.396 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.396 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.396 INFO:teuthology.orchestra.run.smithi117.stdout: "warn_threshold": { 2024-06-03T03:28:26.396 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "warn_threshold", 2024-06-03T03:28:26.396 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-06-03T03:28:26.396 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.396 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.396 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "7257600", 2024-06-03T03:28:26.397 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.397 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.397 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.397 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-06-03T03:28:26.397 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.397 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.397 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.397 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.397 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.397 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.397 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:26.397 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "diskprediction_local", 2024-06-03T03:28:26.397 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:26.397 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:26.398 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:26.398 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:26.398 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:26.398 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.398 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.398 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.398 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.398 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.398 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.398 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.398 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.398 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.398 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.399 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.399 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.399 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.399 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.399 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.399 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.399 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.399 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.399 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.399 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:26.399 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:26.399 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.399 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.400 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.400 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.400 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.400 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.400 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.400 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.400 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.400 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.400 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.400 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.400 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:26.400 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:26.400 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.400 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.401 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.401 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:26.401 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.401 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.401 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.401 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.401 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.401 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.401 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.401 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.401 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.401 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.401 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.401 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.402 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.402 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.402 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.402 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:26.402 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:26.402 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.402 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.402 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.402 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.402 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.402 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.402 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.402 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.402 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.403 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.403 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.403 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.403 INFO:teuthology.orchestra.run.smithi117.stdout: "predict_interval": { 2024-06-03T03:28:26.403 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "predict_interval", 2024-06-03T03:28:26.403 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.403 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.403 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.404 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:28:25 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/2189770831' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-03T03:28:26.404 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:28:25 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/2684813020' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-06-03T03:28:26.404 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "86400", 2024-06-03T03:28:26.404 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.404 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.404 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.405 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.405 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.405 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.405 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.405 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.405 INFO:teuthology.orchestra.run.smithi117.stdout: "predictor_model": { 2024-06-03T03:28:26.405 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "predictor_model", 2024-06-03T03:28:26.405 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.405 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.405 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.405 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "prophetstor", 2024-06-03T03:28:26.405 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.406 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.406 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.406 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.406 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.406 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.406 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.406 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.406 INFO:teuthology.orchestra.run.smithi117.stdout: "sleep_interval": { 2024-06-03T03:28:26.406 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sleep_interval", 2024-06-03T03:28:26.406 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.406 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.406 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.406 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "600", 2024-06-03T03:28:26.407 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.407 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.407 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.407 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.407 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.407 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.407 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.407 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.407 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.407 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.407 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:26.407 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "influx", 2024-06-03T03:28:26.407 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": false, 2024-06-03T03:28:26.408 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "influxdb python module not found", 2024-06-03T03:28:26.408 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:26.408 INFO:teuthology.orchestra.run.smithi117.stdout: "batch_size": { 2024-06-03T03:28:26.408 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "batch_size", 2024-06-03T03:28:26.408 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:26.408 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.408 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.408 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "5000", 2024-06-03T03:28:26.408 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.408 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.408 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.408 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-06-03T03:28:26.408 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.409 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.409 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.409 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.409 INFO:teuthology.orchestra.run.smithi117.stdout: "database": { 2024-06-03T03:28:26.409 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "database", 2024-06-03T03:28:26.409 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.409 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.409 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.409 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "ceph", 2024-06-03T03:28:26.409 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.409 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.409 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.409 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-06-03T03:28:26.409 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.410 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.410 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.410 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.410 INFO:teuthology.orchestra.run.smithi117.stdout: "hostname": { 2024-06-03T03:28:26.410 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "hostname", 2024-06-03T03:28:26.410 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.410 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.410 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.410 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.410 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.410 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.410 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.410 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "InfluxDB server hostname", 2024-06-03T03:28:26.411 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.411 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.411 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.411 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.411 INFO:teuthology.orchestra.run.smithi117.stdout: "interval": { 2024-06-03T03:28:26.411 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "interval", 2024-06-03T03:28:26.411 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-06-03T03:28:26.411 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.411 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.411 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "30", 2024-06-03T03:28:26.411 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "5", 2024-06-03T03:28:26.411 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.411 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.412 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-06-03T03:28:26.412 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.412 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.412 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.412 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.412 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:26.412 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:26.412 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.412 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.412 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.412 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.412 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.412 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.413 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.413 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.413 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.413 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.413 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.413 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.413 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.413 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.413 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.413 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.413 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.413 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.413 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.413 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:26.414 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:26.414 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.414 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.414 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.414 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.414 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.414 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.414 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.414 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.414 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.414 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.414 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.414 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.414 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:26.415 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:26.415 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.415 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.415 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.415 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:26.415 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.415 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.415 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.415 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.415 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.415 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.415 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.415 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.416 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.416 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.416 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.416 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.416 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.416 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.416 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.416 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:26.416 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:26.416 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.416 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.416 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.416 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.417 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.417 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.417 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.417 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.417 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.417 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.417 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.417 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.417 INFO:teuthology.orchestra.run.smithi117.stdout: "password": { 2024-06-03T03:28:26.417 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "password", 2024-06-03T03:28:26.417 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.417 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.417 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.417 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.417 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.418 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.418 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.418 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "password of InfluxDB server user", 2024-06-03T03:28:26.418 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.418 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.418 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.418 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.418 INFO:teuthology.orchestra.run.smithi117.stdout: "port": { 2024-06-03T03:28:26.418 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "port", 2024-06-03T03:28:26.418 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:26.418 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.418 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.419 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "8086", 2024-06-03T03:28:26.419 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.419 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.419 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.419 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "InfluxDB server port", 2024-06-03T03:28:26.419 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.419 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.419 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.419 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.419 INFO:teuthology.orchestra.run.smithi117.stdout: "ssl": { 2024-06-03T03:28:26.419 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "ssl", 2024-06-03T03:28:26.419 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.419 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.420 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.420 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "false", 2024-06-03T03:28:26.420 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.420 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.420 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.420 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-06-03T03:28:26.420 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.420 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.420 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.420 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.420 INFO:teuthology.orchestra.run.smithi117.stdout: "threads": { 2024-06-03T03:28:26.420 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "threads", 2024-06-03T03:28:26.420 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:26.420 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.421 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.421 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "5", 2024-06-03T03:28:26.421 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "1", 2024-06-03T03:28:26.421 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "32", 2024-06-03T03:28:26.421 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.421 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-06-03T03:28:26.421 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.421 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.421 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.421 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.421 INFO:teuthology.orchestra.run.smithi117.stdout: "username": { 2024-06-03T03:28:26.421 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "username", 2024-06-03T03:28:26.421 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.421 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.422 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.422 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.422 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.422 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.422 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.422 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "username of InfluxDB server user", 2024-06-03T03:28:26.422 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.422 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.422 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.422 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.422 INFO:teuthology.orchestra.run.smithi117.stdout: "verify_ssl": { 2024-06-03T03:28:26.422 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "verify_ssl", 2024-06-03T03:28:26.422 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.423 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.423 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.423 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "true", 2024-06-03T03:28:26.423 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.426 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.426 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.426 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-06-03T03:28:26.426 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.426 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.426 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.426 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.426 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.426 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.426 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:26.426 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "insights", 2024-06-03T03:28:26.426 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:26.427 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:26.427 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:26.427 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:26.427 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:26.427 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.427 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.427 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.427 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.427 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.427 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.427 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.427 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.428 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.428 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.428 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.428 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.428 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.428 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.428 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.428 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.428 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.428 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.428 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.428 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:26.428 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:26.428 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.429 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.429 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.429 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.429 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.429 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.429 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.429 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.429 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.429 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.429 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.429 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.429 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:26.429 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:26.430 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.430 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.430 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.430 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:26.430 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.430 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.430 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.430 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.430 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.430 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.430 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.430 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.430 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.431 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.431 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.431 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.431 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.431 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.431 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.431 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:26.431 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:26.431 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.431 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.431 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.431 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.431 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.431 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.432 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.432 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.432 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.432 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.432 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.432 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.432 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.432 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.432 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:26.432 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "iostat", 2024-06-03T03:28:26.432 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:26.432 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:26.432 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:26.433 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:26.433 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:26.433 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.433 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.433 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.433 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.433 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.433 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.433 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.433 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.433 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.433 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.433 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.434 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.434 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.434 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.434 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.434 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.434 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.434 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.434 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.434 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:26.434 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:26.434 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.434 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.434 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.434 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.435 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.435 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.435 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.435 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.435 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.435 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.435 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.435 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.435 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:26.435 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:26.435 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.435 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.435 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.436 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:26.436 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.436 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.436 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.436 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.436 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.436 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.436 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.436 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.436 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.436 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.436 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.436 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.436 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.437 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.437 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.437 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:26.437 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:26.437 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.437 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.437 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.437 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.437 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.437 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.437 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.437 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.437 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.437 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.438 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.438 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.438 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.438 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.438 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:26.438 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "k8sevents", 2024-06-03T03:28:26.438 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:26.438 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:26.438 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:26.438 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_event_retention_days": { 2024-06-03T03:28:26.438 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "ceph_event_retention_days", 2024-06-03T03:28:26.438 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:26.438 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.439 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.439 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "7", 2024-06-03T03:28:26.439 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.439 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.439 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.439 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Days to hold ceph event information within local cache", 2024-06-03T03:28:26.439 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.439 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.439 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.439 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.439 INFO:teuthology.orchestra.run.smithi117.stdout: "config_check_secs": { 2024-06-03T03:28:26.439 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "config_check_secs", 2024-06-03T03:28:26.439 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:26.440 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.440 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.440 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "10", 2024-06-03T03:28:26.440 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "10", 2024-06-03T03:28:26.440 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.440 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.440 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-06-03T03:28:26.440 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.440 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.440 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.440 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.440 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:26.440 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:26.440 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.441 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.441 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.441 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.441 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.441 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.441 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.441 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.441 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.441 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.441 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.441 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.441 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.441 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.442 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.442 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.442 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.442 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.442 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.442 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:26.442 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:26.442 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.442 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.442 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.442 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.442 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.442 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.443 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.443 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.443 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.443 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.443 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.443 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.443 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:26.443 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:26.443 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.443 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.443 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.443 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:26.443 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.443 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.444 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.444 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.444 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.444 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.444 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.444 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.444 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.444 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.444 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.444 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.444 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.444 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.444 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.445 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:26.445 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:26.445 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.445 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.445 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.445 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.445 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.445 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.445 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.445 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.445 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.445 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.445 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.445 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.446 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.446 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.446 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:26.446 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "localpool", 2024-06-03T03:28:26.446 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:26.446 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:26.446 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:26.446 INFO:teuthology.orchestra.run.smithi117.stdout: "failure_domain": { 2024-06-03T03:28:26.446 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "failure_domain", 2024-06-03T03:28:26.446 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.446 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.446 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.446 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "host", 2024-06-03T03:28:26.447 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.447 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.447 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.447 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "failure domain for any created local pool", 2024-06-03T03:28:26.447 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-06-03T03:28:26.447 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.447 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.447 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.447 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:26.447 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:26.447 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.447 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.447 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.448 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.448 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.448 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.448 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.448 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.448 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.448 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.448 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.448 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.448 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.448 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.448 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.448 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.448 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.449 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.449 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.449 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:26.449 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:26.449 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.449 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.449 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.449 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.449 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.449 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.449 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.449 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.449 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.450 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.450 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.450 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.450 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:26.450 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:26.450 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.450 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.450 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.450 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:26.450 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.450 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.450 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.450 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.451 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.451 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.451 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.451 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.451 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.451 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.451 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.451 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.451 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.451 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.451 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.451 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:26.451 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:26.451 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.452 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.452 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.452 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.452 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.452 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.452 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.452 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.452 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.452 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.452 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.452 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.452 INFO:teuthology.orchestra.run.smithi117.stdout: "min_size": { 2024-06-03T03:28:26.452 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "min_size", 2024-06-03T03:28:26.452 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:26.453 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.453 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.453 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.453 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.453 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.453 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.453 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "default min_size for any created local pool", 2024-06-03T03:28:26.453 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-06-03T03:28:26.453 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.453 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.453 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.453 INFO:teuthology.orchestra.run.smithi117.stdout: "num_rep": { 2024-06-03T03:28:26.453 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "num_rep", 2024-06-03T03:28:26.453 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:26.454 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.454 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.454 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "3", 2024-06-03T03:28:26.454 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.454 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.454 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.454 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "default replica count for any created local pool", 2024-06-03T03:28:26.454 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.454 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.454 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.454 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.454 INFO:teuthology.orchestra.run.smithi117.stdout: "pg_num": { 2024-06-03T03:28:26.454 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "pg_num", 2024-06-03T03:28:26.455 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:26.455 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.455 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.455 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "128", 2024-06-03T03:28:26.455 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.455 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.455 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.455 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "default pg_num for any created local pool", 2024-06-03T03:28:26.455 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.455 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.455 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.455 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.455 INFO:teuthology.orchestra.run.smithi117.stdout: "prefix": { 2024-06-03T03:28:26.455 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "prefix", 2024-06-03T03:28:26.456 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.456 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.456 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.456 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.456 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.456 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.456 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.456 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "name prefix for any created local pool", 2024-06-03T03:28:26.456 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.456 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.456 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.456 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.456 INFO:teuthology.orchestra.run.smithi117.stdout: "subtree": { 2024-06-03T03:28:26.456 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "subtree", 2024-06-03T03:28:26.457 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.457 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.457 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.457 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "rack", 2024-06-03T03:28:26.457 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.457 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.457 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.457 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "CRUSH level for which to create a local pool", 2024-06-03T03:28:26.457 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-06-03T03:28:26.457 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.457 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.457 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.457 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.457 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.458 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:26.458 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "mds_autoscaler", 2024-06-03T03:28:26.458 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:26.458 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:26.458 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:26.458 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:26.458 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:26.458 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.458 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.458 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.458 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.458 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.458 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.459 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.459 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.459 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.459 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.459 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.459 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.459 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.459 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.459 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.459 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.459 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.459 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.459 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.459 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:26.460 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:26.460 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.460 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.460 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.460 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.460 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.460 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.460 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.460 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.460 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.460 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.460 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.460 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.461 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:26.461 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:26.461 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.461 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.461 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.461 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:26.461 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.461 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.461 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.461 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.461 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.461 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.461 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.461 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.462 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.462 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.462 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.462 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.462 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.462 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.462 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.462 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:26.462 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:26.462 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.462 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.462 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.463 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.463 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.463 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.463 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.463 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.463 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.463 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.463 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.463 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.463 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.463 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.463 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:26.463 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "mirroring", 2024-06-03T03:28:26.464 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:26.464 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:26.464 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:26.464 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:26.464 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:26.464 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.464 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.464 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.464 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.464 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.464 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.464 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.464 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.465 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.465 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.465 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.465 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.465 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.465 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.465 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.465 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.465 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.465 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.465 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.465 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:26.465 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:26.465 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.466 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.466 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.466 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.466 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.466 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.466 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.466 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.466 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.466 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.466 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.466 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.466 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:26.467 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:26.467 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.467 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.467 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.467 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:26.467 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.467 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.467 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.467 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.467 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.467 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.467 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.467 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.468 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.468 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.468 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.468 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.468 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.468 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.468 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.468 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:26.468 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:26.468 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.468 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.468 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.468 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.468 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.469 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.469 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.469 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.469 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.469 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.469 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.469 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.469 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.469 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.469 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:26.469 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "nfs", 2024-06-03T03:28:26.469 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:26.469 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:26.469 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:26.470 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:26.470 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:26.470 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.470 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.470 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.470 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.470 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.470 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.470 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.470 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.470 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.470 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.471 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.471 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.471 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.471 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.471 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.471 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.471 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.471 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.471 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.471 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:26.471 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:26.471 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.471 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.471 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.472 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.472 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.472 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.472 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.472 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.472 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.472 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.472 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.472 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.472 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:26.472 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:26.472 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.472 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.472 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.473 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:26.473 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.473 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.473 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.473 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.473 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.473 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.473 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.473 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.473 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.473 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.473 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.473 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.474 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.474 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.474 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.474 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:26.474 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:26.474 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.474 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.474 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.474 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.474 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.474 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.474 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.474 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.474 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.475 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.475 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.475 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.475 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.475 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.475 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:26.475 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "orchestrator", 2024-06-03T03:28:26.475 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:26.475 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:26.475 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:26.475 INFO:teuthology.orchestra.run.smithi117.stdout: "fail_fs": { 2024-06-03T03:28:26.475 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "fail_fs", 2024-06-03T03:28:26.475 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.475 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.476 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.476 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.476 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.476 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.476 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.476 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-06-03T03:28:26.476 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.476 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.476 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.476 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.476 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:26.476 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:26.476 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.477 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.477 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.477 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.477 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.477 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.477 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.477 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.477 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.477 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.477 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.477 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.478 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:25.301+0000 7f5a8a200640 1 -- 172.21.15.117:0/3573246030 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f5a680a2b90 msgr2=0x7f5a680a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:28:26.478 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:25.301+0000 7f5a8a200640 1 --2- 172.21.15.117:0/3573246030 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f5a680a2b90 0x7f5a680a5050 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f5a74009bd0 tx=0x7f5a74005f90 comp rx=0 tx=0).stop 2024-06-03T03:28:26.478 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:25.301+0000 7f5a8a200640 1 -- 172.21.15.117:0/3573246030 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a84100bd0 msgr2=0x7f5a8419a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:28:26.478 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:25.301+0000 7f5a8a200640 1 --2- 172.21.15.117:0/3573246030 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a84100bd0 0x7f5a8419a3c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5a7000ea60 tx=0x7f5a7000ef20 comp rx=0 tx=0).stop 2024-06-03T03:28:26.478 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:25.301+0000 7f5a8a200640 1 -- 172.21.15.117:0/3573246030 shutdown_connections 2024-06-03T03:28:26.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:25.301+0000 7f5a8a200640 1 --2- 172.21.15.117:0/3573246030 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a84101570 0x7f5a841a1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:26.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:25.301+0000 7f5a8a200640 1 --2- 172.21.15.117:0/3573246030 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a84100bd0 0x7f5a8419a3c0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:26.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:25.301+0000 7f5a8a200640 1 --2- 172.21.15.117:0/3573246030 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f5a680a2b90 0x7f5a680a5050 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:26.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:25.301+0000 7f5a8a200640 1 --2- 172.21.15.117:0/3573246030 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a84100270 0x7f5a84199e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:28:26.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:25.301+0000 7f5a8a200640 1 -- 172.21.15.117:0/3573246030 >> 172.21.15.117:0/3573246030 conn(0x7f5a840fbe90 msgr2=0x7f5a840fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:28:26.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:25.302+0000 7f5a8a200640 1 -- 172.21.15.117:0/3573246030 shutdown_connections 2024-06-03T03:28:26.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:28:25.302+0000 7f5a8a200640 1 -- 172.21.15.117:0/3573246030 wait complete. 2024-06-03T03:28:26.480 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:28:26 smithi151 ceph-mon[37043]: pgmap v599: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-06-03T03:28:26.480 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:28:26 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/3573246030' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-06-03T03:28:26.481 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.481 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.481 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.481 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.481 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.481 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.481 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.481 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:26.481 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:26.481 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.481 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.481 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.482 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.482 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.482 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.482 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.482 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.482 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.482 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.482 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.482 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.482 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:26.482 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:26.482 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.482 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.483 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.483 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:26.483 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.483 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.483 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.483 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.483 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.483 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.483 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.483 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.483 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.483 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.483 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.484 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.484 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.484 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.484 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.484 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:26.484 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:26.484 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.484 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.484 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.484 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.484 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.484 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.484 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.484 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.485 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.485 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.485 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.485 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.485 INFO:teuthology.orchestra.run.smithi117.stdout: "orchestrator": { 2024-06-03T03:28:26.485 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "orchestrator", 2024-06-03T03:28:26.485 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.485 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.485 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.485 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.485 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.485 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.485 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.486 INFO:teuthology.orchestra.run.smithi117.stdout: "cephadm", 2024-06-03T03:28:26.486 INFO:teuthology.orchestra.run.smithi117.stdout: "rook", 2024-06-03T03:28:26.486 INFO:teuthology.orchestra.run.smithi117.stdout: "test_orchestrator" 2024-06-03T03:28:26.486 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.486 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Orchestrator backend", 2024-06-03T03:28:26.486 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.486 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.486 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.486 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.486 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.486 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.486 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:26.486 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "osd_perf_query", 2024-06-03T03:28:26.487 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:26.487 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:26.487 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:26.487 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:26.487 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:26.487 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.487 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.487 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.487 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.487 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.487 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.487 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.487 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.488 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.488 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.488 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.488 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.488 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.488 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.488 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.488 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.488 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.488 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.488 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.488 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:26.488 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:26.488 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.489 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.489 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.489 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.489 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.489 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.489 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.489 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.489 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.489 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.489 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.489 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.489 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:26.489 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:26.490 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.490 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.490 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.490 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:26.490 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.490 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.490 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.490 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.490 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.490 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.490 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.490 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.490 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.490 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.491 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.491 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.491 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.491 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.491 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.491 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:26.491 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:26.491 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.491 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.491 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.491 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.491 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.492 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.492 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.492 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.492 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.492 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.492 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.492 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.492 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.492 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.492 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:26.492 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "osd_support", 2024-06-03T03:28:26.492 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:26.492 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:26.493 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:26.493 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:26.493 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:26.493 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.493 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.493 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.493 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.493 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.493 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.493 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.493 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.493 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.493 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.493 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.494 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.494 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.494 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.494 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.494 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.494 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.494 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.494 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.494 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:26.494 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:26.494 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.494 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.495 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.495 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.495 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.495 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.495 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.495 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.495 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.495 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.495 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.495 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.495 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:26.495 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:26.495 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.495 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.496 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.496 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:26.496 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.496 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.496 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.496 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.496 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.496 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.496 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.496 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.496 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.496 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.496 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.496 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.497 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.497 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.497 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.497 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:26.497 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:26.497 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.497 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.497 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.497 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.497 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.497 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.497 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.497 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.498 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.498 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.498 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.498 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.498 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.498 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.498 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:26.498 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "pg_autoscaler", 2024-06-03T03:28:26.498 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:26.498 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:26.498 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:26.498 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:26.499 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:26.499 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.499 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.499 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.499 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.499 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.499 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.499 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.499 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.499 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.499 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.499 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.499 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.500 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.500 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.500 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.500 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.500 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.500 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.500 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.500 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:26.500 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:26.500 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.500 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.500 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.500 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.500 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.501 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.501 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.501 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.501 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.501 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.501 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.501 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.501 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:26.501 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:26.501 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.501 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.501 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.502 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:26.502 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.502 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.502 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.502 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.502 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.502 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.502 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.502 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.502 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.502 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.502 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.502 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.503 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.503 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.503 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.503 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:26.503 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:26.503 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.503 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.503 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.503 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.503 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.503 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.503 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.503 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.504 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.504 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.504 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.504 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.504 INFO:teuthology.orchestra.run.smithi117.stdout: "sleep_interval": { 2024-06-03T03:28:26.504 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sleep_interval", 2024-06-03T03:28:26.504 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-06-03T03:28:26.504 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.504 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.504 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "60", 2024-06-03T03:28:26.504 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.504 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.504 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.505 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.505 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.505 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.505 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.505 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.505 INFO:teuthology.orchestra.run.smithi117.stdout: "threshold": { 2024-06-03T03:28:26.505 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "threshold", 2024-06-03T03:28:26.505 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "float", 2024-06-03T03:28:26.505 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.505 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.505 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "3.0", 2024-06-03T03:28:26.505 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "1.0", 2024-06-03T03:28:26.505 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.506 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.506 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "scaling threshold", 2024-06-03T03:28:26.506 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-06-03T03:28:26.506 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.506 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.506 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.506 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.506 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.506 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:26.506 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "progress", 2024-06-03T03:28:26.506 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:26.506 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:26.507 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:26.507 INFO:teuthology.orchestra.run.smithi117.stdout: "allow_pg_recovery_event": { 2024-06-03T03:28:26.507 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "allow_pg_recovery_event", 2024-06-03T03:28:26.507 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.507 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.507 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.507 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.507 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.507 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.507 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.507 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "allow the module to show pg recovery progress", 2024-06-03T03:28:26.507 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.507 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.508 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.508 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.508 INFO:teuthology.orchestra.run.smithi117.stdout: "enabled": { 2024-06-03T03:28:26.508 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "enabled", 2024-06-03T03:28:26.508 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.508 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.508 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.508 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:26.508 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.508 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.508 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.508 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.508 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.508 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.509 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.509 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.509 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:26.509 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:26.509 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.509 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.509 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.509 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.509 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.509 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.509 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.509 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.509 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.510 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.510 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.510 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.510 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.510 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.510 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.510 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.510 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.510 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.510 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.510 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:26.510 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:26.510 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.511 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.511 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.511 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.511 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.511 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.511 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.511 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.511 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.511 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.511 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.511 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.511 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:26.511 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:26.512 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.512 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.512 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.512 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:26.512 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.512 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.512 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.512 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.512 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.512 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.512 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.512 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.513 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.513 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.513 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.513 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.513 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.513 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.513 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.513 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:26.513 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:26.513 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.513 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.513 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.513 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.513 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.514 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.514 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.514 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.514 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.514 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.514 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.514 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.514 INFO:teuthology.orchestra.run.smithi117.stdout: "max_completed_events": { 2024-06-03T03:28:26.514 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "max_completed_events", 2024-06-03T03:28:26.514 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:26.514 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.514 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.514 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "50", 2024-06-03T03:28:26.515 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.515 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.515 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.515 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "number of past completed events to remember", 2024-06-03T03:28:26.515 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.515 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.515 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.515 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.515 INFO:teuthology.orchestra.run.smithi117.stdout: "sleep_interval": { 2024-06-03T03:28:26.515 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sleep_interval", 2024-06-03T03:28:26.515 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-06-03T03:28:26.515 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.515 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.515 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "5", 2024-06-03T03:28:26.516 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.516 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.516 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.516 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "how long the module is going to sleep", 2024-06-03T03:28:26.516 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.516 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.516 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.516 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.516 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.516 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.516 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:26.516 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "prometheus", 2024-06-03T03:28:26.516 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:26.516 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:26.517 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:26.517 INFO:teuthology.orchestra.run.smithi117.stdout: "cache": { 2024-06-03T03:28:26.517 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "cache", 2024-06-03T03:28:26.517 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.517 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.517 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.517 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:26.517 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.517 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.517 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.517 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.517 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.517 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.518 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.518 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.518 INFO:teuthology.orchestra.run.smithi117.stdout: "exclude_perf_counters": { 2024-06-03T03:28:26.518 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "exclude_perf_counters", 2024-06-03T03:28:26.518 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.518 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.518 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.518 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:26.518 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.518 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.518 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.518 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-06-03T03:28:26.518 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-06-03T03:28:26.519 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.519 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.519 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.519 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:26.519 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:26.519 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.519 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.519 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.519 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.519 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.519 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.519 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.519 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.520 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.520 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.520 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.520 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.520 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.520 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.520 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.520 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.520 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.520 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.520 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.520 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:26.520 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:26.521 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.521 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.521 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.521 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.521 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.521 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.521 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.521 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.521 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.521 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.521 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.522 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.522 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:26.522 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:26.522 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.522 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.522 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.522 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:26.522 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.522 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.522 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.522 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.522 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.522 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.522 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.523 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.523 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.523 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.523 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.523 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.523 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.523 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.523 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.523 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:26.523 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:26.523 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.523 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.523 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.524 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.524 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.524 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.524 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.524 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.524 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.524 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.524 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.524 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.524 INFO:teuthology.orchestra.run.smithi117.stdout: "rbd_stats_pools": { 2024-06-03T03:28:26.524 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "rbd_stats_pools", 2024-06-03T03:28:26.524 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.524 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.525 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.525 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.525 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.525 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.525 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.525 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.525 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.525 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.525 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.525 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.525 INFO:teuthology.orchestra.run.smithi117.stdout: "rbd_stats_pools_refresh_interval": { 2024-06-03T03:28:26.525 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-06-03T03:28:26.525 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:26.525 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.526 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.526 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "300", 2024-06-03T03:28:26.526 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.526 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.526 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.526 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.526 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.526 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.526 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.526 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.526 INFO:teuthology.orchestra.run.smithi117.stdout: "scrape_interval": { 2024-06-03T03:28:26.526 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "scrape_interval", 2024-06-03T03:28:26.526 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "float", 2024-06-03T03:28:26.527 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.527 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.527 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "15.0", 2024-06-03T03:28:26.527 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.527 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.527 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.527 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.527 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.527 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.527 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.527 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.527 INFO:teuthology.orchestra.run.smithi117.stdout: "server_addr": { 2024-06-03T03:28:26.527 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "server_addr", 2024-06-03T03:28:26.527 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.528 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.528 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.528 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "::", 2024-06-03T03:28:26.528 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.528 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.528 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.528 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-06-03T03:28:26.528 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.528 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.528 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.528 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.528 INFO:teuthology.orchestra.run.smithi117.stdout: "server_port": { 2024-06-03T03:28:26.528 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "server_port", 2024-06-03T03:28:26.528 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:26.529 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.529 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.529 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "9283", 2024-06-03T03:28:26.529 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.529 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.529 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.529 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-06-03T03:28:26.529 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.529 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.529 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.529 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.529 INFO:teuthology.orchestra.run.smithi117.stdout: "stale_cache_strategy": { 2024-06-03T03:28:26.529 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "stale_cache_strategy", 2024-06-03T03:28:26.529 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.530 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.530 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.530 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "log", 2024-06-03T03:28:26.530 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.530 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.530 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.530 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.530 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.530 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.530 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.530 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.530 INFO:teuthology.orchestra.run.smithi117.stdout: "standby_behaviour": { 2024-06-03T03:28:26.530 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "standby_behaviour", 2024-06-03T03:28:26.531 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.531 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.531 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.531 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "default", 2024-06-03T03:28:26.531 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.531 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.531 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.531 INFO:teuthology.orchestra.run.smithi117.stdout: "default", 2024-06-03T03:28:26.531 INFO:teuthology.orchestra.run.smithi117.stdout: "error" 2024-06-03T03:28:26.531 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.531 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.531 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.531 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.531 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.532 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.532 INFO:teuthology.orchestra.run.smithi117.stdout: "standby_error_status_code": { 2024-06-03T03:28:26.532 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "standby_error_status_code", 2024-06-03T03:28:26.532 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:26.532 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.532 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.532 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "500", 2024-06-03T03:28:26.532 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "400", 2024-06-03T03:28:26.532 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "599", 2024-06-03T03:28:26.532 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.532 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.532 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.532 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.533 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.533 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.533 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.533 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.533 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:26.533 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "rbd_support", 2024-06-03T03:28:26.533 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:26.533 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:26.533 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:26.533 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:26.533 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:26.533 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.533 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.533 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.534 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.534 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.534 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.534 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.534 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.534 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.534 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.534 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.534 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.534 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.534 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.534 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.535 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.535 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.535 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.535 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.535 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:26.535 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:26.535 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.535 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.535 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.535 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.535 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.535 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.535 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.536 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.536 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.536 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.536 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.536 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.536 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:26.536 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:26.536 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.536 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.536 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.536 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:26.536 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.536 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.537 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.537 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.537 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.537 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.537 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.537 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.537 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.537 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.537 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.537 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.537 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.537 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.537 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.537 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:26.538 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:26.538 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.538 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.538 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.538 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.538 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.538 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.538 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.538 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.538 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.538 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.538 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.539 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.539 INFO:teuthology.orchestra.run.smithi117.stdout: "max_concurrent_snap_create": { 2024-06-03T03:28:26.539 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "max_concurrent_snap_create", 2024-06-03T03:28:26.539 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:26.539 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.539 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.539 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "10", 2024-06-03T03:28:26.539 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.539 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.539 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.539 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.539 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.539 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.540 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.540 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.540 INFO:teuthology.orchestra.run.smithi117.stdout: "mirror_snapshot_schedule": { 2024-06-03T03:28:26.540 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "mirror_snapshot_schedule", 2024-06-03T03:28:26.540 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.540 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.540 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.540 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.540 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.540 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.540 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.540 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.540 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.540 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.541 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.541 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.541 INFO:teuthology.orchestra.run.smithi117.stdout: "trash_purge_schedule": { 2024-06-03T03:28:26.541 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "trash_purge_schedule", 2024-06-03T03:28:26.541 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.541 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.541 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.541 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.541 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.541 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.541 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.541 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.541 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.542 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.542 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.542 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.542 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.542 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.542 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:26.542 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "restful", 2024-06-03T03:28:26.542 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:26.542 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:26.542 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:26.542 INFO:teuthology.orchestra.run.smithi117.stdout: "enable_auth": { 2024-06-03T03:28:26.542 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "enable_auth", 2024-06-03T03:28:26.543 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.543 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.543 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.543 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:26.543 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.543 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.543 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.543 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.543 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.543 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.543 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.543 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.543 INFO:teuthology.orchestra.run.smithi117.stdout: "key_file": { 2024-06-03T03:28:26.543 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "key_file", 2024-06-03T03:28:26.544 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.544 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.544 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.544 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.544 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.544 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.544 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.544 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.544 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.544 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.544 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.544 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.544 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:26.545 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:26.545 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.545 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.545 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.545 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.545 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.545 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.545 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.545 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.545 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.545 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.545 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.545 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.545 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.546 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.546 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.546 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.546 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.546 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.546 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.546 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:26.546 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:26.546 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.546 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.546 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.546 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.546 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.546 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.547 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.547 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.547 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.547 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.547 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.547 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.547 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:26.547 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:26.547 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.547 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.547 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.547 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:26.547 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.547 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.548 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.548 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.548 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.548 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.548 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.548 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.548 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.548 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.548 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.548 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.548 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.548 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.548 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.549 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:26.549 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:26.549 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.549 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.549 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.549 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.549 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.549 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.549 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.549 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.549 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.549 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.549 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.549 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.550 INFO:teuthology.orchestra.run.smithi117.stdout: "server_addr": { 2024-06-03T03:28:26.550 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "server_addr", 2024-06-03T03:28:26.550 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.550 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.550 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.550 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.550 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.550 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.550 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.550 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.550 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.550 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.550 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.551 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.551 INFO:teuthology.orchestra.run.smithi117.stdout: "server_port": { 2024-06-03T03:28:26.551 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "server_port", 2024-06-03T03:28:26.551 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.551 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.551 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.551 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.551 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.551 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.551 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.551 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.551 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.551 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.551 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.552 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.552 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.552 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.552 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:26.552 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "rgw", 2024-06-03T03:28:26.552 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:26.552 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:26.552 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:26.552 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:26.552 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:26.552 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.552 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.552 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.553 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.553 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.553 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.553 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.553 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.553 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.553 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.553 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.553 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.553 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.553 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.553 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.553 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.553 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.554 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.554 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.554 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:26.554 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:26.554 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.554 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.554 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.554 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.554 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.554 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.554 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.554 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.554 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.555 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.555 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.555 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.555 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:26.555 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:26.555 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.555 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.555 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.555 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:26.555 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.555 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.555 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.555 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.555 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.556 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.556 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.556 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.556 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.556 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.556 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.556 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.556 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.556 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.556 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.556 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:26.556 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:26.556 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.557 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.557 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.557 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.557 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.557 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.557 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.557 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.557 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.557 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.557 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.557 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.557 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.558 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.558 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:26.558 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "rook", 2024-06-03T03:28:26.558 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:26.558 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:26.558 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:26.558 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:26.558 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:26.558 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.558 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.558 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.558 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.558 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.559 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.559 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.559 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.559 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.559 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.559 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.559 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.559 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.559 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.559 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.559 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.559 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.560 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.560 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.560 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:26.560 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:26.560 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.560 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.560 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.560 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.560 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.560 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.560 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.560 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.560 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.561 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.561 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.561 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.561 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:26.561 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:26.561 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.561 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.561 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.561 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:26.561 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.561 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.561 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.561 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.562 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.562 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.562 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.562 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.562 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.562 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.562 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.562 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.562 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.562 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.562 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.562 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:26.563 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:26.563 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.563 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.563 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.563 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.563 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.563 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.563 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.563 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.563 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.563 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.563 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.563 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.564 INFO:teuthology.orchestra.run.smithi117.stdout: "storage_class": { 2024-06-03T03:28:26.564 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "storage_class", 2024-06-03T03:28:26.564 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.564 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.564 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.564 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "local", 2024-06-03T03:28:26.564 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.564 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.564 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.564 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-06-03T03:28:26.564 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.564 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.564 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.565 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.565 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.565 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.565 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:26.565 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "selftest", 2024-06-03T03:28:26.565 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:26.565 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:26.565 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:26.565 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:26.565 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:26.565 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.565 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.565 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.566 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.566 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.566 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.566 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.566 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.566 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.566 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.566 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.566 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.566 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.566 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.566 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.566 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.567 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.567 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.567 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.567 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:26.567 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:26.567 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.567 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.567 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.567 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.567 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.567 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.567 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.567 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.567 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.568 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.568 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.568 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.568 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:26.568 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:26.568 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.568 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.568 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.568 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:26.568 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.568 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.568 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.568 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.568 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.569 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.569 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.569 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.569 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.569 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.569 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.569 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.569 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.569 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.569 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.569 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:26.569 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:26.569 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.570 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.570 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.570 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.570 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.570 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.570 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.570 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.570 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.570 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.570 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.570 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.570 INFO:teuthology.orchestra.run.smithi117.stdout: "roption1": { 2024-06-03T03:28:26.570 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "roption1", 2024-06-03T03:28:26.571 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.571 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.571 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.571 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.571 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.571 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.571 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.571 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.571 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.571 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.571 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.571 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.572 INFO:teuthology.orchestra.run.smithi117.stdout: "roption2": { 2024-06-03T03:28:26.572 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "roption2", 2024-06-03T03:28:26.572 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.572 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.572 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.572 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "xyz", 2024-06-03T03:28:26.572 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.572 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.572 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.572 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.572 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.572 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.572 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.573 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.573 INFO:teuthology.orchestra.run.smithi117.stdout: "rwoption1": { 2024-06-03T03:28:26.573 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "rwoption1", 2024-06-03T03:28:26.573 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.573 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.573 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.573 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.573 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.573 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.573 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.573 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.573 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.573 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.573 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.574 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.574 INFO:teuthology.orchestra.run.smithi117.stdout: "rwoption2": { 2024-06-03T03:28:26.574 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "rwoption2", 2024-06-03T03:28:26.574 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:26.574 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.574 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.574 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.574 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.574 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.574 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.574 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.574 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.574 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.575 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.575 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.575 INFO:teuthology.orchestra.run.smithi117.stdout: "rwoption3": { 2024-06-03T03:28:26.575 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "rwoption3", 2024-06-03T03:28:26.575 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "float", 2024-06-03T03:28:26.575 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.575 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.575 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.575 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.575 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.575 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.575 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.575 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.576 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.576 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.576 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.576 INFO:teuthology.orchestra.run.smithi117.stdout: "rwoption4": { 2024-06-03T03:28:26.576 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "rwoption4", 2024-06-03T03:28:26.576 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.576 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.576 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.576 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.576 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.576 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.576 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.577 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.577 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.577 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.577 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.577 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.577 INFO:teuthology.orchestra.run.smithi117.stdout: "rwoption5": { 2024-06-03T03:28:26.577 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "rwoption5", 2024-06-03T03:28:26.577 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.577 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.577 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.577 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.577 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.577 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.577 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.578 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.578 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.578 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.578 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.578 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.578 INFO:teuthology.orchestra.run.smithi117.stdout: "rwoption6": { 2024-06-03T03:28:26.578 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "rwoption6", 2024-06-03T03:28:26.578 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.578 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.578 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.578 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:26.578 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.578 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.579 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.579 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.579 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.579 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.579 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.579 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.579 INFO:teuthology.orchestra.run.smithi117.stdout: "rwoption7": { 2024-06-03T03:28:26.579 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "rwoption7", 2024-06-03T03:28:26.579 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:26.579 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.579 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.579 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.579 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "1", 2024-06-03T03:28:26.580 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "42", 2024-06-03T03:28:26.580 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.580 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.580 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.580 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.580 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.580 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.580 INFO:teuthology.orchestra.run.smithi117.stdout: "testkey": { 2024-06-03T03:28:26.580 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "testkey", 2024-06-03T03:28:26.580 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.580 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.580 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.580 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.580 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.581 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.581 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.581 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.581 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.581 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.581 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.581 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.581 INFO:teuthology.orchestra.run.smithi117.stdout: "testlkey": { 2024-06-03T03:28:26.581 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "testlkey", 2024-06-03T03:28:26.581 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.581 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.581 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.581 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.582 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.582 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.582 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.582 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.582 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.582 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.582 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.582 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.582 INFO:teuthology.orchestra.run.smithi117.stdout: "testnewline": { 2024-06-03T03:28:26.582 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "testnewline", 2024-06-03T03:28:26.582 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.582 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.582 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.583 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.583 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.583 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.583 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.583 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.583 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.583 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.583 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.583 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.583 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.583 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.583 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:26.583 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "snap_schedule", 2024-06-03T03:28:26.583 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:26.584 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:26.584 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:26.584 INFO:teuthology.orchestra.run.smithi117.stdout: "allow_m_granularity": { 2024-06-03T03:28:26.584 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "allow_m_granularity", 2024-06-03T03:28:26.584 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.584 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.584 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.584 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.584 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.584 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.584 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.584 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "allow minute scheduled snapshots", 2024-06-03T03:28:26.584 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.584 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.585 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.585 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.585 INFO:teuthology.orchestra.run.smithi117.stdout: "dump_on_update": { 2024-06-03T03:28:26.585 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "dump_on_update", 2024-06-03T03:28:26.585 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.585 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.585 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.585 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.585 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.585 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.585 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.585 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "dump database to debug log on update", 2024-06-03T03:28:26.585 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.586 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.586 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.586 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.586 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:26.586 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:26.586 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.586 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.586 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.586 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.586 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.586 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.586 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.586 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.587 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.587 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.587 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.587 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.587 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.587 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.587 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.587 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.587 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.587 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.587 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.587 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:26.587 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:26.587 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.587 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.588 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.588 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.588 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.588 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.588 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.588 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.588 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.588 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.588 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.588 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.588 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:26.588 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:26.588 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.589 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.589 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.589 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:26.589 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.589 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.589 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.589 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.589 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.589 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.589 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.589 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.589 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.589 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.590 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.590 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.590 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.590 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.590 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.590 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:26.590 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:26.590 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.590 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.590 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.590 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.590 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.590 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.591 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.591 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.591 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.591 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.591 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.591 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.591 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.591 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.591 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:26.591 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "stats", 2024-06-03T03:28:26.591 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:26.591 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:26.591 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:26.592 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:26.592 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:26.592 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.592 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.592 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.592 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.592 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.592 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.592 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.592 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.592 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.592 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.592 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.593 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.593 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.593 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.593 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.593 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.593 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.593 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.593 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.593 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:26.593 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:26.593 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.593 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.593 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.594 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.594 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.594 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.594 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.594 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.594 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.594 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.594 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.594 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.594 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:26.594 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:26.594 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.594 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.595 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.595 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:26.595 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.595 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.595 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.595 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.595 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.595 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.595 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.595 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.595 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.595 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.595 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.596 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.596 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.596 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.596 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.596 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:26.596 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:26.596 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.596 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.596 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.596 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.596 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.596 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.596 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.596 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.597 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.597 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.597 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.597 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.597 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.597 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.597 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:26.597 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "status", 2024-06-03T03:28:26.597 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:26.597 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:26.597 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:26.597 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:26.597 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:26.598 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.598 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.598 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.598 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.598 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.598 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.598 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.598 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.598 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.598 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.598 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.598 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.599 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.599 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.599 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.599 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.599 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.599 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.599 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.599 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:26.599 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:26.599 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.599 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.599 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.599 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.599 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.600 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.600 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.600 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.600 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.600 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.600 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.600 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.600 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:26.600 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:26.600 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.600 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.600 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.600 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:26.601 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.601 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.601 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.601 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.601 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.601 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.601 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.601 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.601 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.601 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.601 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.601 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.601 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.601 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.602 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.602 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:26.602 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:26.602 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.602 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.602 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.602 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.602 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.602 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.602 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.602 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.602 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.602 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.603 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.603 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.603 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.603 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.603 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:26.603 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "telegraf", 2024-06-03T03:28:26.603 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:26.603 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:26.603 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:26.603 INFO:teuthology.orchestra.run.smithi117.stdout: "address": { 2024-06-03T03:28:26.603 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "address", 2024-06-03T03:28:26.603 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.603 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.604 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.604 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-06-03T03:28:26.604 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.604 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.604 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.604 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.604 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.604 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.604 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.604 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.604 INFO:teuthology.orchestra.run.smithi117.stdout: "interval": { 2024-06-03T03:28:26.604 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "interval", 2024-06-03T03:28:26.604 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-06-03T03:28:26.604 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.605 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.605 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "15", 2024-06-03T03:28:26.605 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.605 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.605 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.605 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.605 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.605 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.605 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.605 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.605 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:26.605 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:26.605 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.606 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.606 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.606 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.606 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.606 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.606 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.606 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.606 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.606 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.606 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.606 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.606 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.606 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.607 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.607 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.607 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.607 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.607 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.607 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:26.607 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:26.607 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.607 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.607 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.607 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.607 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.607 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.608 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.608 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.608 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.608 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.608 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.608 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.608 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:26.608 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:26.608 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.608 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.608 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.608 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:26.608 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.609 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.609 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.609 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.609 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.609 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.609 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.609 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.609 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.609 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.609 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.609 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.609 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.610 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.610 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.610 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:26.610 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:26.610 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.610 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.610 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.610 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.610 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.610 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.610 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.610 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.610 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.611 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.611 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.611 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.611 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.611 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.611 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:26.611 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "telemetry", 2024-06-03T03:28:26.611 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:26.611 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:26.611 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:26.611 INFO:teuthology.orchestra.run.smithi117.stdout: "channel_basic": { 2024-06-03T03:28:26.611 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "channel_basic", 2024-06-03T03:28:26.611 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.612 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.612 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.612 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:26.612 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.612 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.612 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.612 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Share basic cluster information (size, version)", 2024-06-03T03:28:26.612 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.612 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.612 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.612 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.612 INFO:teuthology.orchestra.run.smithi117.stdout: "channel_crash": { 2024-06-03T03:28:26.612 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "channel_crash", 2024-06-03T03:28:26.613 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.613 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.613 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.613 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:26.613 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.613 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.613 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.613 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-06-03T03:28:26.613 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.613 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.613 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.613 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.613 INFO:teuthology.orchestra.run.smithi117.stdout: "channel_device": { 2024-06-03T03:28:26.614 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "channel_device", 2024-06-03T03:28:26.614 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.614 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.614 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.614 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:26.614 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.614 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.614 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.614 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-06-03T03:28:26.614 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.614 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.614 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.614 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.615 INFO:teuthology.orchestra.run.smithi117.stdout: "channel_ident": { 2024-06-03T03:28:26.615 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "channel_ident", 2024-06-03T03:28:26.615 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.615 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.615 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.615 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.615 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.615 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.615 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.615 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-06-03T03:28:26.615 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.615 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.615 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.615 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.616 INFO:teuthology.orchestra.run.smithi117.stdout: "channel_perf": { 2024-06-03T03:28:26.616 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "channel_perf", 2024-06-03T03:28:26.616 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.616 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.616 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.616 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.616 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.616 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.616 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.616 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Share various performance metrics of a cluster", 2024-06-03T03:28:26.616 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.616 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.616 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.617 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.617 INFO:teuthology.orchestra.run.smithi117.stdout: "contact": { 2024-06-03T03:28:26.617 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "contact", 2024-06-03T03:28:26.617 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.617 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.617 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.617 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.617 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.617 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.617 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.617 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.617 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.617 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.617 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.618 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.618 INFO:teuthology.orchestra.run.smithi117.stdout: "description": { 2024-06-03T03:28:26.618 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "description", 2024-06-03T03:28:26.618 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.618 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.618 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.618 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.618 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.618 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.618 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.618 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.618 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.618 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.619 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.619 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.619 INFO:teuthology.orchestra.run.smithi117.stdout: "device_url": { 2024-06-03T03:28:26.619 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "device_url", 2024-06-03T03:28:26.619 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.619 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.619 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.619 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-06-03T03:28:26.619 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.619 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.619 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.619 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.619 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.620 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.620 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.620 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.620 INFO:teuthology.orchestra.run.smithi117.stdout: "enabled": { 2024-06-03T03:28:26.620 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "enabled", 2024-06-03T03:28:26.620 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.620 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.620 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.620 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.620 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.620 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.620 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.620 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.620 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.621 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.621 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.621 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.621 INFO:teuthology.orchestra.run.smithi117.stdout: "interval": { 2024-06-03T03:28:26.621 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "interval", 2024-06-03T03:28:26.621 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:26.621 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.621 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.621 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "24", 2024-06-03T03:28:26.621 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "8", 2024-06-03T03:28:26.621 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.621 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.621 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.621 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.622 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.622 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.622 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.622 INFO:teuthology.orchestra.run.smithi117.stdout: "last_opt_revision": { 2024-06-03T03:28:26.622 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "last_opt_revision", 2024-06-03T03:28:26.622 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:26.622 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.622 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.622 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "1", 2024-06-03T03:28:26.622 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.622 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.622 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.622 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.623 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.623 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.623 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.623 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.623 INFO:teuthology.orchestra.run.smithi117.stdout: "leaderboard": { 2024-06-03T03:28:26.623 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "leaderboard", 2024-06-03T03:28:26.623 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.623 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.623 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.623 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.623 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.623 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.623 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.623 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.624 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.624 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.624 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.624 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.624 INFO:teuthology.orchestra.run.smithi117.stdout: "leaderboard_description": { 2024-06-03T03:28:26.624 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "leaderboard_description", 2024-06-03T03:28:26.624 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.624 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.624 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.624 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.624 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.624 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.624 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.625 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.625 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.625 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.625 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.625 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.625 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:26.625 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:26.625 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.625 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.625 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.625 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.625 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.625 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.625 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.626 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.626 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.626 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.626 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.626 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.626 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.626 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.626 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.626 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.626 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.626 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.626 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.627 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:26.627 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:26.627 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.627 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.627 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.627 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.627 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.627 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.627 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.627 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.627 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.627 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.627 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.628 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.628 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:26.628 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:26.628 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.628 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.630 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.630 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:26.630 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.630 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.630 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.630 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.630 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.630 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.630 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.631 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.631 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.631 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.631 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.631 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.631 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.631 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.631 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.631 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:26.631 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:26.631 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.631 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.631 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.632 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.632 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.632 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.632 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.632 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.632 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.632 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.632 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.632 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.632 INFO:teuthology.orchestra.run.smithi117.stdout: "organization": { 2024-06-03T03:28:26.632 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "organization", 2024-06-03T03:28:26.632 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.632 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.633 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.633 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.633 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.633 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.633 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.633 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.633 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.633 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.633 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.633 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.633 INFO:teuthology.orchestra.run.smithi117.stdout: "proxy": { 2024-06-03T03:28:26.633 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "proxy", 2024-06-03T03:28:26.633 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.634 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.634 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.634 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.634 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.634 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.634 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.634 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.634 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.634 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.634 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.634 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.634 INFO:teuthology.orchestra.run.smithi117.stdout: "url": { 2024-06-03T03:28:26.634 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "url", 2024-06-03T03:28:26.635 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.635 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.635 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.635 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-06-03T03:28:26.635 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.635 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.635 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.635 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.635 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.635 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.635 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.635 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.635 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.636 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.636 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:26.636 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "test_orchestrator", 2024-06-03T03:28:26.636 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:26.636 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:26.636 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:26.636 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:26.636 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:26.636 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.636 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.636 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.636 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.636 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.636 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.637 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.637 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.637 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.637 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.637 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.637 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.637 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.637 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.637 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.637 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.637 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.637 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.637 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.637 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:26.638 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:26.638 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.638 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.638 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.638 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.638 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.638 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.638 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.638 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.638 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.638 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.638 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.639 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.639 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:26.639 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:26.639 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.639 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.639 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.639 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:26.639 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.639 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.639 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.639 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.639 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.639 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.639 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.640 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.640 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.640 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.640 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.640 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.640 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.640 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.640 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.640 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:26.640 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:26.640 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.640 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.640 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.641 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.641 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.641 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.641 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.641 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.641 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.641 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.641 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.641 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.641 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.641 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.641 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:26.641 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "volumes", 2024-06-03T03:28:26.641 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:26.642 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:26.642 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:26.642 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:26.642 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:26.642 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.642 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.642 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.642 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.642 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.642 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.642 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.642 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.642 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.643 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.643 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.643 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.643 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.643 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.643 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.643 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.643 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.643 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.643 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.643 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:26.643 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:26.643 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.643 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.644 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.644 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.644 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.644 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.644 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.644 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.644 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.644 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.644 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.644 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.644 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:26.644 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:26.644 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.644 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.645 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.645 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:26.645 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.645 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.645 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.645 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.645 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.645 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.645 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.645 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.645 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.645 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.645 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.646 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.646 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.646 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.646 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.646 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:26.646 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:26.646 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.646 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.646 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.646 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.646 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.647 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.647 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.647 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.647 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.647 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.647 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.647 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.647 INFO:teuthology.orchestra.run.smithi117.stdout: "max_concurrent_clones": { 2024-06-03T03:28:26.647 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "max_concurrent_clones", 2024-06-03T03:28:26.647 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:26.647 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.647 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.647 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "4", 2024-06-03T03:28:26.648 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.648 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.648 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.648 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Number of asynchronous cloner threads", 2024-06-03T03:28:26.648 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.648 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.648 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.648 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.648 INFO:teuthology.orchestra.run.smithi117.stdout: "periodic_async_work": { 2024-06-03T03:28:26.648 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "periodic_async_work", 2024-06-03T03:28:26.648 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.648 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.648 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.648 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.649 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.649 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.649 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.649 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Periodically check for async work", 2024-06-03T03:28:26.649 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.649 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.649 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.649 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.649 INFO:teuthology.orchestra.run.smithi117.stdout: "snapshot_clone_delay": { 2024-06-03T03:28:26.649 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "snapshot_clone_delay", 2024-06-03T03:28:26.649 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:26.649 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.649 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.650 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-06-03T03:28:26.650 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.650 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.650 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.650 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-06-03T03:28:26.650 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.650 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.650 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.650 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.650 INFO:teuthology.orchestra.run.smithi117.stdout: "snapshot_clone_no_wait": { 2024-06-03T03:28:26.650 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "snapshot_clone_no_wait", 2024-06-03T03:28:26.650 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.650 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.651 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.651 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-06-03T03:28:26.651 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.651 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.651 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.651 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-06-03T03:28:26.651 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.651 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.651 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.651 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.651 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.651 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.651 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:26.652 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "zabbix", 2024-06-03T03:28:26.652 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-06-03T03:28:26.652 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-06-03T03:28:26.652 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-06-03T03:28:26.652 INFO:teuthology.orchestra.run.smithi117.stdout: "discovery_interval": { 2024-06-03T03:28:26.652 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "discovery_interval", 2024-06-03T03:28:26.652 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "uint", 2024-06-03T03:28:26.652 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.652 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.652 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "100", 2024-06-03T03:28:26.652 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.652 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.652 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.652 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.653 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.653 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.653 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.653 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.653 INFO:teuthology.orchestra.run.smithi117.stdout: "identifier": { 2024-06-03T03:28:26.653 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "identifier", 2024-06-03T03:28:26.653 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.653 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.653 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.653 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.653 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.653 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.653 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.654 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.654 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.654 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.654 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.654 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.654 INFO:teuthology.orchestra.run.smithi117.stdout: "interval": { 2024-06-03T03:28:26.654 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "interval", 2024-06-03T03:28:26.654 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-06-03T03:28:26.654 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.654 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.654 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "60", 2024-06-03T03:28:26.654 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.654 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.655 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.655 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.655 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.655 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.655 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.655 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.655 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-06-03T03:28:26.655 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-06-03T03:28:26.655 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.655 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.655 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.655 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.656 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.656 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.656 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.656 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.656 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.656 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.656 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.656 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.656 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.656 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.656 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.656 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.656 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.656 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.657 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.657 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-06-03T03:28:26.657 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-06-03T03:28:26.657 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.657 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.657 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.657 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.657 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.657 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.657 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.657 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.657 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.658 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.658 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.658 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.658 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-06-03T03:28:26.658 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-06-03T03:28:26.658 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.658 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.658 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.658 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-06-03T03:28:26.658 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.658 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.658 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-06-03T03:28:26.658 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-06-03T03:28:26.659 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-06-03T03:28:26.659 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-06-03T03:28:26.659 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-06-03T03:28:26.659 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-06-03T03:28:26.659 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-06-03T03:28:26.659 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.659 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.659 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.659 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.659 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.659 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.659 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-06-03T03:28:26.660 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-06-03T03:28:26.660 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-06-03T03:28:26.660 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.660 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-06-03T03:28:26.660 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-06-03T03:28:26.660 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.660 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.660 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.660 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.660 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.660 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.660 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.660 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.660 INFO:teuthology.orchestra.run.smithi117.stdout: "zabbix_host": { 2024-06-03T03:28:26.661 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "zabbix_host", 2024-06-03T03:28:26.661 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.661 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.661 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.661 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-06-03T03:28:26.661 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.661 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.661 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.661 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.661 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.661 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.661 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.661 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.662 INFO:teuthology.orchestra.run.smithi117.stdout: "zabbix_port": { 2024-06-03T03:28:26.662 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "zabbix_port", 2024-06-03T03:28:26.662 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-06-03T03:28:26.662 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.662 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.662 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "10051", 2024-06-03T03:28:26.662 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.662 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.662 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.662 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.662 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.662 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.662 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.663 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.663 INFO:teuthology.orchestra.run.smithi117.stdout: "zabbix_sender": { 2024-06-03T03:28:26.663 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "zabbix_sender", 2024-06-03T03:28:26.663 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-06-03T03:28:26.663 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-06-03T03:28:26.663 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-06-03T03:28:26.663 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-06-03T03:28:26.663 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-06-03T03:28:26.663 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-06-03T03:28:26.663 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-06-03T03:28:26.663 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-06-03T03:28:26.663 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-06-03T03:28:26.663 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-06-03T03:28:26.664 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-06-03T03:28:26.664 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.664 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.664 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.664 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.664 INFO:teuthology.orchestra.run.smithi117.stdout: "services": { 2024-06-03T03:28:26.664 INFO:teuthology.orchestra.run.smithi117.stdout: "dashboard": "https://172.21.15.117:8443/" 2024-06-03T03:28:26.664 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.664 INFO:teuthology.orchestra.run.smithi117.stdout: "always_on_modules": { 2024-06-03T03:28:26.664 INFO:teuthology.orchestra.run.smithi117.stdout: "octopus": [ 2024-06-03T03:28:26.664 INFO:teuthology.orchestra.run.smithi117.stdout: "balancer", 2024-06-03T03:28:26.664 INFO:teuthology.orchestra.run.smithi117.stdout: "crash", 2024-06-03T03:28:26.664 INFO:teuthology.orchestra.run.smithi117.stdout: "devicehealth", 2024-06-03T03:28:26.665 INFO:teuthology.orchestra.run.smithi117.stdout: "orchestrator", 2024-06-03T03:28:26.665 INFO:teuthology.orchestra.run.smithi117.stdout: "pg_autoscaler", 2024-06-03T03:28:26.665 INFO:teuthology.orchestra.run.smithi117.stdout: "progress", 2024-06-03T03:28:26.665 INFO:teuthology.orchestra.run.smithi117.stdout: "rbd_support", 2024-06-03T03:28:26.665 INFO:teuthology.orchestra.run.smithi117.stdout: "status", 2024-06-03T03:28:26.665 INFO:teuthology.orchestra.run.smithi117.stdout: "telemetry", 2024-06-03T03:28:26.665 INFO:teuthology.orchestra.run.smithi117.stdout: "volumes" 2024-06-03T03:28:26.665 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.665 INFO:teuthology.orchestra.run.smithi117.stdout: "pacific": [ 2024-06-03T03:28:26.665 INFO:teuthology.orchestra.run.smithi117.stdout: "balancer", 2024-06-03T03:28:26.665 INFO:teuthology.orchestra.run.smithi117.stdout: "crash", 2024-06-03T03:28:26.665 INFO:teuthology.orchestra.run.smithi117.stdout: "devicehealth", 2024-06-03T03:28:26.665 INFO:teuthology.orchestra.run.smithi117.stdout: "orchestrator", 2024-06-03T03:28:26.665 INFO:teuthology.orchestra.run.smithi117.stdout: "pg_autoscaler", 2024-06-03T03:28:26.666 INFO:teuthology.orchestra.run.smithi117.stdout: "progress", 2024-06-03T03:28:26.666 INFO:teuthology.orchestra.run.smithi117.stdout: "rbd_support", 2024-06-03T03:28:26.666 INFO:teuthology.orchestra.run.smithi117.stdout: "status", 2024-06-03T03:28:26.666 INFO:teuthology.orchestra.run.smithi117.stdout: "telemetry", 2024-06-03T03:28:26.666 INFO:teuthology.orchestra.run.smithi117.stdout: "volumes" 2024-06-03T03:28:26.666 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.666 INFO:teuthology.orchestra.run.smithi117.stdout: "quincy": [ 2024-06-03T03:28:26.666 INFO:teuthology.orchestra.run.smithi117.stdout: "balancer", 2024-06-03T03:28:26.666 INFO:teuthology.orchestra.run.smithi117.stdout: "crash", 2024-06-03T03:28:26.666 INFO:teuthology.orchestra.run.smithi117.stdout: "devicehealth", 2024-06-03T03:28:26.666 INFO:teuthology.orchestra.run.smithi117.stdout: "orchestrator", 2024-06-03T03:28:26.666 INFO:teuthology.orchestra.run.smithi117.stdout: "pg_autoscaler", 2024-06-03T03:28:26.667 INFO:teuthology.orchestra.run.smithi117.stdout: "progress", 2024-06-03T03:28:26.667 INFO:teuthology.orchestra.run.smithi117.stdout: "rbd_support", 2024-06-03T03:28:26.667 INFO:teuthology.orchestra.run.smithi117.stdout: "status", 2024-06-03T03:28:26.667 INFO:teuthology.orchestra.run.smithi117.stdout: "telemetry", 2024-06-03T03:28:26.667 INFO:teuthology.orchestra.run.smithi117.stdout: "volumes" 2024-06-03T03:28:26.667 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.667 INFO:teuthology.orchestra.run.smithi117.stdout: "reef": [ 2024-06-03T03:28:26.667 INFO:teuthology.orchestra.run.smithi117.stdout: "balancer", 2024-06-03T03:28:26.667 INFO:teuthology.orchestra.run.smithi117.stdout: "crash", 2024-06-03T03:28:26.667 INFO:teuthology.orchestra.run.smithi117.stdout: "devicehealth", 2024-06-03T03:28:26.667 INFO:teuthology.orchestra.run.smithi117.stdout: "orchestrator", 2024-06-03T03:28:26.667 INFO:teuthology.orchestra.run.smithi117.stdout: "pg_autoscaler", 2024-06-03T03:28:26.667 INFO:teuthology.orchestra.run.smithi117.stdout: "progress", 2024-06-03T03:28:26.667 INFO:teuthology.orchestra.run.smithi117.stdout: "rbd_support", 2024-06-03T03:28:26.668 INFO:teuthology.orchestra.run.smithi117.stdout: "status", 2024-06-03T03:28:26.668 INFO:teuthology.orchestra.run.smithi117.stdout: "telemetry", 2024-06-03T03:28:26.668 INFO:teuthology.orchestra.run.smithi117.stdout: "volumes" 2024-06-03T03:28:26.668 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-06-03T03:28:26.668 INFO:teuthology.orchestra.run.smithi117.stdout: "squid": [ 2024-06-03T03:28:26.668 INFO:teuthology.orchestra.run.smithi117.stdout: "balancer", 2024-06-03T03:28:26.668 INFO:teuthology.orchestra.run.smithi117.stdout: "crash", 2024-06-03T03:28:26.668 INFO:teuthology.orchestra.run.smithi117.stdout: "devicehealth", 2024-06-03T03:28:26.668 INFO:teuthology.orchestra.run.smithi117.stdout: "orchestrator", 2024-06-03T03:28:26.668 INFO:teuthology.orchestra.run.smithi117.stdout: "pg_autoscaler", 2024-06-03T03:28:26.668 INFO:teuthology.orchestra.run.smithi117.stdout: "progress", 2024-06-03T03:28:26.668 INFO:teuthology.orchestra.run.smithi117.stdout: "rbd_support", 2024-06-03T03:28:26.668 INFO:teuthology.orchestra.run.smithi117.stdout: "status", 2024-06-03T03:28:26.668 INFO:teuthology.orchestra.run.smithi117.stdout: "telemetry", 2024-06-03T03:28:26.669 INFO:teuthology.orchestra.run.smithi117.stdout: "volumes" 2024-06-03T03:28:26.669 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-06-03T03:28:26.669 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.669 INFO:teuthology.orchestra.run.smithi117.stdout: "last_failure_osd_epoch": 3, 2024-06-03T03:28:26.669 INFO:teuthology.orchestra.run.smithi117.stdout: "active_clients": [ 2024-06-03T03:28:26.669 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:26.669 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "devicehealth", 2024-06-03T03:28:26.669 INFO:teuthology.orchestra.run.smithi117.stdout: "addrvec": [ 2024-06-03T03:28:26.669 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:26.669 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v2", 2024-06-03T03:28:26.669 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.117:0", 2024-06-03T03:28:26.669 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 1130602469 2024-06-03T03:28:26.669 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.669 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-06-03T03:28:26.670 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.670 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:26.670 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "libcephsqlite", 2024-06-03T03:28:26.670 INFO:teuthology.orchestra.run.smithi117.stdout: "addrvec": [ 2024-06-03T03:28:26.670 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:26.670 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v2", 2024-06-03T03:28:26.670 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.117:0", 2024-06-03T03:28:26.670 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 1908883785 2024-06-03T03:28:26.670 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.670 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-06-03T03:28:26.670 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.670 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:26.670 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "rbd_support", 2024-06-03T03:28:26.671 INFO:teuthology.orchestra.run.smithi117.stdout: "addrvec": [ 2024-06-03T03:28:26.671 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:26.671 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v2", 2024-06-03T03:28:26.671 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.117:0", 2024-06-03T03:28:26.671 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 2581180598 2024-06-03T03:28:26.671 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.671 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-06-03T03:28:26.671 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:26.671 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:26.671 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "volumes", 2024-06-03T03:28:26.671 INFO:teuthology.orchestra.run.smithi117.stdout: "addrvec": [ 2024-06-03T03:28:26.671 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-06-03T03:28:26.671 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v2", 2024-06-03T03:28:26.671 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.117:0", 2024-06-03T03:28:26.672 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 2352316279 2024-06-03T03:28:26.672 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.672 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-06-03T03:28:26.672 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:26.672 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-06-03T03:28:26.672 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-06-03T03:28:26.678 DEBUG:tasks.check_counter:Getting stats from a 2024-06-03T03:28:26.678 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph daemon mds.a perf dump 2024-06-03T03:28:26.679 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:28:26 smithi117 ceph-mon[31649]: pgmap v599: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-06-03T03:28:26.679 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:28:26 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/3573246030' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-06-03T03:28:26.679 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:28:26 smithi183 ceph-mon[37326]: pgmap v599: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-06-03T03:28:26.680 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:28:26 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/3573246030' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-06-03T03:28:27.130 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:28:28.197 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:28:28 smithi117 ceph-mon[31649]: pgmap v600: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 8 op/s 2024-06-03T03:28:28.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:28:28 smithi151 ceph-mon[37043]: pgmap v600: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 8 op/s 2024-06-03T03:28:28.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:28:28 smithi183 ceph-mon[37326]: pgmap v600: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 8 op/s 2024-06-03T03:28:29.775 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-06-03T03:28:29.775 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-0": { 2024-06-03T03:28:29.775 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 224, 2024-06-03T03:28:29.776 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 208, 2024-06-03T03:28:29.776 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 801349, 2024-06-03T03:28:29.776 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 225186, 2024-06-03T03:28:29.776 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 17, 2024-06-03T03:28:29.776 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 4, 2024-06-03T03:28:29.776 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.073557283, 2024-06-03T03:28:29.776 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.017027005, 2024-06-03T03:28:29.776 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.055090581, 2024-06-03T03:28:29.776 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.007949063, 2024-06-03T03:28:29.776 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T03:28:29.776 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 208, 2024-06-03T03:28:29.776 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.026446314, 2024-06-03T03:28:29.777 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000127145 2024-06-03T03:28:29.777 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:29.777 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-06-03T03:28:29.777 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 10, 2024-06-03T03:28:29.777 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000002119, 2024-06-03T03:28:29.777 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000211 2024-06-03T03:28:29.777 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:29.777 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 797360, 2024-06-03T03:28:29.777 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 98000 2024-06-03T03:28:29.777 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:29.777 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-1": { 2024-06-03T03:28:29.777 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 12, 2024-06-03T03:28:29.778 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 124, 2024-06-03T03:28:29.778 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 3122, 2024-06-03T03:28:29.778 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 260924, 2024-06-03T03:28:29.778 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 6, 2024-06-03T03:28:29.778 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 4, 2024-06-03T03:28:29.778 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.026369279, 2024-06-03T03:28:29.778 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.016601645, 2024-06-03T03:28:29.778 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.005719385, 2024-06-03T03:28:29.778 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.001555452, 2024-06-03T03:28:29.778 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T03:28:29.778 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 122, 2024-06-03T03:28:29.779 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.028313116, 2024-06-03T03:28:29.779 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000232074 2024-06-03T03:28:29.779 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:29.779 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-06-03T03:28:29.779 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:29.779 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:29.779 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:29.779 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:29.779 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 160, 2024-06-03T03:28:29.779 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 250816 2024-06-03T03:28:29.779 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:29.779 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-2": { 2024-06-03T03:28:29.779 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 171, 2024-06-03T03:28:29.780 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 281, 2024-06-03T03:28:29.780 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 43107, 2024-06-03T03:28:29.780 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 55291, 2024-06-03T03:28:29.780 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 6, 2024-06-03T03:28:29.780 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 4, 2024-06-03T03:28:29.780 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.054854225, 2024-06-03T03:28:29.780 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.022758542, 2024-06-03T03:28:29.780 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.022872780, 2024-06-03T03:28:29.780 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.005725365, 2024-06-03T03:28:29.780 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T03:28:29.780 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 279, 2024-06-03T03:28:29.780 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.046515481, 2024-06-03T03:28:29.780 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000166722 2024-06-03T03:28:29.781 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:29.781 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-06-03T03:28:29.781 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 417, 2024-06-03T03:28:29.781 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000248013, 2024-06-03T03:28:29.781 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000594 2024-06-03T03:28:29.781 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:29.781 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-03T03:28:29.781 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 0 2024-06-03T03:28:29.781 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:29.781 INFO:teuthology.orchestra.run.smithi117.stdout: "cct": { 2024-06-03T03:28:29.781 INFO:teuthology.orchestra.run.smithi117.stdout: "total_workers": 1, 2024-06-03T03:28:29.781 INFO:teuthology.orchestra.run.smithi117.stdout: "unhealthy_workers": 0 2024-06-03T03:28:29.781 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:29.782 INFO:teuthology.orchestra.run.smithi117.stdout: "finisher-MDSRank": { 2024-06-03T03:28:29.782 INFO:teuthology.orchestra.run.smithi117.stdout: "queue_len": 0, 2024-06-03T03:28:29.782 INFO:teuthology.orchestra.run.smithi117.stdout: "complete_latency": { 2024-06-03T03:28:29.782 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 38, 2024-06-03T03:28:29.782 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.001714033, 2024-06-03T03:28:29.782 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000045106 2024-06-03T03:28:29.782 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:29.782 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:29.782 INFO:teuthology.orchestra.run.smithi117.stdout: "finisher-PurgeQueue": { 2024-06-03T03:28:29.782 INFO:teuthology.orchestra.run.smithi117.stdout: "queue_len": 0, 2024-06-03T03:28:29.782 INFO:teuthology.orchestra.run.smithi117.stdout: "complete_latency": { 2024-06-03T03:28:29.782 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 2, 2024-06-03T03:28:29.783 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000008244, 2024-06-03T03:28:29.783 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000004122 2024-06-03T03:28:29.783 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:29.783 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:29.783 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-06-03T03:28:29.783 INFO:teuthology.orchestra.run.smithi117.stdout: "request": 0, 2024-06-03T03:28:29.783 INFO:teuthology.orchestra.run.smithi117.stdout: "reply": 0, 2024-06-03T03:28:29.783 INFO:teuthology.orchestra.run.smithi117.stdout: "reply_latency": { 2024-06-03T03:28:29.783 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:29.783 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:29.783 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:29.783 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:29.783 INFO:teuthology.orchestra.run.smithi117.stdout: "slow_reply": 0, 2024-06-03T03:28:29.784 INFO:teuthology.orchestra.run.smithi117.stdout: "forward": 0, 2024-06-03T03:28:29.784 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_fetch_complete": 0, 2024-06-03T03:28:29.784 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_fetch_keys": 0, 2024-06-03T03:28:29.784 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_commit": 11, 2024-06-03T03:28:29.784 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_split": 0, 2024-06-03T03:28:29.784 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_merge": 0, 2024-06-03T03:28:29.784 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes": 10, 2024-06-03T03:28:29.784 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_top": 0, 2024-06-03T03:28:29.784 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_bottom": 10, 2024-06-03T03:28:29.784 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_pin_tail": 0, 2024-06-03T03:28:29.784 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_pinned": 10, 2024-06-03T03:28:29.784 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_expired": 0, 2024-06-03T03:28:29.784 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_with_caps": 0, 2024-06-03T03:28:29.785 INFO:teuthology.orchestra.run.smithi117.stdout: "caps": 0, 2024-06-03T03:28:29.785 INFO:teuthology.orchestra.run.smithi117.stdout: "subtrees": 1, 2024-06-03T03:28:29.785 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse": 0, 2024-06-03T03:28:29.785 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_hit": 0, 2024-06-03T03:28:29.785 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_forward": 0, 2024-06-03T03:28:29.785 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_discover": 0, 2024-06-03T03:28:29.785 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_dir_fetch": 0, 2024-06-03T03:28:29.785 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_remote_ino": 0, 2024-06-03T03:28:29.785 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_lock": 0, 2024-06-03T03:28:29.785 INFO:teuthology.orchestra.run.smithi117.stdout: "load_cent": 0, 2024-06-03T03:28:29.785 INFO:teuthology.orchestra.run.smithi117.stdout: "q": 0, 2024-06-03T03:28:29.785 INFO:teuthology.orchestra.run.smithi117.stdout: "exported": 0, 2024-06-03T03:28:29.785 INFO:teuthology.orchestra.run.smithi117.stdout: "exported_inodes": 0, 2024-06-03T03:28:29.786 INFO:teuthology.orchestra.run.smithi117.stdout: "imported": 0, 2024-06-03T03:28:29.786 INFO:teuthology.orchestra.run.smithi117.stdout: "imported_inodes": 0, 2024-06-03T03:28:29.786 INFO:teuthology.orchestra.run.smithi117.stdout: "openino_dir_fetch": 0, 2024-06-03T03:28:29.786 INFO:teuthology.orchestra.run.smithi117.stdout: "openino_backtrace_fetch": 0, 2024-06-03T03:28:29.786 INFO:teuthology.orchestra.run.smithi117.stdout: "openino_peer_discover": 0, 2024-06-03T03:28:29.786 INFO:teuthology.orchestra.run.smithi117.stdout: "root_rfiles": 4, 2024-06-03T03:28:29.786 INFO:teuthology.orchestra.run.smithi117.stdout: "root_rbytes": 234, 2024-06-03T03:28:29.786 INFO:teuthology.orchestra.run.smithi117.stdout: "root_rsnaps": 0, 2024-06-03T03:28:29.786 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_backtrace_fetch": 0, 2024-06-03T03:28:29.786 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_set_tag": 0, 2024-06-03T03:28:29.786 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_backtrace_repaired": 0, 2024-06-03T03:28:29.786 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_inotable_repaired": 0, 2024-06-03T03:28:29.786 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_dir_inodes": 0, 2024-06-03T03:28:29.787 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_dir_base_inodes": 0, 2024-06-03T03:28:29.787 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_dirfrag_rstats": 0, 2024-06-03T03:28:29.787 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_file_inodes": 0, 2024-06-03T03:28:29.787 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_inode_file_caps": 0, 2024-06-03T03:28:29.787 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_revoke": 0, 2024-06-03T03:28:29.787 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_grant": 0, 2024-06-03T03:28:29.787 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_trunc": 0, 2024-06-03T03:28:29.787 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-03T03:28:29.787 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-03T03:28:29.787 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_caps": 0, 2024-06-03T03:28:29.787 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_caps_dirty": 0, 2024-06-03T03:28:29.787 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_cap_release": 0, 2024-06-03T03:28:29.787 INFO:teuthology.orchestra.run.smithi117.stdout: "process_request_cap_release": 0 2024-06-03T03:28:29.788 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:29.788 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_cache": { 2024-06-03T03:28:29.788 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_update": 0, 2024-06-03T03:28:29.788 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_update_receipt": 0, 2024-06-03T03:28:29.788 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_try_discover": 0, 2024-06-03T03:28:29.788 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_send_discover": 1, 2024-06-03T03:28:29.788 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_handle_discover": 0, 2024-06-03T03:28:29.788 INFO:teuthology.orchestra.run.smithi117.stdout: "num_strays": 0, 2024-06-03T03:28:29.788 INFO:teuthology.orchestra.run.smithi117.stdout: "num_strays_delayed": 0, 2024-06-03T03:28:29.788 INFO:teuthology.orchestra.run.smithi117.stdout: "num_strays_enqueuing": 0, 2024-06-03T03:28:29.788 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_created": 0, 2024-06-03T03:28:29.788 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_enqueued": 0, 2024-06-03T03:28:29.788 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_reintegrated": 0, 2024-06-03T03:28:29.789 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_migrated": 0, 2024-06-03T03:28:29.789 INFO:teuthology.orchestra.run.smithi117.stdout: "num_recovering_processing": 0, 2024-06-03T03:28:29.789 INFO:teuthology.orchestra.run.smithi117.stdout: "num_recovering_enqueued": 0, 2024-06-03T03:28:29.789 INFO:teuthology.orchestra.run.smithi117.stdout: "num_recovering_prioritized": 0, 2024-06-03T03:28:29.789 INFO:teuthology.orchestra.run.smithi117.stdout: "recovery_started": 0, 2024-06-03T03:28:29.789 INFO:teuthology.orchestra.run.smithi117.stdout: "recovery_completed": 0, 2024-06-03T03:28:29.789 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_quiesce_path": 0, 2024-06-03T03:28:29.789 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_quiesce_inode": 0, 2024-06-03T03:28:29.789 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_enqueue_scrub": 0, 2024-06-03T03:28:29.789 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_exportdir": 0, 2024-06-03T03:28:29.789 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_flush": 0, 2024-06-03T03:28:29.789 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_fragmentdir": 0, 2024-06-03T03:28:29.789 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_fragstats": 0, 2024-06-03T03:28:29.789 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_inodestats": 0 2024-06-03T03:28:29.790 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:29.790 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_log": { 2024-06-03T03:28:29.790 INFO:teuthology.orchestra.run.smithi117.stdout: "evlrg": 0, 2024-06-03T03:28:29.790 INFO:teuthology.orchestra.run.smithi117.stdout: "evadd": 2, 2024-06-03T03:28:29.790 INFO:teuthology.orchestra.run.smithi117.stdout: "evex": 0, 2024-06-03T03:28:29.790 INFO:teuthology.orchestra.run.smithi117.stdout: "evtrm": 0, 2024-06-03T03:28:29.790 INFO:teuthology.orchestra.run.smithi117.stdout: "ev": 2, 2024-06-03T03:28:29.790 INFO:teuthology.orchestra.run.smithi117.stdout: "evexg": 0, 2024-06-03T03:28:29.790 INFO:teuthology.orchestra.run.smithi117.stdout: "evexd": 0, 2024-06-03T03:28:29.790 INFO:teuthology.orchestra.run.smithi117.stdout: "segadd": 2, 2024-06-03T03:28:29.790 INFO:teuthology.orchestra.run.smithi117.stdout: "segex": 0, 2024-06-03T03:28:29.790 INFO:teuthology.orchestra.run.smithi117.stdout: "segtrm": 0, 2024-06-03T03:28:29.790 INFO:teuthology.orchestra.run.smithi117.stdout: "seg": 2, 2024-06-03T03:28:29.791 INFO:teuthology.orchestra.run.smithi117.stdout: "segmjr": 2, 2024-06-03T03:28:29.791 INFO:teuthology.orchestra.run.smithi117.stdout: "segexg": 0, 2024-06-03T03:28:29.791 INFO:teuthology.orchestra.run.smithi117.stdout: "segexd": 0, 2024-06-03T03:28:29.791 INFO:teuthology.orchestra.run.smithi117.stdout: "expos": 4194304, 2024-06-03T03:28:29.791 INFO:teuthology.orchestra.run.smithi117.stdout: "wrpos": 4194892, 2024-06-03T03:28:29.791 INFO:teuthology.orchestra.run.smithi117.stdout: "rdpos": 0, 2024-06-03T03:28:29.791 INFO:teuthology.orchestra.run.smithi117.stdout: "jlat": { 2024-06-03T03:28:29.791 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 1, 2024-06-03T03:28:29.791 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.001859229, 2024-06-03T03:28:29.791 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.001859229 2024-06-03T03:28:29.791 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:29.791 INFO:teuthology.orchestra.run.smithi117.stdout: "replayed": 0 2024-06-03T03:28:29.791 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:29.791 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_mem": { 2024-06-03T03:28:29.792 INFO:teuthology.orchestra.run.smithi117.stdout: "ino": 13, 2024-06-03T03:28:29.792 INFO:teuthology.orchestra.run.smithi117.stdout: "ino+": 13, 2024-06-03T03:28:29.792 INFO:teuthology.orchestra.run.smithi117.stdout: "ino-": 0, 2024-06-03T03:28:29.792 INFO:teuthology.orchestra.run.smithi117.stdout: "dir": 11, 2024-06-03T03:28:29.792 INFO:teuthology.orchestra.run.smithi117.stdout: "dir+": 12, 2024-06-03T03:28:29.792 INFO:teuthology.orchestra.run.smithi117.stdout: "dir-": 1, 2024-06-03T03:28:29.792 INFO:teuthology.orchestra.run.smithi117.stdout: "dn": 10, 2024-06-03T03:28:29.792 INFO:teuthology.orchestra.run.smithi117.stdout: "dn+": 10, 2024-06-03T03:28:29.792 INFO:teuthology.orchestra.run.smithi117.stdout: "dn-": 0, 2024-06-03T03:28:29.792 INFO:teuthology.orchestra.run.smithi117.stdout: "cap": 0, 2024-06-03T03:28:29.792 INFO:teuthology.orchestra.run.smithi117.stdout: "cap+": 0, 2024-06-03T03:28:29.792 INFO:teuthology.orchestra.run.smithi117.stdout: "cap-": 0, 2024-06-03T03:28:29.792 INFO:teuthology.orchestra.run.smithi117.stdout: "rss": 49660, 2024-06-03T03:28:29.792 INFO:teuthology.orchestra.run.smithi117.stdout: "heap": 223612 2024-06-03T03:28:29.793 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:29.793 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_server": { 2024-06-03T03:28:29.793 INFO:teuthology.orchestra.run.smithi117.stdout: "dispatch_client_request": 0, 2024-06-03T03:28:29.793 INFO:teuthology.orchestra.run.smithi117.stdout: "dispatch_server_request": 0, 2024-06-03T03:28:29.793 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_request": 0, 2024-06-03T03:28:29.793 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_session": 0, 2024-06-03T03:28:29.793 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_peer_request": 0, 2024-06-03T03:28:29.793 INFO:teuthology.orchestra.run.smithi117.stdout: "req_create_latency": { 2024-06-03T03:28:29.793 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:29.793 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:29.793 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:29.793 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:29.793 INFO:teuthology.orchestra.run.smithi117.stdout: "req_getattr_latency": { 2024-06-03T03:28:29.794 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:29.794 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:29.794 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:29.794 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:29.794 INFO:teuthology.orchestra.run.smithi117.stdout: "req_getfilelock_latency": { 2024-06-03T03:28:29.794 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:29.794 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:29.794 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:29.794 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:29.794 INFO:teuthology.orchestra.run.smithi117.stdout: "req_link_latency": { 2024-06-03T03:28:29.794 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:29.794 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:29.794 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:29.795 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:29.795 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookup_latency": { 2024-06-03T03:28:29.795 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:29.795 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:29.795 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:29.795 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:29.795 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookuphash_latency": { 2024-06-03T03:28:29.795 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:29.795 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:29.795 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:29.795 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:29.795 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupino_latency": { 2024-06-03T03:28:29.795 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:29.796 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:29.796 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:29.796 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:29.796 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupname_latency": { 2024-06-03T03:28:29.796 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:29.796 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:29.796 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:29.796 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:29.796 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupparent_latency": { 2024-06-03T03:28:29.796 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:29.796 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:29.796 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:29.796 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:29.796 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupsnap_latency": { 2024-06-03T03:28:29.797 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:29.797 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:29.797 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:29.797 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:29.797 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lssnap_latency": { 2024-06-03T03:28:29.797 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:29.797 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:29.797 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:29.797 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:29.797 INFO:teuthology.orchestra.run.smithi117.stdout: "req_mkdir_latency": { 2024-06-03T03:28:29.797 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:29.798 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:29.798 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:29.798 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:29.798 INFO:teuthology.orchestra.run.smithi117.stdout: "req_mknod_latency": { 2024-06-03T03:28:29.798 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:29.798 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:29.798 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:29.799 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:29.799 INFO:teuthology.orchestra.run.smithi117.stdout: "req_mksnap_latency": { 2024-06-03T03:28:29.799 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:29.799 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:29.799 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:29.799 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:29.799 INFO:teuthology.orchestra.run.smithi117.stdout: "req_open_latency": { 2024-06-03T03:28:29.799 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:29.799 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:29.799 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:29.799 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:29.799 INFO:teuthology.orchestra.run.smithi117.stdout: "req_readdir_latency": { 2024-06-03T03:28:29.799 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:29.800 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:29.800 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:29.800 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:29.800 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rename_latency": { 2024-06-03T03:28:29.800 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:29.800 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:29.800 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:29.800 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:29.800 INFO:teuthology.orchestra.run.smithi117.stdout: "req_renamesnap_latency": { 2024-06-03T03:28:29.800 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:29.800 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:29.800 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:29.800 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:29.800 INFO:teuthology.orchestra.run.smithi117.stdout: "req_snapdiff_latency": { 2024-06-03T03:28:29.801 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:29.801 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:29.801 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:29.801 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:29.801 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rmdir_latency": { 2024-06-03T03:28:29.801 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:29.801 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:29.801 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:29.801 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:29.801 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rmsnap_latency": { 2024-06-03T03:28:29.801 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:29.801 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:29.801 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:29.801 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:29.802 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rmxattr_latency": { 2024-06-03T03:28:29.802 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:29.802 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:29.802 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:29.802 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:29.802 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setattr_latency": { 2024-06-03T03:28:29.802 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:29.802 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:29.802 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:29.802 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:29.802 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setdirlayout_latency": { 2024-06-03T03:28:29.802 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:29.802 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:29.803 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:29.803 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:29.803 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setfilelock_latency": { 2024-06-03T03:28:29.803 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:29.803 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:29.803 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:29.803 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:29.803 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setlayout_latency": { 2024-06-03T03:28:29.803 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:29.803 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:29.803 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:29.803 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:29.803 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setxattr_latency": { 2024-06-03T03:28:29.804 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:29.804 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:29.804 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:29.804 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:29.804 INFO:teuthology.orchestra.run.smithi117.stdout: "req_symlink_latency": { 2024-06-03T03:28:29.804 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:29.804 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:29.804 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:29.804 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:29.804 INFO:teuthology.orchestra.run.smithi117.stdout: "req_unlink_latency": { 2024-06-03T03:28:29.804 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:29.804 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:29.804 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:29.804 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:29.805 INFO:teuthology.orchestra.run.smithi117.stdout: "cap_revoke_eviction": 0, 2024-06-03T03:28:29.805 INFO:teuthology.orchestra.run.smithi117.stdout: "cap_acquisition_throttle": 0, 2024-06-03T03:28:29.805 INFO:teuthology.orchestra.run.smithi117.stdout: "req_getvxattr_latency": { 2024-06-03T03:28:29.805 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:29.805 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:29.805 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:29.805 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:29.805 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:29.805 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_sessions": { 2024-06-03T03:28:29.805 INFO:teuthology.orchestra.run.smithi117.stdout: "session_count": 0, 2024-06-03T03:28:29.805 INFO:teuthology.orchestra.run.smithi117.stdout: "session_add": 0, 2024-06-03T03:28:29.805 INFO:teuthology.orchestra.run.smithi117.stdout: "session_remove": 0, 2024-06-03T03:28:29.805 INFO:teuthology.orchestra.run.smithi117.stdout: "sessions_open": 0, 2024-06-03T03:28:29.805 INFO:teuthology.orchestra.run.smithi117.stdout: "sessions_stale": 0, 2024-06-03T03:28:29.806 INFO:teuthology.orchestra.run.smithi117.stdout: "total_load": 0, 2024-06-03T03:28:29.806 INFO:teuthology.orchestra.run.smithi117.stdout: "average_load": 0, 2024-06-03T03:28:29.806 INFO:teuthology.orchestra.run.smithi117.stdout: "avg_session_uptime": 0, 2024-06-03T03:28:29.806 INFO:teuthology.orchestra.run.smithi117.stdout: "mdthresh_evicted": 0 2024-06-03T03:28:29.806 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:29.806 INFO:teuthology.orchestra.run.smithi117.stdout: "mempool": { 2024-06-03T03:28:29.806 INFO:teuthology.orchestra.run.smithi117.stdout: "bloom_filter_bytes": 0, 2024-06-03T03:28:29.806 INFO:teuthology.orchestra.run.smithi117.stdout: "bloom_filter_items": 0, 2024-06-03T03:28:29.806 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_alloc_bytes": 0, 2024-06-03T03:28:29.807 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_alloc_items": 0, 2024-06-03T03:28:29.807 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_data_bytes": 0, 2024-06-03T03:28:29.807 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_data_items": 0, 2024-06-03T03:28:29.807 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-03T03:28:29.807 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_onode_items": 0, 2024-06-03T03:28:29.807 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-03T03:28:29.807 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_meta_items": 0, 2024-06-03T03:28:29.807 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_other_bytes": 0, 2024-06-03T03:28:29.807 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_other_items": 0, 2024-06-03T03:28:29.807 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-03T03:28:29.807 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_buffer_items": 0, 2024-06-03T03:28:29.807 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_extent_bytes": 0, 2024-06-03T03:28:29.807 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_extent_items": 0, 2024-06-03T03:28:29.808 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_blob_bytes": 0, 2024-06-03T03:28:29.808 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_blob_items": 0, 2024-06-03T03:28:29.808 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-03T03:28:29.808 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_shared_blob_items": 0, 2024-06-03T03:28:29.808 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-03T03:28:29.808 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_inline_bl_items": 0, 2024-06-03T03:28:29.808 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_fsck_bytes": 0, 2024-06-03T03:28:29.808 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_fsck_items": 0, 2024-06-03T03:28:29.808 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_txc_bytes": 0, 2024-06-03T03:28:29.808 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_txc_items": 0, 2024-06-03T03:28:29.808 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-03T03:28:29.808 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_deferred_items": 0, 2024-06-03T03:28:29.808 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_bytes": 0, 2024-06-03T03:28:29.808 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_items": 0, 2024-06-03T03:28:29.809 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_bytes": 0, 2024-06-03T03:28:29.809 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_items": 0, 2024-06-03T03:28:29.809 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_reader_bytes": 0, 2024-06-03T03:28:29.809 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_reader_items": 0, 2024-06-03T03:28:29.809 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_writer_bytes": 0, 2024-06-03T03:28:29.809 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_writer_items": 0, 2024-06-03T03:28:29.809 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_anon_bytes": 230103, 2024-06-03T03:28:29.809 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_anon_items": 73, 2024-06-03T03:28:29.809 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_meta_bytes": 0, 2024-06-03T03:28:29.809 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_meta_items": 0, 2024-06-03T03:28:29.809 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_bytes": 0, 2024-06-03T03:28:29.809 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_items": 0, 2024-06-03T03:28:29.809 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_mapbl_bytes": 0, 2024-06-03T03:28:29.809 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_mapbl_items": 0, 2024-06-03T03:28:29.810 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_pglog_bytes": 0, 2024-06-03T03:28:29.810 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_pglog_items": 0, 2024-06-03T03:28:29.810 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_bytes": 8888, 2024-06-03T03:28:29.810 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_items": 170, 2024-06-03T03:28:29.810 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_mapping_bytes": 0, 2024-06-03T03:28:29.810 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_mapping_items": 0, 2024-06-03T03:28:29.810 INFO:teuthology.orchestra.run.smithi117.stdout: "pgmap_bytes": 0, 2024-06-03T03:28:29.810 INFO:teuthology.orchestra.run.smithi117.stdout: "pgmap_items": 0, 2024-06-03T03:28:29.810 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_co_bytes": 50664, 2024-06-03T03:28:29.810 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_co_items": 198, 2024-06-03T03:28:29.810 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_1_bytes": 0, 2024-06-03T03:28:29.810 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_1_items": 0, 2024-06-03T03:28:29.810 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_2_bytes": 0, 2024-06-03T03:28:29.810 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_2_items": 0 2024-06-03T03:28:29.811 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:29.811 INFO:teuthology.orchestra.run.smithi117.stdout: "objecter": { 2024-06-03T03:28:29.811 INFO:teuthology.orchestra.run.smithi117.stdout: "op_active": 0, 2024-06-03T03:28:29.811 INFO:teuthology.orchestra.run.smithi117.stdout: "op_laggy": 0, 2024-06-03T03:28:29.811 INFO:teuthology.orchestra.run.smithi117.stdout: "op_send": 35, 2024-06-03T03:28:29.811 INFO:teuthology.orchestra.run.smithi117.stdout: "op_send_bytes": 10710, 2024-06-03T03:28:29.811 INFO:teuthology.orchestra.run.smithi117.stdout: "op_resend": 33, 2024-06-03T03:28:29.811 INFO:teuthology.orchestra.run.smithi117.stdout: "op_reply": 35, 2024-06-03T03:28:29.811 INFO:teuthology.orchestra.run.smithi117.stdout: "op_latency": { 2024-06-03T03:28:29.811 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 35, 2024-06-03T03:28:29.811 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.268994770, 2024-06-03T03:28:29.811 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.007685564 2024-06-03T03:28:29.811 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:29.811 INFO:teuthology.orchestra.run.smithi117.stdout: "op_inflight": 0, 2024-06-03T03:28:29.812 INFO:teuthology.orchestra.run.smithi117.stdout: "oplen_avg": { 2024-06-03T03:28:29.812 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 35, 2024-06-03T03:28:29.812 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 57 2024-06-03T03:28:29.812 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:29.812 INFO:teuthology.orchestra.run.smithi117.stdout: "op": 35, 2024-06-03T03:28:29.812 INFO:teuthology.orchestra.run.smithi117.stdout: "op_r": 0, 2024-06-03T03:28:29.812 INFO:teuthology.orchestra.run.smithi117.stdout: "op_w": 35, 2024-06-03T03:28:29.812 INFO:teuthology.orchestra.run.smithi117.stdout: "op_rmw": 0, 2024-06-03T03:28:29.812 INFO:teuthology.orchestra.run.smithi117.stdout: "op_pg": 0, 2024-06-03T03:28:29.812 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_stat": 0, 2024-06-03T03:28:29.812 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_create": 10, 2024-06-03T03:28:29.812 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_read": 0, 2024-06-03T03:28:29.812 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_write": 1, 2024-06-03T03:28:29.812 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_writefull": 5, 2024-06-03T03:28:29.813 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_writesame": 0, 2024-06-03T03:28:29.813 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_append": 0, 2024-06-03T03:28:29.813 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_zero": 0, 2024-06-03T03:28:29.813 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_truncate": 0, 2024-06-03T03:28:29.813 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_delete": 6, 2024-06-03T03:28:29.813 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_mapext": 0, 2024-06-03T03:28:29.813 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_sparse_read": 0, 2024-06-03T03:28:29.813 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_clonerange": 0, 2024-06-03T03:28:29.813 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_getxattr": 0, 2024-06-03T03:28:29.813 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_setxattr": 20, 2024-06-03T03:28:29.813 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_cmpxattr": 0, 2024-06-03T03:28:29.813 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_rmxattr": 0, 2024-06-03T03:28:29.813 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_resetxattrs": 0, 2024-06-03T03:28:29.813 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_call": 0, 2024-06-03T03:28:29.814 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_watch": 0, 2024-06-03T03:28:29.814 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_notify": 0, 2024-06-03T03:28:29.814 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_src_cmpxattr": 0, 2024-06-03T03:28:29.814 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_pgls": 0, 2024-06-03T03:28:29.814 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_pgls_filter": 0, 2024-06-03T03:28:29.814 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_other": 0, 2024-06-03T03:28:29.814 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_active": 0, 2024-06-03T03:28:29.814 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_send": 0, 2024-06-03T03:28:29.814 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_resend": 0, 2024-06-03T03:28:29.814 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_ping": 0, 2024-06-03T03:28:29.814 INFO:teuthology.orchestra.run.smithi117.stdout: "poolop_active": 0, 2024-06-03T03:28:29.814 INFO:teuthology.orchestra.run.smithi117.stdout: "poolop_send": 0, 2024-06-03T03:28:29.814 INFO:teuthology.orchestra.run.smithi117.stdout: "poolop_resend": 0, 2024-06-03T03:28:29.814 INFO:teuthology.orchestra.run.smithi117.stdout: "poolstat_active": 0, 2024-06-03T03:28:29.815 INFO:teuthology.orchestra.run.smithi117.stdout: "poolstat_send": 0, 2024-06-03T03:28:29.815 INFO:teuthology.orchestra.run.smithi117.stdout: "poolstat_resend": 0, 2024-06-03T03:28:29.815 INFO:teuthology.orchestra.run.smithi117.stdout: "statfs_active": 0, 2024-06-03T03:28:29.815 INFO:teuthology.orchestra.run.smithi117.stdout: "statfs_send": 0, 2024-06-03T03:28:29.815 INFO:teuthology.orchestra.run.smithi117.stdout: "statfs_resend": 0, 2024-06-03T03:28:29.815 INFO:teuthology.orchestra.run.smithi117.stdout: "command_active": 0, 2024-06-03T03:28:29.815 INFO:teuthology.orchestra.run.smithi117.stdout: "command_send": 0, 2024-06-03T03:28:29.815 INFO:teuthology.orchestra.run.smithi117.stdout: "command_resend": 0, 2024-06-03T03:28:29.815 INFO:teuthology.orchestra.run.smithi117.stdout: "map_epoch": 99, 2024-06-03T03:28:29.815 INFO:teuthology.orchestra.run.smithi117.stdout: "map_full": 0, 2024-06-03T03:28:29.815 INFO:teuthology.orchestra.run.smithi117.stdout: "map_inc": 2, 2024-06-03T03:28:29.815 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_sessions": 8, 2024-06-03T03:28:29.816 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_session_open": 8, 2024-06-03T03:28:29.816 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_session_close": 0, 2024-06-03T03:28:29.816 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_laggy": 0, 2024-06-03T03:28:29.816 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_wr": 14, 2024-06-03T03:28:29.816 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_rd": 0, 2024-06-03T03:28:29.816 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_del": 1 2024-06-03T03:28:29.816 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:29.816 INFO:teuthology.orchestra.run.smithi117.stdout: "oft": { 2024-06-03T03:28:29.816 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_objs": 1, 2024-06-03T03:28:29.816 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_kv_pairs": 0, 2024-06-03T03:28:29.816 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_updates": 0, 2024-06-03T03:28:29.816 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_removes": 0 2024-06-03T03:28:29.816 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:29.817 INFO:teuthology.orchestra.run.smithi117.stdout: "osd-slow-ops": { 2024-06-03T03:28:29.817 INFO:teuthology.orchestra.run.smithi117.stdout: "slow_ops_count": 0 2024-06-03T03:28:29.817 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:29.817 INFO:teuthology.orchestra.run.smithi117.stdout: "purge_queue": { 2024-06-03T03:28:29.817 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing_ops": 0, 2024-06-03T03:28:29.817 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing_ops_high_water": 0, 2024-06-03T03:28:29.817 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing": 0, 2024-06-03T03:28:29.817 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing_high_water": 0, 2024-06-03T03:28:29.817 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executed_ops": 0, 2024-06-03T03:28:29.817 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executed": 0, 2024-06-03T03:28:29.817 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_item_in_journal": 0 2024-06-03T03:28:29.817 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:29.817 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-03T03:28:29.817 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-06-03T03:28:29.817 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 104857600, 2024-06-03T03:28:29.818 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-06-03T03:28:29.818 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 403, 2024-06-03T03:28:29.818 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 798219, 2024-06-03T03:28:29.818 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-06-03T03:28:29.818 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 403, 2024-06-03T03:28:29.818 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2024-06-03T03:28:29.818 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2024-06-03T03:28:29.818 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 403, 2024-06-03T03:28:29.818 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 798219, 2024-06-03T03:28:29.818 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-06-03T03:28:29.818 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:29.818 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:29.818 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:29.819 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:29.819 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:29.819 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-objecter_bytes": { 2024-06-03T03:28:29.819 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-06-03T03:28:29.819 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 104857600, 2024-06-03T03:28:29.819 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-06-03T03:28:29.819 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 0, 2024-06-03T03:28:29.819 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 0, 2024-06-03T03:28:29.819 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-06-03T03:28:29.819 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 0, 2024-06-03T03:28:29.819 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 35, 2024-06-03T03:28:29.819 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 10710, 2024-06-03T03:28:29.819 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 35, 2024-06-03T03:28:29.819 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 10710, 2024-06-03T03:28:29.820 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-06-03T03:28:29.820 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:29.820 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:29.820 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:29.820 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:29.820 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:29.820 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-objecter_ops": { 2024-06-03T03:28:29.820 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-06-03T03:28:29.820 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 1024, 2024-06-03T03:28:29.820 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-06-03T03:28:29.820 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 0, 2024-06-03T03:28:29.820 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 0, 2024-06-03T03:28:29.820 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-06-03T03:28:29.820 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 0, 2024-06-03T03:28:29.821 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 35, 2024-06-03T03:28:29.821 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 35, 2024-06-03T03:28:29.821 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 35, 2024-06-03T03:28:29.821 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 35, 2024-06-03T03:28:29.821 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-06-03T03:28:29.821 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:29.821 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:29.821 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:29.821 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:29.821 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:29.821 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-write_buf_throttle": { 2024-06-03T03:28:29.821 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-06-03T03:28:29.821 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 3758096384, 2024-06-03T03:28:29.821 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-06-03T03:28:29.822 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 0, 2024-06-03T03:28:29.822 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 0, 2024-06-03T03:28:29.822 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-06-03T03:28:29.822 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 0, 2024-06-03T03:28:29.822 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2024-06-03T03:28:29.822 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2024-06-03T03:28:29.822 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 0, 2024-06-03T03:28:29.822 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 0, 2024-06-03T03:28:29.822 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-06-03T03:28:29.822 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:29.822 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:29.822 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:29.822 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:29.822 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:29.823 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-write_buf_throttle-0x55a1eece2500": { 2024-06-03T03:28:29.823 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-06-03T03:28:29.823 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 3758096384, 2024-06-03T03:28:29.823 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-06-03T03:28:29.823 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 2, 2024-06-03T03:28:29.823 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 588, 2024-06-03T03:28:29.823 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-06-03T03:28:29.823 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 2, 2024-06-03T03:28:29.823 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2024-06-03T03:28:29.823 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2024-06-03T03:28:29.823 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 1, 2024-06-03T03:28:29.823 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 588, 2024-06-03T03:28:29.823 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-06-03T03:28:29.823 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:29.823 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:29.824 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:29.824 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:29.824 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:29.824 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-06-03T03:28:30.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:28:30 smithi151 ceph-mon[37043]: pgmap v601: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 170 B/s wr, 11 op/s 2024-06-03T03:28:30.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:28:30 smithi117 ceph-mon[31649]: pgmap v601: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 170 B/s wr, 11 op/s 2024-06-03T03:28:30.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:28:30 smithi183 ceph-mon[37326]: pgmap v601: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 170 B/s wr, 11 op/s 2024-06-03T03:28:30.881 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-06-03T03:28:30.881 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-06-03T03:28:30.881 DEBUG:tasks.check_counter:Getting stats from d 2024-06-03T03:28:30.882 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph daemon mds.d perf dump 2024-06-03T03:28:31.318 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:28:32.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:28:32 smithi117 ceph-mon[31649]: pgmap v602: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:28:32.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:28:32 smithi151 ceph-mon[37043]: pgmap v602: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:28:32.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:28:32 smithi183 ceph-mon[37326]: pgmap v602: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:28:33.910 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-06-03T03:28:33.911 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-0": { 2024-06-03T03:28:33.911 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 166, 2024-06-03T03:28:33.911 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 159, 2024-06-03T03:28:33.911 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 740480, 2024-06-03T03:28:33.911 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 96496, 2024-06-03T03:28:33.911 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 4, 2024-06-03T03:28:33.912 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 0, 2024-06-03T03:28:33.912 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.060794922, 2024-06-03T03:28:33.912 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.023897091, 2024-06-03T03:28:33.912 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.032571861, 2024-06-03T03:28:33.912 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.006734424, 2024-06-03T03:28:33.912 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T03:28:33.912 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 159, 2024-06-03T03:28:33.912 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.015594548, 2024-06-03T03:28:33.912 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000098078 2024-06-03T03:28:33.912 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:33.913 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-06-03T03:28:33.913 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:33.913 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:33.913 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:33.913 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:33.913 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 740480, 2024-06-03T03:28:33.913 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 96496 2024-06-03T03:28:33.913 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:33.913 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-1": { 2024-06-03T03:28:33.913 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 1, 2024-06-03T03:28:33.913 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 119, 2024-06-03T03:28:33.913 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 160, 2024-06-03T03:28:33.914 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 98768, 2024-06-03T03:28:33.914 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 3, 2024-06-03T03:28:33.914 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 2, 2024-06-03T03:28:33.914 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.021300336, 2024-06-03T03:28:33.914 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.015911203, 2024-06-03T03:28:33.914 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.002884948, 2024-06-03T03:28:33.914 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-03T03:28:33.914 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T03:28:33.914 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 117, 2024-06-03T03:28:33.914 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.010137814, 2024-06-03T03:28:33.914 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000086647 2024-06-03T03:28:33.914 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:33.915 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-06-03T03:28:33.915 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:33.915 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:33.915 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:33.915 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:33.915 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 160, 2024-06-03T03:28:33.915 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 98768 2024-06-03T03:28:33.915 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:33.915 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-2": { 2024-06-03T03:28:33.915 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 0, 2024-06-03T03:28:33.915 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 2, 2024-06-03T03:28:33.915 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 0, 2024-06-03T03:28:33.915 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 0, 2024-06-03T03:28:33.915 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 2, 2024-06-03T03:28:33.915 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 1, 2024-06-03T03:28:33.916 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.000979442, 2024-06-03T03:28:33.916 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.000000000, 2024-06-03T03:28:33.916 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.000835023, 2024-06-03T03:28:33.916 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-03T03:28:33.916 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T03:28:33.916 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:33.916 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:33.916 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:33.916 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:33.916 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-06-03T03:28:33.916 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:33.916 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:33.916 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:33.916 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:33.917 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-03T03:28:33.917 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 0 2024-06-03T03:28:33.917 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:33.917 INFO:teuthology.orchestra.run.smithi117.stdout: "cct": { 2024-06-03T03:28:33.917 INFO:teuthology.orchestra.run.smithi117.stdout: "total_workers": 0, 2024-06-03T03:28:33.917 INFO:teuthology.orchestra.run.smithi117.stdout: "unhealthy_workers": 0 2024-06-03T03:28:33.917 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:33.917 INFO:teuthology.orchestra.run.smithi117.stdout: "mempool": { 2024-06-03T03:28:33.917 INFO:teuthology.orchestra.run.smithi117.stdout: "bloom_filter_bytes": 0, 2024-06-03T03:28:33.917 INFO:teuthology.orchestra.run.smithi117.stdout: "bloom_filter_items": 0, 2024-06-03T03:28:33.917 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_alloc_bytes": 0, 2024-06-03T03:28:33.917 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_alloc_items": 0, 2024-06-03T03:28:33.917 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_data_bytes": 0, 2024-06-03T03:28:33.918 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_data_items": 0, 2024-06-03T03:28:33.918 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-03T03:28:33.918 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_onode_items": 0, 2024-06-03T03:28:33.918 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-03T03:28:33.918 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_meta_items": 0, 2024-06-03T03:28:33.918 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_other_bytes": 0, 2024-06-03T03:28:33.918 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_other_items": 0, 2024-06-03T03:28:33.918 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-03T03:28:33.918 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_buffer_items": 0, 2024-06-03T03:28:33.918 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_extent_bytes": 0, 2024-06-03T03:28:33.918 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_extent_items": 0, 2024-06-03T03:28:33.918 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_blob_bytes": 0, 2024-06-03T03:28:33.918 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_blob_items": 0, 2024-06-03T03:28:33.919 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-03T03:28:33.919 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_shared_blob_items": 0, 2024-06-03T03:28:33.919 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-03T03:28:33.919 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_inline_bl_items": 0, 2024-06-03T03:28:33.919 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_fsck_bytes": 0, 2024-06-03T03:28:33.919 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_fsck_items": 0, 2024-06-03T03:28:33.919 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_txc_bytes": 0, 2024-06-03T03:28:33.919 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_txc_items": 0, 2024-06-03T03:28:33.919 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-03T03:28:33.919 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_deferred_items": 0, 2024-06-03T03:28:33.919 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_bytes": 0, 2024-06-03T03:28:33.919 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_items": 0, 2024-06-03T03:28:33.919 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_bytes": 0, 2024-06-03T03:28:33.919 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_items": 0, 2024-06-03T03:28:33.919 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_reader_bytes": 0, 2024-06-03T03:28:33.920 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_reader_items": 0, 2024-06-03T03:28:33.920 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_writer_bytes": 0, 2024-06-03T03:28:33.920 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_writer_items": 0, 2024-06-03T03:28:33.920 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_anon_bytes": 148984, 2024-06-03T03:28:33.920 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_anon_items": 30, 2024-06-03T03:28:33.920 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_meta_bytes": 0, 2024-06-03T03:28:33.920 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_meta_items": 0, 2024-06-03T03:28:33.920 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_bytes": 0, 2024-06-03T03:28:33.920 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_items": 0, 2024-06-03T03:28:33.920 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_mapbl_bytes": 0, 2024-06-03T03:28:33.920 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_mapbl_items": 0, 2024-06-03T03:28:33.920 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_pglog_bytes": 0, 2024-06-03T03:28:33.920 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_pglog_items": 0, 2024-06-03T03:28:33.920 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_bytes": 0, 2024-06-03T03:28:33.921 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_items": 0, 2024-06-03T03:28:33.921 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_mapping_bytes": 0, 2024-06-03T03:28:33.921 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_mapping_items": 0, 2024-06-03T03:28:33.921 INFO:teuthology.orchestra.run.smithi117.stdout: "pgmap_bytes": 0, 2024-06-03T03:28:33.921 INFO:teuthology.orchestra.run.smithi117.stdout: "pgmap_items": 0, 2024-06-03T03:28:33.921 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_co_bytes": 928, 2024-06-03T03:28:33.921 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_co_items": 2, 2024-06-03T03:28:33.921 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_1_bytes": 0, 2024-06-03T03:28:33.921 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_1_items": 0, 2024-06-03T03:28:33.922 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_2_bytes": 0, 2024-06-03T03:28:33.922 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_2_items": 0 2024-06-03T03:28:33.922 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:33.922 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-03T03:28:33.922 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-06-03T03:28:33.922 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 104857600, 2024-06-03T03:28:33.922 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-06-03T03:28:33.922 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 163, 2024-06-03T03:28:33.922 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 714622, 2024-06-03T03:28:33.922 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-06-03T03:28:33.922 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 163, 2024-06-03T03:28:33.922 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2024-06-03T03:28:33.922 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2024-06-03T03:28:33.922 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 163, 2024-06-03T03:28:33.923 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 714622, 2024-06-03T03:28:33.923 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-06-03T03:28:33.923 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:33.923 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:33.923 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:33.923 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:33.923 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:33.923 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-06-03T03:28:34.379 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:28:34 smithi117 ceph-mon[31649]: pgmap v603: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-06-03T03:28:34.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:28:34 smithi151 ceph-mon[37043]: pgmap v603: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-06-03T03:28:34.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:28:34 smithi183 ceph-mon[37326]: pgmap v603: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-06-03T03:28:34.968 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-06-03T03:28:34.969 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-06-03T03:28:34.969 DEBUG:tasks.check_counter:Getting stats from g 2024-06-03T03:28:34.969 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph daemon mds.g perf dump 2024-06-03T03:28:35.415 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:28:35.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:28:35 smithi117 ceph-mon[31649]: pgmap v604: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:28:35.981 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:28:35 smithi183 ceph-mon[37326]: pgmap v604: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:28:36.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:28:35 smithi151 ceph-mon[37043]: pgmap v604: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:28:37.393 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:28:37 smithi117 ceph-mon[31649]: pgmap v605: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-03T03:28:37.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:28:37 smithi151 ceph-mon[37043]: pgmap v605: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-03T03:28:37.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:28:37 smithi183 ceph-mon[37326]: pgmap v605: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-03T03:28:38.020 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-06-03T03:28:38.020 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-0": { 2024-06-03T03:28:38.020 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 0, 2024-06-03T03:28:38.020 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 2, 2024-06-03T03:28:38.020 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 0, 2024-06-03T03:28:38.020 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 0, 2024-06-03T03:28:38.020 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 4, 2024-06-03T03:28:38.020 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 0, 2024-06-03T03:28:38.020 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.003781202, 2024-06-03T03:28:38.021 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.000000000, 2024-06-03T03:28:38.021 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.000669244, 2024-06-03T03:28:38.021 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-03T03:28:38.021 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T03:28:38.021 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:38.021 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:38.021 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:38.021 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:38.021 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-06-03T03:28:38.021 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:38.021 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:38.021 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:38.021 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:38.021 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-03T03:28:38.022 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 0 2024-06-03T03:28:38.022 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:38.022 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-1": { 2024-06-03T03:28:38.022 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 158, 2024-06-03T03:28:38.022 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 153, 2024-06-03T03:28:38.022 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 736368, 2024-06-03T03:28:38.022 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 93488, 2024-06-03T03:28:38.022 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 2, 2024-06-03T03:28:38.022 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 1, 2024-06-03T03:28:38.022 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.055513238, 2024-06-03T03:28:38.022 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.023402216, 2024-06-03T03:28:38.022 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.029248775, 2024-06-03T03:28:38.022 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.006610075, 2024-06-03T03:28:38.023 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T03:28:38.023 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 152, 2024-06-03T03:28:38.023 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.014382619, 2024-06-03T03:28:38.023 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000094622 2024-06-03T03:28:38.023 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:38.023 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-06-03T03:28:38.023 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:38.023 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:38.023 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:38.023 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:38.023 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 736368, 2024-06-03T03:28:38.023 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 93488 2024-06-03T03:28:38.023 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:38.024 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-2": { 2024-06-03T03:28:38.024 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 5, 2024-06-03T03:28:38.024 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 118, 2024-06-03T03:28:38.024 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 3184, 2024-06-03T03:28:38.024 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 98688, 2024-06-03T03:28:38.024 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 3, 2024-06-03T03:28:38.024 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 2, 2024-06-03T03:28:38.024 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.021238311, 2024-06-03T03:28:38.024 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.016565983, 2024-06-03T03:28:38.024 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.002266523, 2024-06-03T03:28:38.024 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-03T03:28:38.024 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T03:28:38.024 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 117, 2024-06-03T03:28:38.024 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.011154987, 2024-06-03T03:28:38.025 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000095341 2024-06-03T03:28:38.025 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:38.025 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-06-03T03:28:38.025 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:38.025 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:38.025 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:38.025 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:38.025 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 3184, 2024-06-03T03:28:38.025 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 98688 2024-06-03T03:28:38.025 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:38.025 INFO:teuthology.orchestra.run.smithi117.stdout: "cct": { 2024-06-03T03:28:38.025 INFO:teuthology.orchestra.run.smithi117.stdout: "total_workers": 0, 2024-06-03T03:28:38.025 INFO:teuthology.orchestra.run.smithi117.stdout: "unhealthy_workers": 0 2024-06-03T03:28:38.025 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:38.026 INFO:teuthology.orchestra.run.smithi117.stdout: "mempool": { 2024-06-03T03:28:38.026 INFO:teuthology.orchestra.run.smithi117.stdout: "bloom_filter_bytes": 0, 2024-06-03T03:28:38.026 INFO:teuthology.orchestra.run.smithi117.stdout: "bloom_filter_items": 0, 2024-06-03T03:28:38.026 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_alloc_bytes": 0, 2024-06-03T03:28:38.026 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_alloc_items": 0, 2024-06-03T03:28:38.026 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_data_bytes": 0, 2024-06-03T03:28:38.026 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_data_items": 0, 2024-06-03T03:28:38.026 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-03T03:28:38.026 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_onode_items": 0, 2024-06-03T03:28:38.026 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-03T03:28:38.026 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_meta_items": 0, 2024-06-03T03:28:38.026 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_other_bytes": 0, 2024-06-03T03:28:38.026 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_other_items": 0, 2024-06-03T03:28:38.027 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-03T03:28:38.027 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_buffer_items": 0, 2024-06-03T03:28:38.027 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_extent_bytes": 0, 2024-06-03T03:28:38.027 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_extent_items": 0, 2024-06-03T03:28:38.027 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_blob_bytes": 0, 2024-06-03T03:28:38.027 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_blob_items": 0, 2024-06-03T03:28:38.027 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-03T03:28:38.027 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_shared_blob_items": 0, 2024-06-03T03:28:38.027 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-03T03:28:38.027 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_inline_bl_items": 0, 2024-06-03T03:28:38.027 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_fsck_bytes": 0, 2024-06-03T03:28:38.027 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_fsck_items": 0, 2024-06-03T03:28:38.027 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_txc_bytes": 0, 2024-06-03T03:28:38.028 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_txc_items": 0, 2024-06-03T03:28:38.028 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-03T03:28:38.028 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_deferred_items": 0, 2024-06-03T03:28:38.028 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_bytes": 0, 2024-06-03T03:28:38.028 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_items": 0, 2024-06-03T03:28:38.028 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_bytes": 0, 2024-06-03T03:28:38.028 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_items": 0, 2024-06-03T03:28:38.028 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_reader_bytes": 0, 2024-06-03T03:28:38.028 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_reader_items": 0, 2024-06-03T03:28:38.028 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_writer_bytes": 0, 2024-06-03T03:28:38.028 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_writer_items": 0, 2024-06-03T03:28:38.028 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_anon_bytes": 149998, 2024-06-03T03:28:38.028 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_anon_items": 30, 2024-06-03T03:28:38.028 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_meta_bytes": 0, 2024-06-03T03:28:38.029 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_meta_items": 0, 2024-06-03T03:28:38.029 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_bytes": 0, 2024-06-03T03:28:38.029 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_items": 0, 2024-06-03T03:28:38.029 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_mapbl_bytes": 0, 2024-06-03T03:28:38.029 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_mapbl_items": 0, 2024-06-03T03:28:38.029 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_pglog_bytes": 0, 2024-06-03T03:28:38.029 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_pglog_items": 0, 2024-06-03T03:28:38.029 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_bytes": 0, 2024-06-03T03:28:38.029 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_items": 0, 2024-06-03T03:28:38.029 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_mapping_bytes": 0, 2024-06-03T03:28:38.029 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_mapping_items": 0, 2024-06-03T03:28:38.029 INFO:teuthology.orchestra.run.smithi117.stdout: "pgmap_bytes": 0, 2024-06-03T03:28:38.029 INFO:teuthology.orchestra.run.smithi117.stdout: "pgmap_items": 0, 2024-06-03T03:28:38.029 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_co_bytes": 928, 2024-06-03T03:28:38.030 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_co_items": 2, 2024-06-03T03:28:38.030 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_1_bytes": 0, 2024-06-03T03:28:38.030 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_1_items": 0, 2024-06-03T03:28:38.030 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_2_bytes": 0, 2024-06-03T03:28:38.030 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_2_items": 0 2024-06-03T03:28:38.030 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:38.030 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-03T03:28:38.030 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-06-03T03:28:38.030 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 104857600, 2024-06-03T03:28:38.030 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-06-03T03:28:38.030 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 159, 2024-06-03T03:28:38.030 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 714102, 2024-06-03T03:28:38.030 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-06-03T03:28:38.031 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 159, 2024-06-03T03:28:38.031 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2024-06-03T03:28:38.031 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2024-06-03T03:28:38.031 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 159, 2024-06-03T03:28:38.031 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 714102, 2024-06-03T03:28:38.031 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-06-03T03:28:38.031 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:38.031 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:38.031 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:38.031 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:38.031 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:38.031 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-06-03T03:28:39.122 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-06-03T03:28:39.123 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-06-03T03:28:39.123 DEBUG:tasks.check_counter:Getting stats from j 2024-06-03T03:28:39.123 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph daemon mds.j perf dump 2024-06-03T03:28:39.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:28:39 smithi151 ceph-mon[37043]: pgmap v606: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-03T03:28:39.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:28:39 smithi117 ceph-mon[31649]: pgmap v606: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-03T03:28:39.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:28:39 smithi183 ceph-mon[37326]: pgmap v606: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-03T03:28:39.566 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:28:41.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:28:41 smithi151 ceph-mon[37043]: pgmap v607: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:28:41.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:28:41 smithi117 ceph-mon[31649]: pgmap v607: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:28:41.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:28:41 smithi183 ceph-mon[37326]: pgmap v607: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:28:42.162 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-06-03T03:28:42.162 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-0": { 2024-06-03T03:28:42.162 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 1263, 2024-06-03T03:28:42.162 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 1265, 2024-06-03T03:28:42.162 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 361401, 2024-06-03T03:28:42.163 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 401641, 2024-06-03T03:28:42.163 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 6, 2024-06-03T03:28:42.163 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 2, 2024-06-03T03:28:42.163 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.200433015, 2024-06-03T03:28:42.163 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.066374192, 2024-06-03T03:28:42.163 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.052532722, 2024-06-03T03:28:42.163 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.061763862, 2024-06-03T03:28:42.163 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T03:28:42.163 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 1263, 2024-06-03T03:28:42.163 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.084160253, 2024-06-03T03:28:42.163 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000066635 2024-06-03T03:28:42.163 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:42.163 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-06-03T03:28:42.164 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:42.164 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:42.164 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:42.164 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:42.164 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-03T03:28:42.164 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 0 2024-06-03T03:28:42.164 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:42.164 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-1": { 2024-06-03T03:28:42.164 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 180, 2024-06-03T03:28:42.164 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 171, 2024-06-03T03:28:42.164 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 772684, 2024-06-03T03:28:42.164 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 100298, 2024-06-03T03:28:42.164 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 3, 2024-06-03T03:28:42.165 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 2, 2024-06-03T03:28:42.165 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.067829620, 2024-06-03T03:28:42.165 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.018533262, 2024-06-03T03:28:42.165 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.060387151, 2024-06-03T03:28:42.165 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.006859017, 2024-06-03T03:28:42.165 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T03:28:42.165 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 171, 2024-06-03T03:28:42.165 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.018746298, 2024-06-03T03:28:42.165 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000109627 2024-06-03T03:28:42.165 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:42.165 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-06-03T03:28:42.165 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:42.166 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:42.166 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:42.166 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:42.166 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 772128, 2024-06-03T03:28:42.166 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 99600 2024-06-03T03:28:42.166 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:42.166 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-2": { 2024-06-03T03:28:42.166 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 2, 2024-06-03T03:28:42.166 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 122, 2024-06-03T03:28:42.166 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 428, 2024-06-03T03:28:42.166 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 254158, 2024-06-03T03:28:42.166 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 4, 2024-06-03T03:28:42.167 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 3, 2024-06-03T03:28:42.167 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.023013303, 2024-06-03T03:28:42.167 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.016986594, 2024-06-03T03:28:42.167 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.003305619, 2024-06-03T03:28:42.167 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.000029430, 2024-06-03T03:28:42.167 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T03:28:42.167 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 120, 2024-06-03T03:28:42.167 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.011664320, 2024-06-03T03:28:42.167 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000097202 2024-06-03T03:28:42.167 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:42.167 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-06-03T03:28:42.167 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:42.167 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:42.168 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:42.168 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:42.168 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 160, 2024-06-03T03:28:42.168 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 253840 2024-06-03T03:28:42.168 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:42.168 INFO:teuthology.orchestra.run.smithi117.stdout: "cct": { 2024-06-03T03:28:42.168 INFO:teuthology.orchestra.run.smithi117.stdout: "total_workers": 1, 2024-06-03T03:28:42.168 INFO:teuthology.orchestra.run.smithi117.stdout: "unhealthy_workers": 0 2024-06-03T03:28:42.168 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:42.168 INFO:teuthology.orchestra.run.smithi117.stdout: "finisher-MDSRank": { 2024-06-03T03:28:42.168 INFO:teuthology.orchestra.run.smithi117.stdout: "queue_len": 0, 2024-06-03T03:28:42.168 INFO:teuthology.orchestra.run.smithi117.stdout: "complete_latency": { 2024-06-03T03:28:42.168 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 2152, 2024-06-03T03:28:42.169 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.098675237, 2024-06-03T03:28:42.169 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000045852 2024-06-03T03:28:42.169 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:42.169 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:42.169 INFO:teuthology.orchestra.run.smithi117.stdout: "finisher-PurgeQueue": { 2024-06-03T03:28:42.169 INFO:teuthology.orchestra.run.smithi117.stdout: "queue_len": 0, 2024-06-03T03:28:42.169 INFO:teuthology.orchestra.run.smithi117.stdout: "complete_latency": { 2024-06-03T03:28:42.169 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:42.169 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:42.169 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:42.169 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:42.169 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:42.169 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-06-03T03:28:42.169 INFO:teuthology.orchestra.run.smithi117.stdout: "request": 0, 2024-06-03T03:28:42.170 INFO:teuthology.orchestra.run.smithi117.stdout: "reply": 0, 2024-06-03T03:28:42.170 INFO:teuthology.orchestra.run.smithi117.stdout: "reply_latency": { 2024-06-03T03:28:42.170 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:42.170 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:42.170 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:42.170 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:42.170 INFO:teuthology.orchestra.run.smithi117.stdout: "slow_reply": 0, 2024-06-03T03:28:42.170 INFO:teuthology.orchestra.run.smithi117.stdout: "forward": 0, 2024-06-03T03:28:42.170 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_fetch_complete": 0, 2024-06-03T03:28:42.170 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_fetch_keys": 0, 2024-06-03T03:28:42.170 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_commit": 0, 2024-06-03T03:28:42.170 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_split": 0, 2024-06-03T03:28:42.170 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_merge": 0, 2024-06-03T03:28:42.171 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes": 0, 2024-06-03T03:28:42.171 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_top": 0, 2024-06-03T03:28:42.171 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_bottom": 0, 2024-06-03T03:28:42.171 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_pin_tail": 0, 2024-06-03T03:28:42.171 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_pinned": 0, 2024-06-03T03:28:42.171 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_expired": 0, 2024-06-03T03:28:42.171 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_with_caps": 0, 2024-06-03T03:28:42.171 INFO:teuthology.orchestra.run.smithi117.stdout: "caps": 0, 2024-06-03T03:28:42.171 INFO:teuthology.orchestra.run.smithi117.stdout: "subtrees": 1, 2024-06-03T03:28:42.171 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse": 0, 2024-06-03T03:28:42.171 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_hit": 0, 2024-06-03T03:28:42.171 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_forward": 0, 2024-06-03T03:28:42.171 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_discover": 0, 2024-06-03T03:28:42.171 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_dir_fetch": 0, 2024-06-03T03:28:42.172 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_remote_ino": 0, 2024-06-03T03:28:42.172 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_lock": 0, 2024-06-03T03:28:42.172 INFO:teuthology.orchestra.run.smithi117.stdout: "load_cent": 0, 2024-06-03T03:28:42.172 INFO:teuthology.orchestra.run.smithi117.stdout: "q": 0, 2024-06-03T03:28:42.172 INFO:teuthology.orchestra.run.smithi117.stdout: "exported": 0, 2024-06-03T03:28:42.172 INFO:teuthology.orchestra.run.smithi117.stdout: "exported_inodes": 0, 2024-06-03T03:28:42.172 INFO:teuthology.orchestra.run.smithi117.stdout: "imported": 0, 2024-06-03T03:28:42.172 INFO:teuthology.orchestra.run.smithi117.stdout: "imported_inodes": 0, 2024-06-03T03:28:42.172 INFO:teuthology.orchestra.run.smithi117.stdout: "openino_dir_fetch": 0, 2024-06-03T03:28:42.172 INFO:teuthology.orchestra.run.smithi117.stdout: "openino_backtrace_fetch": 0, 2024-06-03T03:28:42.172 INFO:teuthology.orchestra.run.smithi117.stdout: "openino_peer_discover": 0, 2024-06-03T03:28:42.172 INFO:teuthology.orchestra.run.smithi117.stdout: "root_rfiles": 0, 2024-06-03T03:28:42.172 INFO:teuthology.orchestra.run.smithi117.stdout: "root_rbytes": 0, 2024-06-03T03:28:42.173 INFO:teuthology.orchestra.run.smithi117.stdout: "root_rsnaps": 0, 2024-06-03T03:28:42.173 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_backtrace_fetch": 0, 2024-06-03T03:28:42.173 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_set_tag": 0, 2024-06-03T03:28:42.173 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_backtrace_repaired": 0, 2024-06-03T03:28:42.173 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_inotable_repaired": 0, 2024-06-03T03:28:42.173 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_dir_inodes": 0, 2024-06-03T03:28:42.173 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_dir_base_inodes": 0, 2024-06-03T03:28:42.173 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_dirfrag_rstats": 0, 2024-06-03T03:28:42.173 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_file_inodes": 0, 2024-06-03T03:28:42.173 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_inode_file_caps": 0, 2024-06-03T03:28:42.173 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_revoke": 0, 2024-06-03T03:28:42.173 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_grant": 0, 2024-06-03T03:28:42.173 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_trunc": 0, 2024-06-03T03:28:42.173 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-03T03:28:42.174 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-03T03:28:42.174 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_caps": 0, 2024-06-03T03:28:42.174 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_caps_dirty": 0, 2024-06-03T03:28:42.174 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_cap_release": 0, 2024-06-03T03:28:42.174 INFO:teuthology.orchestra.run.smithi117.stdout: "process_request_cap_release": 0 2024-06-03T03:28:42.174 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:42.174 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_cache": { 2024-06-03T03:28:42.174 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_update": 0, 2024-06-03T03:28:42.174 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_update_receipt": 0, 2024-06-03T03:28:42.174 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_try_discover": 0, 2024-06-03T03:28:42.174 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_send_discover": 0, 2024-06-03T03:28:42.174 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_handle_discover": 0, 2024-06-03T03:28:42.174 INFO:teuthology.orchestra.run.smithi117.stdout: "num_strays": 0, 2024-06-03T03:28:42.175 INFO:teuthology.orchestra.run.smithi117.stdout: "num_strays_delayed": 0, 2024-06-03T03:28:42.175 INFO:teuthology.orchestra.run.smithi117.stdout: "num_strays_enqueuing": 0, 2024-06-03T03:28:42.175 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_created": 0, 2024-06-03T03:28:42.175 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_enqueued": 0, 2024-06-03T03:28:42.175 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_reintegrated": 0, 2024-06-03T03:28:42.175 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_migrated": 0, 2024-06-03T03:28:42.175 INFO:teuthology.orchestra.run.smithi117.stdout: "num_recovering_processing": 0, 2024-06-03T03:28:42.175 INFO:teuthology.orchestra.run.smithi117.stdout: "num_recovering_enqueued": 0, 2024-06-03T03:28:42.175 INFO:teuthology.orchestra.run.smithi117.stdout: "num_recovering_prioritized": 0, 2024-06-03T03:28:42.175 INFO:teuthology.orchestra.run.smithi117.stdout: "recovery_started": 0, 2024-06-03T03:28:42.175 INFO:teuthology.orchestra.run.smithi117.stdout: "recovery_completed": 0, 2024-06-03T03:28:42.175 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_quiesce_path": 0, 2024-06-03T03:28:42.175 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_quiesce_inode": 0, 2024-06-03T03:28:42.175 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_enqueue_scrub": 0, 2024-06-03T03:28:42.176 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_exportdir": 0, 2024-06-03T03:28:42.176 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_flush": 0, 2024-06-03T03:28:42.176 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_fragmentdir": 0, 2024-06-03T03:28:42.176 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_fragstats": 0, 2024-06-03T03:28:42.176 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_inodestats": 0 2024-06-03T03:28:42.176 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:42.176 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_log": { 2024-06-03T03:28:42.176 INFO:teuthology.orchestra.run.smithi117.stdout: "evlrg": 0, 2024-06-03T03:28:42.176 INFO:teuthology.orchestra.run.smithi117.stdout: "evadd": 0, 2024-06-03T03:28:42.176 INFO:teuthology.orchestra.run.smithi117.stdout: "evex": 0, 2024-06-03T03:28:42.176 INFO:teuthology.orchestra.run.smithi117.stdout: "evtrm": 0, 2024-06-03T03:28:42.176 INFO:teuthology.orchestra.run.smithi117.stdout: "ev": 7, 2024-06-03T03:28:42.176 INFO:teuthology.orchestra.run.smithi117.stdout: "evexg": 0, 2024-06-03T03:28:42.176 INFO:teuthology.orchestra.run.smithi117.stdout: "evexd": 0, 2024-06-03T03:28:42.176 INFO:teuthology.orchestra.run.smithi117.stdout: "segadd": 0, 2024-06-03T03:28:42.177 INFO:teuthology.orchestra.run.smithi117.stdout: "segex": 0, 2024-06-03T03:28:42.177 INFO:teuthology.orchestra.run.smithi117.stdout: "segtrm": 0, 2024-06-03T03:28:42.177 INFO:teuthology.orchestra.run.smithi117.stdout: "seg": 2, 2024-06-03T03:28:42.177 INFO:teuthology.orchestra.run.smithi117.stdout: "segmjr": 2, 2024-06-03T03:28:42.177 INFO:teuthology.orchestra.run.smithi117.stdout: "segexg": 0, 2024-06-03T03:28:42.177 INFO:teuthology.orchestra.run.smithi117.stdout: "segexd": 0, 2024-06-03T03:28:42.177 INFO:teuthology.orchestra.run.smithi117.stdout: "expos": 4194304, 2024-06-03T03:28:42.177 INFO:teuthology.orchestra.run.smithi117.stdout: "wrpos": 4196260, 2024-06-03T03:28:42.177 INFO:teuthology.orchestra.run.smithi117.stdout: "rdpos": 4196119, 2024-06-03T03:28:42.177 INFO:teuthology.orchestra.run.smithi117.stdout: "jlat": { 2024-06-03T03:28:42.177 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:42.177 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:42.177 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:42.178 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:42.178 INFO:teuthology.orchestra.run.smithi117.stdout: "replayed": 7 2024-06-03T03:28:42.178 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:42.178 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_mem": { 2024-06-03T03:28:42.178 INFO:teuthology.orchestra.run.smithi117.stdout: "ino": 3, 2024-06-03T03:28:42.178 INFO:teuthology.orchestra.run.smithi117.stdout: "ino+": 3, 2024-06-03T03:28:42.178 INFO:teuthology.orchestra.run.smithi117.stdout: "ino-": 0, 2024-06-03T03:28:42.178 INFO:teuthology.orchestra.run.smithi117.stdout: "dir": 1, 2024-06-03T03:28:42.178 INFO:teuthology.orchestra.run.smithi117.stdout: "dir+": 1, 2024-06-03T03:28:42.178 INFO:teuthology.orchestra.run.smithi117.stdout: "dir-": 0, 2024-06-03T03:28:42.178 INFO:teuthology.orchestra.run.smithi117.stdout: "dn": 0, 2024-06-03T03:28:42.178 INFO:teuthology.orchestra.run.smithi117.stdout: "dn+": 0, 2024-06-03T03:28:42.178 INFO:teuthology.orchestra.run.smithi117.stdout: "dn-": 0, 2024-06-03T03:28:42.179 INFO:teuthology.orchestra.run.smithi117.stdout: "cap": 0, 2024-06-03T03:28:42.179 INFO:teuthology.orchestra.run.smithi117.stdout: "cap+": 0, 2024-06-03T03:28:42.179 INFO:teuthology.orchestra.run.smithi117.stdout: "cap-": 0, 2024-06-03T03:28:42.179 INFO:teuthology.orchestra.run.smithi117.stdout: "rss": 48640, 2024-06-03T03:28:42.179 INFO:teuthology.orchestra.run.smithi117.stdout: "heap": 215420 2024-06-03T03:28:42.179 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:42.179 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_server": { 2024-06-03T03:28:42.179 INFO:teuthology.orchestra.run.smithi117.stdout: "dispatch_client_request": 0, 2024-06-03T03:28:42.179 INFO:teuthology.orchestra.run.smithi117.stdout: "dispatch_server_request": 0, 2024-06-03T03:28:42.179 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_request": 0, 2024-06-03T03:28:42.179 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_session": 0, 2024-06-03T03:28:42.179 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_peer_request": 0, 2024-06-03T03:28:42.179 INFO:teuthology.orchestra.run.smithi117.stdout: "req_create_latency": { 2024-06-03T03:28:42.179 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:42.179 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:42.180 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:42.180 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:42.180 INFO:teuthology.orchestra.run.smithi117.stdout: "req_getattr_latency": { 2024-06-03T03:28:42.180 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:42.180 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:42.180 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:42.180 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:42.180 INFO:teuthology.orchestra.run.smithi117.stdout: "req_getfilelock_latency": { 2024-06-03T03:28:42.180 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:42.180 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:42.180 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:42.180 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:42.180 INFO:teuthology.orchestra.run.smithi117.stdout: "req_link_latency": { 2024-06-03T03:28:42.181 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:42.181 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:42.181 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:42.181 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:42.181 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookup_latency": { 2024-06-03T03:28:42.181 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:42.181 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:42.181 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:42.181 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:42.181 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookuphash_latency": { 2024-06-03T03:28:42.181 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:42.181 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:42.181 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:42.181 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:42.182 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupino_latency": { 2024-06-03T03:28:42.182 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:42.182 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:42.182 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:42.182 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:42.182 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupname_latency": { 2024-06-03T03:28:42.182 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:42.182 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:42.182 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:42.182 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:42.182 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupparent_latency": { 2024-06-03T03:28:42.182 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:42.182 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:42.183 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:42.183 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:42.183 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupsnap_latency": { 2024-06-03T03:28:42.183 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:42.183 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:42.183 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:42.183 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:42.183 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lssnap_latency": { 2024-06-03T03:28:42.183 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:42.183 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:42.183 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:42.183 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:42.183 INFO:teuthology.orchestra.run.smithi117.stdout: "req_mkdir_latency": { 2024-06-03T03:28:42.183 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:42.184 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:42.184 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:42.184 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:42.184 INFO:teuthology.orchestra.run.smithi117.stdout: "req_mknod_latency": { 2024-06-03T03:28:42.184 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:42.184 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:42.185 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:42.185 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:42.185 INFO:teuthology.orchestra.run.smithi117.stdout: "req_mksnap_latency": { 2024-06-03T03:28:42.185 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:42.185 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:42.185 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:42.185 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:42.185 INFO:teuthology.orchestra.run.smithi117.stdout: "req_open_latency": { 2024-06-03T03:28:42.185 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:42.185 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:42.185 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:42.185 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:42.185 INFO:teuthology.orchestra.run.smithi117.stdout: "req_readdir_latency": { 2024-06-03T03:28:42.185 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:42.186 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:42.186 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:42.186 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:42.186 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rename_latency": { 2024-06-03T03:28:42.186 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:42.186 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:42.186 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:42.186 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:42.186 INFO:teuthology.orchestra.run.smithi117.stdout: "req_renamesnap_latency": { 2024-06-03T03:28:42.186 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:42.186 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:42.186 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:42.187 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:42.187 INFO:teuthology.orchestra.run.smithi117.stdout: "req_snapdiff_latency": { 2024-06-03T03:28:42.187 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:42.187 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:42.187 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:42.187 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:42.187 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rmdir_latency": { 2024-06-03T03:28:42.187 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:42.187 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:42.187 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:42.187 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:42.187 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rmsnap_latency": { 2024-06-03T03:28:42.187 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:42.188 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:42.188 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:42.188 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:42.188 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rmxattr_latency": { 2024-06-03T03:28:42.188 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:42.188 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:42.188 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:42.188 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:42.188 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setattr_latency": { 2024-06-03T03:28:42.188 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:42.188 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:42.188 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:42.188 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:42.188 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setdirlayout_latency": { 2024-06-03T03:28:42.189 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:42.189 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:42.189 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:42.189 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:42.189 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setfilelock_latency": { 2024-06-03T03:28:42.189 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:42.189 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:42.189 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:42.189 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:42.189 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setlayout_latency": { 2024-06-03T03:28:42.189 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:42.189 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:42.189 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:42.189 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:42.190 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setxattr_latency": { 2024-06-03T03:28:42.190 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:42.190 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:42.190 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:42.190 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:42.190 INFO:teuthology.orchestra.run.smithi117.stdout: "req_symlink_latency": { 2024-06-03T03:28:42.190 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:42.190 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:42.190 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:42.190 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:42.190 INFO:teuthology.orchestra.run.smithi117.stdout: "req_unlink_latency": { 2024-06-03T03:28:42.190 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:42.190 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:42.191 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:42.191 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:42.191 INFO:teuthology.orchestra.run.smithi117.stdout: "cap_revoke_eviction": 0, 2024-06-03T03:28:42.191 INFO:teuthology.orchestra.run.smithi117.stdout: "cap_acquisition_throttle": 0, 2024-06-03T03:28:42.191 INFO:teuthology.orchestra.run.smithi117.stdout: "req_getvxattr_latency": { 2024-06-03T03:28:42.191 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:42.191 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:42.191 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:42.191 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:42.191 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:42.191 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_sessions": { 2024-06-03T03:28:42.191 INFO:teuthology.orchestra.run.smithi117.stdout: "session_count": 1, 2024-06-03T03:28:42.191 INFO:teuthology.orchestra.run.smithi117.stdout: "session_add": 3, 2024-06-03T03:28:42.191 INFO:teuthology.orchestra.run.smithi117.stdout: "session_remove": 2, 2024-06-03T03:28:42.191 INFO:teuthology.orchestra.run.smithi117.stdout: "sessions_open": 2, 2024-06-03T03:28:42.192 INFO:teuthology.orchestra.run.smithi117.stdout: "sessions_stale": 0, 2024-06-03T03:28:42.192 INFO:teuthology.orchestra.run.smithi117.stdout: "total_load": 0, 2024-06-03T03:28:42.192 INFO:teuthology.orchestra.run.smithi117.stdout: "average_load": 0, 2024-06-03T03:28:42.192 INFO:teuthology.orchestra.run.smithi117.stdout: "avg_session_uptime": 3282, 2024-06-03T03:28:42.192 INFO:teuthology.orchestra.run.smithi117.stdout: "mdthresh_evicted": 0 2024-06-03T03:28:42.192 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:42.192 INFO:teuthology.orchestra.run.smithi117.stdout: "mempool": { 2024-06-03T03:28:42.192 INFO:teuthology.orchestra.run.smithi117.stdout: "bloom_filter_bytes": 0, 2024-06-03T03:28:42.192 INFO:teuthology.orchestra.run.smithi117.stdout: "bloom_filter_items": 0, 2024-06-03T03:28:42.192 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_alloc_bytes": 0, 2024-06-03T03:28:42.192 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_alloc_items": 0, 2024-06-03T03:28:42.192 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_data_bytes": 0, 2024-06-03T03:28:42.192 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_data_items": 0, 2024-06-03T03:28:42.192 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-03T03:28:42.193 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_onode_items": 0, 2024-06-03T03:28:42.193 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-03T03:28:42.193 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_meta_items": 0, 2024-06-03T03:28:42.193 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_other_bytes": 0, 2024-06-03T03:28:42.193 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_other_items": 0, 2024-06-03T03:28:42.193 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-03T03:28:42.193 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_buffer_items": 0, 2024-06-03T03:28:42.193 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_extent_bytes": 0, 2024-06-03T03:28:42.193 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_extent_items": 0, 2024-06-03T03:28:42.193 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_blob_bytes": 0, 2024-06-03T03:28:42.193 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_blob_items": 0, 2024-06-03T03:28:42.193 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-03T03:28:42.193 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_shared_blob_items": 0, 2024-06-03T03:28:42.194 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-03T03:28:42.194 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_inline_bl_items": 0, 2024-06-03T03:28:42.194 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_fsck_bytes": 0, 2024-06-03T03:28:42.194 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_fsck_items": 0, 2024-06-03T03:28:42.194 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_txc_bytes": 0, 2024-06-03T03:28:42.194 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_txc_items": 0, 2024-06-03T03:28:42.194 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-03T03:28:42.194 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_deferred_items": 0, 2024-06-03T03:28:42.194 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_bytes": 0, 2024-06-03T03:28:42.194 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_items": 0, 2024-06-03T03:28:42.194 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_bytes": 0, 2024-06-03T03:28:42.194 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_items": 0, 2024-06-03T03:28:42.194 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_reader_bytes": 0, 2024-06-03T03:28:42.194 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_reader_items": 0, 2024-06-03T03:28:42.195 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_writer_bytes": 0, 2024-06-03T03:28:42.195 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_writer_items": 0, 2024-06-03T03:28:42.195 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_anon_bytes": 182334, 2024-06-03T03:28:42.195 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_anon_items": 47, 2024-06-03T03:28:42.195 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_meta_bytes": 0, 2024-06-03T03:28:42.195 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_meta_items": 0, 2024-06-03T03:28:42.195 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_bytes": 0, 2024-06-03T03:28:42.195 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_items": 0, 2024-06-03T03:28:42.195 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_mapbl_bytes": 0, 2024-06-03T03:28:42.195 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_mapbl_items": 0, 2024-06-03T03:28:42.195 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_pglog_bytes": 0, 2024-06-03T03:28:42.195 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_pglog_items": 0, 2024-06-03T03:28:42.195 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_bytes": 8888, 2024-06-03T03:28:42.195 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_items": 170, 2024-06-03T03:28:42.195 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_mapping_bytes": 0, 2024-06-03T03:28:42.196 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_mapping_items": 0, 2024-06-03T03:28:42.196 INFO:teuthology.orchestra.run.smithi117.stdout: "pgmap_bytes": 0, 2024-06-03T03:28:42.196 INFO:teuthology.orchestra.run.smithi117.stdout: "pgmap_items": 0, 2024-06-03T03:28:42.196 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_co_bytes": 8464, 2024-06-03T03:28:42.196 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_co_items": 13, 2024-06-03T03:28:42.196 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_1_bytes": 0, 2024-06-03T03:28:42.196 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_1_items": 0, 2024-06-03T03:28:42.196 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_2_bytes": 0, 2024-06-03T03:28:42.196 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_2_items": 0 2024-06-03T03:28:42.196 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:42.196 INFO:teuthology.orchestra.run.smithi117.stdout: "objecter": { 2024-06-03T03:28:42.196 INFO:teuthology.orchestra.run.smithi117.stdout: "op_active": 0, 2024-06-03T03:28:42.196 INFO:teuthology.orchestra.run.smithi117.stdout: "op_laggy": 0, 2024-06-03T03:28:42.196 INFO:teuthology.orchestra.run.smithi117.stdout: "op_send": 1266, 2024-06-03T03:28:42.196 INFO:teuthology.orchestra.run.smithi117.stdout: "op_send_bytes": 21, 2024-06-03T03:28:42.197 INFO:teuthology.orchestra.run.smithi117.stdout: "op_resend": 3, 2024-06-03T03:28:42.197 INFO:teuthology.orchestra.run.smithi117.stdout: "op_reply": 1266, 2024-06-03T03:28:42.197 INFO:teuthology.orchestra.run.smithi117.stdout: "op_latency": { 2024-06-03T03:28:42.197 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 1266, 2024-06-03T03:28:42.197 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 1.049979532, 2024-06-03T03:28:42.197 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000829367 2024-06-03T03:28:42.197 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:42.197 INFO:teuthology.orchestra.run.smithi117.stdout: "op_inflight": 0, 2024-06-03T03:28:42.197 INFO:teuthology.orchestra.run.smithi117.stdout: "oplen_avg": { 2024-06-03T03:28:42.197 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 1266, 2024-06-03T03:28:42.197 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 1267 2024-06-03T03:28:42.197 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:42.197 INFO:teuthology.orchestra.run.smithi117.stdout: "op": 1266, 2024-06-03T03:28:42.197 INFO:teuthology.orchestra.run.smithi117.stdout: "op_r": 1266, 2024-06-03T03:28:42.198 INFO:teuthology.orchestra.run.smithi117.stdout: "op_w": 0, 2024-06-03T03:28:42.198 INFO:teuthology.orchestra.run.smithi117.stdout: "op_rmw": 0, 2024-06-03T03:28:42.198 INFO:teuthology.orchestra.run.smithi117.stdout: "op_pg": 0, 2024-06-03T03:28:42.198 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_stat": 801, 2024-06-03T03:28:42.198 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_create": 0, 2024-06-03T03:28:42.198 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_read": 463, 2024-06-03T03:28:42.198 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_write": 0, 2024-06-03T03:28:42.198 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_writefull": 0, 2024-06-03T03:28:42.198 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_writesame": 0, 2024-06-03T03:28:42.198 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_append": 0, 2024-06-03T03:28:42.198 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_zero": 0, 2024-06-03T03:28:42.198 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_truncate": 0, 2024-06-03T03:28:42.198 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_delete": 0, 2024-06-03T03:28:42.198 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_mapext": 0, 2024-06-03T03:28:42.199 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_sparse_read": 0, 2024-06-03T03:28:42.199 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_clonerange": 0, 2024-06-03T03:28:42.199 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_getxattr": 1, 2024-06-03T03:28:42.199 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_setxattr": 0, 2024-06-03T03:28:42.199 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_cmpxattr": 0, 2024-06-03T03:28:42.199 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_rmxattr": 0, 2024-06-03T03:28:42.199 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_resetxattrs": 0, 2024-06-03T03:28:42.199 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_call": 0, 2024-06-03T03:28:42.199 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_watch": 0, 2024-06-03T03:28:42.199 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_notify": 0, 2024-06-03T03:28:42.199 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_src_cmpxattr": 0, 2024-06-03T03:28:42.199 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_pgls": 0, 2024-06-03T03:28:42.199 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_pgls_filter": 0, 2024-06-03T03:28:42.199 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_other": 0, 2024-06-03T03:28:42.200 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_active": 0, 2024-06-03T03:28:42.200 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_send": 0, 2024-06-03T03:28:42.200 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_resend": 0, 2024-06-03T03:28:42.200 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_ping": 0, 2024-06-03T03:28:42.200 INFO:teuthology.orchestra.run.smithi117.stdout: "poolop_active": 0, 2024-06-03T03:28:42.200 INFO:teuthology.orchestra.run.smithi117.stdout: "poolop_send": 0, 2024-06-03T03:28:42.200 INFO:teuthology.orchestra.run.smithi117.stdout: "poolop_resend": 0, 2024-06-03T03:28:42.200 INFO:teuthology.orchestra.run.smithi117.stdout: "poolstat_active": 0, 2024-06-03T03:28:42.200 INFO:teuthology.orchestra.run.smithi117.stdout: "poolstat_send": 0, 2024-06-03T03:28:42.200 INFO:teuthology.orchestra.run.smithi117.stdout: "poolstat_resend": 0, 2024-06-03T03:28:42.200 INFO:teuthology.orchestra.run.smithi117.stdout: "statfs_active": 0, 2024-06-03T03:28:42.200 INFO:teuthology.orchestra.run.smithi117.stdout: "statfs_send": 0, 2024-06-03T03:28:42.200 INFO:teuthology.orchestra.run.smithi117.stdout: "statfs_resend": 0, 2024-06-03T03:28:42.200 INFO:teuthology.orchestra.run.smithi117.stdout: "command_active": 0, 2024-06-03T03:28:42.200 INFO:teuthology.orchestra.run.smithi117.stdout: "command_send": 0, 2024-06-03T03:28:42.201 INFO:teuthology.orchestra.run.smithi117.stdout: "command_resend": 0, 2024-06-03T03:28:42.201 INFO:teuthology.orchestra.run.smithi117.stdout: "map_epoch": 99, 2024-06-03T03:28:42.201 INFO:teuthology.orchestra.run.smithi117.stdout: "map_full": 0, 2024-06-03T03:28:42.201 INFO:teuthology.orchestra.run.smithi117.stdout: "map_inc": 2, 2024-06-03T03:28:42.201 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_sessions": 4, 2024-06-03T03:28:42.201 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_session_open": 4, 2024-06-03T03:28:42.201 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_session_close": 0, 2024-06-03T03:28:42.201 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_laggy": 0, 2024-06-03T03:28:42.201 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_wr": 0, 2024-06-03T03:28:42.201 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_rd": 2, 2024-06-03T03:28:42.201 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_del": 0 2024-06-03T03:28:42.201 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:42.201 INFO:teuthology.orchestra.run.smithi117.stdout: "oft": { 2024-06-03T03:28:42.201 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_objs": 0, 2024-06-03T03:28:42.201 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_kv_pairs": 0, 2024-06-03T03:28:42.202 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_updates": 0, 2024-06-03T03:28:42.202 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_removes": 0 2024-06-03T03:28:42.202 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:42.202 INFO:teuthology.orchestra.run.smithi117.stdout: "osd-slow-ops": { 2024-06-03T03:28:42.202 INFO:teuthology.orchestra.run.smithi117.stdout: "slow_ops_count": 0 2024-06-03T03:28:42.202 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:42.202 INFO:teuthology.orchestra.run.smithi117.stdout: "purge_queue": { 2024-06-03T03:28:42.202 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing_ops": 0, 2024-06-03T03:28:42.202 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing_ops_high_water": 0, 2024-06-03T03:28:42.202 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing": 0, 2024-06-03T03:28:42.202 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing_high_water": 0, 2024-06-03T03:28:42.202 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executed_ops": 0, 2024-06-03T03:28:42.202 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executed": 0, 2024-06-03T03:28:42.203 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_item_in_journal": 0 2024-06-03T03:28:42.203 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:42.203 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-03T03:28:42.203 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-06-03T03:28:42.203 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 104857600, 2024-06-03T03:28:42.203 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-06-03T03:28:42.203 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 1441, 2024-06-03T03:28:42.203 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 992918, 2024-06-03T03:28:42.203 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-06-03T03:28:42.203 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 1441, 2024-06-03T03:28:42.203 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2024-06-03T03:28:42.203 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2024-06-03T03:28:42.203 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 1441, 2024-06-03T03:28:42.203 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 992918, 2024-06-03T03:28:42.204 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-06-03T03:28:42.204 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:42.204 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:42.204 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:42.204 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:42.204 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:42.204 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-objecter_bytes": { 2024-06-03T03:28:42.204 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-06-03T03:28:42.204 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 104857600, 2024-06-03T03:28:42.204 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-06-03T03:28:42.204 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 0, 2024-06-03T03:28:42.204 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 0, 2024-06-03T03:28:42.204 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-06-03T03:28:42.204 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 0, 2024-06-03T03:28:42.205 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 1266, 2024-06-03T03:28:42.205 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 1961, 2024-06-03T03:28:42.205 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 1266, 2024-06-03T03:28:42.205 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 1961, 2024-06-03T03:28:42.205 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-06-03T03:28:42.205 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:42.205 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:42.205 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:42.205 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:42.205 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:42.205 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-objecter_ops": { 2024-06-03T03:28:42.205 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-06-03T03:28:42.205 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 1024, 2024-06-03T03:28:42.206 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-06-03T03:28:42.206 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 0, 2024-06-03T03:28:42.206 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 0, 2024-06-03T03:28:42.206 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-06-03T03:28:42.206 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 0, 2024-06-03T03:28:42.206 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 1266, 2024-06-03T03:28:42.206 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 1266, 2024-06-03T03:28:42.206 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 1266, 2024-06-03T03:28:42.206 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 1266, 2024-06-03T03:28:42.206 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-06-03T03:28:42.206 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:42.206 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:42.206 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:42.206 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:42.206 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:42.207 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-write_buf_throttle": { 2024-06-03T03:28:42.207 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-06-03T03:28:42.207 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 3758096384, 2024-06-03T03:28:42.207 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-06-03T03:28:42.207 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 0, 2024-06-03T03:28:42.207 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 0, 2024-06-03T03:28:42.207 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-06-03T03:28:42.207 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 0, 2024-06-03T03:28:42.207 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2024-06-03T03:28:42.207 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2024-06-03T03:28:42.207 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 0, 2024-06-03T03:28:42.207 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 0, 2024-06-03T03:28:42.207 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-06-03T03:28:42.208 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:42.208 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:42.208 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:42.208 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:42.208 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-06-03T03:28:42.208 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-write_buf_throttle-0x55cbedb9c780": { 2024-06-03T03:28:42.208 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-06-03T03:28:42.208 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 3758096384, 2024-06-03T03:28:42.208 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-06-03T03:28:42.208 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 0, 2024-06-03T03:28:42.208 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 0, 2024-06-03T03:28:42.208 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-06-03T03:28:42.208 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 0, 2024-06-03T03:28:42.208 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2024-06-03T03:28:42.209 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2024-06-03T03:28:42.209 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 0, 2024-06-03T03:28:42.209 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 0, 2024-06-03T03:28:42.209 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-06-03T03:28:42.209 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-06-03T03:28:42.209 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-06-03T03:28:42.209 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-06-03T03:28:42.209 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:42.209 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-06-03T03:28:42.209 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-06-03T03:28:43.312 INFO:tasks.check_counter:Daemon mds.j mds.exported=0 2024-06-03T03:28:43.312 INFO:tasks.check_counter:Daemon mds.j mds.imported=0 2024-06-03T03:28:43.313 DEBUG:tasks.check_counter:Getting stats from b 2024-06-03T03:28:43.313 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph daemon mds.b perf dump 2024-06-03T03:28:43.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:28:43 smithi151 ceph-mon[37043]: pgmap v608: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-06-03T03:28:43.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:28:43 smithi117 ceph-mon[31649]: pgmap v608: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-06-03T03:28:43.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:28:43 smithi183 ceph-mon[37326]: pgmap v608: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-06-03T03:28:43.563 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.b/config 2024-06-03T03:28:45.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:28:45 smithi151 ceph-mon[37043]: pgmap v609: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:28:45.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:28:45 smithi117 ceph-mon[31649]: pgmap v609: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:28:45.454 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-06-03T03:28:45.454 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-0": { 2024-06-03T03:28:45.454 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 219, 2024-06-03T03:28:45.454 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 203, 2024-06-03T03:28:45.454 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 802405, 2024-06-03T03:28:45.454 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 176549, 2024-06-03T03:28:45.454 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 14, 2024-06-03T03:28:45.454 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 3, 2024-06-03T03:28:45.454 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.035631956, 2024-06-03T03:28:45.454 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.011220056, 2024-06-03T03:28:45.455 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.021541217, 2024-06-03T03:28:45.455 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.004071330, 2024-06-03T03:28:45.455 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T03:28:45.455 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 202, 2024-06-03T03:28:45.455 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.018229843, 2024-06-03T03:28:45.455 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000090246 2024-06-03T03:28:45.455 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:45.455 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-06-03T03:28:45.455 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 13, 2024-06-03T03:28:45.455 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000001822, 2024-06-03T03:28:45.455 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000140 2024-06-03T03:28:45.455 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:45.455 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 799264, 2024-06-03T03:28:45.456 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 99408 2024-06-03T03:28:45.456 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:45.456 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-1": { 2024-06-03T03:28:45.456 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 43, 2024-06-03T03:28:45.456 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 161, 2024-06-03T03:28:45.456 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 13495, 2024-06-03T03:28:45.456 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 322962, 2024-06-03T03:28:45.456 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 13, 2024-06-03T03:28:45.456 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 6, 2024-06-03T03:28:45.456 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.019109293, 2024-06-03T03:28:45.456 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.010722001, 2024-06-03T03:28:45.456 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.006487005, 2024-06-03T03:28:45.456 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.000522870, 2024-06-03T03:28:45.456 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T03:28:45.457 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 158, 2024-06-03T03:28:45.457 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.023447027, 2024-06-03T03:28:45.457 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000148398 2024-06-03T03:28:45.457 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:45.457 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-06-03T03:28:45.457 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 31, 2024-06-03T03:28:45.457 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000006393, 2024-06-03T03:28:45.457 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000206 2024-06-03T03:28:45.457 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:45.457 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 7744, 2024-06-03T03:28:45.457 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 263152 2024-06-03T03:28:45.457 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:45.457 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-2": { 2024-06-03T03:28:45.457 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 246, 2024-06-03T03:28:45.458 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 319, 2024-06-03T03:28:45.458 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 64536, 2024-06-03T03:28:45.458 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 134197, 2024-06-03T03:28:45.458 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 7, 2024-06-03T03:28:45.458 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 4, 2024-06-03T03:28:45.458 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.033361594, 2024-06-03T03:28:45.458 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.012817783, 2024-06-03T03:28:45.458 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.015667398, 2024-06-03T03:28:45.458 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.003490529, 2024-06-03T03:28:45.458 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T03:28:45.458 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 319, 2024-06-03T03:28:45.458 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.034102004, 2024-06-03T03:28:45.458 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000106902 2024-06-03T03:28:45.459 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:45.459 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-06-03T03:28:45.459 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 520, 2024-06-03T03:28:45.459 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000175330, 2024-06-03T03:28:45.459 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000337 2024-06-03T03:28:45.459 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:45.459 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-03T03:28:45.459 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 0 2024-06-03T03:28:45.459 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:45.459 INFO:teuthology.orchestra.run.smithi151.stdout: "cct": { 2024-06-03T03:28:45.459 INFO:teuthology.orchestra.run.smithi151.stdout: "total_workers": 1, 2024-06-03T03:28:45.459 INFO:teuthology.orchestra.run.smithi151.stdout: "unhealthy_workers": 0 2024-06-03T03:28:45.460 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:45.460 INFO:teuthology.orchestra.run.smithi151.stdout: "finisher-MDSRank": { 2024-06-03T03:28:45.460 INFO:teuthology.orchestra.run.smithi151.stdout: "queue_len": 0, 2024-06-03T03:28:45.460 INFO:teuthology.orchestra.run.smithi151.stdout: "complete_latency": { 2024-06-03T03:28:45.460 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 55, 2024-06-03T03:28:45.460 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.001685234, 2024-06-03T03:28:45.460 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000030640 2024-06-03T03:28:45.460 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-06-03T03:28:45.460 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:45.460 INFO:teuthology.orchestra.run.smithi151.stdout: "finisher-PurgeQueue": { 2024-06-03T03:28:45.460 INFO:teuthology.orchestra.run.smithi151.stdout: "queue_len": 0, 2024-06-03T03:28:45.460 INFO:teuthology.orchestra.run.smithi151.stdout: "complete_latency": { 2024-06-03T03:28:45.461 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 2, 2024-06-03T03:28:45.461 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000007747, 2024-06-03T03:28:45.461 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000003873 2024-06-03T03:28:45.461 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-06-03T03:28:45.461 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:45.461 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": { 2024-06-03T03:28:45.461 INFO:teuthology.orchestra.run.smithi151.stdout: "request": 5, 2024-06-03T03:28:45.461 INFO:teuthology.orchestra.run.smithi151.stdout: "reply": 2, 2024-06-03T03:28:45.461 INFO:teuthology.orchestra.run.smithi151.stdout: "reply_latency": { 2024-06-03T03:28:45.461 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 2, 2024-06-03T03:28:45.461 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.001293740, 2024-06-03T03:28:45.461 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000646870 2024-06-03T03:28:45.461 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:45.462 INFO:teuthology.orchestra.run.smithi151.stdout: "slow_reply": 0, 2024-06-03T03:28:45.462 INFO:teuthology.orchestra.run.smithi151.stdout: "forward": 3, 2024-06-03T03:28:45.462 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_fetch_complete": 0, 2024-06-03T03:28:45.462 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_fetch_keys": 0, 2024-06-03T03:28:45.462 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_commit": 11, 2024-06-03T03:28:45.462 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_split": 0, 2024-06-03T03:28:45.462 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_merge": 0, 2024-06-03T03:28:45.462 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes": 14, 2024-06-03T03:28:45.462 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_top": 0, 2024-06-03T03:28:45.462 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_bottom": 14, 2024-06-03T03:28:45.462 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_pin_tail": 0, 2024-06-03T03:28:45.462 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_pinned": 14, 2024-06-03T03:28:45.462 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_expired": 1, 2024-06-03T03:28:45.462 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_with_caps": 1, 2024-06-03T03:28:45.463 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": 1, 2024-06-03T03:28:45.463 INFO:teuthology.orchestra.run.smithi151.stdout: "subtrees": 2, 2024-06-03T03:28:45.463 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse": 8, 2024-06-03T03:28:45.463 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_hit": 3, 2024-06-03T03:28:45.463 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_forward": 0, 2024-06-03T03:28:45.463 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_discover": 2, 2024-06-03T03:28:45.463 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_dir_fetch": 0, 2024-06-03T03:28:45.463 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_remote_ino": 0, 2024-06-03T03:28:45.463 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_lock": 0, 2024-06-03T03:28:45.463 INFO:teuthology.orchestra.run.smithi151.stdout: "load_cent": 0, 2024-06-03T03:28:45.463 INFO:teuthology.orchestra.run.smithi151.stdout: "q": 0, 2024-06-03T03:28:45.463 INFO:teuthology.orchestra.run.smithi151.stdout: "exported": 0, 2024-06-03T03:28:45.463 INFO:teuthology.orchestra.run.smithi151.stdout: "exported_inodes": 0, 2024-06-03T03:28:45.463 INFO:teuthology.orchestra.run.smithi151.stdout: "imported": 0, 2024-06-03T03:28:45.464 INFO:teuthology.orchestra.run.smithi151.stdout: "imported_inodes": 0, 2024-06-03T03:28:45.464 INFO:teuthology.orchestra.run.smithi151.stdout: "openino_dir_fetch": 0, 2024-06-03T03:28:45.464 INFO:teuthology.orchestra.run.smithi151.stdout: "openino_backtrace_fetch": 0, 2024-06-03T03:28:45.464 INFO:teuthology.orchestra.run.smithi151.stdout: "openino_peer_discover": 0, 2024-06-03T03:28:45.464 INFO:teuthology.orchestra.run.smithi151.stdout: "root_rfiles": 4, 2024-06-03T03:28:45.464 INFO:teuthology.orchestra.run.smithi151.stdout: "root_rbytes": 234, 2024-06-03T03:28:45.464 INFO:teuthology.orchestra.run.smithi151.stdout: "root_rsnaps": 0, 2024-06-03T03:28:45.464 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_backtrace_fetch": 0, 2024-06-03T03:28:45.464 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_set_tag": 0, 2024-06-03T03:28:45.464 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_backtrace_repaired": 0, 2024-06-03T03:28:45.465 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_inotable_repaired": 0, 2024-06-03T03:28:45.465 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_dir_inodes": 0, 2024-06-03T03:28:45.465 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_dir_base_inodes": 0, 2024-06-03T03:28:45.465 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_dirfrag_rstats": 0, 2024-06-03T03:28:45.465 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_file_inodes": 0, 2024-06-03T03:28:45.465 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_inode_file_caps": 0, 2024-06-03T03:28:45.465 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_revoke": 2, 2024-06-03T03:28:45.465 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_grant": 3, 2024-06-03T03:28:45.465 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_trunc": 0, 2024-06-03T03:28:45.465 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-03T03:28:45.465 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-03T03:28:45.465 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_caps": 2, 2024-06-03T03:28:45.465 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_caps_dirty": 0, 2024-06-03T03:28:45.465 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_cap_release": 0, 2024-06-03T03:28:45.466 INFO:teuthology.orchestra.run.smithi151.stdout: "process_request_cap_release": 3 2024-06-03T03:28:45.466 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:45.466 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_cache": { 2024-06-03T03:28:45.466 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_update": 0, 2024-06-03T03:28:45.466 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_update_receipt": 0, 2024-06-03T03:28:45.466 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_try_discover": 0, 2024-06-03T03:28:45.466 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_send_discover": 3, 2024-06-03T03:28:45.466 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_handle_discover": 0, 2024-06-03T03:28:45.466 INFO:teuthology.orchestra.run.smithi151.stdout: "num_strays": 0, 2024-06-03T03:28:45.466 INFO:teuthology.orchestra.run.smithi151.stdout: "num_strays_delayed": 0, 2024-06-03T03:28:45.466 INFO:teuthology.orchestra.run.smithi151.stdout: "num_strays_enqueuing": 0, 2024-06-03T03:28:45.466 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_created": 0, 2024-06-03T03:28:45.466 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_enqueued": 0, 2024-06-03T03:28:45.467 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_reintegrated": 0, 2024-06-03T03:28:45.467 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_migrated": 0, 2024-06-03T03:28:45.467 INFO:teuthology.orchestra.run.smithi151.stdout: "num_recovering_processing": 0, 2024-06-03T03:28:45.467 INFO:teuthology.orchestra.run.smithi151.stdout: "num_recovering_enqueued": 0, 2024-06-03T03:28:45.467 INFO:teuthology.orchestra.run.smithi151.stdout: "num_recovering_prioritized": 0, 2024-06-03T03:28:45.467 INFO:teuthology.orchestra.run.smithi151.stdout: "recovery_started": 0, 2024-06-03T03:28:45.467 INFO:teuthology.orchestra.run.smithi151.stdout: "recovery_completed": 0, 2024-06-03T03:28:45.467 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_quiesce_path": 0, 2024-06-03T03:28:45.467 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_quiesce_inode": 0, 2024-06-03T03:28:45.467 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_enqueue_scrub": 0, 2024-06-03T03:28:45.467 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_exportdir": 0, 2024-06-03T03:28:45.467 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_flush": 0, 2024-06-03T03:28:45.467 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_fragmentdir": 0, 2024-06-03T03:28:45.467 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_fragstats": 0, 2024-06-03T03:28:45.468 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_inodestats": 0 2024-06-03T03:28:45.468 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:45.468 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_log": { 2024-06-03T03:28:45.468 INFO:teuthology.orchestra.run.smithi151.stdout: "evlrg": 0, 2024-06-03T03:28:45.468 INFO:teuthology.orchestra.run.smithi151.stdout: "evadd": 7, 2024-06-03T03:28:45.468 INFO:teuthology.orchestra.run.smithi151.stdout: "evex": 0, 2024-06-03T03:28:45.468 INFO:teuthology.orchestra.run.smithi151.stdout: "evtrm": 0, 2024-06-03T03:28:45.468 INFO:teuthology.orchestra.run.smithi151.stdout: "ev": 7, 2024-06-03T03:28:45.468 INFO:teuthology.orchestra.run.smithi151.stdout: "evexg": 0, 2024-06-03T03:28:45.468 INFO:teuthology.orchestra.run.smithi151.stdout: "evexd": 0, 2024-06-03T03:28:45.468 INFO:teuthology.orchestra.run.smithi151.stdout: "segadd": 2, 2024-06-03T03:28:45.468 INFO:teuthology.orchestra.run.smithi151.stdout: "segex": 0, 2024-06-03T03:28:45.468 INFO:teuthology.orchestra.run.smithi151.stdout: "segtrm": 0, 2024-06-03T03:28:45.468 INFO:teuthology.orchestra.run.smithi151.stdout: "seg": 2, 2024-06-03T03:28:45.468 INFO:teuthology.orchestra.run.smithi151.stdout: "segmjr": 2, 2024-06-03T03:28:45.469 INFO:teuthology.orchestra.run.smithi151.stdout: "segexg": 0, 2024-06-03T03:28:45.469 INFO:teuthology.orchestra.run.smithi151.stdout: "segexd": 0, 2024-06-03T03:28:45.469 INFO:teuthology.orchestra.run.smithi151.stdout: "expos": 4194304, 2024-06-03T03:28:45.469 INFO:teuthology.orchestra.run.smithi151.stdout: "wrpos": 4196260, 2024-06-03T03:28:45.469 INFO:teuthology.orchestra.run.smithi151.stdout: "rdpos": 0, 2024-06-03T03:28:45.469 INFO:teuthology.orchestra.run.smithi151.stdout: "jlat": { 2024-06-03T03:28:45.469 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 6, 2024-06-03T03:28:45.469 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.012020426, 2024-06-03T03:28:45.469 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.002003404 2024-06-03T03:28:45.469 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:45.469 INFO:teuthology.orchestra.run.smithi151.stdout: "replayed": 0 2024-06-03T03:28:45.469 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:45.469 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_mem": { 2024-06-03T03:28:45.469 INFO:teuthology.orchestra.run.smithi151.stdout: "ino": 17, 2024-06-03T03:28:45.470 INFO:teuthology.orchestra.run.smithi151.stdout: "ino+": 17, 2024-06-03T03:28:45.470 INFO:teuthology.orchestra.run.smithi151.stdout: "ino-": 0, 2024-06-03T03:28:45.470 INFO:teuthology.orchestra.run.smithi151.stdout: "dir": 15, 2024-06-03T03:28:45.470 INFO:teuthology.orchestra.run.smithi151.stdout: "dir+": 17, 2024-06-03T03:28:45.470 INFO:teuthology.orchestra.run.smithi151.stdout: "dir-": 2, 2024-06-03T03:28:45.470 INFO:teuthology.orchestra.run.smithi151.stdout: "dn": 14, 2024-06-03T03:28:45.470 INFO:teuthology.orchestra.run.smithi151.stdout: "dn+": 15, 2024-06-03T03:28:45.470 INFO:teuthology.orchestra.run.smithi151.stdout: "dn-": 1, 2024-06-03T03:28:45.470 INFO:teuthology.orchestra.run.smithi151.stdout: "cap": 1, 2024-06-03T03:28:45.471 INFO:teuthology.orchestra.run.smithi151.stdout: "cap+": 2, 2024-06-03T03:28:45.471 INFO:teuthology.orchestra.run.smithi151.stdout: "cap-": 1, 2024-06-03T03:28:45.471 INFO:teuthology.orchestra.run.smithi151.stdout: "rss": 52048, 2024-06-03T03:28:45.471 INFO:teuthology.orchestra.run.smithi151.stdout: "heap": 223612 2024-06-03T03:28:45.471 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:45.471 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_server": { 2024-06-03T03:28:45.471 INFO:teuthology.orchestra.run.smithi151.stdout: "dispatch_client_request": 7, 2024-06-03T03:28:45.471 INFO:teuthology.orchestra.run.smithi151.stdout: "dispatch_server_request": 0, 2024-06-03T03:28:45.471 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_request": 5, 2024-06-03T03:28:45.471 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_session": 11, 2024-06-03T03:28:45.471 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_peer_request": 0, 2024-06-03T03:28:45.471 INFO:teuthology.orchestra.run.smithi151.stdout: "req_create_latency": { 2024-06-03T03:28:45.471 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:45.471 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:45.472 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:45.472 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:45.472 INFO:teuthology.orchestra.run.smithi151.stdout: "req_getattr_latency": { 2024-06-03T03:28:45.472 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 2, 2024-06-03T03:28:45.472 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.001293740, 2024-06-03T03:28:45.472 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000646870 2024-06-03T03:28:45.472 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:45.472 INFO:teuthology.orchestra.run.smithi151.stdout: "req_getfilelock_latency": { 2024-06-03T03:28:45.472 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:45.472 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:45.472 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:45.472 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:45.472 INFO:teuthology.orchestra.run.smithi151.stdout: "req_link_latency": { 2024-06-03T03:28:45.472 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:45.472 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:45.473 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:45.473 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:45.473 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookup_latency": { 2024-06-03T03:28:45.473 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:45.473 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:45.473 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:45.473 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:45.473 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookuphash_latency": { 2024-06-03T03:28:45.473 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:45.473 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:45.473 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:45.473 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:45.473 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupino_latency": { 2024-06-03T03:28:45.473 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:45.474 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:45.474 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:45.474 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:45.474 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupname_latency": { 2024-06-03T03:28:45.474 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:45.474 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:45.474 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:45.474 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:45.474 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupparent_latency": { 2024-06-03T03:28:45.474 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:45.474 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:45.474 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:45.474 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:45.475 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupsnap_latency": { 2024-06-03T03:28:45.475 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:45.475 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:45.475 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:45.475 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:45.475 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lssnap_latency": { 2024-06-03T03:28:45.475 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:45.475 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:45.475 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:45.475 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:45.475 INFO:teuthology.orchestra.run.smithi151.stdout: "req_mkdir_latency": { 2024-06-03T03:28:45.475 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:45.476 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:45.476 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:45.476 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:45.476 INFO:teuthology.orchestra.run.smithi151.stdout: "req_mknod_latency": { 2024-06-03T03:28:45.476 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:45.476 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:45.476 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:45.477 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:45.477 INFO:teuthology.orchestra.run.smithi151.stdout: "req_mksnap_latency": { 2024-06-03T03:28:45.477 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:45.477 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:45.477 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:45.477 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:45.477 INFO:teuthology.orchestra.run.smithi151.stdout: "req_open_latency": { 2024-06-03T03:28:45.477 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:45.477 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:45.477 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:45.477 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:45.477 INFO:teuthology.orchestra.run.smithi151.stdout: "req_readdir_latency": { 2024-06-03T03:28:45.477 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:45.477 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:45.478 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:45.478 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:45.478 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rename_latency": { 2024-06-03T03:28:45.478 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:45.478 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:45.478 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:45.478 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:45.478 INFO:teuthology.orchestra.run.smithi151.stdout: "req_renamesnap_latency": { 2024-06-03T03:28:45.478 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:45.478 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:45.478 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:45.478 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:45.478 INFO:teuthology.orchestra.run.smithi151.stdout: "req_snapdiff_latency": { 2024-06-03T03:28:45.479 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:45.479 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:45.479 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:45.479 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:45.479 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rmdir_latency": { 2024-06-03T03:28:45.479 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:45.479 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:45.479 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:45.479 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:45.479 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rmsnap_latency": { 2024-06-03T03:28:45.479 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:45.479 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:45.479 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:45.480 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:45.480 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rmxattr_latency": { 2024-06-03T03:28:45.480 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:45.480 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:45.480 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:45.480 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:45.480 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setattr_latency": { 2024-06-03T03:28:45.480 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:45.480 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:45.480 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:45.480 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:45.481 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setdirlayout_latency": { 2024-06-03T03:28:45.481 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:45.481 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:45.481 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:45.481 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:45.481 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setfilelock_latency": { 2024-06-03T03:28:45.481 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:45.481 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:45.481 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:45.481 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:45.481 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setlayout_latency": { 2024-06-03T03:28:45.482 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:45.482 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:45.482 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:45.482 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:45.482 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setxattr_latency": { 2024-06-03T03:28:45.482 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:45.482 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:45.482 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:45.482 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:45.482 INFO:teuthology.orchestra.run.smithi151.stdout: "req_symlink_latency": { 2024-06-03T03:28:45.483 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:45.483 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:45.483 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:45.483 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:45.483 INFO:teuthology.orchestra.run.smithi151.stdout: "req_unlink_latency": { 2024-06-03T03:28:45.483 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:45.483 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:45.483 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:45.483 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:45.483 INFO:teuthology.orchestra.run.smithi151.stdout: "cap_revoke_eviction": 0, 2024-06-03T03:28:45.483 INFO:teuthology.orchestra.run.smithi151.stdout: "cap_acquisition_throttle": 0, 2024-06-03T03:28:45.483 INFO:teuthology.orchestra.run.smithi151.stdout: "req_getvxattr_latency": { 2024-06-03T03:28:45.483 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:45.483 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:45.484 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:45.484 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-06-03T03:28:45.484 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:45.484 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_sessions": { 2024-06-03T03:28:45.484 INFO:teuthology.orchestra.run.smithi151.stdout: "session_count": 1, 2024-06-03T03:28:45.484 INFO:teuthology.orchestra.run.smithi151.stdout: "session_add": 3, 2024-06-03T03:28:45.484 INFO:teuthology.orchestra.run.smithi151.stdout: "session_remove": 2, 2024-06-03T03:28:45.484 INFO:teuthology.orchestra.run.smithi151.stdout: "sessions_open": 1, 2024-06-03T03:28:45.484 INFO:teuthology.orchestra.run.smithi151.stdout: "sessions_stale": 0, 2024-06-03T03:28:45.484 INFO:teuthology.orchestra.run.smithi151.stdout: "total_load": 1, 2024-06-03T03:28:45.484 INFO:teuthology.orchestra.run.smithi151.stdout: "average_load": 1, 2024-06-03T03:28:45.484 INFO:teuthology.orchestra.run.smithi151.stdout: "avg_session_uptime": 214, 2024-06-03T03:28:45.484 INFO:teuthology.orchestra.run.smithi151.stdout: "mdthresh_evicted": 0 2024-06-03T03:28:45.485 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:45.485 INFO:teuthology.orchestra.run.smithi151.stdout: "mempool": { 2024-06-03T03:28:45.485 INFO:teuthology.orchestra.run.smithi151.stdout: "bloom_filter_bytes": 0, 2024-06-03T03:28:45.485 INFO:teuthology.orchestra.run.smithi151.stdout: "bloom_filter_items": 0, 2024-06-03T03:28:45.485 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_alloc_bytes": 0, 2024-06-03T03:28:45.485 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_alloc_items": 0, 2024-06-03T03:28:45.485 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_data_bytes": 0, 2024-06-03T03:28:45.485 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_data_items": 0, 2024-06-03T03:28:45.485 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-03T03:28:45.485 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_onode_items": 0, 2024-06-03T03:28:45.485 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-03T03:28:45.485 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_meta_items": 0, 2024-06-03T03:28:45.485 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_other_bytes": 0, 2024-06-03T03:28:45.485 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_other_items": 0, 2024-06-03T03:28:45.486 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-03T03:28:45.486 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_buffer_items": 0, 2024-06-03T03:28:45.486 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_extent_bytes": 0, 2024-06-03T03:28:45.486 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_extent_items": 0, 2024-06-03T03:28:45.486 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_blob_bytes": 0, 2024-06-03T03:28:45.486 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_blob_items": 0, 2024-06-03T03:28:45.486 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-03T03:28:45.486 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_shared_blob_items": 0, 2024-06-03T03:28:45.486 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-03T03:28:45.486 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_inline_bl_items": 0, 2024-06-03T03:28:45.486 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_fsck_bytes": 0, 2024-06-03T03:28:45.486 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_fsck_items": 0, 2024-06-03T03:28:45.486 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_txc_bytes": 0, 2024-06-03T03:28:45.487 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_txc_items": 0, 2024-06-03T03:28:45.487 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-03T03:28:45.487 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_deferred_items": 0, 2024-06-03T03:28:45.487 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_bytes": 0, 2024-06-03T03:28:45.487 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_items": 0, 2024-06-03T03:28:45.487 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_bytes": 0, 2024-06-03T03:28:45.487 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_items": 0, 2024-06-03T03:28:45.487 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_reader_bytes": 0, 2024-06-03T03:28:45.487 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_reader_items": 0, 2024-06-03T03:28:45.487 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_writer_bytes": 0, 2024-06-03T03:28:45.487 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_writer_items": 0, 2024-06-03T03:28:45.487 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_anon_bytes": 255319, 2024-06-03T03:28:45.487 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_anon_items": 85, 2024-06-03T03:28:45.487 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_meta_bytes": 0, 2024-06-03T03:28:45.488 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_meta_items": 0, 2024-06-03T03:28:45.488 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_bytes": 0, 2024-06-03T03:28:45.488 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_items": 0, 2024-06-03T03:28:45.488 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_mapbl_bytes": 0, 2024-06-03T03:28:45.488 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_mapbl_items": 0, 2024-06-03T03:28:45.488 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_pglog_bytes": 0, 2024-06-03T03:28:45.488 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_pglog_items": 0, 2024-06-03T03:28:45.488 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_bytes": 8888, 2024-06-03T03:28:45.488 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_items": 170, 2024-06-03T03:28:45.488 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_mapping_bytes": 0, 2024-06-03T03:28:45.488 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_mapping_items": 0, 2024-06-03T03:28:45.488 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap_bytes": 0, 2024-06-03T03:28:45.488 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap_items": 0, 2024-06-03T03:28:45.488 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_co_bytes": 67781, 2024-06-03T03:28:45.489 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_co_items": 290, 2024-06-03T03:28:45.489 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_1_bytes": 0, 2024-06-03T03:28:45.489 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_1_items": 0, 2024-06-03T03:28:45.489 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_2_bytes": 0, 2024-06-03T03:28:45.489 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_2_items": 0 2024-06-03T03:28:45.489 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:45.489 INFO:teuthology.orchestra.run.smithi151.stdout: "objecter": { 2024-06-03T03:28:45.489 INFO:teuthology.orchestra.run.smithi151.stdout: "op_active": 0, 2024-06-03T03:28:45.489 INFO:teuthology.orchestra.run.smithi151.stdout: "op_laggy": 0, 2024-06-03T03:28:45.489 INFO:teuthology.orchestra.run.smithi151.stdout: "op_send": 45, 2024-06-03T03:28:45.489 INFO:teuthology.orchestra.run.smithi151.stdout: "op_send_bytes": 12753, 2024-06-03T03:28:45.489 INFO:teuthology.orchestra.run.smithi151.stdout: "op_resend": 28, 2024-06-03T03:28:45.489 INFO:teuthology.orchestra.run.smithi151.stdout: "op_reply": 45, 2024-06-03T03:28:45.489 INFO:teuthology.orchestra.run.smithi151.stdout: "op_latency": { 2024-06-03T03:28:45.490 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 45, 2024-06-03T03:28:45.490 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.297996001, 2024-06-03T03:28:45.490 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.006622133 2024-06-03T03:28:45.490 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:45.490 INFO:teuthology.orchestra.run.smithi151.stdout: "op_inflight": 0, 2024-06-03T03:28:45.490 INFO:teuthology.orchestra.run.smithi151.stdout: "oplen_avg": { 2024-06-03T03:28:45.490 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 45, 2024-06-03T03:28:45.490 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 69 2024-06-03T03:28:45.490 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:45.490 INFO:teuthology.orchestra.run.smithi151.stdout: "op": 45, 2024-06-03T03:28:45.490 INFO:teuthology.orchestra.run.smithi151.stdout: "op_r": 0, 2024-06-03T03:28:45.490 INFO:teuthology.orchestra.run.smithi151.stdout: "op_w": 45, 2024-06-03T03:28:45.490 INFO:teuthology.orchestra.run.smithi151.stdout: "op_rmw": 0, 2024-06-03T03:28:45.490 INFO:teuthology.orchestra.run.smithi151.stdout: "op_pg": 0, 2024-06-03T03:28:45.491 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_stat": 0, 2024-06-03T03:28:45.491 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_create": 10, 2024-06-03T03:28:45.491 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_read": 0, 2024-06-03T03:28:45.491 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_write": 6, 2024-06-03T03:28:45.491 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_writefull": 8, 2024-06-03T03:28:45.491 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_writesame": 0, 2024-06-03T03:28:45.491 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_append": 0, 2024-06-03T03:28:45.491 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_zero": 0, 2024-06-03T03:28:45.491 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_truncate": 0, 2024-06-03T03:28:45.491 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_delete": 6, 2024-06-03T03:28:45.491 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_mapext": 0, 2024-06-03T03:28:45.491 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_sparse_read": 0, 2024-06-03T03:28:45.491 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_clonerange": 0, 2024-06-03T03:28:45.491 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_getxattr": 0, 2024-06-03T03:28:45.492 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_setxattr": 20, 2024-06-03T03:28:45.492 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_cmpxattr": 0, 2024-06-03T03:28:45.492 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_rmxattr": 0, 2024-06-03T03:28:45.492 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_resetxattrs": 0, 2024-06-03T03:28:45.492 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_call": 0, 2024-06-03T03:28:45.492 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_watch": 0, 2024-06-03T03:28:45.492 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_notify": 0, 2024-06-03T03:28:45.492 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_src_cmpxattr": 0, 2024-06-03T03:28:45.492 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_pgls": 0, 2024-06-03T03:28:45.492 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_pgls_filter": 0, 2024-06-03T03:28:45.492 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_other": 0, 2024-06-03T03:28:45.492 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_active": 0, 2024-06-03T03:28:45.492 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_send": 0, 2024-06-03T03:28:45.492 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_resend": 0, 2024-06-03T03:28:45.493 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_ping": 0, 2024-06-03T03:28:45.493 INFO:teuthology.orchestra.run.smithi151.stdout: "poolop_active": 0, 2024-06-03T03:28:45.493 INFO:teuthology.orchestra.run.smithi151.stdout: "poolop_send": 0, 2024-06-03T03:28:45.493 INFO:teuthology.orchestra.run.smithi151.stdout: "poolop_resend": 0, 2024-06-03T03:28:45.493 INFO:teuthology.orchestra.run.smithi151.stdout: "poolstat_active": 0, 2024-06-03T03:28:45.493 INFO:teuthology.orchestra.run.smithi151.stdout: "poolstat_send": 0, 2024-06-03T03:28:45.493 INFO:teuthology.orchestra.run.smithi151.stdout: "poolstat_resend": 0, 2024-06-03T03:28:45.493 INFO:teuthology.orchestra.run.smithi151.stdout: "statfs_active": 0, 2024-06-03T03:28:45.493 INFO:teuthology.orchestra.run.smithi151.stdout: "statfs_send": 0, 2024-06-03T03:28:45.493 INFO:teuthology.orchestra.run.smithi151.stdout: "statfs_resend": 0, 2024-06-03T03:28:45.494 INFO:teuthology.orchestra.run.smithi151.stdout: "command_active": 0, 2024-06-03T03:28:45.494 INFO:teuthology.orchestra.run.smithi151.stdout: "command_send": 0, 2024-06-03T03:28:45.494 INFO:teuthology.orchestra.run.smithi151.stdout: "command_resend": 0, 2024-06-03T03:28:45.494 INFO:teuthology.orchestra.run.smithi151.stdout: "map_epoch": 99, 2024-06-03T03:28:45.494 INFO:teuthology.orchestra.run.smithi151.stdout: "map_full": 0, 2024-06-03T03:28:45.494 INFO:teuthology.orchestra.run.smithi151.stdout: "map_inc": 2, 2024-06-03T03:28:45.494 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_sessions": 10, 2024-06-03T03:28:45.494 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_session_open": 10, 2024-06-03T03:28:45.494 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_session_close": 0, 2024-06-03T03:28:45.494 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_laggy": 0, 2024-06-03T03:28:45.494 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_wr": 18, 2024-06-03T03:28:45.494 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_rd": 0, 2024-06-03T03:28:45.495 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_del": 1 2024-06-03T03:28:45.495 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:45.495 INFO:teuthology.orchestra.run.smithi151.stdout: "oft": { 2024-06-03T03:28:45.495 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_objs": 1, 2024-06-03T03:28:45.495 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_kv_pairs": 0, 2024-06-03T03:28:45.495 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_updates": 0, 2024-06-03T03:28:45.495 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_removes": 0 2024-06-03T03:28:45.495 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:45.495 INFO:teuthology.orchestra.run.smithi151.stdout: "osd-slow-ops": { 2024-06-03T03:28:45.495 INFO:teuthology.orchestra.run.smithi151.stdout: "slow_ops_count": 0 2024-06-03T03:28:45.495 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:45.495 INFO:teuthology.orchestra.run.smithi151.stdout: "purge_queue": { 2024-06-03T03:28:45.495 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing_ops": 0, 2024-06-03T03:28:45.495 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing_ops_high_water": 0, 2024-06-03T03:28:45.496 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing": 0, 2024-06-03T03:28:45.496 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing_high_water": 0, 2024-06-03T03:28:45.496 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executed_ops": 0, 2024-06-03T03:28:45.496 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executed": 0, 2024-06-03T03:28:45.496 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_item_in_journal": 0 2024-06-03T03:28:45.496 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:45.496 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-03T03:28:45.496 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-06-03T03:28:45.496 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 104857600, 2024-06-03T03:28:45.496 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-06-03T03:28:45.496 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 504, 2024-06-03T03:28:45.496 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 821164, 2024-06-03T03:28:45.496 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-06-03T03:28:45.497 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 504, 2024-06-03T03:28:45.497 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-06-03T03:28:45.497 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-06-03T03:28:45.497 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 504, 2024-06-03T03:28:45.497 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 821164, 2024-06-03T03:28:45.497 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-06-03T03:28:45.497 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:45.497 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:45.497 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:45.497 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-06-03T03:28:45.497 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:45.497 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-objecter_bytes": { 2024-06-03T03:28:45.497 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-06-03T03:28:45.497 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 104857600, 2024-06-03T03:28:45.498 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-06-03T03:28:45.498 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-06-03T03:28:45.498 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-06-03T03:28:45.498 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-06-03T03:28:45.498 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-06-03T03:28:45.498 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 45, 2024-06-03T03:28:45.498 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 12753, 2024-06-03T03:28:45.498 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 45, 2024-06-03T03:28:45.498 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 12753, 2024-06-03T03:28:45.498 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-06-03T03:28:45.498 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:45.498 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:45.498 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:45.498 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-06-03T03:28:45.499 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:45.499 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-objecter_ops": { 2024-06-03T03:28:45.499 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-06-03T03:28:45.499 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 1024, 2024-06-03T03:28:45.499 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-06-03T03:28:45.499 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-06-03T03:28:45.499 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-06-03T03:28:45.499 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-06-03T03:28:45.499 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-06-03T03:28:45.499 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 45, 2024-06-03T03:28:45.499 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 45, 2024-06-03T03:28:45.499 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 45, 2024-06-03T03:28:45.499 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 45, 2024-06-03T03:28:45.500 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-06-03T03:28:45.500 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:45.500 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:45.500 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:45.500 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-06-03T03:28:45.500 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:45.500 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-write_buf_throttle": { 2024-06-03T03:28:45.500 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-06-03T03:28:45.500 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 3758096384, 2024-06-03T03:28:45.500 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-06-03T03:28:45.500 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-06-03T03:28:45.500 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-06-03T03:28:45.500 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-06-03T03:28:45.500 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-06-03T03:28:45.501 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-06-03T03:28:45.501 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-06-03T03:28:45.501 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 0, 2024-06-03T03:28:45.501 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 0, 2024-06-03T03:28:45.501 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-06-03T03:28:45.501 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:45.501 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:45.501 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:45.501 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-06-03T03:28:45.501 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:45.501 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-write_buf_throttle-0x563399cb6500": { 2024-06-03T03:28:45.501 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-06-03T03:28:45.501 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 3758096384, 2024-06-03T03:28:45.501 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-06-03T03:28:45.502 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 7, 2024-06-03T03:28:45.502 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 1956, 2024-06-03T03:28:45.502 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-06-03T03:28:45.502 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 7, 2024-06-03T03:28:45.502 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-06-03T03:28:45.502 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-06-03T03:28:45.502 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 6, 2024-06-03T03:28:45.502 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 1956, 2024-06-03T03:28:45.502 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-06-03T03:28:45.502 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:45.502 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:45.502 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:45.502 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-06-03T03:28:45.502 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-06-03T03:28:45.503 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-06-03T03:28:45.503 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:28:45 smithi183 ceph-mon[37326]: pgmap v609: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:28:46.245 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-06-03T03:28:46.245 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-06-03T03:28:46.245 DEBUG:tasks.check_counter:Getting stats from e 2024-06-03T03:28:46.246 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph daemon mds.e perf dump 2024-06-03T03:28:46.496 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.b/config 2024-06-03T03:28:47.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:28:47 smithi151 ceph-mon[37043]: pgmap v610: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-03T03:28:47.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:28:47 smithi117 ceph-mon[31649]: pgmap v610: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-03T03:28:47.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:28:47 smithi183 ceph-mon[37326]: pgmap v610: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-03T03:28:48.213 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-06-03T03:28:48.213 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-0": { 2024-06-03T03:28:48.213 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 468, 2024-06-03T03:28:48.214 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 468, 2024-06-03T03:28:48.214 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 125803, 2024-06-03T03:28:48.214 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 148064, 2024-06-03T03:28:48.214 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 5, 2024-06-03T03:28:48.214 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 1, 2024-06-03T03:28:48.214 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.043093558, 2024-06-03T03:28:48.214 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.016526635, 2024-06-03T03:28:48.214 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.010463147, 2024-06-03T03:28:48.214 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.011053039, 2024-06-03T03:28:48.214 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T03:28:48.215 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 467, 2024-06-03T03:28:48.215 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.019936062, 2024-06-03T03:28:48.215 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000042689 2024-06-03T03:28:48.215 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:48.215 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-06-03T03:28:48.215 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:48.215 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:48.215 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:48.215 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:48.215 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-06-03T03:28:48.215 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 448 2024-06-03T03:28:48.215 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:48.216 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-1": { 2024-06-03T03:28:48.216 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 175, 2024-06-03T03:28:48.216 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 168, 2024-06-03T03:28:48.216 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 769346, 2024-06-03T03:28:48.216 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 99137, 2024-06-03T03:28:48.216 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 3, 2024-06-03T03:28:48.216 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 2, 2024-06-03T03:28:48.216 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.030764886, 2024-06-03T03:28:48.216 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.010706478, 2024-06-03T03:28:48.216 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.018422136, 2024-06-03T03:28:48.216 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.004184862, 2024-06-03T03:28:48.216 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T03:28:48.216 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 167, 2024-06-03T03:28:48.217 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.010836174, 2024-06-03T03:28:48.217 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000064887 2024-06-03T03:28:48.217 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:48.217 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-06-03T03:28:48.217 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:48.217 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:48.217 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:48.217 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:48.217 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 768832, 2024-06-03T03:28:48.217 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 98496 2024-06-03T03:28:48.217 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:48.217 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-2": { 2024-06-03T03:28:48.217 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 461, 2024-06-03T03:28:48.218 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 581, 2024-06-03T03:28:48.218 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 154720, 2024-06-03T03:28:48.218 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 403352, 2024-06-03T03:28:48.218 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 4, 2024-06-03T03:28:48.218 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 3, 2024-06-03T03:28:48.218 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.061737937, 2024-06-03T03:28:48.218 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.022426198, 2024-06-03T03:28:48.218 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.014863418, 2024-06-03T03:28:48.218 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.017827283, 2024-06-03T03:28:48.218 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T03:28:48.218 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 579, 2024-06-03T03:28:48.219 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.033489317, 2024-06-03T03:28:48.219 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000057839 2024-06-03T03:28:48.219 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:48.219 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-06-03T03:28:48.219 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:48.219 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:48.219 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:48.219 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:48.219 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 160, 2024-06-03T03:28:48.219 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 257072 2024-06-03T03:28:48.219 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:48.219 INFO:teuthology.orchestra.run.smithi151.stdout: "cct": { 2024-06-03T03:28:48.219 INFO:teuthology.orchestra.run.smithi151.stdout: "total_workers": 1, 2024-06-03T03:28:48.220 INFO:teuthology.orchestra.run.smithi151.stdout: "unhealthy_workers": 0 2024-06-03T03:28:48.220 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:48.220 INFO:teuthology.orchestra.run.smithi151.stdout: "finisher-MDSRank": { 2024-06-03T03:28:48.220 INFO:teuthology.orchestra.run.smithi151.stdout: "queue_len": 0, 2024-06-03T03:28:48.220 INFO:teuthology.orchestra.run.smithi151.stdout: "complete_latency": { 2024-06-03T03:28:48.220 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 1845, 2024-06-03T03:28:48.220 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.053990701, 2024-06-03T03:28:48.220 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000029263 2024-06-03T03:28:48.220 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-06-03T03:28:48.220 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:48.220 INFO:teuthology.orchestra.run.smithi151.stdout: "finisher-PurgeQueue": { 2024-06-03T03:28:48.220 INFO:teuthology.orchestra.run.smithi151.stdout: "queue_len": 0, 2024-06-03T03:28:48.220 INFO:teuthology.orchestra.run.smithi151.stdout: "complete_latency": { 2024-06-03T03:28:48.221 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:48.221 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:48.221 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:48.221 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-06-03T03:28:48.221 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:48.221 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": { 2024-06-03T03:28:48.221 INFO:teuthology.orchestra.run.smithi151.stdout: "request": 0, 2024-06-03T03:28:48.221 INFO:teuthology.orchestra.run.smithi151.stdout: "reply": 0, 2024-06-03T03:28:48.221 INFO:teuthology.orchestra.run.smithi151.stdout: "reply_latency": { 2024-06-03T03:28:48.221 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:48.221 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:48.221 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:48.222 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:48.222 INFO:teuthology.orchestra.run.smithi151.stdout: "slow_reply": 0, 2024-06-03T03:28:48.222 INFO:teuthology.orchestra.run.smithi151.stdout: "forward": 0, 2024-06-03T03:28:48.222 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_fetch_complete": 0, 2024-06-03T03:28:48.222 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_fetch_keys": 0, 2024-06-03T03:28:48.222 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_commit": 0, 2024-06-03T03:28:48.222 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_split": 0, 2024-06-03T03:28:48.222 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_merge": 0, 2024-06-03T03:28:48.222 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes": 0, 2024-06-03T03:28:48.222 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_top": 0, 2024-06-03T03:28:48.222 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_bottom": 0, 2024-06-03T03:28:48.222 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_pin_tail": 0, 2024-06-03T03:28:48.223 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_pinned": 0, 2024-06-03T03:28:48.223 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_expired": 0, 2024-06-03T03:28:48.223 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_with_caps": 0, 2024-06-03T03:28:48.223 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": 0, 2024-06-03T03:28:48.223 INFO:teuthology.orchestra.run.smithi151.stdout: "subtrees": 1, 2024-06-03T03:28:48.223 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse": 0, 2024-06-03T03:28:48.223 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_hit": 0, 2024-06-03T03:28:48.223 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_forward": 0, 2024-06-03T03:28:48.223 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_discover": 0, 2024-06-03T03:28:48.223 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_dir_fetch": 0, 2024-06-03T03:28:48.223 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_remote_ino": 0, 2024-06-03T03:28:48.223 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_lock": 0, 2024-06-03T03:28:48.223 INFO:teuthology.orchestra.run.smithi151.stdout: "load_cent": 0, 2024-06-03T03:28:48.224 INFO:teuthology.orchestra.run.smithi151.stdout: "q": 0, 2024-06-03T03:28:48.224 INFO:teuthology.orchestra.run.smithi151.stdout: "exported": 0, 2024-06-03T03:28:48.224 INFO:teuthology.orchestra.run.smithi151.stdout: "exported_inodes": 0, 2024-06-03T03:28:48.224 INFO:teuthology.orchestra.run.smithi151.stdout: "imported": 0, 2024-06-03T03:28:48.224 INFO:teuthology.orchestra.run.smithi151.stdout: "imported_inodes": 0, 2024-06-03T03:28:48.224 INFO:teuthology.orchestra.run.smithi151.stdout: "openino_dir_fetch": 0, 2024-06-03T03:28:48.224 INFO:teuthology.orchestra.run.smithi151.stdout: "openino_backtrace_fetch": 0, 2024-06-03T03:28:48.224 INFO:teuthology.orchestra.run.smithi151.stdout: "openino_peer_discover": 0, 2024-06-03T03:28:48.224 INFO:teuthology.orchestra.run.smithi151.stdout: "root_rfiles": 0, 2024-06-03T03:28:48.224 INFO:teuthology.orchestra.run.smithi151.stdout: "root_rbytes": 0, 2024-06-03T03:28:48.224 INFO:teuthology.orchestra.run.smithi151.stdout: "root_rsnaps": 0, 2024-06-03T03:28:48.224 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_backtrace_fetch": 0, 2024-06-03T03:28:48.224 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_set_tag": 0, 2024-06-03T03:28:48.224 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_backtrace_repaired": 0, 2024-06-03T03:28:48.225 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_inotable_repaired": 0, 2024-06-03T03:28:48.225 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_dir_inodes": 0, 2024-06-03T03:28:48.225 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_dir_base_inodes": 0, 2024-06-03T03:28:48.225 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_dirfrag_rstats": 0, 2024-06-03T03:28:48.225 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_file_inodes": 0, 2024-06-03T03:28:48.225 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_inode_file_caps": 0, 2024-06-03T03:28:48.225 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_revoke": 0, 2024-06-03T03:28:48.225 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_grant": 0, 2024-06-03T03:28:48.225 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_trunc": 0, 2024-06-03T03:28:48.225 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-03T03:28:48.225 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-03T03:28:48.225 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_caps": 0, 2024-06-03T03:28:48.225 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_caps_dirty": 0, 2024-06-03T03:28:48.225 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_cap_release": 0, 2024-06-03T03:28:48.226 INFO:teuthology.orchestra.run.smithi151.stdout: "process_request_cap_release": 0 2024-06-03T03:28:48.226 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:48.226 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_cache": { 2024-06-03T03:28:48.226 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_update": 0, 2024-06-03T03:28:48.226 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_update_receipt": 0, 2024-06-03T03:28:48.226 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_try_discover": 0, 2024-06-03T03:28:48.226 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_send_discover": 0, 2024-06-03T03:28:48.226 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_handle_discover": 0, 2024-06-03T03:28:48.226 INFO:teuthology.orchestra.run.smithi151.stdout: "num_strays": 0, 2024-06-03T03:28:48.226 INFO:teuthology.orchestra.run.smithi151.stdout: "num_strays_delayed": 0, 2024-06-03T03:28:48.226 INFO:teuthology.orchestra.run.smithi151.stdout: "num_strays_enqueuing": 0, 2024-06-03T03:28:48.226 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_created": 0, 2024-06-03T03:28:48.226 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_enqueued": 0, 2024-06-03T03:28:48.226 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_reintegrated": 0, 2024-06-03T03:28:48.227 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_migrated": 0, 2024-06-03T03:28:48.227 INFO:teuthology.orchestra.run.smithi151.stdout: "num_recovering_processing": 0, 2024-06-03T03:28:48.227 INFO:teuthology.orchestra.run.smithi151.stdout: "num_recovering_enqueued": 0, 2024-06-03T03:28:48.227 INFO:teuthology.orchestra.run.smithi151.stdout: "num_recovering_prioritized": 0, 2024-06-03T03:28:48.227 INFO:teuthology.orchestra.run.smithi151.stdout: "recovery_started": 0, 2024-06-03T03:28:48.227 INFO:teuthology.orchestra.run.smithi151.stdout: "recovery_completed": 0, 2024-06-03T03:28:48.227 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_quiesce_path": 0, 2024-06-03T03:28:48.227 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_quiesce_inode": 0, 2024-06-03T03:28:48.227 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_enqueue_scrub": 0, 2024-06-03T03:28:48.227 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_exportdir": 0, 2024-06-03T03:28:48.227 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_flush": 0, 2024-06-03T03:28:48.227 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_fragmentdir": 0, 2024-06-03T03:28:48.228 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_fragstats": 0, 2024-06-03T03:28:48.228 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_inodestats": 0 2024-06-03T03:28:48.228 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:48.228 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_log": { 2024-06-03T03:28:48.228 INFO:teuthology.orchestra.run.smithi151.stdout: "evlrg": 0, 2024-06-03T03:28:48.228 INFO:teuthology.orchestra.run.smithi151.stdout: "evadd": 0, 2024-06-03T03:28:48.228 INFO:teuthology.orchestra.run.smithi151.stdout: "evex": 0, 2024-06-03T03:28:48.228 INFO:teuthology.orchestra.run.smithi151.stdout: "evtrm": 0, 2024-06-03T03:28:48.228 INFO:teuthology.orchestra.run.smithi151.stdout: "ev": 2, 2024-06-03T03:28:48.228 INFO:teuthology.orchestra.run.smithi151.stdout: "evexg": 0, 2024-06-03T03:28:48.228 INFO:teuthology.orchestra.run.smithi151.stdout: "evexd": 0, 2024-06-03T03:28:48.228 INFO:teuthology.orchestra.run.smithi151.stdout: "segadd": 0, 2024-06-03T03:28:48.228 INFO:teuthology.orchestra.run.smithi151.stdout: "segex": 0, 2024-06-03T03:28:48.228 INFO:teuthology.orchestra.run.smithi151.stdout: "segtrm": 0, 2024-06-03T03:28:48.229 INFO:teuthology.orchestra.run.smithi151.stdout: "seg": 2, 2024-06-03T03:28:48.229 INFO:teuthology.orchestra.run.smithi151.stdout: "segmjr": 2, 2024-06-03T03:28:48.229 INFO:teuthology.orchestra.run.smithi151.stdout: "segexg": 0, 2024-06-03T03:28:48.229 INFO:teuthology.orchestra.run.smithi151.stdout: "segexd": 0, 2024-06-03T03:28:48.229 INFO:teuthology.orchestra.run.smithi151.stdout: "expos": 4194304, 2024-06-03T03:28:48.229 INFO:teuthology.orchestra.run.smithi151.stdout: "wrpos": 4194892, 2024-06-03T03:28:48.229 INFO:teuthology.orchestra.run.smithi151.stdout: "rdpos": 4194352, 2024-06-03T03:28:48.229 INFO:teuthology.orchestra.run.smithi151.stdout: "jlat": { 2024-06-03T03:28:48.229 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:48.229 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:48.229 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:48.229 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:48.229 INFO:teuthology.orchestra.run.smithi151.stdout: "replayed": 2 2024-06-03T03:28:48.229 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:48.230 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_mem": { 2024-06-03T03:28:48.230 INFO:teuthology.orchestra.run.smithi151.stdout: "ino": 3, 2024-06-03T03:28:48.230 INFO:teuthology.orchestra.run.smithi151.stdout: "ino+": 3, 2024-06-03T03:28:48.230 INFO:teuthology.orchestra.run.smithi151.stdout: "ino-": 0, 2024-06-03T03:28:48.230 INFO:teuthology.orchestra.run.smithi151.stdout: "dir": 1, 2024-06-03T03:28:48.230 INFO:teuthology.orchestra.run.smithi151.stdout: "dir+": 1, 2024-06-03T03:28:48.230 INFO:teuthology.orchestra.run.smithi151.stdout: "dir-": 0, 2024-06-03T03:28:48.230 INFO:teuthology.orchestra.run.smithi151.stdout: "dn": 0, 2024-06-03T03:28:48.230 INFO:teuthology.orchestra.run.smithi151.stdout: "dn+": 0, 2024-06-03T03:28:48.230 INFO:teuthology.orchestra.run.smithi151.stdout: "dn-": 0, 2024-06-03T03:28:48.230 INFO:teuthology.orchestra.run.smithi151.stdout: "cap": 0, 2024-06-03T03:28:48.230 INFO:teuthology.orchestra.run.smithi151.stdout: "cap+": 0, 2024-06-03T03:28:48.230 INFO:teuthology.orchestra.run.smithi151.stdout: "cap-": 0, 2024-06-03T03:28:48.231 INFO:teuthology.orchestra.run.smithi151.stdout: "rss": 49244, 2024-06-03T03:28:48.231 INFO:teuthology.orchestra.run.smithi151.stdout: "heap": 215420 2024-06-03T03:28:48.231 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:48.231 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_server": { 2024-06-03T03:28:48.231 INFO:teuthology.orchestra.run.smithi151.stdout: "dispatch_client_request": 0, 2024-06-03T03:28:48.231 INFO:teuthology.orchestra.run.smithi151.stdout: "dispatch_server_request": 0, 2024-06-03T03:28:48.231 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_request": 0, 2024-06-03T03:28:48.231 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_session": 0, 2024-06-03T03:28:48.231 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_peer_request": 0, 2024-06-03T03:28:48.231 INFO:teuthology.orchestra.run.smithi151.stdout: "req_create_latency": { 2024-06-03T03:28:48.231 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:48.231 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:48.231 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:48.231 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:48.232 INFO:teuthology.orchestra.run.smithi151.stdout: "req_getattr_latency": { 2024-06-03T03:28:48.232 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:48.232 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:48.232 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:48.232 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:48.232 INFO:teuthology.orchestra.run.smithi151.stdout: "req_getfilelock_latency": { 2024-06-03T03:28:48.232 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:48.232 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:48.232 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:48.232 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:48.232 INFO:teuthology.orchestra.run.smithi151.stdout: "req_link_latency": { 2024-06-03T03:28:48.232 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:48.232 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:48.232 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:48.233 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:48.233 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookup_latency": { 2024-06-03T03:28:48.233 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:48.233 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:48.233 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:48.233 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:48.233 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookuphash_latency": { 2024-06-03T03:28:48.233 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:48.233 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:48.233 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:48.233 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:48.233 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupino_latency": { 2024-06-03T03:28:48.233 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:48.233 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:48.234 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:48.234 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:48.234 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupname_latency": { 2024-06-03T03:28:48.234 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:48.234 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:48.234 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:48.234 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:48.234 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupparent_latency": { 2024-06-03T03:28:48.234 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:48.234 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:48.234 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:48.234 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:48.234 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupsnap_latency": { 2024-06-03T03:28:48.235 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:48.235 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:48.235 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:48.235 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:48.235 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lssnap_latency": { 2024-06-03T03:28:48.235 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:48.235 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:48.235 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:48.235 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:48.235 INFO:teuthology.orchestra.run.smithi151.stdout: "req_mkdir_latency": { 2024-06-03T03:28:48.235 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:48.236 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:48.236 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:48.236 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:48.236 INFO:teuthology.orchestra.run.smithi151.stdout: "req_mknod_latency": { 2024-06-03T03:28:48.236 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:48.236 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:48.236 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:48.237 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:48.237 INFO:teuthology.orchestra.run.smithi151.stdout: "req_mksnap_latency": { 2024-06-03T03:28:48.237 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:48.237 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:48.237 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:48.237 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:48.237 INFO:teuthology.orchestra.run.smithi151.stdout: "req_open_latency": { 2024-06-03T03:28:48.237 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:48.237 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:48.237 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:48.237 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:48.237 INFO:teuthology.orchestra.run.smithi151.stdout: "req_readdir_latency": { 2024-06-03T03:28:48.237 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:48.237 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:48.238 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:48.238 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:48.238 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rename_latency": { 2024-06-03T03:28:48.238 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:48.238 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:48.238 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:48.238 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:48.238 INFO:teuthology.orchestra.run.smithi151.stdout: "req_renamesnap_latency": { 2024-06-03T03:28:48.238 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:48.238 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:48.238 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:48.238 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:48.239 INFO:teuthology.orchestra.run.smithi151.stdout: "req_snapdiff_latency": { 2024-06-03T03:28:48.239 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:48.239 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:48.239 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:48.239 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:48.239 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rmdir_latency": { 2024-06-03T03:28:48.239 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:48.239 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:48.239 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:48.239 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:48.239 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rmsnap_latency": { 2024-06-03T03:28:48.239 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:48.240 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:48.240 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:48.240 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:48.240 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rmxattr_latency": { 2024-06-03T03:28:48.240 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:48.240 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:48.240 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:48.240 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:48.240 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setattr_latency": { 2024-06-03T03:28:48.240 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:48.240 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:48.240 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:48.240 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:48.240 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setdirlayout_latency": { 2024-06-03T03:28:48.241 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:48.241 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:48.241 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:48.241 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:48.241 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setfilelock_latency": { 2024-06-03T03:28:48.241 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:48.241 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:48.241 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:48.241 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:48.241 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setlayout_latency": { 2024-06-03T03:28:48.241 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:48.241 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:48.241 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:48.241 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:48.242 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setxattr_latency": { 2024-06-03T03:28:48.242 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:48.242 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:48.242 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:48.242 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:48.242 INFO:teuthology.orchestra.run.smithi151.stdout: "req_symlink_latency": { 2024-06-03T03:28:48.242 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:48.242 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:48.242 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:48.242 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:48.242 INFO:teuthology.orchestra.run.smithi151.stdout: "req_unlink_latency": { 2024-06-03T03:28:48.242 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:48.242 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:48.243 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:48.243 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:48.243 INFO:teuthology.orchestra.run.smithi151.stdout: "cap_revoke_eviction": 0, 2024-06-03T03:28:48.243 INFO:teuthology.orchestra.run.smithi151.stdout: "cap_acquisition_throttle": 0, 2024-06-03T03:28:48.243 INFO:teuthology.orchestra.run.smithi151.stdout: "req_getvxattr_latency": { 2024-06-03T03:28:48.243 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:48.243 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:48.243 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:48.243 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-06-03T03:28:48.243 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:48.243 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_sessions": { 2024-06-03T03:28:48.243 INFO:teuthology.orchestra.run.smithi151.stdout: "session_count": 0, 2024-06-03T03:28:48.243 INFO:teuthology.orchestra.run.smithi151.stdout: "session_add": 0, 2024-06-03T03:28:48.244 INFO:teuthology.orchestra.run.smithi151.stdout: "session_remove": 0, 2024-06-03T03:28:48.244 INFO:teuthology.orchestra.run.smithi151.stdout: "sessions_open": 0, 2024-06-03T03:28:48.244 INFO:teuthology.orchestra.run.smithi151.stdout: "sessions_stale": 0, 2024-06-03T03:28:48.244 INFO:teuthology.orchestra.run.smithi151.stdout: "total_load": 0, 2024-06-03T03:28:48.244 INFO:teuthology.orchestra.run.smithi151.stdout: "average_load": 0, 2024-06-03T03:28:48.244 INFO:teuthology.orchestra.run.smithi151.stdout: "avg_session_uptime": 0, 2024-06-03T03:28:48.244 INFO:teuthology.orchestra.run.smithi151.stdout: "mdthresh_evicted": 0 2024-06-03T03:28:48.244 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:48.244 INFO:teuthology.orchestra.run.smithi151.stdout: "mempool": { 2024-06-03T03:28:48.244 INFO:teuthology.orchestra.run.smithi151.stdout: "bloom_filter_bytes": 0, 2024-06-03T03:28:48.244 INFO:teuthology.orchestra.run.smithi151.stdout: "bloom_filter_items": 0, 2024-06-03T03:28:48.244 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_alloc_bytes": 0, 2024-06-03T03:28:48.244 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_alloc_items": 0, 2024-06-03T03:28:48.245 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_data_bytes": 0, 2024-06-03T03:28:48.245 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_data_items": 0, 2024-06-03T03:28:48.245 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-03T03:28:48.245 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_onode_items": 0, 2024-06-03T03:28:48.245 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-03T03:28:48.245 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_meta_items": 0, 2024-06-03T03:28:48.245 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_other_bytes": 0, 2024-06-03T03:28:48.245 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_other_items": 0, 2024-06-03T03:28:48.245 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-03T03:28:48.245 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_buffer_items": 0, 2024-06-03T03:28:48.245 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_extent_bytes": 0, 2024-06-03T03:28:48.245 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_extent_items": 0, 2024-06-03T03:28:48.245 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_blob_bytes": 0, 2024-06-03T03:28:48.246 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_blob_items": 0, 2024-06-03T03:28:48.246 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-03T03:28:48.246 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_shared_blob_items": 0, 2024-06-03T03:28:48.246 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-03T03:28:48.246 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_inline_bl_items": 0, 2024-06-03T03:28:48.246 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_fsck_bytes": 0, 2024-06-03T03:28:48.246 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_fsck_items": 0, 2024-06-03T03:28:48.246 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_txc_bytes": 0, 2024-06-03T03:28:48.246 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_txc_items": 0, 2024-06-03T03:28:48.246 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-03T03:28:48.246 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_deferred_items": 0, 2024-06-03T03:28:48.246 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_bytes": 0, 2024-06-03T03:28:48.247 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_items": 0, 2024-06-03T03:28:48.247 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_bytes": 0, 2024-06-03T03:28:48.247 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_items": 0, 2024-06-03T03:28:48.247 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_reader_bytes": 0, 2024-06-03T03:28:48.247 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_reader_items": 0, 2024-06-03T03:28:48.247 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_writer_bytes": 0, 2024-06-03T03:28:48.247 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_writer_items": 0, 2024-06-03T03:28:48.247 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_anon_bytes": 173201, 2024-06-03T03:28:48.247 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_anon_items": 42, 2024-06-03T03:28:48.247 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_meta_bytes": 0, 2024-06-03T03:28:48.247 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_meta_items": 0, 2024-06-03T03:28:48.247 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_bytes": 0, 2024-06-03T03:28:48.247 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_items": 0, 2024-06-03T03:28:48.247 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_mapbl_bytes": 0, 2024-06-03T03:28:48.248 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_mapbl_items": 0, 2024-06-03T03:28:48.248 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_pglog_bytes": 0, 2024-06-03T03:28:48.248 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_pglog_items": 0, 2024-06-03T03:28:48.248 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_bytes": 8888, 2024-06-03T03:28:48.248 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_items": 170, 2024-06-03T03:28:48.248 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_mapping_bytes": 0, 2024-06-03T03:28:48.248 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_mapping_items": 0, 2024-06-03T03:28:48.248 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap_bytes": 0, 2024-06-03T03:28:48.248 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap_items": 0, 2024-06-03T03:28:48.248 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_co_bytes": 8464, 2024-06-03T03:28:48.248 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_co_items": 13, 2024-06-03T03:28:48.248 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_1_bytes": 0, 2024-06-03T03:28:48.248 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_1_items": 0, 2024-06-03T03:28:48.248 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_2_bytes": 0, 2024-06-03T03:28:48.248 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_2_items": 0 2024-06-03T03:28:48.249 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:48.249 INFO:teuthology.orchestra.run.smithi151.stdout: "objecter": { 2024-06-03T03:28:48.249 INFO:teuthology.orchestra.run.smithi151.stdout: "op_active": 0, 2024-06-03T03:28:48.249 INFO:teuthology.orchestra.run.smithi151.stdout: "op_laggy": 0, 2024-06-03T03:28:48.249 INFO:teuthology.orchestra.run.smithi151.stdout: "op_send": 926, 2024-06-03T03:28:48.249 INFO:teuthology.orchestra.run.smithi151.stdout: "op_send_bytes": 21, 2024-06-03T03:28:48.249 INFO:teuthology.orchestra.run.smithi151.stdout: "op_resend": 3, 2024-06-03T03:28:48.249 INFO:teuthology.orchestra.run.smithi151.stdout: "op_reply": 926, 2024-06-03T03:28:48.249 INFO:teuthology.orchestra.run.smithi151.stdout: "op_latency": { 2024-06-03T03:28:48.249 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 926, 2024-06-03T03:28:48.249 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.647991371, 2024-06-03T03:28:48.249 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000699774 2024-06-03T03:28:48.249 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:48.249 INFO:teuthology.orchestra.run.smithi151.stdout: "op_inflight": 0, 2024-06-03T03:28:48.250 INFO:teuthology.orchestra.run.smithi151.stdout: "oplen_avg": { 2024-06-03T03:28:48.250 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 926, 2024-06-03T03:28:48.250 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 927 2024-06-03T03:28:48.250 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:48.250 INFO:teuthology.orchestra.run.smithi151.stdout: "op": 926, 2024-06-03T03:28:48.250 INFO:teuthology.orchestra.run.smithi151.stdout: "op_r": 926, 2024-06-03T03:28:48.250 INFO:teuthology.orchestra.run.smithi151.stdout: "op_w": 0, 2024-06-03T03:28:48.250 INFO:teuthology.orchestra.run.smithi151.stdout: "op_rmw": 0, 2024-06-03T03:28:48.250 INFO:teuthology.orchestra.run.smithi151.stdout: "op_pg": 0, 2024-06-03T03:28:48.250 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_stat": 460, 2024-06-03T03:28:48.250 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_create": 0, 2024-06-03T03:28:48.250 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_read": 464, 2024-06-03T03:28:48.250 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_write": 0, 2024-06-03T03:28:48.251 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_writefull": 0, 2024-06-03T03:28:48.251 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_writesame": 0, 2024-06-03T03:28:48.251 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_append": 0, 2024-06-03T03:28:48.251 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_zero": 0, 2024-06-03T03:28:48.251 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_truncate": 0, 2024-06-03T03:28:48.251 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_delete": 0, 2024-06-03T03:28:48.251 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_mapext": 0, 2024-06-03T03:28:48.251 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_sparse_read": 0, 2024-06-03T03:28:48.251 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_clonerange": 0, 2024-06-03T03:28:48.251 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_getxattr": 1, 2024-06-03T03:28:48.251 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_setxattr": 0, 2024-06-03T03:28:48.251 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_cmpxattr": 0, 2024-06-03T03:28:48.251 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_rmxattr": 0, 2024-06-03T03:28:48.252 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_resetxattrs": 0, 2024-06-03T03:28:48.252 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_call": 0, 2024-06-03T03:28:48.252 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_watch": 0, 2024-06-03T03:28:48.252 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_notify": 0, 2024-06-03T03:28:48.252 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_src_cmpxattr": 0, 2024-06-03T03:28:48.252 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_pgls": 0, 2024-06-03T03:28:48.252 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_pgls_filter": 0, 2024-06-03T03:28:48.252 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_other": 0, 2024-06-03T03:28:48.252 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_active": 0, 2024-06-03T03:28:48.252 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_send": 0, 2024-06-03T03:28:48.252 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_resend": 0, 2024-06-03T03:28:48.252 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_ping": 0, 2024-06-03T03:28:48.252 INFO:teuthology.orchestra.run.smithi151.stdout: "poolop_active": 0, 2024-06-03T03:28:48.252 INFO:teuthology.orchestra.run.smithi151.stdout: "poolop_send": 0, 2024-06-03T03:28:48.253 INFO:teuthology.orchestra.run.smithi151.stdout: "poolop_resend": 0, 2024-06-03T03:28:48.253 INFO:teuthology.orchestra.run.smithi151.stdout: "poolstat_active": 0, 2024-06-03T03:28:48.253 INFO:teuthology.orchestra.run.smithi151.stdout: "poolstat_send": 0, 2024-06-03T03:28:48.253 INFO:teuthology.orchestra.run.smithi151.stdout: "poolstat_resend": 0, 2024-06-03T03:28:48.253 INFO:teuthology.orchestra.run.smithi151.stdout: "statfs_active": 0, 2024-06-03T03:28:48.253 INFO:teuthology.orchestra.run.smithi151.stdout: "statfs_send": 0, 2024-06-03T03:28:48.253 INFO:teuthology.orchestra.run.smithi151.stdout: "statfs_resend": 0, 2024-06-03T03:28:48.253 INFO:teuthology.orchestra.run.smithi151.stdout: "command_active": 0, 2024-06-03T03:28:48.253 INFO:teuthology.orchestra.run.smithi151.stdout: "command_send": 0, 2024-06-03T03:28:48.253 INFO:teuthology.orchestra.run.smithi151.stdout: "command_resend": 0, 2024-06-03T03:28:48.253 INFO:teuthology.orchestra.run.smithi151.stdout: "map_epoch": 99, 2024-06-03T03:28:48.253 INFO:teuthology.orchestra.run.smithi151.stdout: "map_full": 0, 2024-06-03T03:28:48.253 INFO:teuthology.orchestra.run.smithi151.stdout: "map_inc": 2, 2024-06-03T03:28:48.253 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_sessions": 3, 2024-06-03T03:28:48.253 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_session_open": 3, 2024-06-03T03:28:48.254 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_session_close": 0, 2024-06-03T03:28:48.254 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_laggy": 0, 2024-06-03T03:28:48.254 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_wr": 0, 2024-06-03T03:28:48.254 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_rd": 2, 2024-06-03T03:28:48.254 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_del": 0 2024-06-03T03:28:48.254 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:48.254 INFO:teuthology.orchestra.run.smithi151.stdout: "oft": { 2024-06-03T03:28:48.254 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_objs": 0, 2024-06-03T03:28:48.254 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_kv_pairs": 0, 2024-06-03T03:28:48.254 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_updates": 0, 2024-06-03T03:28:48.254 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_removes": 0 2024-06-03T03:28:48.254 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:48.254 INFO:teuthology.orchestra.run.smithi151.stdout: "osd-slow-ops": { 2024-06-03T03:28:48.254 INFO:teuthology.orchestra.run.smithi151.stdout: "slow_ops_count": 0 2024-06-03T03:28:48.255 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:48.255 INFO:teuthology.orchestra.run.smithi151.stdout: "purge_queue": { 2024-06-03T03:28:48.255 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing_ops": 0, 2024-06-03T03:28:48.255 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing_ops_high_water": 0, 2024-06-03T03:28:48.255 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing": 0, 2024-06-03T03:28:48.255 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing_high_water": 0, 2024-06-03T03:28:48.255 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executed_ops": 0, 2024-06-03T03:28:48.255 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executed": 0, 2024-06-03T03:28:48.255 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_item_in_journal": 0 2024-06-03T03:28:48.255 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:48.255 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-03T03:28:48.255 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-06-03T03:28:48.255 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 104857600, 2024-06-03T03:28:48.255 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-06-03T03:28:48.256 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 1100, 2024-06-03T03:28:48.256 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 939016, 2024-06-03T03:28:48.256 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-06-03T03:28:48.256 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 1100, 2024-06-03T03:28:48.256 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-06-03T03:28:48.256 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-06-03T03:28:48.256 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 1100, 2024-06-03T03:28:48.256 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 939016, 2024-06-03T03:28:48.256 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-06-03T03:28:48.256 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:48.256 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:48.256 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:48.256 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-06-03T03:28:48.256 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:48.257 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-objecter_bytes": { 2024-06-03T03:28:48.257 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-06-03T03:28:48.257 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 104857600, 2024-06-03T03:28:48.257 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-06-03T03:28:48.257 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-06-03T03:28:48.257 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-06-03T03:28:48.257 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-06-03T03:28:48.257 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-06-03T03:28:48.257 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 926, 2024-06-03T03:28:48.257 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 593, 2024-06-03T03:28:48.257 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 926, 2024-06-03T03:28:48.257 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 593, 2024-06-03T03:28:48.257 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-06-03T03:28:48.257 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:48.258 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:48.258 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:48.258 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-06-03T03:28:48.258 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:48.258 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-objecter_ops": { 2024-06-03T03:28:48.258 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-06-03T03:28:48.258 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 1024, 2024-06-03T03:28:48.258 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-06-03T03:28:48.258 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-06-03T03:28:48.258 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-06-03T03:28:48.258 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-06-03T03:28:48.258 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-06-03T03:28:48.258 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 926, 2024-06-03T03:28:48.259 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 926, 2024-06-03T03:28:48.259 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 926, 2024-06-03T03:28:48.259 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 926, 2024-06-03T03:28:48.259 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-06-03T03:28:48.259 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:48.259 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:48.259 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:48.259 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-06-03T03:28:48.259 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:48.259 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-write_buf_throttle": { 2024-06-03T03:28:48.259 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-06-03T03:28:48.259 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 3758096384, 2024-06-03T03:28:48.259 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-06-03T03:28:48.259 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-06-03T03:28:48.260 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-06-03T03:28:48.260 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-06-03T03:28:48.260 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-06-03T03:28:48.260 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-06-03T03:28:48.260 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-06-03T03:28:48.260 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 0, 2024-06-03T03:28:48.260 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 0, 2024-06-03T03:28:48.260 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-06-03T03:28:48.260 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:48.260 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:48.260 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:48.260 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-06-03T03:28:48.260 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:48.260 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-write_buf_throttle-0x556067c546e0": { 2024-06-03T03:28:48.260 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-06-03T03:28:48.261 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 3758096384, 2024-06-03T03:28:48.261 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-06-03T03:28:48.261 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-06-03T03:28:48.261 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-06-03T03:28:48.261 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-06-03T03:28:48.261 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-06-03T03:28:48.261 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-06-03T03:28:48.261 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-06-03T03:28:48.261 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 0, 2024-06-03T03:28:48.261 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 0, 2024-06-03T03:28:48.261 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-06-03T03:28:48.261 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:48.261 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:48.261 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:48.262 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-06-03T03:28:48.262 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-06-03T03:28:48.262 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-06-03T03:28:49.022 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-06-03T03:28:49.023 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-06-03T03:28:49.023 DEBUG:tasks.check_counter:Getting stats from h 2024-06-03T03:28:49.023 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph daemon mds.h perf dump 2024-06-03T03:28:49.265 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.b/config 2024-06-03T03:28:49.386 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:28:49 smithi151 ceph-mon[37043]: pgmap v611: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-03T03:28:49.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:28:49 smithi117 ceph-mon[31649]: pgmap v611: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-03T03:28:49.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:28:49 smithi183 ceph-mon[37326]: pgmap v611: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-03T03:28:51.036 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-06-03T03:28:51.037 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-0": { 2024-06-03T03:28:51.037 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 466, 2024-06-03T03:28:51.037 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 468, 2024-06-03T03:28:51.037 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 114742, 2024-06-03T03:28:51.037 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 148195, 2024-06-03T03:28:51.037 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 6, 2024-06-03T03:28:51.037 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 2, 2024-06-03T03:28:51.037 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.041901393, 2024-06-03T03:28:51.037 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.011472446, 2024-06-03T03:28:51.037 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.009985710, 2024-06-03T03:28:51.037 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.016209616, 2024-06-03T03:28:51.038 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T03:28:51.038 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 466, 2024-06-03T03:28:51.038 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.014311427, 2024-06-03T03:28:51.038 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000030711 2024-06-03T03:28:51.038 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:51.038 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-06-03T03:28:51.038 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:51.038 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:51.038 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:51.038 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:51.038 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-03T03:28:51.038 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 0 2024-06-03T03:28:51.038 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:51.039 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-1": { 2024-06-03T03:28:51.039 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 1018, 2024-06-03T03:28:51.039 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 1137, 2024-06-03T03:28:51.039 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 970413, 2024-06-03T03:28:51.039 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 581979, 2024-06-03T03:28:51.039 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 5, 2024-06-03T03:28:51.039 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 4, 2024-06-03T03:28:51.039 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.100271256, 2024-06-03T03:28:51.039 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.041522563, 2024-06-03T03:28:51.039 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.022634735, 2024-06-03T03:28:51.039 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.026640299, 2024-06-03T03:28:51.039 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T03:28:51.040 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 1135, 2024-06-03T03:28:51.040 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.047290310, 2024-06-03T03:28:51.040 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000041665 2024-06-03T03:28:51.040 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:51.040 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-06-03T03:28:51.040 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:51.040 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:51.040 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:51.040 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:51.040 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 160, 2024-06-03T03:28:51.040 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 258512 2024-06-03T03:28:51.040 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:51.040 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-2": { 2024-06-03T03:28:51.041 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 180, 2024-06-03T03:28:51.041 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 171, 2024-06-03T03:28:51.041 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 773090, 2024-06-03T03:28:51.041 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 99443, 2024-06-03T03:28:51.041 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 4, 2024-06-03T03:28:51.041 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 3, 2024-06-03T03:28:51.041 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.031442782, 2024-06-03T03:28:51.041 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.010645142, 2024-06-03T03:28:51.041 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.020030020, 2024-06-03T03:28:51.041 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.003977804, 2024-06-03T03:28:51.041 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T03:28:51.041 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 171, 2024-06-03T03:28:51.041 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.013047869, 2024-06-03T03:28:51.042 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000076303 2024-06-03T03:28:51.042 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:51.042 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-06-03T03:28:51.042 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:51.042 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:51.042 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:51.042 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:51.042 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 771744, 2024-06-03T03:28:51.042 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 98480 2024-06-03T03:28:51.042 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:51.042 INFO:teuthology.orchestra.run.smithi151.stdout: "cct": { 2024-06-03T03:28:51.042 INFO:teuthology.orchestra.run.smithi151.stdout: "total_workers": 1, 2024-06-03T03:28:51.042 INFO:teuthology.orchestra.run.smithi151.stdout: "unhealthy_workers": 0 2024-06-03T03:28:51.043 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:51.043 INFO:teuthology.orchestra.run.smithi151.stdout: "finisher-MDSRank": { 2024-06-03T03:28:51.043 INFO:teuthology.orchestra.run.smithi151.stdout: "queue_len": 0, 2024-06-03T03:28:51.043 INFO:teuthology.orchestra.run.smithi151.stdout: "complete_latency": { 2024-06-03T03:28:51.043 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 2495, 2024-06-03T03:28:51.043 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.061623808, 2024-06-03T03:28:51.043 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000024698 2024-06-03T03:28:51.043 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-06-03T03:28:51.043 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:51.043 INFO:teuthology.orchestra.run.smithi151.stdout: "finisher-PurgeQueue": { 2024-06-03T03:28:51.043 INFO:teuthology.orchestra.run.smithi151.stdout: "queue_len": 0, 2024-06-03T03:28:51.043 INFO:teuthology.orchestra.run.smithi151.stdout: "complete_latency": { 2024-06-03T03:28:51.043 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:51.044 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:51.044 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:51.044 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-06-03T03:28:51.044 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:51.044 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": { 2024-06-03T03:28:51.044 INFO:teuthology.orchestra.run.smithi151.stdout: "request": 0, 2024-06-03T03:28:51.044 INFO:teuthology.orchestra.run.smithi151.stdout: "reply": 0, 2024-06-03T03:28:51.044 INFO:teuthology.orchestra.run.smithi151.stdout: "reply_latency": { 2024-06-03T03:28:51.044 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:51.044 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:51.044 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:51.044 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:51.044 INFO:teuthology.orchestra.run.smithi151.stdout: "slow_reply": 0, 2024-06-03T03:28:51.044 INFO:teuthology.orchestra.run.smithi151.stdout: "forward": 0, 2024-06-03T03:28:51.045 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_fetch_complete": 0, 2024-06-03T03:28:51.045 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_fetch_keys": 0, 2024-06-03T03:28:51.045 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_commit": 0, 2024-06-03T03:28:51.045 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_split": 0, 2024-06-03T03:28:51.045 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_merge": 0, 2024-06-03T03:28:51.045 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes": 20, 2024-06-03T03:28:51.045 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_top": 4, 2024-06-03T03:28:51.045 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_bottom": 16, 2024-06-03T03:28:51.045 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_pin_tail": 0, 2024-06-03T03:28:51.045 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_pinned": 13, 2024-06-03T03:28:51.045 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_expired": 0, 2024-06-03T03:28:51.045 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_with_caps": 0, 2024-06-03T03:28:51.045 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": 0, 2024-06-03T03:28:51.045 INFO:teuthology.orchestra.run.smithi151.stdout: "subtrees": 2, 2024-06-03T03:28:51.046 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse": 0, 2024-06-03T03:28:51.046 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_hit": 0, 2024-06-03T03:28:51.046 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_forward": 0, 2024-06-03T03:28:51.046 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_discover": 0, 2024-06-03T03:28:51.046 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_dir_fetch": 0, 2024-06-03T03:28:51.046 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_remote_ino": 0, 2024-06-03T03:28:51.046 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_lock": 0, 2024-06-03T03:28:51.046 INFO:teuthology.orchestra.run.smithi151.stdout: "load_cent": 0, 2024-06-03T03:28:51.046 INFO:teuthology.orchestra.run.smithi151.stdout: "q": 0, 2024-06-03T03:28:51.046 INFO:teuthology.orchestra.run.smithi151.stdout: "exported": 0, 2024-06-03T03:28:51.046 INFO:teuthology.orchestra.run.smithi151.stdout: "exported_inodes": 0, 2024-06-03T03:28:51.046 INFO:teuthology.orchestra.run.smithi151.stdout: "imported": 0, 2024-06-03T03:28:51.046 INFO:teuthology.orchestra.run.smithi151.stdout: "imported_inodes": 0, 2024-06-03T03:28:51.047 INFO:teuthology.orchestra.run.smithi151.stdout: "openino_dir_fetch": 0, 2024-06-03T03:28:51.047 INFO:teuthology.orchestra.run.smithi151.stdout: "openino_backtrace_fetch": 0, 2024-06-03T03:28:51.047 INFO:teuthology.orchestra.run.smithi151.stdout: "openino_peer_discover": 0, 2024-06-03T03:28:51.047 INFO:teuthology.orchestra.run.smithi151.stdout: "root_rfiles": 4, 2024-06-03T03:28:51.047 INFO:teuthology.orchestra.run.smithi151.stdout: "root_rbytes": 234, 2024-06-03T03:28:51.047 INFO:teuthology.orchestra.run.smithi151.stdout: "root_rsnaps": 0, 2024-06-03T03:28:51.047 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_backtrace_fetch": 0, 2024-06-03T03:28:51.047 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_set_tag": 0, 2024-06-03T03:28:51.047 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_backtrace_repaired": 0, 2024-06-03T03:28:51.047 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_inotable_repaired": 0, 2024-06-03T03:28:51.047 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_dir_inodes": 0, 2024-06-03T03:28:51.047 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_dir_base_inodes": 0, 2024-06-03T03:28:51.047 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_dirfrag_rstats": 0, 2024-06-03T03:28:51.048 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_file_inodes": 0, 2024-06-03T03:28:51.048 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_inode_file_caps": 0, 2024-06-03T03:28:51.048 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_revoke": 0, 2024-06-03T03:28:51.048 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_grant": 0, 2024-06-03T03:28:51.048 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_trunc": 0, 2024-06-03T03:28:51.048 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-03T03:28:51.048 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-03T03:28:51.048 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_caps": 0, 2024-06-03T03:28:51.048 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_caps_dirty": 0, 2024-06-03T03:28:51.048 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_cap_release": 0, 2024-06-03T03:28:51.048 INFO:teuthology.orchestra.run.smithi151.stdout: "process_request_cap_release": 0 2024-06-03T03:28:51.048 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:51.048 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_cache": { 2024-06-03T03:28:51.048 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_update": 0, 2024-06-03T03:28:51.049 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_update_receipt": 0, 2024-06-03T03:28:51.049 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_try_discover": 0, 2024-06-03T03:28:51.049 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_send_discover": 0, 2024-06-03T03:28:51.049 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_handle_discover": 0, 2024-06-03T03:28:51.049 INFO:teuthology.orchestra.run.smithi151.stdout: "num_strays": 0, 2024-06-03T03:28:51.049 INFO:teuthology.orchestra.run.smithi151.stdout: "num_strays_delayed": 0, 2024-06-03T03:28:51.049 INFO:teuthology.orchestra.run.smithi151.stdout: "num_strays_enqueuing": 0, 2024-06-03T03:28:51.049 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_created": 0, 2024-06-03T03:28:51.049 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_enqueued": 0, 2024-06-03T03:28:51.049 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_reintegrated": 0, 2024-06-03T03:28:51.049 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_migrated": 0, 2024-06-03T03:28:51.049 INFO:teuthology.orchestra.run.smithi151.stdout: "num_recovering_processing": 0, 2024-06-03T03:28:51.049 INFO:teuthology.orchestra.run.smithi151.stdout: "num_recovering_enqueued": 0, 2024-06-03T03:28:51.050 INFO:teuthology.orchestra.run.smithi151.stdout: "num_recovering_prioritized": 0, 2024-06-03T03:28:51.050 INFO:teuthology.orchestra.run.smithi151.stdout: "recovery_started": 0, 2024-06-03T03:28:51.050 INFO:teuthology.orchestra.run.smithi151.stdout: "recovery_completed": 0, 2024-06-03T03:28:51.050 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_quiesce_path": 0, 2024-06-03T03:28:51.050 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_quiesce_inode": 0, 2024-06-03T03:28:51.050 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_enqueue_scrub": 0, 2024-06-03T03:28:51.050 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_exportdir": 0, 2024-06-03T03:28:51.050 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_flush": 0, 2024-06-03T03:28:51.050 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_fragmentdir": 0, 2024-06-03T03:28:51.050 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_fragstats": 0, 2024-06-03T03:28:51.050 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_inodestats": 0 2024-06-03T03:28:51.050 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:51.051 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_log": { 2024-06-03T03:28:51.051 INFO:teuthology.orchestra.run.smithi151.stdout: "evlrg": 0, 2024-06-03T03:28:51.051 INFO:teuthology.orchestra.run.smithi151.stdout: "evadd": 0, 2024-06-03T03:28:51.051 INFO:teuthology.orchestra.run.smithi151.stdout: "evex": 0, 2024-06-03T03:28:51.051 INFO:teuthology.orchestra.run.smithi151.stdout: "evtrm": 0, 2024-06-03T03:28:51.051 INFO:teuthology.orchestra.run.smithi151.stdout: "ev": 189, 2024-06-03T03:28:51.051 INFO:teuthology.orchestra.run.smithi151.stdout: "evexg": 0, 2024-06-03T03:28:51.051 INFO:teuthology.orchestra.run.smithi151.stdout: "evexd": 0, 2024-06-03T03:28:51.051 INFO:teuthology.orchestra.run.smithi151.stdout: "segadd": 0, 2024-06-03T03:28:51.051 INFO:teuthology.orchestra.run.smithi151.stdout: "segex": 0, 2024-06-03T03:28:51.051 INFO:teuthology.orchestra.run.smithi151.stdout: "segtrm": 0, 2024-06-03T03:28:51.051 INFO:teuthology.orchestra.run.smithi151.stdout: "seg": 2, 2024-06-03T03:28:51.051 INFO:teuthology.orchestra.run.smithi151.stdout: "segmjr": 13, 2024-06-03T03:28:51.051 INFO:teuthology.orchestra.run.smithi151.stdout: "segexg": 0, 2024-06-03T03:28:51.052 INFO:teuthology.orchestra.run.smithi151.stdout: "segexd": 0, 2024-06-03T03:28:51.052 INFO:teuthology.orchestra.run.smithi151.stdout: "expos": 4792199, 2024-06-03T03:28:51.052 INFO:teuthology.orchestra.run.smithi151.stdout: "wrpos": 4864647, 2024-06-03T03:28:51.052 INFO:teuthology.orchestra.run.smithi151.stdout: "rdpos": 4864048, 2024-06-03T03:28:51.052 INFO:teuthology.orchestra.run.smithi151.stdout: "jlat": { 2024-06-03T03:28:51.052 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:51.052 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:51.052 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:51.052 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:51.052 INFO:teuthology.orchestra.run.smithi151.stdout: "replayed": 189 2024-06-03T03:28:51.052 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:51.052 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_mem": { 2024-06-03T03:28:51.053 INFO:teuthology.orchestra.run.smithi151.stdout: "ino": 14, 2024-06-03T03:28:51.053 INFO:teuthology.orchestra.run.smithi151.stdout: "ino+": 20, 2024-06-03T03:28:51.053 INFO:teuthology.orchestra.run.smithi151.stdout: "ino-": 6, 2024-06-03T03:28:51.053 INFO:teuthology.orchestra.run.smithi151.stdout: "dir": 9, 2024-06-03T03:28:51.053 INFO:teuthology.orchestra.run.smithi151.stdout: "dir+": 11, 2024-06-03T03:28:51.053 INFO:teuthology.orchestra.run.smithi151.stdout: "dir-": 2, 2024-06-03T03:28:51.053 INFO:teuthology.orchestra.run.smithi151.stdout: "dn": 20, 2024-06-03T03:28:51.053 INFO:teuthology.orchestra.run.smithi151.stdout: "dn+": 25, 2024-06-03T03:28:51.053 INFO:teuthology.orchestra.run.smithi151.stdout: "dn-": 5, 2024-06-03T03:28:51.053 INFO:teuthology.orchestra.run.smithi151.stdout: "cap": 0, 2024-06-03T03:28:51.053 INFO:teuthology.orchestra.run.smithi151.stdout: "cap+": 0, 2024-06-03T03:28:51.053 INFO:teuthology.orchestra.run.smithi151.stdout: "cap-": 0, 2024-06-03T03:28:51.053 INFO:teuthology.orchestra.run.smithi151.stdout: "rss": 49648, 2024-06-03T03:28:51.054 INFO:teuthology.orchestra.run.smithi151.stdout: "heap": 215420 2024-06-03T03:28:51.054 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:51.054 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_server": { 2024-06-03T03:28:51.054 INFO:teuthology.orchestra.run.smithi151.stdout: "dispatch_client_request": 0, 2024-06-03T03:28:51.054 INFO:teuthology.orchestra.run.smithi151.stdout: "dispatch_server_request": 0, 2024-06-03T03:28:51.054 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_request": 0, 2024-06-03T03:28:51.054 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_session": 0, 2024-06-03T03:28:51.054 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_peer_request": 0, 2024-06-03T03:28:51.054 INFO:teuthology.orchestra.run.smithi151.stdout: "req_create_latency": { 2024-06-03T03:28:51.054 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:51.054 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:51.054 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:51.054 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:51.055 INFO:teuthology.orchestra.run.smithi151.stdout: "req_getattr_latency": { 2024-06-03T03:28:51.055 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:51.055 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:51.055 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:51.055 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:51.055 INFO:teuthology.orchestra.run.smithi151.stdout: "req_getfilelock_latency": { 2024-06-03T03:28:51.055 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:51.055 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:51.055 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:51.055 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:51.055 INFO:teuthology.orchestra.run.smithi151.stdout: "req_link_latency": { 2024-06-03T03:28:51.055 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:51.055 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:51.055 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:51.056 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:51.056 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookup_latency": { 2024-06-03T03:28:51.056 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:51.056 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:51.056 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:51.056 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:51.056 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookuphash_latency": { 2024-06-03T03:28:51.056 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:51.056 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:51.056 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:51.056 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:51.056 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupino_latency": { 2024-06-03T03:28:51.056 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:51.056 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:51.057 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:51.057 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:51.057 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupname_latency": { 2024-06-03T03:28:51.057 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:51.057 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:51.057 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:51.057 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:51.057 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupparent_latency": { 2024-06-03T03:28:51.057 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:51.057 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:51.057 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:51.057 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:51.058 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupsnap_latency": { 2024-06-03T03:28:51.058 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:51.058 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:51.058 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:51.058 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:51.058 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lssnap_latency": { 2024-06-03T03:28:51.058 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:51.058 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:51.058 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:51.058 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:51.058 INFO:teuthology.orchestra.run.smithi151.stdout: "req_mkdir_latency": { 2024-06-03T03:28:51.058 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:51.059 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:51.059 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:51.059 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:51.060 INFO:teuthology.orchestra.run.smithi151.stdout: "req_mknod_latency": { 2024-06-03T03:28:51.060 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:51.060 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:51.060 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:51.060 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:51.060 INFO:teuthology.orchestra.run.smithi151.stdout: "req_mksnap_latency": { 2024-06-03T03:28:51.060 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:51.060 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:51.060 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:51.060 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:51.060 INFO:teuthology.orchestra.run.smithi151.stdout: "req_open_latency": { 2024-06-03T03:28:51.060 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:51.061 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:51.061 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:51.061 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:51.061 INFO:teuthology.orchestra.run.smithi151.stdout: "req_readdir_latency": { 2024-06-03T03:28:51.061 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:51.061 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:51.061 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:51.061 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:51.061 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rename_latency": { 2024-06-03T03:28:51.061 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:51.061 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:51.061 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:51.061 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:51.061 INFO:teuthology.orchestra.run.smithi151.stdout: "req_renamesnap_latency": { 2024-06-03T03:28:51.062 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:51.062 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:51.062 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:51.062 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:51.062 INFO:teuthology.orchestra.run.smithi151.stdout: "req_snapdiff_latency": { 2024-06-03T03:28:51.062 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:51.062 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:51.062 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:51.062 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:51.062 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rmdir_latency": { 2024-06-03T03:28:51.062 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:51.062 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:51.063 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:51.063 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:51.063 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rmsnap_latency": { 2024-06-03T03:28:51.063 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:51.063 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:51.063 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:51.063 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:51.063 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rmxattr_latency": { 2024-06-03T03:28:51.063 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:51.063 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:51.063 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:51.063 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:51.063 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setattr_latency": { 2024-06-03T03:28:51.063 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:51.064 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:51.064 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:51.064 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:51.064 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setdirlayout_latency": { 2024-06-03T03:28:51.064 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:51.064 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:51.064 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:51.064 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:51.064 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setfilelock_latency": { 2024-06-03T03:28:51.064 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:51.064 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:51.064 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:51.064 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:51.064 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setlayout_latency": { 2024-06-03T03:28:51.065 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:51.065 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:51.065 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:51.065 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:51.065 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setxattr_latency": { 2024-06-03T03:28:51.065 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:51.065 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:51.065 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:51.065 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:51.065 INFO:teuthology.orchestra.run.smithi151.stdout: "req_symlink_latency": { 2024-06-03T03:28:51.065 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:51.065 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:51.065 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:51.066 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:51.066 INFO:teuthology.orchestra.run.smithi151.stdout: "req_unlink_latency": { 2024-06-03T03:28:51.066 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:51.066 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:51.066 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:51.066 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:51.066 INFO:teuthology.orchestra.run.smithi151.stdout: "cap_revoke_eviction": 0, 2024-06-03T03:28:51.066 INFO:teuthology.orchestra.run.smithi151.stdout: "cap_acquisition_throttle": 0, 2024-06-03T03:28:51.066 INFO:teuthology.orchestra.run.smithi151.stdout: "req_getvxattr_latency": { 2024-06-03T03:28:51.066 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:51.066 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:51.066 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:51.066 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-06-03T03:28:51.067 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:51.067 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_sessions": { 2024-06-03T03:28:51.067 INFO:teuthology.orchestra.run.smithi151.stdout: "session_count": 1, 2024-06-03T03:28:51.067 INFO:teuthology.orchestra.run.smithi151.stdout: "session_add": 4, 2024-06-03T03:28:51.067 INFO:teuthology.orchestra.run.smithi151.stdout: "session_remove": 3, 2024-06-03T03:28:51.067 INFO:teuthology.orchestra.run.smithi151.stdout: "sessions_open": 2, 2024-06-03T03:28:51.067 INFO:teuthology.orchestra.run.smithi151.stdout: "sessions_stale": 0, 2024-06-03T03:28:51.067 INFO:teuthology.orchestra.run.smithi151.stdout: "total_load": 0, 2024-06-03T03:28:51.067 INFO:teuthology.orchestra.run.smithi151.stdout: "average_load": 0, 2024-06-03T03:28:51.067 INFO:teuthology.orchestra.run.smithi151.stdout: "avg_session_uptime": 3727, 2024-06-03T03:28:51.067 INFO:teuthology.orchestra.run.smithi151.stdout: "mdthresh_evicted": 0 2024-06-03T03:28:51.067 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:51.067 INFO:teuthology.orchestra.run.smithi151.stdout: "mempool": { 2024-06-03T03:28:51.068 INFO:teuthology.orchestra.run.smithi151.stdout: "bloom_filter_bytes": 0, 2024-06-03T03:28:51.068 INFO:teuthology.orchestra.run.smithi151.stdout: "bloom_filter_items": 0, 2024-06-03T03:28:51.068 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_alloc_bytes": 0, 2024-06-03T03:28:51.068 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_alloc_items": 0, 2024-06-03T03:28:51.068 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_data_bytes": 0, 2024-06-03T03:28:51.068 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_data_items": 0, 2024-06-03T03:28:51.068 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-03T03:28:51.068 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_onode_items": 0, 2024-06-03T03:28:51.068 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-03T03:28:51.068 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_meta_items": 0, 2024-06-03T03:28:51.068 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_other_bytes": 0, 2024-06-03T03:28:51.068 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_other_items": 0, 2024-06-03T03:28:51.068 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-03T03:28:51.068 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_buffer_items": 0, 2024-06-03T03:28:51.069 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_extent_bytes": 0, 2024-06-03T03:28:51.069 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_extent_items": 0, 2024-06-03T03:28:51.069 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_blob_bytes": 0, 2024-06-03T03:28:51.069 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_blob_items": 0, 2024-06-03T03:28:51.069 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-03T03:28:51.069 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_shared_blob_items": 0, 2024-06-03T03:28:51.069 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-03T03:28:51.069 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_inline_bl_items": 0, 2024-06-03T03:28:51.069 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_fsck_bytes": 0, 2024-06-03T03:28:51.069 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_fsck_items": 0, 2024-06-03T03:28:51.069 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_txc_bytes": 0, 2024-06-03T03:28:51.069 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_txc_items": 0, 2024-06-03T03:28:51.069 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-03T03:28:51.069 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_deferred_items": 0, 2024-06-03T03:28:51.070 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_bytes": 0, 2024-06-03T03:28:51.070 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_items": 0, 2024-06-03T03:28:51.070 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_bytes": 0, 2024-06-03T03:28:51.070 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_items": 0, 2024-06-03T03:28:51.070 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_reader_bytes": 0, 2024-06-03T03:28:51.070 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_reader_items": 0, 2024-06-03T03:28:51.070 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_writer_bytes": 0, 2024-06-03T03:28:51.070 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_writer_items": 0, 2024-06-03T03:28:51.070 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_anon_bytes": 197463, 2024-06-03T03:28:51.070 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_anon_items": 54, 2024-06-03T03:28:51.070 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_meta_bytes": 0, 2024-06-03T03:28:51.070 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_meta_items": 0, 2024-06-03T03:28:51.070 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_bytes": 0, 2024-06-03T03:28:51.071 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_items": 0, 2024-06-03T03:28:51.071 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_mapbl_bytes": 0, 2024-06-03T03:28:51.071 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_mapbl_items": 0, 2024-06-03T03:28:51.071 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_pglog_bytes": 0, 2024-06-03T03:28:51.071 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_pglog_items": 0, 2024-06-03T03:28:51.071 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_bytes": 8888, 2024-06-03T03:28:51.071 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_items": 170, 2024-06-03T03:28:51.071 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_mapping_bytes": 0, 2024-06-03T03:28:51.071 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_mapping_items": 0, 2024-06-03T03:28:51.071 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap_bytes": 0, 2024-06-03T03:28:51.071 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap_items": 0, 2024-06-03T03:28:51.071 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_co_bytes": 56866, 2024-06-03T03:28:51.071 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_co_items": 273, 2024-06-03T03:28:51.072 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_1_bytes": 0, 2024-06-03T03:28:51.072 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_1_items": 0, 2024-06-03T03:28:51.072 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_2_bytes": 0, 2024-06-03T03:28:51.072 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_2_items": 0 2024-06-03T03:28:51.072 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:51.072 INFO:teuthology.orchestra.run.smithi151.stdout: "objecter": { 2024-06-03T03:28:51.072 INFO:teuthology.orchestra.run.smithi151.stdout: "op_active": 0, 2024-06-03T03:28:51.072 INFO:teuthology.orchestra.run.smithi151.stdout: "op_laggy": 0, 2024-06-03T03:28:51.072 INFO:teuthology.orchestra.run.smithi151.stdout: "op_send": 1486, 2024-06-03T03:28:51.072 INFO:teuthology.orchestra.run.smithi151.stdout: "op_send_bytes": 26, 2024-06-03T03:28:51.072 INFO:teuthology.orchestra.run.smithi151.stdout: "op_resend": 4, 2024-06-03T03:28:51.072 INFO:teuthology.orchestra.run.smithi151.stdout: "op_reply": 1486, 2024-06-03T03:28:51.072 INFO:teuthology.orchestra.run.smithi151.stdout: "op_latency": { 2024-06-03T03:28:51.072 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 1486, 2024-06-03T03:28:51.073 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.577992302, 2024-06-03T03:28:51.073 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000388958 2024-06-03T03:28:51.073 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:51.073 INFO:teuthology.orchestra.run.smithi151.stdout: "op_inflight": 0, 2024-06-03T03:28:51.073 INFO:teuthology.orchestra.run.smithi151.stdout: "oplen_avg": { 2024-06-03T03:28:51.073 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 1486, 2024-06-03T03:28:51.073 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 1487 2024-06-03T03:28:51.073 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:51.073 INFO:teuthology.orchestra.run.smithi151.stdout: "op": 1486, 2024-06-03T03:28:51.073 INFO:teuthology.orchestra.run.smithi151.stdout: "op_r": 1486, 2024-06-03T03:28:51.073 INFO:teuthology.orchestra.run.smithi151.stdout: "op_w": 0, 2024-06-03T03:28:51.073 INFO:teuthology.orchestra.run.smithi151.stdout: "op_rmw": 0, 2024-06-03T03:28:51.073 INFO:teuthology.orchestra.run.smithi151.stdout: "op_pg": 0, 2024-06-03T03:28:51.073 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_stat": 926, 2024-06-03T03:28:51.074 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_create": 0, 2024-06-03T03:28:51.074 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_read": 557, 2024-06-03T03:28:51.074 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_write": 0, 2024-06-03T03:28:51.074 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_writefull": 0, 2024-06-03T03:28:51.074 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_writesame": 0, 2024-06-03T03:28:51.074 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_append": 0, 2024-06-03T03:28:51.074 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_zero": 0, 2024-06-03T03:28:51.074 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_truncate": 0, 2024-06-03T03:28:51.074 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_delete": 0, 2024-06-03T03:28:51.074 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_mapext": 0, 2024-06-03T03:28:51.074 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_sparse_read": 0, 2024-06-03T03:28:51.074 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_clonerange": 0, 2024-06-03T03:28:51.074 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_getxattr": 2, 2024-06-03T03:28:51.075 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_setxattr": 0, 2024-06-03T03:28:51.075 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_cmpxattr": 0, 2024-06-03T03:28:51.075 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_rmxattr": 0, 2024-06-03T03:28:51.075 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_resetxattrs": 0, 2024-06-03T03:28:51.075 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_call": 0, 2024-06-03T03:28:51.075 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_watch": 0, 2024-06-03T03:28:51.075 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_notify": 0, 2024-06-03T03:28:51.075 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_src_cmpxattr": 0, 2024-06-03T03:28:51.075 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_pgls": 0, 2024-06-03T03:28:51.075 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_pgls_filter": 0, 2024-06-03T03:28:51.075 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_other": 0, 2024-06-03T03:28:51.075 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_active": 0, 2024-06-03T03:28:51.075 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_send": 0, 2024-06-03T03:28:51.076 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_resend": 0, 2024-06-03T03:28:51.076 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_ping": 0, 2024-06-03T03:28:51.076 INFO:teuthology.orchestra.run.smithi151.stdout: "poolop_active": 0, 2024-06-03T03:28:51.076 INFO:teuthology.orchestra.run.smithi151.stdout: "poolop_send": 0, 2024-06-03T03:28:51.076 INFO:teuthology.orchestra.run.smithi151.stdout: "poolop_resend": 0, 2024-06-03T03:28:51.076 INFO:teuthology.orchestra.run.smithi151.stdout: "poolstat_active": 0, 2024-06-03T03:28:51.076 INFO:teuthology.orchestra.run.smithi151.stdout: "poolstat_send": 0, 2024-06-03T03:28:51.076 INFO:teuthology.orchestra.run.smithi151.stdout: "poolstat_resend": 0, 2024-06-03T03:28:51.076 INFO:teuthology.orchestra.run.smithi151.stdout: "statfs_active": 0, 2024-06-03T03:28:51.076 INFO:teuthology.orchestra.run.smithi151.stdout: "statfs_send": 0, 2024-06-03T03:28:51.076 INFO:teuthology.orchestra.run.smithi151.stdout: "statfs_resend": 0, 2024-06-03T03:28:51.076 INFO:teuthology.orchestra.run.smithi151.stdout: "command_active": 0, 2024-06-03T03:28:51.076 INFO:teuthology.orchestra.run.smithi151.stdout: "command_send": 0, 2024-06-03T03:28:51.076 INFO:teuthology.orchestra.run.smithi151.stdout: "command_resend": 0, 2024-06-03T03:28:51.077 INFO:teuthology.orchestra.run.smithi151.stdout: "map_epoch": 99, 2024-06-03T03:28:51.077 INFO:teuthology.orchestra.run.smithi151.stdout: "map_full": 0, 2024-06-03T03:28:51.077 INFO:teuthology.orchestra.run.smithi151.stdout: "map_inc": 2, 2024-06-03T03:28:51.077 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_sessions": 6, 2024-06-03T03:28:51.077 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_session_open": 6, 2024-06-03T03:28:51.077 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_session_close": 0, 2024-06-03T03:28:51.077 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_laggy": 0, 2024-06-03T03:28:51.077 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_wr": 0, 2024-06-03T03:28:51.077 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_rd": 2, 2024-06-03T03:28:51.077 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_del": 0 2024-06-03T03:28:51.077 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:51.077 INFO:teuthology.orchestra.run.smithi151.stdout: "oft": { 2024-06-03T03:28:51.077 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_objs": 0, 2024-06-03T03:28:51.078 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_kv_pairs": 0, 2024-06-03T03:28:51.078 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_updates": 0, 2024-06-03T03:28:51.078 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_removes": 0 2024-06-03T03:28:51.078 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:51.078 INFO:teuthology.orchestra.run.smithi151.stdout: "osd-slow-ops": { 2024-06-03T03:28:51.078 INFO:teuthology.orchestra.run.smithi151.stdout: "slow_ops_count": 0 2024-06-03T03:28:51.078 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:51.078 INFO:teuthology.orchestra.run.smithi151.stdout: "purge_queue": { 2024-06-03T03:28:51.078 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing_ops": 0, 2024-06-03T03:28:51.078 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing_ops_high_water": 0, 2024-06-03T03:28:51.078 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing": 0, 2024-06-03T03:28:51.078 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing_high_water": 0, 2024-06-03T03:28:51.078 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executed_ops": 0, 2024-06-03T03:28:51.078 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executed": 0, 2024-06-03T03:28:51.079 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_item_in_journal": 0 2024-06-03T03:28:51.079 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:51.079 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-03T03:28:51.079 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-06-03T03:28:51.079 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 104857600, 2024-06-03T03:28:51.079 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-06-03T03:28:51.079 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 1660, 2024-06-03T03:28:51.079 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 1696998, 2024-06-03T03:28:51.079 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-06-03T03:28:51.079 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 1660, 2024-06-03T03:28:51.079 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-06-03T03:28:51.079 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-06-03T03:28:51.080 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 1660, 2024-06-03T03:28:51.080 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 1696998, 2024-06-03T03:28:51.080 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-06-03T03:28:51.080 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:51.080 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:51.080 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:51.080 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-06-03T03:28:51.080 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:51.080 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-objecter_bytes": { 2024-06-03T03:28:51.080 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-06-03T03:28:51.080 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 104857600, 2024-06-03T03:28:51.080 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-06-03T03:28:51.080 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-06-03T03:28:51.081 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-06-03T03:28:51.081 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-06-03T03:28:51.081 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-06-03T03:28:51.081 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 1486, 2024-06-03T03:28:51.081 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 670353, 2024-06-03T03:28:51.081 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 1486, 2024-06-03T03:28:51.081 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 670353, 2024-06-03T03:28:51.081 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-06-03T03:28:51.081 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:51.081 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:51.081 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:51.081 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-06-03T03:28:51.081 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:51.082 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-objecter_ops": { 2024-06-03T03:28:51.082 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-06-03T03:28:51.082 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 1024, 2024-06-03T03:28:51.082 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-06-03T03:28:51.082 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-06-03T03:28:51.082 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-06-03T03:28:51.082 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-06-03T03:28:51.082 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-06-03T03:28:51.082 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 1486, 2024-06-03T03:28:51.082 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 1486, 2024-06-03T03:28:51.082 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 1486, 2024-06-03T03:28:51.082 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 1486, 2024-06-03T03:28:51.082 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-06-03T03:28:51.083 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:51.083 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:51.083 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:51.083 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-06-03T03:28:51.083 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:51.083 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-write_buf_throttle": { 2024-06-03T03:28:51.083 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-06-03T03:28:51.083 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 3758096384, 2024-06-03T03:28:51.083 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-06-03T03:28:51.083 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-06-03T03:28:51.083 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-06-03T03:28:51.083 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-06-03T03:28:51.083 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-06-03T03:28:51.083 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-06-03T03:28:51.084 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-06-03T03:28:51.084 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 0, 2024-06-03T03:28:51.084 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 0, 2024-06-03T03:28:51.084 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-06-03T03:28:51.084 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:51.084 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:51.084 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:51.084 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-06-03T03:28:51.084 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:51.084 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-write_buf_throttle-0x55e84f6706e0": { 2024-06-03T03:28:51.084 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-06-03T03:28:51.084 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 3758096384, 2024-06-03T03:28:51.084 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-06-03T03:28:51.084 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-06-03T03:28:51.085 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-06-03T03:28:51.085 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-06-03T03:28:51.085 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-06-03T03:28:51.085 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-06-03T03:28:51.085 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-06-03T03:28:51.085 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 0, 2024-06-03T03:28:51.085 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 0, 2024-06-03T03:28:51.085 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-06-03T03:28:51.085 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:51.085 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:51.085 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:51.085 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-06-03T03:28:51.085 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-06-03T03:28:51.085 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-06-03T03:28:51.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:28:51 smithi151 ceph-mon[37043]: pgmap v612: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-06-03T03:28:51.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:28:51 smithi117 ceph-mon[31649]: pgmap v612: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-06-03T03:28:51.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:28:51 smithi183 ceph-mon[37326]: pgmap v612: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-06-03T03:28:51.932 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-06-03T03:28:51.932 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-06-03T03:28:51.932 DEBUG:tasks.check_counter:Getting stats from k 2024-06-03T03:28:51.933 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph daemon mds.k perf dump 2024-06-03T03:28:52.172 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.b/config 2024-06-03T03:28:53.318 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:28:53 smithi151 ceph-mon[37043]: pgmap v613: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 11 op/s 2024-06-03T03:28:53.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:28:53 smithi117 ceph-mon[31649]: pgmap v613: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 11 op/s 2024-06-03T03:28:53.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:28:53 smithi183 ceph-mon[37326]: pgmap v613: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 11 op/s 2024-06-03T03:28:53.908 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-06-03T03:28:53.908 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-0": { 2024-06-03T03:28:53.908 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 469, 2024-06-03T03:28:53.908 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 470, 2024-06-03T03:28:53.908 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 159209, 2024-06-03T03:28:53.908 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 148637, 2024-06-03T03:28:53.908 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 6, 2024-06-03T03:28:53.908 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 2, 2024-06-03T03:28:53.908 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.058284255, 2024-06-03T03:28:53.908 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.017826110, 2024-06-03T03:28:53.909 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.015596908, 2024-06-03T03:28:53.909 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.018298961, 2024-06-03T03:28:53.909 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T03:28:53.909 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 469, 2024-06-03T03:28:53.909 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.027939431, 2024-06-03T03:28:53.909 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000059572 2024-06-03T03:28:53.909 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:53.909 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-06-03T03:28:53.909 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:53.909 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:53.909 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:53.909 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:53.909 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 2688, 2024-06-03T03:28:53.909 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 448 2024-06-03T03:28:53.910 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:53.910 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-1": { 2024-06-03T03:28:53.910 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 867, 2024-06-03T03:28:53.910 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 860, 2024-06-03T03:28:53.910 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 948309, 2024-06-03T03:28:53.910 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 320871, 2024-06-03T03:28:53.910 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 4, 2024-06-03T03:28:53.910 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 3, 2024-06-03T03:28:53.910 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.084817559, 2024-06-03T03:28:53.910 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.030865097, 2024-06-03T03:28:53.910 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.032186471, 2024-06-03T03:28:53.910 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.020738980, 2024-06-03T03:28:53.910 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T03:28:53.911 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 859, 2024-06-03T03:28:53.911 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.037577826, 2024-06-03T03:28:53.911 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000043746 2024-06-03T03:28:53.911 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:53.911 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-06-03T03:28:53.911 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:53.911 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:53.911 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:53.911 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:53.911 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 770192, 2024-06-03T03:28:53.911 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 101712 2024-06-03T03:28:53.911 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:53.911 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-2": { 2024-06-03T03:28:53.912 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 4, 2024-06-03T03:28:53.912 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 128, 2024-06-03T03:28:53.912 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 1508, 2024-06-03T03:28:53.912 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 260357, 2024-06-03T03:28:53.912 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 5, 2024-06-03T03:28:53.912 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 4, 2024-06-03T03:28:53.912 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.007959239, 2024-06-03T03:28:53.912 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.005154098, 2024-06-03T03:28:53.912 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.001399732, 2024-06-03T03:28:53.912 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.000074781, 2024-06-03T03:28:53.912 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T03:28:53.912 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 126, 2024-06-03T03:28:53.912 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.010564843, 2024-06-03T03:28:53.912 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000083847 2024-06-03T03:28:53.913 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:53.913 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-06-03T03:28:53.913 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:53.913 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:53.913 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:53.913 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:53.913 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 160, 2024-06-03T03:28:53.913 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 259392 2024-06-03T03:28:53.913 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:53.913 INFO:teuthology.orchestra.run.smithi151.stdout: "cct": { 2024-06-03T03:28:53.913 INFO:teuthology.orchestra.run.smithi151.stdout: "total_workers": 1, 2024-06-03T03:28:53.913 INFO:teuthology.orchestra.run.smithi151.stdout: "unhealthy_workers": 0 2024-06-03T03:28:53.913 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:53.914 INFO:teuthology.orchestra.run.smithi151.stdout: "finisher-MDSRank": { 2024-06-03T03:28:53.914 INFO:teuthology.orchestra.run.smithi151.stdout: "queue_len": 0, 2024-06-03T03:28:53.914 INFO:teuthology.orchestra.run.smithi151.stdout: "complete_latency": { 2024-06-03T03:28:53.914 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 2088, 2024-06-03T03:28:53.914 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.056204296, 2024-06-03T03:28:53.914 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000026917 2024-06-03T03:28:53.914 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-06-03T03:28:53.914 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:53.914 INFO:teuthology.orchestra.run.smithi151.stdout: "finisher-PurgeQueue": { 2024-06-03T03:28:53.914 INFO:teuthology.orchestra.run.smithi151.stdout: "queue_len": 0, 2024-06-03T03:28:53.914 INFO:teuthology.orchestra.run.smithi151.stdout: "complete_latency": { 2024-06-03T03:28:53.914 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:53.914 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:53.914 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:53.915 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-06-03T03:28:53.915 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:53.915 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": { 2024-06-03T03:28:53.915 INFO:teuthology.orchestra.run.smithi151.stdout: "request": 0, 2024-06-03T03:28:53.915 INFO:teuthology.orchestra.run.smithi151.stdout: "reply": 0, 2024-06-03T03:28:53.915 INFO:teuthology.orchestra.run.smithi151.stdout: "reply_latency": { 2024-06-03T03:28:53.915 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:53.915 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:53.915 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:53.915 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:53.915 INFO:teuthology.orchestra.run.smithi151.stdout: "slow_reply": 0, 2024-06-03T03:28:53.915 INFO:teuthology.orchestra.run.smithi151.stdout: "forward": 0, 2024-06-03T03:28:53.915 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_fetch_complete": 0, 2024-06-03T03:28:53.916 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_fetch_keys": 0, 2024-06-03T03:28:53.916 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_commit": 0, 2024-06-03T03:28:53.916 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_split": 0, 2024-06-03T03:28:53.916 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_merge": 0, 2024-06-03T03:28:53.916 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes": 0, 2024-06-03T03:28:53.916 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_top": 0, 2024-06-03T03:28:53.916 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_bottom": 0, 2024-06-03T03:28:53.917 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_pin_tail": 0, 2024-06-03T03:28:53.917 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_pinned": 0, 2024-06-03T03:28:53.917 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_expired": 0, 2024-06-03T03:28:53.917 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_with_caps": 0, 2024-06-03T03:28:53.917 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": 0, 2024-06-03T03:28:53.917 INFO:teuthology.orchestra.run.smithi151.stdout: "subtrees": 1, 2024-06-03T03:28:53.917 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse": 0, 2024-06-03T03:28:53.917 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_hit": 0, 2024-06-03T03:28:53.917 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_forward": 0, 2024-06-03T03:28:53.918 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_discover": 0, 2024-06-03T03:28:53.918 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_dir_fetch": 0, 2024-06-03T03:28:53.918 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_remote_ino": 0, 2024-06-03T03:28:53.918 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_lock": 0, 2024-06-03T03:28:53.918 INFO:teuthology.orchestra.run.smithi151.stdout: "load_cent": 0, 2024-06-03T03:28:53.918 INFO:teuthology.orchestra.run.smithi151.stdout: "q": 0, 2024-06-03T03:28:53.918 INFO:teuthology.orchestra.run.smithi151.stdout: "exported": 0, 2024-06-03T03:28:53.918 INFO:teuthology.orchestra.run.smithi151.stdout: "exported_inodes": 0, 2024-06-03T03:28:53.918 INFO:teuthology.orchestra.run.smithi151.stdout: "imported": 0, 2024-06-03T03:28:53.918 INFO:teuthology.orchestra.run.smithi151.stdout: "imported_inodes": 0, 2024-06-03T03:28:53.918 INFO:teuthology.orchestra.run.smithi151.stdout: "openino_dir_fetch": 0, 2024-06-03T03:28:53.919 INFO:teuthology.orchestra.run.smithi151.stdout: "openino_backtrace_fetch": 0, 2024-06-03T03:28:53.919 INFO:teuthology.orchestra.run.smithi151.stdout: "openino_peer_discover": 0, 2024-06-03T03:28:53.919 INFO:teuthology.orchestra.run.smithi151.stdout: "root_rfiles": 0, 2024-06-03T03:28:53.919 INFO:teuthology.orchestra.run.smithi151.stdout: "root_rbytes": 0, 2024-06-03T03:28:53.919 INFO:teuthology.orchestra.run.smithi151.stdout: "root_rsnaps": 0, 2024-06-03T03:28:53.919 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_backtrace_fetch": 0, 2024-06-03T03:28:53.919 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_set_tag": 0, 2024-06-03T03:28:53.919 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_backtrace_repaired": 0, 2024-06-03T03:28:53.919 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_inotable_repaired": 0, 2024-06-03T03:28:53.919 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_dir_inodes": 0, 2024-06-03T03:28:53.919 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_dir_base_inodes": 0, 2024-06-03T03:28:53.919 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_dirfrag_rstats": 0, 2024-06-03T03:28:53.919 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_file_inodes": 0, 2024-06-03T03:28:53.919 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_inode_file_caps": 0, 2024-06-03T03:28:53.920 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_revoke": 0, 2024-06-03T03:28:53.920 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_grant": 0, 2024-06-03T03:28:53.920 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_trunc": 0, 2024-06-03T03:28:53.920 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-03T03:28:53.920 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-03T03:28:53.920 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_caps": 0, 2024-06-03T03:28:53.920 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_caps_dirty": 0, 2024-06-03T03:28:53.920 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_cap_release": 0, 2024-06-03T03:28:53.920 INFO:teuthology.orchestra.run.smithi151.stdout: "process_request_cap_release": 0 2024-06-03T03:28:53.920 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:53.920 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_cache": { 2024-06-03T03:28:53.920 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_update": 0, 2024-06-03T03:28:53.921 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_update_receipt": 0, 2024-06-03T03:28:53.921 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_try_discover": 0, 2024-06-03T03:28:53.921 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_send_discover": 0, 2024-06-03T03:28:53.921 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_handle_discover": 0, 2024-06-03T03:28:53.921 INFO:teuthology.orchestra.run.smithi151.stdout: "num_strays": 0, 2024-06-03T03:28:53.921 INFO:teuthology.orchestra.run.smithi151.stdout: "num_strays_delayed": 0, 2024-06-03T03:28:53.921 INFO:teuthology.orchestra.run.smithi151.stdout: "num_strays_enqueuing": 0, 2024-06-03T03:28:53.921 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_created": 0, 2024-06-03T03:28:53.921 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_enqueued": 0, 2024-06-03T03:28:53.921 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_reintegrated": 0, 2024-06-03T03:28:53.921 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_migrated": 0, 2024-06-03T03:28:53.921 INFO:teuthology.orchestra.run.smithi151.stdout: "num_recovering_processing": 0, 2024-06-03T03:28:53.921 INFO:teuthology.orchestra.run.smithi151.stdout: "num_recovering_enqueued": 0, 2024-06-03T03:28:53.922 INFO:teuthology.orchestra.run.smithi151.stdout: "num_recovering_prioritized": 0, 2024-06-03T03:28:53.922 INFO:teuthology.orchestra.run.smithi151.stdout: "recovery_started": 0, 2024-06-03T03:28:53.922 INFO:teuthology.orchestra.run.smithi151.stdout: "recovery_completed": 0, 2024-06-03T03:28:53.922 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_quiesce_path": 0, 2024-06-03T03:28:53.922 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_quiesce_inode": 0, 2024-06-03T03:28:53.922 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_enqueue_scrub": 0, 2024-06-03T03:28:53.922 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_exportdir": 0, 2024-06-03T03:28:53.922 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_flush": 0, 2024-06-03T03:28:53.922 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_fragmentdir": 0, 2024-06-03T03:28:53.922 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_fragstats": 0, 2024-06-03T03:28:53.922 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_inodestats": 0 2024-06-03T03:28:53.922 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:53.922 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_log": { 2024-06-03T03:28:53.923 INFO:teuthology.orchestra.run.smithi151.stdout: "evlrg": 0, 2024-06-03T03:28:53.923 INFO:teuthology.orchestra.run.smithi151.stdout: "evadd": 0, 2024-06-03T03:28:53.923 INFO:teuthology.orchestra.run.smithi151.stdout: "evex": 0, 2024-06-03T03:28:53.923 INFO:teuthology.orchestra.run.smithi151.stdout: "evtrm": 0, 2024-06-03T03:28:53.923 INFO:teuthology.orchestra.run.smithi151.stdout: "ev": 4, 2024-06-03T03:28:53.923 INFO:teuthology.orchestra.run.smithi151.stdout: "evexg": 0, 2024-06-03T03:28:53.923 INFO:teuthology.orchestra.run.smithi151.stdout: "evexd": 0, 2024-06-03T03:28:53.923 INFO:teuthology.orchestra.run.smithi151.stdout: "segadd": 0, 2024-06-03T03:28:53.923 INFO:teuthology.orchestra.run.smithi151.stdout: "segex": 0, 2024-06-03T03:28:53.923 INFO:teuthology.orchestra.run.smithi151.stdout: "segtrm": 0, 2024-06-03T03:28:53.923 INFO:teuthology.orchestra.run.smithi151.stdout: "seg": 2, 2024-06-03T03:28:53.923 INFO:teuthology.orchestra.run.smithi151.stdout: "segmjr": 2, 2024-06-03T03:28:53.923 INFO:teuthology.orchestra.run.smithi151.stdout: "segexg": 0, 2024-06-03T03:28:53.923 INFO:teuthology.orchestra.run.smithi151.stdout: "segexd": 0, 2024-06-03T03:28:53.924 INFO:teuthology.orchestra.run.smithi151.stdout: "expos": 4194304, 2024-06-03T03:28:53.924 INFO:teuthology.orchestra.run.smithi151.stdout: "wrpos": 4195423, 2024-06-03T03:28:53.924 INFO:teuthology.orchestra.run.smithi151.stdout: "rdpos": 4195282, 2024-06-03T03:28:53.924 INFO:teuthology.orchestra.run.smithi151.stdout: "jlat": { 2024-06-03T03:28:53.924 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:53.924 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:53.924 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:53.924 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:53.924 INFO:teuthology.orchestra.run.smithi151.stdout: "replayed": 4 2024-06-03T03:28:53.924 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:53.924 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_mem": { 2024-06-03T03:28:53.924 INFO:teuthology.orchestra.run.smithi151.stdout: "ino": 3, 2024-06-03T03:28:53.924 INFO:teuthology.orchestra.run.smithi151.stdout: "ino+": 3, 2024-06-03T03:28:53.925 INFO:teuthology.orchestra.run.smithi151.stdout: "ino-": 0, 2024-06-03T03:28:53.925 INFO:teuthology.orchestra.run.smithi151.stdout: "dir": 1, 2024-06-03T03:28:53.925 INFO:teuthology.orchestra.run.smithi151.stdout: "dir+": 1, 2024-06-03T03:28:53.925 INFO:teuthology.orchestra.run.smithi151.stdout: "dir-": 0, 2024-06-03T03:28:53.925 INFO:teuthology.orchestra.run.smithi151.stdout: "dn": 0, 2024-06-03T03:28:53.925 INFO:teuthology.orchestra.run.smithi151.stdout: "dn+": 0, 2024-06-03T03:28:53.925 INFO:teuthology.orchestra.run.smithi151.stdout: "dn-": 0, 2024-06-03T03:28:53.925 INFO:teuthology.orchestra.run.smithi151.stdout: "cap": 0, 2024-06-03T03:28:53.925 INFO:teuthology.orchestra.run.smithi151.stdout: "cap+": 0, 2024-06-03T03:28:53.925 INFO:teuthology.orchestra.run.smithi151.stdout: "cap-": 0, 2024-06-03T03:28:53.925 INFO:teuthology.orchestra.run.smithi151.stdout: "rss": 54372, 2024-06-03T03:28:53.926 INFO:teuthology.orchestra.run.smithi151.stdout: "heap": 215420 2024-06-03T03:28:53.926 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:53.926 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_server": { 2024-06-03T03:28:53.926 INFO:teuthology.orchestra.run.smithi151.stdout: "dispatch_client_request": 0, 2024-06-03T03:28:53.926 INFO:teuthology.orchestra.run.smithi151.stdout: "dispatch_server_request": 0, 2024-06-03T03:28:53.926 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_request": 0, 2024-06-03T03:28:53.926 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_session": 0, 2024-06-03T03:28:53.926 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_peer_request": 0, 2024-06-03T03:28:53.926 INFO:teuthology.orchestra.run.smithi151.stdout: "req_create_latency": { 2024-06-03T03:28:53.926 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:53.926 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:53.927 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:53.927 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:53.928 INFO:teuthology.orchestra.run.smithi151.stdout: "req_getattr_latency": { 2024-06-03T03:28:53.928 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:53.928 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:53.928 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:53.928 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:53.928 INFO:teuthology.orchestra.run.smithi151.stdout: "req_getfilelock_latency": { 2024-06-03T03:28:53.928 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:53.928 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:53.928 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:53.928 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:53.928 INFO:teuthology.orchestra.run.smithi151.stdout: "req_link_latency": { 2024-06-03T03:28:53.929 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:53.929 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:53.929 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:53.929 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:53.929 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookup_latency": { 2024-06-03T03:28:53.929 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:53.930 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:53.930 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:53.930 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:53.930 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookuphash_latency": { 2024-06-03T03:28:53.930 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:53.930 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:53.930 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:53.930 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:53.930 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupino_latency": { 2024-06-03T03:28:53.930 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:53.930 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:53.930 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:53.930 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:53.931 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupname_latency": { 2024-06-03T03:28:53.931 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:53.931 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:53.931 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:53.931 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:53.931 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupparent_latency": { 2024-06-03T03:28:53.931 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:53.931 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:53.931 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:53.931 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:53.931 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupsnap_latency": { 2024-06-03T03:28:53.931 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:53.931 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:53.931 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:53.932 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:53.932 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lssnap_latency": { 2024-06-03T03:28:53.932 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:53.932 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:53.932 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:53.932 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:53.932 INFO:teuthology.orchestra.run.smithi151.stdout: "req_mkdir_latency": { 2024-06-03T03:28:53.932 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:53.933 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:53.933 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:53.933 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:53.933 INFO:teuthology.orchestra.run.smithi151.stdout: "req_mknod_latency": { 2024-06-03T03:28:53.933 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:53.933 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:53.933 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:53.933 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:53.933 INFO:teuthology.orchestra.run.smithi151.stdout: "req_mksnap_latency": { 2024-06-03T03:28:53.933 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:53.934 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:53.934 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:53.934 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:53.934 INFO:teuthology.orchestra.run.smithi151.stdout: "req_open_latency": { 2024-06-03T03:28:53.934 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:53.934 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:53.934 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:53.934 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:53.934 INFO:teuthology.orchestra.run.smithi151.stdout: "req_readdir_latency": { 2024-06-03T03:28:53.934 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:53.934 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:53.934 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:53.934 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:53.935 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rename_latency": { 2024-06-03T03:28:53.935 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:53.935 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:53.935 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:53.935 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:53.935 INFO:teuthology.orchestra.run.smithi151.stdout: "req_renamesnap_latency": { 2024-06-03T03:28:53.935 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:53.935 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:53.935 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:53.935 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:53.935 INFO:teuthology.orchestra.run.smithi151.stdout: "req_snapdiff_latency": { 2024-06-03T03:28:53.935 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:53.935 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:53.935 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:53.936 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:53.936 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rmdir_latency": { 2024-06-03T03:28:53.936 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:53.936 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:53.936 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:53.936 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:53.936 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rmsnap_latency": { 2024-06-03T03:28:53.936 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:53.936 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:53.936 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:53.936 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:53.936 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rmxattr_latency": { 2024-06-03T03:28:53.936 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:53.936 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:53.937 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:53.937 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:53.937 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setattr_latency": { 2024-06-03T03:28:53.937 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:53.937 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:53.937 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:53.937 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:53.937 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setdirlayout_latency": { 2024-06-03T03:28:53.937 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:53.937 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:53.937 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:53.937 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:53.937 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setfilelock_latency": { 2024-06-03T03:28:53.937 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:53.938 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:53.938 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:53.938 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:53.938 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setlayout_latency": { 2024-06-03T03:28:53.938 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:53.938 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:53.938 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:53.938 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:53.938 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setxattr_latency": { 2024-06-03T03:28:53.938 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:53.938 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:53.938 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:53.938 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:53.939 INFO:teuthology.orchestra.run.smithi151.stdout: "req_symlink_latency": { 2024-06-03T03:28:53.939 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:53.939 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:53.939 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:53.939 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:53.939 INFO:teuthology.orchestra.run.smithi151.stdout: "req_unlink_latency": { 2024-06-03T03:28:53.939 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:53.939 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:53.939 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:53.939 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:53.939 INFO:teuthology.orchestra.run.smithi151.stdout: "cap_revoke_eviction": 0, 2024-06-03T03:28:53.939 INFO:teuthology.orchestra.run.smithi151.stdout: "cap_acquisition_throttle": 0, 2024-06-03T03:28:53.939 INFO:teuthology.orchestra.run.smithi151.stdout: "req_getvxattr_latency": { 2024-06-03T03:28:53.939 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:53.940 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:53.940 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:53.940 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-06-03T03:28:53.940 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:53.940 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_sessions": { 2024-06-03T03:28:53.940 INFO:teuthology.orchestra.run.smithi151.stdout: "session_count": 0, 2024-06-03T03:28:53.940 INFO:teuthology.orchestra.run.smithi151.stdout: "session_add": 1, 2024-06-03T03:28:53.940 INFO:teuthology.orchestra.run.smithi151.stdout: "session_remove": 1, 2024-06-03T03:28:53.940 INFO:teuthology.orchestra.run.smithi151.stdout: "sessions_open": 1, 2024-06-03T03:28:53.940 INFO:teuthology.orchestra.run.smithi151.stdout: "sessions_stale": 0, 2024-06-03T03:28:53.940 INFO:teuthology.orchestra.run.smithi151.stdout: "total_load": 0, 2024-06-03T03:28:53.940 INFO:teuthology.orchestra.run.smithi151.stdout: "average_load": 0, 2024-06-03T03:28:53.940 INFO:teuthology.orchestra.run.smithi151.stdout: "avg_session_uptime": 1824, 2024-06-03T03:28:53.940 INFO:teuthology.orchestra.run.smithi151.stdout: "mdthresh_evicted": 0 2024-06-03T03:28:53.941 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:53.941 INFO:teuthology.orchestra.run.smithi151.stdout: "mempool": { 2024-06-03T03:28:53.941 INFO:teuthology.orchestra.run.smithi151.stdout: "bloom_filter_bytes": 0, 2024-06-03T03:28:53.941 INFO:teuthology.orchestra.run.smithi151.stdout: "bloom_filter_items": 0, 2024-06-03T03:28:53.941 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_alloc_bytes": 0, 2024-06-03T03:28:53.941 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_alloc_items": 0, 2024-06-03T03:28:53.941 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_data_bytes": 0, 2024-06-03T03:28:53.941 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_data_items": 0, 2024-06-03T03:28:53.941 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-03T03:28:53.941 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_onode_items": 0, 2024-06-03T03:28:53.941 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-03T03:28:53.941 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_meta_items": 0, 2024-06-03T03:28:53.941 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_other_bytes": 0, 2024-06-03T03:28:53.941 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_other_items": 0, 2024-06-03T03:28:53.942 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-03T03:28:53.942 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_buffer_items": 0, 2024-06-03T03:28:53.942 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_extent_bytes": 0, 2024-06-03T03:28:53.942 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_extent_items": 0, 2024-06-03T03:28:53.942 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_blob_bytes": 0, 2024-06-03T03:28:53.942 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_blob_items": 0, 2024-06-03T03:28:53.942 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-03T03:28:53.942 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_shared_blob_items": 0, 2024-06-03T03:28:53.942 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-03T03:28:53.942 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_inline_bl_items": 0, 2024-06-03T03:28:53.942 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_fsck_bytes": 0, 2024-06-03T03:28:53.942 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_fsck_items": 0, 2024-06-03T03:28:53.942 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_txc_bytes": 0, 2024-06-03T03:28:53.942 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_txc_items": 0, 2024-06-03T03:28:53.943 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-03T03:28:53.943 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_deferred_items": 0, 2024-06-03T03:28:53.943 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_bytes": 0, 2024-06-03T03:28:53.943 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_items": 0, 2024-06-03T03:28:53.943 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_bytes": 0, 2024-06-03T03:28:53.943 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_items": 0, 2024-06-03T03:28:53.943 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_reader_bytes": 0, 2024-06-03T03:28:53.943 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_reader_items": 0, 2024-06-03T03:28:53.943 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_writer_bytes": 0, 2024-06-03T03:28:53.943 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_writer_items": 0, 2024-06-03T03:28:53.943 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_anon_bytes": 197481, 2024-06-03T03:28:53.943 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_anon_items": 55, 2024-06-03T03:28:53.943 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_meta_bytes": 0, 2024-06-03T03:28:53.943 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_meta_items": 0, 2024-06-03T03:28:53.944 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_bytes": 0, 2024-06-03T03:28:53.944 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_items": 0, 2024-06-03T03:28:53.944 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_mapbl_bytes": 0, 2024-06-03T03:28:53.944 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_mapbl_items": 0, 2024-06-03T03:28:53.944 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_pglog_bytes": 0, 2024-06-03T03:28:53.944 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_pglog_items": 0, 2024-06-03T03:28:53.944 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_bytes": 8888, 2024-06-03T03:28:53.944 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_items": 170, 2024-06-03T03:28:53.944 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_mapping_bytes": 0, 2024-06-03T03:28:53.944 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_mapping_items": 0, 2024-06-03T03:28:53.944 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap_bytes": 0, 2024-06-03T03:28:53.944 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap_items": 0, 2024-06-03T03:28:53.944 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_co_bytes": 8464, 2024-06-03T03:28:53.944 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_co_items": 13, 2024-06-03T03:28:53.945 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_1_bytes": 0, 2024-06-03T03:28:53.945 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_1_items": 0, 2024-06-03T03:28:53.945 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_2_bytes": 0, 2024-06-03T03:28:53.945 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_2_items": 0 2024-06-03T03:28:53.945 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:53.945 INFO:teuthology.orchestra.run.smithi151.stdout: "objecter": { 2024-06-03T03:28:53.945 INFO:teuthology.orchestra.run.smithi151.stdout: "op_active": 0, 2024-06-03T03:28:53.945 INFO:teuthology.orchestra.run.smithi151.stdout: "op_laggy": 0, 2024-06-03T03:28:53.945 INFO:teuthology.orchestra.run.smithi151.stdout: "op_send": 1158, 2024-06-03T03:28:53.945 INFO:teuthology.orchestra.run.smithi151.stdout: "op_send_bytes": 21, 2024-06-03T03:28:53.945 INFO:teuthology.orchestra.run.smithi151.stdout: "op_resend": 3, 2024-06-03T03:28:53.945 INFO:teuthology.orchestra.run.smithi151.stdout: "op_reply": 1158, 2024-06-03T03:28:53.945 INFO:teuthology.orchestra.run.smithi151.stdout: "op_latency": { 2024-06-03T03:28:53.945 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 1158, 2024-06-03T03:28:53.945 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.740990123, 2024-06-03T03:28:53.946 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000639887 2024-06-03T03:28:53.946 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:53.946 INFO:teuthology.orchestra.run.smithi151.stdout: "op_inflight": 0, 2024-06-03T03:28:53.946 INFO:teuthology.orchestra.run.smithi151.stdout: "oplen_avg": { 2024-06-03T03:28:53.946 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 1158, 2024-06-03T03:28:53.946 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 1159 2024-06-03T03:28:53.946 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:53.946 INFO:teuthology.orchestra.run.smithi151.stdout: "op": 1158, 2024-06-03T03:28:53.946 INFO:teuthology.orchestra.run.smithi151.stdout: "op_r": 1158, 2024-06-03T03:28:53.946 INFO:teuthology.orchestra.run.smithi151.stdout: "op_w": 0, 2024-06-03T03:28:53.946 INFO:teuthology.orchestra.run.smithi151.stdout: "op_rmw": 0, 2024-06-03T03:28:53.946 INFO:teuthology.orchestra.run.smithi151.stdout: "op_pg": 0, 2024-06-03T03:28:53.946 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_stat": 685, 2024-06-03T03:28:53.946 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_create": 0, 2024-06-03T03:28:53.947 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_read": 471, 2024-06-03T03:28:53.947 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_write": 0, 2024-06-03T03:28:53.947 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_writefull": 0, 2024-06-03T03:28:53.947 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_writesame": 0, 2024-06-03T03:28:53.947 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_append": 0, 2024-06-03T03:28:53.947 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_zero": 0, 2024-06-03T03:28:53.947 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_truncate": 0, 2024-06-03T03:28:53.947 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_delete": 0, 2024-06-03T03:28:53.947 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_mapext": 0, 2024-06-03T03:28:53.947 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_sparse_read": 0, 2024-06-03T03:28:53.947 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_clonerange": 0, 2024-06-03T03:28:53.947 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_getxattr": 1, 2024-06-03T03:28:53.947 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_setxattr": 0, 2024-06-03T03:28:53.947 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_cmpxattr": 0, 2024-06-03T03:28:53.948 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_rmxattr": 0, 2024-06-03T03:28:53.948 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_resetxattrs": 0, 2024-06-03T03:28:53.948 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_call": 0, 2024-06-03T03:28:53.948 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_watch": 0, 2024-06-03T03:28:53.948 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_notify": 0, 2024-06-03T03:28:53.948 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_src_cmpxattr": 0, 2024-06-03T03:28:53.948 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_pgls": 0, 2024-06-03T03:28:53.948 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_pgls_filter": 0, 2024-06-03T03:28:53.948 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_other": 0, 2024-06-03T03:28:53.948 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_active": 0, 2024-06-03T03:28:53.948 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_send": 0, 2024-06-03T03:28:53.948 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_resend": 0, 2024-06-03T03:28:53.948 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_ping": 0, 2024-06-03T03:28:53.948 INFO:teuthology.orchestra.run.smithi151.stdout: "poolop_active": 0, 2024-06-03T03:28:53.949 INFO:teuthology.orchestra.run.smithi151.stdout: "poolop_send": 0, 2024-06-03T03:28:53.949 INFO:teuthology.orchestra.run.smithi151.stdout: "poolop_resend": 0, 2024-06-03T03:28:53.949 INFO:teuthology.orchestra.run.smithi151.stdout: "poolstat_active": 0, 2024-06-03T03:28:53.949 INFO:teuthology.orchestra.run.smithi151.stdout: "poolstat_send": 0, 2024-06-03T03:28:53.949 INFO:teuthology.orchestra.run.smithi151.stdout: "poolstat_resend": 0, 2024-06-03T03:28:53.949 INFO:teuthology.orchestra.run.smithi151.stdout: "statfs_active": 0, 2024-06-03T03:28:53.949 INFO:teuthology.orchestra.run.smithi151.stdout: "statfs_send": 0, 2024-06-03T03:28:53.949 INFO:teuthology.orchestra.run.smithi151.stdout: "statfs_resend": 0, 2024-06-03T03:28:53.949 INFO:teuthology.orchestra.run.smithi151.stdout: "command_active": 0, 2024-06-03T03:28:53.949 INFO:teuthology.orchestra.run.smithi151.stdout: "command_send": 0, 2024-06-03T03:28:53.949 INFO:teuthology.orchestra.run.smithi151.stdout: "command_resend": 0, 2024-06-03T03:28:53.949 INFO:teuthology.orchestra.run.smithi151.stdout: "map_epoch": 99, 2024-06-03T03:28:53.949 INFO:teuthology.orchestra.run.smithi151.stdout: "map_full": 0, 2024-06-03T03:28:53.949 INFO:teuthology.orchestra.run.smithi151.stdout: "map_inc": 2, 2024-06-03T03:28:53.949 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_sessions": 6, 2024-06-03T03:28:53.950 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_session_open": 6, 2024-06-03T03:28:53.950 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_session_close": 0, 2024-06-03T03:28:53.950 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_laggy": 0, 2024-06-03T03:28:53.950 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_wr": 0, 2024-06-03T03:28:53.950 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_rd": 2, 2024-06-03T03:28:53.950 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_del": 0 2024-06-03T03:28:53.950 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:53.950 INFO:teuthology.orchestra.run.smithi151.stdout: "oft": { 2024-06-03T03:28:53.950 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_objs": 0, 2024-06-03T03:28:53.950 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_kv_pairs": 0, 2024-06-03T03:28:53.950 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_updates": 0, 2024-06-03T03:28:53.950 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_removes": 0 2024-06-03T03:28:53.950 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:53.951 INFO:teuthology.orchestra.run.smithi151.stdout: "osd-slow-ops": { 2024-06-03T03:28:53.951 INFO:teuthology.orchestra.run.smithi151.stdout: "slow_ops_count": 0 2024-06-03T03:28:53.951 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:53.951 INFO:teuthology.orchestra.run.smithi151.stdout: "purge_queue": { 2024-06-03T03:28:53.951 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing_ops": 0, 2024-06-03T03:28:53.951 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing_ops_high_water": 0, 2024-06-03T03:28:53.951 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing": 0, 2024-06-03T03:28:53.951 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing_high_water": 0, 2024-06-03T03:28:53.951 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executed_ops": 0, 2024-06-03T03:28:53.951 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executed": 0, 2024-06-03T03:28:53.951 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_item_in_journal": 0 2024-06-03T03:28:53.951 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:53.951 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-03T03:28:53.951 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-06-03T03:28:53.952 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 104857600, 2024-06-03T03:28:53.952 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-06-03T03:28:53.952 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 1337, 2024-06-03T03:28:53.952 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 976919, 2024-06-03T03:28:53.952 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-06-03T03:28:53.952 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 1337, 2024-06-03T03:28:53.952 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-06-03T03:28:53.952 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-06-03T03:28:53.952 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 1337, 2024-06-03T03:28:53.952 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 976919, 2024-06-03T03:28:53.952 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-06-03T03:28:53.952 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:53.952 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:53.952 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:53.952 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-06-03T03:28:53.953 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:53.953 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-objecter_bytes": { 2024-06-03T03:28:53.953 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-06-03T03:28:53.953 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 104857600, 2024-06-03T03:28:53.953 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-06-03T03:28:53.953 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-06-03T03:28:53.953 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-06-03T03:28:53.953 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-06-03T03:28:53.953 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-06-03T03:28:53.953 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 1158, 2024-06-03T03:28:53.953 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 1124, 2024-06-03T03:28:53.953 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 1158, 2024-06-03T03:28:53.953 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 1124, 2024-06-03T03:28:53.953 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-06-03T03:28:53.953 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:53.954 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:53.954 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:53.954 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-06-03T03:28:53.954 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:53.954 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-objecter_ops": { 2024-06-03T03:28:53.954 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-06-03T03:28:53.954 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 1024, 2024-06-03T03:28:53.954 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-06-03T03:28:53.954 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-06-03T03:28:53.954 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-06-03T03:28:53.954 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-06-03T03:28:53.954 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-06-03T03:28:53.954 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 1158, 2024-06-03T03:28:53.955 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 1158, 2024-06-03T03:28:53.955 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 1158, 2024-06-03T03:28:53.955 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 1158, 2024-06-03T03:28:53.955 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-06-03T03:28:53.955 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:53.955 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:53.955 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:53.955 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-06-03T03:28:53.955 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:53.955 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-write_buf_throttle": { 2024-06-03T03:28:53.955 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-06-03T03:28:53.955 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 3758096384, 2024-06-03T03:28:53.955 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-06-03T03:28:53.955 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-06-03T03:28:53.956 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-06-03T03:28:53.956 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-06-03T03:28:53.956 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-06-03T03:28:53.956 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-06-03T03:28:53.956 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-06-03T03:28:53.956 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 0, 2024-06-03T03:28:53.956 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 0, 2024-06-03T03:28:53.956 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-06-03T03:28:53.956 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:53.956 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:53.956 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:53.956 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-06-03T03:28:53.956 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-06-03T03:28:53.956 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-write_buf_throttle-0x55f0c3d6c6e0": { 2024-06-03T03:28:53.956 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-06-03T03:28:53.957 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 3758096384, 2024-06-03T03:28:53.957 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-06-03T03:28:53.957 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-06-03T03:28:53.957 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-06-03T03:28:53.957 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-06-03T03:28:53.957 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-06-03T03:28:53.957 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-06-03T03:28:53.957 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-06-03T03:28:53.957 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 0, 2024-06-03T03:28:53.957 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 0, 2024-06-03T03:28:53.957 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-06-03T03:28:53.957 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-06-03T03:28:53.957 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-06-03T03:28:53.957 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-06-03T03:28:53.957 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-06-03T03:28:53.958 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-06-03T03:28:53.958 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-06-03T03:28:54.724 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-06-03T03:28:54.725 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-06-03T03:28:54.725 DEBUG:tasks.check_counter:Getting stats from c 2024-06-03T03:28:54.725 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph daemon mds.c perf dump 2024-06-03T03:28:54.973 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.c/config 2024-06-03T03:28:55.403 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:28:55 smithi183 ceph-mon[37326]: pgmap v614: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-06-03T03:28:55.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:28:55 smithi151 ceph-mon[37043]: pgmap v614: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-06-03T03:28:55.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:28:55 smithi117 ceph-mon[31649]: pgmap v614: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-06-03T03:28:56.790 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-06-03T03:28:56.790 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-0": { 2024-06-03T03:28:56.790 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 413, 2024-06-03T03:28:56.790 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 498, 2024-06-03T03:28:56.791 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 844441, 2024-06-03T03:28:56.791 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 178122, 2024-06-03T03:28:56.791 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 7, 2024-06-03T03:28:56.791 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 2, 2024-06-03T03:28:56.791 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.067162376, 2024-06-03T03:28:56.791 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.027556985, 2024-06-03T03:28:56.791 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.033052349, 2024-06-03T03:28:56.791 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.006944376, 2024-06-03T03:28:56.791 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T03:28:56.791 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 499, 2024-06-03T03:28:56.791 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.044010899, 2024-06-03T03:28:56.791 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000088198 2024-06-03T03:28:56.791 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:56.792 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-06-03T03:28:56.792 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 475, 2024-06-03T03:28:56.792 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000140759, 2024-06-03T03:28:56.792 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000296 2024-06-03T03:28:56.792 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:56.792 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 791840, 2024-06-03T03:28:56.792 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 103952 2024-06-03T03:28:56.792 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:56.792 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-1": { 2024-06-03T03:28:56.792 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 17, 2024-06-03T03:28:56.792 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 138, 2024-06-03T03:28:56.792 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 4435, 2024-06-03T03:28:56.792 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 346718, 2024-06-03T03:28:56.793 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 7, 2024-06-03T03:28:56.793 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 4, 2024-06-03T03:28:56.793 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.011941506, 2024-06-03T03:28:56.793 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.005553380, 2024-06-03T03:28:56.793 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.004568668, 2024-06-03T03:28:56.793 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.000432766, 2024-06-03T03:28:56.793 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T03:28:56.793 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 136, 2024-06-03T03:28:56.793 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.024328984, 2024-06-03T03:28:56.793 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000178889 2024-06-03T03:28:56.793 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:56.793 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-06-03T03:28:56.793 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:28:56.794 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:28:56.794 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:28:56.794 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:56.794 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 160, 2024-06-03T03:28:56.794 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 338656 2024-06-03T03:28:56.794 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:56.794 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-2": { 2024-06-03T03:28:56.794 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 35, 2024-06-03T03:28:56.794 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 37, 2024-06-03T03:28:56.794 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 10137, 2024-06-03T03:28:56.794 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 138234, 2024-06-03T03:28:56.794 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 17, 2024-06-03T03:28:56.794 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 8, 2024-06-03T03:28:56.795 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.008095308, 2024-06-03T03:28:56.795 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.000762482, 2024-06-03T03:28:56.795 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.006490357, 2024-06-03T03:28:56.795 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.000415519, 2024-06-03T03:28:56.795 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T03:28:56.795 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 34, 2024-06-03T03:28:56.795 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.012438946, 2024-06-03T03:28:56.795 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000365851 2024-06-03T03:28:56.795 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:56.795 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-06-03T03:28:56.795 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 25, 2024-06-03T03:28:56.795 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000003677, 2024-06-03T03:28:56.795 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000147 2024-06-03T03:28:56.796 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:56.796 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 5200, 2024-06-03T03:28:56.796 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 3776 2024-06-03T03:28:56.796 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:56.796 INFO:teuthology.orchestra.run.smithi183.stdout: "cct": { 2024-06-03T03:28:56.796 INFO:teuthology.orchestra.run.smithi183.stdout: "total_workers": 1, 2024-06-03T03:28:56.796 INFO:teuthology.orchestra.run.smithi183.stdout: "unhealthy_workers": 0 2024-06-03T03:28:56.796 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:56.796 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-MDSRank": { 2024-06-03T03:28:56.796 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-06-03T03:28:56.796 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-06-03T03:28:56.796 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 53, 2024-06-03T03:28:56.796 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.002293814, 2024-06-03T03:28:56.796 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000043279 2024-06-03T03:28:56.797 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-06-03T03:28:56.797 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:56.797 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-PurgeQueue": { 2024-06-03T03:28:56.797 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-06-03T03:28:56.797 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-06-03T03:28:56.797 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 2, 2024-06-03T03:28:56.797 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000012309, 2024-06-03T03:28:56.797 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000006154 2024-06-03T03:28:56.797 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-06-03T03:28:56.797 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:56.798 INFO:teuthology.orchestra.run.smithi183.stdout: "mds": { 2024-06-03T03:28:56.798 INFO:teuthology.orchestra.run.smithi183.stdout: "request": 4, 2024-06-03T03:28:56.798 INFO:teuthology.orchestra.run.smithi183.stdout: "reply": 3, 2024-06-03T03:28:56.798 INFO:teuthology.orchestra.run.smithi183.stdout: "reply_latency": { 2024-06-03T03:28:56.798 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 3, 2024-06-03T03:28:56.798 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.002770638, 2024-06-03T03:28:56.799 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000923546 2024-06-03T03:28:56.799 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:56.799 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_reply": 0, 2024-06-03T03:28:56.799 INFO:teuthology.orchestra.run.smithi183.stdout: "forward": 1, 2024-06-03T03:28:56.799 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_complete": 0, 2024-06-03T03:28:56.799 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_keys": 0, 2024-06-03T03:28:56.799 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_commit": 11, 2024-06-03T03:28:56.799 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_split": 0, 2024-06-03T03:28:56.799 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_merge": 0, 2024-06-03T03:28:56.799 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes": 12, 2024-06-03T03:28:56.799 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_top": 0, 2024-06-03T03:28:56.799 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_bottom": 12, 2024-06-03T03:28:56.799 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pin_tail": 0, 2024-06-03T03:28:56.799 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pinned": 11, 2024-06-03T03:28:56.800 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_expired": 0, 2024-06-03T03:28:56.800 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_with_caps": 0, 2024-06-03T03:28:56.800 INFO:teuthology.orchestra.run.smithi183.stdout: "caps": 0, 2024-06-03T03:28:56.800 INFO:teuthology.orchestra.run.smithi183.stdout: "subtrees": 2, 2024-06-03T03:28:56.800 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse": 8, 2024-06-03T03:28:56.800 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_hit": 6, 2024-06-03T03:28:56.800 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_forward": 0, 2024-06-03T03:28:56.800 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_discover": 2, 2024-06-03T03:28:56.800 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_dir_fetch": 0, 2024-06-03T03:28:56.800 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_remote_ino": 0, 2024-06-03T03:28:56.800 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_lock": 0, 2024-06-03T03:28:56.800 INFO:teuthology.orchestra.run.smithi183.stdout: "load_cent": 0, 2024-06-03T03:28:56.800 INFO:teuthology.orchestra.run.smithi183.stdout: "q": 0, 2024-06-03T03:28:56.801 INFO:teuthology.orchestra.run.smithi183.stdout: "exported": 0, 2024-06-03T03:28:56.801 INFO:teuthology.orchestra.run.smithi183.stdout: "exported_inodes": 0, 2024-06-03T03:28:56.801 INFO:teuthology.orchestra.run.smithi183.stdout: "imported": 0, 2024-06-03T03:28:56.801 INFO:teuthology.orchestra.run.smithi183.stdout: "imported_inodes": 0, 2024-06-03T03:28:56.801 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_dir_fetch": 0, 2024-06-03T03:28:56.801 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_backtrace_fetch": 0, 2024-06-03T03:28:56.801 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_peer_discover": 0, 2024-06-03T03:28:56.801 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rfiles": 4, 2024-06-03T03:28:56.801 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rbytes": 234, 2024-06-03T03:28:56.801 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rsnaps": 0, 2024-06-03T03:28:56.801 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_fetch": 0, 2024-06-03T03:28:56.801 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_set_tag": 0, 2024-06-03T03:28:56.801 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_repaired": 0, 2024-06-03T03:28:56.801 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_inotable_repaired": 0, 2024-06-03T03:28:56.802 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_inodes": 0, 2024-06-03T03:28:56.802 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_base_inodes": 0, 2024-06-03T03:28:56.802 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dirfrag_rstats": 0, 2024-06-03T03:28:56.802 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_file_inodes": 0, 2024-06-03T03:28:56.802 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_inode_file_caps": 0, 2024-06-03T03:28:56.802 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_revoke": 0, 2024-06-03T03:28:56.802 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_grant": 0, 2024-06-03T03:28:56.802 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_trunc": 0, 2024-06-03T03:28:56.802 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-03T03:28:56.803 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-03T03:28:56.803 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps": 0, 2024-06-03T03:28:56.803 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps_dirty": 0, 2024-06-03T03:28:56.803 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_cap_release": 0, 2024-06-03T03:28:56.803 INFO:teuthology.orchestra.run.smithi183.stdout: "process_request_cap_release": 0 2024-06-03T03:28:56.803 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:56.803 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_cache": { 2024-06-03T03:28:56.803 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update": 0, 2024-06-03T03:28:56.803 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update_receipt": 0, 2024-06-03T03:28:56.803 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_try_discover": 0, 2024-06-03T03:28:56.803 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_send_discover": 3, 2024-06-03T03:28:56.803 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_handle_discover": 0, 2024-06-03T03:28:56.803 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays": 0, 2024-06-03T03:28:56.803 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_delayed": 0, 2024-06-03T03:28:56.804 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_enqueuing": 0, 2024-06-03T03:28:56.804 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_created": 0, 2024-06-03T03:28:56.804 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_enqueued": 0, 2024-06-03T03:28:56.804 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_reintegrated": 0, 2024-06-03T03:28:56.804 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_migrated": 0, 2024-06-03T03:28:56.804 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_processing": 0, 2024-06-03T03:28:56.804 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_enqueued": 0, 2024-06-03T03:28:56.804 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_prioritized": 0, 2024-06-03T03:28:56.804 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_started": 0, 2024-06-03T03:28:56.804 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_completed": 0, 2024-06-03T03:28:56.804 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_path": 0, 2024-06-03T03:28:56.804 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_inode": 0, 2024-06-03T03:28:56.804 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_enqueue_scrub": 0, 2024-06-03T03:28:56.805 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_exportdir": 0, 2024-06-03T03:28:56.805 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_flush": 0, 2024-06-03T03:28:56.805 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragmentdir": 0, 2024-06-03T03:28:56.805 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragstats": 0, 2024-06-03T03:28:56.805 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_inodestats": 0 2024-06-03T03:28:56.805 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:56.805 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_log": { 2024-06-03T03:28:56.805 INFO:teuthology.orchestra.run.smithi183.stdout: "evlrg": 0, 2024-06-03T03:28:56.805 INFO:teuthology.orchestra.run.smithi183.stdout: "evadd": 4, 2024-06-03T03:28:56.805 INFO:teuthology.orchestra.run.smithi183.stdout: "evex": 0, 2024-06-03T03:28:56.805 INFO:teuthology.orchestra.run.smithi183.stdout: "evtrm": 0, 2024-06-03T03:28:56.805 INFO:teuthology.orchestra.run.smithi183.stdout: "ev": 4, 2024-06-03T03:28:56.805 INFO:teuthology.orchestra.run.smithi183.stdout: "evexg": 0, 2024-06-03T03:28:56.806 INFO:teuthology.orchestra.run.smithi183.stdout: "evexd": 0, 2024-06-03T03:28:56.806 INFO:teuthology.orchestra.run.smithi183.stdout: "segadd": 2, 2024-06-03T03:28:56.806 INFO:teuthology.orchestra.run.smithi183.stdout: "segex": 0, 2024-06-03T03:28:56.806 INFO:teuthology.orchestra.run.smithi183.stdout: "segtrm": 0, 2024-06-03T03:28:56.806 INFO:teuthology.orchestra.run.smithi183.stdout: "seg": 2, 2024-06-03T03:28:56.806 INFO:teuthology.orchestra.run.smithi183.stdout: "segmjr": 2, 2024-06-03T03:28:56.806 INFO:teuthology.orchestra.run.smithi183.stdout: "segexg": 0, 2024-06-03T03:28:56.806 INFO:teuthology.orchestra.run.smithi183.stdout: "segexd": 0, 2024-06-03T03:28:56.806 INFO:teuthology.orchestra.run.smithi183.stdout: "expos": 4194304, 2024-06-03T03:28:56.806 INFO:teuthology.orchestra.run.smithi183.stdout: "wrpos": 4195423, 2024-06-03T03:28:56.806 INFO:teuthology.orchestra.run.smithi183.stdout: "rdpos": 0, 2024-06-03T03:28:56.806 INFO:teuthology.orchestra.run.smithi183.stdout: "jlat": { 2024-06-03T03:28:56.806 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 3, 2024-06-03T03:28:56.806 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.007674364, 2024-06-03T03:28:56.807 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.002558121 2024-06-03T03:28:56.807 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:56.807 INFO:teuthology.orchestra.run.smithi183.stdout: "replayed": 0 2024-06-03T03:28:56.807 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:56.807 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_mem": { 2024-06-03T03:28:56.807 INFO:teuthology.orchestra.run.smithi183.stdout: "ino": 15, 2024-06-03T03:28:56.807 INFO:teuthology.orchestra.run.smithi183.stdout: "ino+": 15, 2024-06-03T03:28:56.807 INFO:teuthology.orchestra.run.smithi183.stdout: "ino-": 0, 2024-06-03T03:28:56.807 INFO:teuthology.orchestra.run.smithi183.stdout: "dir": 13, 2024-06-03T03:28:56.807 INFO:teuthology.orchestra.run.smithi183.stdout: "dir+": 14, 2024-06-03T03:28:56.807 INFO:teuthology.orchestra.run.smithi183.stdout: "dir-": 1, 2024-06-03T03:28:56.807 INFO:teuthology.orchestra.run.smithi183.stdout: "dn": 12, 2024-06-03T03:28:56.807 INFO:teuthology.orchestra.run.smithi183.stdout: "dn+": 12, 2024-06-03T03:28:56.807 INFO:teuthology.orchestra.run.smithi183.stdout: "dn-": 0, 2024-06-03T03:28:56.808 INFO:teuthology.orchestra.run.smithi183.stdout: "cap": 0, 2024-06-03T03:28:56.808 INFO:teuthology.orchestra.run.smithi183.stdout: "cap+": 3, 2024-06-03T03:28:56.808 INFO:teuthology.orchestra.run.smithi183.stdout: "cap-": 3, 2024-06-03T03:28:56.808 INFO:teuthology.orchestra.run.smithi183.stdout: "rss": 51072, 2024-06-03T03:28:56.808 INFO:teuthology.orchestra.run.smithi183.stdout: "heap": 223612 2024-06-03T03:28:56.808 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:56.808 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_server": { 2024-06-03T03:28:56.808 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_client_request": 9, 2024-06-03T03:28:56.808 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_server_request": 0, 2024-06-03T03:28:56.808 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_request": 4, 2024-06-03T03:28:56.808 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_session": 4, 2024-06-03T03:28:56.808 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_peer_request": 1, 2024-06-03T03:28:56.808 INFO:teuthology.orchestra.run.smithi183.stdout: "req_create_latency": { 2024-06-03T03:28:56.809 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:28:56.809 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:28:56.809 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:28:56.809 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:56.809 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getattr_latency": { 2024-06-03T03:28:56.809 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1, 2024-06-03T03:28:56.809 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.001691782, 2024-06-03T03:28:56.809 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.001691782 2024-06-03T03:28:56.809 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:56.809 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getfilelock_latency": { 2024-06-03T03:28:56.809 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:28:56.809 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:28:56.809 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:28:56.810 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:56.810 INFO:teuthology.orchestra.run.smithi183.stdout: "req_link_latency": { 2024-06-03T03:28:56.810 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:28:56.810 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:28:56.810 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:28:56.810 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:56.810 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookup_latency": { 2024-06-03T03:28:56.810 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 2, 2024-06-03T03:28:56.810 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.001078856, 2024-06-03T03:28:56.810 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000539428 2024-06-03T03:28:56.810 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:56.810 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookuphash_latency": { 2024-06-03T03:28:56.810 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:28:56.811 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:28:56.811 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:28:56.811 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:56.811 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupino_latency": { 2024-06-03T03:28:56.811 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:28:56.811 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:28:56.811 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:28:56.811 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:56.811 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupname_latency": { 2024-06-03T03:28:56.811 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:28:56.811 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:28:56.811 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:28:56.811 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:56.812 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupparent_latency": { 2024-06-03T03:28:56.812 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:28:56.812 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:28:56.812 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:28:56.812 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:56.812 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupsnap_latency": { 2024-06-03T03:28:56.812 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:28:56.812 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:28:56.812 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:28:56.812 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:56.812 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lssnap_latency": { 2024-06-03T03:28:56.812 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:28:56.812 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:28:56.813 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:28:56.813 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:56.813 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mkdir_latency": { 2024-06-03T03:28:56.813 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:28:56.813 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:28:56.814 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:28:56.814 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:56.814 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mknod_latency": { 2024-06-03T03:28:56.814 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:28:56.814 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:28:56.814 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:28:56.814 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:56.814 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mksnap_latency": { 2024-06-03T03:28:56.814 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:28:56.814 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:28:56.814 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:28:56.814 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:56.814 INFO:teuthology.orchestra.run.smithi183.stdout: "req_open_latency": { 2024-06-03T03:28:56.815 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:28:56.815 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:28:56.815 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:28:56.815 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:56.815 INFO:teuthology.orchestra.run.smithi183.stdout: "req_readdir_latency": { 2024-06-03T03:28:56.815 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:28:56.815 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:28:56.815 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:28:56.815 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:56.815 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rename_latency": { 2024-06-03T03:28:56.815 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:28:56.815 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:28:56.815 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:28:56.816 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:56.816 INFO:teuthology.orchestra.run.smithi183.stdout: "req_renamesnap_latency": { 2024-06-03T03:28:56.816 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:28:56.816 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:28:56.816 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:28:56.816 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:56.816 INFO:teuthology.orchestra.run.smithi183.stdout: "req_snapdiff_latency": { 2024-06-03T03:28:56.816 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:28:56.816 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:28:56.816 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:28:56.816 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:56.816 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmdir_latency": { 2024-06-03T03:28:56.816 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:28:56.816 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:28:56.817 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:28:56.817 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:56.817 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmsnap_latency": { 2024-06-03T03:28:56.817 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:28:56.817 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:28:56.817 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:28:56.817 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:56.817 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmxattr_latency": { 2024-06-03T03:28:56.817 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:28:56.817 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:28:56.817 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:28:56.817 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:56.817 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setattr_latency": { 2024-06-03T03:28:56.817 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:28:56.818 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:28:56.818 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:28:56.818 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:56.818 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setdirlayout_latency": { 2024-06-03T03:28:56.818 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:28:56.818 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:28:56.818 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:28:56.818 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:56.818 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setfilelock_latency": { 2024-06-03T03:28:56.818 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:28:56.818 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:28:56.818 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:28:56.818 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:56.819 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setlayout_latency": { 2024-06-03T03:28:56.819 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:28:56.819 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:28:56.819 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:28:56.819 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:56.819 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setxattr_latency": { 2024-06-03T03:28:56.819 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:28:56.819 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:28:56.819 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:28:56.819 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:56.819 INFO:teuthology.orchestra.run.smithi183.stdout: "req_symlink_latency": { 2024-06-03T03:28:56.819 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:28:56.819 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:28:56.820 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:28:56.820 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:56.820 INFO:teuthology.orchestra.run.smithi183.stdout: "req_unlink_latency": { 2024-06-03T03:28:56.820 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:28:56.820 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:28:56.820 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:28:56.820 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:56.820 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_revoke_eviction": 0, 2024-06-03T03:28:56.820 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_acquisition_throttle": 0, 2024-06-03T03:28:56.820 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getvxattr_latency": { 2024-06-03T03:28:56.820 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:28:56.820 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:28:56.820 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:28:56.820 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-06-03T03:28:56.821 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:56.821 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_sessions": { 2024-06-03T03:28:56.821 INFO:teuthology.orchestra.run.smithi183.stdout: "session_count": 0, 2024-06-03T03:28:56.821 INFO:teuthology.orchestra.run.smithi183.stdout: "session_add": 1, 2024-06-03T03:28:56.821 INFO:teuthology.orchestra.run.smithi183.stdout: "session_remove": 1, 2024-06-03T03:28:56.821 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_open": 0, 2024-06-03T03:28:56.821 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_stale": 0, 2024-06-03T03:28:56.821 INFO:teuthology.orchestra.run.smithi183.stdout: "total_load": 2, 2024-06-03T03:28:56.821 INFO:teuthology.orchestra.run.smithi183.stdout: "average_load": 2, 2024-06-03T03:28:56.821 INFO:teuthology.orchestra.run.smithi183.stdout: "avg_session_uptime": 69, 2024-06-03T03:28:56.821 INFO:teuthology.orchestra.run.smithi183.stdout: "mdthresh_evicted": 0 2024-06-03T03:28:56.821 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:56.821 INFO:teuthology.orchestra.run.smithi183.stdout: "mempool": { 2024-06-03T03:28:56.822 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_bytes": 0, 2024-06-03T03:28:56.822 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_items": 0, 2024-06-03T03:28:56.822 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_bytes": 0, 2024-06-03T03:28:56.822 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_items": 0, 2024-06-03T03:28:56.822 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_bytes": 0, 2024-06-03T03:28:56.822 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_items": 0, 2024-06-03T03:28:56.822 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-03T03:28:56.822 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_items": 0, 2024-06-03T03:28:56.822 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-03T03:28:56.822 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_items": 0, 2024-06-03T03:28:56.822 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_bytes": 0, 2024-06-03T03:28:56.822 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_items": 0, 2024-06-03T03:28:56.822 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-03T03:28:56.822 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_items": 0, 2024-06-03T03:28:56.823 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_bytes": 0, 2024-06-03T03:28:56.823 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_items": 0, 2024-06-03T03:28:56.823 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_bytes": 0, 2024-06-03T03:28:56.823 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_items": 0, 2024-06-03T03:28:56.823 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-03T03:28:56.823 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_items": 0, 2024-06-03T03:28:56.823 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-03T03:28:56.823 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_items": 0, 2024-06-03T03:28:56.823 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_bytes": 0, 2024-06-03T03:28:56.823 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_items": 0, 2024-06-03T03:28:56.823 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_bytes": 0, 2024-06-03T03:28:56.823 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_items": 0, 2024-06-03T03:28:56.823 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-03T03:28:56.823 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_items": 0, 2024-06-03T03:28:56.824 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_bytes": 0, 2024-06-03T03:28:56.824 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_items": 0, 2024-06-03T03:28:56.824 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_bytes": 0, 2024-06-03T03:28:56.824 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_items": 0, 2024-06-03T03:28:56.824 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_bytes": 0, 2024-06-03T03:28:56.824 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_items": 0, 2024-06-03T03:28:56.824 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_bytes": 0, 2024-06-03T03:28:56.824 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_items": 0, 2024-06-03T03:28:56.824 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_bytes": 231143, 2024-06-03T03:28:56.824 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_items": 80, 2024-06-03T03:28:56.824 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_bytes": 88, 2024-06-03T03:28:56.824 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_items": 1, 2024-06-03T03:28:56.824 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_bytes": 0, 2024-06-03T03:28:56.824 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_items": 0, 2024-06-03T03:28:56.825 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_bytes": 0, 2024-06-03T03:28:56.825 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_items": 0, 2024-06-03T03:28:56.825 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_bytes": 0, 2024-06-03T03:28:56.825 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_items": 0, 2024-06-03T03:28:56.825 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_bytes": 8888, 2024-06-03T03:28:56.825 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_items": 170, 2024-06-03T03:28:56.825 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_bytes": 0, 2024-06-03T03:28:56.825 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_items": 0, 2024-06-03T03:28:56.825 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_bytes": 0, 2024-06-03T03:28:56.825 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_items": 0, 2024-06-03T03:28:56.825 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_bytes": 59120, 2024-06-03T03:28:56.825 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_items": 237, 2024-06-03T03:28:56.825 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_bytes": 0, 2024-06-03T03:28:56.825 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_items": 0, 2024-06-03T03:28:56.825 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_bytes": 0, 2024-06-03T03:28:56.826 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_items": 0 2024-06-03T03:28:56.826 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:56.826 INFO:teuthology.orchestra.run.smithi183.stdout: "objecter": { 2024-06-03T03:28:56.826 INFO:teuthology.orchestra.run.smithi183.stdout: "op_active": 0, 2024-06-03T03:28:56.826 INFO:teuthology.orchestra.run.smithi183.stdout: "op_laggy": 0, 2024-06-03T03:28:56.826 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send": 41, 2024-06-03T03:28:56.826 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send_bytes": 11646, 2024-06-03T03:28:56.826 INFO:teuthology.orchestra.run.smithi183.stdout: "op_resend": 33, 2024-06-03T03:28:56.826 INFO:teuthology.orchestra.run.smithi183.stdout: "op_reply": 41, 2024-06-03T03:28:56.826 INFO:teuthology.orchestra.run.smithi183.stdout: "op_latency": { 2024-06-03T03:28:56.826 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 41, 2024-06-03T03:28:56.826 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.273992602, 2024-06-03T03:28:56.826 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.006682746 2024-06-03T03:28:56.827 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:56.827 INFO:teuthology.orchestra.run.smithi183.stdout: "op_inflight": 0, 2024-06-03T03:28:56.827 INFO:teuthology.orchestra.run.smithi183.stdout: "oplen_avg": { 2024-06-03T03:28:56.827 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 41, 2024-06-03T03:28:56.827 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 65 2024-06-03T03:28:56.827 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:56.827 INFO:teuthology.orchestra.run.smithi183.stdout: "op": 41, 2024-06-03T03:28:56.827 INFO:teuthology.orchestra.run.smithi183.stdout: "op_r": 0, 2024-06-03T03:28:56.827 INFO:teuthology.orchestra.run.smithi183.stdout: "op_w": 41, 2024-06-03T03:28:56.827 INFO:teuthology.orchestra.run.smithi183.stdout: "op_rmw": 0, 2024-06-03T03:28:56.827 INFO:teuthology.orchestra.run.smithi183.stdout: "op_pg": 0, 2024-06-03T03:28:56.827 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_stat": 0, 2024-06-03T03:28:56.827 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_create": 10, 2024-06-03T03:28:56.827 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_read": 0, 2024-06-03T03:28:56.827 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_write": 3, 2024-06-03T03:28:56.828 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writefull": 7, 2024-06-03T03:28:56.828 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writesame": 0, 2024-06-03T03:28:56.828 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_append": 0, 2024-06-03T03:28:56.828 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_zero": 0, 2024-06-03T03:28:56.828 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_truncate": 0, 2024-06-03T03:28:56.828 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_delete": 6, 2024-06-03T03:28:56.828 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_mapext": 0, 2024-06-03T03:28:56.828 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_sparse_read": 0, 2024-06-03T03:28:56.828 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_clonerange": 0, 2024-06-03T03:28:56.828 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_getxattr": 0, 2024-06-03T03:28:56.828 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_setxattr": 20, 2024-06-03T03:28:56.828 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_cmpxattr": 0, 2024-06-03T03:28:56.828 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_rmxattr": 0, 2024-06-03T03:28:56.828 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_resetxattrs": 0, 2024-06-03T03:28:56.829 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_call": 0, 2024-06-03T03:28:56.829 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_watch": 0, 2024-06-03T03:28:56.829 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_notify": 0, 2024-06-03T03:28:56.829 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_src_cmpxattr": 0, 2024-06-03T03:28:56.829 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls": 0, 2024-06-03T03:28:56.829 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls_filter": 0, 2024-06-03T03:28:56.829 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_other": 0, 2024-06-03T03:28:56.829 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_active": 0, 2024-06-03T03:28:56.829 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_send": 0, 2024-06-03T03:28:56.829 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_resend": 0, 2024-06-03T03:28:56.829 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_ping": 0, 2024-06-03T03:28:56.829 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_active": 0, 2024-06-03T03:28:56.829 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_send": 0, 2024-06-03T03:28:56.829 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_resend": 0, 2024-06-03T03:28:56.830 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_active": 0, 2024-06-03T03:28:56.830 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_send": 0, 2024-06-03T03:28:56.830 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_resend": 0, 2024-06-03T03:28:56.830 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_active": 0, 2024-06-03T03:28:56.830 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_send": 0, 2024-06-03T03:28:56.830 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_resend": 0, 2024-06-03T03:28:56.830 INFO:teuthology.orchestra.run.smithi183.stdout: "command_active": 0, 2024-06-03T03:28:56.830 INFO:teuthology.orchestra.run.smithi183.stdout: "command_send": 0, 2024-06-03T03:28:56.830 INFO:teuthology.orchestra.run.smithi183.stdout: "command_resend": 0, 2024-06-03T03:28:56.830 INFO:teuthology.orchestra.run.smithi183.stdout: "map_epoch": 99, 2024-06-03T03:28:56.830 INFO:teuthology.orchestra.run.smithi183.stdout: "map_full": 0, 2024-06-03T03:28:56.830 INFO:teuthology.orchestra.run.smithi183.stdout: "map_inc": 2, 2024-06-03T03:28:56.830 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_sessions": 8, 2024-06-03T03:28:56.830 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_open": 8, 2024-06-03T03:28:56.831 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_close": 0, 2024-06-03T03:28:56.831 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_laggy": 0, 2024-06-03T03:28:56.831 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_wr": 17, 2024-06-03T03:28:56.831 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_rd": 0, 2024-06-03T03:28:56.831 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_del": 2 2024-06-03T03:28:56.831 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:56.831 INFO:teuthology.orchestra.run.smithi183.stdout: "oft": { 2024-06-03T03:28:56.831 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_objs": 1, 2024-06-03T03:28:56.831 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_kv_pairs": 0, 2024-06-03T03:28:56.831 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_updates": 0, 2024-06-03T03:28:56.831 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_removes": 0 2024-06-03T03:28:56.831 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:56.831 INFO:teuthology.orchestra.run.smithi183.stdout: "osd-slow-ops": { 2024-06-03T03:28:56.831 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_ops_count": 0 2024-06-03T03:28:56.832 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:56.832 INFO:teuthology.orchestra.run.smithi183.stdout: "purge_queue": { 2024-06-03T03:28:56.832 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops": 0, 2024-06-03T03:28:56.832 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops_high_water": 0, 2024-06-03T03:28:56.832 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing": 0, 2024-06-03T03:28:56.832 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_high_water": 0, 2024-06-03T03:28:56.832 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed_ops": 0, 2024-06-03T03:28:56.832 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed": 0, 2024-06-03T03:28:56.832 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_item_in_journal": 0 2024-06-03T03:28:56.832 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:56.832 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-03T03:28:56.832 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-06-03T03:28:56.832 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-06-03T03:28:56.832 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-06-03T03:28:56.833 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 461, 2024-06-03T03:28:56.833 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 803652, 2024-06-03T03:28:56.833 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-06-03T03:28:56.833 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 461, 2024-06-03T03:28:56.833 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-06-03T03:28:56.833 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-06-03T03:28:56.833 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 461, 2024-06-03T03:28:56.833 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 803652, 2024-06-03T03:28:56.833 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-06-03T03:28:56.833 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:28:56.833 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:28:56.833 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:28:56.833 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-06-03T03:28:56.833 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:56.833 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_bytes": { 2024-06-03T03:28:56.834 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-06-03T03:28:56.834 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-06-03T03:28:56.834 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-06-03T03:28:56.834 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-06-03T03:28:56.834 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-06-03T03:28:56.834 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-06-03T03:28:56.834 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-06-03T03:28:56.834 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 41, 2024-06-03T03:28:56.834 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 11646, 2024-06-03T03:28:56.834 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 41, 2024-06-03T03:28:56.834 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 11646, 2024-06-03T03:28:56.834 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-06-03T03:28:56.834 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:28:56.834 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:28:56.835 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:28:56.835 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-06-03T03:28:56.835 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:56.835 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_ops": { 2024-06-03T03:28:56.835 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-06-03T03:28:56.835 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 1024, 2024-06-03T03:28:56.835 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-06-03T03:28:56.835 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-06-03T03:28:56.835 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-06-03T03:28:56.835 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-06-03T03:28:56.835 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-06-03T03:28:56.835 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 41, 2024-06-03T03:28:56.835 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 41, 2024-06-03T03:28:56.835 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 41, 2024-06-03T03:28:56.836 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 41, 2024-06-03T03:28:56.836 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-06-03T03:28:56.836 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:28:56.836 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:28:56.836 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:28:56.836 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-06-03T03:28:56.836 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:56.836 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle": { 2024-06-03T03:28:56.836 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-06-03T03:28:56.836 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-06-03T03:28:56.836 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-06-03T03:28:56.836 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-06-03T03:28:56.836 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-06-03T03:28:56.836 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-06-03T03:28:56.837 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-06-03T03:28:56.837 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-06-03T03:28:56.837 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-06-03T03:28:56.837 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 0, 2024-06-03T03:28:56.837 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 0, 2024-06-03T03:28:56.837 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-06-03T03:28:56.837 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:28:56.837 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:28:56.837 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:28:56.837 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-06-03T03:28:56.837 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:56.837 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle-0x55a11a266500": { 2024-06-03T03:28:56.837 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-06-03T03:28:56.837 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-06-03T03:28:56.837 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-06-03T03:28:56.838 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 4, 2024-06-03T03:28:56.838 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 1119, 2024-06-03T03:28:56.838 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-06-03T03:28:56.838 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 4, 2024-06-03T03:28:56.838 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-06-03T03:28:56.838 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-06-03T03:28:56.838 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 3, 2024-06-03T03:28:56.838 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 1119, 2024-06-03T03:28:56.838 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-06-03T03:28:56.838 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:28:56.839 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:28:56.839 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:28:56.839 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-06-03T03:28:56.839 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-06-03T03:28:56.839 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-06-03T03:28:57.288 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:28:57 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:28:57.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:28:57 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:28:57.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:28:57 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T03:28:57.517 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-06-03T03:28:57.517 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-06-03T03:28:57.517 DEBUG:tasks.check_counter:Getting stats from f 2024-06-03T03:28:57.518 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph daemon mds.f perf dump 2024-06-03T03:28:57.761 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.c/config 2024-06-03T03:28:58.205 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:28:58 smithi183 ceph-mon[37326]: pgmap v615: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-06-03T03:28:58.205 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:28:58 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:28:58.205 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:28:58 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:28:58.205 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:28:58 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:28:58.205 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:28:58 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:28:58.205 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:28:58 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:28:58.205 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:28:58 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:28:58.205 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:28:58 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:28:58.206 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:28:58 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:28:58.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:28:58 smithi151 ceph-mon[37043]: pgmap v615: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-06-03T03:28:58.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:28:58 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:28:58.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:28:58 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:28:58.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:28:58 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:28:58.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:28:58 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:28:58.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:28:58 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:28:58.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:28:58 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:28:58.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:28:58 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:28:58.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:28:58 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:28:58.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:28:58 smithi117 ceph-mon[31649]: pgmap v615: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-06-03T03:28:58.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:28:58 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:28:58.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:28:58 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:28:58.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:28:58 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:28:58.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:28:58 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:28:58.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:28:58 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:28:58.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:28:58 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:28:58.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:28:58 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:28:58.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:28:58 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-03T03:28:59.179 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:28:59 smithi183 ceph-mon[37326]: Adjusting osd_memory_target on smithi151 to 176.2M 2024-06-03T03:28:59.179 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:28:59 smithi183 ceph-mon[37326]: Unable to set osd_memory_target on smithi151 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-06-03T03:28:59.179 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:28:59 smithi183 ceph-mon[37326]: Adjusting osd_memory_target on smithi183 to 176.2M 2024-06-03T03:28:59.179 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:28:59 smithi183 ceph-mon[37326]: Unable to set osd_memory_target on smithi183 to 184769536: error parsing value: Value '184769536' is below minimum 939524096 2024-06-03T03:28:59.179 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:28:59 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:28:59.179 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:28:59 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:28:59.179 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:28:59 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:28:59.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:28:59 smithi151 ceph-mon[37043]: Adjusting osd_memory_target on smithi151 to 176.2M 2024-06-03T03:28:59.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:28:59 smithi151 ceph-mon[37043]: Unable to set osd_memory_target on smithi151 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-06-03T03:28:59.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:28:59 smithi151 ceph-mon[37043]: Adjusting osd_memory_target on smithi183 to 176.2M 2024-06-03T03:28:59.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:28:59 smithi151 ceph-mon[37043]: Unable to set osd_memory_target on smithi183 to 184769536: error parsing value: Value '184769536' is below minimum 939524096 2024-06-03T03:28:59.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:28:59 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:28:59.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:28:59 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:28:59.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:28:59 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:28:59.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:28:59 smithi117 ceph-mon[31649]: Adjusting osd_memory_target on smithi151 to 176.2M 2024-06-03T03:28:59.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:28:59 smithi117 ceph-mon[31649]: Unable to set osd_memory_target on smithi151 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-06-03T03:28:59.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:28:59 smithi117 ceph-mon[31649]: Adjusting osd_memory_target on smithi183 to 176.2M 2024-06-03T03:28:59.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:28:59 smithi117 ceph-mon[31649]: Unable to set osd_memory_target on smithi183 to 184769536: error parsing value: Value '184769536' is below minimum 939524096 2024-06-03T03:28:59.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:28:59 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T03:28:59.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:28:59 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T03:28:59.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:28:59 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' 2024-06-03T03:28:59.468 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-06-03T03:28:59.468 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-0": { 2024-06-03T03:28:59.468 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 176, 2024-06-03T03:28:59.468 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 293, 2024-06-03T03:28:59.468 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 47193, 2024-06-03T03:28:59.468 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 69837, 2024-06-03T03:28:59.468 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 8, 2024-06-03T03:28:59.468 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 3, 2024-06-03T03:28:59.469 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.035027020, 2024-06-03T03:28:59.469 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.015531525, 2024-06-03T03:28:59.469 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.012608351, 2024-06-03T03:28:59.469 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.002794482, 2024-06-03T03:28:59.469 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T03:28:59.469 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 292, 2024-06-03T03:28:59.469 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.034829170, 2024-06-03T03:28:59.469 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000119277 2024-06-03T03:28:59.469 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:59.469 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-06-03T03:28:59.469 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 427, 2024-06-03T03:28:59.469 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000117111, 2024-06-03T03:28:59.469 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000274 2024-06-03T03:28:59.469 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:59.470 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-06-03T03:28:59.470 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 448 2024-06-03T03:28:59.470 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:59.470 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-1": { 2024-06-03T03:28:59.470 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 230, 2024-06-03T03:28:59.470 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 220, 2024-06-03T03:28:59.470 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 790539, 2024-06-03T03:28:59.470 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 213901, 2024-06-03T03:28:59.470 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 15, 2024-06-03T03:28:59.470 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 7, 2024-06-03T03:28:59.470 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.037026840, 2024-06-03T03:28:59.470 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.011712663, 2024-06-03T03:28:59.470 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.027023799, 2024-06-03T03:28:59.471 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.004369100, 2024-06-03T03:28:59.471 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T03:28:59.471 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 219, 2024-06-03T03:28:59.471 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.028811763, 2024-06-03T03:28:59.471 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000131560 2024-06-03T03:28:59.471 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:59.471 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-06-03T03:28:59.471 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 15, 2024-06-03T03:28:59.471 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000002499, 2024-06-03T03:28:59.471 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000166 2024-06-03T03:28:59.471 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:59.471 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 786880, 2024-06-03T03:28:59.471 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 105968 2024-06-03T03:28:59.471 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:59.472 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-2": { 2024-06-03T03:28:59.472 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 19, 2024-06-03T03:28:59.472 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 141, 2024-06-03T03:28:59.472 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 4690, 2024-06-03T03:28:59.472 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 370745, 2024-06-03T03:28:59.472 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 9, 2024-06-03T03:28:59.472 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 4, 2024-06-03T03:28:59.472 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.011359677, 2024-06-03T03:28:59.472 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.005609433, 2024-06-03T03:28:59.472 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.002991915, 2024-06-03T03:28:59.472 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.001015449, 2024-06-03T03:28:59.472 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T03:28:59.472 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 139, 2024-06-03T03:28:59.473 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.022593085, 2024-06-03T03:28:59.473 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000162540 2024-06-03T03:28:59.473 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:59.473 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-06-03T03:28:59.473 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 2, 2024-06-03T03:28:59.473 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000204, 2024-06-03T03:28:59.473 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000102 2024-06-03T03:28:59.473 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:59.473 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 320, 2024-06-03T03:28:59.473 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 339008 2024-06-03T03:28:59.473 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:59.473 INFO:teuthology.orchestra.run.smithi183.stdout: "cct": { 2024-06-03T03:28:59.473 INFO:teuthology.orchestra.run.smithi183.stdout: "total_workers": 1, 2024-06-03T03:28:59.474 INFO:teuthology.orchestra.run.smithi183.stdout: "unhealthy_workers": 0 2024-06-03T03:28:59.474 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:59.474 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-MDSRank": { 2024-06-03T03:28:59.474 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-06-03T03:28:59.474 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-06-03T03:28:59.474 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 51, 2024-06-03T03:28:59.474 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.001466132, 2024-06-03T03:28:59.474 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000028747 2024-06-03T03:28:59.474 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-06-03T03:28:59.474 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:59.474 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-PurgeQueue": { 2024-06-03T03:28:59.474 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-06-03T03:28:59.474 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-06-03T03:28:59.475 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 2, 2024-06-03T03:28:59.475 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000008348, 2024-06-03T03:28:59.475 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000004174 2024-06-03T03:28:59.475 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-06-03T03:28:59.475 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:59.475 INFO:teuthology.orchestra.run.smithi183.stdout: "mds": { 2024-06-03T03:28:59.475 INFO:teuthology.orchestra.run.smithi183.stdout: "request": 0, 2024-06-03T03:28:59.475 INFO:teuthology.orchestra.run.smithi183.stdout: "reply": 0, 2024-06-03T03:28:59.475 INFO:teuthology.orchestra.run.smithi183.stdout: "reply_latency": { 2024-06-03T03:28:59.475 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:28:59.475 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:28:59.475 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:28:59.475 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:59.476 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_reply": 0, 2024-06-03T03:28:59.476 INFO:teuthology.orchestra.run.smithi183.stdout: "forward": 0, 2024-06-03T03:28:59.476 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_complete": 0, 2024-06-03T03:28:59.476 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_keys": 0, 2024-06-03T03:28:59.476 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_commit": 11, 2024-06-03T03:28:59.476 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_split": 0, 2024-06-03T03:28:59.476 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_merge": 0, 2024-06-03T03:28:59.476 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes": 10, 2024-06-03T03:28:59.476 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_top": 0, 2024-06-03T03:28:59.476 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_bottom": 10, 2024-06-03T03:28:59.476 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pin_tail": 0, 2024-06-03T03:28:59.476 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pinned": 10, 2024-06-03T03:28:59.476 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_expired": 0, 2024-06-03T03:28:59.476 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_with_caps": 0, 2024-06-03T03:28:59.477 INFO:teuthology.orchestra.run.smithi183.stdout: "caps": 0, 2024-06-03T03:28:59.477 INFO:teuthology.orchestra.run.smithi183.stdout: "subtrees": 1, 2024-06-03T03:28:59.477 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse": 0, 2024-06-03T03:28:59.477 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_hit": 0, 2024-06-03T03:28:59.477 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_forward": 0, 2024-06-03T03:28:59.477 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_discover": 0, 2024-06-03T03:28:59.477 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_dir_fetch": 0, 2024-06-03T03:28:59.477 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_remote_ino": 0, 2024-06-03T03:28:59.477 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_lock": 0, 2024-06-03T03:28:59.477 INFO:teuthology.orchestra.run.smithi183.stdout: "load_cent": 0, 2024-06-03T03:28:59.477 INFO:teuthology.orchestra.run.smithi183.stdout: "q": 0, 2024-06-03T03:28:59.477 INFO:teuthology.orchestra.run.smithi183.stdout: "exported": 0, 2024-06-03T03:28:59.477 INFO:teuthology.orchestra.run.smithi183.stdout: "exported_inodes": 0, 2024-06-03T03:28:59.477 INFO:teuthology.orchestra.run.smithi183.stdout: "imported": 0, 2024-06-03T03:28:59.478 INFO:teuthology.orchestra.run.smithi183.stdout: "imported_inodes": 0, 2024-06-03T03:28:59.478 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_dir_fetch": 0, 2024-06-03T03:28:59.478 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_backtrace_fetch": 0, 2024-06-03T03:28:59.478 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_peer_discover": 0, 2024-06-03T03:28:59.478 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rfiles": 4, 2024-06-03T03:28:59.478 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rbytes": 234, 2024-06-03T03:28:59.478 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rsnaps": 0, 2024-06-03T03:28:59.478 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_fetch": 0, 2024-06-03T03:28:59.478 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_set_tag": 0, 2024-06-03T03:28:59.478 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_repaired": 0, 2024-06-03T03:28:59.478 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_inotable_repaired": 0, 2024-06-03T03:28:59.478 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_inodes": 0, 2024-06-03T03:28:59.478 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_base_inodes": 0, 2024-06-03T03:28:59.479 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dirfrag_rstats": 0, 2024-06-03T03:28:59.479 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_file_inodes": 0, 2024-06-03T03:28:59.479 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_inode_file_caps": 0, 2024-06-03T03:28:59.479 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_revoke": 0, 2024-06-03T03:28:59.479 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_grant": 0, 2024-06-03T03:28:59.479 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_trunc": 0, 2024-06-03T03:28:59.479 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-03T03:28:59.479 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-03T03:28:59.479 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps": 0, 2024-06-03T03:28:59.479 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps_dirty": 0, 2024-06-03T03:28:59.479 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_cap_release": 0, 2024-06-03T03:28:59.479 INFO:teuthology.orchestra.run.smithi183.stdout: "process_request_cap_release": 0 2024-06-03T03:28:59.479 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:59.479 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_cache": { 2024-06-03T03:28:59.480 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update": 0, 2024-06-03T03:28:59.480 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update_receipt": 0, 2024-06-03T03:28:59.480 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_try_discover": 0, 2024-06-03T03:28:59.480 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_send_discover": 1, 2024-06-03T03:28:59.480 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_handle_discover": 0, 2024-06-03T03:28:59.480 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays": 0, 2024-06-03T03:28:59.480 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_delayed": 0, 2024-06-03T03:28:59.480 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_enqueuing": 0, 2024-06-03T03:28:59.480 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_created": 0, 2024-06-03T03:28:59.480 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_enqueued": 0, 2024-06-03T03:28:59.480 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_reintegrated": 0, 2024-06-03T03:28:59.480 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_migrated": 0, 2024-06-03T03:28:59.480 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_processing": 0, 2024-06-03T03:28:59.480 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_enqueued": 0, 2024-06-03T03:28:59.481 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_prioritized": 0, 2024-06-03T03:28:59.481 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_started": 0, 2024-06-03T03:28:59.481 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_completed": 0, 2024-06-03T03:28:59.481 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_path": 0, 2024-06-03T03:28:59.481 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_inode": 0, 2024-06-03T03:28:59.481 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_enqueue_scrub": 0, 2024-06-03T03:28:59.481 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_exportdir": 0, 2024-06-03T03:28:59.481 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_flush": 0, 2024-06-03T03:28:59.481 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragmentdir": 0, 2024-06-03T03:28:59.481 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragstats": 0, 2024-06-03T03:28:59.481 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_inodestats": 0 2024-06-03T03:28:59.481 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:59.481 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_log": { 2024-06-03T03:28:59.481 INFO:teuthology.orchestra.run.smithi183.stdout: "evlrg": 0, 2024-06-03T03:28:59.482 INFO:teuthology.orchestra.run.smithi183.stdout: "evadd": 4, 2024-06-03T03:28:59.482 INFO:teuthology.orchestra.run.smithi183.stdout: "evex": 0, 2024-06-03T03:28:59.482 INFO:teuthology.orchestra.run.smithi183.stdout: "evtrm": 0, 2024-06-03T03:28:59.482 INFO:teuthology.orchestra.run.smithi183.stdout: "ev": 4, 2024-06-03T03:28:59.482 INFO:teuthology.orchestra.run.smithi183.stdout: "evexg": 0, 2024-06-03T03:28:59.482 INFO:teuthology.orchestra.run.smithi183.stdout: "evexd": 0, 2024-06-03T03:28:59.482 INFO:teuthology.orchestra.run.smithi183.stdout: "segadd": 2, 2024-06-03T03:28:59.482 INFO:teuthology.orchestra.run.smithi183.stdout: "segex": 0, 2024-06-03T03:28:59.482 INFO:teuthology.orchestra.run.smithi183.stdout: "segtrm": 0, 2024-06-03T03:28:59.482 INFO:teuthology.orchestra.run.smithi183.stdout: "seg": 2, 2024-06-03T03:28:59.482 INFO:teuthology.orchestra.run.smithi183.stdout: "segmjr": 2, 2024-06-03T03:28:59.482 INFO:teuthology.orchestra.run.smithi183.stdout: "segexg": 0, 2024-06-03T03:28:59.482 INFO:teuthology.orchestra.run.smithi183.stdout: "segexd": 0, 2024-06-03T03:28:59.482 INFO:teuthology.orchestra.run.smithi183.stdout: "expos": 4194304, 2024-06-03T03:28:59.483 INFO:teuthology.orchestra.run.smithi183.stdout: "wrpos": 4195423, 2024-06-03T03:28:59.483 INFO:teuthology.orchestra.run.smithi183.stdout: "rdpos": 0, 2024-06-03T03:28:59.483 INFO:teuthology.orchestra.run.smithi183.stdout: "jlat": { 2024-06-03T03:28:59.483 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 3, 2024-06-03T03:28:59.483 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.008038080, 2024-06-03T03:28:59.483 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.002679360 2024-06-03T03:28:59.483 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:59.483 INFO:teuthology.orchestra.run.smithi183.stdout: "replayed": 0 2024-06-03T03:28:59.483 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:59.483 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_mem": { 2024-06-03T03:28:59.483 INFO:teuthology.orchestra.run.smithi183.stdout: "ino": 13, 2024-06-03T03:28:59.483 INFO:teuthology.orchestra.run.smithi183.stdout: "ino+": 13, 2024-06-03T03:28:59.483 INFO:teuthology.orchestra.run.smithi183.stdout: "ino-": 0, 2024-06-03T03:28:59.483 INFO:teuthology.orchestra.run.smithi183.stdout: "dir": 11, 2024-06-03T03:28:59.484 INFO:teuthology.orchestra.run.smithi183.stdout: "dir+": 12, 2024-06-03T03:28:59.484 INFO:teuthology.orchestra.run.smithi183.stdout: "dir-": 1, 2024-06-03T03:28:59.484 INFO:teuthology.orchestra.run.smithi183.stdout: "dn": 10, 2024-06-03T03:28:59.484 INFO:teuthology.orchestra.run.smithi183.stdout: "dn+": 10, 2024-06-03T03:28:59.484 INFO:teuthology.orchestra.run.smithi183.stdout: "dn-": 0, 2024-06-03T03:28:59.484 INFO:teuthology.orchestra.run.smithi183.stdout: "cap": 0, 2024-06-03T03:28:59.484 INFO:teuthology.orchestra.run.smithi183.stdout: "cap+": 0, 2024-06-03T03:28:59.484 INFO:teuthology.orchestra.run.smithi183.stdout: "cap-": 0, 2024-06-03T03:28:59.484 INFO:teuthology.orchestra.run.smithi183.stdout: "rss": 51464, 2024-06-03T03:28:59.484 INFO:teuthology.orchestra.run.smithi183.stdout: "heap": 223612 2024-06-03T03:28:59.484 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:59.484 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_server": { 2024-06-03T03:28:59.484 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_client_request": 0, 2024-06-03T03:28:59.484 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_server_request": 0, 2024-06-03T03:28:59.485 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_request": 0, 2024-06-03T03:28:59.485 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_session": 4, 2024-06-03T03:28:59.485 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_peer_request": 0, 2024-06-03T03:28:59.485 INFO:teuthology.orchestra.run.smithi183.stdout: "req_create_latency": { 2024-06-03T03:28:59.485 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:28:59.485 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:28:59.485 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:28:59.485 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:59.485 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getattr_latency": { 2024-06-03T03:28:59.485 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:28:59.485 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:28:59.485 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:28:59.485 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:59.485 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getfilelock_latency": { 2024-06-03T03:28:59.486 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:28:59.486 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:28:59.486 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:28:59.486 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:59.486 INFO:teuthology.orchestra.run.smithi183.stdout: "req_link_latency": { 2024-06-03T03:28:59.486 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:28:59.486 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:28:59.486 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:28:59.486 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:59.486 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookup_latency": { 2024-06-03T03:28:59.486 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:28:59.486 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:28:59.486 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:28:59.486 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:59.487 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookuphash_latency": { 2024-06-03T03:28:59.487 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:28:59.487 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:28:59.487 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:28:59.487 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:59.487 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupino_latency": { 2024-06-03T03:28:59.487 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:28:59.487 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:28:59.487 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:28:59.487 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:59.487 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupname_latency": { 2024-06-03T03:28:59.487 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:28:59.487 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:28:59.487 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:28:59.488 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:59.488 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupparent_latency": { 2024-06-03T03:28:59.488 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:28:59.488 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:28:59.488 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:28:59.488 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:59.488 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupsnap_latency": { 2024-06-03T03:28:59.488 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:28:59.488 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:28:59.488 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:28:59.488 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:59.488 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lssnap_latency": { 2024-06-03T03:28:59.488 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:28:59.488 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:28:59.489 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:28:59.489 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:59.489 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mkdir_latency": { 2024-06-03T03:28:59.489 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:28:59.489 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:28:59.489 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:28:59.490 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:59.490 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mknod_latency": { 2024-06-03T03:28:59.490 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:28:59.490 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:28:59.490 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:28:59.490 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:59.490 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mksnap_latency": { 2024-06-03T03:28:59.490 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:28:59.490 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:28:59.490 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:28:59.490 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:59.490 INFO:teuthology.orchestra.run.smithi183.stdout: "req_open_latency": { 2024-06-03T03:28:59.491 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:28:59.491 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:28:59.491 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:28:59.491 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:59.491 INFO:teuthology.orchestra.run.smithi183.stdout: "req_readdir_latency": { 2024-06-03T03:28:59.491 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:28:59.491 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:28:59.491 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:28:59.491 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:59.491 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rename_latency": { 2024-06-03T03:28:59.491 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:28:59.491 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:28:59.491 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:28:59.491 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:59.492 INFO:teuthology.orchestra.run.smithi183.stdout: "req_renamesnap_latency": { 2024-06-03T03:28:59.492 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:28:59.492 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:28:59.492 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:28:59.492 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:59.492 INFO:teuthology.orchestra.run.smithi183.stdout: "req_snapdiff_latency": { 2024-06-03T03:28:59.492 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:28:59.492 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:28:59.492 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:28:59.492 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:59.492 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmdir_latency": { 2024-06-03T03:28:59.492 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:28:59.492 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:28:59.493 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:28:59.493 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:59.493 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmsnap_latency": { 2024-06-03T03:28:59.493 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:28:59.493 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:28:59.493 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:28:59.493 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:59.493 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmxattr_latency": { 2024-06-03T03:28:59.493 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:28:59.493 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:28:59.493 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:28:59.493 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:59.493 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setattr_latency": { 2024-06-03T03:28:59.494 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:28:59.494 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:28:59.494 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:28:59.494 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:59.494 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setdirlayout_latency": { 2024-06-03T03:28:59.494 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:28:59.494 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:28:59.494 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:28:59.494 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:59.494 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setfilelock_latency": { 2024-06-03T03:28:59.494 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:28:59.494 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:28:59.494 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:28:59.495 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:59.495 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setlayout_latency": { 2024-06-03T03:28:59.495 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:28:59.495 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:28:59.495 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:28:59.495 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:59.495 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setxattr_latency": { 2024-06-03T03:28:59.495 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:28:59.495 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:28:59.495 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:28:59.495 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:59.495 INFO:teuthology.orchestra.run.smithi183.stdout: "req_symlink_latency": { 2024-06-03T03:28:59.495 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:28:59.495 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:28:59.496 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:28:59.496 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:59.496 INFO:teuthology.orchestra.run.smithi183.stdout: "req_unlink_latency": { 2024-06-03T03:28:59.496 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:28:59.496 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:28:59.496 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:28:59.496 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:59.496 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_revoke_eviction": 0, 2024-06-03T03:28:59.496 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_acquisition_throttle": 0, 2024-06-03T03:28:59.496 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getvxattr_latency": { 2024-06-03T03:28:59.496 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:28:59.496 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:28:59.496 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:28:59.497 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-06-03T03:28:59.497 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:59.497 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_sessions": { 2024-06-03T03:28:59.497 INFO:teuthology.orchestra.run.smithi183.stdout: "session_count": 0, 2024-06-03T03:28:59.497 INFO:teuthology.orchestra.run.smithi183.stdout: "session_add": 1, 2024-06-03T03:28:59.497 INFO:teuthology.orchestra.run.smithi183.stdout: "session_remove": 1, 2024-06-03T03:28:59.497 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_open": 0, 2024-06-03T03:28:59.497 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_stale": 0, 2024-06-03T03:28:59.497 INFO:teuthology.orchestra.run.smithi183.stdout: "total_load": 0, 2024-06-03T03:28:59.497 INFO:teuthology.orchestra.run.smithi183.stdout: "average_load": 0, 2024-06-03T03:28:59.497 INFO:teuthology.orchestra.run.smithi183.stdout: "avg_session_uptime": 71, 2024-06-03T03:28:59.497 INFO:teuthology.orchestra.run.smithi183.stdout: "mdthresh_evicted": 0 2024-06-03T03:28:59.497 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:59.497 INFO:teuthology.orchestra.run.smithi183.stdout: "mempool": { 2024-06-03T03:28:59.498 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_bytes": 0, 2024-06-03T03:28:59.498 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_items": 0, 2024-06-03T03:28:59.498 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_bytes": 0, 2024-06-03T03:28:59.498 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_items": 0, 2024-06-03T03:28:59.498 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_bytes": 0, 2024-06-03T03:28:59.498 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_items": 0, 2024-06-03T03:28:59.498 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-03T03:28:59.498 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_items": 0, 2024-06-03T03:28:59.498 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-03T03:28:59.498 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_items": 0, 2024-06-03T03:28:59.498 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_bytes": 0, 2024-06-03T03:28:59.498 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_items": 0, 2024-06-03T03:28:59.498 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-03T03:28:59.498 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_items": 0, 2024-06-03T03:28:59.499 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_bytes": 0, 2024-06-03T03:28:59.499 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_items": 0, 2024-06-03T03:28:59.499 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_bytes": 0, 2024-06-03T03:28:59.499 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_items": 0, 2024-06-03T03:28:59.499 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-03T03:28:59.499 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_items": 0, 2024-06-03T03:28:59.499 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-03T03:28:59.499 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_items": 0, 2024-06-03T03:28:59.499 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_bytes": 0, 2024-06-03T03:28:59.499 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_items": 0, 2024-06-03T03:28:59.499 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_bytes": 0, 2024-06-03T03:28:59.499 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_items": 0, 2024-06-03T03:28:59.499 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-03T03:28:59.499 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_items": 0, 2024-06-03T03:28:59.500 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_bytes": 0, 2024-06-03T03:28:59.500 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_items": 0, 2024-06-03T03:28:59.500 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_bytes": 0, 2024-06-03T03:28:59.500 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_items": 0, 2024-06-03T03:28:59.500 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_bytes": 0, 2024-06-03T03:28:59.500 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_items": 0, 2024-06-03T03:28:59.500 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_bytes": 0, 2024-06-03T03:28:59.500 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_items": 0, 2024-06-03T03:28:59.500 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_bytes": 238327, 2024-06-03T03:28:59.500 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_items": 79, 2024-06-03T03:28:59.500 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_bytes": 0, 2024-06-03T03:28:59.500 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_items": 0, 2024-06-03T03:28:59.500 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_bytes": 0, 2024-06-03T03:28:59.500 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_items": 0, 2024-06-03T03:28:59.501 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_bytes": 0, 2024-06-03T03:28:59.501 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_items": 0, 2024-06-03T03:28:59.501 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_bytes": 0, 2024-06-03T03:28:59.501 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_items": 0, 2024-06-03T03:28:59.501 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_bytes": 8888, 2024-06-03T03:28:59.501 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_items": 170, 2024-06-03T03:28:59.501 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_bytes": 0, 2024-06-03T03:28:59.501 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_items": 0, 2024-06-03T03:28:59.501 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_bytes": 0, 2024-06-03T03:28:59.501 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_items": 0, 2024-06-03T03:28:59.501 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_bytes": 50664, 2024-06-03T03:28:59.501 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_items": 198, 2024-06-03T03:28:59.501 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_bytes": 0, 2024-06-03T03:28:59.501 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_items": 0, 2024-06-03T03:28:59.502 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_bytes": 0, 2024-06-03T03:28:59.502 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_items": 0 2024-06-03T03:28:59.502 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:59.502 INFO:teuthology.orchestra.run.smithi183.stdout: "objecter": { 2024-06-03T03:28:59.502 INFO:teuthology.orchestra.run.smithi183.stdout: "op_active": 0, 2024-06-03T03:28:59.502 INFO:teuthology.orchestra.run.smithi183.stdout: "op_laggy": 0, 2024-06-03T03:28:59.502 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send": 39, 2024-06-03T03:28:59.502 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send_bytes": 11421, 2024-06-03T03:28:59.502 INFO:teuthology.orchestra.run.smithi183.stdout: "op_resend": 33, 2024-06-03T03:28:59.502 INFO:teuthology.orchestra.run.smithi183.stdout: "op_reply": 39, 2024-06-03T03:28:59.502 INFO:teuthology.orchestra.run.smithi183.stdout: "op_latency": { 2024-06-03T03:28:59.502 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 39, 2024-06-03T03:28:59.502 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.311991601, 2024-06-03T03:28:59.502 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.007999784 2024-06-03T03:28:59.503 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:59.503 INFO:teuthology.orchestra.run.smithi183.stdout: "op_inflight": 0, 2024-06-03T03:28:59.503 INFO:teuthology.orchestra.run.smithi183.stdout: "oplen_avg": { 2024-06-03T03:28:59.503 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 39, 2024-06-03T03:28:59.503 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 61 2024-06-03T03:28:59.503 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:59.503 INFO:teuthology.orchestra.run.smithi183.stdout: "op": 39, 2024-06-03T03:28:59.503 INFO:teuthology.orchestra.run.smithi183.stdout: "op_r": 0, 2024-06-03T03:28:59.503 INFO:teuthology.orchestra.run.smithi183.stdout: "op_w": 39, 2024-06-03T03:28:59.503 INFO:teuthology.orchestra.run.smithi183.stdout: "op_rmw": 0, 2024-06-03T03:28:59.503 INFO:teuthology.orchestra.run.smithi183.stdout: "op_pg": 0, 2024-06-03T03:28:59.503 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_stat": 0, 2024-06-03T03:28:59.503 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_create": 10, 2024-06-03T03:28:59.503 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_read": 0, 2024-06-03T03:28:59.503 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_write": 3, 2024-06-03T03:28:59.504 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writefull": 7, 2024-06-03T03:28:59.504 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writesame": 0, 2024-06-03T03:28:59.504 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_append": 0, 2024-06-03T03:28:59.504 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_zero": 0, 2024-06-03T03:28:59.504 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_truncate": 0, 2024-06-03T03:28:59.504 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_delete": 6, 2024-06-03T03:28:59.504 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_mapext": 0, 2024-06-03T03:28:59.504 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_sparse_read": 0, 2024-06-03T03:28:59.504 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_clonerange": 0, 2024-06-03T03:28:59.504 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_getxattr": 0, 2024-06-03T03:28:59.504 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_setxattr": 20, 2024-06-03T03:28:59.504 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_cmpxattr": 0, 2024-06-03T03:28:59.504 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_rmxattr": 0, 2024-06-03T03:28:59.504 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_resetxattrs": 0, 2024-06-03T03:28:59.505 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_call": 0, 2024-06-03T03:28:59.505 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_watch": 0, 2024-06-03T03:28:59.505 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_notify": 0, 2024-06-03T03:28:59.505 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_src_cmpxattr": 0, 2024-06-03T03:28:59.505 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls": 0, 2024-06-03T03:28:59.505 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls_filter": 0, 2024-06-03T03:28:59.505 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_other": 0, 2024-06-03T03:28:59.505 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_active": 0, 2024-06-03T03:28:59.505 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_send": 0, 2024-06-03T03:28:59.505 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_resend": 0, 2024-06-03T03:28:59.505 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_ping": 0, 2024-06-03T03:28:59.505 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_active": 0, 2024-06-03T03:28:59.505 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_send": 0, 2024-06-03T03:28:59.505 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_resend": 0, 2024-06-03T03:28:59.505 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_active": 0, 2024-06-03T03:28:59.506 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_send": 0, 2024-06-03T03:28:59.506 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_resend": 0, 2024-06-03T03:28:59.506 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_active": 0, 2024-06-03T03:28:59.506 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_send": 0, 2024-06-03T03:28:59.506 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_resend": 0, 2024-06-03T03:28:59.506 INFO:teuthology.orchestra.run.smithi183.stdout: "command_active": 0, 2024-06-03T03:28:59.506 INFO:teuthology.orchestra.run.smithi183.stdout: "command_send": 0, 2024-06-03T03:28:59.506 INFO:teuthology.orchestra.run.smithi183.stdout: "command_resend": 0, 2024-06-03T03:28:59.506 INFO:teuthology.orchestra.run.smithi183.stdout: "map_epoch": 99, 2024-06-03T03:28:59.506 INFO:teuthology.orchestra.run.smithi183.stdout: "map_full": 0, 2024-06-03T03:28:59.506 INFO:teuthology.orchestra.run.smithi183.stdout: "map_inc": 2, 2024-06-03T03:28:59.506 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_sessions": 9, 2024-06-03T03:28:59.506 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_open": 9, 2024-06-03T03:28:59.507 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_close": 0, 2024-06-03T03:28:59.507 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_laggy": 0, 2024-06-03T03:28:59.507 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_wr": 14, 2024-06-03T03:28:59.507 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_rd": 0, 2024-06-03T03:28:59.507 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_del": 1 2024-06-03T03:28:59.507 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:59.507 INFO:teuthology.orchestra.run.smithi183.stdout: "oft": { 2024-06-03T03:28:59.507 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_objs": 1, 2024-06-03T03:28:59.507 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_kv_pairs": 0, 2024-06-03T03:28:59.507 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_updates": 0, 2024-06-03T03:28:59.507 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_removes": 0 2024-06-03T03:28:59.507 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:59.507 INFO:teuthology.orchestra.run.smithi183.stdout: "osd-slow-ops": { 2024-06-03T03:28:59.507 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_ops_count": 0 2024-06-03T03:28:59.508 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:59.508 INFO:teuthology.orchestra.run.smithi183.stdout: "purge_queue": { 2024-06-03T03:28:59.508 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops": 0, 2024-06-03T03:28:59.508 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops_high_water": 0, 2024-06-03T03:28:59.508 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing": 0, 2024-06-03T03:28:59.508 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_high_water": 0, 2024-06-03T03:28:59.508 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed_ops": 0, 2024-06-03T03:28:59.508 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed": 0, 2024-06-03T03:28:59.508 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_item_in_journal": 0 2024-06-03T03:28:59.508 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:59.508 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-03T03:28:59.508 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-06-03T03:28:59.508 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-06-03T03:28:59.508 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-06-03T03:28:59.508 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 421, 2024-06-03T03:28:59.509 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 790785, 2024-06-03T03:28:59.509 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-06-03T03:28:59.509 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 421, 2024-06-03T03:28:59.509 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-06-03T03:28:59.509 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-06-03T03:28:59.509 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 421, 2024-06-03T03:28:59.509 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 790785, 2024-06-03T03:28:59.509 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-06-03T03:28:59.509 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:28:59.509 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:28:59.509 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:28:59.509 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-06-03T03:28:59.509 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:59.509 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_bytes": { 2024-06-03T03:28:59.510 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-06-03T03:28:59.510 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-06-03T03:28:59.510 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-06-03T03:28:59.510 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-06-03T03:28:59.510 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-06-03T03:28:59.510 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-06-03T03:28:59.510 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-06-03T03:28:59.510 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 39, 2024-06-03T03:28:59.510 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 11421, 2024-06-03T03:28:59.510 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 39, 2024-06-03T03:28:59.510 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 11421, 2024-06-03T03:28:59.510 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-06-03T03:28:59.510 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:28:59.510 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:28:59.511 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:28:59.511 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-06-03T03:28:59.511 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:59.511 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_ops": { 2024-06-03T03:28:59.511 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-06-03T03:28:59.511 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 1024, 2024-06-03T03:28:59.511 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-06-03T03:28:59.511 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-06-03T03:28:59.511 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-06-03T03:28:59.511 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-06-03T03:28:59.511 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-06-03T03:28:59.511 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 39, 2024-06-03T03:28:59.511 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 39, 2024-06-03T03:28:59.511 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 39, 2024-06-03T03:28:59.512 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 39, 2024-06-03T03:28:59.512 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-06-03T03:28:59.512 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:28:59.512 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:28:59.512 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:28:59.512 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-06-03T03:28:59.512 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:59.512 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle": { 2024-06-03T03:28:59.512 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-06-03T03:28:59.512 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-06-03T03:28:59.512 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-06-03T03:28:59.512 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-06-03T03:28:59.512 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-06-03T03:28:59.512 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-06-03T03:28:59.512 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-06-03T03:28:59.513 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-06-03T03:28:59.513 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-06-03T03:28:59.513 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 0, 2024-06-03T03:28:59.513 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 0, 2024-06-03T03:28:59.513 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-06-03T03:28:59.513 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:28:59.513 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:28:59.513 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:28:59.513 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-06-03T03:28:59.513 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:28:59.513 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle-0x55e5c0fb8500": { 2024-06-03T03:28:59.513 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-06-03T03:28:59.513 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-06-03T03:28:59.513 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-06-03T03:28:59.514 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 4, 2024-06-03T03:28:59.514 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 1119, 2024-06-03T03:28:59.514 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-06-03T03:28:59.514 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 4, 2024-06-03T03:28:59.514 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-06-03T03:28:59.514 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-06-03T03:28:59.514 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 3, 2024-06-03T03:28:59.514 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 1119, 2024-06-03T03:28:59.514 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-06-03T03:28:59.514 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:28:59.514 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:28:59.514 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:28:59.514 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-06-03T03:28:59.514 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-06-03T03:28:59.515 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-06-03T03:29:00.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:29:00 smithi183 ceph-mon[37326]: pgmap v616: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-06-03T03:29:00.337 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-06-03T03:29:00.337 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-06-03T03:29:00.338 DEBUG:tasks.check_counter:Getting stats from i 2024-06-03T03:29:00.338 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph daemon mds.i perf dump 2024-06-03T03:29:00.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:29:00 smithi151 ceph-mon[37043]: pgmap v616: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-06-03T03:29:00.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:29:00 smithi117 ceph-mon[31649]: pgmap v616: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-06-03T03:29:00.581 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.c/config 2024-06-03T03:29:01.231 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:29:01 smithi183 ceph-mon[37326]: pgmap v617: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-06-03T03:29:01.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:29:01 smithi151 ceph-mon[37043]: pgmap v617: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-06-03T03:29:01.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:29:01 smithi117 ceph-mon[31649]: pgmap v617: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-06-03T03:29:02.287 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-06-03T03:29:02.287 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-0": { 2024-06-03T03:29:02.288 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 1056, 2024-06-03T03:29:02.288 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 840, 2024-06-03T03:29:02.288 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 256928, 2024-06-03T03:29:02.288 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 1110278, 2024-06-03T03:29:02.288 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 26, 2024-06-03T03:29:02.288 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 4, 2024-06-03T03:29:02.288 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.103940121, 2024-06-03T03:29:02.288 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.034919531, 2024-06-03T03:29:02.288 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.034991982, 2024-06-03T03:29:02.288 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.025134321, 2024-06-03T03:29:02.288 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T03:29:02.288 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 838, 2024-06-03T03:29:02.288 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.184597137, 2024-06-03T03:29:02.289 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000220282 2024-06-03T03:29:02.289 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:02.289 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-06-03T03:29:02.289 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 917, 2024-06-03T03:29:02.289 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000217466, 2024-06-03T03:29:02.289 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000237 2024-06-03T03:29:02.289 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:02.289 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-03T03:29:02.289 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 0 2024-06-03T03:29:02.289 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:02.289 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-1": { 2024-06-03T03:29:02.289 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 621, 2024-06-03T03:29:02.289 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 621, 2024-06-03T03:29:02.290 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 139604, 2024-06-03T03:29:02.290 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 469995, 2024-06-03T03:29:02.290 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 9, 2024-06-03T03:29:02.290 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 6, 2024-06-03T03:29:02.290 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.058696546, 2024-06-03T03:29:02.290 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.020597692, 2024-06-03T03:29:02.290 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.018579708, 2024-06-03T03:29:02.290 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.015404333, 2024-06-03T03:29:02.290 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T03:29:02.290 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 622, 2024-06-03T03:29:02.290 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.542367513, 2024-06-03T03:29:02.290 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000871973 2024-06-03T03:29:02.291 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:02.291 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-06-03T03:29:02.291 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 434, 2024-06-03T03:29:02.291 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000108238, 2024-06-03T03:29:02.291 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000249 2024-06-03T03:29:02.291 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:02.291 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 160, 2024-06-03T03:29:02.291 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 289712 2024-06-03T03:29:02.291 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:02.291 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-2": { 2024-06-03T03:29:02.291 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 1423, 2024-06-03T03:29:02.291 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 964, 2024-06-03T03:29:02.291 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 1273126, 2024-06-03T03:29:02.292 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 822980, 2024-06-03T03:29:02.292 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 48, 2024-06-03T03:29:02.292 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 10, 2024-06-03T03:29:02.292 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.184337981, 2024-06-03T03:29:02.292 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.050737541, 2024-06-03T03:29:02.292 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.089501831, 2024-06-03T03:29:02.292 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.048586401, 2024-06-03T03:29:02.292 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T03:29:02.292 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 961, 2024-06-03T03:29:02.292 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.169752008, 2024-06-03T03:29:02.292 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000176641 2024-06-03T03:29:02.292 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:02.292 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-06-03T03:29:02.293 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1364, 2024-06-03T03:29:02.293 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000304269, 2024-06-03T03:29:02.293 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000223 2024-06-03T03:29:02.293 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:02.293 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 1070160, 2024-06-03T03:29:02.293 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 210320 2024-06-03T03:29:02.293 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:02.293 INFO:teuthology.orchestra.run.smithi183.stdout: "cct": { 2024-06-03T03:29:02.293 INFO:teuthology.orchestra.run.smithi183.stdout: "total_workers": 1, 2024-06-03T03:29:02.293 INFO:teuthology.orchestra.run.smithi183.stdout: "unhealthy_workers": 0 2024-06-03T03:29:02.293 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:02.294 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-MDSRank": { 2024-06-03T03:29:02.294 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-06-03T03:29:02.294 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-06-03T03:29:02.294 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 555, 2024-06-03T03:29:02.294 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.120261635, 2024-06-03T03:29:02.294 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000216687 2024-06-03T03:29:02.294 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-06-03T03:29:02.294 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:02.294 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-PurgeQueue": { 2024-06-03T03:29:02.295 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-06-03T03:29:02.295 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-06-03T03:29:02.295 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 145, 2024-06-03T03:29:02.295 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.005474267, 2024-06-03T03:29:02.295 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000037753 2024-06-03T03:29:02.295 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-06-03T03:29:02.295 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:02.295 INFO:teuthology.orchestra.run.smithi183.stdout: "mds": { 2024-06-03T03:29:02.295 INFO:teuthology.orchestra.run.smithi183.stdout: "request": 59, 2024-06-03T03:29:02.295 INFO:teuthology.orchestra.run.smithi183.stdout: "reply": 59, 2024-06-03T03:29:02.295 INFO:teuthology.orchestra.run.smithi183.stdout: "reply_latency": { 2024-06-03T03:29:02.295 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 59, 2024-06-03T03:29:02.295 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.074551620, 2024-06-03T03:29:02.295 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.001263586 2024-06-03T03:29:02.296 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:02.296 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_reply": 0, 2024-06-03T03:29:02.296 INFO:teuthology.orchestra.run.smithi183.stdout: "forward": 0, 2024-06-03T03:29:02.296 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_complete": 0, 2024-06-03T03:29:02.296 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_keys": 0, 2024-06-03T03:29:02.296 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_commit": 60, 2024-06-03T03:29:02.296 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_split": 0, 2024-06-03T03:29:02.296 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_merge": 0, 2024-06-03T03:29:02.296 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes": 24, 2024-06-03T03:29:02.296 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_top": 5, 2024-06-03T03:29:02.297 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_bottom": 19, 2024-06-03T03:29:02.297 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pin_tail": 0, 2024-06-03T03:29:02.297 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pinned": 16, 2024-06-03T03:29:02.297 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_expired": 0, 2024-06-03T03:29:02.297 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_with_caps": 3, 2024-06-03T03:29:02.297 INFO:teuthology.orchestra.run.smithi183.stdout: "caps": 3, 2024-06-03T03:29:02.297 INFO:teuthology.orchestra.run.smithi183.stdout: "subtrees": 2, 2024-06-03T03:29:02.297 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse": 79, 2024-06-03T03:29:02.297 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_hit": 71, 2024-06-03T03:29:02.297 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_forward": 0, 2024-06-03T03:29:02.297 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_discover": 0, 2024-06-03T03:29:02.297 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_dir_fetch": 0, 2024-06-03T03:29:02.297 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_remote_ino": 0, 2024-06-03T03:29:02.298 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_lock": 0, 2024-06-03T03:29:02.298 INFO:teuthology.orchestra.run.smithi183.stdout: "load_cent": 0, 2024-06-03T03:29:02.298 INFO:teuthology.orchestra.run.smithi183.stdout: "q": 0, 2024-06-03T03:29:02.298 INFO:teuthology.orchestra.run.smithi183.stdout: "exported": 0, 2024-06-03T03:29:02.298 INFO:teuthology.orchestra.run.smithi183.stdout: "exported_inodes": 0, 2024-06-03T03:29:02.298 INFO:teuthology.orchestra.run.smithi183.stdout: "imported": 0, 2024-06-03T03:29:02.298 INFO:teuthology.orchestra.run.smithi183.stdout: "imported_inodes": 0, 2024-06-03T03:29:02.298 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_dir_fetch": 0, 2024-06-03T03:29:02.298 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_backtrace_fetch": 41, 2024-06-03T03:29:02.299 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_peer_discover": 0, 2024-06-03T03:29:02.299 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rfiles": 4, 2024-06-03T03:29:02.299 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rbytes": 234, 2024-06-03T03:29:02.299 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rsnaps": 0, 2024-06-03T03:29:02.299 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_fetch": 41, 2024-06-03T03:29:02.299 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_set_tag": 0, 2024-06-03T03:29:02.299 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_repaired": 0, 2024-06-03T03:29:02.299 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_inotable_repaired": 0, 2024-06-03T03:29:02.299 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_inodes": 25, 2024-06-03T03:29:02.299 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_base_inodes": 3, 2024-06-03T03:29:02.299 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dirfrag_rstats": 22, 2024-06-03T03:29:02.299 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_file_inodes": 16, 2024-06-03T03:29:02.299 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_inode_file_caps": 1, 2024-06-03T03:29:02.299 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_revoke": 15, 2024-06-03T03:29:02.300 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_grant": 32, 2024-06-03T03:29:02.300 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_trunc": 0, 2024-06-03T03:29:02.300 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-03T03:29:02.300 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-03T03:29:02.300 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps": 110, 2024-06-03T03:29:02.300 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps_dirty": 90, 2024-06-03T03:29:02.300 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_cap_release": 2, 2024-06-03T03:29:02.300 INFO:teuthology.orchestra.run.smithi183.stdout: "process_request_cap_release": 27 2024-06-03T03:29:02.300 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:02.300 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_cache": { 2024-06-03T03:29:02.301 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update": 0, 2024-06-03T03:29:02.301 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update_receipt": 0, 2024-06-03T03:29:02.301 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_try_discover": 0, 2024-06-03T03:29:02.301 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_send_discover": 0, 2024-06-03T03:29:02.301 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_handle_discover": 8, 2024-06-03T03:29:02.301 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays": 0, 2024-06-03T03:29:02.301 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_delayed": 0, 2024-06-03T03:29:02.301 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_enqueuing": 0, 2024-06-03T03:29:02.301 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_created": 6, 2024-06-03T03:29:02.301 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_enqueued": 6, 2024-06-03T03:29:02.301 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_reintegrated": 0, 2024-06-03T03:29:02.302 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_migrated": 0, 2024-06-03T03:29:02.302 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_processing": 0, 2024-06-03T03:29:02.302 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_enqueued": 0, 2024-06-03T03:29:02.302 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_prioritized": 0, 2024-06-03T03:29:02.302 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_started": 0, 2024-06-03T03:29:02.302 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_completed": 0, 2024-06-03T03:29:02.302 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_path": 0, 2024-06-03T03:29:02.302 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_inode": 0, 2024-06-03T03:29:02.302 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_enqueue_scrub": 3, 2024-06-03T03:29:02.302 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_exportdir": 0, 2024-06-03T03:29:02.302 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_flush": 0, 2024-06-03T03:29:02.303 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragmentdir": 0, 2024-06-03T03:29:02.303 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragstats": 0, 2024-06-03T03:29:02.303 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_inodestats": 0 2024-06-03T03:29:02.303 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:02.303 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_log": { 2024-06-03T03:29:02.303 INFO:teuthology.orchestra.run.smithi183.stdout: "evlrg": 0, 2024-06-03T03:29:02.303 INFO:teuthology.orchestra.run.smithi183.stdout: "evadd": 189, 2024-06-03T03:29:02.303 INFO:teuthology.orchestra.run.smithi183.stdout: "evex": 164, 2024-06-03T03:29:02.303 INFO:teuthology.orchestra.run.smithi183.stdout: "evtrm": 164, 2024-06-03T03:29:02.303 INFO:teuthology.orchestra.run.smithi183.stdout: "ev": 25, 2024-06-03T03:29:02.304 INFO:teuthology.orchestra.run.smithi183.stdout: "evexg": 0, 2024-06-03T03:29:02.304 INFO:teuthology.orchestra.run.smithi183.stdout: "evexd": 22, 2024-06-03T03:29:02.304 INFO:teuthology.orchestra.run.smithi183.stdout: "segadd": 13, 2024-06-03T03:29:02.304 INFO:teuthology.orchestra.run.smithi183.stdout: "segex": 12, 2024-06-03T03:29:02.304 INFO:teuthology.orchestra.run.smithi183.stdout: "segtrm": 12, 2024-06-03T03:29:02.304 INFO:teuthology.orchestra.run.smithi183.stdout: "seg": 1, 2024-06-03T03:29:02.304 INFO:teuthology.orchestra.run.smithi183.stdout: "segmjr": 1, 2024-06-03T03:29:02.304 INFO:teuthology.orchestra.run.smithi183.stdout: "segexg": 0, 2024-06-03T03:29:02.305 INFO:teuthology.orchestra.run.smithi183.stdout: "segexd": 1, 2024-06-03T03:29:02.305 INFO:teuthology.orchestra.run.smithi183.stdout: "expos": 4792199, 2024-06-03T03:29:02.305 INFO:teuthology.orchestra.run.smithi183.stdout: "wrpos": 4864647, 2024-06-03T03:29:02.305 INFO:teuthology.orchestra.run.smithi183.stdout: "rdpos": 0, 2024-06-03T03:29:02.305 INFO:teuthology.orchestra.run.smithi183.stdout: "jlat": { 2024-06-03T03:29:02.305 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 162, 2024-06-03T03:29:02.305 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.334921247, 2024-06-03T03:29:02.305 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.002067415 2024-06-03T03:29:02.305 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:02.305 INFO:teuthology.orchestra.run.smithi183.stdout: "replayed": 0 2024-06-03T03:29:02.305 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:02.305 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_mem": { 2024-06-03T03:29:02.305 INFO:teuthology.orchestra.run.smithi183.stdout: "ino": 23, 2024-06-03T03:29:02.306 INFO:teuthology.orchestra.run.smithi183.stdout: "ino+": 32, 2024-06-03T03:29:02.306 INFO:teuthology.orchestra.run.smithi183.stdout: "ino-": 9, 2024-06-03T03:29:02.306 INFO:teuthology.orchestra.run.smithi183.stdout: "dir": 18, 2024-06-03T03:29:02.306 INFO:teuthology.orchestra.run.smithi183.stdout: "dir+": 20, 2024-06-03T03:29:02.306 INFO:teuthology.orchestra.run.smithi183.stdout: "dir-": 2, 2024-06-03T03:29:02.306 INFO:teuthology.orchestra.run.smithi183.stdout: "dn": 24, 2024-06-03T03:29:02.306 INFO:teuthology.orchestra.run.smithi183.stdout: "dn+": 35, 2024-06-03T03:29:02.306 INFO:teuthology.orchestra.run.smithi183.stdout: "dn-": 11, 2024-06-03T03:29:02.306 INFO:teuthology.orchestra.run.smithi183.stdout: "cap": 3, 2024-06-03T03:29:02.306 INFO:teuthology.orchestra.run.smithi183.stdout: "cap+": 22, 2024-06-03T03:29:02.306 INFO:teuthology.orchestra.run.smithi183.stdout: "cap-": 19, 2024-06-03T03:29:02.306 INFO:teuthology.orchestra.run.smithi183.stdout: "rss": 57020, 2024-06-03T03:29:02.306 INFO:teuthology.orchestra.run.smithi183.stdout: "heap": 231804 2024-06-03T03:29:02.307 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:02.307 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_server": { 2024-06-03T03:29:02.307 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_client_request": 80, 2024-06-03T03:29:02.307 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_server_request": 1, 2024-06-03T03:29:02.307 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_request": 59, 2024-06-03T03:29:02.308 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_session": 19, 2024-06-03T03:29:02.308 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_peer_request": 2, 2024-06-03T03:29:02.308 INFO:teuthology.orchestra.run.smithi183.stdout: "req_create_latency": { 2024-06-03T03:29:02.308 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 8, 2024-06-03T03:29:02.308 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.007717139, 2024-06-03T03:29:02.308 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000964642 2024-06-03T03:29:02.308 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:02.308 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getattr_latency": { 2024-06-03T03:29:02.308 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 19, 2024-06-03T03:29:02.308 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.004478613, 2024-06-03T03:29:02.309 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000235716 2024-06-03T03:29:02.309 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:02.309 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getfilelock_latency": { 2024-06-03T03:29:02.309 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:29:02.309 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:29:02.309 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:29:02.309 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:02.309 INFO:teuthology.orchestra.run.smithi183.stdout: "req_link_latency": { 2024-06-03T03:29:02.309 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:29:02.309 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:29:02.309 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:29:02.309 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:02.309 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookup_latency": { 2024-06-03T03:29:02.309 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:29:02.310 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:29:02.310 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:29:02.310 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:02.310 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookuphash_latency": { 2024-06-03T03:29:02.310 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:29:02.310 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:29:02.310 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:29:02.310 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:02.310 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupino_latency": { 2024-06-03T03:29:02.310 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 2, 2024-06-03T03:29:02.310 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000339952, 2024-06-03T03:29:02.310 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000169976 2024-06-03T03:29:02.310 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:02.311 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupname_latency": { 2024-06-03T03:29:02.311 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:29:02.311 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:29:02.311 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:29:02.311 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:02.311 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupparent_latency": { 2024-06-03T03:29:02.311 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:29:02.311 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:29:02.311 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:29:02.311 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:02.311 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupsnap_latency": { 2024-06-03T03:29:02.311 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:29:02.311 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:29:02.312 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:29:02.312 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:02.312 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lssnap_latency": { 2024-06-03T03:29:02.312 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:29:02.312 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:29:02.312 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:29:02.312 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:02.313 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mkdir_latency": { 2024-06-03T03:29:02.313 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 8, 2024-06-03T03:29:02.313 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.015524921, 2024-06-03T03:29:02.313 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.001940615 2024-06-03T03:29:02.313 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:02.313 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mknod_latency": { 2024-06-03T03:29:02.313 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:29:02.314 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:29:02.314 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:29:02.314 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:02.314 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mksnap_latency": { 2024-06-03T03:29:02.314 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:29:02.314 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:29:02.314 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:29:02.314 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:02.314 INFO:teuthology.orchestra.run.smithi183.stdout: "req_open_latency": { 2024-06-03T03:29:02.314 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1, 2024-06-03T03:29:02.314 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000204508, 2024-06-03T03:29:02.315 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000204508 2024-06-03T03:29:02.315 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:02.315 INFO:teuthology.orchestra.run.smithi183.stdout: "req_readdir_latency": { 2024-06-03T03:29:02.315 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 2, 2024-06-03T03:29:02.315 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.003218587, 2024-06-03T03:29:02.315 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.001609293 2024-06-03T03:29:02.315 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:02.315 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rename_latency": { 2024-06-03T03:29:02.315 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 2, 2024-06-03T03:29:02.315 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.003043638, 2024-06-03T03:29:02.315 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.001521819 2024-06-03T03:29:02.315 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:02.315 INFO:teuthology.orchestra.run.smithi183.stdout: "req_renamesnap_latency": { 2024-06-03T03:29:02.315 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:29:02.316 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:29:02.316 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:29:02.316 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:02.316 INFO:teuthology.orchestra.run.smithi183.stdout: "req_snapdiff_latency": { 2024-06-03T03:29:02.316 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:29:02.316 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:29:02.316 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:29:02.316 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:02.316 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmdir_latency": { 2024-06-03T03:29:02.316 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 2, 2024-06-03T03:29:02.316 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.009742288, 2024-06-03T03:29:02.316 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.004871144 2024-06-03T03:29:02.316 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:02.317 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmsnap_latency": { 2024-06-03T03:29:02.317 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:29:02.317 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:29:02.317 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:29:02.317 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:02.317 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmxattr_latency": { 2024-06-03T03:29:02.317 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:29:02.317 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:29:02.317 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:29:02.317 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:02.317 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setattr_latency": { 2024-06-03T03:29:02.317 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 2, 2024-06-03T03:29:02.317 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.001514478, 2024-06-03T03:29:02.318 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000757239 2024-06-03T03:29:02.318 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:02.318 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setdirlayout_latency": { 2024-06-03T03:29:02.318 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:29:02.318 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:29:02.318 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:29:02.318 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:02.318 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setfilelock_latency": { 2024-06-03T03:29:02.318 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:29:02.318 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:29:02.318 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:29:02.318 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:02.318 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setlayout_latency": { 2024-06-03T03:29:02.319 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:29:02.319 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:29:02.319 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:29:02.319 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:02.319 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setxattr_latency": { 2024-06-03T03:29:02.319 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 9, 2024-06-03T03:29:02.319 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.023824404, 2024-06-03T03:29:02.319 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.002647156 2024-06-03T03:29:02.319 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:02.319 INFO:teuthology.orchestra.run.smithi183.stdout: "req_symlink_latency": { 2024-06-03T03:29:02.319 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:29:02.319 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:29:02.319 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:29:02.319 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:02.320 INFO:teuthology.orchestra.run.smithi183.stdout: "req_unlink_latency": { 2024-06-03T03:29:02.320 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 4, 2024-06-03T03:29:02.320 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.004943092, 2024-06-03T03:29:02.320 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.001235773 2024-06-03T03:29:02.320 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:02.320 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_revoke_eviction": 0, 2024-06-03T03:29:02.320 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_acquisition_throttle": 0, 2024-06-03T03:29:02.320 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getvxattr_latency": { 2024-06-03T03:29:02.320 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:29:02.320 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:29:02.320 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:29:02.320 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-06-03T03:29:02.320 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:02.321 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_sessions": { 2024-06-03T03:29:02.321 INFO:teuthology.orchestra.run.smithi183.stdout: "session_count": 1, 2024-06-03T03:29:02.321 INFO:teuthology.orchestra.run.smithi183.stdout: "session_add": 4, 2024-06-03T03:29:02.321 INFO:teuthology.orchestra.run.smithi183.stdout: "session_remove": 3, 2024-06-03T03:29:02.321 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_open": 1, 2024-06-03T03:29:02.321 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_stale": 0, 2024-06-03T03:29:02.321 INFO:teuthology.orchestra.run.smithi183.stdout: "total_load": 12, 2024-06-03T03:29:02.321 INFO:teuthology.orchestra.run.smithi183.stdout: "average_load": 12, 2024-06-03T03:29:02.321 INFO:teuthology.orchestra.run.smithi183.stdout: "avg_session_uptime": 226, 2024-06-03T03:29:02.321 INFO:teuthology.orchestra.run.smithi183.stdout: "mdthresh_evicted": 0 2024-06-03T03:29:02.321 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:02.321 INFO:teuthology.orchestra.run.smithi183.stdout: "mempool": { 2024-06-03T03:29:02.321 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_bytes": 0, 2024-06-03T03:29:02.322 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_items": 0, 2024-06-03T03:29:02.322 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_bytes": 0, 2024-06-03T03:29:02.322 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_items": 0, 2024-06-03T03:29:02.322 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_bytes": 0, 2024-06-03T03:29:02.322 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_items": 0, 2024-06-03T03:29:02.322 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-03T03:29:02.322 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_items": 0, 2024-06-03T03:29:02.322 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-03T03:29:02.322 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_items": 0, 2024-06-03T03:29:02.322 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_bytes": 0, 2024-06-03T03:29:02.322 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_items": 0, 2024-06-03T03:29:02.322 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-03T03:29:02.322 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_items": 0, 2024-06-03T03:29:02.323 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_bytes": 0, 2024-06-03T03:29:02.323 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_items": 0, 2024-06-03T03:29:02.323 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_bytes": 0, 2024-06-03T03:29:02.323 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_items": 0, 2024-06-03T03:29:02.323 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-03T03:29:02.323 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_items": 0, 2024-06-03T03:29:02.323 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-03T03:29:02.323 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_items": 0, 2024-06-03T03:29:02.323 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_bytes": 0, 2024-06-03T03:29:02.323 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_items": 0, 2024-06-03T03:29:02.323 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_bytes": 0, 2024-06-03T03:29:02.323 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_items": 0, 2024-06-03T03:29:02.324 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-03T03:29:02.324 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_items": 0, 2024-06-03T03:29:02.324 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_bytes": 0, 2024-06-03T03:29:02.324 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_items": 0, 2024-06-03T03:29:02.324 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_bytes": 0, 2024-06-03T03:29:02.324 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_items": 0, 2024-06-03T03:29:02.324 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_bytes": 0, 2024-06-03T03:29:02.324 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_items": 0, 2024-06-03T03:29:02.324 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_bytes": 0, 2024-06-03T03:29:02.324 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_items": 0, 2024-06-03T03:29:02.324 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_bytes": 309555, 2024-06-03T03:29:02.324 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_items": 142, 2024-06-03T03:29:02.324 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_bytes": 528, 2024-06-03T03:29:02.325 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_items": 6, 2024-06-03T03:29:02.325 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_bytes": 0, 2024-06-03T03:29:02.325 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_items": 0, 2024-06-03T03:29:02.325 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_bytes": 0, 2024-06-03T03:29:02.325 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_items": 0, 2024-06-03T03:29:02.325 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_bytes": 0, 2024-06-03T03:29:02.325 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_items": 0, 2024-06-03T03:29:02.325 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_bytes": 8888, 2024-06-03T03:29:02.325 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_items": 170, 2024-06-03T03:29:02.325 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_bytes": 0, 2024-06-03T03:29:02.325 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_items": 0, 2024-06-03T03:29:02.325 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_bytes": 0, 2024-06-03T03:29:02.325 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_items": 0, 2024-06-03T03:29:02.326 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_bytes": 94786, 2024-06-03T03:29:02.326 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_items": 616, 2024-06-03T03:29:02.326 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_bytes": 0, 2024-06-03T03:29:02.326 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_items": 0, 2024-06-03T03:29:02.326 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_bytes": 0, 2024-06-03T03:29:02.326 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_items": 0 2024-06-03T03:29:02.326 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:02.326 INFO:teuthology.orchestra.run.smithi183.stdout: "objecter": { 2024-06-03T03:29:02.326 INFO:teuthology.orchestra.run.smithi183.stdout: "op_active": 0, 2024-06-03T03:29:02.326 INFO:teuthology.orchestra.run.smithi183.stdout: "op_laggy": 0, 2024-06-03T03:29:02.326 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send": 1054, 2024-06-03T03:29:02.326 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send_bytes": 749150, 2024-06-03T03:29:02.326 INFO:teuthology.orchestra.run.smithi183.stdout: "op_resend": 31, 2024-06-03T03:29:02.327 INFO:teuthology.orchestra.run.smithi183.stdout: "op_reply": 1054, 2024-06-03T03:29:02.327 INFO:teuthology.orchestra.run.smithi183.stdout: "op_latency": { 2024-06-03T03:29:02.327 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1054, 2024-06-03T03:29:02.327 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 25.166320949, 2024-06-03T03:29:02.327 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.023876964 2024-06-03T03:29:02.327 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:02.327 INFO:teuthology.orchestra.run.smithi183.stdout: "op_inflight": 0, 2024-06-03T03:29:02.327 INFO:teuthology.orchestra.run.smithi183.stdout: "oplen_avg": { 2024-06-03T03:29:02.327 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1054, 2024-06-03T03:29:02.327 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 1239 2024-06-03T03:29:02.327 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:02.327 INFO:teuthology.orchestra.run.smithi183.stdout: "op": 1054, 2024-06-03T03:29:02.327 INFO:teuthology.orchestra.run.smithi183.stdout: "op_r": 49, 2024-06-03T03:29:02.327 INFO:teuthology.orchestra.run.smithi183.stdout: "op_w": 1005, 2024-06-03T03:29:02.328 INFO:teuthology.orchestra.run.smithi183.stdout: "op_rmw": 0, 2024-06-03T03:29:02.328 INFO:teuthology.orchestra.run.smithi183.stdout: "op_pg": 0, 2024-06-03T03:29:02.328 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_stat": 40, 2024-06-03T03:29:02.328 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_create": 36, 2024-06-03T03:29:02.328 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_read": 5, 2024-06-03T03:29:02.328 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_write": 164, 2024-06-03T03:29:02.328 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writefull": 50, 2024-06-03T03:29:02.328 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writesame": 0, 2024-06-03T03:29:02.328 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_append": 0, 2024-06-03T03:29:02.328 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_zero": 0, 2024-06-03T03:29:02.328 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_truncate": 0, 2024-06-03T03:29:02.328 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_delete": 650, 2024-06-03T03:29:02.329 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_mapext": 0, 2024-06-03T03:29:02.329 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_sparse_read": 0, 2024-06-03T03:29:02.329 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_clonerange": 0, 2024-06-03T03:29:02.329 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_getxattr": 44, 2024-06-03T03:29:02.329 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_setxattr": 64, 2024-06-03T03:29:02.329 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_cmpxattr": 0, 2024-06-03T03:29:02.329 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_rmxattr": 0, 2024-06-03T03:29:02.329 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_resetxattrs": 0, 2024-06-03T03:29:02.329 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_call": 0, 2024-06-03T03:29:02.329 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_watch": 0, 2024-06-03T03:29:02.330 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_notify": 0, 2024-06-03T03:29:02.330 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_src_cmpxattr": 0, 2024-06-03T03:29:02.330 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls": 0, 2024-06-03T03:29:02.330 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls_filter": 0, 2024-06-03T03:29:02.330 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_other": 0, 2024-06-03T03:29:02.330 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_active": 0, 2024-06-03T03:29:02.330 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_send": 0, 2024-06-03T03:29:02.330 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_resend": 0, 2024-06-03T03:29:02.330 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_ping": 0, 2024-06-03T03:29:02.330 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_active": 0, 2024-06-03T03:29:02.330 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_send": 0, 2024-06-03T03:29:02.330 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_resend": 0, 2024-06-03T03:29:02.330 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_active": 0, 2024-06-03T03:29:02.331 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_send": 0, 2024-06-03T03:29:02.331 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_resend": 0, 2024-06-03T03:29:02.331 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_active": 0, 2024-06-03T03:29:02.331 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_send": 0, 2024-06-03T03:29:02.331 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_resend": 0, 2024-06-03T03:29:02.331 INFO:teuthology.orchestra.run.smithi183.stdout: "command_active": 0, 2024-06-03T03:29:02.331 INFO:teuthology.orchestra.run.smithi183.stdout: "command_send": 0, 2024-06-03T03:29:02.331 INFO:teuthology.orchestra.run.smithi183.stdout: "command_resend": 0, 2024-06-03T03:29:02.331 INFO:teuthology.orchestra.run.smithi183.stdout: "map_epoch": 99, 2024-06-03T03:29:02.332 INFO:teuthology.orchestra.run.smithi183.stdout: "map_full": 0, 2024-06-03T03:29:02.332 INFO:teuthology.orchestra.run.smithi183.stdout: "map_inc": 11, 2024-06-03T03:29:02.332 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_sessions": 12, 2024-06-03T03:29:02.332 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_open": 12, 2024-06-03T03:29:02.332 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_close": 0, 2024-06-03T03:29:02.332 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_laggy": 0, 2024-06-03T03:29:02.332 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_wr": 171, 2024-06-03T03:29:02.332 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_rd": 0, 2024-06-03T03:29:02.332 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_del": 15 2024-06-03T03:29:02.332 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:02.332 INFO:teuthology.orchestra.run.smithi183.stdout: "oft": { 2024-06-03T03:29:02.332 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_objs": 1, 2024-06-03T03:29:02.332 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_kv_pairs": 0, 2024-06-03T03:29:02.332 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_updates": 0, 2024-06-03T03:29:02.333 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_removes": 0 2024-06-03T03:29:02.333 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:02.333 INFO:teuthology.orchestra.run.smithi183.stdout: "osd-slow-ops": { 2024-06-03T03:29:02.333 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_ops_count": 0 2024-06-03T03:29:02.333 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:02.333 INFO:teuthology.orchestra.run.smithi183.stdout: "purge_queue": { 2024-06-03T03:29:02.333 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops": 0, 2024-06-03T03:29:02.334 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops_high_water": 137, 2024-06-03T03:29:02.334 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing": 0, 2024-06-03T03:29:02.334 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_high_water": 5, 2024-06-03T03:29:02.334 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed_ops": 138, 2024-06-03T03:29:02.334 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed": 6, 2024-06-03T03:29:02.334 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_item_in_journal": 0 2024-06-03T03:29:02.334 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:02.334 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-03T03:29:02.334 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-06-03T03:29:02.334 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-06-03T03:29:02.334 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-06-03T03:29:02.334 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 3096, 2024-06-03T03:29:02.334 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 1350141, 2024-06-03T03:29:02.335 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-06-03T03:29:02.335 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 3096, 2024-06-03T03:29:02.335 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-06-03T03:29:02.335 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-06-03T03:29:02.335 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 3096, 2024-06-03T03:29:02.335 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 1350141, 2024-06-03T03:29:02.335 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-06-03T03:29:02.335 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:29:02.335 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:29:02.335 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:29:02.336 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-06-03T03:29:02.336 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:02.336 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_bytes": { 2024-06-03T03:29:02.336 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-06-03T03:29:02.336 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-06-03T03:29:02.336 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-06-03T03:29:02.336 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-06-03T03:29:02.337 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-06-03T03:29:02.337 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-06-03T03:29:02.337 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-06-03T03:29:02.337 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 1054, 2024-06-03T03:29:02.337 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 749788, 2024-06-03T03:29:02.337 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 1054, 2024-06-03T03:29:02.337 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 749788, 2024-06-03T03:29:02.337 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-06-03T03:29:02.337 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:29:02.337 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:29:02.337 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:29:02.337 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-06-03T03:29:02.338 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:02.338 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_ops": { 2024-06-03T03:29:02.338 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-06-03T03:29:02.338 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 1024, 2024-06-03T03:29:02.338 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-06-03T03:29:02.338 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-06-03T03:29:02.338 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-06-03T03:29:02.338 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-06-03T03:29:02.338 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-06-03T03:29:02.338 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 1054, 2024-06-03T03:29:02.339 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 1054, 2024-06-03T03:29:02.339 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 1054, 2024-06-03T03:29:02.339 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 1054, 2024-06-03T03:29:02.339 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-06-03T03:29:02.339 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:29:02.339 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:29:02.339 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:29:02.339 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-06-03T03:29:02.340 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:02.340 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle": { 2024-06-03T03:29:02.340 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-06-03T03:29:02.340 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-06-03T03:29:02.340 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-06-03T03:29:02.340 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 6, 2024-06-03T03:29:02.340 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 638, 2024-06-03T03:29:02.340 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-06-03T03:29:02.340 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 6, 2024-06-03T03:29:02.341 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-06-03T03:29:02.341 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-06-03T03:29:02.341 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 2, 2024-06-03T03:29:02.341 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 638, 2024-06-03T03:29:02.341 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-06-03T03:29:02.341 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:29:02.341 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:29:02.341 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:29:02.342 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-06-03T03:29:02.342 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:02.342 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle-0x5626c96f2500": { 2024-06-03T03:29:02.342 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-06-03T03:29:02.342 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-06-03T03:29:02.342 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-06-03T03:29:02.342 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 189, 2024-06-03T03:29:02.342 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 670343, 2024-06-03T03:29:02.342 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-06-03T03:29:02.342 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 189, 2024-06-03T03:29:02.342 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-06-03T03:29:02.342 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-06-03T03:29:02.342 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 162, 2024-06-03T03:29:02.343 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 670343, 2024-06-03T03:29:02.343 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-06-03T03:29:02.343 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:29:02.343 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:29:02.343 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:29:02.343 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-06-03T03:29:02.343 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-06-03T03:29:02.343 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-06-03T03:29:03.086 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-06-03T03:29:03.086 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-06-03T03:29:03.086 DEBUG:tasks.check_counter:Getting stats from l 2024-06-03T03:29:03.087 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph daemon mds.l perf dump 2024-06-03T03:29:03.231 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:29:03 smithi183 ceph-mon[37326]: pgmap v618: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-06-03T03:29:03.361 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.c/config 2024-06-03T03:29:03.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:29:03 smithi151 ceph-mon[37043]: pgmap v618: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-06-03T03:29:03.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:29:03 smithi117 ceph-mon[31649]: pgmap v618: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-06-03T03:29:05.082 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-06-03T03:29:05.082 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-0": { 2024-06-03T03:29:05.082 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 663, 2024-06-03T03:29:05.082 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 654, 2024-06-03T03:29:05.082 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 934576, 2024-06-03T03:29:05.082 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 256406, 2024-06-03T03:29:05.083 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 5, 2024-06-03T03:29:05.083 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 1, 2024-06-03T03:29:05.083 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.093188814, 2024-06-03T03:29:05.083 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.029443382, 2024-06-03T03:29:05.083 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.034719693, 2024-06-03T03:29:05.083 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.025528633, 2024-06-03T03:29:05.083 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T03:29:05.083 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 654, 2024-06-03T03:29:05.083 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.038575107, 2024-06-03T03:29:05.083 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000058983 2024-06-03T03:29:05.083 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:05.083 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-06-03T03:29:05.084 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:29:05.084 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:29:05.084 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:29:05.084 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:05.084 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 774304, 2024-06-03T03:29:05.084 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 104720 2024-06-03T03:29:05.084 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:05.084 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-1": { 2024-06-03T03:29:05.084 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 236, 2024-06-03T03:29:05.084 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 363, 2024-06-03T03:29:05.084 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 58622, 2024-06-03T03:29:05.084 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 339566, 2024-06-03T03:29:05.084 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 5, 2024-06-03T03:29:05.085 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 4, 2024-06-03T03:29:05.085 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.030375315, 2024-06-03T03:29:05.085 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.013908346, 2024-06-03T03:29:05.085 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.005559795, 2024-06-03T03:29:05.085 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.007599606, 2024-06-03T03:29:05.085 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T03:29:05.085 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 361, 2024-06-03T03:29:05.085 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.016858944, 2024-06-03T03:29:05.085 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000046700 2024-06-03T03:29:05.085 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:05.085 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-06-03T03:29:05.085 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:29:05.086 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:29:05.086 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:29:05.086 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:05.086 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 160, 2024-06-03T03:29:05.086 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 264768 2024-06-03T03:29:05.086 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:05.086 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-2": { 2024-06-03T03:29:05.086 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 482, 2024-06-03T03:29:05.086 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 484, 2024-06-03T03:29:05.086 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 127380, 2024-06-03T03:29:05.086 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 153277, 2024-06-03T03:29:05.086 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 4, 2024-06-03T03:29:05.087 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 3, 2024-06-03T03:29:05.087 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.041238895, 2024-06-03T03:29:05.087 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.014610090, 2024-06-03T03:29:05.087 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.010301204, 2024-06-03T03:29:05.087 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.011668947, 2024-06-03T03:29:05.087 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T03:29:05.087 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 482, 2024-06-03T03:29:05.087 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.020545437, 2024-06-03T03:29:05.087 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000042625 2024-06-03T03:29:05.087 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:05.087 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-06-03T03:29:05.087 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:29:05.087 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:29:05.088 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:29:05.088 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:05.088 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-03T03:29:05.088 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 0 2024-06-03T03:29:05.088 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:05.088 INFO:teuthology.orchestra.run.smithi183.stdout: "cct": { 2024-06-03T03:29:05.088 INFO:teuthology.orchestra.run.smithi183.stdout: "total_workers": 1, 2024-06-03T03:29:05.088 INFO:teuthology.orchestra.run.smithi183.stdout: "unhealthy_workers": 0 2024-06-03T03:29:05.088 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:05.089 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-MDSRank": { 2024-06-03T03:29:05.089 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-06-03T03:29:05.089 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-06-03T03:29:05.089 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 2133, 2024-06-03T03:29:05.089 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.064492362, 2024-06-03T03:29:05.089 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000030235 2024-06-03T03:29:05.089 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-06-03T03:29:05.089 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:05.089 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-PurgeQueue": { 2024-06-03T03:29:05.089 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-06-03T03:29:05.089 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-06-03T03:29:05.089 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:29:05.090 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:29:05.090 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:29:05.090 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-06-03T03:29:05.090 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:05.090 INFO:teuthology.orchestra.run.smithi183.stdout: "mds": { 2024-06-03T03:29:05.090 INFO:teuthology.orchestra.run.smithi183.stdout: "request": 0, 2024-06-03T03:29:05.090 INFO:teuthology.orchestra.run.smithi183.stdout: "reply": 0, 2024-06-03T03:29:05.090 INFO:teuthology.orchestra.run.smithi183.stdout: "reply_latency": { 2024-06-03T03:29:05.090 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:29:05.090 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:29:05.090 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:29:05.090 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:05.090 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_reply": 0, 2024-06-03T03:29:05.091 INFO:teuthology.orchestra.run.smithi183.stdout: "forward": 0, 2024-06-03T03:29:05.091 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_complete": 0, 2024-06-03T03:29:05.091 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_keys": 0, 2024-06-03T03:29:05.091 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_commit": 0, 2024-06-03T03:29:05.091 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_split": 0, 2024-06-03T03:29:05.091 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_merge": 0, 2024-06-03T03:29:05.091 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes": 0, 2024-06-03T03:29:05.091 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_top": 0, 2024-06-03T03:29:05.091 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_bottom": 0, 2024-06-03T03:29:05.091 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pin_tail": 0, 2024-06-03T03:29:05.091 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pinned": 0, 2024-06-03T03:29:05.091 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_expired": 0, 2024-06-03T03:29:05.091 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_with_caps": 0, 2024-06-03T03:29:05.092 INFO:teuthology.orchestra.run.smithi183.stdout: "caps": 0, 2024-06-03T03:29:05.092 INFO:teuthology.orchestra.run.smithi183.stdout: "subtrees": 1, 2024-06-03T03:29:05.092 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse": 0, 2024-06-03T03:29:05.092 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_hit": 0, 2024-06-03T03:29:05.092 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_forward": 0, 2024-06-03T03:29:05.092 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_discover": 0, 2024-06-03T03:29:05.092 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_dir_fetch": 0, 2024-06-03T03:29:05.092 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_remote_ino": 0, 2024-06-03T03:29:05.092 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_lock": 0, 2024-06-03T03:29:05.092 INFO:teuthology.orchestra.run.smithi183.stdout: "load_cent": 0, 2024-06-03T03:29:05.092 INFO:teuthology.orchestra.run.smithi183.stdout: "q": 0, 2024-06-03T03:29:05.092 INFO:teuthology.orchestra.run.smithi183.stdout: "exported": 0, 2024-06-03T03:29:05.092 INFO:teuthology.orchestra.run.smithi183.stdout: "exported_inodes": 0, 2024-06-03T03:29:05.092 INFO:teuthology.orchestra.run.smithi183.stdout: "imported": 0, 2024-06-03T03:29:05.093 INFO:teuthology.orchestra.run.smithi183.stdout: "imported_inodes": 0, 2024-06-03T03:29:05.093 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_dir_fetch": 0, 2024-06-03T03:29:05.093 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_backtrace_fetch": 0, 2024-06-03T03:29:05.093 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_peer_discover": 0, 2024-06-03T03:29:05.093 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rfiles": 0, 2024-06-03T03:29:05.093 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rbytes": 0, 2024-06-03T03:29:05.093 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rsnaps": 0, 2024-06-03T03:29:05.093 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_fetch": 0, 2024-06-03T03:29:05.093 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_set_tag": 0, 2024-06-03T03:29:05.093 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_repaired": 0, 2024-06-03T03:29:05.093 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_inotable_repaired": 0, 2024-06-03T03:29:05.093 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_inodes": 0, 2024-06-03T03:29:05.093 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_base_inodes": 0, 2024-06-03T03:29:05.094 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dirfrag_rstats": 0, 2024-06-03T03:29:05.094 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_file_inodes": 0, 2024-06-03T03:29:05.094 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_inode_file_caps": 0, 2024-06-03T03:29:05.094 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_revoke": 0, 2024-06-03T03:29:05.094 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_grant": 0, 2024-06-03T03:29:05.094 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_trunc": 0, 2024-06-03T03:29:05.094 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-03T03:29:05.094 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-03T03:29:05.094 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps": 0, 2024-06-03T03:29:05.094 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps_dirty": 0, 2024-06-03T03:29:05.094 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_cap_release": 0, 2024-06-03T03:29:05.094 INFO:teuthology.orchestra.run.smithi183.stdout: "process_request_cap_release": 0 2024-06-03T03:29:05.094 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:05.095 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_cache": { 2024-06-03T03:29:05.095 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update": 0, 2024-06-03T03:29:05.095 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update_receipt": 0, 2024-06-03T03:29:05.095 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_try_discover": 0, 2024-06-03T03:29:05.095 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_send_discover": 0, 2024-06-03T03:29:05.095 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_handle_discover": 0, 2024-06-03T03:29:05.095 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays": 0, 2024-06-03T03:29:05.095 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_delayed": 0, 2024-06-03T03:29:05.095 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_enqueuing": 0, 2024-06-03T03:29:05.095 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_created": 0, 2024-06-03T03:29:05.095 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_enqueued": 0, 2024-06-03T03:29:05.095 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_reintegrated": 0, 2024-06-03T03:29:05.095 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_migrated": 0, 2024-06-03T03:29:05.096 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_processing": 0, 2024-06-03T03:29:05.096 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_enqueued": 0, 2024-06-03T03:29:05.096 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_prioritized": 0, 2024-06-03T03:29:05.096 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_started": 0, 2024-06-03T03:29:05.096 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_completed": 0, 2024-06-03T03:29:05.096 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_path": 0, 2024-06-03T03:29:05.096 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_inode": 0, 2024-06-03T03:29:05.096 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_enqueue_scrub": 0, 2024-06-03T03:29:05.096 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_exportdir": 0, 2024-06-03T03:29:05.096 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_flush": 0, 2024-06-03T03:29:05.096 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragmentdir": 0, 2024-06-03T03:29:05.096 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragstats": 0, 2024-06-03T03:29:05.096 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_inodestats": 0 2024-06-03T03:29:05.097 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:05.097 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_log": { 2024-06-03T03:29:05.097 INFO:teuthology.orchestra.run.smithi183.stdout: "evlrg": 0, 2024-06-03T03:29:05.097 INFO:teuthology.orchestra.run.smithi183.stdout: "evadd": 0, 2024-06-03T03:29:05.097 INFO:teuthology.orchestra.run.smithi183.stdout: "evex": 0, 2024-06-03T03:29:05.097 INFO:teuthology.orchestra.run.smithi183.stdout: "evtrm": 0, 2024-06-03T03:29:05.097 INFO:teuthology.orchestra.run.smithi183.stdout: "ev": 4, 2024-06-03T03:29:05.097 INFO:teuthology.orchestra.run.smithi183.stdout: "evexg": 0, 2024-06-03T03:29:05.097 INFO:teuthology.orchestra.run.smithi183.stdout: "evexd": 0, 2024-06-03T03:29:05.097 INFO:teuthology.orchestra.run.smithi183.stdout: "segadd": 0, 2024-06-03T03:29:05.097 INFO:teuthology.orchestra.run.smithi183.stdout: "segex": 0, 2024-06-03T03:29:05.097 INFO:teuthology.orchestra.run.smithi183.stdout: "segtrm": 0, 2024-06-03T03:29:05.097 INFO:teuthology.orchestra.run.smithi183.stdout: "seg": 2, 2024-06-03T03:29:05.098 INFO:teuthology.orchestra.run.smithi183.stdout: "segmjr": 2, 2024-06-03T03:29:05.098 INFO:teuthology.orchestra.run.smithi183.stdout: "segexg": 0, 2024-06-03T03:29:05.098 INFO:teuthology.orchestra.run.smithi183.stdout: "segexd": 0, 2024-06-03T03:29:05.098 INFO:teuthology.orchestra.run.smithi183.stdout: "expos": 4194304, 2024-06-03T03:29:05.098 INFO:teuthology.orchestra.run.smithi183.stdout: "wrpos": 4195423, 2024-06-03T03:29:05.098 INFO:teuthology.orchestra.run.smithi183.stdout: "rdpos": 4195282, 2024-06-03T03:29:05.098 INFO:teuthology.orchestra.run.smithi183.stdout: "jlat": { 2024-06-03T03:29:05.098 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:29:05.098 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:29:05.098 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:29:05.098 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:05.098 INFO:teuthology.orchestra.run.smithi183.stdout: "replayed": 4 2024-06-03T03:29:05.099 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:05.099 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_mem": { 2024-06-03T03:29:05.099 INFO:teuthology.orchestra.run.smithi183.stdout: "ino": 3, 2024-06-03T03:29:05.099 INFO:teuthology.orchestra.run.smithi183.stdout: "ino+": 3, 2024-06-03T03:29:05.099 INFO:teuthology.orchestra.run.smithi183.stdout: "ino-": 0, 2024-06-03T03:29:05.099 INFO:teuthology.orchestra.run.smithi183.stdout: "dir": 1, 2024-06-03T03:29:05.099 INFO:teuthology.orchestra.run.smithi183.stdout: "dir+": 1, 2024-06-03T03:29:05.099 INFO:teuthology.orchestra.run.smithi183.stdout: "dir-": 0, 2024-06-03T03:29:05.099 INFO:teuthology.orchestra.run.smithi183.stdout: "dn": 0, 2024-06-03T03:29:05.099 INFO:teuthology.orchestra.run.smithi183.stdout: "dn+": 0, 2024-06-03T03:29:05.099 INFO:teuthology.orchestra.run.smithi183.stdout: "dn-": 0, 2024-06-03T03:29:05.100 INFO:teuthology.orchestra.run.smithi183.stdout: "cap": 0, 2024-06-03T03:29:05.100 INFO:teuthology.orchestra.run.smithi183.stdout: "cap+": 0, 2024-06-03T03:29:05.100 INFO:teuthology.orchestra.run.smithi183.stdout: "cap-": 0, 2024-06-03T03:29:05.100 INFO:teuthology.orchestra.run.smithi183.stdout: "rss": 50128, 2024-06-03T03:29:05.100 INFO:teuthology.orchestra.run.smithi183.stdout: "heap": 215420 2024-06-03T03:29:05.100 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:05.100 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_server": { 2024-06-03T03:29:05.100 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_client_request": 0, 2024-06-03T03:29:05.100 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_server_request": 0, 2024-06-03T03:29:05.100 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_request": 0, 2024-06-03T03:29:05.100 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_session": 0, 2024-06-03T03:29:05.100 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_peer_request": 0, 2024-06-03T03:29:05.101 INFO:teuthology.orchestra.run.smithi183.stdout: "req_create_latency": { 2024-06-03T03:29:05.101 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:29:05.101 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:29:05.101 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:29:05.101 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:05.101 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getattr_latency": { 2024-06-03T03:29:05.101 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:29:05.101 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:29:05.101 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:29:05.101 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:05.101 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getfilelock_latency": { 2024-06-03T03:29:05.101 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:29:05.101 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:29:05.101 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:29:05.102 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:05.102 INFO:teuthology.orchestra.run.smithi183.stdout: "req_link_latency": { 2024-06-03T03:29:05.102 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:29:05.102 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:29:05.102 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:29:05.102 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:05.102 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookup_latency": { 2024-06-03T03:29:05.102 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:29:05.102 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:29:05.102 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:29:05.102 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:05.102 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookuphash_latency": { 2024-06-03T03:29:05.102 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:29:05.103 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:29:05.103 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:29:05.103 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:05.103 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupino_latency": { 2024-06-03T03:29:05.103 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:29:05.103 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:29:05.103 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:29:05.103 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:05.103 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupname_latency": { 2024-06-03T03:29:05.103 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:29:05.103 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:29:05.103 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:29:05.103 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:05.103 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupparent_latency": { 2024-06-03T03:29:05.104 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:29:05.104 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:29:05.104 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:29:05.104 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:05.104 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupsnap_latency": { 2024-06-03T03:29:05.104 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:29:05.104 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:29:05.104 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:29:05.104 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:05.104 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lssnap_latency": { 2024-06-03T03:29:05.104 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:29:05.104 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:29:05.104 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:29:05.104 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:05.105 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mkdir_latency": { 2024-06-03T03:29:05.105 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:29:05.105 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:29:05.105 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:29:05.105 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:05.106 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mknod_latency": { 2024-06-03T03:29:05.106 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:29:05.106 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:29:05.106 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:29:05.106 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:05.106 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mksnap_latency": { 2024-06-03T03:29:05.106 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:29:05.106 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:29:05.106 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:29:05.106 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:05.106 INFO:teuthology.orchestra.run.smithi183.stdout: "req_open_latency": { 2024-06-03T03:29:05.106 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:29:05.106 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:29:05.107 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:29:05.107 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:05.107 INFO:teuthology.orchestra.run.smithi183.stdout: "req_readdir_latency": { 2024-06-03T03:29:05.107 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:29:05.107 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:29:05.107 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:29:05.107 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:05.107 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rename_latency": { 2024-06-03T03:29:05.107 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:29:05.107 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:29:05.107 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:29:05.107 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:05.107 INFO:teuthology.orchestra.run.smithi183.stdout: "req_renamesnap_latency": { 2024-06-03T03:29:05.108 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:29:05.108 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:29:05.108 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:29:05.108 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:05.108 INFO:teuthology.orchestra.run.smithi183.stdout: "req_snapdiff_latency": { 2024-06-03T03:29:05.108 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:29:05.108 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:29:05.108 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:29:05.108 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:05.108 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmdir_latency": { 2024-06-03T03:29:05.108 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:29:05.108 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:29:05.108 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:29:05.108 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:05.109 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmsnap_latency": { 2024-06-03T03:29:05.109 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:29:05.109 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:29:05.109 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:29:05.109 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:05.109 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmxattr_latency": { 2024-06-03T03:29:05.109 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:29:05.109 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:29:05.109 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:29:05.109 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:05.109 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setattr_latency": { 2024-06-03T03:29:05.109 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:29:05.109 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:29:05.109 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:29:05.109 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:05.110 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setdirlayout_latency": { 2024-06-03T03:29:05.110 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:29:05.110 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:29:05.110 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:29:05.110 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:05.110 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setfilelock_latency": { 2024-06-03T03:29:05.110 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:29:05.110 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:29:05.110 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:29:05.110 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:05.110 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setlayout_latency": { 2024-06-03T03:29:05.110 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:29:05.111 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:29:05.111 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:29:05.111 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:05.111 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setxattr_latency": { 2024-06-03T03:29:05.111 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:29:05.111 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:29:05.111 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:29:05.111 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:05.111 INFO:teuthology.orchestra.run.smithi183.stdout: "req_symlink_latency": { 2024-06-03T03:29:05.111 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:29:05.111 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:29:05.111 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:29:05.111 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:05.111 INFO:teuthology.orchestra.run.smithi183.stdout: "req_unlink_latency": { 2024-06-03T03:29:05.111 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:29:05.112 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:29:05.112 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:29:05.112 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:05.112 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_revoke_eviction": 0, 2024-06-03T03:29:05.112 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_acquisition_throttle": 0, 2024-06-03T03:29:05.112 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getvxattr_latency": { 2024-06-03T03:29:05.112 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:29:05.112 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:29:05.112 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:29:05.112 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-06-03T03:29:05.112 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:05.112 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_sessions": { 2024-06-03T03:29:05.112 INFO:teuthology.orchestra.run.smithi183.stdout: "session_count": 0, 2024-06-03T03:29:05.112 INFO:teuthology.orchestra.run.smithi183.stdout: "session_add": 1, 2024-06-03T03:29:05.113 INFO:teuthology.orchestra.run.smithi183.stdout: "session_remove": 1, 2024-06-03T03:29:05.113 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_open": 1, 2024-06-03T03:29:05.113 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_stale": 0, 2024-06-03T03:29:05.113 INFO:teuthology.orchestra.run.smithi183.stdout: "total_load": 0, 2024-06-03T03:29:05.113 INFO:teuthology.orchestra.run.smithi183.stdout: "average_load": 0, 2024-06-03T03:29:05.113 INFO:teuthology.orchestra.run.smithi183.stdout: "avg_session_uptime": 1844, 2024-06-03T03:29:05.113 INFO:teuthology.orchestra.run.smithi183.stdout: "mdthresh_evicted": 0 2024-06-03T03:29:05.113 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:05.113 INFO:teuthology.orchestra.run.smithi183.stdout: "mempool": { 2024-06-03T03:29:05.113 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_bytes": 0, 2024-06-03T03:29:05.113 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_items": 0, 2024-06-03T03:29:05.113 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_bytes": 0, 2024-06-03T03:29:05.113 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_items": 0, 2024-06-03T03:29:05.114 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_bytes": 0, 2024-06-03T03:29:05.114 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_items": 0, 2024-06-03T03:29:05.114 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-03T03:29:05.114 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_items": 0, 2024-06-03T03:29:05.114 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-03T03:29:05.114 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_items": 0, 2024-06-03T03:29:05.114 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_bytes": 0, 2024-06-03T03:29:05.114 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_items": 0, 2024-06-03T03:29:05.114 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-03T03:29:05.114 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_items": 0, 2024-06-03T03:29:05.114 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_bytes": 0, 2024-06-03T03:29:05.114 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_items": 0, 2024-06-03T03:29:05.114 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_bytes": 0, 2024-06-03T03:29:05.115 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_items": 0, 2024-06-03T03:29:05.115 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-03T03:29:05.115 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_items": 0, 2024-06-03T03:29:05.115 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-03T03:29:05.115 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_items": 0, 2024-06-03T03:29:05.115 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_bytes": 0, 2024-06-03T03:29:05.115 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_items": 0, 2024-06-03T03:29:05.115 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_bytes": 0, 2024-06-03T03:29:05.115 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_items": 0, 2024-06-03T03:29:05.115 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-03T03:29:05.115 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_items": 0, 2024-06-03T03:29:05.115 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_bytes": 0, 2024-06-03T03:29:05.115 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_items": 0, 2024-06-03T03:29:05.116 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_bytes": 0, 2024-06-03T03:29:05.116 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_items": 0, 2024-06-03T03:29:05.116 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_bytes": 0, 2024-06-03T03:29:05.116 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_items": 0, 2024-06-03T03:29:05.116 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_bytes": 0, 2024-06-03T03:29:05.116 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_items": 0, 2024-06-03T03:29:05.116 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_bytes": 189401, 2024-06-03T03:29:05.116 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_items": 51, 2024-06-03T03:29:05.116 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_bytes": 0, 2024-06-03T03:29:05.116 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_items": 0, 2024-06-03T03:29:05.116 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_bytes": 0, 2024-06-03T03:29:05.116 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_items": 0, 2024-06-03T03:29:05.116 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_bytes": 0, 2024-06-03T03:29:05.116 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_items": 0, 2024-06-03T03:29:05.116 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_bytes": 0, 2024-06-03T03:29:05.117 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_items": 0, 2024-06-03T03:29:05.117 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_bytes": 8888, 2024-06-03T03:29:05.117 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_items": 170, 2024-06-03T03:29:05.117 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_bytes": 0, 2024-06-03T03:29:05.117 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_items": 0, 2024-06-03T03:29:05.117 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_bytes": 0, 2024-06-03T03:29:05.117 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_items": 0, 2024-06-03T03:29:05.117 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_bytes": 8464, 2024-06-03T03:29:05.117 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_items": 13, 2024-06-03T03:29:05.117 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_bytes": 0, 2024-06-03T03:29:05.117 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_items": 0, 2024-06-03T03:29:05.117 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_bytes": 0, 2024-06-03T03:29:05.117 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_items": 0 2024-06-03T03:29:05.117 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:05.118 INFO:teuthology.orchestra.run.smithi183.stdout: "objecter": { 2024-06-03T03:29:05.118 INFO:teuthology.orchestra.run.smithi183.stdout: "op_active": 0, 2024-06-03T03:29:05.118 INFO:teuthology.orchestra.run.smithi183.stdout: "op_laggy": 0, 2024-06-03T03:29:05.118 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send": 1194, 2024-06-03T03:29:05.118 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send_bytes": 21, 2024-06-03T03:29:05.118 INFO:teuthology.orchestra.run.smithi183.stdout: "op_resend": 3, 2024-06-03T03:29:05.118 INFO:teuthology.orchestra.run.smithi183.stdout: "op_reply": 1194, 2024-06-03T03:29:05.118 INFO:teuthology.orchestra.run.smithi183.stdout: "op_latency": { 2024-06-03T03:29:05.118 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1194, 2024-06-03T03:29:05.118 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.558984914, 2024-06-03T03:29:05.118 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000468161 2024-06-03T03:29:05.118 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:05.119 INFO:teuthology.orchestra.run.smithi183.stdout: "op_inflight": 0, 2024-06-03T03:29:05.119 INFO:teuthology.orchestra.run.smithi183.stdout: "oplen_avg": { 2024-06-03T03:29:05.119 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1194, 2024-06-03T03:29:05.119 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 1195 2024-06-03T03:29:05.119 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:05.119 INFO:teuthology.orchestra.run.smithi183.stdout: "op": 1194, 2024-06-03T03:29:05.119 INFO:teuthology.orchestra.run.smithi183.stdout: "op_r": 1194, 2024-06-03T03:29:05.119 INFO:teuthology.orchestra.run.smithi183.stdout: "op_w": 0, 2024-06-03T03:29:05.119 INFO:teuthology.orchestra.run.smithi183.stdout: "op_rmw": 0, 2024-06-03T03:29:05.119 INFO:teuthology.orchestra.run.smithi183.stdout: "op_pg": 0, 2024-06-03T03:29:05.119 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_stat": 709, 2024-06-03T03:29:05.119 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_create": 0, 2024-06-03T03:29:05.119 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_read": 483, 2024-06-03T03:29:05.119 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_write": 0, 2024-06-03T03:29:05.120 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writefull": 0, 2024-06-03T03:29:05.120 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writesame": 0, 2024-06-03T03:29:05.120 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_append": 0, 2024-06-03T03:29:05.120 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_zero": 0, 2024-06-03T03:29:05.120 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_truncate": 0, 2024-06-03T03:29:05.120 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_delete": 0, 2024-06-03T03:29:05.120 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_mapext": 0, 2024-06-03T03:29:05.120 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_sparse_read": 0, 2024-06-03T03:29:05.120 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_clonerange": 0, 2024-06-03T03:29:05.120 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_getxattr": 1, 2024-06-03T03:29:05.120 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_setxattr": 0, 2024-06-03T03:29:05.120 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_cmpxattr": 0, 2024-06-03T03:29:05.120 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_rmxattr": 0, 2024-06-03T03:29:05.120 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_resetxattrs": 0, 2024-06-03T03:29:05.121 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_call": 0, 2024-06-03T03:29:05.121 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_watch": 0, 2024-06-03T03:29:05.121 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_notify": 0, 2024-06-03T03:29:05.121 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_src_cmpxattr": 0, 2024-06-03T03:29:05.121 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls": 0, 2024-06-03T03:29:05.121 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls_filter": 0, 2024-06-03T03:29:05.121 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_other": 0, 2024-06-03T03:29:05.121 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_active": 0, 2024-06-03T03:29:05.121 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_send": 0, 2024-06-03T03:29:05.121 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_resend": 0, 2024-06-03T03:29:05.121 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_ping": 0, 2024-06-03T03:29:05.121 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_active": 0, 2024-06-03T03:29:05.121 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_send": 0, 2024-06-03T03:29:05.121 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_resend": 0, 2024-06-03T03:29:05.122 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_active": 0, 2024-06-03T03:29:05.122 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_send": 0, 2024-06-03T03:29:05.122 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_resend": 0, 2024-06-03T03:29:05.122 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_active": 0, 2024-06-03T03:29:05.122 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_send": 0, 2024-06-03T03:29:05.122 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_resend": 0, 2024-06-03T03:29:05.122 INFO:teuthology.orchestra.run.smithi183.stdout: "command_active": 0, 2024-06-03T03:29:05.122 INFO:teuthology.orchestra.run.smithi183.stdout: "command_send": 0, 2024-06-03T03:29:05.122 INFO:teuthology.orchestra.run.smithi183.stdout: "command_resend": 0, 2024-06-03T03:29:05.122 INFO:teuthology.orchestra.run.smithi183.stdout: "map_epoch": 99, 2024-06-03T03:29:05.122 INFO:teuthology.orchestra.run.smithi183.stdout: "map_full": 0, 2024-06-03T03:29:05.122 INFO:teuthology.orchestra.run.smithi183.stdout: "map_inc": 2, 2024-06-03T03:29:05.122 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_sessions": 5, 2024-06-03T03:29:05.123 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_open": 5, 2024-06-03T03:29:05.123 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_close": 0, 2024-06-03T03:29:05.123 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_laggy": 0, 2024-06-03T03:29:05.123 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_wr": 0, 2024-06-03T03:29:05.123 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_rd": 2, 2024-06-03T03:29:05.123 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_del": 0 2024-06-03T03:29:05.123 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:05.123 INFO:teuthology.orchestra.run.smithi183.stdout: "oft": { 2024-06-03T03:29:05.123 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_objs": 0, 2024-06-03T03:29:05.123 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_kv_pairs": 0, 2024-06-03T03:29:05.123 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_updates": 0, 2024-06-03T03:29:05.123 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_removes": 0 2024-06-03T03:29:05.123 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:05.123 INFO:teuthology.orchestra.run.smithi183.stdout: "osd-slow-ops": { 2024-06-03T03:29:05.124 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_ops_count": 0 2024-06-03T03:29:05.124 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:05.124 INFO:teuthology.orchestra.run.smithi183.stdout: "purge_queue": { 2024-06-03T03:29:05.124 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops": 0, 2024-06-03T03:29:05.124 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops_high_water": 0, 2024-06-03T03:29:05.124 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing": 0, 2024-06-03T03:29:05.124 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_high_water": 0, 2024-06-03T03:29:05.124 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed_ops": 0, 2024-06-03T03:29:05.124 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed": 0, 2024-06-03T03:29:05.124 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_item_in_journal": 0 2024-06-03T03:29:05.124 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:05.124 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-03T03:29:05.124 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-06-03T03:29:05.124 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-06-03T03:29:05.125 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-06-03T03:29:05.125 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 1377, 2024-06-03T03:29:05.125 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 984327, 2024-06-03T03:29:05.125 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-06-03T03:29:05.125 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 1377, 2024-06-03T03:29:05.125 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-06-03T03:29:05.125 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-06-03T03:29:05.125 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 1377, 2024-06-03T03:29:05.125 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 984327, 2024-06-03T03:29:05.125 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-06-03T03:29:05.125 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:29:05.125 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:29:05.125 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:29:05.126 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-06-03T03:29:05.126 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:05.126 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_bytes": { 2024-06-03T03:29:05.126 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-06-03T03:29:05.126 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-06-03T03:29:05.126 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-06-03T03:29:05.126 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-06-03T03:29:05.126 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-06-03T03:29:05.126 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-06-03T03:29:05.126 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-06-03T03:29:05.126 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 1194, 2024-06-03T03:29:05.126 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 1124, 2024-06-03T03:29:05.126 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 1194, 2024-06-03T03:29:05.127 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 1124, 2024-06-03T03:29:05.127 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-06-03T03:29:05.127 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:29:05.127 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:29:05.127 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:29:05.127 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-06-03T03:29:05.127 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:05.127 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_ops": { 2024-06-03T03:29:05.127 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-06-03T03:29:05.127 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 1024, 2024-06-03T03:29:05.127 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-06-03T03:29:05.127 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-06-03T03:29:05.127 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-06-03T03:29:05.127 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-06-03T03:29:05.128 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-06-03T03:29:05.128 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 1194, 2024-06-03T03:29:05.128 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 1194, 2024-06-03T03:29:05.128 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 1194, 2024-06-03T03:29:05.128 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 1194, 2024-06-03T03:29:05.128 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-06-03T03:29:05.128 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:29:05.128 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:29:05.128 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:29:05.128 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-06-03T03:29:05.128 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:05.128 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle": { 2024-06-03T03:29:05.128 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-06-03T03:29:05.128 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-06-03T03:29:05.129 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-06-03T03:29:05.129 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-06-03T03:29:05.129 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-06-03T03:29:05.129 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-06-03T03:29:05.129 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-06-03T03:29:05.129 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-06-03T03:29:05.129 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-06-03T03:29:05.129 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 0, 2024-06-03T03:29:05.129 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 0, 2024-06-03T03:29:05.129 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-06-03T03:29:05.129 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:29:05.129 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:29:05.129 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:29:05.129 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-06-03T03:29:05.129 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-06-03T03:29:05.130 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle-0x55e0be6e46e0": { 2024-06-03T03:29:05.130 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-06-03T03:29:05.130 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-06-03T03:29:05.130 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-06-03T03:29:05.130 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-06-03T03:29:05.130 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-06-03T03:29:05.130 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-06-03T03:29:05.130 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-06-03T03:29:05.130 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-06-03T03:29:05.130 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-06-03T03:29:05.130 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 0, 2024-06-03T03:29:05.130 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 0, 2024-06-03T03:29:05.131 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-06-03T03:29:05.131 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-06-03T03:29:05.131 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-06-03T03:29:05.131 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-06-03T03:29:05.131 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-06-03T03:29:05.131 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-06-03T03:29:05.132 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-06-03T03:29:05.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:29:05 smithi151 ceph-mon[37043]: pgmap v619: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:29:05.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:29:05 smithi117 ceph-mon[31649]: pgmap v619: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:29:05.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:29:05 smithi183 ceph-mon[37326]: pgmap v619: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T03:29:05.863 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-06-03T03:29:05.863 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-06-03T03:29:05.863 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-06-03T03:29:05.875 INFO:tasks.kclient:Unmounting kernel clients... 2024-06-03T03:29:05.875 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-06-03T03:29:05.875 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-03T03:29:05.935 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-06-03T03:29:05.935 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-03T03:29:05.986 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph osd blocklist ls 2024-06-03T03:29:06.490 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:29:07.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:29:07 smithi117 ceph-mon[31649]: pgmap v620: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-03T03:29:07.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:29:07 smithi151 ceph-mon[37043]: pgmap v620: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-03T03:29:07.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:29:07 smithi183 ceph-mon[37326]: pgmap v620: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-03T03:29:09.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:09.092+0000 7f925fe00700 1 -- 172.21.15.117:0/1687226828 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9260109390 msgr2=0x7f9260115830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:29:09.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:09.092+0000 7f925fe00700 1 --2- 172.21.15.117:0/1687226828 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9260109390 0x7f9260115830 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f925800b0a0 tx=0x7f925800b3b0 comp rx=0 tx=0).stop 2024-06-03T03:29:09.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:09.092+0000 7f925fe00700 1 -- 172.21.15.117:0/1687226828 shutdown_connections 2024-06-03T03:29:09.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:09.092+0000 7f925fe00700 1 --2- 172.21.15.117:0/1687226828 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9260109390 0x7f9260115830 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:29:09.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:09.092+0000 7f925fe00700 1 --2- 172.21.15.117:0/1687226828 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f926010d8a0 0x7f926010dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:29:09.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:09.092+0000 7f925fe00700 1 --2- 172.21.15.117:0/1687226828 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f92601089d0 0x7f9260108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:29:09.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:09.092+0000 7f925fe00700 1 -- 172.21.15.117:0/1687226828 >> 172.21.15.117:0/1687226828 conn(0x7f9260078220 msgr2=0x7f92601072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:29:09.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:09.092+0000 7f925fe00700 1 -- 172.21.15.117:0/1687226828 shutdown_connections 2024-06-03T03:29:09.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:09.092+0000 7f925fe00700 1 -- 172.21.15.117:0/1687226828 wait complete. 2024-06-03T03:29:09.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:09.093+0000 7f925fe00700 1 Processor -- start 2024-06-03T03:29:09.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:09.093+0000 7f925fe00700 1 -- start start 2024-06-03T03:29:09.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:09.094+0000 7f925fe00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f92601089d0 0x7f926019f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:29:09.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:09.094+0000 7f925fe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9260109390 0x7f926019f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:29:09.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:09.094+0000 7f925fe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f926010d8a0 0x7f92601a3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:29:09.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:09.094+0000 7f925fe00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9260118d20 con 0x7f9260109390 2024-06-03T03:29:09.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:09.094+0000 7f925fe00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9260118ba0 con 0x7f926010d8a0 2024-06-03T03:29:09.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:09.094+0000 7f925fe00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9260118ea0 con 0x7f92601089d0 2024-06-03T03:29:09.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:09.094+0000 7f9257e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9260109390 0x7f926019f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:29:09.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:09.094+0000 7f9257e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9260109390 0x7f926019f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37330/0 (socket says 172.21.15.117:37330) 2024-06-03T03:29:09.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:09.094+0000 7f9257e00700 1 -- 172.21.15.117:0/877396109 learned_addr learned my addr 172.21.15.117:0/877396109 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:29:09.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:09.094+0000 7f925ea00700 1 --2- 172.21.15.117:0/877396109 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f92601089d0 0x7f926019f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:29:09.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:09.094+0000 7f925f400700 1 --2- 172.21.15.117:0/877396109 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f926010d8a0 0x7f92601a3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:29:09.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:09.094+0000 7f9257e00700 1 -- 172.21.15.117:0/877396109 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f92601089d0 msgr2=0x7f926019f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:29:09.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:09.094+0000 7f9257e00700 1 --2- 172.21.15.117:0/877396109 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f92601089d0 0x7f926019f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:29:09.100 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:09.094+0000 7f9257e00700 1 -- 172.21.15.117:0/877396109 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f926010d8a0 msgr2=0x7f92601a3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:29:09.100 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:09.094+0000 7f9257e00700 1 --2- 172.21.15.117:0/877396109 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f926010d8a0 0x7f92601a3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:29:09.100 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:09.094+0000 7f9257e00700 1 -- 172.21.15.117:0/877396109 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92601a41d0 con 0x7f9260109390 2024-06-03T03:29:09.100 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:09.095+0000 7f9257e00700 1 --2- 172.21.15.117:0/877396109 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9260109390 0x7f926019f750 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f924800d950 tx=0x7f924800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:29:09.100 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:09.095+0000 7f9257400700 1 -- 172.21.15.117:0/877396109 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9248016070 con 0x7f9260109390 2024-06-03T03:29:09.100 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:09.095+0000 7f9257400700 1 -- 172.21.15.117:0/877396109 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f924800fd70 con 0x7f9260109390 2024-06-03T03:29:09.100 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:09.095+0000 7f925fe00700 1 -- 172.21.15.117:0/877396109 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92601a44c0 con 0x7f9260109390 2024-06-03T03:29:09.100 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:09.095+0000 7f9257400700 1 -- 172.21.15.117:0/877396109 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f924800e610 con 0x7f9260109390 2024-06-03T03:29:09.100 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:09.095+0000 7f925fe00700 1 -- 172.21.15.117:0/877396109 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92601abd40 con 0x7f9260109390 2024-06-03T03:29:09.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:09.096+0000 7f925fe00700 1 -- 172.21.15.117:0/877396109 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9260069180 con 0x7f9260109390 2024-06-03T03:29:09.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:09.098+0000 7f9257400700 1 -- 172.21.15.117:0/877396109 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f9248019070 con 0x7f9260109390 2024-06-03T03:29:09.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:09.102+0000 7f9257400700 1 --2- 172.21.15.117:0/877396109 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f92440a2ca0 0x7f92440a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:29:09.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:09.102+0000 7f9257400700 1 -- 172.21.15.117:0/877396109 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f92480cee30 con 0x7f9260109390 2024-06-03T03:29:09.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:09.102+0000 7f925ea00700 1 --2- 172.21.15.117:0/877396109 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f92440a2ca0 0x7f92440a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:29:09.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:09.102+0000 7f925ea00700 1 --2- 172.21.15.117:0/877396109 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f92440a2ca0 0x7f92440a5160 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f9250009be0 tx=0x7f9250009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:29:09.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:09.104+0000 7f9257400700 1 -- 172.21.15.117:0/877396109 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f924809b9a0 con 0x7f9260109390 2024-06-03T03:29:09.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:09.345+0000 7f925fe00700 1 -- 172.21.15.117:0/877396109 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f92601a0520 con 0x7f9260109390 2024-06-03T03:29:09.347 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:29:09 smithi117 ceph-mon[31649]: pgmap v621: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-03T03:29:09.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:09.346+0000 7f9257400700 1 -- 172.21.15.117:0/877396109 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v99) ==== 81+0+601 (secure 0 0 0) 0x7f924809b2a0 con 0x7f9260109390 2024-06-03T03:29:09.348 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:0/186309008 2024-06-04T03:11:24.655516+0000 2024-06-03T03:29:09.348 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:6801/2142313590 2024-06-04T03:11:24.655516+0000 2024-06-03T03:29:09.348 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:6800/2142313590 2024-06-04T03:11:24.655516+0000 2024-06-03T03:29:09.348 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:0/2399725156 2024-06-04T03:11:24.655516+0000 2024-06-03T03:29:09.348 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:0/1453232061 2024-06-04T03:10:50.552899+0000 2024-06-03T03:29:09.348 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:0/4281674974 2024-06-04T03:10:50.552899+0000 2024-06-03T03:29:09.348 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:6800/2424554985 2024-06-04T03:10:50.552899+0000 2024-06-03T03:29:09.348 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:0/3946365225 2024-06-04T03:11:24.655516+0000 2024-06-03T03:29:09.349 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:0/1146162354 2024-06-04T03:10:50.552899+0000 2024-06-03T03:29:09.349 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:6801/2424554985 2024-06-04T03:10:50.552899+0000 2024-06-03T03:29:09.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:09.349+0000 7f925fe00700 1 -- 172.21.15.117:0/877396109 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f92440a2ca0 msgr2=0x7f92440a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:29:09.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:09.349+0000 7f925fe00700 1 --2- 172.21.15.117:0/877396109 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f92440a2ca0 0x7f92440a5160 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f9250009be0 tx=0x7f9250009260 comp rx=0 tx=0).stop 2024-06-03T03:29:09.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:09.349+0000 7f925fe00700 1 -- 172.21.15.117:0/877396109 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9260109390 msgr2=0x7f926019f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:29:09.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:09.349+0000 7f925fe00700 1 --2- 172.21.15.117:0/877396109 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9260109390 0x7f926019f750 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f924800d950 tx=0x7f924800dc60 comp rx=0 tx=0).stop 2024-06-03T03:29:09.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:09.349+0000 7f925fe00700 1 -- 172.21.15.117:0/877396109 shutdown_connections 2024-06-03T03:29:09.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:09.349+0000 7f925fe00700 1 --2- 172.21.15.117:0/877396109 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f92440a2ca0 0x7f92440a5160 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:29:09.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:09.349+0000 7f925fe00700 1 --2- 172.21.15.117:0/877396109 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f926010d8a0 0x7f92601a3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:29:09.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:09.349+0000 7f925fe00700 1 --2- 172.21.15.117:0/877396109 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f92601089d0 0x7f926019f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:29:09.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:09.349+0000 7f925fe00700 1 --2- 172.21.15.117:0/877396109 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9260109390 0x7f926019f750 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:29:09.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:09.349+0000 7f925fe00700 1 -- 172.21.15.117:0/877396109 >> 172.21.15.117:0/877396109 conn(0x7f9260078220 msgr2=0x7f9260078880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:29:09.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:09.349+0000 7f925fe00700 1 -- 172.21.15.117:0/877396109 shutdown_connections 2024-06-03T03:29:09.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:09.349+0000 7f925fe00700 1 -- 172.21.15.117:0/877396109 wait complete. 2024-06-03T03:29:09.352 INFO:teuthology.orchestra.run.smithi117.stderr:listed 10 entries 2024-06-03T03:29:09.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:29:09 smithi151 ceph-mon[37043]: pgmap v621: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-03T03:29:09.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:29:09 smithi183 ceph-mon[37326]: pgmap v621: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-03T03:29:10.348 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-06-03T03:29:10.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T03:29:10.349 DEBUG:teuthology.orchestra.run.smithi117:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-06-03T03:29:10.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:29:10 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/877396109' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-03T03:29:10.426 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:29:10 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/877396109' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-03T03:29:10.478 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi117.front.sepia.ceph.com 2024-06-03T03:29:10.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T03:29:10.478 DEBUG:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-06-03T03:29:10.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:29:10 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/877396109' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-03T03:29:10.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T03:29:10.528 DEBUG:teuthology.orchestra.run.smithi117:> ip netns list 2024-06-03T03:29:10.608 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-06-03T03:29:10.609 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T03:29:10.609 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-06-03T03:29:10.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T03:29:10.726 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip link delete ceph-brx 2024-06-03T03:29:10.842 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-06-03T03:29:10.859 INFO:tasks.cephadm:Teardown begin 2024-06-03T03:29:10.859 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-06-03T03:29:10.859 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-03T03:29:11.278 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:29:11.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:29:11 smithi151 ceph-mon[37043]: pgmap v622: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-06-03T03:29:11.426 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:29:11 smithi117 ceph-mon[31649]: pgmap v622: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-06-03T03:29:11.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:29:11 smithi183 ceph-mon[37326]: pgmap v622: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-06-03T03:29:13.365 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:29:13 smithi117 ceph-mon[31649]: pgmap v623: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 11 op/s 2024-06-03T03:29:13.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:29:13 smithi151 ceph-mon[37043]: pgmap v623: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 11 op/s 2024-06-03T03:29:13.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:29:13 smithi183 ceph-mon[37326]: pgmap v623: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 11 op/s 2024-06-03T03:29:13.930 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:13.929+0000 7f0aeaa00700 1 -- 172.21.15.117:0/550470684 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0ae410a6a0 msgr2=0x7f0ae410aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:29:13.930 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:13.929+0000 7f0aeaa00700 1 --2- 172.21.15.117:0/550470684 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0ae410a6a0 0x7f0ae410aa80 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0ad40098f0 tx=0x7f0ad4009c00 comp rx=0 tx=0).stop 2024-06-03T03:29:13.930 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:13.929+0000 7f0aeaa00700 1 -- 172.21.15.117:0/550470684 shutdown_connections 2024-06-03T03:29:13.930 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:13.929+0000 7f0aeaa00700 1 --2- 172.21.15.117:0/550470684 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ae4105060 0x7f0ae4111570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:29:13.930 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:13.929+0000 7f0aeaa00700 1 --2- 172.21.15.117:0/550470684 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0ae410a6a0 0x7f0ae410aa80 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:29:13.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:13.929+0000 7f0aeaa00700 1 --2- 172.21.15.117:0/550470684 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ae41046a0 0x7f0ae4104b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:29:13.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:13.929+0000 7f0aeaa00700 1 -- 172.21.15.117:0/550470684 >> 172.21.15.117:0/550470684 conn(0x7f0ae41001d0 msgr2=0x7f0ae41025f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:29:13.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:13.929+0000 7f0aeaa00700 1 -- 172.21.15.117:0/550470684 shutdown_connections 2024-06-03T03:29:13.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:13.929+0000 7f0aeaa00700 1 -- 172.21.15.117:0/550470684 wait complete. 2024-06-03T03:29:13.932 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:13.930+0000 7f0aeaa00700 1 Processor -- start 2024-06-03T03:29:13.932 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:13.930+0000 7f0aeaa00700 1 -- start start 2024-06-03T03:29:13.932 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:13.931+0000 7f0aeaa00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ae41046a0 0x7f0ae419aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:29:13.932 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:13.931+0000 7f0aeaa00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ae4105060 0x7f0ae419b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:29:13.932 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:13.931+0000 7f0aeaa00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0ae410a6a0 0x7f0ae419f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:29:13.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:13.931+0000 7f0aeaa00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae4114a30 con 0x7f0ae41046a0 2024-06-03T03:29:13.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:13.931+0000 7f0aeaa00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae41148b0 con 0x7f0ae4105060 2024-06-03T03:29:13.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:13.931+0000 7f0aeaa00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae4114bb0 con 0x7f0ae410a6a0 2024-06-03T03:29:13.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:13.931+0000 7f0ae3e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ae41046a0 0x7f0ae419aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:29:13.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:13.931+0000 7f0ae8a00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0ae410a6a0 0x7f0ae419f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:29:13.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:13.931+0000 7f0ae3e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ae41046a0 0x7f0ae419aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37342/0 (socket says 172.21.15.117:37342) 2024-06-03T03:29:13.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:13.931+0000 7f0ae8a00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0ae410a6a0 0x7f0ae419f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:51738/0 (socket says 172.21.15.117:51738) 2024-06-03T03:29:13.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:13.931+0000 7f0ae3e00700 1 -- 172.21.15.117:0/2118894426 learned_addr learned my addr 172.21.15.117:0/2118894426 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:29:13.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:13.931+0000 7f0ae3400700 1 --2- 172.21.15.117:0/2118894426 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ae4105060 0x7f0ae419b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:29:13.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:13.932+0000 7f0ae8a00700 1 -- 172.21.15.117:0/2118894426 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ae4105060 msgr2=0x7f0ae419b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:29:13.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:13.932+0000 7f0ae8a00700 1 --2- 172.21.15.117:0/2118894426 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ae4105060 0x7f0ae419b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:29:13.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:13.932+0000 7f0ae8a00700 1 -- 172.21.15.117:0/2118894426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ae41046a0 msgr2=0x7f0ae419aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:29:13.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:13.932+0000 7f0ae8a00700 1 --2- 172.21.15.117:0/2118894426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ae41046a0 0x7f0ae419aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:29:13.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:13.932+0000 7f0ae8a00700 1 -- 172.21.15.117:0/2118894426 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ae419feb0 con 0x7f0ae410a6a0 2024-06-03T03:29:13.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:13.932+0000 7f0ae8a00700 1 --2- 172.21.15.117:0/2118894426 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0ae410a6a0 0x7f0ae419f770 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0adc009cd0 tx=0x7f0adc00c620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:29:13.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:13.932+0000 7f0ae0c00700 1 -- 172.21.15.117:0/2118894426 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0adc00cdc0 con 0x7f0ae410a6a0 2024-06-03T03:29:13.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:13.932+0000 7f0ae0c00700 1 -- 172.21.15.117:0/2118894426 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0adc003a00 con 0x7f0ae410a6a0 2024-06-03T03:29:13.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:13.932+0000 7f0aeaa00700 1 -- 172.21.15.117:0/2118894426 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ae41a0140 con 0x7f0ae410a6a0 2024-06-03T03:29:13.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:13.932+0000 7f0ae0c00700 1 -- 172.21.15.117:0/2118894426 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0adc004650 con 0x7f0ae410a6a0 2024-06-03T03:29:13.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:13.932+0000 7f0aeaa00700 1 -- 172.21.15.117:0/2118894426 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ae41a7a20 con 0x7f0ae410a6a0 2024-06-03T03:29:13.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:13.933+0000 7f0aeaa00700 1 -- 172.21.15.117:0/2118894426 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ae4069180 con 0x7f0ae410a6a0 2024-06-03T03:29:13.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:13.936+0000 7f0ae0c00700 1 -- 172.21.15.117:0/2118894426 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f0adc01a020 con 0x7f0ae410a6a0 2024-06-03T03:29:13.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:13.937+0000 7f0ae0c00700 1 --2- 172.21.15.117:0/2118894426 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f0ac80a2ca0 0x7f0ac80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:29:13.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:13.937+0000 7f0ae0c00700 1 -- 172.21.15.117:0/2118894426 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f0adc0cec60 con 0x7f0ae410a6a0 2024-06-03T03:29:13.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:13.937+0000 7f0ae3e00700 1 --2- 172.21.15.117:0/2118894426 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f0ac80a2ca0 0x7f0ac80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:29:13.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:13.938+0000 7f0ae3e00700 1 --2- 172.21.15.117:0/2118894426 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f0ac80a2ca0 0x7f0ac80a5160 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0ad400c010 tx=0x7f0ad4019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:29:13.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:13.941+0000 7f0ae0c00700 1 -- 172.21.15.117:0/2118894426 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0adc010040 con 0x7f0ae410a6a0 2024-06-03T03:29:14.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:14.253+0000 7f0aeaa00700 1 -- 172.21.15.117:0/2118894426 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0ae402dca0 con 0x7f0ae410a6a0 2024-06-03T03:29:14.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:14.254+0000 7f0ae0c00700 1 -- 172.21.15.117:0/2118894426 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11955 (secure 0 0 0) 0x7f0adc09b7d0 con 0x7f0ae410a6a0 2024-06-03T03:29:14.255 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T03:29:14.255 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3353001119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3353001119},{"type":"v1","addr":"172.21.15.117:6837","nonce":3353001119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14388,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6839/405782304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":405782304},{"type":"v1","addr":"172.21.15.117:6839","nonce":405782304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-03T03:19:44.629822+0000","modified":"2024-06-03T03:25:11.576498+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24517,"mds_1":24512,"mds_2":24503,"mds_3":24499,"mds_4":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24431":{"gid":24431,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6833/3355662416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3355662416},{"type":"v1","addr":"172.21.15.151:6833","nonce":3355662416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/2429725085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2429725085},{"type":"v1","addr":"172.21.15.183:6833","nonce":2429725085}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/31153579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":31153579},{"type":"v1","addr":"172.21.15.151:6835","nonce":31153579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"j","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/2884460088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2884460088},{"type":"v1","addr":"172.21.15.117:6835","nonce":2884460088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/2392407848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":2392407848},{"type":"v1","addr":"172.21.15.151:6837","nonce":2392407848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":32,"addr":"172.21.15.183:6835/1009004320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1009004320},{"type":"v1","addr":"172.21.15.183:6835","nonce":1009004320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.183:6837/2697518526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":2697518526},{"type":"v1","addr":"172.21.15.183:6837","nonce":2697518526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"a","rank":2,"incarnation":22,"state":"up:active","state_seq":27,"addr":"172.21.15.117:6841/1212242967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6840","nonce":1212242967},{"type":"v1","addr":"172.21.15.117:6841","nonce":1212242967}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.151:6839/3623444616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":3623444616},{"type":"v1","addr":"172.21.15.151:6839","nonce":3623444616}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":10,"addr":"172.21.15.183:6839/1439615083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1439615083},{"type":"v1","addr":"172.21.15.183:6839","nonce":1439615083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24517,"qdb_cluster":[24499,24487,24512,24503,24517]},"id":1}]} 2024-06-03T03:29:14.259 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:14.258+0000 7f0aeaa00700 1 -- 172.21.15.117:0/2118894426 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f0ac80a2ca0 msgr2=0x7f0ac80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:29:14.259 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:14.258+0000 7f0aeaa00700 1 --2- 172.21.15.117:0/2118894426 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f0ac80a2ca0 0x7f0ac80a5160 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0ad400c010 tx=0x7f0ad4019040 comp rx=0 tx=0).stop 2024-06-03T03:29:14.259 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:14.258+0000 7f0aeaa00700 1 -- 172.21.15.117:0/2118894426 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0ae410a6a0 msgr2=0x7f0ae419f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:29:14.259 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:14.258+0000 7f0aeaa00700 1 --2- 172.21.15.117:0/2118894426 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0ae410a6a0 0x7f0ae419f770 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0adc009cd0 tx=0x7f0adc00c620 comp rx=0 tx=0).stop 2024-06-03T03:29:14.259 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:14.258+0000 7f0aeaa00700 1 -- 172.21.15.117:0/2118894426 shutdown_connections 2024-06-03T03:29:14.259 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:14.258+0000 7f0aeaa00700 1 --2- 172.21.15.117:0/2118894426 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f0ac80a2ca0 0x7f0ac80a5160 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:29:14.259 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:14.258+0000 7f0aeaa00700 1 --2- 172.21.15.117:0/2118894426 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0ae410a6a0 0x7f0ae419f770 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:29:14.259 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:14.258+0000 7f0aeaa00700 1 --2- 172.21.15.117:0/2118894426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ae41046a0 0x7f0ae419aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:29:14.259 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:14.258+0000 7f0aeaa00700 1 --2- 172.21.15.117:0/2118894426 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ae4105060 0x7f0ae419b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:29:14.259 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:14.258+0000 7f0aeaa00700 1 -- 172.21.15.117:0/2118894426 >> 172.21.15.117:0/2118894426 conn(0x7f0ae41001d0 msgr2=0x7f0ae4100950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:29:14.259 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:14.258+0000 7f0aeaa00700 1 -- 172.21.15.117:0/2118894426 shutdown_connections 2024-06-03T03:29:14.260 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:14.258+0000 7f0aeaa00700 1 -- 172.21.15.117:0/2118894426 wait complete. 2024-06-03T03:29:14.261 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 31 2024-06-03T03:29:14.656 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:29:14 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/2118894426' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T03:29:14.696 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:29:14 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/2118894426' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T03:29:14.731 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:29:14 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/2118894426' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T03:29:15.268 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-03T03:29:15.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:29:15 smithi117 ceph-mon[31649]: pgmap v624: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-06-03T03:29:15.656 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:29:15 smithi151 ceph-mon[37043]: pgmap v624: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-06-03T03:29:15.699 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:29:15.731 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:29:15 smithi183 ceph-mon[37326]: pgmap v624: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-06-03T03:29:17.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:29:17 smithi151 ceph-mon[37043]: pgmap v625: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2024-06-03T03:29:17.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:29:17 smithi117 ceph-mon[31649]: pgmap v625: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2024-06-03T03:29:17.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:29:17 smithi183 ceph-mon[37326]: pgmap v625: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2024-06-03T03:29:18.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:18.225+0000 7f73aa400700 1 -- 172.21.15.117:0/2286528845 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73a40772e0 msgr2=0x7f73a4112d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:29:18.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:18.225+0000 7f73aa400700 1 --2- 172.21.15.117:0/2286528845 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73a40772e0 0x7f73a4112d00 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7390009990 tx=0x7f7390009ca0 comp rx=0 tx=0).stop 2024-06-03T03:29:18.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:18.225+0000 7f73aa400700 1 -- 172.21.15.117:0/2286528845 shutdown_connections 2024-06-03T03:29:18.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:18.225+0000 7f73aa400700 1 --2- 172.21.15.117:0/2286528845 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73a4113240 0x7f73a4115630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:29:18.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:18.225+0000 7f73aa400700 1 --2- 172.21.15.117:0/2286528845 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73a40769c0 0x7f73a4076da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:29:18.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:18.225+0000 7f73aa400700 1 --2- 172.21.15.117:0/2286528845 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73a40772e0 0x7f73a4112d00 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:29:18.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:18.225+0000 7f73aa400700 1 -- 172.21.15.117:0/2286528845 >> 172.21.15.117:0/2286528845 conn(0x7f73a41000d0 msgr2=0x7f73a41024f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:29:18.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:18.225+0000 7f73aa400700 1 -- 172.21.15.117:0/2286528845 shutdown_connections 2024-06-03T03:29:18.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:18.225+0000 7f73aa400700 1 -- 172.21.15.117:0/2286528845 wait complete. 2024-06-03T03:29:18.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:18.226+0000 7f73aa400700 1 Processor -- start 2024-06-03T03:29:18.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:18.227+0000 7f73aa400700 1 -- start start 2024-06-03T03:29:18.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:18.227+0000 7f73aa400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73a40769c0 0x7f73a419f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:29:18.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:18.227+0000 7f73aa400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73a40772e0 0x7f73a419f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:29:18.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:18.227+0000 7f73aa400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73a4113240 0x7f73a41a3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:29:18.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:18.227+0000 7f73aa400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f73a4118d20 con 0x7f73a40769c0 2024-06-03T03:29:18.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:18.227+0000 7f73aa400700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f73a4118ba0 con 0x7f73a40772e0 2024-06-03T03:29:18.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:18.227+0000 7f73aa400700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f73a4118ea0 con 0x7f73a4113240 2024-06-03T03:29:18.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:18.227+0000 7f73a9000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73a40769c0 0x7f73a419f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:29:18.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:18.228+0000 7f73a9a00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73a4113240 0x7f73a41a3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:29:18.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:18.228+0000 7f73a3e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73a40772e0 0x7f73a419f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:29:18.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:18.228+0000 7f73a9000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73a40769c0 0x7f73a419f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:59660/0 (socket says 172.21.15.117:59660) 2024-06-03T03:29:18.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:18.228+0000 7f73a9000700 1 -- 172.21.15.117:0/2195543297 learned_addr learned my addr 172.21.15.117:0/2195543297 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:29:18.230 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:18.228+0000 7f73a9a00700 1 -- 172.21.15.117:0/2195543297 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73a40772e0 msgr2=0x7f73a419f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:29:18.230 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:18.228+0000 7f73a9a00700 1 --2- 172.21.15.117:0/2195543297 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73a40772e0 0x7f73a419f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:29:18.230 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:18.228+0000 7f73a9a00700 1 -- 172.21.15.117:0/2195543297 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73a40769c0 msgr2=0x7f73a419f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:29:18.230 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:18.228+0000 7f73a9a00700 1 --2- 172.21.15.117:0/2195543297 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73a40769c0 0x7f73a419f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:29:18.230 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:18.228+0000 7f73a9a00700 1 -- 172.21.15.117:0/2195543297 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73a41a4240 con 0x7f73a4113240 2024-06-03T03:29:18.230 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:18.228+0000 7f73a3e00700 1 --2- 172.21.15.117:0/2195543297 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73a40772e0 0x7f73a419f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:29:18.230 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:18.228+0000 7f73a9a00700 1 --2- 172.21.15.117:0/2195543297 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73a4113240 0x7f73a41a3b00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f73a407acb0 tx=0x7f739c00ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:29:18.230 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:18.229+0000 7f73a1600700 1 -- 172.21.15.117:0/2195543297 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f739c0092e0 con 0x7f73a4113240 2024-06-03T03:29:18.230 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:18.229+0000 7f73a1600700 1 -- 172.21.15.117:0/2195543297 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f739c004bb0 con 0x7f73a4113240 2024-06-03T03:29:18.231 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:18.229+0000 7f73aa400700 1 -- 172.21.15.117:0/2195543297 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73a41a44d0 con 0x7f73a4113240 2024-06-03T03:29:18.231 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:18.229+0000 7f73a1600700 1 -- 172.21.15.117:0/2195543297 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f739c004280 con 0x7f73a4113240 2024-06-03T03:29:18.231 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:18.229+0000 7f73aa400700 1 -- 172.21.15.117:0/2195543297 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73a41abdb0 con 0x7f73a4113240 2024-06-03T03:29:18.231 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:18.230+0000 7f73aa400700 1 -- 172.21.15.117:0/2195543297 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73a4069180 con 0x7f73a4113240 2024-06-03T03:29:18.232 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:18.232+0000 7f73a1600700 1 -- 172.21.15.117:0/2195543297 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f739c01b030 con 0x7f73a4113240 2024-06-03T03:29:18.234 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:18.233+0000 7f73a1600700 1 --2- 172.21.15.117:0/2195543297 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f738c0a2c20 0x7f738c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:29:18.234 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:18.233+0000 7f73a1600700 1 -- 172.21.15.117:0/2195543297 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f739c0cf870 con 0x7f73a4113240 2024-06-03T03:29:18.234 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:18.234+0000 7f73a9000700 1 --2- 172.21.15.117:0/2195543297 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f738c0a2c20 0x7f738c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:29:18.234 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:18.234+0000 7f73a9000700 1 --2- 172.21.15.117:0/2195543297 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f738c0a2c20 0x7f738c0a50e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7398009d20 tx=0x7f7398009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:29:18.236 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:18.236+0000 7f73a1600700 1 -- 172.21.15.117:0/2195543297 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f739c022020 con 0x7f73a4113240 2024-06-03T03:29:18.506 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:18.506+0000 7f73aa400700 1 -- 172.21.15.117:0/2195543297 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f73a402dca0 con 0x7f73a4113240 2024-06-03T03:29:18.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:18.507+0000 7f73a1600700 1 -- 172.21.15.117:0/2195543297 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11955 (secure 0 0 0) 0x7f739c010040 con 0x7f73a4113240 2024-06-03T03:29:18.508 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T03:29:18.508 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14367,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3353001119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3353001119},{"type":"v1","addr":"172.21.15.117:6837","nonce":3353001119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14388,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6839/405782304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":405782304},{"type":"v1","addr":"172.21.15.117:6839","nonce":405782304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-03T03:19:44.629822+0000","modified":"2024-06-03T03:25:11.576498+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24517,"mds_1":24512,"mds_2":24503,"mds_3":24499,"mds_4":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24431":{"gid":24431,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6833/3355662416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3355662416},{"type":"v1","addr":"172.21.15.151:6833","nonce":3355662416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/2429725085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2429725085},{"type":"v1","addr":"172.21.15.183:6833","nonce":2429725085}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/31153579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":31153579},{"type":"v1","addr":"172.21.15.151:6835","nonce":31153579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"j","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/2884460088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2884460088},{"type":"v1","addr":"172.21.15.117:6835","nonce":2884460088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/2392407848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":2392407848},{"type":"v1","addr":"172.21.15.151:6837","nonce":2392407848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":32,"addr":"172.21.15.183:6835/1009004320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1009004320},{"type":"v1","addr":"172.21.15.183:6835","nonce":1009004320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.183:6837/2697518526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":2697518526},{"type":"v1","addr":"172.21.15.183:6837","nonce":2697518526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"a","rank":2,"incarnation":22,"state":"up:active","state_seq":27,"addr":"172.21.15.117:6841/1212242967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6840","nonce":1212242967},{"type":"v1","addr":"172.21.15.117:6841","nonce":1212242967}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.151:6839/3623444616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":3623444616},{"type":"v1","addr":"172.21.15.151:6839","nonce":3623444616}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":10,"addr":"172.21.15.183:6839/1439615083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1439615083},{"type":"v1","addr":"172.21.15.183:6839","nonce":1439615083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24517,"qdb_cluster":[24499,24487,24512,24503,24517]},"id":1}]} 2024-06-03T03:29:18.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:18.510+0000 7f73aa400700 1 -- 172.21.15.117:0/2195543297 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f738c0a2c20 msgr2=0x7f738c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:29:18.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:18.510+0000 7f73aa400700 1 --2- 172.21.15.117:0/2195543297 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f738c0a2c20 0x7f738c0a50e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7398009d20 tx=0x7f7398009260 comp rx=0 tx=0).stop 2024-06-03T03:29:18.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:18.510+0000 7f73aa400700 1 -- 172.21.15.117:0/2195543297 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73a4113240 msgr2=0x7f73a41a3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:29:18.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:18.510+0000 7f73aa400700 1 --2- 172.21.15.117:0/2195543297 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73a4113240 0x7f73a41a3b00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f73a407acb0 tx=0x7f739c00ef90 comp rx=0 tx=0).stop 2024-06-03T03:29:18.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:18.510+0000 7f73aa400700 1 -- 172.21.15.117:0/2195543297 shutdown_connections 2024-06-03T03:29:18.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:18.510+0000 7f73aa400700 1 --2- 172.21.15.117:0/2195543297 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f738c0a2c20 0x7f738c0a50e0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:29:18.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:18.510+0000 7f73aa400700 1 --2- 172.21.15.117:0/2195543297 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73a4113240 0x7f73a41a3b00 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:29:18.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:18.510+0000 7f73aa400700 1 --2- 172.21.15.117:0/2195543297 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73a40769c0 0x7f73a419f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:29:18.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:18.510+0000 7f73aa400700 1 --2- 172.21.15.117:0/2195543297 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73a40772e0 0x7f73a419f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:29:18.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:18.510+0000 7f73aa400700 1 -- 172.21.15.117:0/2195543297 >> 172.21.15.117:0/2195543297 conn(0x7f73a41000d0 msgr2=0x7f73a41024c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:29:18.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:18.510+0000 7f73aa400700 1 -- 172.21.15.117:0/2195543297 shutdown_connections 2024-06-03T03:29:18.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:18.510+0000 7f73aa400700 1 -- 172.21.15.117:0/2195543297 wait complete. 2024-06-03T03:29:18.512 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 31 2024-06-03T03:29:18.906 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:29:18 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/2195543297' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T03:29:18.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:29:18 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/2195543297' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T03:29:18.981 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:29:18 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/2195543297' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T03:29:19.568 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-03T03:29:19.906 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:29:19 smithi151 ceph-mon[37043]: pgmap v626: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 12 op/s 2024-06-03T03:29:19.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:29:19 smithi117 ceph-mon[31649]: pgmap v626: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 12 op/s 2024-06-03T03:29:19.981 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:29:19 smithi183 ceph-mon[37326]: pgmap v626: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 12 op/s 2024-06-03T03:29:19.999 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:29:21.181 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:29:21 smithi117 ceph-mon[31649]: pgmap v627: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-06-03T03:29:21.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:29:21 smithi151 ceph-mon[37043]: pgmap v627: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-06-03T03:29:21.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:29:21 smithi183 ceph-mon[37326]: pgmap v627: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-06-03T03:29:22.477 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:22.476+0000 7f1ec5400700 1 -- 172.21.15.117:0/1274754636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ec006b3c0 msgr2=0x7f1ec010f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:29:22.477 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:22.476+0000 7f1ec5400700 1 --2- 172.21.15.117:0/1274754636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ec006b3c0 0x7f1ec010f310 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f1eb800b0a0 tx=0x7f1eb800b3b0 comp rx=0 tx=0).stop 2024-06-03T03:29:22.477 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:22.476+0000 7f1ec5400700 1 -- 172.21.15.117:0/1274754636 shutdown_connections 2024-06-03T03:29:22.477 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:22.476+0000 7f1ec5400700 1 --2- 172.21.15.117:0/1274754636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ec006b3c0 0x7f1ec010f310 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:29:22.478 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:22.476+0000 7f1ec5400700 1 --2- 172.21.15.117:0/1274754636 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ec01084f0 0x7f1ec01088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:29:22.478 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:22.476+0000 7f1ec5400700 1 --2- 172.21.15.117:0/1274754636 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1ec006aa00 0x7f1ec006ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:29:22.478 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:22.476+0000 7f1ec5400700 1 -- 172.21.15.117:0/1274754636 >> 172.21.15.117:0/1274754636 conn(0x7f1ec0078300 msgr2=0x7f1ec0078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:29:22.478 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:22.476+0000 7f1ec5400700 1 -- 172.21.15.117:0/1274754636 shutdown_connections 2024-06-03T03:29:22.478 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:22.476+0000 7f1ec5400700 1 -- 172.21.15.117:0/1274754636 wait complete. 2024-06-03T03:29:22.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:22.477+0000 7f1ec5400700 1 Processor -- start 2024-06-03T03:29:22.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:22.478+0000 7f1ec5400700 1 -- start start 2024-06-03T03:29:22.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:22.478+0000 7f1ec5400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1ec006aa00 0x7f1ec019e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:29:22.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:22.478+0000 7f1ec5400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ec006b3c0 0x7f1ec019ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:29:22.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:22.478+0000 7f1ec5400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ec01084f0 0x7f1ec0198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:29:22.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:22.478+0000 7f1ec5400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec01127a0 con 0x7f1ec006b3c0 2024-06-03T03:29:22.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:22.478+0000 7f1ec5400700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec0112620 con 0x7f1ec006aa00 2024-06-03T03:29:22.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:22.478+0000 7f1ec5400700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec0112920 con 0x7f1ec01084f0 2024-06-03T03:29:22.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:22.478+0000 7f1ebde00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ec006b3c0 0x7f1ec019ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:29:22.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:22.478+0000 7f1ebe800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1ec006aa00 0x7f1ec019e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:29:22.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:22.478+0000 7f1ebf200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ec01084f0 0x7f1ec0198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:29:22.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:22.478+0000 7f1ebde00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ec006b3c0 0x7f1ec019ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:59678/0 (socket says 172.21.15.117:59678) 2024-06-03T03:29:22.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:22.478+0000 7f1ebe800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1ec006aa00 0x7f1ec019e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.117:57790/0 (socket says 172.21.15.117:57790) 2024-06-03T03:29:22.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:22.478+0000 7f1ebde00700 1 -- 172.21.15.117:0/3161961333 learned_addr learned my addr 172.21.15.117:0/3161961333 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:29:22.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:22.479+0000 7f1ebf200700 1 -- 172.21.15.117:0/3161961333 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1ec006aa00 msgr2=0x7f1ec019e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:29:22.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:22.479+0000 7f1ebf200700 1 --2- 172.21.15.117:0/3161961333 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1ec006aa00 0x7f1ec019e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:29:22.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:22.479+0000 7f1ebf200700 1 -- 172.21.15.117:0/3161961333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ec006b3c0 msgr2=0x7f1ec019ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:29:22.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:22.479+0000 7f1ebf200700 1 --2- 172.21.15.117:0/3161961333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ec006b3c0 0x7f1ec019ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:29:22.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:22.479+0000 7f1ebf200700 1 -- 172.21.15.117:0/3161961333 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ec0199160 con 0x7f1ec01084f0 2024-06-03T03:29:22.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:22.479+0000 7f1ebf200700 1 --2- 172.21.15.117:0/3161961333 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ec01084f0 0x7f1ec0198a00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1eb800b070 tx=0x7f1eb8004c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:29:22.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:22.479+0000 7f1eb7400700 1 -- 172.21.15.117:0/3161961333 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb8021070 con 0x7f1ec01084f0 2024-06-03T03:29:22.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:22.480+0000 7f1eb7400700 1 -- 172.21.15.117:0/3161961333 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1eb8007b80 con 0x7f1ec01084f0 2024-06-03T03:29:22.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:22.480+0000 7f1ec5400700 1 -- 172.21.15.117:0/3161961333 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ec01993f0 con 0x7f1ec01084f0 2024-06-03T03:29:22.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:22.480+0000 7f1eb7400700 1 -- 172.21.15.117:0/3161961333 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb80052c0 con 0x7f1ec01084f0 2024-06-03T03:29:22.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:22.480+0000 7f1ec5400700 1 -- 172.21.15.117:0/3161961333 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ec01a5710 con 0x7f1ec01084f0 2024-06-03T03:29:22.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:22.481+0000 7f1ec5400700 1 -- 172.21.15.117:0/3161961333 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ec0069180 con 0x7f1ec01084f0 2024-06-03T03:29:22.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:22.483+0000 7f1eb7400700 1 -- 172.21.15.117:0/3161961333 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f1eb8024070 con 0x7f1ec01084f0 2024-06-03T03:29:22.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:22.484+0000 7f1eb7400700 1 --2- 172.21.15.117:0/3161961333 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f1ea40a2ca0 0x7f1ea40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:29:22.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:22.484+0000 7f1eb7400700 1 -- 172.21.15.117:0/3161961333 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7f1eb800f020 con 0x7f1ec01084f0 2024-06-03T03:29:22.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:22.485+0000 7f1ebe800700 1 --2- 172.21.15.117:0/3161961333 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f1ea40a2ca0 0x7f1ea40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:29:22.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:22.485+0000 7f1ebe800700 1 --2- 172.21.15.117:0/3161961333 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f1ea40a2ca0 0x7f1ea40a5160 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1ec0199b40 tx=0x7f1eb00073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:29:22.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:22.487+0000 7f1eb7400700 1 -- 172.21.15.117:0/3161961333 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1eb802b020 con 0x7f1ec01084f0 2024-06-03T03:29:22.712 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:22.712+0000 7f1ec5400700 1 -- 172.21.15.117:0/3161961333 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1ec002dca0 con 0x7f1ec01084f0 2024-06-03T03:29:22.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:22.714+0000 7f1eb7400700 1 -- 172.21.15.117:0/3161961333 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+24807 (secure 0 0 0) 0x7f1eb809d7a0 con 0x7f1ec01084f0 2024-06-03T03:29:22.716 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-06-03T03:29:22.716 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":99,"fsid":"a0e8473a-2156-11ef-bc9b-c7b262605968","created":"2024-06-03T03:10:08.446859+0000","modified":"2024-06-03T03:21:25.966405+0000","last_up_change":"2024-06-03T03:18:28.736904+0000","last_in_change":"2024-06-03T03:18:08.485711+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-03T03:14:09.370986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-03T03:19:35.044683+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-03T03:19:40.097569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-03T03:19:55.394480+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8fefa089-9401-4a1c-b9fb-fe30c63e6a1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":4286644492},{"type":"v1","addr":"172.21.15.117:6803","nonce":4286644492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":4286644492},{"type":"v1","addr":"172.21.15.117:6805","nonce":4286644492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":4286644492},{"type":"v1","addr":"172.21.15.117:6809","nonce":4286644492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":4286644492},{"type":"v1","addr":"172.21.15.117:6807","nonce":4286644492}]},"public_addr":"172.21.15.117:6803/4286644492","cluster_addr":"172.21.15.117:6805/4286644492","heartbeat_back_addr":"172.21.15.117:6809/4286644492","heartbeat_front_addr":"172.21.15.117:6807/4286644492","state":["exists","up"]},{"osd":1,"uuid":"5b12b371-f155-4b67-a470-25cbe59a2999","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":262491158},{"type":"v1","addr":"172.21.15.151:6801","nonce":262491158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":262491158},{"type":"v1","addr":"172.21.15.151:6803","nonce":262491158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":262491158},{"type":"v1","addr":"172.21.15.151:6807","nonce":262491158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":262491158},{"type":"v1","addr":"172.21.15.151:6805","nonce":262491158}]},"public_addr":"172.21.15.151:6801/262491158","cluster_addr":"172.21.15.151:6803/262491158","heartbeat_back_addr":"172.21.15.151:6807/262491158","heartbeat_front_addr":"172.21.15.151:6805/262491158","state":["exists","up"]},{"osd":2,"uuid":"dbc17818-c54c-4ecf-b656-03578b95d8c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":3542453623},{"type":"v1","addr":"172.21.15.183:6801","nonce":3542453623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":3542453623},{"type":"v1","addr":"172.21.15.183:6803","nonce":3542453623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":3542453623},{"type":"v1","addr":"172.21.15.183:6807","nonce":3542453623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":3542453623},{"type":"v1","addr":"172.21.15.183:6805","nonce":3542453623}]},"public_addr":"172.21.15.183:6801/3542453623","cluster_addr":"172.21.15.183:6803/3542453623","heartbeat_back_addr":"172.21.15.183:6807/3542453623","heartbeat_front_addr":"172.21.15.183:6805/3542453623","state":["exists","up"]},{"osd":3,"uuid":"ca08d2ac-b1d6-4a49-9ae8-e3ec146899a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":589012315},{"type":"v1","addr":"172.21.15.117:6811","nonce":589012315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":589012315},{"type":"v1","addr":"172.21.15.117:6813","nonce":589012315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":589012315},{"type":"v1","addr":"172.21.15.117:6817","nonce":589012315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":589012315},{"type":"v1","addr":"172.21.15.117:6815","nonce":589012315}]},"public_addr":"172.21.15.117:6811/589012315","cluster_addr":"172.21.15.117:6813/589012315","heartbeat_back_addr":"172.21.15.117:6817/589012315","heartbeat_front_addr":"172.21.15.117:6815/589012315","state":["exists","up"]},{"osd":4,"uuid":"810d6c7a-023f-4930-82af-bc2bbfd647da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":1239048647},{"type":"v1","addr":"172.21.15.151:6809","nonce":1239048647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":1239048647},{"type":"v1","addr":"172.21.15.151:6811","nonce":1239048647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":1239048647},{"type":"v1","addr":"172.21.15.151:6815","nonce":1239048647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":1239048647},{"type":"v1","addr":"172.21.15.151:6813","nonce":1239048647}]},"public_addr":"172.21.15.151:6809/1239048647","cluster_addr":"172.21.15.151:6811/1239048647","heartbeat_back_addr":"172.21.15.151:6815/1239048647","heartbeat_front_addr":"172.21.15.151:6813/1239048647","state":["exists","up"]},{"osd":5,"uuid":"8ab6a5f0-34bb-48d8-9471-a7dc836e486e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":2985874686},{"type":"v1","addr":"172.21.15.183:6809","nonce":2985874686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":2985874686},{"type":"v1","addr":"172.21.15.183:6811","nonce":2985874686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":2985874686},{"type":"v1","addr":"172.21.15.183:6815","nonce":2985874686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":2985874686},{"type":"v1","addr":"172.21.15.183:6813","nonce":2985874686}]},"public_addr":"172.21.15.183:6809/2985874686","cluster_addr":"172.21.15.183:6811/2985874686","heartbeat_back_addr":"172.21.15.183:6815/2985874686","heartbeat_front_addr":"172.21.15.183:6813/2985874686","state":["exists","up"]},{"osd":6,"uuid":"78d15739-e9e0-4f19-ae5f-ee8527a5b527","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4250583193},{"type":"v1","addr":"172.21.15.117:6819","nonce":4250583193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":4250583193},{"type":"v1","addr":"172.21.15.117:6821","nonce":4250583193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":4250583193},{"type":"v1","addr":"172.21.15.117:6825","nonce":4250583193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4250583193},{"type":"v1","addr":"172.21.15.117:6823","nonce":4250583193}]},"public_addr":"172.21.15.117:6819/4250583193","cluster_addr":"172.21.15.117:6821/4250583193","heartbeat_back_addr":"172.21.15.117:6825/4250583193","heartbeat_front_addr":"172.21.15.117:6823/4250583193","state":["exists","up"]},{"osd":7,"uuid":"3724fe1f-f4d7-443b-aea8-ff3ab02bfdc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":386094645},{"type":"v1","addr":"172.21.15.151:6817","nonce":386094645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":386094645},{"type":"v1","addr":"172.21.15.151:6819","nonce":386094645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":386094645},{"type":"v1","addr":"172.21.15.151:6823","nonce":386094645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":386094645},{"type":"v1","addr":"172.21.15.151:6821","nonce":386094645}]},"public_addr":"172.21.15.151:6817/386094645","cluster_addr":"172.21.15.151:6819/386094645","heartbeat_back_addr":"172.21.15.151:6823/386094645","heartbeat_front_addr":"172.21.15.151:6821/386094645","state":["exists","up"]},{"osd":8,"uuid":"006ac092-f015-448e-a0c0-f7b0c4597a18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1829748222},{"type":"v1","addr":"172.21.15.183:6817","nonce":1829748222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1829748222},{"type":"v1","addr":"172.21.15.183:6819","nonce":1829748222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1829748222},{"type":"v1","addr":"172.21.15.183:6823","nonce":1829748222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1829748222},{"type":"v1","addr":"172.21.15.183:6821","nonce":1829748222}]},"public_addr":"172.21.15.183:6817/1829748222","cluster_addr":"172.21.15.183:6819/1829748222","heartbeat_back_addr":"172.21.15.183:6823/1829748222","heartbeat_front_addr":"172.21.15.183:6821/1829748222","state":["exists","up"]},{"osd":9,"uuid":"00a970e6-2502-459e-975c-badac93c9bd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2538315141},{"type":"v1","addr":"172.21.15.117:6827","nonce":2538315141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2538315141},{"type":"v1","addr":"172.21.15.117:6829","nonce":2538315141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2538315141},{"type":"v1","addr":"172.21.15.117:6833","nonce":2538315141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2538315141},{"type":"v1","addr":"172.21.15.117:6831","nonce":2538315141}]},"public_addr":"172.21.15.117:6827/2538315141","cluster_addr":"172.21.15.117:6829/2538315141","heartbeat_back_addr":"172.21.15.117:6833/2538315141","heartbeat_front_addr":"172.21.15.117:6831/2538315141","state":["exists","up"]},{"osd":10,"uuid":"ba2333cd-32a3-41ef-9ea5-287c707f512b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":3801045133},{"type":"v1","addr":"172.21.15.151:6825","nonce":3801045133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3801045133},{"type":"v1","addr":"172.21.15.151:6827","nonce":3801045133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3801045133},{"type":"v1","addr":"172.21.15.151:6831","nonce":3801045133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3801045133},{"type":"v1","addr":"172.21.15.151:6829","nonce":3801045133}]},"public_addr":"172.21.15.151:6825/3801045133","cluster_addr":"172.21.15.151:6827/3801045133","heartbeat_back_addr":"172.21.15.151:6831/3801045133","heartbeat_front_addr":"172.21.15.151:6829/3801045133","state":["exists","up"]},{"osd":11,"uuid":"1556d654-e6fa-42c1-b399-aa53f6837a1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3513899696},{"type":"v1","addr":"172.21.15.183:6825","nonce":3513899696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3513899696},{"type":"v1","addr":"172.21.15.183:6827","nonce":3513899696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3513899696},{"type":"v1","addr":"172.21.15.183:6831","nonce":3513899696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3513899696},{"type":"v1","addr":"172.21.15.183:6829","nonce":3513899696}]},"public_addr":"172.21.15.183:6825/3513899696","cluster_addr":"172.21.15.183:6827/3513899696","heartbeat_back_addr":"172.21.15.183:6831/3513899696","heartbeat_front_addr":"172.21.15.183:6829/3513899696","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:13:17.174041+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:13:42.237001+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:14:06.273315+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:14:39.912564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:15:06.063202+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:15:30.424818+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:16:05.623554+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:16:32.926393+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:16:58.165899+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:17:32.897477+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:18:00.720866+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T03:18:26.474441+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.117:0/186309008":"2024-06-04T03:11:24.655516+0000","172.21.15.117:6801/2142313590":"2024-06-04T03:11:24.655516+0000","172.21.15.117:6800/2142313590":"2024-06-04T03:11:24.655516+0000","172.21.15.117:0/2399725156":"2024-06-04T03:11:24.655516+0000","172.21.15.117:0/1453232061":"2024-06-04T03:10:50.552899+0000","172.21.15.117:0/4281674974":"2024-06-04T03:10:50.552899+0000","172.21.15.117:6800/2424554985":"2024-06-04T03:10:50.552899+0000","172.21.15.117:0/3946365225":"2024-06-04T03:11:24.655516+0000","172.21.15.117:0/1146162354":"2024-06-04T03:10:50.552899+0000","172.21.15.117:6801/2424554985":"2024-06-04T03:10:50.552899+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-03T03:29:22.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:22.717+0000 7f1ec5400700 1 -- 172.21.15.117:0/3161961333 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f1ea40a2ca0 msgr2=0x7f1ea40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:29:22.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:22.717+0000 7f1ec5400700 1 --2- 172.21.15.117:0/3161961333 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f1ea40a2ca0 0x7f1ea40a5160 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1ec0199b40 tx=0x7f1eb00073d0 comp rx=0 tx=0).stop 2024-06-03T03:29:22.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:22.717+0000 7f1ec5400700 1 -- 172.21.15.117:0/3161961333 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ec01084f0 msgr2=0x7f1ec0198a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:29:22.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:22.717+0000 7f1ec5400700 1 --2- 172.21.15.117:0/3161961333 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ec01084f0 0x7f1ec0198a00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1eb800b070 tx=0x7f1eb8004c80 comp rx=0 tx=0).stop 2024-06-03T03:29:22.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:22.717+0000 7f1ec5400700 1 -- 172.21.15.117:0/3161961333 shutdown_connections 2024-06-03T03:29:22.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:22.717+0000 7f1ec5400700 1 --2- 172.21.15.117:0/3161961333 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f1ea40a2ca0 0x7f1ea40a5160 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:29:22.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:22.717+0000 7f1ec5400700 1 --2- 172.21.15.117:0/3161961333 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ec01084f0 0x7f1ec0198a00 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:29:22.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:22.717+0000 7f1ec5400700 1 --2- 172.21.15.117:0/3161961333 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1ec006aa00 0x7f1ec019e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:29:22.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:22.717+0000 7f1ec5400700 1 --2- 172.21.15.117:0/3161961333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ec006b3c0 0x7f1ec019ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:29:22.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:22.717+0000 7f1ec5400700 1 -- 172.21.15.117:0/3161961333 >> 172.21.15.117:0/3161961333 conn(0x7f1ec0078300 msgr2=0x7f1ec00ffef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:29:22.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:22.717+0000 7f1ec5400700 1 -- 172.21.15.117:0/3161961333 shutdown_connections 2024-06-03T03:29:22.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:22.717+0000 7f1ec5400700 1 -- 172.21.15.117:0/3161961333 wait complete. 2024-06-03T03:29:23.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:29:22 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/3161961333' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T03:29:23.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:29:22 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/3161961333' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T03:29:23.231 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:29:22 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/3161961333' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T03:29:23.761 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph fs fail cephfs 2024-06-03T03:29:24.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:29:23 smithi151 ceph-mon[37043]: pgmap v628: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 426 B/s wr, 11 op/s 2024-06-03T03:29:24.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:29:23 smithi117 ceph-mon[31649]: pgmap v628: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 426 B/s wr, 11 op/s 2024-06-03T03:29:24.206 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:29:24.231 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:29:23 smithi183 ceph-mon[37326]: pgmap v628: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 426 B/s wr, 11 op/s 2024-06-03T03:29:25.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:29:25 smithi151 ceph-mon[37043]: pgmap v629: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 0 B/s wr, 9 op/s 2024-06-03T03:29:25.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:29:25 smithi117 ceph-mon[31649]: pgmap v629: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 0 B/s wr, 9 op/s 2024-06-03T03:29:25.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:29:25 smithi183 ceph-mon[37326]: pgmap v629: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 0 B/s wr, 9 op/s 2024-06-03T03:29:26.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:26.748+0000 7fb123600700 1 -- 172.21.15.117:0/2789267944 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb11c102ef0 msgr2=0x7fb11c10f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:29:26.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:26.748+0000 7fb123600700 1 --2- 172.21.15.117:0/2789267944 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb11c102ef0 0x7fb11c10f3a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb10c0098f0 tx=0x7fb10c009c00 comp rx=0 tx=0).stop 2024-06-03T03:29:26.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:26.748+0000 7fb123600700 1 -- 172.21.15.117:0/2789267944 shutdown_connections 2024-06-03T03:29:26.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:26.748+0000 7fb123600700 1 --2- 172.21.15.117:0/2789267944 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb11c102ef0 0x7fb11c10f3a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:29:26.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:26.748+0000 7fb123600700 1 --2- 172.21.15.117:0/2789267944 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb11c1085d0 0x7fb11c1089b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:29:26.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:26.748+0000 7fb123600700 1 --2- 172.21.15.117:0/2789267944 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb11c102530 0x7fb11c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:29:26.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:26.748+0000 7fb123600700 1 -- 172.21.15.117:0/2789267944 >> 172.21.15.117:0/2789267944 conn(0x7fb11c079640 msgr2=0x7fb11c079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:29:26.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:26.748+0000 7fb123600700 1 -- 172.21.15.117:0/2789267944 shutdown_connections 2024-06-03T03:29:26.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:26.748+0000 7fb123600700 1 -- 172.21.15.117:0/2789267944 wait complete. 2024-06-03T03:29:26.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:26.749+0000 7fb123600700 1 Processor -- start 2024-06-03T03:29:26.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:26.749+0000 7fb123600700 1 -- start start 2024-06-03T03:29:26.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:26.749+0000 7fb123600700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb11c102530 0x7fb11c19e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:29:26.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:26.749+0000 7fb123600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb11c102ef0 0x7fb11c19ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:29:26.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:26.750+0000 7fb123600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb11c1085d0 0x7fb11c1989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:29:26.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:26.750+0000 7fb123600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb11c112810 con 0x7fb11c102ef0 2024-06-03T03:29:26.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:26.750+0000 7fb123600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb11c112690 con 0x7fb11c1085d0 2024-06-03T03:29:26.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:26.750+0000 7fb123600700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb11c112990 con 0x7fb11c102530 2024-06-03T03:29:26.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:26.750+0000 7fb11be00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb11c102ef0 0x7fb11c19ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:29:26.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:26.750+0000 7fb120c00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb11c102530 0x7fb11c19e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:29:26.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:26.750+0000 7fb121600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb11c1085d0 0x7fb11c1989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:29:26.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:26.750+0000 7fb120c00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb11c102530 0x7fb11c19e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:57846/0 (socket says 172.21.15.117:57846) 2024-06-03T03:29:26.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:26.750+0000 7fb11be00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb11c102ef0 0x7fb11c19ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:59690/0 (socket says 172.21.15.117:59690) 2024-06-03T03:29:26.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:26.750+0000 7fb120c00700 1 -- 172.21.15.117:0/435379451 learned_addr learned my addr 172.21.15.117:0/435379451 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:29:26.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:26.750+0000 7fb120c00700 1 -- 172.21.15.117:0/435379451 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb11c1085d0 msgr2=0x7fb11c1989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:29:26.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:26.750+0000 7fb120c00700 1 --2- 172.21.15.117:0/435379451 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb11c1085d0 0x7fb11c1989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:29:26.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:26.750+0000 7fb120c00700 1 -- 172.21.15.117:0/435379451 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb11c102ef0 msgr2=0x7fb11c19ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:29:26.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:26.750+0000 7fb120c00700 1 --2- 172.21.15.117:0/435379451 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb11c102ef0 0x7fb11c19ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:29:26.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:26.750+0000 7fb120c00700 1 -- 172.21.15.117:0/435379451 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb11c199150 con 0x7fb11c102530 2024-06-03T03:29:26.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:26.751+0000 7fb120c00700 1 --2- 172.21.15.117:0/435379451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb11c102530 0x7fb11c19e830 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb10400ec10 tx=0x7fb10400ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:29:26.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:26.751+0000 7fb119600700 1 -- 172.21.15.117:0/435379451 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb10400cc90 con 0x7fb11c102530 2024-06-03T03:29:26.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:26.751+0000 7fb119600700 1 -- 172.21.15.117:0/435379451 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb10400ce30 con 0x7fb11c102530 2024-06-03T03:29:26.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:26.751+0000 7fb123600700 1 -- 172.21.15.117:0/435379451 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb11c199440 con 0x7fb11c102530 2024-06-03T03:29:26.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:26.751+0000 7fb119600700 1 -- 172.21.15.117:0/435379451 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb104010690 con 0x7fb11c102530 2024-06-03T03:29:26.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:26.751+0000 7fb123600700 1 -- 172.21.15.117:0/435379451 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb11c0746c0 con 0x7fb11c102530 2024-06-03T03:29:26.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:26.752+0000 7fb123600700 1 -- 172.21.15.117:0/435379451 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb11c103c40 con 0x7fb11c102530 2024-06-03T03:29:26.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:26.754+0000 7fb119600700 1 -- 172.21.15.117:0/435379451 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fb104003680 con 0x7fb11c102530 2024-06-03T03:29:26.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:26.756+0000 7fb119600700 1 --2- 172.21.15.117:0/435379451 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fb1000a2ca0 0x7fb1000a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:29:26.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:26.756+0000 7fb119600700 1 -- 172.21.15.117:0/435379451 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9150+0+0 (secure 0 0 0) 0x7fb1040cf720 con 0x7fb11c102530 2024-06-03T03:29:26.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:26.756+0000 7fb11be00700 1 --2- 172.21.15.117:0/435379451 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fb1000a2ca0 0x7fb1000a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:29:26.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:26.757+0000 7fb11be00700 1 --2- 172.21.15.117:0/435379451 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fb1000a2ca0 0x7fb1000a5160 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb11c079bd0 tx=0x7fb114012040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:29:26.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:26.760+0000 7fb119600700 1 -- 172.21.15.117:0/435379451 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb10409c290 con 0x7fb11c102530 2024-06-03T03:29:27.033 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:27.032+0000 7fb123600700 1 -- 172.21.15.117:0/435379451 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fb11c199f50 con 0x7fb11c102530 2024-06-03T03:29:27.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:27.047+0000 7fb119600700 1 -- 172.21.15.117:0/435379451 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v32) ==== 161+0+0 (secure 0 0 0) 0x7fb11c199f50 con 0x7fb11c102530 2024-06-03T03:29:27.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:27.050+0000 7fb123600700 1 -- 172.21.15.117:0/435379451 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fb1000a2ca0 msgr2=0x7fb1000a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:29:27.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:27.050+0000 7fb123600700 1 --2- 172.21.15.117:0/435379451 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fb1000a2ca0 0x7fb1000a5160 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb11c079bd0 tx=0x7fb114012040 comp rx=0 tx=0).stop 2024-06-03T03:29:27.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:27.050+0000 7fb123600700 1 -- 172.21.15.117:0/435379451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb11c102530 msgr2=0x7fb11c19e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:29:27.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:27.050+0000 7fb123600700 1 --2- 172.21.15.117:0/435379451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb11c102530 0x7fb11c19e830 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb10400ec10 tx=0x7fb10400ef20 comp rx=0 tx=0).stop 2024-06-03T03:29:27.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:27.050+0000 7fb123600700 1 -- 172.21.15.117:0/435379451 shutdown_connections 2024-06-03T03:29:27.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:27.050+0000 7fb123600700 1 --2- 172.21.15.117:0/435379451 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7fb1000a2ca0 0x7fb1000a5160 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:29:27.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:27.050+0000 7fb123600700 1 --2- 172.21.15.117:0/435379451 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb11c1085d0 0x7fb11c1989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:29:27.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:27.050+0000 7fb123600700 1 --2- 172.21.15.117:0/435379451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb11c102530 0x7fb11c19e830 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:29:27.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:27.050+0000 7fb123600700 1 --2- 172.21.15.117:0/435379451 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb11c102ef0 0x7fb11c19ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:29:27.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:27.050+0000 7fb123600700 1 -- 172.21.15.117:0/435379451 >> 172.21.15.117:0/435379451 conn(0x7fb11c079640 msgr2=0x7fb11c101040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:29:27.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:27.050+0000 7fb123600700 1 -- 172.21.15.117:0/435379451 shutdown_connections 2024-06-03T03:29:27.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:27.051+0000 7fb123600700 1 -- 172.21.15.117:0/435379451 wait complete. 2024-06-03T03:29:27.052 INFO:teuthology.orchestra.run.smithi117.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-06-03T03:29:27.406 INFO:journalctl@ceph.mds.b.smithi151.stdout:Jun 03 03:29:27 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-b[67242]: ignoring --setuser ceph since I am not root 2024-06-03T03:29:27.406 INFO:journalctl@ceph.mds.b.smithi151.stdout:Jun 03 03:29:27 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-b[67242]: ignoring --setgroup ceph since I am not root 2024-06-03T03:29:27.406 INFO:journalctl@ceph.mds.b.smithi151.stdout:Jun 03 03:29:27 smithi151 ceph-mds[67246]: -- 172.21.15.151:0/3404770377 <== mon.2 v2:172.21.15.151:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a6e31f3a40 con 0x55a6e31c6c00 2024-06-03T03:29:27.407 INFO:journalctl@ceph.mds.b.smithi151.stdout:Jun 03 03:29:27 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-b[67242]: starting mds.b at 2024-06-03T03:29:27.407 INFO:journalctl@ceph.mds.e.smithi151.stdout:Jun 03 03:29:27 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-e[66639]: ignoring --setuser ceph since I am not root 2024-06-03T03:29:27.407 INFO:journalctl@ceph.mds.e.smithi151.stdout:Jun 03 03:29:27 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-e[66639]: ignoring --setgroup ceph since I am not root 2024-06-03T03:29:27.407 INFO:journalctl@ceph.mds.e.smithi151.stdout:Jun 03 03:29:27 smithi151 ceph-mds[66643]: -- 172.21.15.151:0/2275613758 <== mon.1 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f5bdec1a40 con 0x55f5bde95000 2024-06-03T03:29:27.407 INFO:journalctl@ceph.mds.e.smithi151.stdout:Jun 03 03:29:27 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-e[66639]: starting mds.e at 2024-06-03T03:29:27.407 INFO:journalctl@ceph.mds.h.smithi151.stdout:Jun 03 03:29:27 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-h[66942]: ignoring --setuser ceph since I am not root 2024-06-03T03:29:27.407 INFO:journalctl@ceph.mds.h.smithi151.stdout:Jun 03 03:29:27 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-h[66942]: ignoring --setgroup ceph since I am not root 2024-06-03T03:29:27.408 INFO:journalctl@ceph.mds.h.smithi151.stdout:Jun 03 03:29:27 smithi151 ceph-mds[66946]: -- 172.21.15.151:0/569037762 <== mon.2 v2:172.21.15.151:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x565224147a40 con 0x56522411ac00 2024-06-03T03:29:27.408 INFO:journalctl@ceph.mds.h.smithi151.stdout:Jun 03 03:29:27 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-h[66942]: starting mds.h at 2024-06-03T03:29:27.408 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:29:27 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/435379451' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-06-03T03:29:27.408 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:29:27 smithi151 ceph-mon[37043]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-06-03T03:29:27.408 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:29:27 smithi151 ceph-mon[37043]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-06-03T03:29:27.408 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:29:27 smithi151 ceph-mon[37043]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-06-03T03:29:27.408 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:29:27 smithi151 ceph-mon[37043]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-06-03T03:29:27.408 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:29:27 smithi151 ceph-mon[37043]: osdmap e100: 12 total, 12 up, 12 in 2024-06-03T03:29:27.408 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:29:27 smithi151 ceph-mon[37043]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-06-03T03:29:27.409 INFO:journalctl@ceph.mds.k.smithi151.stdout:Jun 03 03:29:27 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-k[66328]: ignoring --setuser ceph since I am not root 2024-06-03T03:29:27.409 INFO:journalctl@ceph.mds.k.smithi151.stdout:Jun 03 03:29:27 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-k[66328]: ignoring --setgroup ceph since I am not root 2024-06-03T03:29:27.409 INFO:journalctl@ceph.mds.k.smithi151.stdout:Jun 03 03:29:27 smithi151 ceph-mds[66332]: -- 172.21.15.151:0/4294728554 <== mon.2 v2:172.21.15.151:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55998586ba40 con 0x55998583f000 2024-06-03T03:29:27.409 INFO:journalctl@ceph.mds.k.smithi151.stdout:Jun 03 03:29:27 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-k[66328]: starting mds.k at 2024-06-03T03:29:27.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:29:27 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mon-a[31645]: 2024-06-03T03:29:27.035+0000 7f3785200700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-06-03T03:29:27.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:29:27 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/435379451' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-06-03T03:29:27.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:29:27 smithi117 ceph-mon[31649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-06-03T03:29:27.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:29:27 smithi117 ceph-mon[31649]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-06-03T03:29:27.448 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:29:27 smithi117 ceph-mon[31649]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-06-03T03:29:27.448 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:29:27 smithi117 ceph-mon[31649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-06-03T03:29:27.448 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:29:27 smithi117 ceph-mon[31649]: osdmap e100: 12 total, 12 up, 12 in 2024-06-03T03:29:27.448 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:29:27 smithi117 ceph-mon[31649]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-06-03T03:29:27.448 INFO:journalctl@ceph.mds.j.smithi117.stdout:Jun 03 03:29:27 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-j[69608]: ignoring --setuser ceph since I am not root 2024-06-03T03:29:27.448 INFO:journalctl@ceph.mds.j.smithi117.stdout:Jun 03 03:29:27 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-j[69608]: ignoring --setgroup ceph since I am not root 2024-06-03T03:29:27.448 INFO:journalctl@ceph.mds.j.smithi117.stdout:Jun 03 03:29:27 smithi117 ceph-mds[69612]: -- 172.21.15.117:0/1704402150 <== mon.1 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b4eed97a40 con 0x55b4eed6b000 2024-06-03T03:29:27.448 INFO:journalctl@ceph.mds.j.smithi117.stdout:Jun 03 03:29:27 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-j[69608]: starting mds.j at 2024-06-03T03:29:27.449 INFO:journalctl@ceph.mds.a.smithi117.stdout:Jun 03 03:29:27 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-a[71198]: ignoring --setuser ceph since I am not root 2024-06-03T03:29:27.449 INFO:journalctl@ceph.mds.a.smithi117.stdout:Jun 03 03:29:27 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-a[71198]: ignoring --setgroup ceph since I am not root 2024-06-03T03:29:27.449 INFO:journalctl@ceph.mds.a.smithi117.stdout:Jun 03 03:29:27 smithi117 ceph-mds[71202]: -- 172.21.15.117:0/3731072382 <== mon.1 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d12fbeba40 con 0x55d12fbbf000 2024-06-03T03:29:27.449 INFO:journalctl@ceph.mds.a.smithi117.stdout:Jun 03 03:29:27 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-a[71198]: starting mds.a at 2024-06-03T03:29:27.482 INFO:journalctl@ceph.mds.i.smithi183.stdout:Jun 03 03:29:27 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-i[67909]: ignoring --setuser ceph since I am not root 2024-06-03T03:29:27.482 INFO:journalctl@ceph.mds.i.smithi183.stdout:Jun 03 03:29:27 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-i[67909]: ignoring --setgroup ceph since I am not root 2024-06-03T03:29:27.482 INFO:journalctl@ceph.mds.i.smithi183.stdout:Jun 03 03:29:27 smithi183 ceph-mds[67913]: -- 172.21.15.183:0/3177388002 <== mon.2 v2:172.21.15.151:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555b02637a40 con 0x555b0260ac00 2024-06-03T03:29:27.482 INFO:journalctl@ceph.mds.i.smithi183.stdout:Jun 03 03:29:27 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-i[67909]: starting mds.i at 2024-06-03T03:29:27.482 INFO:journalctl@ceph.mds.f.smithi183.stdout:Jun 03 03:29:27 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-f[67293]: ignoring --setuser ceph since I am not root 2024-06-03T03:29:27.483 INFO:journalctl@ceph.mds.f.smithi183.stdout:Jun 03 03:29:27 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-f[67293]: ignoring --setgroup ceph since I am not root 2024-06-03T03:29:27.483 INFO:journalctl@ceph.mds.f.smithi183.stdout:Jun 03 03:29:27 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-f[67293]: starting mds.f at 2024-06-03T03:29:27.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:29:27 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/435379451' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-06-03T03:29:27.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:29:27 smithi183 ceph-mon[37326]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-06-03T03:29:27.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:29:27 smithi183 ceph-mon[37326]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-06-03T03:29:27.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:29:27 smithi183 ceph-mon[37326]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-06-03T03:29:27.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:29:27 smithi183 ceph-mon[37326]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-06-03T03:29:27.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:29:27 smithi183 ceph-mon[37326]: osdmap e100: 12 total, 12 up, 12 in 2024-06-03T03:29:27.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:29:27 smithi183 ceph-mon[37326]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-06-03T03:29:27.484 INFO:journalctl@ceph.mds.l.smithi183.stdout:Jun 03 03:29:27 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-l[66988]: ignoring --setuser ceph since I am not root 2024-06-03T03:29:27.484 INFO:journalctl@ceph.mds.l.smithi183.stdout:Jun 03 03:29:27 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-l[66988]: ignoring --setgroup ceph since I am not root 2024-06-03T03:29:27.484 INFO:journalctl@ceph.mds.l.smithi183.stdout:Jun 03 03:29:27 smithi183 ceph-mds[66992]: -- 172.21.15.183:0/3271377905 <== mon.1 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555d40fc9a40 con 0x555d40f9cc00 2024-06-03T03:29:27.484 INFO:journalctl@ceph.mds.l.smithi183.stdout:Jun 03 03:29:27 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-l[66988]: starting mds.l at 2024-06-03T03:29:27.484 INFO:journalctl@ceph.mds.c.smithi183.stdout:Jun 03 03:29:27 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-c[67597]: ignoring --setuser ceph since I am not root 2024-06-03T03:29:27.484 INFO:journalctl@ceph.mds.c.smithi183.stdout:Jun 03 03:29:27 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-c[67597]: ignoring --setgroup ceph since I am not root 2024-06-03T03:29:27.484 INFO:journalctl@ceph.mds.c.smithi183.stdout:Jun 03 03:29:27 smithi183 ceph-mds[67601]: -- 172.21.15.183:0/1054714237 <== mon.2 v2:172.21.15.151:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562c61665a40 con 0x562c61638800 2024-06-03T03:29:27.484 INFO:journalctl@ceph.mds.c.smithi183.stdout:Jun 03 03:29:27 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-c[67597]: starting mds.c at 2024-06-03T03:29:28.119 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-06-03T03:29:28.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:29:28 smithi151 ceph-mon[37043]: pgmap v631: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-06-03T03:29:28.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:29:28 smithi117 ceph-mon[31649]: pgmap v631: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-06-03T03:29:28.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:29:28 smithi183 ceph-mon[37326]: pgmap v631: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-06-03T03:29:28.556 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:29:29.272 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:29:29 smithi117 ceph-mon[31649]: mds.? [v2:172.21.15.151:6833/1542140846,v1:172.21.15.151:6835/1542140846] up:boot 2024-06-03T03:29:29.273 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:29:29 smithi117 ceph-mon[31649]: mds.? [v2:172.21.15.151:6832/3534598918,v1:172.21.15.151:6834/3534598918] up:boot 2024-06-03T03:29:29.273 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:29:29 smithi117 ceph-mon[31649]: mds.? [v2:172.21.15.183:6833/956348146,v1:172.21.15.183:6835/956348146] up:boot 2024-06-03T03:29:29.273 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:29:29 smithi117 ceph-mon[31649]: mds.? [v2:172.21.15.183:6832/1027854905,v1:172.21.15.183:6834/1027854905] up:boot 2024-06-03T03:29:29.273 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:29:29 smithi117 ceph-mon[31649]: mds.? [v2:172.21.15.183:6836/3521874354,v1:172.21.15.183:6838/3521874354] up:boot 2024-06-03T03:29:29.273 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:29:29 smithi117 ceph-mon[31649]: mds.? [v2:172.21.15.183:6837/1403380998,v1:172.21.15.183:6839/1403380998] up:boot 2024-06-03T03:29:29.273 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:29:29 smithi117 ceph-mon[31649]: mds.? [v2:172.21.15.151:6836/4168474958,v1:172.21.15.151:6838/4168474958] up:boot 2024-06-03T03:29:29.273 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:29:29 smithi117 ceph-mon[31649]: mds.? [v2:172.21.15.151:6837/3583039713,v1:172.21.15.151:6839/3583039713] up:boot 2024-06-03T03:29:29.273 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:29:29 smithi117 ceph-mon[31649]: mds.? [v2:172.21.15.117:6834/2243825865,v1:172.21.15.117:6835/2243825865] up:boot 2024-06-03T03:29:29.273 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:29:29 smithi117 ceph-mon[31649]: mds.? [v2:172.21.15.117:6840/2029483785,v1:172.21.15.117:6841/2029483785] up:boot 2024-06-03T03:29:29.273 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:29:29 smithi117 ceph-mon[31649]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-06-03T03:29:29.273 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:29:29 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-03T03:29:29.273 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:29:29 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-03T03:29:29.274 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:29:29 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-03T03:29:29.274 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:29:29 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-03T03:29:29.274 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:29:29 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-03T03:29:29.274 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:29:29 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-03T03:29:29.274 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:29:29 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-03T03:29:29.274 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:29:29 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-03T03:29:29.274 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:29:29 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-03T03:29:29.274 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:29:29 smithi117 ceph-mon[31649]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-06-03T03:29:29.274 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:29:29 smithi117 ceph-mon[31649]: pgmap v632: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-06-03T03:29:29.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:29:29 smithi151 ceph-mon[37043]: mds.? [v2:172.21.15.151:6833/1542140846,v1:172.21.15.151:6835/1542140846] up:boot 2024-06-03T03:29:29.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:29:29 smithi151 ceph-mon[37043]: mds.? [v2:172.21.15.151:6832/3534598918,v1:172.21.15.151:6834/3534598918] up:boot 2024-06-03T03:29:29.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:29:29 smithi151 ceph-mon[37043]: mds.? [v2:172.21.15.183:6833/956348146,v1:172.21.15.183:6835/956348146] up:boot 2024-06-03T03:29:29.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:29:29 smithi151 ceph-mon[37043]: mds.? [v2:172.21.15.183:6832/1027854905,v1:172.21.15.183:6834/1027854905] up:boot 2024-06-03T03:29:29.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:29:29 smithi151 ceph-mon[37043]: mds.? [v2:172.21.15.183:6836/3521874354,v1:172.21.15.183:6838/3521874354] up:boot 2024-06-03T03:29:29.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:29:29 smithi151 ceph-mon[37043]: mds.? [v2:172.21.15.183:6837/1403380998,v1:172.21.15.183:6839/1403380998] up:boot 2024-06-03T03:29:29.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:29:29 smithi151 ceph-mon[37043]: mds.? [v2:172.21.15.151:6836/4168474958,v1:172.21.15.151:6838/4168474958] up:boot 2024-06-03T03:29:29.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:29:29 smithi151 ceph-mon[37043]: mds.? [v2:172.21.15.151:6837/3583039713,v1:172.21.15.151:6839/3583039713] up:boot 2024-06-03T03:29:29.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:29:29 smithi151 ceph-mon[37043]: mds.? [v2:172.21.15.117:6834/2243825865,v1:172.21.15.117:6835/2243825865] up:boot 2024-06-03T03:29:29.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:29:29 smithi151 ceph-mon[37043]: mds.? [v2:172.21.15.117:6840/2029483785,v1:172.21.15.117:6841/2029483785] up:boot 2024-06-03T03:29:29.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:29:29 smithi151 ceph-mon[37043]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-06-03T03:29:29.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:29:29 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-03T03:29:29.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:29:29 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-03T03:29:29.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:29:29 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-03T03:29:29.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:29:29 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-03T03:29:29.408 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:29:29 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-03T03:29:29.408 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:29:29 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-03T03:29:29.408 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:29:29 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-03T03:29:29.408 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:29:29 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-03T03:29:29.408 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:29:29 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-03T03:29:29.408 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:29:29 smithi151 ceph-mon[37043]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-06-03T03:29:29.408 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:29:29 smithi151 ceph-mon[37043]: pgmap v632: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-06-03T03:29:29.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:29:29 smithi183 ceph-mon[37326]: mds.? [v2:172.21.15.151:6833/1542140846,v1:172.21.15.151:6835/1542140846] up:boot 2024-06-03T03:29:29.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:29:29 smithi183 ceph-mon[37326]: mds.? [v2:172.21.15.151:6832/3534598918,v1:172.21.15.151:6834/3534598918] up:boot 2024-06-03T03:29:29.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:29:29 smithi183 ceph-mon[37326]: mds.? [v2:172.21.15.183:6833/956348146,v1:172.21.15.183:6835/956348146] up:boot 2024-06-03T03:29:29.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:29:29 smithi183 ceph-mon[37326]: mds.? [v2:172.21.15.183:6832/1027854905,v1:172.21.15.183:6834/1027854905] up:boot 2024-06-03T03:29:29.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:29:29 smithi183 ceph-mon[37326]: mds.? [v2:172.21.15.183:6836/3521874354,v1:172.21.15.183:6838/3521874354] up:boot 2024-06-03T03:29:29.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:29:29 smithi183 ceph-mon[37326]: mds.? [v2:172.21.15.183:6837/1403380998,v1:172.21.15.183:6839/1403380998] up:boot 2024-06-03T03:29:29.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:29:29 smithi183 ceph-mon[37326]: mds.? [v2:172.21.15.151:6836/4168474958,v1:172.21.15.151:6838/4168474958] up:boot 2024-06-03T03:29:29.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:29:29 smithi183 ceph-mon[37326]: mds.? [v2:172.21.15.151:6837/3583039713,v1:172.21.15.151:6839/3583039713] up:boot 2024-06-03T03:29:29.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:29:29 smithi183 ceph-mon[37326]: mds.? [v2:172.21.15.117:6834/2243825865,v1:172.21.15.117:6835/2243825865] up:boot 2024-06-03T03:29:29.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:29:29 smithi183 ceph-mon[37326]: mds.? [v2:172.21.15.117:6840/2029483785,v1:172.21.15.117:6841/2029483785] up:boot 2024-06-03T03:29:29.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:29:29 smithi183 ceph-mon[37326]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-06-03T03:29:29.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:29:29 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-03T03:29:29.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:29:29 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-03T03:29:29.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:29:29 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-03T03:29:29.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:29:29 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-03T03:29:29.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:29:29 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-03T03:29:29.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:29:29 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-03T03:29:29.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:29:29 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-03T03:29:29.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:29:29 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-03T03:29:29.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:29:29 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-03T03:29:29.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:29:29 smithi183 ceph-mon[37326]: from='mgr.14152 172.21.15.117:0/649815484' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-06-03T03:29:29.483 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:29:29 smithi183 ceph-mon[37326]: pgmap v632: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-06-03T03:29:31.237 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:31.236+0000 7f8de2200700 1 -- 172.21.15.117:0/614837549 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ddc074fe0 msgr2=0x7f8ddc113120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:29:31.237 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:31.236+0000 7f8de2200700 1 --2- 172.21.15.117:0/614837549 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ddc074fe0 0x7f8ddc113120 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f8dc4009990 tx=0x7f8dc4009ca0 comp rx=0 tx=0).stop 2024-06-03T03:29:31.237 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:31.236+0000 7f8de2200700 1 -- 172.21.15.117:0/614837549 shutdown_connections 2024-06-03T03:29:31.238 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:31.236+0000 7f8de2200700 1 --2- 172.21.15.117:0/614837549 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ddc113660 0x7f8ddc115a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:29:31.238 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:31.236+0000 7f8de2200700 1 --2- 172.21.15.117:0/614837549 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ddc0746c0 0x7f8ddc074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:29:31.238 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:31.236+0000 7f8de2200700 1 --2- 172.21.15.117:0/614837549 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ddc074fe0 0x7f8ddc113120 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:29:31.238 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:31.236+0000 7f8de2200700 1 -- 172.21.15.117:0/614837549 >> 172.21.15.117:0/614837549 conn(0x7f8ddc0795f0 msgr2=0x7f8ddc079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:29:31.238 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:31.236+0000 7f8de2200700 1 -- 172.21.15.117:0/614837549 shutdown_connections 2024-06-03T03:29:31.238 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:31.236+0000 7f8de2200700 1 -- 172.21.15.117:0/614837549 wait complete. 2024-06-03T03:29:31.239 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:31.237+0000 7f8de2200700 1 Processor -- start 2024-06-03T03:29:31.239 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:31.238+0000 7f8de2200700 1 -- start start 2024-06-03T03:29:31.239 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:31.238+0000 7f8de2200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ddc0746c0 0x7f8ddc19f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:29:31.239 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:31.238+0000 7f8de2200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ddc074fe0 0x7f8ddc19f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:29:31.240 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:31.238+0000 7f8de2200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ddc113660 0x7f8ddc1a3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:29:31.240 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:31.238+0000 7f8de2200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8ddc118e10 con 0x7f8ddc074fe0 2024-06-03T03:29:31.240 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:31.238+0000 7f8de2200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8ddc118c90 con 0x7f8ddc0746c0 2024-06-03T03:29:31.240 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:31.238+0000 7f8de2200700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8ddc118f90 con 0x7f8ddc113660 2024-06-03T03:29:31.240 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:31.238+0000 7f8dd3e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ddc074fe0 0x7f8ddc19f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:29:31.240 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:31.238+0000 7f8dd3e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ddc074fe0 0x7f8ddc19f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50780/0 (socket says 172.21.15.117:50780) 2024-06-03T03:29:31.240 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:31.238+0000 7f8dd3e00700 1 -- 172.21.15.117:0/3289349933 learned_addr learned my addr 172.21.15.117:0/3289349933 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:29:31.240 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:31.238+0000 7f8ddbe00700 1 --2- 172.21.15.117:0/3289349933 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ddc113660 0x7f8ddc1a3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:29:31.240 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:31.238+0000 7f8ddb400700 1 --2- 172.21.15.117:0/3289349933 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ddc0746c0 0x7f8ddc19f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:29:31.240 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:31.239+0000 7f8ddbe00700 1 -- 172.21.15.117:0/3289349933 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ddc0746c0 msgr2=0x7f8ddc19f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:29:31.240 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:31.239+0000 7f8ddbe00700 1 --2- 172.21.15.117:0/3289349933 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ddc0746c0 0x7f8ddc19f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:29:31.240 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:31.239+0000 7f8ddbe00700 1 -- 172.21.15.117:0/3289349933 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ddc074fe0 msgr2=0x7f8ddc19f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:29:31.240 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:31.239+0000 7f8ddbe00700 1 --2- 172.21.15.117:0/3289349933 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ddc074fe0 0x7f8ddc19f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:29:31.241 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:31.239+0000 7f8ddbe00700 1 -- 172.21.15.117:0/3289349933 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ddc1a4290 con 0x7f8ddc113660 2024-06-03T03:29:31.241 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:31.239+0000 7f8ddb400700 1 --2- 172.21.15.117:0/3289349933 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ddc0746c0 0x7f8ddc19f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:29:31.241 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:31.239+0000 7f8dd3e00700 1 --2- 172.21.15.117:0/3289349933 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ddc074fe0 0x7f8ddc19f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:29:31.241 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:31.239+0000 7f8ddbe00700 1 --2- 172.21.15.117:0/3289349933 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ddc113660 0x7f8ddc1a3b50 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8dd400ef90 tx=0x7f8dd400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:29:31.242 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:31.240+0000 7f8dd8c00700 1 -- 172.21.15.117:0/3289349933 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd40092e0 con 0x7f8ddc113660 2024-06-03T03:29:31.242 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:31.240+0000 7f8dd8c00700 1 -- 172.21.15.117:0/3289349933 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8dd4004bb0 con 0x7f8ddc113660 2024-06-03T03:29:31.242 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:31.240+0000 7f8de2200700 1 -- 172.21.15.117:0/3289349933 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ddc1a4520 con 0x7f8ddc113660 2024-06-03T03:29:31.242 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:31.240+0000 7f8dd8c00700 1 -- 172.21.15.117:0/3289349933 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd4004280 con 0x7f8ddc113660 2024-06-03T03:29:31.242 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:31.240+0000 7f8de2200700 1 -- 172.21.15.117:0/3289349933 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ddc1abe00 con 0x7f8ddc113660 2024-06-03T03:29:31.242 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:31.241+0000 7f8de2200700 1 -- 172.21.15.117:0/3289349933 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ddc069180 con 0x7f8ddc113660 2024-06-03T03:29:31.243 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:31.243+0000 7f8dd8c00700 1 -- 172.21.15.117:0/3289349933 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f8dd401b030 con 0x7f8ddc113660 2024-06-03T03:29:31.244 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:31.244+0000 7f8dd8c00700 1 --2- 172.21.15.117:0/3289349933 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f8dc00a2ca0 0x7f8dc00a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:29:31.245 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:31.244+0000 7f8dd8c00700 1 -- 172.21.15.117:0/3289349933 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9580+0+0 (secure 0 0 0) 0x7f8dd40cf0a0 con 0x7f8ddc113660 2024-06-03T03:29:31.245 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:31.244+0000 7f8ddb400700 1 --2- 172.21.15.117:0/3289349933 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f8dc00a2ca0 0x7f8dc00a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:29:31.245 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:31.245+0000 7f8ddb400700 1 --2- 172.21.15.117:0/3289349933 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f8dc00a2ca0 0x7f8dc00a5160 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc009680 tx=0x7f8dcc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:29:31.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:31.249+0000 7f8dd8c00700 1 -- 172.21.15.117:0/3289349933 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8dd4022020 con 0x7f8ddc113660 2024-06-03T03:29:31.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:29:31 smithi151 ceph-mon[37043]: pgmap v633: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-06-03T03:29:31.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:29:31 smithi117 ceph-mon[31649]: pgmap v633: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-06-03T03:29:31.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:29:31 smithi183 ceph-mon[37326]: pgmap v633: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-06-03T03:29:31.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:31.520+0000 7f8de2200700 1 -- 172.21.15.117:0/3289349933 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f8ddc1abfd0 con 0x7f8ddc113660 2024-06-03T03:29:32.166 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:32.165+0000 7f8dd8c00700 1 -- 172.21.15.117:0/3289349933 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v34) ==== 104+0+0 (secure 0 0 0) 0x7f8dd4010040 con 0x7f8ddc113660 2024-06-03T03:29:32.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:32.169+0000 7f8de2200700 1 -- 172.21.15.117:0/3289349933 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f8dc00a2ca0 msgr2=0x7f8dc00a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:29:32.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:32.169+0000 7f8de2200700 1 --2- 172.21.15.117:0/3289349933 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f8dc00a2ca0 0x7f8dc00a5160 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc009680 tx=0x7f8dcc009260 comp rx=0 tx=0).stop 2024-06-03T03:29:32.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:32.169+0000 7f8de2200700 1 -- 172.21.15.117:0/3289349933 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ddc113660 msgr2=0x7f8ddc1a3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:29:32.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:32.169+0000 7f8de2200700 1 --2- 172.21.15.117:0/3289349933 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ddc113660 0x7f8ddc1a3b50 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8dd400ef90 tx=0x7f8dd400c3a0 comp rx=0 tx=0).stop 2024-06-03T03:29:32.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:32.170+0000 7f8de2200700 1 -- 172.21.15.117:0/3289349933 shutdown_connections 2024-06-03T03:29:32.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:32.170+0000 7f8de2200700 1 --2- 172.21.15.117:0/3289349933 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f8dc00a2ca0 0x7f8dc00a5160 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:29:32.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:32.170+0000 7f8de2200700 1 --2- 172.21.15.117:0/3289349933 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ddc113660 0x7f8ddc1a3b50 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:29:32.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:32.170+0000 7f8de2200700 1 --2- 172.21.15.117:0/3289349933 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ddc0746c0 0x7f8ddc19f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:29:32.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:32.170+0000 7f8de2200700 1 --2- 172.21.15.117:0/3289349933 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ddc074fe0 0x7f8ddc19f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:29:32.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:32.170+0000 7f8de2200700 1 -- 172.21.15.117:0/3289349933 >> 172.21.15.117:0/3289349933 conn(0x7f8ddc0795f0 msgr2=0x7f8ddc112420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:29:32.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:32.170+0000 7f8de2200700 1 -- 172.21.15.117:0/3289349933 shutdown_connections 2024-06-03T03:29:32.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:32.170+0000 7f8de2200700 1 -- 172.21.15.117:0/3289349933 wait complete. 2024-06-03T03:29:32.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:29:32 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/3289349933' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-06-03T03:29:32.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:29:32 smithi151 ceph-mon[37043]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-06-03T03:29:32.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:29:32 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/3289349933' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-06-03T03:29:32.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:29:32 smithi117 ceph-mon[31649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-06-03T03:29:32.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:29:32 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/3289349933' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-06-03T03:29:32.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:29:32 smithi183 ceph-mon[37326]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-06-03T03:29:33.222 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-06-03T03:29:33.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:29:33 smithi151 ceph-mon[37043]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-06-03T03:29:33.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:29:33 smithi151 ceph-mon[37043]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-06-03T03:29:33.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:29:33 smithi151 ceph-mon[37043]: Cluster is now healthy 2024-06-03T03:29:33.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:29:33 smithi151 ceph-mon[37043]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-06-03T03:29:33.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:29:33 smithi151 ceph-mon[37043]: fsmap 12 up:standby 2024-06-03T03:29:33.407 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:29:33 smithi151 ceph-mon[37043]: pgmap v634: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-06-03T03:29:33.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:29:33 smithi117 ceph-mon[31649]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-06-03T03:29:33.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:29:33 smithi117 ceph-mon[31649]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-06-03T03:29:33.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:29:33 smithi117 ceph-mon[31649]: Cluster is now healthy 2024-06-03T03:29:33.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:29:33 smithi117 ceph-mon[31649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-06-03T03:29:33.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:29:33 smithi117 ceph-mon[31649]: fsmap 12 up:standby 2024-06-03T03:29:33.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:29:33 smithi117 ceph-mon[31649]: pgmap v634: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-06-03T03:29:33.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:29:33 smithi183 ceph-mon[37326]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-06-03T03:29:33.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:29:33 smithi183 ceph-mon[37326]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-06-03T03:29:33.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:29:33 smithi183 ceph-mon[37326]: Cluster is now healthy 2024-06-03T03:29:33.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:29:33 smithi183 ceph-mon[37326]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-06-03T03:29:33.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:29:33 smithi183 ceph-mon[37326]: fsmap 12 up:standby 2024-06-03T03:29:33.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:29:33 smithi183 ceph-mon[37326]: pgmap v634: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-06-03T03:29:33.649 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:29:35.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:29:35 smithi151 ceph-mon[37043]: pgmap v635: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 4 op/s 2024-06-03T03:29:35.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:29:35 smithi117 ceph-mon[31649]: pgmap v635: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 4 op/s 2024-06-03T03:29:35.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:29:35 smithi183 ceph-mon[37326]: pgmap v635: 129 pgs: 129 active+clean; 1.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 4 op/s 2024-06-03T03:29:36.443 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:36.441+0000 7f6a96200700 1 -- 172.21.15.117:0/3312185893 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a90102600 msgr2=0x7f6a901029e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:29:36.443 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:36.441+0000 7f6a96200700 1 --2- 172.21.15.117:0/3312185893 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a90102600 0x7f6a901029e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6a840099c0 tx=0x7f6a84009cd0 comp rx=0 tx=0).stop 2024-06-03T03:29:36.443 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:36.442+0000 7f6a96200700 1 -- 172.21.15.117:0/3312185893 shutdown_connections 2024-06-03T03:29:36.443 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:36.442+0000 7f6a96200700 1 --2- 172.21.15.117:0/3312185893 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6a901076e0 0x7f6a9010f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:29:36.443 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:36.442+0000 7f6a96200700 1 --2- 172.21.15.117:0/3312185893 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a90102600 0x7f6a901029e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:29:36.443 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:36.442+0000 7f6a96200700 1 --2- 172.21.15.117:0/3312185893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a90102fb0 0x7f6a90107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:29:36.443 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:36.442+0000 7f6a96200700 1 -- 172.21.15.117:0/3312185893 >> 172.21.15.117:0/3312185893 conn(0x7f6a900fdeb0 msgr2=0x7f6a901002d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:29:36.443 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:36.442+0000 7f6a96200700 1 -- 172.21.15.117:0/3312185893 shutdown_connections 2024-06-03T03:29:36.443 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:36.442+0000 7f6a96200700 1 -- 172.21.15.117:0/3312185893 wait complete. 2024-06-03T03:29:36.444 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:36.443+0000 7f6a96200700 1 Processor -- start 2024-06-03T03:29:36.445 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:36.444+0000 7f6a96200700 1 -- start start 2024-06-03T03:29:36.445 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:36.444+0000 7f6a96200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a90102600 0x7f6a9019adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:29:36.445 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:36.444+0000 7f6a96200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a90102fb0 0x7f6a9019b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:29:36.445 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:36.444+0000 7f6a96200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6a901076e0 0x7f6a9019f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:29:36.445 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:36.444+0000 7f6a96200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6a90112710 con 0x7f6a90102600 2024-06-03T03:29:36.445 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:36.444+0000 7f6a96200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6a90112590 con 0x7f6a901076e0 2024-06-03T03:29:36.445 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:36.444+0000 7f6a96200700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6a90112890 con 0x7f6a90102fb0 2024-06-03T03:29:36.446 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:36.445+0000 7f6a94e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a90102600 0x7f6a9019adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:29:36.446 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:36.445+0000 7f6a8fe00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a90102fb0 0x7f6a9019b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:29:36.446 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:36.445+0000 7f6a8fe00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a90102fb0 0x7f6a9019b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.117:34826/0 (socket says 172.21.15.117:34826) 2024-06-03T03:29:36.447 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:36.445+0000 7f6a8fe00700 1 -- 172.21.15.117:0/2875262008 learned_addr learned my addr 172.21.15.117:0/2875262008 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:29:36.447 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:36.445+0000 7f6a8fe00700 1 -- 172.21.15.117:0/2875262008 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6a901076e0 msgr2=0x7f6a9019f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:29:36.447 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:36.445+0000 7f6a95800700 1 --2- 172.21.15.117:0/2875262008 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6a901076e0 0x7f6a9019f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:29:36.447 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:36.445+0000 7f6a8fe00700 1 --2- 172.21.15.117:0/2875262008 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6a901076e0 0x7f6a9019f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:29:36.447 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:36.445+0000 7f6a8fe00700 1 -- 172.21.15.117:0/2875262008 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a90102600 msgr2=0x7f6a9019adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:29:36.447 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:36.445+0000 7f6a8fe00700 1 --2- 172.21.15.117:0/2875262008 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a90102600 0x7f6a9019adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:29:36.447 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:36.445+0000 7f6a8fe00700 1 -- 172.21.15.117:0/2875262008 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a9019fdc0 con 0x7f6a90102fb0 2024-06-03T03:29:36.447 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:36.445+0000 7f6a95800700 1 --2- 172.21.15.117:0/2875262008 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6a901076e0 0x7f6a9019f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T03:29:36.447 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:36.445+0000 7f6a94e00700 1 --2- 172.21.15.117:0/2875262008 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a90102600 0x7f6a9019adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:29:36.447 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:36.445+0000 7f6a8fe00700 1 --2- 172.21.15.117:0/2875262008 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a90102fb0 0x7f6a9019b2f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c00e970 tx=0x7f6a7c00ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:29:36.447 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:36.446+0000 7f6a8d600700 1 -- 172.21.15.117:0/2875262008 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a7c00cbd0 con 0x7f6a90102fb0 2024-06-03T03:29:36.447 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:36.446+0000 7f6a8d600700 1 -- 172.21.15.117:0/2875262008 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a7c00cd70 con 0x7f6a90102fb0 2024-06-03T03:29:36.447 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:36.446+0000 7f6a8d600700 1 -- 172.21.15.117:0/2875262008 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a7c0056b0 con 0x7f6a90102fb0 2024-06-03T03:29:36.447 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:36.446+0000 7f6a96200700 1 -- 172.21.15.117:0/2875262008 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a901a00b0 con 0x7f6a90102fb0 2024-06-03T03:29:36.448 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:36.446+0000 7f6a96200700 1 -- 172.21.15.117:0/2875262008 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a901a7930 con 0x7f6a90102fb0 2024-06-03T03:29:36.448 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:36.447+0000 7f6a96200700 1 -- 172.21.15.117:0/2875262008 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a90069180 con 0x7f6a90102fb0 2024-06-03T03:29:36.449 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:36.449+0000 7f6a8d600700 1 -- 172.21.15.117:0/2875262008 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f6a7c004750 con 0x7f6a90102fb0 2024-06-03T03:29:36.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:36.450+0000 7f6a8d600700 1 --2- 172.21.15.117:0/2875262008 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f6a780a2ca0 0x7f6a780a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:29:36.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:36.451+0000 7f6a8d600700 1 -- 172.21.15.117:0/2875262008 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9580+0+0 (secure 0 0 0) 0x7f6a7c0cfcd0 con 0x7f6a90102fb0 2024-06-03T03:29:36.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:36.451+0000 7f6a94e00700 1 --2- 172.21.15.117:0/2875262008 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f6a780a2ca0 0x7f6a780a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:29:36.452 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:36.451+0000 7f6a94e00700 1 --2- 172.21.15.117:0/2875262008 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f6a780a2ca0 0x7f6a780a5160 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f6a84000c00 tx=0x7f6a84005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:29:36.453 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:36.453+0000 7f6a8d600700 1 -- 172.21.15.117:0/2875262008 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6a7c09c690 con 0x7f6a90102fb0 2024-06-03T03:29:36.680 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:36.679+0000 7f6a96200700 1 -- 172.21.15.117:0/2875262008 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f6a901a7b00 con 0x7f6a90102fb0 2024-06-03T03:29:36.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:36.740+0000 7f6a8d600700 1 -- 172.21.15.117:0/2875262008 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v101) ==== 181+0+0 (secure 0 0 0) 0x7f6a7c09bf90 con 0x7f6a90102fb0 2024-06-03T03:29:36.748 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:36.746+0000 7f6a96200700 1 -- 172.21.15.117:0/2875262008 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f6a780a2ca0 msgr2=0x7f6a780a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:29:36.748 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:36.746+0000 7f6a96200700 1 --2- 172.21.15.117:0/2875262008 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f6a780a2ca0 0x7f6a780a5160 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f6a84000c00 tx=0x7f6a84005fb0 comp rx=0 tx=0).stop 2024-06-03T03:29:36.748 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:36.746+0000 7f6a96200700 1 -- 172.21.15.117:0/2875262008 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a90102fb0 msgr2=0x7f6a9019b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:29:36.748 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:36.746+0000 7f6a96200700 1 --2- 172.21.15.117:0/2875262008 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a90102fb0 0x7f6a9019b2f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c00e970 tx=0x7f6a7c00ec80 comp rx=0 tx=0).stop 2024-06-03T03:29:36.748 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:36.747+0000 7f6a96200700 1 -- 172.21.15.117:0/2875262008 shutdown_connections 2024-06-03T03:29:36.748 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:36.747+0000 7f6a96200700 1 --2- 172.21.15.117:0/2875262008 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f6a780a2ca0 0x7f6a780a5160 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:29:36.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:36.747+0000 7f6a96200700 1 --2- 172.21.15.117:0/2875262008 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6a901076e0 0x7f6a9019f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:29:36.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:36.747+0000 7f6a96200700 1 --2- 172.21.15.117:0/2875262008 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a90102600 0x7f6a9019adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:29:36.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:36.747+0000 7f6a96200700 1 --2- 172.21.15.117:0/2875262008 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a90102fb0 0x7f6a9019b2f0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:29:36.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:36.747+0000 7f6a96200700 1 -- 172.21.15.117:0/2875262008 >> 172.21.15.117:0/2875262008 conn(0x7f6a900fdeb0 msgr2=0x7f6a9010e580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:29:36.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:36.748+0000 7f6a96200700 1 -- 172.21.15.117:0/2875262008 shutdown_connections 2024-06-03T03:29:36.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:36.748+0000 7f6a96200700 1 -- 172.21.15.117:0/2875262008 wait complete. 2024-06-03T03:29:36.755 INFO:teuthology.orchestra.run.smithi117.stderr:pool 'cephfs_metadata' removed 2024-06-03T03:29:36.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:29:36 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/2875262008' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-03T03:29:36.947 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:29:36 smithi117 ceph-mon[31649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-03T03:29:37.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:29:36 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/2875262008' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-03T03:29:37.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:29:36 smithi151 ceph-mon[37043]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-03T03:29:37.231 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:29:36 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/2875262008' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-03T03:29:37.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:29:36 smithi183 ceph-mon[37326]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-03T03:29:37.832 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-06-03T03:29:38.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:29:37 smithi151 ceph-mon[37043]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-06-03T03:29:38.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:29:37 smithi151 ceph-mon[37043]: osdmap e101: 12 total, 12 up, 12 in 2024-06-03T03:29:38.157 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:29:37 smithi151 ceph-mon[37043]: pgmap v637: 97 pgs: 97 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:29:38.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:29:37 smithi117 ceph-mon[31649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-06-03T03:29:38.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:29:37 smithi117 ceph-mon[31649]: osdmap e101: 12 total, 12 up, 12 in 2024-06-03T03:29:38.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:29:37 smithi117 ceph-mon[31649]: pgmap v637: 97 pgs: 97 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:29:38.231 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:29:37 smithi183 ceph-mon[37326]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-06-03T03:29:38.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:29:37 smithi183 ceph-mon[37326]: osdmap e101: 12 total, 12 up, 12 in 2024-06-03T03:29:38.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:29:37 smithi183 ceph-mon[37326]: pgmap v637: 97 pgs: 97 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:29:38.277 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:29:39.404 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:29:39 smithi117 ceph-mon[31649]: pgmap v638: 97 pgs: 97 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:29:39.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:29:39 smithi151 ceph-mon[37043]: pgmap v638: 97 pgs: 97 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:29:39.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:29:39 smithi183 ceph-mon[37326]: pgmap v638: 97 pgs: 97 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:29:40.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:40.832+0000 7f7265000700 1 -- 172.21.15.117:0/204492528 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7260105070 msgr2=0x7f7260111580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:29:40.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:40.832+0000 7f7265000700 1 --2- 172.21.15.117:0/204492528 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7260105070 0x7f7260111580 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f725800b0a0 tx=0x7f725800b3b0 comp rx=0 tx=0).stop 2024-06-03T03:29:40.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:40.832+0000 7f7265000700 1 -- 172.21.15.117:0/204492528 shutdown_connections 2024-06-03T03:29:40.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:40.832+0000 7f7265000700 1 --2- 172.21.15.117:0/204492528 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7260105070 0x7f7260111580 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:29:40.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:40.832+0000 7f7265000700 1 --2- 172.21.15.117:0/204492528 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f726010a6b0 0x7f726010aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:29:40.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:40.832+0000 7f7265000700 1 --2- 172.21.15.117:0/204492528 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f72601046b0 0x7f7260104b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:29:40.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:40.832+0000 7f7265000700 1 -- 172.21.15.117:0/204492528 >> 172.21.15.117:0/204492528 conn(0x7f72601001a0 msgr2=0x7f72601025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:29:40.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:40.833+0000 7f7265000700 1 -- 172.21.15.117:0/204492528 shutdown_connections 2024-06-03T03:29:40.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:40.833+0000 7f7265000700 1 -- 172.21.15.117:0/204492528 wait complete. 2024-06-03T03:29:40.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:40.834+0000 7f7265000700 1 Processor -- start 2024-06-03T03:29:40.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:40.834+0000 7f7265000700 1 -- start start 2024-06-03T03:29:40.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:40.835+0000 7f7265000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72601046b0 0x7f726019ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:29:40.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:40.835+0000 7f7265000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7260105070 0x7f726019b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:29:40.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:40.835+0000 7f7265000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f726010a6b0 0x7f726019f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:29:40.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:40.835+0000 7f7265000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7260114a40 con 0x7f72601046b0 2024-06-03T03:29:40.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:40.835+0000 7f7265000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f72601148c0 con 0x7f7260105070 2024-06-03T03:29:40.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:40.835+0000 7f7265000700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7260114bc0 con 0x7f726010a6b0 2024-06-03T03:29:40.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:40.835+0000 7f725de00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72601046b0 0x7f726019ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:29:40.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:40.835+0000 7f725e800700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f726010a6b0 0x7f726019f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:29:40.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:40.835+0000 7f725de00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72601046b0 0x7f726019ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56108/0 (socket says 172.21.15.117:56108) 2024-06-03T03:29:40.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:40.835+0000 7f725de00700 1 -- 172.21.15.117:0/3662466696 learned_addr learned my addr 172.21.15.117:0/3662466696 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:29:40.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:40.835+0000 7f725d400700 1 --2- 172.21.15.117:0/3662466696 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7260105070 0x7f726019b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:29:40.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:40.836+0000 7f725e800700 1 -- 172.21.15.117:0/3662466696 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7260105070 msgr2=0x7f726019b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:29:40.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:40.836+0000 7f725e800700 1 --2- 172.21.15.117:0/3662466696 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7260105070 0x7f726019b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:29:40.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:40.836+0000 7f725e800700 1 -- 172.21.15.117:0/3662466696 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72601046b0 msgr2=0x7f726019ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:29:40.838 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:40.836+0000 7f725e800700 1 --2- 172.21.15.117:0/3662466696 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72601046b0 0x7f726019ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:29:40.838 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:40.836+0000 7f725e800700 1 -- 172.21.15.117:0/3662466696 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f726019fdf0 con 0x7f726010a6b0 2024-06-03T03:29:40.838 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:40.836+0000 7f725e800700 1 --2- 172.21.15.117:0/3662466696 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f726010a6b0 0x7f726019f6b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f725800b070 tx=0x7f7258004c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:29:40.838 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:40.836+0000 7f7256a00700 1 -- 172.21.15.117:0/3662466696 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7258021070 con 0x7f726010a6b0 2024-06-03T03:29:40.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:40.836+0000 7f7256a00700 1 -- 172.21.15.117:0/3662466696 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7258007b80 con 0x7f726010a6b0 2024-06-03T03:29:40.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:40.836+0000 7f7256a00700 1 -- 172.21.15.117:0/3662466696 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7258005040 con 0x7f726010a6b0 2024-06-03T03:29:40.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:40.836+0000 7f7265000700 1 -- 172.21.15.117:0/3662466696 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72601a0080 con 0x7f726010a6b0 2024-06-03T03:29:40.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:40.836+0000 7f7265000700 1 -- 172.21.15.117:0/3662466696 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72601a7960 con 0x7f726010a6b0 2024-06-03T03:29:40.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:40.837+0000 7f7265000700 1 -- 172.21.15.117:0/3662466696 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7260105df0 con 0x7f726010a6b0 2024-06-03T03:29:40.840 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:40.840+0000 7f7256a00700 1 -- 172.21.15.117:0/3662466696 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f7258024070 con 0x7f726010a6b0 2024-06-03T03:29:40.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:40.841+0000 7f7256a00700 1 --2- 172.21.15.117:0/3662466696 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f72440a2c20 0x7f72440a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:29:40.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:40.841+0000 7f7256a00700 1 -- 172.21.15.117:0/3662466696 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9055+0+0 (secure 0 0 0) 0x7f72580d0200 con 0x7f726010a6b0 2024-06-03T03:29:40.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:40.841+0000 7f725de00700 1 --2- 172.21.15.117:0/3662466696 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f72440a2c20 0x7f72440a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:29:40.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:40.842+0000 7f725de00700 1 --2- 172.21.15.117:0/3662466696 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f72440a2c20 0x7f72440a50e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f7250009ca0 tx=0x7f7250009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:29:40.844 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:40.843+0000 7f7256a00700 1 -- 172.21.15.117:0/3662466696 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7258007d30 con 0x7f726010a6b0 2024-06-03T03:29:41.077 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:41.076+0000 7f7265000700 1 -- 172.21.15.117:0/3662466696 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f726002dca0 con 0x7f726010a6b0 2024-06-03T03:29:41.138 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:41.136+0000 7f7256a00700 1 -- 172.21.15.117:0/3662466696 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v102) ==== 169+0+0 (secure 0 0 0) 0x7f725809ce60 con 0x7f726010a6b0 2024-06-03T03:29:41.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:41.142+0000 7f7265000700 1 -- 172.21.15.117:0/3662466696 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f72440a2c20 msgr2=0x7f72440a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:29:41.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:41.142+0000 7f7265000700 1 --2- 172.21.15.117:0/3662466696 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f72440a2c20 0x7f72440a50e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f7250009ca0 tx=0x7f7250009260 comp rx=0 tx=0).stop 2024-06-03T03:29:41.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:41.142+0000 7f7265000700 1 -- 172.21.15.117:0/3662466696 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f726010a6b0 msgr2=0x7f726019f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:29:41.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:41.142+0000 7f7265000700 1 --2- 172.21.15.117:0/3662466696 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f726010a6b0 0x7f726019f6b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f725800b070 tx=0x7f7258004c80 comp rx=0 tx=0).stop 2024-06-03T03:29:41.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:41.142+0000 7f7265000700 1 -- 172.21.15.117:0/3662466696 shutdown_connections 2024-06-03T03:29:41.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:41.142+0000 7f7265000700 1 --2- 172.21.15.117:0/3662466696 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f72440a2c20 0x7f72440a50e0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:29:41.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:41.142+0000 7f7265000700 1 --2- 172.21.15.117:0/3662466696 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f726010a6b0 0x7f726019f6b0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:29:41.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:41.142+0000 7f7265000700 1 --2- 172.21.15.117:0/3662466696 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72601046b0 0x7f726019ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:29:41.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:41.142+0000 7f7265000700 1 --2- 172.21.15.117:0/3662466696 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7260105070 0x7f726019b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:29:41.144 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:41.142+0000 7f7265000700 1 -- 172.21.15.117:0/3662466696 >> 172.21.15.117:0/3662466696 conn(0x7f72601001a0 msgr2=0x7f72601006c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:29:41.144 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:41.143+0000 7f7265000700 1 -- 172.21.15.117:0/3662466696 shutdown_connections 2024-06-03T03:29:41.144 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:41.143+0000 7f7265000700 1 -- 172.21.15.117:0/3662466696 wait complete. 2024-06-03T03:29:41.146 INFO:teuthology.orchestra.run.smithi117.stderr:pool 'cephfs_data' removed 2024-06-03T03:29:41.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:29:41 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/3662466696' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-03T03:29:41.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:29:41 smithi151 ceph-mon[37043]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-03T03:29:41.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:29:41 smithi151 ceph-mon[37043]: pgmap v639: 97 pgs: 97 active+clean; 577 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:29:41.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:29:41 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/3662466696' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-03T03:29:41.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:29:41 smithi117 ceph-mon[31649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-03T03:29:41.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:29:41 smithi117 ceph-mon[31649]: pgmap v639: 97 pgs: 97 active+clean; 577 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:29:41.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:29:41 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/3662466696' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-03T03:29:41.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:29:41 smithi183 ceph-mon[37326]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-03T03:29:41.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:29:41 smithi183 ceph-mon[37326]: pgmap v639: 97 pgs: 97 active+clean; 577 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:29:42.235 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 -- ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-06-03T03:29:42.408 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:29:42 smithi151 ceph-mon[37043]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-06-03T03:29:42.408 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:29:42 smithi151 ceph-mon[37043]: osdmap e102: 12 total, 12 up, 12 in 2024-06-03T03:29:42.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:29:42 smithi117 ceph-mon[31649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-06-03T03:29:42.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:29:42 smithi117 ceph-mon[31649]: osdmap e102: 12 total, 12 up, 12 in 2024-06-03T03:29:42.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:29:42 smithi183 ceph-mon[37326]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-06-03T03:29:42.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:29:42 smithi183 ceph-mon[37326]: osdmap e102: 12 total, 12 up, 12 in 2024-06-03T03:29:42.695 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T03:29:43.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:29:43 smithi117 ceph-mon[31649]: pgmap v641: 33 pgs: 33 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:29:43.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:29:43 smithi183 ceph-mon[37326]: pgmap v641: 33 pgs: 33 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:29:43.656 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:29:43 smithi151 ceph-mon[37043]: pgmap v641: 33 pgs: 33 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:29:45.181 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:29:45 smithi117 ceph-mon[31649]: pgmap v642: 33 pgs: 33 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:29:45.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:45.249+0000 7f69ad800700 1 -- 172.21.15.117:0/3499594519 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69a81089d0 msgr2=0x7f69a8108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:29:45.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:45.249+0000 7f69ad800700 1 --2- 172.21.15.117:0/3499594519 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69a81089d0 0x7f69a8108e50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f69940098f0 tx=0x7f6994009c00 comp rx=0 tx=0).stop 2024-06-03T03:29:45.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:45.249+0000 7f69ad800700 1 -- 172.21.15.117:0/3499594519 shutdown_connections 2024-06-03T03:29:45.251 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:45.249+0000 7f69ad800700 1 --2- 172.21.15.117:0/3499594519 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69a8109390 0x7f69a8115830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:29:45.251 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:45.249+0000 7f69ad800700 1 --2- 172.21.15.117:0/3499594519 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69a810d8a0 0x7f69a810dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:29:45.251 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:45.249+0000 7f69ad800700 1 --2- 172.21.15.117:0/3499594519 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69a81089d0 0x7f69a8108e50 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:29:45.251 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:45.249+0000 7f69ad800700 1 -- 172.21.15.117:0/3499594519 >> 172.21.15.117:0/3499594519 conn(0x7f69a8078220 msgr2=0x7f69a81072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:29:45.251 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:45.250+0000 7f69ad800700 1 -- 172.21.15.117:0/3499594519 shutdown_connections 2024-06-03T03:29:45.251 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:45.250+0000 7f69ad800700 1 -- 172.21.15.117:0/3499594519 wait complete. 2024-06-03T03:29:45.252 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:45.251+0000 7f69ad800700 1 Processor -- start 2024-06-03T03:29:45.252 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:45.251+0000 7f69ad800700 1 -- start start 2024-06-03T03:29:45.252 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:45.251+0000 7f69ad800700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69a81089d0 0x7f69a819f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:29:45.252 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:45.251+0000 7f69ad800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69a8109390 0x7f69a819f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:29:45.252 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:45.251+0000 7f69ad800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69a810d8a0 0x7f69a81a3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:29:45.252 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:45.251+0000 7f69ad800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f69a8118d20 con 0x7f69a810d8a0 2024-06-03T03:29:45.252 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:45.251+0000 7f69ad800700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f69a8118ba0 con 0x7f69a8109390 2024-06-03T03:29:45.252 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:45.251+0000 7f69ad800700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f69a8118ea0 con 0x7f69a81089d0 2024-06-03T03:29:45.252 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:45.252+0000 7f69ace00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69a810d8a0 0x7f69a81a3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:29:45.253 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:45.252+0000 7f69ace00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69a810d8a0 0x7f69a81a3a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56120/0 (socket says 172.21.15.117:56120) 2024-06-03T03:29:45.253 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:45.252+0000 7f69a7400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69a8109390 0x7f69a819f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:29:45.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:45.252+0000 7f69a7e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69a81089d0 0x7f69a819f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:29:45.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:45.252+0000 7f69ace00700 1 -- 172.21.15.117:0/1652078040 learned_addr learned my addr 172.21.15.117:0/1652078040 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-06-03T03:29:45.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:45.252+0000 7f69a7400700 1 -- 172.21.15.117:0/1652078040 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69a81089d0 msgr2=0x7f69a819f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:29:45.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:45.252+0000 7f69a7400700 1 --2- 172.21.15.117:0/1652078040 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69a81089d0 0x7f69a819f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:29:45.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:45.252+0000 7f69a7400700 1 -- 172.21.15.117:0/1652078040 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69a810d8a0 msgr2=0x7f69a81a3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:29:45.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:45.252+0000 7f69a7400700 1 --2- 172.21.15.117:0/1652078040 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69a810d8a0 0x7f69a81a3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:29:45.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:45.252+0000 7f69a7400700 1 -- 172.21.15.117:0/1652078040 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69a81a41d0 con 0x7f69a8109390 2024-06-03T03:29:45.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:45.252+0000 7f69ace00700 1 --2- 172.21.15.117:0/1652078040 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69a810d8a0 0x7f69a81a3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T03:29:45.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:45.252+0000 7f69a7400700 1 --2- 172.21.15.117:0/1652078040 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69a8109390 0x7f69a819f750 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6994000c00 tx=0x7f699400fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:29:45.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:45.253+0000 7f69a4c00700 1 -- 172.21.15.117:0/1652078040 <== mon.2 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6994005040 con 0x7f69a8109390 2024-06-03T03:29:45.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:45.253+0000 7f69a4c00700 1 -- 172.21.15.117:0/1652078040 <== mon.2 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69940056c0 con 0x7f69a8109390 2024-06-03T03:29:45.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:45.253+0000 7f69ad800700 1 -- 172.21.15.117:0/1652078040 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69a81a4460 con 0x7f69a8109390 2024-06-03T03:29:45.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:45.253+0000 7f69a4c00700 1 -- 172.21.15.117:0/1652078040 <== mon.2 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6994003a00 con 0x7f69a8109390 2024-06-03T03:29:45.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:45.253+0000 7f69ad800700 1 -- 172.21.15.117:0/1652078040 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69a81abd40 con 0x7f69a8109390 2024-06-03T03:29:45.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:45.254+0000 7f69ad800700 1 -- 172.21.15.117:0/1652078040 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69a8069180 con 0x7f69a8109390 2024-06-03T03:29:45.257 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:45.257+0000 7f69a4c00700 1 -- 172.21.15.117:0/1652078040 <== mon.2 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f699400b560 con 0x7f69a8109390 2024-06-03T03:29:45.259 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:45.258+0000 7f69a4c00700 1 --2- 172.21.15.117:0/1652078040 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f698c0a2c20 0x7f698c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T03:29:45.259 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:45.258+0000 7f69a4c00700 1 -- 172.21.15.117:0/1652078040 <== mon.2 v2:172.21.15.151:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8496+0+0 (secure 0 0 0) 0x7f69940d0300 con 0x7f69a8109390 2024-06-03T03:29:45.259 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:45.258+0000 7f69a7e00700 1 --2- 172.21.15.117:0/1652078040 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f698c0a2c20 0x7f698c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T03:29:45.259 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:45.259+0000 7f69a7e00700 1 --2- 172.21.15.117:0/1652078040 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f698c0a2c20 0x7f698c0a50e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f6998009750 tx=0x7f6998009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T03:29:45.260 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:45.260+0000 7f69a4c00700 1 -- 172.21.15.117:0/1652078040 <== mon.2 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6994013030 con 0x7f69a8109390 2024-06-03T03:29:45.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:29:45 smithi151 ceph-mon[37043]: pgmap v642: 33 pgs: 33 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:29:45.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:29:45 smithi183 ceph-mon[37326]: pgmap v642: 33 pgs: 33 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:29:45.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:45.481+0000 7f69ad800700 1 -- 172.21.15.117:0/1652078040 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f69a802dca0 con 0x7f69a8109390 2024-06-03T03:29:46.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:46.166+0000 7f69a4c00700 1 -- 172.21.15.117:0/1652078040 <== mon.2 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v103) ==== 178+0+0 (secure 0 0 0) 0x7f699409d1b0 con 0x7f69a8109390 2024-06-03T03:29:46.173 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:46.172+0000 7f69ad800700 1 -- 172.21.15.117:0/1652078040 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f698c0a2c20 msgr2=0x7f698c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:29:46.173 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:46.172+0000 7f69ad800700 1 --2- 172.21.15.117:0/1652078040 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f698c0a2c20 0x7f698c0a50e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f6998009750 tx=0x7f6998009310 comp rx=0 tx=0).stop 2024-06-03T03:29:46.173 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:46.172+0000 7f69ad800700 1 -- 172.21.15.117:0/1652078040 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69a8109390 msgr2=0x7f69a819f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T03:29:46.173 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:46.172+0000 7f69ad800700 1 --2- 172.21.15.117:0/1652078040 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69a8109390 0x7f69a819f750 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6994000c00 tx=0x7f699400fa00 comp rx=0 tx=0).stop 2024-06-03T03:29:46.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:46.172+0000 7f69ad800700 1 -- 172.21.15.117:0/1652078040 shutdown_connections 2024-06-03T03:29:46.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:46.172+0000 7f69ad800700 1 --2- 172.21.15.117:0/1652078040 >> [v2:172.21.15.117:6800/1398160887,v1:172.21.15.117:6801/1398160887] conn(0x7f698c0a2c20 0x7f698c0a50e0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:29:46.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:46.172+0000 7f69ad800700 1 --2- 172.21.15.117:0/1652078040 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69a810d8a0 0x7f69a81a3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:29:46.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:46.172+0000 7f69ad800700 1 --2- 172.21.15.117:0/1652078040 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69a81089d0 0x7f69a819f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T03:29:46.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:46.172+0000 7f69ad800700 1 --2- 172.21.15.117:0/1652078040 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69a8109390 0x7f69a819f750 secure :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6994000c00 tx=0x7f699400fa00 comp rx=0 tx=0).stop 2024-06-03T03:29:46.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:46.172+0000 7f69ad800700 1 -- 172.21.15.117:0/1652078040 >> 172.21.15.117:0/1652078040 conn(0x7f69a8078220 msgr2=0x7f69a8078880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T03:29:46.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:46.173+0000 7f69ad800700 1 -- 172.21.15.117:0/1652078040 shutdown_connections 2024-06-03T03:29:46.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-06-03T03:29:46.173+0000 7f69ad800700 1 -- 172.21.15.117:0/1652078040 wait complete. 2024-06-03T03:29:46.177 INFO:teuthology.orchestra.run.smithi117.stderr:pool 'cephfs_data_ec' removed 2024-06-03T03:29:46.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:29:46 smithi151 ceph-mon[37043]: from='client.? 172.21.15.117:0/1652078040' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-03T03:29:46.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:29:46 smithi151 ceph-mon[37043]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-03T03:29:46.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:29:46 smithi117 ceph-mon[31649]: from='client.? 172.21.15.117:0/1652078040' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-03T03:29:46.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:29:46 smithi117 ceph-mon[31649]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-03T03:29:46.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:29:46 smithi183 ceph-mon[37326]: from='client.? 172.21.15.117:0/1652078040' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-03T03:29:46.482 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:29:46 smithi183 ceph-mon[37326]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-03T03:29:47.236 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-03T03:29:47.324 DEBUG:teuthology.orchestra.run.smithi151:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-03T03:29:47.357 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-03T03:29:47.389 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-06-03T03:29:47.390 DEBUG:teuthology.orchestra.run.smithi117:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-06-03T03:29:47.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:29:47 smithi117 ceph-mon[31649]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-06-03T03:29:47.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:29:47 smithi117 ceph-mon[31649]: osdmap e103: 12 total, 12 up, 12 in 2024-06-03T03:29:47.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:29:47 smithi117 ceph-mon[31649]: pgmap v644: 1 pgs: 1 active+clean; 577 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:29:47.447 DEBUG:teuthology.orchestra.run.smithi151:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-06-03T03:29:47.465 DEBUG:teuthology.orchestra.run.smithi183:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-06-03T03:29:47.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:29:47 smithi183 ceph-mon[37326]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-06-03T03:29:47.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:29:47 smithi183 ceph-mon[37326]: osdmap e103: 12 total, 12 up, 12 in 2024-06-03T03:29:47.481 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:29:47 smithi183 ceph-mon[37326]: pgmap v644: 1 pgs: 1 active+clean; 577 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:29:47.483 INFO:tasks.cephadm:Stopping all daemons... 2024-06-03T03:29:47.483 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-06-03T03:29:47.483 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@mon.a 2024-06-03T03:29:47.656 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:29:47 smithi151 ceph-mon[37043]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-06-03T03:29:47.656 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:29:47 smithi151 ceph-mon[37043]: osdmap e103: 12 total, 12 up, 12 in 2024-06-03T03:29:47.656 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:29:47 smithi151 ceph-mon[37043]: pgmap v644: 1 pgs: 1 active+clean; 577 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T03:29:47.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:29:47 smithi117 systemd[1]: Stopping Ceph mon.a for a0e8473a-2156-11ef-bc9b-c7b262605968... 2024-06-03T03:29:48.446 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:29:48 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mon-a[31645]: 2024-06-03T03:29:48.099+0000 7f378c000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T03:29:48.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:29:48 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mon-a[31645]: 2024-06-03T03:29:48.099+0000 7f378c000700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-06-03T03:29:48.447 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:29:48 smithi117 podman[97978]: 2024-06-03 03:29:48.171690041 +0000 UTC m=+0.231079528 container died 92ba733bffb8ae7bb2913d8cd0c8a11cfbc4718d08f5f82680e75aed6d029674 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=) 2024-06-03T03:29:49.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:29:48 smithi117 podman[97978]: 2024-06-03 03:29:48.835686803 +0000 UTC m=+0.895076296 container cleanup 92ba733bffb8ae7bb2913d8cd0c8a11cfbc4718d08f5f82680e75aed6d029674 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mon-a, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528) 2024-06-03T03:29:49.196 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:29:48 smithi117 bash[97978]: ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mon-a 2024-06-03T03:29:49.492 INFO:journalctl@ceph.mon.a.smithi117.stdout:Jun 03 03:29:49 smithi117 podman[97991]: 2024-06-03 03:29:49.201159236 +0000 UTC m=+1.011112109 container remove 92ba733bffb8ae7bb2913d8cd0c8a11cfbc4718d08f5f82680e75aed6d029674 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mon-a, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0) 2024-06-03T03:29:49.530 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@mon.a.service' 2024-06-03T03:29:49.655 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T03:29:49.656 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-06-03T03:29:49.656 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-06-03T03:29:49.656 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@mon.b 2024-06-03T03:29:50.016 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:29:49 smithi151 systemd[1]: Stopping Ceph mon.b for a0e8473a-2156-11ef-bc9b-c7b262605968... 2024-06-03T03:29:50.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:29:50 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mon-b[37039]: 2024-06-03T03:29:50.010+0000 7f2e50b41700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T03:29:50.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:29:50 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mon-b[37039]: 2024-06-03T03:29:50.010+0000 7f2e50b41700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-06-03T03:29:50.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:29:50 smithi151 podman[70973]: 2024-06-03 03:29:50.082712272 +0000 UTC m=+0.225265547 container died 193aee5b1d93f2a5ea2f4c162941a9064cb59f5c91830f5f31f1c59d0414fa4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mon-b, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-06-03T03:29:50.838 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:29:50 smithi151 podman[70973]: 2024-06-03 03:29:50.544853675 +0000 UTC m=+0.687406935 container cleanup 193aee5b1d93f2a5ea2f4c162941a9064cb59f5c91830f5f31f1c59d0414fa4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mon-b, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-06-03T03:29:50.839 INFO:journalctl@ceph.mon.b.smithi151.stdout:Jun 03 03:29:50 smithi151 bash[70973]: ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mon-b 2024-06-03T03:29:50.966 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@mon.b.service' 2024-06-03T03:29:51.011 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T03:29:51.011 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-06-03T03:29:51.011 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-06-03T03:29:51.011 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@mon.c 2024-06-03T03:29:51.336 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:29:51 smithi183 systemd[1]: Stopping Ceph mon.c for a0e8473a-2156-11ef-bc9b-c7b262605968... 2024-06-03T03:29:51.591 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:29:51 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mon-c[37322]: 2024-06-03T03:29:51.336+0000 7fc6a6293700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T03:29:51.591 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:29:51 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mon-c[37322]: 2024-06-03T03:29:51.336+0000 7fc6a6293700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-06-03T03:29:51.887 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:29:51 smithi183 podman[71571]: 2024-06-03 03:29:51.591018066 +0000 UTC m=+0.378682280 container stop 24b9dbb1fddb5933dc92127d68c469848c0cd4e5dbdeea8fa475c3ce5cc1819a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mon-c, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, ceph=True) 2024-06-03T03:29:51.887 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:29:51 smithi183 podman[71571]: 2024-06-03 03:29:51.617483663 +0000 UTC m=+0.405147877 container died 24b9dbb1fddb5933dc92127d68c469848c0cd4e5dbdeea8fa475c3ce5cc1819a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-06-03T03:29:52.188 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:29:51 smithi183 podman[71571]: 2024-06-03 03:29:51.978382073 +0000 UTC m=+0.766046288 container cleanup 24b9dbb1fddb5933dc92127d68c469848c0cd4e5dbdeea8fa475c3ce5cc1819a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mon-c, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-06-03T03:29:52.188 INFO:journalctl@ceph.mon.c.smithi183.stdout:Jun 03 03:29:51 smithi183 bash[71571]: ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mon-c 2024-06-03T03:29:52.412 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@mon.c.service' 2024-06-03T03:29:52.458 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T03:29:52.458 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-06-03T03:29:52.459 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-06-03T03:29:52.459 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@mgr.x 2024-06-03T03:29:52.946 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:29:52 smithi117 systemd[1]: Stopping Ceph mgr.x for a0e8473a-2156-11ef-bc9b-c7b262605968... 2024-06-03T03:29:53.448 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:29:53 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:29:53.051+0000 7fbc35600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T03:29:53.448 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:29:53 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x[31883]: 2024-06-03T03:29:53.051+0000 7fbc35600700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-06-03T03:29:53.448 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:29:53 smithi117 podman[98097]: 2024-06-03 03:29:53.15668974 +0000 UTC m=+0.276048746 container died 31f67206095a987561f775a5b1af7dffd82d46ca8b1fd2e7e43d1c74edfbbbd6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-06-03T03:29:54.100 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:29:53 smithi117 podman[98097]: 2024-06-03 03:29:53.813226193 +0000 UTC m=+0.932585239 container cleanup 31f67206095a987561f775a5b1af7dffd82d46ca8b1fd2e7e43d1c74edfbbbd6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=) 2024-06-03T03:29:54.100 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:29:53 smithi117 bash[98097]: ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x 2024-06-03T03:29:54.446 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Jun 03 03:29:54 smithi117 podman[98112]: 2024-06-03 03:29:54.099082295 +0000 UTC m=+0.926702476 container remove 31f67206095a987561f775a5b1af7dffd82d46ca8b1fd2e7e43d1c74edfbbbd6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-x, GIT_CLEAN=True, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS) 2024-06-03T03:29:54.593 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@mgr.x.service' 2024-06-03T03:29:54.706 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T03:29:54.706 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-06-03T03:29:54.707 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-06-03T03:29:54.707 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@mgr.y 2024-06-03T03:29:55.042 INFO:journalctl@ceph.mgr.y.smithi151.stdout:Jun 03 03:29:54 smithi151 systemd[1]: Stopping Ceph mgr.y for a0e8473a-2156-11ef-bc9b-c7b262605968... 2024-06-03T03:29:55.336 INFO:journalctl@ceph.mgr.y.smithi151.stdout:Jun 03 03:29:55 smithi151 podman[71095]: 2024-06-03 03:29:55.042382014 +0000 UTC m=+0.141264475 container died 11784dd35c97acae90e740bd4df326c57b9cf9902d84e9d60baf47df567740da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-y, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True) 2024-06-03T03:29:55.656 INFO:journalctl@ceph.mgr.y.smithi151.stdout:Jun 03 03:29:55 smithi151 podman[71095]: 2024-06-03 03:29:55.410791813 +0000 UTC m=+0.509674312 container cleanup 11784dd35c97acae90e740bd4df326c57b9cf9902d84e9d60baf47df567740da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-y, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc) 2024-06-03T03:29:55.656 INFO:journalctl@ceph.mgr.y.smithi151.stdout:Jun 03 03:29:55 smithi151 bash[71095]: ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-y 2024-06-03T03:29:55.835 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@mgr.y.service' 2024-06-03T03:29:55.868 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T03:29:55.869 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-06-03T03:29:55.869 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-06-03T03:29:55.869 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@mgr.z 2024-06-03T03:29:56.220 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Jun 03 03:29:55 smithi183 systemd[1]: Stopping Ceph mgr.z for a0e8473a-2156-11ef-bc9b-c7b262605968... 2024-06-03T03:29:56.481 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Jun 03 03:29:56 smithi183 podman[71693]: 2024-06-03 03:29:56.220517991 +0000 UTC m=+0.159649630 container died 275464c5a73749c329419ea95dff4897704b80b3175c39fae5c47a83490f765e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-z, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-06-03T03:29:56.860 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Jun 03 03:29:56 smithi183 podman[71693]: 2024-06-03 03:29:56.659299448 +0000 UTC m=+0.598431084 container cleanup 275464c5a73749c329419ea95dff4897704b80b3175c39fae5c47a83490f765e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-z, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-03T03:29:56.861 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Jun 03 03:29:56 smithi183 bash[71693]: ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mgr-z 2024-06-03T03:29:57.085 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@mgr.z.service' 2024-06-03T03:29:57.118 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T03:29:57.118 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-06-03T03:29:57.118 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-06-03T03:29:57.119 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@osd.0 2024-06-03T03:29:57.696 INFO:journalctl@ceph.osd.0.smithi117.stdout:Jun 03 03:29:57 smithi117 systemd[1]: Stopping Ceph osd.0 for a0e8473a-2156-11ef-bc9b-c7b262605968... 2024-06-03T03:29:58.197 INFO:journalctl@ceph.osd.0.smithi117.stdout:Jun 03 03:29:57 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-0[44092]: 2024-06-03T03:29:57.749+0000 7f759b200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T03:29:58.197 INFO:journalctl@ceph.osd.0.smithi117.stdout:Jun 03 03:29:57 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-0[44092]: 2024-06-03T03:29:57.749+0000 7f759b200700 -1 osd.0 103 *** Got signal Terminated *** 2024-06-03T03:29:58.197 INFO:journalctl@ceph.osd.0.smithi117.stdout:Jun 03 03:29:57 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-0[44092]: 2024-06-03T03:29:57.749+0000 7f759b200700 -1 osd.0 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-03T03:30:03.646 INFO:journalctl@ceph.osd.0.smithi117.stdout:Jun 03 03:30:03 smithi117 podman[98219]: 2024-06-03 03:30:03.246463306 +0000 UTC m=+5.694823644 container died 242d07237290038d73c1c7c0074891fdbb010fc7a4d8f0fb085b69376d88cbc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-3d89acc, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-06-03T03:30:03.904 INFO:journalctl@ceph.osd.0.smithi117.stdout:Jun 03 03:30:03 smithi117 podman[98219]: 2024-06-03 03:30:03.868863359 +0000 UTC m=+6.317223741 container cleanup 242d07237290038d73c1c7c0074891fdbb010fc7a4d8f0fb085b69376d88cbc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-0, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-06-03T03:30:03.904 INFO:journalctl@ceph.osd.0.smithi117.stdout:Jun 03 03:30:03 smithi117 bash[98219]: ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-0 2024-06-03T03:30:04.446 INFO:journalctl@ceph.osd.0.smithi117.stdout:Jun 03 03:30:04 smithi117 podman[98239]: 2024-06-03 03:30:04.187277117 +0000 UTC m=+0.926417531 container remove 242d07237290038d73c1c7c0074891fdbb010fc7a4d8f0fb085b69376d88cbc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2) 2024-06-03T03:30:05.196 INFO:journalctl@ceph.osd.0.smithi117.stdout:Jun 03 03:30:04 smithi117 podman[98326]: 2024-06-03 03:30:04.843663231 +0000 UTC m=+0.031652194 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T03:30:05.197 INFO:journalctl@ceph.osd.0.smithi117.stdout:Jun 03 03:30:04 smithi117 podman[98326]: 2024-06-03 03:30:04.996849545 +0000 UTC m=+0.184838562 container create ddbefc9ce52bf6fd04d3fe15a71eb4dc7ecf1ff8173e0b901c691b21eeff9f9e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-06-03T03:30:05.697 INFO:journalctl@ceph.osd.0.smithi117.stdout:Jun 03 03:30:05 smithi117 podman[98326]: 2024-06-03 03:30:05.309169213 +0000 UTC m=+0.497158219 container init ddbefc9ce52bf6fd04d3fe15a71eb4dc7ecf1ff8173e0b901c691b21eeff9f9e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-0-deactivate, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-06-03T03:30:05.697 INFO:journalctl@ceph.osd.0.smithi117.stdout:Jun 03 03:30:05 smithi117 podman[98326]: 2024-06-03 03:30:05.320360519 +0000 UTC m=+0.508349525 container start ddbefc9ce52bf6fd04d3fe15a71eb4dc7ecf1ff8173e0b901c691b21eeff9f9e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-06-03T03:30:05.697 INFO:journalctl@ceph.osd.0.smithi117.stdout:Jun 03 03:30:05 smithi117 podman[98326]: 2024-06-03 03:30:05.424595386 +0000 UTC m=+0.612584372 container attach ddbefc9ce52bf6fd04d3fe15a71eb4dc7ecf1ff8173e0b901c691b21eeff9f9e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-06-03T03:30:06.183 INFO:journalctl@ceph.osd.0.smithi117.stdout:Jun 03 03:30:05 smithi117 podman[98352]: 2024-06-03 03:30:05.828294707 +0000 UTC m=+0.033110063 container died ddbefc9ce52bf6fd04d3fe15a71eb4dc7ecf1ff8173e0b901c691b21eeff9f9e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.build-date=20240528) 2024-06-03T03:30:06.776 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@osd.0.service' 2024-06-03T03:30:06.893 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T03:30:06.894 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-06-03T03:30:06.894 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-06-03T03:30:06.894 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@osd.1 2024-06-03T03:30:07.208 INFO:journalctl@ceph.osd.1.smithi151.stdout:Jun 03 03:30:06 smithi151 systemd[1]: Stopping Ceph osd.1 for a0e8473a-2156-11ef-bc9b-c7b262605968... 2024-06-03T03:30:07.479 INFO:journalctl@ceph.osd.1.smithi151.stdout:Jun 03 03:30:07 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-1[41582]: 2024-06-03T03:30:07.207+0000 7f804846a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T03:30:07.479 INFO:journalctl@ceph.osd.1.smithi151.stdout:Jun 03 03:30:07 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-1[41582]: 2024-06-03T03:30:07.207+0000 7f804846a700 -1 osd.1 103 *** Got signal Terminated *** 2024-06-03T03:30:07.479 INFO:journalctl@ceph.osd.1.smithi151.stdout:Jun 03 03:30:07 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-1[41582]: 2024-06-03T03:30:07.207+0000 7f804846a700 -1 osd.1 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-03T03:30:13.027 INFO:journalctl@ceph.osd.1.smithi151.stdout:Jun 03 03:30:12 smithi151 podman[71219]: 2024-06-03 03:30:12.731171504 +0000 UTC m=+5.641537529 container died ac8f6d836aee166a6e937e246a902610a827ea1e628d59c838259e234706bd09 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-1, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-06-03T03:30:13.319 INFO:journalctl@ceph.osd.1.smithi151.stdout:Jun 03 03:30:13 smithi151 podman[71219]: 2024-06-03 03:30:13.117654563 +0000 UTC m=+6.028020587 container cleanup ac8f6d836aee166a6e937e246a902610a827ea1e628d59c838259e234706bd09 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-1, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-03T03:30:13.319 INFO:journalctl@ceph.osd.1.smithi151.stdout:Jun 03 03:30:13 smithi151 bash[71219]: ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-1 2024-06-03T03:30:13.656 INFO:journalctl@ceph.osd.1.smithi151.stdout:Jun 03 03:30:13 smithi151 podman[71247]: 2024-06-03 03:30:13.319323619 +0000 UTC m=+0.580114833 container remove ac8f6d836aee166a6e937e246a902610a827ea1e628d59c838259e234706bd09 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-03T03:30:14.048 INFO:journalctl@ceph.osd.1.smithi151.stdout:Jun 03 03:30:13 smithi151 podman[71339]: 2024-06-03 03:30:13.646946275 +0000 UTC m=+0.017315290 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T03:30:14.049 INFO:journalctl@ceph.osd.1.smithi151.stdout:Jun 03 03:30:13 smithi151 podman[71339]: 2024-06-03 03:30:13.762190046 +0000 UTC m=+0.132559055 container create 37b78f06e6c3e9bbd0dfb4c2681f885d728db5b8c4888d89796f3e1c5aa15dd7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-06-03T03:30:14.049 INFO:journalctl@ceph.osd.1.smithi151.stdout:Jun 03 03:30:14 smithi151 podman[71339]: 2024-06-03 03:30:14.049032763 +0000 UTC m=+0.419401779 container init 37b78f06e6c3e9bbd0dfb4c2681f885d728db5b8c4888d89796f3e1c5aa15dd7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-06-03T03:30:14.349 INFO:journalctl@ceph.osd.1.smithi151.stdout:Jun 03 03:30:14 smithi151 podman[71339]: 2024-06-03 03:30:14.052240627 +0000 UTC m=+0.422609651 container start 37b78f06e6c3e9bbd0dfb4c2681f885d728db5b8c4888d89796f3e1c5aa15dd7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, ceph=True, org.label-schema.vendor=CentOS) 2024-06-03T03:30:14.349 INFO:journalctl@ceph.osd.1.smithi151.stdout:Jun 03 03:30:14 smithi151 podman[71339]: 2024-06-03 03:30:14.149293976 +0000 UTC m=+0.519663005 container attach 37b78f06e6c3e9bbd0dfb4c2681f885d728db5b8c4888d89796f3e1c5aa15dd7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-03T03:30:14.656 INFO:journalctl@ceph.osd.1.smithi151.stdout:Jun 03 03:30:14 smithi151 podman[71339]: 2024-06-03 03:30:14.349425486 +0000 UTC m=+0.719794505 container died 37b78f06e6c3e9bbd0dfb4c2681f885d728db5b8c4888d89796f3e1c5aa15dd7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-03T03:30:15.109 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@osd.1.service' 2024-06-03T03:30:15.150 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T03:30:15.151 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-06-03T03:30:15.151 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-06-03T03:30:15.151 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@osd.2 2024-06-03T03:30:15.446 INFO:journalctl@ceph.osd.2.smithi183.stdout:Jun 03 03:30:15 smithi183 systemd[1]: Stopping Ceph osd.2 for a0e8473a-2156-11ef-bc9b-c7b262605968... 2024-06-03T03:30:15.706 INFO:journalctl@ceph.osd.2.smithi183.stdout:Jun 03 03:30:15 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-2[42419]: 2024-06-03T03:30:15.545+0000 7ff958101700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T03:30:15.707 INFO:journalctl@ceph.osd.2.smithi183.stdout:Jun 03 03:30:15 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-2[42419]: 2024-06-03T03:30:15.545+0000 7ff958101700 -1 osd.2 103 *** Got signal Terminated *** 2024-06-03T03:30:15.707 INFO:journalctl@ceph.osd.2.smithi183.stdout:Jun 03 03:30:15 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-2[42419]: 2024-06-03T03:30:15.545+0000 7ff958101700 -1 osd.2 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-03T03:30:21.232 INFO:journalctl@ceph.osd.2.smithi183.stdout:Jun 03 03:30:20 smithi183 podman[71820]: 2024-06-03 03:30:20.925919838 +0000 UTC m=+5.584642297 container died d6f5c0d7780d76a93e365d7efe1db4fd4c483b18a969cb899251f53f63e7f6f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-06-03T03:30:21.589 INFO:journalctl@ceph.osd.2.smithi183.stdout:Jun 03 03:30:21 smithi183 podman[71820]: 2024-06-03 03:30:21.362883443 +0000 UTC m=+6.021605893 container cleanup d6f5c0d7780d76a93e365d7efe1db4fd4c483b18a969cb899251f53f63e7f6f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-2, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, io.buildah.version=1.33.5) 2024-06-03T03:30:21.589 INFO:journalctl@ceph.osd.2.smithi183.stdout:Jun 03 03:30:21 smithi183 bash[71820]: ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-2 2024-06-03T03:30:21.981 INFO:journalctl@ceph.osd.2.smithi183.stdout:Jun 03 03:30:21 smithi183 podman[71847]: 2024-06-03 03:30:21.589233039 +0000 UTC m=+0.657754469 container remove d6f5c0d7780d76a93e365d7efe1db4fd4c483b18a969cb899251f53f63e7f6f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-2, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=) 2024-06-03T03:30:22.267 INFO:journalctl@ceph.osd.2.smithi183.stdout:Jun 03 03:30:21 smithi183 podman[71937]: 2024-06-03 03:30:21.896938988 +0000 UTC m=+0.017493390 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T03:30:22.267 INFO:journalctl@ceph.osd.2.smithi183.stdout:Jun 03 03:30:22 smithi183 podman[71937]: 2024-06-03 03:30:22.023754502 +0000 UTC m=+0.144308904 container create 8157b571c39674047bb667b55649baafdd9202d5b26506ad641749c68696940a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux ) 2024-06-03T03:30:22.566 INFO:journalctl@ceph.osd.2.smithi183.stdout:Jun 03 03:30:22 smithi183 podman[71937]: 2024-06-03 03:30:22.267927208 +0000 UTC m=+0.388481610 container init 8157b571c39674047bb667b55649baafdd9202d5b26506ad641749c68696940a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5) 2024-06-03T03:30:22.566 INFO:journalctl@ceph.osd.2.smithi183.stdout:Jun 03 03:30:22 smithi183 podman[71937]: 2024-06-03 03:30:22.271063737 +0000 UTC m=+0.391618145 container start 8157b571c39674047bb667b55649baafdd9202d5b26506ad641749c68696940a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS) 2024-06-03T03:30:22.566 INFO:journalctl@ceph.osd.2.smithi183.stdout:Jun 03 03:30:22 smithi183 podman[71937]: 2024-06-03 03:30:22.368176014 +0000 UTC m=+0.488730418 container attach 8157b571c39674047bb667b55649baafdd9202d5b26506ad641749c68696940a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-3d89acc) 2024-06-03T03:30:22.921 INFO:journalctl@ceph.osd.2.smithi183.stdout:Jun 03 03:30:22 smithi183 podman[71937]: 2024-06-03 03:30:22.566239703 +0000 UTC m=+0.686794106 container died 8157b571c39674047bb667b55649baafdd9202d5b26506ad641749c68696940a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-06-03T03:30:23.238 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@osd.2.service' 2024-06-03T03:30:23.279 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T03:30:23.279 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-06-03T03:30:23.279 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-06-03T03:30:23.279 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@osd.3 2024-06-03T03:30:23.696 INFO:journalctl@ceph.osd.3.smithi117.stdout:Jun 03 03:30:23 smithi117 systemd[1]: Stopping Ceph osd.3 for a0e8473a-2156-11ef-bc9b-c7b262605968... 2024-06-03T03:30:24.197 INFO:journalctl@ceph.osd.3.smithi117.stdout:Jun 03 03:30:23 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-3[51024]: 2024-06-03T03:30:23.862+0000 7f455f800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T03:30:24.197 INFO:journalctl@ceph.osd.3.smithi117.stdout:Jun 03 03:30:23 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-3[51024]: 2024-06-03T03:30:23.862+0000 7f455f800700 -1 osd.3 103 *** Got signal Terminated *** 2024-06-03T03:30:24.197 INFO:journalctl@ceph.osd.3.smithi117.stdout:Jun 03 03:30:23 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-3[51024]: 2024-06-03T03:30:23.862+0000 7f455f800700 -1 osd.3 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-03T03:30:25.900 INFO:journalctl@ceph.osd.5.smithi183.stdout:Jun 03 03:30:25 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-5[49050]: 2024-06-03T03:30:25.597+0000 7f7803952700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:00.624352+0000 front 2024-06-03T03:30:00.624587+0000 (oldest deadline 2024-06-03T03:30:24.724295+0000) 2024-06-03T03:30:26.196 INFO:journalctl@ceph.osd.3.smithi117.stdout:Jun 03 03:30:25 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-3[51024]: 2024-06-03T03:30:25.742+0000 7f455b000700 -1 osd.3 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:01.702708+0000 front 2024-06-03T03:30:01.702905+0000 (oldest deadline 2024-06-03T03:30:25.202498+0000) 2024-06-03T03:30:26.231 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:30:25 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:30:25.900+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:01.379911+0000 front 2024-06-03T03:30:01.380025+0000 (oldest deadline 2024-06-03T03:30:25.479720+0000) 2024-06-03T03:30:26.879 INFO:journalctl@ceph.osd.5.smithi183.stdout:Jun 03 03:30:26 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-5[49050]: 2024-06-03T03:30:26.554+0000 7f7803952700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:00.624352+0000 front 2024-06-03T03:30:00.624587+0000 (oldest deadline 2024-06-03T03:30:24.724295+0000) 2024-06-03T03:30:26.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:26 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:26.409+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:02.742111+0000 front 2024-06-03T03:30:02.742070+0000 (oldest deadline 2024-06-03T03:30:26.241792+0000) 2024-06-03T03:30:27.196 INFO:journalctl@ceph.osd.3.smithi117.stdout:Jun 03 03:30:26 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-3[51024]: 2024-06-03T03:30:26.776+0000 7f455b000700 -1 osd.3 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:01.702708+0000 front 2024-06-03T03:30:01.702905+0000 (oldest deadline 2024-06-03T03:30:25.202498+0000) 2024-06-03T03:30:27.231 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:30:26 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:30:26.879+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:01.379911+0000 front 2024-06-03T03:30:01.380025+0000 (oldest deadline 2024-06-03T03:30:25.479720+0000) 2024-06-03T03:30:27.656 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:27 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:27.393+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:02.742111+0000 front 2024-06-03T03:30:02.742070+0000 (oldest deadline 2024-06-03T03:30:26.241792+0000) 2024-06-03T03:30:27.923 INFO:journalctl@ceph.osd.5.smithi183.stdout:Jun 03 03:30:27 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-5[49050]: 2024-06-03T03:30:27.601+0000 7f7803952700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:00.624352+0000 front 2024-06-03T03:30:00.624587+0000 (oldest deadline 2024-06-03T03:30:24.724295+0000) 2024-06-03T03:30:28.196 INFO:journalctl@ceph.osd.3.smithi117.stdout:Jun 03 03:30:27 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-3[51024]: 2024-06-03T03:30:27.772+0000 7f455b000700 -1 osd.3 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:01.702708+0000 front 2024-06-03T03:30:01.702905+0000 (oldest deadline 2024-06-03T03:30:25.202498+0000) 2024-06-03T03:30:28.232 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:30:27 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:30:27.923+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:01.379911+0000 front 2024-06-03T03:30:01.380025+0000 (oldest deadline 2024-06-03T03:30:25.479720+0000) 2024-06-03T03:30:28.656 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:28 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:28.399+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:02.742111+0000 front 2024-06-03T03:30:02.742070+0000 (oldest deadline 2024-06-03T03:30:26.241792+0000) 2024-06-03T03:30:28.956 INFO:journalctl@ceph.osd.5.smithi183.stdout:Jun 03 03:30:28 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-5[49050]: 2024-06-03T03:30:28.625+0000 7f7803952700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:00.624352+0000 front 2024-06-03T03:30:00.624587+0000 (oldest deadline 2024-06-03T03:30:24.724295+0000) 2024-06-03T03:30:29.196 INFO:journalctl@ceph.osd.3.smithi117.stdout:Jun 03 03:30:28 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-3[51024]: 2024-06-03T03:30:28.819+0000 7f455b000700 -1 osd.3 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:01.702708+0000 front 2024-06-03T03:30:01.702905+0000 (oldest deadline 2024-06-03T03:30:25.202498+0000) 2024-06-03T03:30:29.231 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:30:28 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:30:28.956+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:01.379911+0000 front 2024-06-03T03:30:01.380025+0000 (oldest deadline 2024-06-03T03:30:25.479720+0000) 2024-06-03T03:30:29.663 INFO:journalctl@ceph.osd.3.smithi117.stdout:Jun 03 03:30:29 smithi117 podman[98430]: 2024-06-03 03:30:29.333207097 +0000 UTC m=+5.606059566 container died c78df18b338a1099f51d1a9c9b7c417e8a744bf87f4144e85e179ab31ae8dd39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-3, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-06-03T03:30:29.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:29 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:29.434+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:02.742111+0000 front 2024-06-03T03:30:02.742070+0000 (oldest deadline 2024-06-03T03:30:26.241792+0000) 2024-06-03T03:30:29.946 INFO:journalctl@ceph.osd.3.smithi117.stdout:Jun 03 03:30:29 smithi117 podman[98430]: 2024-06-03 03:30:29.863081376 +0000 UTC m=+6.135933885 container cleanup c78df18b338a1099f51d1a9c9b7c417e8a744bf87f4144e85e179ab31ae8dd39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-3, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-06-03T03:30:29.947 INFO:journalctl@ceph.osd.3.smithi117.stdout:Jun 03 03:30:29 smithi117 bash[98430]: ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-3 2024-06-03T03:30:29.978 INFO:journalctl@ceph.osd.5.smithi183.stdout:Jun 03 03:30:29 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-5[49050]: 2024-06-03T03:30:29.665+0000 7f7803952700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:00.624352+0000 front 2024-06-03T03:30:00.624587+0000 (oldest deadline 2024-06-03T03:30:24.724295+0000) 2024-06-03T03:30:30.231 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:30:29 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:30:29.978+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:01.379911+0000 front 2024-06-03T03:30:01.380025+0000 (oldest deadline 2024-06-03T03:30:25.479720+0000) 2024-06-03T03:30:30.446 INFO:journalctl@ceph.osd.3.smithi117.stdout:Jun 03 03:30:30 smithi117 podman[98450]: 2024-06-03 03:30:30.172904318 +0000 UTC m=+0.827961666 container remove c78df18b338a1099f51d1a9c9b7c417e8a744bf87f4144e85e179ab31ae8dd39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-3, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-06-03T03:30:30.656 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:30 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:30.386+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:02.742111+0000 front 2024-06-03T03:30:02.742070+0000 (oldest deadline 2024-06-03T03:30:26.241792+0000) 2024-06-03T03:30:30.981 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:30:30 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:30:30.932+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:01.379911+0000 front 2024-06-03T03:30:01.380025+0000 (oldest deadline 2024-06-03T03:30:25.479720+0000) 2024-06-03T03:30:30.982 INFO:journalctl@ceph.osd.5.smithi183.stdout:Jun 03 03:30:30 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-5[49050]: 2024-06-03T03:30:30.700+0000 7f7803952700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:00.624352+0000 front 2024-06-03T03:30:00.624587+0000 (oldest deadline 2024-06-03T03:30:24.724295+0000) 2024-06-03T03:30:31.218 INFO:journalctl@ceph.osd.3.smithi117.stdout:Jun 03 03:30:30 smithi117 podman[98541]: 2024-06-03 03:30:30.869140612 +0000 UTC m=+0.032345513 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T03:30:31.219 INFO:journalctl@ceph.osd.3.smithi117.stdout:Jun 03 03:30:31 smithi117 podman[98541]: 2024-06-03 03:30:31.023872416 +0000 UTC m=+0.187077353 container create 30a51f1ab97612f3c66d41bcac5f660cfd6a3da0937381202466cf87f1236bf3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD) 2024-06-03T03:30:31.656 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:31 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:31.368+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:02.742111+0000 front 2024-06-03T03:30:02.742070+0000 (oldest deadline 2024-06-03T03:30:26.241792+0000) 2024-06-03T03:30:31.698 INFO:journalctl@ceph.osd.3.smithi117.stdout:Jun 03 03:30:31 smithi117 podman[98541]: 2024-06-03 03:30:31.352727637 +0000 UTC m=+0.515932654 container init 30a51f1ab97612f3c66d41bcac5f660cfd6a3da0937381202466cf87f1236bf3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-3-deactivate, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-06-03T03:30:31.698 INFO:journalctl@ceph.osd.3.smithi117.stdout:Jun 03 03:30:31 smithi117 podman[98541]: 2024-06-03 03:30:31.364738887 +0000 UTC m=+0.527943858 container start 30a51f1ab97612f3c66d41bcac5f660cfd6a3da0937381202466cf87f1236bf3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-03T03:30:31.698 INFO:journalctl@ceph.osd.3.smithi117.stdout:Jun 03 03:30:31 smithi117 podman[98541]: 2024-06-03 03:30:31.459957542 +0000 UTC m=+0.623162489 container attach 30a51f1ab97612f3c66d41bcac5f660cfd6a3da0937381202466cf87f1236bf3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-03T03:30:31.981 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:30:31 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:30:31.951+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:01.379911+0000 front 2024-06-03T03:30:01.380025+0000 (oldest deadline 2024-06-03T03:30:25.479720+0000) 2024-06-03T03:30:31.982 INFO:journalctl@ceph.osd.5.smithi183.stdout:Jun 03 03:30:31 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-5[49050]: 2024-06-03T03:30:31.722+0000 7f7803952700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:00.624352+0000 front 2024-06-03T03:30:00.624587+0000 (oldest deadline 2024-06-03T03:30:24.724295+0000) 2024-06-03T03:30:32.196 INFO:journalctl@ceph.osd.3.smithi117.stdout:Jun 03 03:30:31 smithi117 podman[98541]: 2024-06-03 03:30:31.814048116 +0000 UTC m=+0.977253088 container died 30a51f1ab97612f3c66d41bcac5f660cfd6a3da0937381202466cf87f1236bf3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0) 2024-06-03T03:30:32.656 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:32 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:32.347+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:02.742111+0000 front 2024-06-03T03:30:02.742070+0000 (oldest deadline 2024-06-03T03:30:26.241792+0000) 2024-06-03T03:30:32.732 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@osd.3.service' 2024-06-03T03:30:32.842 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T03:30:32.842 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-06-03T03:30:32.842 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-06-03T03:30:32.842 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@osd.4 2024-06-03T03:30:33.141 INFO:journalctl@ceph.osd.4.smithi151.stdout:Jun 03 03:30:32 smithi151 systemd[1]: Stopping Ceph osd.4 for a0e8473a-2156-11ef-bc9b-c7b262605968... 2024-06-03T03:30:33.141 INFO:journalctl@ceph.osd.7.smithi151.stdout:Jun 03 03:30:32 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-7[55035]: 2024-06-03T03:30:32.975+0000 7f8e5df2b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:08.069472+0000 front 2024-06-03T03:30:08.069491+0000 (oldest deadline 2024-06-03T03:30:32.769733+0000) 2024-06-03T03:30:33.231 INFO:journalctl@ceph.osd.5.smithi183.stdout:Jun 03 03:30:32 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-5[49050]: 2024-06-03T03:30:32.746+0000 7f7803952700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:00.624352+0000 front 2024-06-03T03:30:00.624587+0000 (oldest deadline 2024-06-03T03:30:24.724295+0000) 2024-06-03T03:30:33.232 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:30:32 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:30:32.915+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:01.379911+0000 front 2024-06-03T03:30:01.380025+0000 (oldest deadline 2024-06-03T03:30:25.479720+0000) 2024-06-03T03:30:33.406 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:33 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:33.362+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:02.742111+0000 front 2024-06-03T03:30:02.742070+0000 (oldest deadline 2024-06-03T03:30:26.241792+0000) 2024-06-03T03:30:33.406 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:33 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:33.362+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.742816+0000 front 2024-06-03T03:30:09.742795+0000 (oldest deadline 2024-06-03T03:30:33.243036+0000) 2024-06-03T03:30:33.406 INFO:journalctl@ceph.osd.4.smithi151.stdout:Jun 03 03:30:33 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-4[48065]: 2024-06-03T03:30:33.140+0000 7fc8abd7b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T03:30:33.407 INFO:journalctl@ceph.osd.4.smithi151.stdout:Jun 03 03:30:33 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-4[48065]: 2024-06-03T03:30:33.140+0000 7fc8abd7b700 -1 osd.4 103 *** Got signal Terminated *** 2024-06-03T03:30:33.407 INFO:journalctl@ceph.osd.4.smithi151.stdout:Jun 03 03:30:33 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-4[48065]: 2024-06-03T03:30:33.140+0000 7fc8abd7b700 -1 osd.4 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-03T03:30:33.727 INFO:journalctl@ceph.osd.5.smithi183.stdout:Jun 03 03:30:33 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-5[49050]: 2024-06-03T03:30:33.726+0000 7f7803952700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:00.624352+0000 front 2024-06-03T03:30:00.624587+0000 (oldest deadline 2024-06-03T03:30:24.724295+0000) 2024-06-03T03:30:33.727 INFO:journalctl@ceph.osd.5.smithi183.stdout:Jun 03 03:30:33 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-5[49050]: 2024-06-03T03:30:33.726+0000 7f7803952700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.425446+0000 front 2024-06-03T03:30:09.425383+0000 (oldest deadline 2024-06-03T03:30:32.925380+0000) 2024-06-03T03:30:33.981 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:30:33 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:30:33.886+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:01.379911+0000 front 2024-06-03T03:30:01.380025+0000 (oldest deadline 2024-06-03T03:30:25.479720+0000) 2024-06-03T03:30:33.982 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:30:33 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:30:33.934+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:06.650666+0000 front 2024-06-03T03:30:06.650530+0000 (oldest deadline 2024-06-03T03:30:33.051199+0000) 2024-06-03T03:30:34.376 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:30:34 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:30:34.121+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.254189+0000 front 2024-06-03T03:30:09.253879+0000 (oldest deadline 2024-06-03T03:30:33.353839+0000) 2024-06-03T03:30:34.392 INFO:journalctl@ceph.osd.7.smithi151.stdout:Jun 03 03:30:33 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-7[55035]: 2024-06-03T03:30:33.962+0000 7f8e5df2b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:08.069472+0000 front 2024-06-03T03:30:08.069491+0000 (oldest deadline 2024-06-03T03:30:32.769733+0000) 2024-06-03T03:30:34.656 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:34 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:34.392+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:02.742111+0000 front 2024-06-03T03:30:02.742070+0000 (oldest deadline 2024-06-03T03:30:26.241792+0000) 2024-06-03T03:30:34.656 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:34 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:34.392+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.742816+0000 front 2024-06-03T03:30:09.742795+0000 (oldest deadline 2024-06-03T03:30:33.243036+0000) 2024-06-03T03:30:34.697 INFO:journalctl@ceph.osd.6.smithi117.stdout:Jun 03 03:30:34 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-6[57519]: 2024-06-03T03:30:34.374+0000 7f50fe600700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:11.044622+0000 front 2024-06-03T03:30:11.044544+0000 (oldest deadline 2024-06-03T03:30:33.944525+0000) 2024-06-03T03:30:34.981 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:30:34 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:30:34.950+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:06.650666+0000 front 2024-06-03T03:30:06.650530+0000 (oldest deadline 2024-06-03T03:30:33.051199+0000) 2024-06-03T03:30:34.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:30:34 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:30:34.872+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:01.379911+0000 front 2024-06-03T03:30:01.380025+0000 (oldest deadline 2024-06-03T03:30:25.479720+0000) 2024-06-03T03:30:34.982 INFO:journalctl@ceph.osd.5.smithi183.stdout:Jun 03 03:30:34 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-5[49050]: 2024-06-03T03:30:34.708+0000 7f7803952700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:00.624352+0000 front 2024-06-03T03:30:00.624587+0000 (oldest deadline 2024-06-03T03:30:24.724295+0000) 2024-06-03T03:30:34.982 INFO:journalctl@ceph.osd.5.smithi183.stdout:Jun 03 03:30:34 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-5[49050]: 2024-06-03T03:30:34.708+0000 7f7803952700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.425446+0000 front 2024-06-03T03:30:09.425383+0000 (oldest deadline 2024-06-03T03:30:32.925380+0000) 2024-06-03T03:30:35.385 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:30:35 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:30:35.102+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.254189+0000 front 2024-06-03T03:30:09.253879+0000 (oldest deadline 2024-06-03T03:30:33.353839+0000) 2024-06-03T03:30:35.406 INFO:journalctl@ceph.osd.7.smithi151.stdout:Jun 03 03:30:35 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-7[55035]: 2024-06-03T03:30:35.008+0000 7f8e5df2b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:08.069472+0000 front 2024-06-03T03:30:08.069491+0000 (oldest deadline 2024-06-03T03:30:32.769733+0000) 2024-06-03T03:30:35.696 INFO:journalctl@ceph.osd.6.smithi117.stdout:Jun 03 03:30:35 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-6[57519]: 2024-06-03T03:30:35.384+0000 7f50fe600700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:11.044622+0000 front 2024-06-03T03:30:11.044544+0000 (oldest deadline 2024-06-03T03:30:33.944525+0000) 2024-06-03T03:30:35.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:35 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:35.408+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:02.742111+0000 front 2024-06-03T03:30:02.742070+0000 (oldest deadline 2024-06-03T03:30:26.241792+0000) 2024-06-03T03:30:35.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:35 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:35.408+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.742816+0000 front 2024-06-03T03:30:09.742795+0000 (oldest deadline 2024-06-03T03:30:33.243036+0000) 2024-06-03T03:30:35.981 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:30:35 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:30:35.907+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:06.650666+0000 front 2024-06-03T03:30:06.650530+0000 (oldest deadline 2024-06-03T03:30:33.051199+0000) 2024-06-03T03:30:35.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:30:35 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:30:35.863+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:01.379911+0000 front 2024-06-03T03:30:01.380025+0000 (oldest deadline 2024-06-03T03:30:25.479720+0000) 2024-06-03T03:30:35.982 INFO:journalctl@ceph.osd.5.smithi183.stdout:Jun 03 03:30:35 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-5[49050]: 2024-06-03T03:30:35.721+0000 7f7803952700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:00.624352+0000 front 2024-06-03T03:30:00.624587+0000 (oldest deadline 2024-06-03T03:30:24.724295+0000) 2024-06-03T03:30:35.982 INFO:journalctl@ceph.osd.5.smithi183.stdout:Jun 03 03:30:35 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-5[49050]: 2024-06-03T03:30:35.721+0000 7f7803952700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.425446+0000 front 2024-06-03T03:30:09.425383+0000 (oldest deadline 2024-06-03T03:30:32.925380+0000) 2024-06-03T03:30:36.406 INFO:journalctl@ceph.osd.7.smithi151.stdout:Jun 03 03:30:36 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-7[55035]: 2024-06-03T03:30:36.017+0000 7f8e5df2b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:08.069472+0000 front 2024-06-03T03:30:08.069491+0000 (oldest deadline 2024-06-03T03:30:32.769733+0000) 2024-06-03T03:30:36.425 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:30:36 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:30:36.066+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.254189+0000 front 2024-06-03T03:30:09.253879+0000 (oldest deadline 2024-06-03T03:30:33.353839+0000) 2024-06-03T03:30:36.696 INFO:journalctl@ceph.osd.6.smithi117.stdout:Jun 03 03:30:36 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-6[57519]: 2024-06-03T03:30:36.424+0000 7f50fe600700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:11.044622+0000 front 2024-06-03T03:30:11.044544+0000 (oldest deadline 2024-06-03T03:30:33.944525+0000) 2024-06-03T03:30:36.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:36 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:36.443+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:02.742111+0000 front 2024-06-03T03:30:02.742070+0000 (oldest deadline 2024-06-03T03:30:26.241792+0000) 2024-06-03T03:30:36.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:36 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:36.443+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.742816+0000 front 2024-06-03T03:30:09.742795+0000 (oldest deadline 2024-06-03T03:30:33.243036+0000) 2024-06-03T03:30:36.957 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:30:36 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:30:36.852+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:01.379911+0000 front 2024-06-03T03:30:01.380025+0000 (oldest deadline 2024-06-03T03:30:25.479720+0000) 2024-06-03T03:30:36.957 INFO:journalctl@ceph.osd.5.smithi183.stdout:Jun 03 03:30:36 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-5[49050]: 2024-06-03T03:30:36.691+0000 7f7803952700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:00.624352+0000 front 2024-06-03T03:30:00.624587+0000 (oldest deadline 2024-06-03T03:30:24.724295+0000) 2024-06-03T03:30:36.957 INFO:journalctl@ceph.osd.5.smithi183.stdout:Jun 03 03:30:36 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-5[49050]: 2024-06-03T03:30:36.691+0000 7f7803952700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.425446+0000 front 2024-06-03T03:30:09.425383+0000 (oldest deadline 2024-06-03T03:30:32.925380+0000) 2024-06-03T03:30:37.234 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:30:36 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:30:36.956+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:06.650666+0000 front 2024-06-03T03:30:06.650530+0000 (oldest deadline 2024-06-03T03:30:33.051199+0000) 2024-06-03T03:30:37.376 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:30:37 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:30:37.023+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.254189+0000 front 2024-06-03T03:30:09.253879+0000 (oldest deadline 2024-06-03T03:30:33.353839+0000) 2024-06-03T03:30:37.406 INFO:journalctl@ceph.osd.7.smithi151.stdout:Jun 03 03:30:37 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-7[55035]: 2024-06-03T03:30:37.019+0000 7f8e5df2b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:08.069472+0000 front 2024-06-03T03:30:08.069491+0000 (oldest deadline 2024-06-03T03:30:32.769733+0000) 2024-06-03T03:30:37.696 INFO:journalctl@ceph.osd.6.smithi117.stdout:Jun 03 03:30:37 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-6[57519]: 2024-06-03T03:30:37.376+0000 7f50fe600700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:11.044622+0000 front 2024-06-03T03:30:11.044544+0000 (oldest deadline 2024-06-03T03:30:33.944525+0000) 2024-06-03T03:30:37.906 INFO:journalctl@ceph.osd.4.smithi151.stdout:Jun 03 03:30:37 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-4[48065]: 2024-06-03T03:30:37.602+0000 7fc8a836f700 -1 osd.4 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:08.389078+0000 front 2024-06-03T03:30:08.389098+0000 (oldest deadline 2024-06-03T03:30:36.589785+0000) 2024-06-03T03:30:37.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:37 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:37.485+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:02.742111+0000 front 2024-06-03T03:30:02.742070+0000 (oldest deadline 2024-06-03T03:30:26.241792+0000) 2024-06-03T03:30:37.907 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:37 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:37.485+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.742816+0000 front 2024-06-03T03:30:09.742795+0000 (oldest deadline 2024-06-03T03:30:33.243036+0000) 2024-06-03T03:30:37.970 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:30:37 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:30:37.822+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:01.379911+0000 front 2024-06-03T03:30:01.380025+0000 (oldest deadline 2024-06-03T03:30:25.479720+0000) 2024-06-03T03:30:37.970 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:30:37 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:30:37.822+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:11.881579+0000 front 2024-06-03T03:30:11.881609+0000 (oldest deadline 2024-06-03T03:30:37.781638+0000) 2024-06-03T03:30:37.970 INFO:journalctl@ceph.osd.5.smithi183.stdout:Jun 03 03:30:37 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-5[49050]: 2024-06-03T03:30:37.652+0000 7f7803952700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:00.624352+0000 front 2024-06-03T03:30:00.624587+0000 (oldest deadline 2024-06-03T03:30:24.724295+0000) 2024-06-03T03:30:37.971 INFO:journalctl@ceph.osd.5.smithi183.stdout:Jun 03 03:30:37 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-5[49050]: 2024-06-03T03:30:37.652+0000 7f7803952700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.425446+0000 front 2024-06-03T03:30:09.425383+0000 (oldest deadline 2024-06-03T03:30:32.925380+0000) 2024-06-03T03:30:38.231 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:30:37 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:30:37.969+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:06.650666+0000 front 2024-06-03T03:30:06.650530+0000 (oldest deadline 2024-06-03T03:30:33.051199+0000) 2024-06-03T03:30:38.406 INFO:journalctl@ceph.osd.7.smithi151.stdout:Jun 03 03:30:38 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-7[55035]: 2024-06-03T03:30:38.061+0000 7f8e5df2b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:08.069472+0000 front 2024-06-03T03:30:08.069491+0000 (oldest deadline 2024-06-03T03:30:32.769733+0000) 2024-06-03T03:30:38.423 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:30:38 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:30:38.050+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.254189+0000 front 2024-06-03T03:30:09.253879+0000 (oldest deadline 2024-06-03T03:30:33.353839+0000) 2024-06-03T03:30:38.696 INFO:journalctl@ceph.osd.6.smithi117.stdout:Jun 03 03:30:38 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-6[57519]: 2024-06-03T03:30:38.422+0000 7f50fe600700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:11.044622+0000 front 2024-06-03T03:30:11.044544+0000 (oldest deadline 2024-06-03T03:30:33.944525+0000) 2024-06-03T03:30:38.801 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:38 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:38.514+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:02.742111+0000 front 2024-06-03T03:30:02.742070+0000 (oldest deadline 2024-06-03T03:30:26.241792+0000) 2024-06-03T03:30:38.801 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:38 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:38.514+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.742816+0000 front 2024-06-03T03:30:09.742795+0000 (oldest deadline 2024-06-03T03:30:33.243036+0000) 2024-06-03T03:30:38.802 INFO:journalctl@ceph.osd.4.smithi151.stdout:Jun 03 03:30:38 smithi151 podman[71444]: 2024-06-03 03:30:38.506386625 +0000 UTC m=+5.463641036 container died dc4177a89dc4b15cf98d4f9f79a4154ba72aa3f314163fa6871b5abe6382b084 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-4, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-06-03T03:30:38.967 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:30:38 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:30:38.863+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:01.379911+0000 front 2024-06-03T03:30:01.380025+0000 (oldest deadline 2024-06-03T03:30:25.479720+0000) 2024-06-03T03:30:38.967 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:30:38 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:30:38.863+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:11.881579+0000 front 2024-06-03T03:30:11.881609+0000 (oldest deadline 2024-06-03T03:30:37.781638+0000) 2024-06-03T03:30:38.967 INFO:journalctl@ceph.osd.5.smithi183.stdout:Jun 03 03:30:38 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-5[49050]: 2024-06-03T03:30:38.659+0000 7f7803952700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:00.624352+0000 front 2024-06-03T03:30:00.624587+0000 (oldest deadline 2024-06-03T03:30:24.724295+0000) 2024-06-03T03:30:38.967 INFO:journalctl@ceph.osd.5.smithi183.stdout:Jun 03 03:30:38 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-5[49050]: 2024-06-03T03:30:38.659+0000 7f7803952700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.425446+0000 front 2024-06-03T03:30:09.425383+0000 (oldest deadline 2024-06-03T03:30:32.925380+0000) 2024-06-03T03:30:39.078 INFO:journalctl@ceph.osd.4.smithi151.stdout:Jun 03 03:30:38 smithi151 podman[71444]: 2024-06-03 03:30:38.900540414 +0000 UTC m=+5.857794822 container cleanup dc4177a89dc4b15cf98d4f9f79a4154ba72aa3f314163fa6871b5abe6382b084 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-4, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-06-03T03:30:39.078 INFO:journalctl@ceph.osd.4.smithi151.stdout:Jun 03 03:30:38 smithi151 bash[71444]: ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-4 2024-06-03T03:30:39.231 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:30:38 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:30:38.966+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:06.650666+0000 front 2024-06-03T03:30:06.650530+0000 (oldest deadline 2024-06-03T03:30:33.051199+0000) 2024-06-03T03:30:39.400 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:30:39 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:30:39.020+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.254189+0000 front 2024-06-03T03:30:09.253879+0000 (oldest deadline 2024-06-03T03:30:33.353839+0000) 2024-06-03T03:30:39.406 INFO:journalctl@ceph.osd.7.smithi151.stdout:Jun 03 03:30:39 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-7[55035]: 2024-06-03T03:30:39.077+0000 7f8e5df2b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:08.069472+0000 front 2024-06-03T03:30:08.069491+0000 (oldest deadline 2024-06-03T03:30:32.769733+0000) 2024-06-03T03:30:39.406 INFO:journalctl@ceph.osd.4.smithi151.stdout:Jun 03 03:30:39 smithi151 podman[71461]: 2024-06-03 03:30:39.17798245 +0000 UTC m=+0.664666667 container remove dc4177a89dc4b15cf98d4f9f79a4154ba72aa3f314163fa6871b5abe6382b084 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-06-03T03:30:39.696 INFO:journalctl@ceph.osd.6.smithi117.stdout:Jun 03 03:30:39 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-6[57519]: 2024-06-03T03:30:39.400+0000 7f50fe600700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:11.044622+0000 front 2024-06-03T03:30:11.044544+0000 (oldest deadline 2024-06-03T03:30:33.944525+0000) 2024-06-03T03:30:39.809 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:39 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:39.563+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:02.742111+0000 front 2024-06-03T03:30:02.742070+0000 (oldest deadline 2024-06-03T03:30:26.241792+0000) 2024-06-03T03:30:39.809 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:39 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:39.563+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.742816+0000 front 2024-06-03T03:30:09.742795+0000 (oldest deadline 2024-06-03T03:30:33.243036+0000) 2024-06-03T03:30:39.810 INFO:journalctl@ceph.osd.4.smithi151.stdout:Jun 03 03:30:39 smithi151 podman[71551]: 2024-06-03 03:30:39.426528438 +0000 UTC m=+0.018167538 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T03:30:39.810 INFO:journalctl@ceph.osd.4.smithi151.stdout:Jun 03 03:30:39 smithi151 podman[71551]: 2024-06-03 03:30:39.614394571 +0000 UTC m=+0.206033668 container create 052f83638c2c274c7e3ce4ef3ca41fe93c6690ec3e2dd8bb06304097b8d021c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-4-deactivate, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS) 2024-06-03T03:30:39.981 INFO:journalctl@ceph.osd.5.smithi183.stdout:Jun 03 03:30:39 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-5[49050]: 2024-06-03T03:30:39.659+0000 7f7803952700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:00.624352+0000 front 2024-06-03T03:30:00.624587+0000 (oldest deadline 2024-06-03T03:30:24.724295+0000) 2024-06-03T03:30:39.982 INFO:journalctl@ceph.osd.5.smithi183.stdout:Jun 03 03:30:39 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-5[49050]: 2024-06-03T03:30:39.659+0000 7f7803952700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.425446+0000 front 2024-06-03T03:30:09.425383+0000 (oldest deadline 2024-06-03T03:30:32.925380+0000) 2024-06-03T03:30:39.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:30:39 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:30:39.903+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:01.379911+0000 front 2024-06-03T03:30:01.380025+0000 (oldest deadline 2024-06-03T03:30:25.479720+0000) 2024-06-03T03:30:39.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:30:39 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:30:39.903+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:11.881579+0000 front 2024-06-03T03:30:11.881609+0000 (oldest deadline 2024-06-03T03:30:37.781638+0000) 2024-06-03T03:30:40.065 INFO:journalctl@ceph.osd.4.smithi151.stdout:Jun 03 03:30:39 smithi151 podman[71551]: 2024-06-03 03:30:39.965469311 +0000 UTC m=+0.557108409 container init 052f83638c2c274c7e3ce4ef3ca41fe93c6690ec3e2dd8bb06304097b8d021c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-4-deactivate, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, GIT_CLEAN=True, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-06-03T03:30:40.065 INFO:journalctl@ceph.osd.4.smithi151.stdout:Jun 03 03:30:39 smithi151 podman[71551]: 2024-06-03 03:30:39.968819235 +0000 UTC m=+0.560458334 container start 052f83638c2c274c7e3ce4ef3ca41fe93c6690ec3e2dd8bb06304097b8d021c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=) 2024-06-03T03:30:40.066 INFO:journalctl@ceph.osd.4.smithi151.stdout:Jun 03 03:30:40 smithi151 podman[71551]: 2024-06-03 03:30:40.065684383 +0000 UTC m=+0.657323503 container attach 052f83638c2c274c7e3ce4ef3ca41fe93c6690ec3e2dd8bb06304097b8d021c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS) 2024-06-03T03:30:40.406 INFO:journalctl@ceph.osd.7.smithi151.stdout:Jun 03 03:30:40 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-7[55035]: 2024-06-03T03:30:40.108+0000 7f8e5df2b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:08.069472+0000 front 2024-06-03T03:30:08.069491+0000 (oldest deadline 2024-06-03T03:30:32.769733+0000) 2024-06-03T03:30:40.407 INFO:journalctl@ceph.osd.4.smithi151.stdout:Jun 03 03:30:40 smithi151 podman[71551]: 2024-06-03 03:30:40.263367577 +0000 UTC m=+0.855006675 container died 052f83638c2c274c7e3ce4ef3ca41fe93c6690ec3e2dd8bb06304097b8d021c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-4-deactivate, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-06-03T03:30:40.420 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:30:39 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:30:39.998+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.254189+0000 front 2024-06-03T03:30:09.253879+0000 (oldest deadline 2024-06-03T03:30:33.353839+0000) 2024-06-03T03:30:40.481 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:30:40 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:30:40.000+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:06.650666+0000 front 2024-06-03T03:30:06.650530+0000 (oldest deadline 2024-06-03T03:30:33.051199+0000) 2024-06-03T03:30:40.696 INFO:journalctl@ceph.osd.6.smithi117.stdout:Jun 03 03:30:40 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-6[57519]: 2024-06-03T03:30:40.419+0000 7f50fe600700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:11.044622+0000 front 2024-06-03T03:30:11.044544+0000 (oldest deadline 2024-06-03T03:30:33.944525+0000) 2024-06-03T03:30:40.879 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:40 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:40.517+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:02.742111+0000 front 2024-06-03T03:30:02.742070+0000 (oldest deadline 2024-06-03T03:30:26.241792+0000) 2024-06-03T03:30:40.879 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:40 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:40.517+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.742816+0000 front 2024-06-03T03:30:09.742795+0000 (oldest deadline 2024-06-03T03:30:33.243036+0000) 2024-06-03T03:30:40.910 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@osd.4.service' 2024-06-03T03:30:40.950 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T03:30:40.951 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-06-03T03:30:40.951 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-06-03T03:30:40.951 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@osd.5 2024-06-03T03:30:40.978 INFO:journalctl@ceph.osd.5.smithi183.stdout:Jun 03 03:30:40 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-5[49050]: 2024-06-03T03:30:40.659+0000 7f7803952700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:00.624352+0000 front 2024-06-03T03:30:00.624587+0000 (oldest deadline 2024-06-03T03:30:24.724295+0000) 2024-06-03T03:30:40.978 INFO:journalctl@ceph.osd.5.smithi183.stdout:Jun 03 03:30:40 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-5[49050]: 2024-06-03T03:30:40.659+0000 7f7803952700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.425446+0000 front 2024-06-03T03:30:09.425383+0000 (oldest deadline 2024-06-03T03:30:32.925380+0000) 2024-06-03T03:30:40.978 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:30:40 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:30:40.876+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:01.379911+0000 front 2024-06-03T03:30:01.380025+0000 (oldest deadline 2024-06-03T03:30:25.479720+0000) 2024-06-03T03:30:40.978 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:30:40 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:30:40.876+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:11.881579+0000 front 2024-06-03T03:30:11.881609+0000 (oldest deadline 2024-06-03T03:30:37.781638+0000) 2024-06-03T03:30:41.231 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:30:40 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:30:40.983+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:06.650666+0000 front 2024-06-03T03:30:06.650530+0000 (oldest deadline 2024-06-03T03:30:33.051199+0000) 2024-06-03T03:30:41.232 INFO:journalctl@ceph.osd.5.smithi183.stdout:Jun 03 03:30:40 smithi183 systemd[1]: Stopping Ceph osd.5 for a0e8473a-2156-11ef-bc9b-c7b262605968... 2024-06-03T03:30:41.401 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:30:40 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:30:40.987+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.254189+0000 front 2024-06-03T03:30:09.253879+0000 (oldest deadline 2024-06-03T03:30:33.353839+0000) 2024-06-03T03:30:41.407 INFO:journalctl@ceph.osd.7.smithi151.stdout:Jun 03 03:30:41 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-7[55035]: 2024-06-03T03:30:41.130+0000 7f8e5df2b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:08.069472+0000 front 2024-06-03T03:30:08.069491+0000 (oldest deadline 2024-06-03T03:30:32.769733+0000) 2024-06-03T03:30:41.651 INFO:journalctl@ceph.osd.5.smithi183.stdout:Jun 03 03:30:41 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-5[49050]: 2024-06-03T03:30:41.281+0000 7f780b366700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T03:30:41.651 INFO:journalctl@ceph.osd.5.smithi183.stdout:Jun 03 03:30:41 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-5[49050]: 2024-06-03T03:30:41.281+0000 7f780b366700 -1 osd.5 103 *** Got signal Terminated *** 2024-06-03T03:30:41.651 INFO:journalctl@ceph.osd.5.smithi183.stdout:Jun 03 03:30:41 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-5[49050]: 2024-06-03T03:30:41.281+0000 7f780b366700 -1 osd.5 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-03T03:30:41.651 INFO:journalctl@ceph.osd.5.smithi183.stdout:Jun 03 03:30:41 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-5[49050]: 2024-06-03T03:30:41.650+0000 7f7803952700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:00.624352+0000 front 2024-06-03T03:30:00.624587+0000 (oldest deadline 2024-06-03T03:30:24.724295+0000) 2024-06-03T03:30:41.651 INFO:journalctl@ceph.osd.5.smithi183.stdout:Jun 03 03:30:41 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-5[49050]: 2024-06-03T03:30:41.650+0000 7f7803952700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.425446+0000 front 2024-06-03T03:30:09.425383+0000 (oldest deadline 2024-06-03T03:30:32.925380+0000) 2024-06-03T03:30:41.696 INFO:journalctl@ceph.osd.6.smithi117.stdout:Jun 03 03:30:41 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-6[57519]: 2024-06-03T03:30:41.400+0000 7f50fe600700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:11.044622+0000 front 2024-06-03T03:30:11.044544+0000 (oldest deadline 2024-06-03T03:30:33.944525+0000) 2024-06-03T03:30:41.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:41 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:41.486+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:02.742111+0000 front 2024-06-03T03:30:02.742070+0000 (oldest deadline 2024-06-03T03:30:26.241792+0000) 2024-06-03T03:30:41.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:41 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:41.486+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.742816+0000 front 2024-06-03T03:30:09.742795+0000 (oldest deadline 2024-06-03T03:30:33.243036+0000) 2024-06-03T03:30:42.231 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:30:41 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:30:41.973+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:06.650666+0000 front 2024-06-03T03:30:06.650530+0000 (oldest deadline 2024-06-03T03:30:33.051199+0000) 2024-06-03T03:30:42.232 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:30:41 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:30:41.900+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:01.379911+0000 front 2024-06-03T03:30:01.380025+0000 (oldest deadline 2024-06-03T03:30:25.479720+0000) 2024-06-03T03:30:42.232 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:30:41 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:30:41.900+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:11.881579+0000 front 2024-06-03T03:30:11.881609+0000 (oldest deadline 2024-06-03T03:30:37.781638+0000) 2024-06-03T03:30:42.232 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:30:41 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:30:41.900+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:17.782146+0000 front 2024-06-03T03:30:17.782047+0000 (oldest deadline 2024-06-03T03:30:41.882207+0000) 2024-06-03T03:30:42.406 INFO:journalctl@ceph.osd.7.smithi151.stdout:Jun 03 03:30:42 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-7[55035]: 2024-06-03T03:30:42.125+0000 7f8e5df2b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:08.069472+0000 front 2024-06-03T03:30:08.069491+0000 (oldest deadline 2024-06-03T03:30:32.769733+0000) 2024-06-03T03:30:42.427 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:30:41 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:30:41.990+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.254189+0000 front 2024-06-03T03:30:09.253879+0000 (oldest deadline 2024-06-03T03:30:33.353839+0000) 2024-06-03T03:30:42.696 INFO:journalctl@ceph.osd.6.smithi117.stdout:Jun 03 03:30:42 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-6[57519]: 2024-06-03T03:30:42.425+0000 7f50fe600700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:11.044622+0000 front 2024-06-03T03:30:11.044544+0000 (oldest deadline 2024-06-03T03:30:33.944525+0000) 2024-06-03T03:30:42.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:42 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:42.506+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:02.742111+0000 front 2024-06-03T03:30:02.742070+0000 (oldest deadline 2024-06-03T03:30:26.241792+0000) 2024-06-03T03:30:42.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:42 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:42.506+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.742816+0000 front 2024-06-03T03:30:09.742795+0000 (oldest deadline 2024-06-03T03:30:33.243036+0000) 2024-06-03T03:30:42.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:42 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:42.506+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:18.444645+0000 front 2024-06-03T03:30:18.444706+0000 (oldest deadline 2024-06-03T03:30:41.944716+0000) 2024-06-03T03:30:42.981 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:30:42 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:30:42.893+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:01.379911+0000 front 2024-06-03T03:30:01.380025+0000 (oldest deadline 2024-06-03T03:30:25.479720+0000) 2024-06-03T03:30:42.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:30:42 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:30:42.893+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:11.881579+0000 front 2024-06-03T03:30:11.881609+0000 (oldest deadline 2024-06-03T03:30:37.781638+0000) 2024-06-03T03:30:42.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:30:42 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:30:42.893+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:17.782146+0000 front 2024-06-03T03:30:17.782047+0000 (oldest deadline 2024-06-03T03:30:41.882207+0000) 2024-06-03T03:30:42.982 INFO:journalctl@ceph.osd.5.smithi183.stdout:Jun 03 03:30:42 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-5[49050]: 2024-06-03T03:30:42.696+0000 7f7803952700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:00.624352+0000 front 2024-06-03T03:30:00.624587+0000 (oldest deadline 2024-06-03T03:30:24.724295+0000) 2024-06-03T03:30:42.982 INFO:journalctl@ceph.osd.5.smithi183.stdout:Jun 03 03:30:42 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-5[49050]: 2024-06-03T03:30:42.696+0000 7f7803952700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.425446+0000 front 2024-06-03T03:30:09.425383+0000 (oldest deadline 2024-06-03T03:30:32.925380+0000) 2024-06-03T03:30:42.982 INFO:journalctl@ceph.osd.5.smithi183.stdout:Jun 03 03:30:42 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-5[49050]: 2024-06-03T03:30:42.696+0000 7f7803952700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:17.526794+0000 front 2024-06-03T03:30:17.526834+0000 (oldest deadline 2024-06-03T03:30:42.226932+0000) 2024-06-03T03:30:43.406 INFO:journalctl@ceph.osd.7.smithi151.stdout:Jun 03 03:30:43 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-7[55035]: 2024-06-03T03:30:43.077+0000 7f8e5df2b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:08.069472+0000 front 2024-06-03T03:30:08.069491+0000 (oldest deadline 2024-06-03T03:30:32.769733+0000) 2024-06-03T03:30:43.417 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:30:42 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:30:42.946+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.254189+0000 front 2024-06-03T03:30:09.253879+0000 (oldest deadline 2024-06-03T03:30:33.353839+0000) 2024-06-03T03:30:43.417 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:30:42 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:30:42.946+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:16.855414+0000 front 2024-06-03T03:30:16.855488+0000 (oldest deadline 2024-06-03T03:30:42.755658+0000) 2024-06-03T03:30:43.481 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:30:43 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:30:43.018+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:06.650666+0000 front 2024-06-03T03:30:06.650530+0000 (oldest deadline 2024-06-03T03:30:33.051199+0000) 2024-06-03T03:30:43.696 INFO:journalctl@ceph.osd.6.smithi117.stdout:Jun 03 03:30:43 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-6[57519]: 2024-06-03T03:30:43.416+0000 7f50fe600700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:11.044622+0000 front 2024-06-03T03:30:11.044544+0000 (oldest deadline 2024-06-03T03:30:33.944525+0000) 2024-06-03T03:30:43.696 INFO:journalctl@ceph.osd.6.smithi117.stdout:Jun 03 03:30:43 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-6[57519]: 2024-06-03T03:30:43.416+0000 7f50fe600700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:17.446468+0000 front 2024-06-03T03:30:17.446276+0000 (oldest deadline 2024-06-03T03:30:43.345900+0000) 2024-06-03T03:30:43.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:43 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:43.524+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:02.742111+0000 front 2024-06-03T03:30:02.742070+0000 (oldest deadline 2024-06-03T03:30:26.241792+0000) 2024-06-03T03:30:43.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:43 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:43.524+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.742816+0000 front 2024-06-03T03:30:09.742795+0000 (oldest deadline 2024-06-03T03:30:33.243036+0000) 2024-06-03T03:30:43.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:43 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:43.524+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:18.444645+0000 front 2024-06-03T03:30:18.444706+0000 (oldest deadline 2024-06-03T03:30:41.944716+0000) 2024-06-03T03:30:43.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:30:43 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:30:43.845+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:01.379911+0000 front 2024-06-03T03:30:01.380025+0000 (oldest deadline 2024-06-03T03:30:25.479720+0000) 2024-06-03T03:30:43.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:30:43 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:30:43.845+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:11.881579+0000 front 2024-06-03T03:30:11.881609+0000 (oldest deadline 2024-06-03T03:30:37.781638+0000) 2024-06-03T03:30:43.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:30:43 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:30:43.845+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:17.782146+0000 front 2024-06-03T03:30:17.782047+0000 (oldest deadline 2024-06-03T03:30:41.882207+0000) 2024-06-03T03:30:43.982 INFO:journalctl@ceph.osd.5.smithi183.stdout:Jun 03 03:30:43 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-5[49050]: 2024-06-03T03:30:43.662+0000 7f7803952700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:00.624352+0000 front 2024-06-03T03:30:00.624587+0000 (oldest deadline 2024-06-03T03:30:24.724295+0000) 2024-06-03T03:30:43.982 INFO:journalctl@ceph.osd.5.smithi183.stdout:Jun 03 03:30:43 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-5[49050]: 2024-06-03T03:30:43.662+0000 7f7803952700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.425446+0000 front 2024-06-03T03:30:09.425383+0000 (oldest deadline 2024-06-03T03:30:32.925380+0000) 2024-06-03T03:30:43.982 INFO:journalctl@ceph.osd.5.smithi183.stdout:Jun 03 03:30:43 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-5[49050]: 2024-06-03T03:30:43.662+0000 7f7803952700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:17.526794+0000 front 2024-06-03T03:30:17.526834+0000 (oldest deadline 2024-06-03T03:30:42.226932+0000) 2024-06-03T03:30:44.406 INFO:journalctl@ceph.osd.7.smithi151.stdout:Jun 03 03:30:44 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-7[55035]: 2024-06-03T03:30:44.080+0000 7f8e5df2b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:08.069472+0000 front 2024-06-03T03:30:08.069491+0000 (oldest deadline 2024-06-03T03:30:32.769733+0000) 2024-06-03T03:30:44.437 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:30:43 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:30:43.993+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.254189+0000 front 2024-06-03T03:30:09.253879+0000 (oldest deadline 2024-06-03T03:30:33.353839+0000) 2024-06-03T03:30:44.437 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:30:43 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:30:43.993+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:16.855414+0000 front 2024-06-03T03:30:16.855488+0000 (oldest deadline 2024-06-03T03:30:42.755658+0000) 2024-06-03T03:30:44.481 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:30:44 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:30:44.024+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:06.650666+0000 front 2024-06-03T03:30:06.650530+0000 (oldest deadline 2024-06-03T03:30:33.051199+0000) 2024-06-03T03:30:44.696 INFO:journalctl@ceph.osd.6.smithi117.stdout:Jun 03 03:30:44 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-6[57519]: 2024-06-03T03:30:44.435+0000 7f50fe600700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:11.044622+0000 front 2024-06-03T03:30:11.044544+0000 (oldest deadline 2024-06-03T03:30:33.944525+0000) 2024-06-03T03:30:44.696 INFO:journalctl@ceph.osd.6.smithi117.stdout:Jun 03 03:30:44 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-6[57519]: 2024-06-03T03:30:44.435+0000 7f50fe600700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:17.446468+0000 front 2024-06-03T03:30:17.446276+0000 (oldest deadline 2024-06-03T03:30:43.345900+0000) 2024-06-03T03:30:44.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:44 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:44.527+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:02.742111+0000 front 2024-06-03T03:30:02.742070+0000 (oldest deadline 2024-06-03T03:30:26.241792+0000) 2024-06-03T03:30:44.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:44 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:44.527+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.742816+0000 front 2024-06-03T03:30:09.742795+0000 (oldest deadline 2024-06-03T03:30:33.243036+0000) 2024-06-03T03:30:44.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:44 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:44.527+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:18.444645+0000 front 2024-06-03T03:30:18.444706+0000 (oldest deadline 2024-06-03T03:30:41.944716+0000) 2024-06-03T03:30:44.981 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:30:44 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:30:44.822+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:01.379911+0000 front 2024-06-03T03:30:01.380025+0000 (oldest deadline 2024-06-03T03:30:25.479720+0000) 2024-06-03T03:30:44.981 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:30:44 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:30:44.822+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:11.881579+0000 front 2024-06-03T03:30:11.881609+0000 (oldest deadline 2024-06-03T03:30:37.781638+0000) 2024-06-03T03:30:44.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:30:44 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:30:44.822+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:17.782146+0000 front 2024-06-03T03:30:17.782047+0000 (oldest deadline 2024-06-03T03:30:41.882207+0000) 2024-06-03T03:30:44.982 INFO:journalctl@ceph.osd.5.smithi183.stdout:Jun 03 03:30:44 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-5[49050]: 2024-06-03T03:30:44.661+0000 7f7803952700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:00.624352+0000 front 2024-06-03T03:30:00.624587+0000 (oldest deadline 2024-06-03T03:30:24.724295+0000) 2024-06-03T03:30:44.982 INFO:journalctl@ceph.osd.5.smithi183.stdout:Jun 03 03:30:44 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-5[49050]: 2024-06-03T03:30:44.661+0000 7f7803952700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.425446+0000 front 2024-06-03T03:30:09.425383+0000 (oldest deadline 2024-06-03T03:30:32.925380+0000) 2024-06-03T03:30:44.982 INFO:journalctl@ceph.osd.5.smithi183.stdout:Jun 03 03:30:44 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-5[49050]: 2024-06-03T03:30:44.661+0000 7f7803952700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:17.526794+0000 front 2024-06-03T03:30:17.526834+0000 (oldest deadline 2024-06-03T03:30:42.226932+0000) 2024-06-03T03:30:45.406 INFO:journalctl@ceph.osd.7.smithi151.stdout:Jun 03 03:30:45 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-7[55035]: 2024-06-03T03:30:45.113+0000 7f8e5df2b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:08.069472+0000 front 2024-06-03T03:30:08.069491+0000 (oldest deadline 2024-06-03T03:30:32.769733+0000) 2024-06-03T03:30:45.446 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:30:45 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:30:45.013+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.254189+0000 front 2024-06-03T03:30:09.253879+0000 (oldest deadline 2024-06-03T03:30:33.353839+0000) 2024-06-03T03:30:45.447 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:30:45 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:30:45.013+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:16.855414+0000 front 2024-06-03T03:30:16.855488+0000 (oldest deadline 2024-06-03T03:30:42.755658+0000) 2024-06-03T03:30:45.481 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:30:45 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:30:45.026+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:06.650666+0000 front 2024-06-03T03:30:06.650530+0000 (oldest deadline 2024-06-03T03:30:33.051199+0000) 2024-06-03T03:30:45.482 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:30:45 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:30:45.026+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:19.452818+0000 front 2024-06-03T03:30:19.452796+0000 (oldest deadline 2024-06-03T03:30:44.153028+0000) 2024-06-03T03:30:45.760 INFO:journalctl@ceph.osd.5.smithi183.stdout:Jun 03 03:30:45 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-5[49050]: 2024-06-03T03:30:45.628+0000 7f7803952700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:00.624352+0000 front 2024-06-03T03:30:00.624587+0000 (oldest deadline 2024-06-03T03:30:24.724295+0000) 2024-06-03T03:30:45.760 INFO:journalctl@ceph.osd.5.smithi183.stdout:Jun 03 03:30:45 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-5[49050]: 2024-06-03T03:30:45.628+0000 7f7803952700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.425446+0000 front 2024-06-03T03:30:09.425383+0000 (oldest deadline 2024-06-03T03:30:32.925380+0000) 2024-06-03T03:30:45.760 INFO:journalctl@ceph.osd.5.smithi183.stdout:Jun 03 03:30:45 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-5[49050]: 2024-06-03T03:30:45.628+0000 7f7803952700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:17.526794+0000 front 2024-06-03T03:30:17.526834+0000 (oldest deadline 2024-06-03T03:30:42.226932+0000) 2024-06-03T03:30:45.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:45 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:45.566+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:02.742111+0000 front 2024-06-03T03:30:02.742070+0000 (oldest deadline 2024-06-03T03:30:26.241792+0000) 2024-06-03T03:30:45.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:45 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:45.566+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.742816+0000 front 2024-06-03T03:30:09.742795+0000 (oldest deadline 2024-06-03T03:30:33.243036+0000) 2024-06-03T03:30:45.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:45 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:45.566+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:18.444645+0000 front 2024-06-03T03:30:18.444706+0000 (oldest deadline 2024-06-03T03:30:41.944716+0000) 2024-06-03T03:30:45.946 INFO:journalctl@ceph.osd.6.smithi117.stdout:Jun 03 03:30:45 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-6[57519]: 2024-06-03T03:30:45.453+0000 7f50fe600700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:11.044622+0000 front 2024-06-03T03:30:11.044544+0000 (oldest deadline 2024-06-03T03:30:33.944525+0000) 2024-06-03T03:30:45.946 INFO:journalctl@ceph.osd.6.smithi117.stdout:Jun 03 03:30:45 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-6[57519]: 2024-06-03T03:30:45.453+0000 7f50fe600700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:17.446468+0000 front 2024-06-03T03:30:17.446276+0000 (oldest deadline 2024-06-03T03:30:43.345900+0000) 2024-06-03T03:30:46.034 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:30:45 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:30:45.790+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:01.379911+0000 front 2024-06-03T03:30:01.380025+0000 (oldest deadline 2024-06-03T03:30:25.479720+0000) 2024-06-03T03:30:46.034 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:30:45 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:30:45.790+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:11.881579+0000 front 2024-06-03T03:30:11.881609+0000 (oldest deadline 2024-06-03T03:30:37.781638+0000) 2024-06-03T03:30:46.035 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:30:45 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:30:45.790+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:17.782146+0000 front 2024-06-03T03:30:17.782047+0000 (oldest deadline 2024-06-03T03:30:41.882207+0000) 2024-06-03T03:30:46.406 INFO:journalctl@ceph.osd.7.smithi151.stdout:Jun 03 03:30:46 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-7[55035]: 2024-06-03T03:30:46.152+0000 7f8e5df2b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:08.069472+0000 front 2024-06-03T03:30:08.069491+0000 (oldest deadline 2024-06-03T03:30:32.769733+0000) 2024-06-03T03:30:46.406 INFO:journalctl@ceph.osd.7.smithi151.stdout:Jun 03 03:30:46 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-7[55035]: 2024-06-03T03:30:46.152+0000 7f8e5df2b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:20.371699+0000 front 2024-06-03T03:30:20.371774+0000 (oldest deadline 2024-06-03T03:30:45.671759+0000) 2024-06-03T03:30:46.446 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:30:45 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:30:45.992+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.254189+0000 front 2024-06-03T03:30:09.253879+0000 (oldest deadline 2024-06-03T03:30:33.353839+0000) 2024-06-03T03:30:46.446 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:30:45 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:30:45.992+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:16.855414+0000 front 2024-06-03T03:30:16.855488+0000 (oldest deadline 2024-06-03T03:30:42.755658+0000) 2024-06-03T03:30:46.481 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:30:46 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:30:46.031+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:06.650666+0000 front 2024-06-03T03:30:06.650530+0000 (oldest deadline 2024-06-03T03:30:33.051199+0000) 2024-06-03T03:30:46.481 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:30:46 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:30:46.031+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:19.452818+0000 front 2024-06-03T03:30:19.452796+0000 (oldest deadline 2024-06-03T03:30:44.153028+0000) 2024-06-03T03:30:46.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:46 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:46.541+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:02.742111+0000 front 2024-06-03T03:30:02.742070+0000 (oldest deadline 2024-06-03T03:30:26.241792+0000) 2024-06-03T03:30:46.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:46 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:46.541+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.742816+0000 front 2024-06-03T03:30:09.742795+0000 (oldest deadline 2024-06-03T03:30:33.243036+0000) 2024-06-03T03:30:46.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:46 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:46.541+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:18.444645+0000 front 2024-06-03T03:30:18.444706+0000 (oldest deadline 2024-06-03T03:30:41.944716+0000) 2024-06-03T03:30:46.946 INFO:journalctl@ceph.osd.6.smithi117.stdout:Jun 03 03:30:46 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-6[57519]: 2024-06-03T03:30:46.498+0000 7f50fe600700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:11.044622+0000 front 2024-06-03T03:30:11.044544+0000 (oldest deadline 2024-06-03T03:30:33.944525+0000) 2024-06-03T03:30:46.946 INFO:journalctl@ceph.osd.6.smithi117.stdout:Jun 03 03:30:46 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-6[57519]: 2024-06-03T03:30:46.498+0000 7f50fe600700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:17.446468+0000 front 2024-06-03T03:30:17.446276+0000 (oldest deadline 2024-06-03T03:30:43.345900+0000) 2024-06-03T03:30:46.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:30:46 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:30:46.834+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:01.379911+0000 front 2024-06-03T03:30:01.380025+0000 (oldest deadline 2024-06-03T03:30:25.479720+0000) 2024-06-03T03:30:46.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:30:46 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:30:46.834+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:11.881579+0000 front 2024-06-03T03:30:11.881609+0000 (oldest deadline 2024-06-03T03:30:37.781638+0000) 2024-06-03T03:30:46.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:30:46 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:30:46.834+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:17.782146+0000 front 2024-06-03T03:30:17.782047+0000 (oldest deadline 2024-06-03T03:30:41.882207+0000) 2024-06-03T03:30:46.982 INFO:journalctl@ceph.osd.5.smithi183.stdout:Jun 03 03:30:46 smithi183 podman[72043]: 2024-06-03 03:30:46.664278688 +0000 UTC m=+5.517922112 container died fdaef9fa776d8b2617760f5a824054efcf42ff85f1d607e6d85e4c459c02ab3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, ceph=True) 2024-06-03T03:30:47.309 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:30:47 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:30:47.077+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:06.650666+0000 front 2024-06-03T03:30:06.650530+0000 (oldest deadline 2024-06-03T03:30:33.051199+0000) 2024-06-03T03:30:47.309 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:30:47 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:30:47.077+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:19.452818+0000 front 2024-06-03T03:30:19.452796+0000 (oldest deadline 2024-06-03T03:30:44.153028+0000) 2024-06-03T03:30:47.309 INFO:journalctl@ceph.osd.5.smithi183.stdout:Jun 03 03:30:47 smithi183 podman[72043]: 2024-06-03 03:30:47.082976033 +0000 UTC m=+5.936619449 container cleanup fdaef9fa776d8b2617760f5a824054efcf42ff85f1d607e6d85e4c459c02ab3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-06-03T03:30:47.309 INFO:journalctl@ceph.osd.5.smithi183.stdout:Jun 03 03:30:47 smithi183 bash[72043]: ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-5 2024-06-03T03:30:47.406 INFO:journalctl@ceph.osd.7.smithi151.stdout:Jun 03 03:30:47 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-7[55035]: 2024-06-03T03:30:47.104+0000 7f8e5df2b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:08.069472+0000 front 2024-06-03T03:30:08.069491+0000 (oldest deadline 2024-06-03T03:30:32.769733+0000) 2024-06-03T03:30:47.406 INFO:journalctl@ceph.osd.7.smithi151.stdout:Jun 03 03:30:47 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-7[55035]: 2024-06-03T03:30:47.104+0000 7f8e5df2b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:20.371699+0000 front 2024-06-03T03:30:20.371774+0000 (oldest deadline 2024-06-03T03:30:45.671759+0000) 2024-06-03T03:30:47.446 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:30:47 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:30:47.004+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.254189+0000 front 2024-06-03T03:30:09.253879+0000 (oldest deadline 2024-06-03T03:30:33.353839+0000) 2024-06-03T03:30:47.447 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:30:47 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:30:47.004+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:16.855414+0000 front 2024-06-03T03:30:16.855488+0000 (oldest deadline 2024-06-03T03:30:42.755658+0000) 2024-06-03T03:30:47.731 INFO:journalctl@ceph.osd.5.smithi183.stdout:Jun 03 03:30:47 smithi183 podman[72069]: 2024-06-03 03:30:47.309609036 +0000 UTC m=+0.645052978 container remove fdaef9fa776d8b2617760f5a824054efcf42ff85f1d607e6d85e4c459c02ab3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528) 2024-06-03T03:30:47.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:47 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:47.529+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:02.742111+0000 front 2024-06-03T03:30:02.742070+0000 (oldest deadline 2024-06-03T03:30:26.241792+0000) 2024-06-03T03:30:47.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:47 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:47.529+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.742816+0000 front 2024-06-03T03:30:09.742795+0000 (oldest deadline 2024-06-03T03:30:33.243036+0000) 2024-06-03T03:30:47.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:47 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:47.529+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:18.444645+0000 front 2024-06-03T03:30:18.444706+0000 (oldest deadline 2024-06-03T03:30:41.944716+0000) 2024-06-03T03:30:47.946 INFO:journalctl@ceph.osd.6.smithi117.stdout:Jun 03 03:30:47 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-6[57519]: 2024-06-03T03:30:47.542+0000 7f50fe600700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:11.044622+0000 front 2024-06-03T03:30:11.044544+0000 (oldest deadline 2024-06-03T03:30:33.944525+0000) 2024-06-03T03:30:47.946 INFO:journalctl@ceph.osd.6.smithi117.stdout:Jun 03 03:30:47 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-6[57519]: 2024-06-03T03:30:47.542+0000 7f50fe600700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:17.446468+0000 front 2024-06-03T03:30:17.446276+0000 (oldest deadline 2024-06-03T03:30:43.345900+0000) 2024-06-03T03:30:48.013 INFO:journalctl@ceph.osd.5.smithi183.stdout:Jun 03 03:30:47 smithi183 podman[72160]: 2024-06-03 03:30:47.647292731 +0000 UTC m=+0.017217218 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T03:30:48.013 INFO:journalctl@ceph.osd.5.smithi183.stdout:Jun 03 03:30:47 smithi183 podman[72160]: 2024-06-03 03:30:47.769040878 +0000 UTC m=+0.138965395 container create f800017edbf2980ba740763302238496f44cfab5c8e27e6853ee9d91ca31a5d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-06-03T03:30:48.013 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:30:47 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:30:47.787+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:01.379911+0000 front 2024-06-03T03:30:01.380025+0000 (oldest deadline 2024-06-03T03:30:25.479720+0000) 2024-06-03T03:30:48.013 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:30:47 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:30:47.787+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:11.881579+0000 front 2024-06-03T03:30:11.881609+0000 (oldest deadline 2024-06-03T03:30:37.781638+0000) 2024-06-03T03:30:48.013 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:30:47 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:30:47.787+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:17.782146+0000 front 2024-06-03T03:30:17.782047+0000 (oldest deadline 2024-06-03T03:30:41.882207+0000) 2024-06-03T03:30:48.315 INFO:journalctl@ceph.osd.5.smithi183.stdout:Jun 03 03:30:48 smithi183 podman[72160]: 2024-06-03 03:30:48.013473485 +0000 UTC m=+0.383397955 container init f800017edbf2980ba740763302238496f44cfab5c8e27e6853ee9d91ca31a5d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-06-03T03:30:48.315 INFO:journalctl@ceph.osd.5.smithi183.stdout:Jun 03 03:30:48 smithi183 podman[72160]: 2024-06-03 03:30:48.016619758 +0000 UTC m=+0.386544237 container start f800017edbf2980ba740763302238496f44cfab5c8e27e6853ee9d91ca31a5d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-5-deactivate, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-03T03:30:48.315 INFO:journalctl@ceph.osd.5.smithi183.stdout:Jun 03 03:30:48 smithi183 podman[72160]: 2024-06-03 03:30:48.130271031 +0000 UTC m=+0.500195521 container attach f800017edbf2980ba740763302238496f44cfab5c8e27e6853ee9d91ca31a5d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc) 2024-06-03T03:30:48.315 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:30:48 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:30:48.104+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:06.650666+0000 front 2024-06-03T03:30:06.650530+0000 (oldest deadline 2024-06-03T03:30:33.051199+0000) 2024-06-03T03:30:48.315 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:30:48 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:30:48.104+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:19.452818+0000 front 2024-06-03T03:30:19.452796+0000 (oldest deadline 2024-06-03T03:30:44.153028+0000) 2024-06-03T03:30:48.406 INFO:journalctl@ceph.osd.7.smithi151.stdout:Jun 03 03:30:48 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-7[55035]: 2024-06-03T03:30:48.154+0000 7f8e5df2b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:08.069472+0000 front 2024-06-03T03:30:08.069491+0000 (oldest deadline 2024-06-03T03:30:32.769733+0000) 2024-06-03T03:30:48.406 INFO:journalctl@ceph.osd.7.smithi151.stdout:Jun 03 03:30:48 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-7[55035]: 2024-06-03T03:30:48.154+0000 7f8e5df2b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:20.371699+0000 front 2024-06-03T03:30:20.371774+0000 (oldest deadline 2024-06-03T03:30:45.671759+0000) 2024-06-03T03:30:48.446 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:30:48 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:30:48.050+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.254189+0000 front 2024-06-03T03:30:09.253879+0000 (oldest deadline 2024-06-03T03:30:33.353839+0000) 2024-06-03T03:30:48.447 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:30:48 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:30:48.050+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:16.855414+0000 front 2024-06-03T03:30:16.855488+0000 (oldest deadline 2024-06-03T03:30:42.755658+0000) 2024-06-03T03:30:48.634 INFO:journalctl@ceph.osd.5.smithi183.stdout:Jun 03 03:30:48 smithi183 podman[72187]: 2024-06-03 03:30:48.3435305 +0000 UTC m=+0.017779014 container died f800017edbf2980ba740763302238496f44cfab5c8e27e6853ee9d91ca31a5d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-5-deactivate, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-03T03:30:48.831 INFO:journalctl@ceph.osd.6.smithi117.stdout:Jun 03 03:30:48 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-6[57519]: 2024-06-03T03:30:48.506+0000 7f50fe600700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:11.044622+0000 front 2024-06-03T03:30:11.044544+0000 (oldest deadline 2024-06-03T03:30:33.944525+0000) 2024-06-03T03:30:48.831 INFO:journalctl@ceph.osd.6.smithi117.stdout:Jun 03 03:30:48 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-6[57519]: 2024-06-03T03:30:48.507+0000 7f50fe600700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:17.446468+0000 front 2024-06-03T03:30:17.446276+0000 (oldest deadline 2024-06-03T03:30:43.345900+0000) 2024-06-03T03:30:48.901 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:30:48 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:30:48.757+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:01.379911+0000 front 2024-06-03T03:30:01.380025+0000 (oldest deadline 2024-06-03T03:30:25.479720+0000) 2024-06-03T03:30:48.901 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:30:48 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:30:48.757+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:11.881579+0000 front 2024-06-03T03:30:11.881609+0000 (oldest deadline 2024-06-03T03:30:37.781638+0000) 2024-06-03T03:30:48.901 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:30:48 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:30:48.757+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:17.782146+0000 front 2024-06-03T03:30:17.782047+0000 (oldest deadline 2024-06-03T03:30:41.882207+0000) 2024-06-03T03:30:48.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:48 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:48.566+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:02.742111+0000 front 2024-06-03T03:30:02.742070+0000 (oldest deadline 2024-06-03T03:30:26.241792+0000) 2024-06-03T03:30:48.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:48 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:48.566+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.742816+0000 front 2024-06-03T03:30:09.742795+0000 (oldest deadline 2024-06-03T03:30:33.243036+0000) 2024-06-03T03:30:48.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:48 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:48.566+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:18.444645+0000 front 2024-06-03T03:30:18.444706+0000 (oldest deadline 2024-06-03T03:30:41.944716+0000) 2024-06-03T03:30:48.925 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@osd.5.service' 2024-06-03T03:30:49.004 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T03:30:49.004 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-06-03T03:30:49.005 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-06-03T03:30:49.005 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@osd.6 2024-06-03T03:30:49.099 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:30:49 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:30:49.039+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.254189+0000 front 2024-06-03T03:30:09.253879+0000 (oldest deadline 2024-06-03T03:30:33.353839+0000) 2024-06-03T03:30:49.099 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:30:49 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:30:49.039+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:16.855414+0000 front 2024-06-03T03:30:16.855488+0000 (oldest deadline 2024-06-03T03:30:42.755658+0000) 2024-06-03T03:30:49.231 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:30:49 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:30:49.123+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:06.650666+0000 front 2024-06-03T03:30:06.650530+0000 (oldest deadline 2024-06-03T03:30:33.051199+0000) 2024-06-03T03:30:49.231 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:30:49 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:30:49.123+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:19.452818+0000 front 2024-06-03T03:30:19.452796+0000 (oldest deadline 2024-06-03T03:30:44.153028+0000) 2024-06-03T03:30:49.446 INFO:journalctl@ceph.osd.6.smithi117.stdout:Jun 03 03:30:49 smithi117 systemd[1]: Stopping Ceph osd.6 for a0e8473a-2156-11ef-bc9b-c7b262605968... 2024-06-03T03:30:49.544 INFO:journalctl@ceph.osd.7.smithi151.stdout:Jun 03 03:30:49 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-7[55035]: 2024-06-03T03:30:49.164+0000 7f8e5df2b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:08.069472+0000 front 2024-06-03T03:30:08.069491+0000 (oldest deadline 2024-06-03T03:30:32.769733+0000) 2024-06-03T03:30:49.544 INFO:journalctl@ceph.osd.7.smithi151.stdout:Jun 03 03:30:49 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-7[55035]: 2024-06-03T03:30:49.164+0000 7f8e5df2b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:20.371699+0000 front 2024-06-03T03:30:20.371774+0000 (oldest deadline 2024-06-03T03:30:45.671759+0000) 2024-06-03T03:30:49.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:49 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:49.543+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:02.742111+0000 front 2024-06-03T03:30:02.742070+0000 (oldest deadline 2024-06-03T03:30:26.241792+0000) 2024-06-03T03:30:49.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:49 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:49.543+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.742816+0000 front 2024-06-03T03:30:09.742795+0000 (oldest deadline 2024-06-03T03:30:33.243036+0000) 2024-06-03T03:30:49.907 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:49 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:49.543+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:18.444645+0000 front 2024-06-03T03:30:18.444706+0000 (oldest deadline 2024-06-03T03:30:41.944716+0000) 2024-06-03T03:30:49.947 INFO:journalctl@ceph.osd.6.smithi117.stdout:Jun 03 03:30:49 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-6[57519]: 2024-06-03T03:30:49.483+0000 7f50fe600700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:11.044622+0000 front 2024-06-03T03:30:11.044544+0000 (oldest deadline 2024-06-03T03:30:33.944525+0000) 2024-06-03T03:30:49.947 INFO:journalctl@ceph.osd.6.smithi117.stdout:Jun 03 03:30:49 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-6[57519]: 2024-06-03T03:30:49.484+0000 7f50fe600700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:17.446468+0000 front 2024-06-03T03:30:17.446276+0000 (oldest deadline 2024-06-03T03:30:43.345900+0000) 2024-06-03T03:30:49.947 INFO:journalctl@ceph.osd.6.smithi117.stdout:Jun 03 03:30:49 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-6[57519]: 2024-06-03T03:30:49.579+0000 7f5102e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T03:30:49.947 INFO:journalctl@ceph.osd.6.smithi117.stdout:Jun 03 03:30:49 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-6[57519]: 2024-06-03T03:30:49.579+0000 7f5102e00700 -1 osd.6 103 *** Got signal Terminated *** 2024-06-03T03:30:49.947 INFO:journalctl@ceph.osd.6.smithi117.stdout:Jun 03 03:30:49 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-6[57519]: 2024-06-03T03:30:49.579+0000 7f5102e00700 -1 osd.6 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-03T03:30:50.113 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:30:49 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:30:49.755+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:01.379911+0000 front 2024-06-03T03:30:01.380025+0000 (oldest deadline 2024-06-03T03:30:25.479720+0000) 2024-06-03T03:30:50.113 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:30:49 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:30:49.755+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:11.881579+0000 front 2024-06-03T03:30:11.881609+0000 (oldest deadline 2024-06-03T03:30:37.781638+0000) 2024-06-03T03:30:50.113 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:30:49 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:30:49.755+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:17.782146+0000 front 2024-06-03T03:30:17.782047+0000 (oldest deadline 2024-06-03T03:30:41.882207+0000) 2024-06-03T03:30:50.406 INFO:journalctl@ceph.osd.7.smithi151.stdout:Jun 03 03:30:50 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-7[55035]: 2024-06-03T03:30:50.121+0000 7f8e5df2b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:08.069472+0000 front 2024-06-03T03:30:08.069491+0000 (oldest deadline 2024-06-03T03:30:32.769733+0000) 2024-06-03T03:30:50.407 INFO:journalctl@ceph.osd.7.smithi151.stdout:Jun 03 03:30:50 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-7[55035]: 2024-06-03T03:30:50.121+0000 7f8e5df2b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:20.371699+0000 front 2024-06-03T03:30:20.371774+0000 (oldest deadline 2024-06-03T03:30:45.671759+0000) 2024-06-03T03:30:50.446 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:30:49 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:30:49.996+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.254189+0000 front 2024-06-03T03:30:09.253879+0000 (oldest deadline 2024-06-03T03:30:33.353839+0000) 2024-06-03T03:30:50.446 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:30:49 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:30:49.996+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:16.855414+0000 front 2024-06-03T03:30:16.855488+0000 (oldest deadline 2024-06-03T03:30:42.755658+0000) 2024-06-03T03:30:50.481 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:30:50 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:30:50.112+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:06.650666+0000 front 2024-06-03T03:30:06.650530+0000 (oldest deadline 2024-06-03T03:30:33.051199+0000) 2024-06-03T03:30:50.481 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:30:50 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:30:50.112+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:19.452818+0000 front 2024-06-03T03:30:19.452796+0000 (oldest deadline 2024-06-03T03:30:44.153028+0000) 2024-06-03T03:30:50.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:50 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:50.556+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:02.742111+0000 front 2024-06-03T03:30:02.742070+0000 (oldest deadline 2024-06-03T03:30:26.241792+0000) 2024-06-03T03:30:50.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:50 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:50.556+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.742816+0000 front 2024-06-03T03:30:09.742795+0000 (oldest deadline 2024-06-03T03:30:33.243036+0000) 2024-06-03T03:30:50.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:50 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:50.556+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:18.444645+0000 front 2024-06-03T03:30:18.444706+0000 (oldest deadline 2024-06-03T03:30:41.944716+0000) 2024-06-03T03:30:50.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:50 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:50.556+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.446009+0000 front 2024-06-03T03:30:25.445806+0000 (oldest deadline 2024-06-03T03:30:50.046043+0000) 2024-06-03T03:30:50.946 INFO:journalctl@ceph.osd.6.smithi117.stdout:Jun 03 03:30:50 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-6[57519]: 2024-06-03T03:30:50.456+0000 7f50fe600700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:11.044622+0000 front 2024-06-03T03:30:11.044544+0000 (oldest deadline 2024-06-03T03:30:33.944525+0000) 2024-06-03T03:30:50.947 INFO:journalctl@ceph.osd.6.smithi117.stdout:Jun 03 03:30:50 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-6[57519]: 2024-06-03T03:30:50.456+0000 7f50fe600700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:17.446468+0000 front 2024-06-03T03:30:17.446276+0000 (oldest deadline 2024-06-03T03:30:43.345900+0000) 2024-06-03T03:30:50.947 INFO:journalctl@ceph.osd.6.smithi117.stdout:Jun 03 03:30:50 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-6[57519]: 2024-06-03T03:30:50.456+0000 7f50fe600700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.648273+0000 front 2024-06-03T03:30:25.648528+0000 (oldest deadline 2024-06-03T03:30:50.347948+0000) 2024-06-03T03:30:50.981 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:30:50 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:30:50.709+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:01.379911+0000 front 2024-06-03T03:30:01.380025+0000 (oldest deadline 2024-06-03T03:30:25.479720+0000) 2024-06-03T03:30:50.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:30:50 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:30:50.709+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:11.881579+0000 front 2024-06-03T03:30:11.881609+0000 (oldest deadline 2024-06-03T03:30:37.781638+0000) 2024-06-03T03:30:50.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:30:50 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:30:50.709+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:17.782146+0000 front 2024-06-03T03:30:17.782047+0000 (oldest deadline 2024-06-03T03:30:41.882207+0000) 2024-06-03T03:30:51.406 INFO:journalctl@ceph.osd.7.smithi151.stdout:Jun 03 03:30:51 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-7[55035]: 2024-06-03T03:30:51.082+0000 7f8e5df2b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:08.069472+0000 front 2024-06-03T03:30:08.069491+0000 (oldest deadline 2024-06-03T03:30:32.769733+0000) 2024-06-03T03:30:51.406 INFO:journalctl@ceph.osd.7.smithi151.stdout:Jun 03 03:30:51 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-7[55035]: 2024-06-03T03:30:51.082+0000 7f8e5df2b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:20.371699+0000 front 2024-06-03T03:30:20.371774+0000 (oldest deadline 2024-06-03T03:30:45.671759+0000) 2024-06-03T03:30:51.406 INFO:journalctl@ceph.osd.7.smithi151.stdout:Jun 03 03:30:51 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-7[55035]: 2024-06-03T03:30:51.082+0000 7f8e5df2b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.672527+0000 front 2024-06-03T03:30:25.672429+0000 (oldest deadline 2024-06-03T03:30:50.972365+0000) 2024-06-03T03:30:51.446 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:30:51 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:30:51.004+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.254189+0000 front 2024-06-03T03:30:09.253879+0000 (oldest deadline 2024-06-03T03:30:33.353839+0000) 2024-06-03T03:30:51.446 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:30:51 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:30:51.004+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:16.855414+0000 front 2024-06-03T03:30:16.855488+0000 (oldest deadline 2024-06-03T03:30:42.755658+0000) 2024-06-03T03:30:51.447 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:30:51 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:30:51.004+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.657633+0000 front 2024-06-03T03:30:25.658290+0000 (oldest deadline 2024-06-03T03:30:50.357531+0000) 2024-06-03T03:30:51.481 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:30:51 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:30:51.103+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:06.650666+0000 front 2024-06-03T03:30:06.650530+0000 (oldest deadline 2024-06-03T03:30:33.051199+0000) 2024-06-03T03:30:51.481 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:30:51 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:30:51.103+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:19.452818+0000 front 2024-06-03T03:30:19.452796+0000 (oldest deadline 2024-06-03T03:30:44.153028+0000) 2024-06-03T03:30:51.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:51 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:51.514+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:02.742111+0000 front 2024-06-03T03:30:02.742070+0000 (oldest deadline 2024-06-03T03:30:26.241792+0000) 2024-06-03T03:30:51.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:51 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:51.514+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.742816+0000 front 2024-06-03T03:30:09.742795+0000 (oldest deadline 2024-06-03T03:30:33.243036+0000) 2024-06-03T03:30:51.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:51 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:51.514+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:18.444645+0000 front 2024-06-03T03:30:18.444706+0000 (oldest deadline 2024-06-03T03:30:41.944716+0000) 2024-06-03T03:30:51.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:51 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:51.514+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.446009+0000 front 2024-06-03T03:30:25.445806+0000 (oldest deadline 2024-06-03T03:30:50.046043+0000) 2024-06-03T03:30:51.946 INFO:journalctl@ceph.osd.6.smithi117.stdout:Jun 03 03:30:51 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-6[57519]: 2024-06-03T03:30:51.498+0000 7f50fe600700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:11.044622+0000 front 2024-06-03T03:30:11.044544+0000 (oldest deadline 2024-06-03T03:30:33.944525+0000) 2024-06-03T03:30:51.947 INFO:journalctl@ceph.osd.6.smithi117.stdout:Jun 03 03:30:51 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-6[57519]: 2024-06-03T03:30:51.498+0000 7f50fe600700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:17.446468+0000 front 2024-06-03T03:30:17.446276+0000 (oldest deadline 2024-06-03T03:30:43.345900+0000) 2024-06-03T03:30:51.947 INFO:journalctl@ceph.osd.6.smithi117.stdout:Jun 03 03:30:51 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-6[57519]: 2024-06-03T03:30:51.498+0000 7f50fe600700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.648273+0000 front 2024-06-03T03:30:25.648528+0000 (oldest deadline 2024-06-03T03:30:50.347948+0000) 2024-06-03T03:30:51.981 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:30:51 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:30:51.710+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:01.379911+0000 front 2024-06-03T03:30:01.380025+0000 (oldest deadline 2024-06-03T03:30:25.479720+0000) 2024-06-03T03:30:51.981 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:30:51 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:30:51.710+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:11.881579+0000 front 2024-06-03T03:30:11.881609+0000 (oldest deadline 2024-06-03T03:30:37.781638+0000) 2024-06-03T03:30:51.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:30:51 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:30:51.710+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:17.782146+0000 front 2024-06-03T03:30:17.782047+0000 (oldest deadline 2024-06-03T03:30:41.882207+0000) 2024-06-03T03:30:52.406 INFO:journalctl@ceph.osd.7.smithi151.stdout:Jun 03 03:30:52 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-7[55035]: 2024-06-03T03:30:52.101+0000 7f8e5df2b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:08.069472+0000 front 2024-06-03T03:30:08.069491+0000 (oldest deadline 2024-06-03T03:30:32.769733+0000) 2024-06-03T03:30:52.406 INFO:journalctl@ceph.osd.7.smithi151.stdout:Jun 03 03:30:52 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-7[55035]: 2024-06-03T03:30:52.101+0000 7f8e5df2b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:20.371699+0000 front 2024-06-03T03:30:20.371774+0000 (oldest deadline 2024-06-03T03:30:45.671759+0000) 2024-06-03T03:30:52.406 INFO:journalctl@ceph.osd.7.smithi151.stdout:Jun 03 03:30:52 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-7[55035]: 2024-06-03T03:30:52.101+0000 7f8e5df2b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.672527+0000 front 2024-06-03T03:30:25.672429+0000 (oldest deadline 2024-06-03T03:30:50.972365+0000) 2024-06-03T03:30:52.446 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:30:51 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:30:51.976+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.254189+0000 front 2024-06-03T03:30:09.253879+0000 (oldest deadline 2024-06-03T03:30:33.353839+0000) 2024-06-03T03:30:52.447 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:30:51 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:30:51.976+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:16.855414+0000 front 2024-06-03T03:30:16.855488+0000 (oldest deadline 2024-06-03T03:30:42.755658+0000) 2024-06-03T03:30:52.447 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:30:51 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:30:51.976+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.657633+0000 front 2024-06-03T03:30:25.658290+0000 (oldest deadline 2024-06-03T03:30:50.357531+0000) 2024-06-03T03:30:52.481 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:30:52 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:30:52.068+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:06.650666+0000 front 2024-06-03T03:30:06.650530+0000 (oldest deadline 2024-06-03T03:30:33.051199+0000) 2024-06-03T03:30:52.481 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:30:52 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:30:52.068+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:19.452818+0000 front 2024-06-03T03:30:19.452796+0000 (oldest deadline 2024-06-03T03:30:44.153028+0000) 2024-06-03T03:30:52.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:52 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:52.474+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:02.742111+0000 front 2024-06-03T03:30:02.742070+0000 (oldest deadline 2024-06-03T03:30:26.241792+0000) 2024-06-03T03:30:52.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:52 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:52.474+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.742816+0000 front 2024-06-03T03:30:09.742795+0000 (oldest deadline 2024-06-03T03:30:33.243036+0000) 2024-06-03T03:30:52.907 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:52 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:52.474+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:18.444645+0000 front 2024-06-03T03:30:18.444706+0000 (oldest deadline 2024-06-03T03:30:41.944716+0000) 2024-06-03T03:30:52.907 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:52 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:52.474+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.446009+0000 front 2024-06-03T03:30:25.445806+0000 (oldest deadline 2024-06-03T03:30:50.046043+0000) 2024-06-03T03:30:52.928 INFO:journalctl@ceph.osd.6.smithi117.stdout:Jun 03 03:30:52 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-6[57519]: 2024-06-03T03:30:52.532+0000 7f50fe600700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:11.044622+0000 front 2024-06-03T03:30:11.044544+0000 (oldest deadline 2024-06-03T03:30:33.944525+0000) 2024-06-03T03:30:52.928 INFO:journalctl@ceph.osd.6.smithi117.stdout:Jun 03 03:30:52 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-6[57519]: 2024-06-03T03:30:52.532+0000 7f50fe600700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:17.446468+0000 front 2024-06-03T03:30:17.446276+0000 (oldest deadline 2024-06-03T03:30:43.345900+0000) 2024-06-03T03:30:52.928 INFO:journalctl@ceph.osd.6.smithi117.stdout:Jun 03 03:30:52 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-6[57519]: 2024-06-03T03:30:52.532+0000 7f50fe600700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.648273+0000 front 2024-06-03T03:30:25.648528+0000 (oldest deadline 2024-06-03T03:30:50.347948+0000) 2024-06-03T03:30:52.981 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:30:52 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:30:52.719+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:01.379911+0000 front 2024-06-03T03:30:01.380025+0000 (oldest deadline 2024-06-03T03:30:25.479720+0000) 2024-06-03T03:30:52.981 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:30:52 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:30:52.719+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:11.881579+0000 front 2024-06-03T03:30:11.881609+0000 (oldest deadline 2024-06-03T03:30:37.781638+0000) 2024-06-03T03:30:52.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:30:52 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:30:52.719+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:17.782146+0000 front 2024-06-03T03:30:17.782047+0000 (oldest deadline 2024-06-03T03:30:41.882207+0000) 2024-06-03T03:30:53.199 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:30:52 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:30:52.927+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.254189+0000 front 2024-06-03T03:30:09.253879+0000 (oldest deadline 2024-06-03T03:30:33.353839+0000) 2024-06-03T03:30:53.199 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:30:52 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:30:52.927+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:16.855414+0000 front 2024-06-03T03:30:16.855488+0000 (oldest deadline 2024-06-03T03:30:42.755658+0000) 2024-06-03T03:30:53.199 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:30:52 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:30:52.927+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.657633+0000 front 2024-06-03T03:30:25.658290+0000 (oldest deadline 2024-06-03T03:30:50.357531+0000) 2024-06-03T03:30:53.406 INFO:journalctl@ceph.osd.7.smithi151.stdout:Jun 03 03:30:53 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-7[55035]: 2024-06-03T03:30:53.131+0000 7f8e5df2b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:08.069472+0000 front 2024-06-03T03:30:08.069491+0000 (oldest deadline 2024-06-03T03:30:32.769733+0000) 2024-06-03T03:30:53.406 INFO:journalctl@ceph.osd.7.smithi151.stdout:Jun 03 03:30:53 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-7[55035]: 2024-06-03T03:30:53.131+0000 7f8e5df2b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:20.371699+0000 front 2024-06-03T03:30:20.371774+0000 (oldest deadline 2024-06-03T03:30:45.671759+0000) 2024-06-03T03:30:53.406 INFO:journalctl@ceph.osd.7.smithi151.stdout:Jun 03 03:30:53 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-7[55035]: 2024-06-03T03:30:53.131+0000 7f8e5df2b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.672527+0000 front 2024-06-03T03:30:25.672429+0000 (oldest deadline 2024-06-03T03:30:50.972365+0000) 2024-06-03T03:30:53.481 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:30:53 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:30:53.031+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:06.650666+0000 front 2024-06-03T03:30:06.650530+0000 (oldest deadline 2024-06-03T03:30:33.051199+0000) 2024-06-03T03:30:53.481 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:30:53 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:30:53.031+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:19.452818+0000 front 2024-06-03T03:30:19.452796+0000 (oldest deadline 2024-06-03T03:30:44.153028+0000) 2024-06-03T03:30:53.482 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:30:53 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:30:53.031+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:26.454415+0000 front 2024-06-03T03:30:26.454587+0000 (oldest deadline 2024-06-03T03:30:52.354304+0000) 2024-06-03T03:30:53.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:53 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:53.449+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:02.742111+0000 front 2024-06-03T03:30:02.742070+0000 (oldest deadline 2024-06-03T03:30:26.241792+0000) 2024-06-03T03:30:53.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:53 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:53.449+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.742816+0000 front 2024-06-03T03:30:09.742795+0000 (oldest deadline 2024-06-03T03:30:33.243036+0000) 2024-06-03T03:30:53.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:53 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:53.449+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:18.444645+0000 front 2024-06-03T03:30:18.444706+0000 (oldest deadline 2024-06-03T03:30:41.944716+0000) 2024-06-03T03:30:53.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:53 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:53.449+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.446009+0000 front 2024-06-03T03:30:25.445806+0000 (oldest deadline 2024-06-03T03:30:50.046043+0000) 2024-06-03T03:30:53.946 INFO:journalctl@ceph.osd.6.smithi117.stdout:Jun 03 03:30:53 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-6[57519]: 2024-06-03T03:30:53.494+0000 7f50fe600700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:11.044622+0000 front 2024-06-03T03:30:11.044544+0000 (oldest deadline 2024-06-03T03:30:33.944525+0000) 2024-06-03T03:30:53.946 INFO:journalctl@ceph.osd.6.smithi117.stdout:Jun 03 03:30:53 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-6[57519]: 2024-06-03T03:30:53.494+0000 7f50fe600700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:17.446468+0000 front 2024-06-03T03:30:17.446276+0000 (oldest deadline 2024-06-03T03:30:43.345900+0000) 2024-06-03T03:30:53.946 INFO:journalctl@ceph.osd.6.smithi117.stdout:Jun 03 03:30:53 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-6[57519]: 2024-06-03T03:30:53.494+0000 7f50fe600700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.648273+0000 front 2024-06-03T03:30:25.648528+0000 (oldest deadline 2024-06-03T03:30:50.347948+0000) 2024-06-03T03:30:53.981 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:30:53 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:30:53.714+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:01.379911+0000 front 2024-06-03T03:30:01.380025+0000 (oldest deadline 2024-06-03T03:30:25.479720+0000) 2024-06-03T03:30:53.981 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:30:53 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:30:53.714+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:11.881579+0000 front 2024-06-03T03:30:11.881609+0000 (oldest deadline 2024-06-03T03:30:37.781638+0000) 2024-06-03T03:30:53.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:30:53 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:30:53.714+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:17.782146+0000 front 2024-06-03T03:30:17.782047+0000 (oldest deadline 2024-06-03T03:30:41.882207+0000) 2024-06-03T03:30:54.406 INFO:journalctl@ceph.osd.7.smithi151.stdout:Jun 03 03:30:54 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-7[55035]: 2024-06-03T03:30:54.083+0000 7f8e5df2b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:08.069472+0000 front 2024-06-03T03:30:08.069491+0000 (oldest deadline 2024-06-03T03:30:32.769733+0000) 2024-06-03T03:30:54.406 INFO:journalctl@ceph.osd.7.smithi151.stdout:Jun 03 03:30:54 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-7[55035]: 2024-06-03T03:30:54.083+0000 7f8e5df2b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:20.371699+0000 front 2024-06-03T03:30:20.371774+0000 (oldest deadline 2024-06-03T03:30:45.671759+0000) 2024-06-03T03:30:54.406 INFO:journalctl@ceph.osd.7.smithi151.stdout:Jun 03 03:30:54 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-7[55035]: 2024-06-03T03:30:54.083+0000 7f8e5df2b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.672527+0000 front 2024-06-03T03:30:25.672429+0000 (oldest deadline 2024-06-03T03:30:50.972365+0000) 2024-06-03T03:30:54.446 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:30:53 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:30:53.959+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.254189+0000 front 2024-06-03T03:30:09.253879+0000 (oldest deadline 2024-06-03T03:30:33.353839+0000) 2024-06-03T03:30:54.447 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:30:53 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:30:53.959+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:16.855414+0000 front 2024-06-03T03:30:16.855488+0000 (oldest deadline 2024-06-03T03:30:42.755658+0000) 2024-06-03T03:30:54.447 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:30:53 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:30:53.959+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.657633+0000 front 2024-06-03T03:30:25.658290+0000 (oldest deadline 2024-06-03T03:30:50.357531+0000) 2024-06-03T03:30:54.481 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:30:53 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:30:53.982+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:06.650666+0000 front 2024-06-03T03:30:06.650530+0000 (oldest deadline 2024-06-03T03:30:33.051199+0000) 2024-06-03T03:30:54.482 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:30:53 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:30:53.982+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:19.452818+0000 front 2024-06-03T03:30:19.452796+0000 (oldest deadline 2024-06-03T03:30:44.153028+0000) 2024-06-03T03:30:54.482 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:30:53 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:30:53.982+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:26.454415+0000 front 2024-06-03T03:30:26.454587+0000 (oldest deadline 2024-06-03T03:30:52.354304+0000) 2024-06-03T03:30:54.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:54 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:54.456+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:02.742111+0000 front 2024-06-03T03:30:02.742070+0000 (oldest deadline 2024-06-03T03:30:26.241792+0000) 2024-06-03T03:30:54.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:54 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:54.456+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.742816+0000 front 2024-06-03T03:30:09.742795+0000 (oldest deadline 2024-06-03T03:30:33.243036+0000) 2024-06-03T03:30:54.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:54 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:54.456+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:18.444645+0000 front 2024-06-03T03:30:18.444706+0000 (oldest deadline 2024-06-03T03:30:41.944716+0000) 2024-06-03T03:30:54.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:54 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:54.456+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.446009+0000 front 2024-06-03T03:30:25.445806+0000 (oldest deadline 2024-06-03T03:30:50.046043+0000) 2024-06-03T03:30:54.932 INFO:journalctl@ceph.osd.6.smithi117.stdout:Jun 03 03:30:54 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-6[57519]: 2024-06-03T03:30:54.501+0000 7f50fe600700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:11.044622+0000 front 2024-06-03T03:30:11.044544+0000 (oldest deadline 2024-06-03T03:30:33.944525+0000) 2024-06-03T03:30:54.932 INFO:journalctl@ceph.osd.6.smithi117.stdout:Jun 03 03:30:54 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-6[57519]: 2024-06-03T03:30:54.501+0000 7f50fe600700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:17.446468+0000 front 2024-06-03T03:30:17.446276+0000 (oldest deadline 2024-06-03T03:30:43.345900+0000) 2024-06-03T03:30:54.932 INFO:journalctl@ceph.osd.6.smithi117.stdout:Jun 03 03:30:54 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-6[57519]: 2024-06-03T03:30:54.501+0000 7f50fe600700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.648273+0000 front 2024-06-03T03:30:25.648528+0000 (oldest deadline 2024-06-03T03:30:50.347948+0000) 2024-06-03T03:30:54.945 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:30:54 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:30:54.694+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:01.379911+0000 front 2024-06-03T03:30:01.380025+0000 (oldest deadline 2024-06-03T03:30:25.479720+0000) 2024-06-03T03:30:54.945 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:30:54 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:30:54.694+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:11.881579+0000 front 2024-06-03T03:30:11.881609+0000 (oldest deadline 2024-06-03T03:30:37.781638+0000) 2024-06-03T03:30:54.945 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:30:54 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:30:54.694+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:17.782146+0000 front 2024-06-03T03:30:17.782047+0000 (oldest deadline 2024-06-03T03:30:41.882207+0000) 2024-06-03T03:30:55.196 INFO:journalctl@ceph.osd.6.smithi117.stdout:Jun 03 03:30:54 smithi117 podman[98653]: 2024-06-03 03:30:54.985538678 +0000 UTC m=+5.607080155 container died 85b94ed18f709179b562ac5855ef55e445250909d1fab5aaff899b45dea16541 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-6, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-06-03T03:30:55.197 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:30:54 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:30:54.931+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.254189+0000 front 2024-06-03T03:30:09.253879+0000 (oldest deadline 2024-06-03T03:30:33.353839+0000) 2024-06-03T03:30:55.197 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:30:54 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:30:54.931+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:16.855414+0000 front 2024-06-03T03:30:16.855488+0000 (oldest deadline 2024-06-03T03:30:42.755658+0000) 2024-06-03T03:30:55.197 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:30:54 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:30:54.931+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.657633+0000 front 2024-06-03T03:30:25.658290+0000 (oldest deadline 2024-06-03T03:30:50.357531+0000) 2024-06-03T03:30:55.231 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:30:54 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:30:54.945+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:06.650666+0000 front 2024-06-03T03:30:06.650530+0000 (oldest deadline 2024-06-03T03:30:33.051199+0000) 2024-06-03T03:30:55.231 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:30:54 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:30:54.945+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:19.452818+0000 front 2024-06-03T03:30:19.452796+0000 (oldest deadline 2024-06-03T03:30:44.153028+0000) 2024-06-03T03:30:55.232 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:30:54 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:30:54.945+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:26.454415+0000 front 2024-06-03T03:30:26.454587+0000 (oldest deadline 2024-06-03T03:30:52.354304+0000) 2024-06-03T03:30:55.406 INFO:journalctl@ceph.osd.7.smithi151.stdout:Jun 03 03:30:55 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-7[55035]: 2024-06-03T03:30:55.098+0000 7f8e5df2b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:08.069472+0000 front 2024-06-03T03:30:08.069491+0000 (oldest deadline 2024-06-03T03:30:32.769733+0000) 2024-06-03T03:30:55.407 INFO:journalctl@ceph.osd.7.smithi151.stdout:Jun 03 03:30:55 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-7[55035]: 2024-06-03T03:30:55.098+0000 7f8e5df2b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:20.371699+0000 front 2024-06-03T03:30:20.371774+0000 (oldest deadline 2024-06-03T03:30:45.671759+0000) 2024-06-03T03:30:55.407 INFO:journalctl@ceph.osd.7.smithi151.stdout:Jun 03 03:30:55 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-7[55035]: 2024-06-03T03:30:55.098+0000 7f8e5df2b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.672527+0000 front 2024-06-03T03:30:25.672429+0000 (oldest deadline 2024-06-03T03:30:50.972365+0000) 2024-06-03T03:30:55.696 INFO:journalctl@ceph.osd.6.smithi117.stdout:Jun 03 03:30:55 smithi117 podman[98653]: 2024-06-03 03:30:55.565162102 +0000 UTC m=+6.186703619 container cleanup 85b94ed18f709179b562ac5855ef55e445250909d1fab5aaff899b45dea16541 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-6, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528) 2024-06-03T03:30:55.697 INFO:journalctl@ceph.osd.6.smithi117.stdout:Jun 03 03:30:55 smithi117 bash[98653]: ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-6 2024-06-03T03:30:55.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:55 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:55.418+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:02.742111+0000 front 2024-06-03T03:30:02.742070+0000 (oldest deadline 2024-06-03T03:30:26.241792+0000) 2024-06-03T03:30:55.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:55 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:55.418+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.742816+0000 front 2024-06-03T03:30:09.742795+0000 (oldest deadline 2024-06-03T03:30:33.243036+0000) 2024-06-03T03:30:55.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:55 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:55.418+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:18.444645+0000 front 2024-06-03T03:30:18.444706+0000 (oldest deadline 2024-06-03T03:30:41.944716+0000) 2024-06-03T03:30:55.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:55 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:55.418+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.446009+0000 front 2024-06-03T03:30:25.445806+0000 (oldest deadline 2024-06-03T03:30:50.046043+0000) 2024-06-03T03:30:55.981 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:30:55 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:30:55.942+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:06.650666+0000 front 2024-06-03T03:30:06.650530+0000 (oldest deadline 2024-06-03T03:30:33.051199+0000) 2024-06-03T03:30:55.982 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:30:55 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:30:55.942+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:19.452818+0000 front 2024-06-03T03:30:19.452796+0000 (oldest deadline 2024-06-03T03:30:44.153028+0000) 2024-06-03T03:30:55.982 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:30:55 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:30:55.942+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:26.454415+0000 front 2024-06-03T03:30:26.454587+0000 (oldest deadline 2024-06-03T03:30:52.354304+0000) 2024-06-03T03:30:55.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:30:55 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:30:55.711+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:01.379911+0000 front 2024-06-03T03:30:01.380025+0000 (oldest deadline 2024-06-03T03:30:25.479720+0000) 2024-06-03T03:30:55.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:30:55 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:30:55.711+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:11.881579+0000 front 2024-06-03T03:30:11.881609+0000 (oldest deadline 2024-06-03T03:30:37.781638+0000) 2024-06-03T03:30:55.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:30:55 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:30:55.711+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:17.782146+0000 front 2024-06-03T03:30:17.782047+0000 (oldest deadline 2024-06-03T03:30:41.882207+0000) 2024-06-03T03:30:56.196 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:30:55 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:30:55.910+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.254189+0000 front 2024-06-03T03:30:09.253879+0000 (oldest deadline 2024-06-03T03:30:33.353839+0000) 2024-06-03T03:30:56.197 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:30:55 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:30:55.910+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:16.855414+0000 front 2024-06-03T03:30:16.855488+0000 (oldest deadline 2024-06-03T03:30:42.755658+0000) 2024-06-03T03:30:56.197 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:30:55 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:30:55.910+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.657633+0000 front 2024-06-03T03:30:25.658290+0000 (oldest deadline 2024-06-03T03:30:50.357531+0000) 2024-06-03T03:30:56.197 INFO:journalctl@ceph.osd.6.smithi117.stdout:Jun 03 03:30:55 smithi117 podman[98672]: 2024-06-03 03:30:55.841723367 +0000 UTC m=+0.832868446 container remove 85b94ed18f709179b562ac5855ef55e445250909d1fab5aaff899b45dea16541 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-6, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-06-03T03:30:56.406 INFO:journalctl@ceph.osd.7.smithi151.stdout:Jun 03 03:30:56 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-7[55035]: 2024-06-03T03:30:56.145+0000 7f8e5df2b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:08.069472+0000 front 2024-06-03T03:30:08.069491+0000 (oldest deadline 2024-06-03T03:30:32.769733+0000) 2024-06-03T03:30:56.406 INFO:journalctl@ceph.osd.7.smithi151.stdout:Jun 03 03:30:56 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-7[55035]: 2024-06-03T03:30:56.145+0000 7f8e5df2b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:20.371699+0000 front 2024-06-03T03:30:20.371774+0000 (oldest deadline 2024-06-03T03:30:45.671759+0000) 2024-06-03T03:30:56.406 INFO:journalctl@ceph.osd.7.smithi151.stdout:Jun 03 03:30:56 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-7[55035]: 2024-06-03T03:30:56.145+0000 7f8e5df2b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.672527+0000 front 2024-06-03T03:30:25.672429+0000 (oldest deadline 2024-06-03T03:30:50.972365+0000) 2024-06-03T03:30:56.899 INFO:journalctl@ceph.osd.6.smithi117.stdout:Jun 03 03:30:56 smithi117 podman[98760]: 2024-06-03 03:30:56.512392702 +0000 UTC m=+0.032363580 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T03:30:56.899 INFO:journalctl@ceph.osd.6.smithi117.stdout:Jun 03 03:30:56 smithi117 podman[98760]: 2024-06-03 03:30:56.642538804 +0000 UTC m=+0.162509704 container create faff0763debf5459632ed36a3bde60db8616fb19ad8d88f92adc1f9a75ff7bb6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-03T03:30:56.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:56 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:56.446+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:02.742111+0000 front 2024-06-03T03:30:02.742070+0000 (oldest deadline 2024-06-03T03:30:26.241792+0000) 2024-06-03T03:30:56.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:56 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:56.446+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.742816+0000 front 2024-06-03T03:30:09.742795+0000 (oldest deadline 2024-06-03T03:30:33.243036+0000) 2024-06-03T03:30:56.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:56 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:56.446+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:18.444645+0000 front 2024-06-03T03:30:18.444706+0000 (oldest deadline 2024-06-03T03:30:41.944716+0000) 2024-06-03T03:30:56.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:56 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:56.446+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.446009+0000 front 2024-06-03T03:30:25.445806+0000 (oldest deadline 2024-06-03T03:30:50.046043+0000) 2024-06-03T03:30:56.947 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:30:56 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:30:56.675+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:01.379911+0000 front 2024-06-03T03:30:01.380025+0000 (oldest deadline 2024-06-03T03:30:25.479720+0000) 2024-06-03T03:30:56.947 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:30:56 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:30:56.675+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:11.881579+0000 front 2024-06-03T03:30:11.881609+0000 (oldest deadline 2024-06-03T03:30:37.781638+0000) 2024-06-03T03:30:56.947 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:30:56 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:30:56.675+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:17.782146+0000 front 2024-06-03T03:30:17.782047+0000 (oldest deadline 2024-06-03T03:30:41.882207+0000) 2024-06-03T03:30:57.196 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:30:56 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:30:56.897+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.254189+0000 front 2024-06-03T03:30:09.253879+0000 (oldest deadline 2024-06-03T03:30:33.353839+0000) 2024-06-03T03:30:57.196 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:30:56 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:30:56.897+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:16.855414+0000 front 2024-06-03T03:30:16.855488+0000 (oldest deadline 2024-06-03T03:30:42.755658+0000) 2024-06-03T03:30:57.196 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:30:56 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:30:56.897+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.657633+0000 front 2024-06-03T03:30:25.658290+0000 (oldest deadline 2024-06-03T03:30:50.357531+0000) 2024-06-03T03:30:57.197 INFO:journalctl@ceph.osd.6.smithi117.stdout:Jun 03 03:30:56 smithi117 podman[98760]: 2024-06-03 03:30:56.946685478 +0000 UTC m=+0.466656405 container init faff0763debf5459632ed36a3bde60db8616fb19ad8d88f92adc1f9a75ff7bb6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-6-deactivate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-03T03:30:57.197 INFO:journalctl@ceph.osd.6.smithi117.stdout:Jun 03 03:30:56 smithi117 podman[98760]: 2024-06-03 03:30:56.957421516 +0000 UTC m=+0.477392414 container start faff0763debf5459632ed36a3bde60db8616fb19ad8d88f92adc1f9a75ff7bb6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-6-deactivate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc) 2024-06-03T03:30:57.197 INFO:journalctl@ceph.osd.6.smithi117.stdout:Jun 03 03:30:57 smithi117 podman[98760]: 2024-06-03 03:30:57.103515949 +0000 UTC m=+0.623486849 container attach faff0763debf5459632ed36a3bde60db8616fb19ad8d88f92adc1f9a75ff7bb6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-6-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-03T03:30:57.231 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:30:56 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:30:56.946+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:06.650666+0000 front 2024-06-03T03:30:06.650530+0000 (oldest deadline 2024-06-03T03:30:33.051199+0000) 2024-06-03T03:30:57.232 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:30:56 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:30:56.946+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:19.452818+0000 front 2024-06-03T03:30:19.452796+0000 (oldest deadline 2024-06-03T03:30:44.153028+0000) 2024-06-03T03:30:57.232 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:30:56 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:30:56.946+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:26.454415+0000 front 2024-06-03T03:30:26.454587+0000 (oldest deadline 2024-06-03T03:30:52.354304+0000) 2024-06-03T03:30:57.431 INFO:journalctl@ceph.osd.7.smithi151.stdout:Jun 03 03:30:57 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-7[55035]: 2024-06-03T03:30:57.166+0000 7f8e5df2b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:08.069472+0000 front 2024-06-03T03:30:08.069491+0000 (oldest deadline 2024-06-03T03:30:32.769733+0000) 2024-06-03T03:30:57.431 INFO:journalctl@ceph.osd.7.smithi151.stdout:Jun 03 03:30:57 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-7[55035]: 2024-06-03T03:30:57.166+0000 7f8e5df2b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:20.371699+0000 front 2024-06-03T03:30:20.371774+0000 (oldest deadline 2024-06-03T03:30:45.671759+0000) 2024-06-03T03:30:57.431 INFO:journalctl@ceph.osd.7.smithi151.stdout:Jun 03 03:30:57 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-7[55035]: 2024-06-03T03:30:57.166+0000 7f8e5df2b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.672527+0000 front 2024-06-03T03:30:25.672429+0000 (oldest deadline 2024-06-03T03:30:50.972365+0000) 2024-06-03T03:30:57.696 INFO:journalctl@ceph.osd.6.smithi117.stdout:Jun 03 03:30:57 smithi117 podman[98786]: 2024-06-03 03:30:57.453216691 +0000 UTC m=+0.032796067 container died faff0763debf5459632ed36a3bde60db8616fb19ad8d88f92adc1f9a75ff7bb6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-03T03:30:57.907 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:57 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:57.430+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:02.742111+0000 front 2024-06-03T03:30:02.742070+0000 (oldest deadline 2024-06-03T03:30:26.241792+0000) 2024-06-03T03:30:57.907 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:57 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:57.430+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.742816+0000 front 2024-06-03T03:30:09.742795+0000 (oldest deadline 2024-06-03T03:30:33.243036+0000) 2024-06-03T03:30:57.907 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:57 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:57.430+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:18.444645+0000 front 2024-06-03T03:30:18.444706+0000 (oldest deadline 2024-06-03T03:30:41.944716+0000) 2024-06-03T03:30:57.907 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:57 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:57.430+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.446009+0000 front 2024-06-03T03:30:25.445806+0000 (oldest deadline 2024-06-03T03:30:50.046043+0000) 2024-06-03T03:30:57.981 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:30:57 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:30:57.908+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:06.650666+0000 front 2024-06-03T03:30:06.650530+0000 (oldest deadline 2024-06-03T03:30:33.051199+0000) 2024-06-03T03:30:57.982 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:30:57 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:30:57.908+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:19.452818+0000 front 2024-06-03T03:30:19.452796+0000 (oldest deadline 2024-06-03T03:30:44.153028+0000) 2024-06-03T03:30:57.982 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:30:57 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:30:57.908+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:26.454415+0000 front 2024-06-03T03:30:26.454587+0000 (oldest deadline 2024-06-03T03:30:52.354304+0000) 2024-06-03T03:30:57.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:30:57 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:30:57.662+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:01.379911+0000 front 2024-06-03T03:30:01.380025+0000 (oldest deadline 2024-06-03T03:30:25.479720+0000) 2024-06-03T03:30:57.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:30:57 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:30:57.662+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:11.881579+0000 front 2024-06-03T03:30:11.881609+0000 (oldest deadline 2024-06-03T03:30:37.781638+0000) 2024-06-03T03:30:57.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:30:57 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:30:57.662+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:17.782146+0000 front 2024-06-03T03:30:17.782047+0000 (oldest deadline 2024-06-03T03:30:41.882207+0000) 2024-06-03T03:30:58.174 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:30:57 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:30:57.934+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.254189+0000 front 2024-06-03T03:30:09.253879+0000 (oldest deadline 2024-06-03T03:30:33.353839+0000) 2024-06-03T03:30:58.174 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:30:57 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:30:57.934+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:16.855414+0000 front 2024-06-03T03:30:16.855488+0000 (oldest deadline 2024-06-03T03:30:42.755658+0000) 2024-06-03T03:30:58.175 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:30:57 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:30:57.934+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.657633+0000 front 2024-06-03T03:30:25.658290+0000 (oldest deadline 2024-06-03T03:30:50.357531+0000) 2024-06-03T03:30:58.220 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@osd.6.service' 2024-06-03T03:30:58.323 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T03:30:58.324 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-06-03T03:30:58.324 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-06-03T03:30:58.324 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@osd.7 2024-06-03T03:30:58.388 INFO:journalctl@ceph.osd.7.smithi151.stdout:Jun 03 03:30:58 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-7[55035]: 2024-06-03T03:30:58.130+0000 7f8e5df2b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:08.069472+0000 front 2024-06-03T03:30:08.069491+0000 (oldest deadline 2024-06-03T03:30:32.769733+0000) 2024-06-03T03:30:58.389 INFO:journalctl@ceph.osd.7.smithi151.stdout:Jun 03 03:30:58 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-7[55035]: 2024-06-03T03:30:58.130+0000 7f8e5df2b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:20.371699+0000 front 2024-06-03T03:30:20.371774+0000 (oldest deadline 2024-06-03T03:30:45.671759+0000) 2024-06-03T03:30:58.389 INFO:journalctl@ceph.osd.7.smithi151.stdout:Jun 03 03:30:58 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-7[55035]: 2024-06-03T03:30:58.130+0000 7f8e5df2b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.672527+0000 front 2024-06-03T03:30:25.672429+0000 (oldest deadline 2024-06-03T03:30:50.972365+0000) 2024-06-03T03:30:58.389 INFO:journalctl@ceph.osd.7.smithi151.stdout:Jun 03 03:30:58 smithi151 systemd[1]: Stopping Ceph osd.7 for a0e8473a-2156-11ef-bc9b-c7b262605968... 2024-06-03T03:30:58.639 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:58 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:58.388+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:02.742111+0000 front 2024-06-03T03:30:02.742070+0000 (oldest deadline 2024-06-03T03:30:26.241792+0000) 2024-06-03T03:30:58.640 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:58 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:58.388+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.742816+0000 front 2024-06-03T03:30:09.742795+0000 (oldest deadline 2024-06-03T03:30:33.243036+0000) 2024-06-03T03:30:58.640 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:58 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:58.388+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:18.444645+0000 front 2024-06-03T03:30:18.444706+0000 (oldest deadline 2024-06-03T03:30:41.944716+0000) 2024-06-03T03:30:58.640 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:58 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:58.388+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.446009+0000 front 2024-06-03T03:30:25.445806+0000 (oldest deadline 2024-06-03T03:30:50.046043+0000) 2024-06-03T03:30:58.894 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:30:58 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:30:58.635+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:01.379911+0000 front 2024-06-03T03:30:01.380025+0000 (oldest deadline 2024-06-03T03:30:25.479720+0000) 2024-06-03T03:30:58.894 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:30:58 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:30:58.635+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:11.881579+0000 front 2024-06-03T03:30:11.881609+0000 (oldest deadline 2024-06-03T03:30:37.781638+0000) 2024-06-03T03:30:58.894 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:30:58 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:30:58.635+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:17.782146+0000 front 2024-06-03T03:30:17.782047+0000 (oldest deadline 2024-06-03T03:30:41.882207+0000) 2024-06-03T03:30:58.906 INFO:journalctl@ceph.osd.7.smithi151.stdout:Jun 03 03:30:58 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-7[55035]: 2024-06-03T03:30:58.639+0000 7f8e6593f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T03:30:58.906 INFO:journalctl@ceph.osd.7.smithi151.stdout:Jun 03 03:30:58 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-7[55035]: 2024-06-03T03:30:58.639+0000 7f8e6593f700 -1 osd.7 103 *** Got signal Terminated *** 2024-06-03T03:30:58.906 INFO:journalctl@ceph.osd.7.smithi151.stdout:Jun 03 03:30:58 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-7[55035]: 2024-06-03T03:30:58.639+0000 7f8e6593f700 -1 osd.7 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-03T03:30:59.197 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:30:58 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:30:58.938+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.254189+0000 front 2024-06-03T03:30:09.253879+0000 (oldest deadline 2024-06-03T03:30:33.353839+0000) 2024-06-03T03:30:59.197 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:30:58 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:30:58.938+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:16.855414+0000 front 2024-06-03T03:30:16.855488+0000 (oldest deadline 2024-06-03T03:30:42.755658+0000) 2024-06-03T03:30:59.197 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:30:58 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:30:58.938+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.657633+0000 front 2024-06-03T03:30:25.658290+0000 (oldest deadline 2024-06-03T03:30:50.357531+0000) 2024-06-03T03:30:59.197 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:30:58 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:30:58.938+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:35.659635+0000 front 2024-06-03T03:30:35.660018+0000 (oldest deadline 2024-06-03T03:30:58.559861+0000) 2024-06-03T03:30:59.231 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:30:58 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:30:58.894+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:06.650666+0000 front 2024-06-03T03:30:06.650530+0000 (oldest deadline 2024-06-03T03:30:33.051199+0000) 2024-06-03T03:30:59.232 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:30:58 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:30:58.894+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:19.452818+0000 front 2024-06-03T03:30:19.452796+0000 (oldest deadline 2024-06-03T03:30:44.153028+0000) 2024-06-03T03:30:59.232 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:30:58 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:30:58.894+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:26.454415+0000 front 2024-06-03T03:30:26.454587+0000 (oldest deadline 2024-06-03T03:30:52.354304+0000) 2024-06-03T03:30:59.406 INFO:journalctl@ceph.osd.7.smithi151.stdout:Jun 03 03:30:59 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-7[55035]: 2024-06-03T03:30:59.139+0000 7f8e5df2b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:08.069472+0000 front 2024-06-03T03:30:08.069491+0000 (oldest deadline 2024-06-03T03:30:32.769733+0000) 2024-06-03T03:30:59.406 INFO:journalctl@ceph.osd.7.smithi151.stdout:Jun 03 03:30:59 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-7[55035]: 2024-06-03T03:30:59.139+0000 7f8e5df2b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:20.371699+0000 front 2024-06-03T03:30:20.371774+0000 (oldest deadline 2024-06-03T03:30:45.671759+0000) 2024-06-03T03:30:59.406 INFO:journalctl@ceph.osd.7.smithi151.stdout:Jun 03 03:30:59 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-7[55035]: 2024-06-03T03:30:59.139+0000 7f8e5df2b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.672527+0000 front 2024-06-03T03:30:25.672429+0000 (oldest deadline 2024-06-03T03:30:50.972365+0000) 2024-06-03T03:30:59.893 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:30:59 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:30:59.589+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:01.379911+0000 front 2024-06-03T03:30:01.380025+0000 (oldest deadline 2024-06-03T03:30:25.479720+0000) 2024-06-03T03:30:59.894 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:30:59 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:30:59.589+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:11.881579+0000 front 2024-06-03T03:30:11.881609+0000 (oldest deadline 2024-06-03T03:30:37.781638+0000) 2024-06-03T03:30:59.894 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:30:59 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:30:59.589+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:17.782146+0000 front 2024-06-03T03:30:17.782047+0000 (oldest deadline 2024-06-03T03:30:41.882207+0000) 2024-06-03T03:30:59.894 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:30:59 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:30:59.589+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:37.585191+0000 front 2024-06-03T03:30:37.585172+0000 (oldest deadline 2024-06-03T03:30:58.685181+0000) 2024-06-03T03:30:59.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:59 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:59.417+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:02.742111+0000 front 2024-06-03T03:30:02.742070+0000 (oldest deadline 2024-06-03T03:30:26.241792+0000) 2024-06-03T03:30:59.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:59 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:59.417+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.742816+0000 front 2024-06-03T03:30:09.742795+0000 (oldest deadline 2024-06-03T03:30:33.243036+0000) 2024-06-03T03:30:59.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:59 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:59.418+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:18.444645+0000 front 2024-06-03T03:30:18.444706+0000 (oldest deadline 2024-06-03T03:30:41.944716+0000) 2024-06-03T03:30:59.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:30:59 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:30:59.418+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.446009+0000 front 2024-06-03T03:30:25.445806+0000 (oldest deadline 2024-06-03T03:30:50.046043+0000) 2024-06-03T03:31:00.196 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:30:59 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:30:59.924+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.254189+0000 front 2024-06-03T03:30:09.253879+0000 (oldest deadline 2024-06-03T03:30:33.353839+0000) 2024-06-03T03:31:00.197 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:30:59 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:30:59.924+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:16.855414+0000 front 2024-06-03T03:30:16.855488+0000 (oldest deadline 2024-06-03T03:30:42.755658+0000) 2024-06-03T03:31:00.197 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:30:59 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:30:59.924+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.657633+0000 front 2024-06-03T03:30:25.658290+0000 (oldest deadline 2024-06-03T03:30:50.357531+0000) 2024-06-03T03:31:00.197 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:30:59 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:30:59.924+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:35.659635+0000 front 2024-06-03T03:30:35.660018+0000 (oldest deadline 2024-06-03T03:30:58.559861+0000) 2024-06-03T03:31:00.231 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:30:59 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:30:59.893+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:06.650666+0000 front 2024-06-03T03:30:06.650530+0000 (oldest deadline 2024-06-03T03:30:33.051199+0000) 2024-06-03T03:31:00.232 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:30:59 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:30:59.893+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:19.452818+0000 front 2024-06-03T03:30:19.452796+0000 (oldest deadline 2024-06-03T03:30:44.153028+0000) 2024-06-03T03:31:00.232 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:30:59 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:30:59.893+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:26.454415+0000 front 2024-06-03T03:30:26.454587+0000 (oldest deadline 2024-06-03T03:30:52.354304+0000) 2024-06-03T03:31:00.406 INFO:journalctl@ceph.osd.7.smithi151.stdout:Jun 03 03:31:00 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-7[55035]: 2024-06-03T03:31:00.113+0000 7f8e5df2b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:08.069472+0000 front 2024-06-03T03:30:08.069491+0000 (oldest deadline 2024-06-03T03:30:32.769733+0000) 2024-06-03T03:31:00.406 INFO:journalctl@ceph.osd.7.smithi151.stdout:Jun 03 03:31:00 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-7[55035]: 2024-06-03T03:31:00.113+0000 7f8e5df2b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:20.371699+0000 front 2024-06-03T03:30:20.371774+0000 (oldest deadline 2024-06-03T03:30:45.671759+0000) 2024-06-03T03:31:00.406 INFO:journalctl@ceph.osd.7.smithi151.stdout:Jun 03 03:31:00 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-7[55035]: 2024-06-03T03:31:00.113+0000 7f8e5df2b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.672527+0000 front 2024-06-03T03:30:25.672429+0000 (oldest deadline 2024-06-03T03:30:50.972365+0000) 2024-06-03T03:31:00.406 INFO:journalctl@ceph.osd.7.smithi151.stdout:Jun 03 03:31:00 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-7[55035]: 2024-06-03T03:31:00.113+0000 7f8e5df2b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:36.873180+0000 front 2024-06-03T03:30:36.873250+0000 (oldest deadline 2024-06-03T03:30:59.773270+0000) 2024-06-03T03:31:00.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:00 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:00.459+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:02.742111+0000 front 2024-06-03T03:30:02.742070+0000 (oldest deadline 2024-06-03T03:30:26.241792+0000) 2024-06-03T03:31:00.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:00 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:00.459+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.742816+0000 front 2024-06-03T03:30:09.742795+0000 (oldest deadline 2024-06-03T03:30:33.243036+0000) 2024-06-03T03:31:00.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:00 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:00.459+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:18.444645+0000 front 2024-06-03T03:30:18.444706+0000 (oldest deadline 2024-06-03T03:30:41.944716+0000) 2024-06-03T03:31:00.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:00 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:00.459+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.446009+0000 front 2024-06-03T03:30:25.445806+0000 (oldest deadline 2024-06-03T03:30:50.046043+0000) 2024-06-03T03:31:00.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:00 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:00.459+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:37.647415+0000 front 2024-06-03T03:30:37.647453+0000 (oldest deadline 2024-06-03T03:30:59.947552+0000) 2024-06-03T03:31:00.919 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:00 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:00.628+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:01.379911+0000 front 2024-06-03T03:30:01.380025+0000 (oldest deadline 2024-06-03T03:30:25.479720+0000) 2024-06-03T03:31:00.919 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:00 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:00.628+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:11.881579+0000 front 2024-06-03T03:30:11.881609+0000 (oldest deadline 2024-06-03T03:30:37.781638+0000) 2024-06-03T03:31:00.919 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:00 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:00.628+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:17.782146+0000 front 2024-06-03T03:30:17.782047+0000 (oldest deadline 2024-06-03T03:30:41.882207+0000) 2024-06-03T03:31:00.919 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:00 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:00.628+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:37.585191+0000 front 2024-06-03T03:30:37.585172+0000 (oldest deadline 2024-06-03T03:30:58.685181+0000) 2024-06-03T03:31:01.196 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:00 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:00.908+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.254189+0000 front 2024-06-03T03:30:09.253879+0000 (oldest deadline 2024-06-03T03:30:33.353839+0000) 2024-06-03T03:31:01.197 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:00 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:00.908+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:16.855414+0000 front 2024-06-03T03:30:16.855488+0000 (oldest deadline 2024-06-03T03:30:42.755658+0000) 2024-06-03T03:31:01.197 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:00 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:00.908+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.657633+0000 front 2024-06-03T03:30:25.658290+0000 (oldest deadline 2024-06-03T03:30:50.357531+0000) 2024-06-03T03:31:01.197 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:00 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:00.908+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:35.659635+0000 front 2024-06-03T03:30:35.660018+0000 (oldest deadline 2024-06-03T03:30:58.559861+0000) 2024-06-03T03:31:01.231 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:31:00 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:31:00.919+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:06.650666+0000 front 2024-06-03T03:30:06.650530+0000 (oldest deadline 2024-06-03T03:30:33.051199+0000) 2024-06-03T03:31:01.232 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:31:00 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:31:00.919+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:19.452818+0000 front 2024-06-03T03:30:19.452796+0000 (oldest deadline 2024-06-03T03:30:44.153028+0000) 2024-06-03T03:31:01.232 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:31:00 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:31:00.919+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:26.454415+0000 front 2024-06-03T03:30:26.454587+0000 (oldest deadline 2024-06-03T03:30:52.354304+0000) 2024-06-03T03:31:01.466 INFO:journalctl@ceph.osd.7.smithi151.stdout:Jun 03 03:31:01 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-7[55035]: 2024-06-03T03:31:01.161+0000 7f8e5df2b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:08.069472+0000 front 2024-06-03T03:30:08.069491+0000 (oldest deadline 2024-06-03T03:30:32.769733+0000) 2024-06-03T03:31:01.466 INFO:journalctl@ceph.osd.7.smithi151.stdout:Jun 03 03:31:01 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-7[55035]: 2024-06-03T03:31:01.161+0000 7f8e5df2b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:20.371699+0000 front 2024-06-03T03:30:20.371774+0000 (oldest deadline 2024-06-03T03:30:45.671759+0000) 2024-06-03T03:31:01.467 INFO:journalctl@ceph.osd.7.smithi151.stdout:Jun 03 03:31:01 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-7[55035]: 2024-06-03T03:31:01.161+0000 7f8e5df2b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.672527+0000 front 2024-06-03T03:30:25.672429+0000 (oldest deadline 2024-06-03T03:30:50.972365+0000) 2024-06-03T03:31:01.467 INFO:journalctl@ceph.osd.7.smithi151.stdout:Jun 03 03:31:01 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-7[55035]: 2024-06-03T03:31:01.161+0000 7f8e5df2b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:36.873180+0000 front 2024-06-03T03:30:36.873250+0000 (oldest deadline 2024-06-03T03:30:59.773270+0000) 2024-06-03T03:31:01.887 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:01 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:01.613+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:01.379911+0000 front 2024-06-03T03:30:01.380025+0000 (oldest deadline 2024-06-03T03:30:25.479720+0000) 2024-06-03T03:31:01.888 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:01 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:01.613+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:11.881579+0000 front 2024-06-03T03:30:11.881609+0000 (oldest deadline 2024-06-03T03:30:37.781638+0000) 2024-06-03T03:31:01.888 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:01 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:01.613+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:17.782146+0000 front 2024-06-03T03:30:17.782047+0000 (oldest deadline 2024-06-03T03:30:41.882207+0000) 2024-06-03T03:31:01.888 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:01 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:01.613+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:37.585191+0000 front 2024-06-03T03:30:37.585172+0000 (oldest deadline 2024-06-03T03:30:58.685181+0000) 2024-06-03T03:31:01.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:01 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:01.466+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:02.742111+0000 front 2024-06-03T03:30:02.742070+0000 (oldest deadline 2024-06-03T03:30:26.241792+0000) 2024-06-03T03:31:01.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:01 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:01.466+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.742816+0000 front 2024-06-03T03:30:09.742795+0000 (oldest deadline 2024-06-03T03:30:33.243036+0000) 2024-06-03T03:31:01.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:01 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:01.466+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:18.444645+0000 front 2024-06-03T03:30:18.444706+0000 (oldest deadline 2024-06-03T03:30:41.944716+0000) 2024-06-03T03:31:01.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:01 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:01.466+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.446009+0000 front 2024-06-03T03:30:25.445806+0000 (oldest deadline 2024-06-03T03:30:50.046043+0000) 2024-06-03T03:31:01.907 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:01 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:01.466+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:37.647415+0000 front 2024-06-03T03:30:37.647453+0000 (oldest deadline 2024-06-03T03:30:59.947552+0000) 2024-06-03T03:31:02.196 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:01 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:01.921+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.254189+0000 front 2024-06-03T03:30:09.253879+0000 (oldest deadline 2024-06-03T03:30:33.353839+0000) 2024-06-03T03:31:02.196 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:01 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:01.921+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:16.855414+0000 front 2024-06-03T03:30:16.855488+0000 (oldest deadline 2024-06-03T03:30:42.755658+0000) 2024-06-03T03:31:02.197 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:01 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:01.921+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.657633+0000 front 2024-06-03T03:30:25.658290+0000 (oldest deadline 2024-06-03T03:30:50.357531+0000) 2024-06-03T03:31:02.197 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:01 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:01.921+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:35.659635+0000 front 2024-06-03T03:30:35.660018+0000 (oldest deadline 2024-06-03T03:30:58.559861+0000) 2024-06-03T03:31:02.232 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:31:01 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:31:01.888+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:06.650666+0000 front 2024-06-03T03:30:06.650530+0000 (oldest deadline 2024-06-03T03:30:33.051199+0000) 2024-06-03T03:31:02.232 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:31:01 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:31:01.888+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:19.452818+0000 front 2024-06-03T03:30:19.452796+0000 (oldest deadline 2024-06-03T03:30:44.153028+0000) 2024-06-03T03:31:02.232 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:31:01 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:31:01.888+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:26.454415+0000 front 2024-06-03T03:30:26.454587+0000 (oldest deadline 2024-06-03T03:30:52.354304+0000) 2024-06-03T03:31:02.232 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:31:01 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:31:01.888+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:34.655361+0000 front 2024-06-03T03:30:34.655377+0000 (oldest deadline 2024-06-03T03:31:01.655600+0000) 2024-06-03T03:31:02.494 INFO:journalctl@ceph.osd.7.smithi151.stdout:Jun 03 03:31:02 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-7[55035]: 2024-06-03T03:31:02.168+0000 7f8e5df2b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:08.069472+0000 front 2024-06-03T03:30:08.069491+0000 (oldest deadline 2024-06-03T03:30:32.769733+0000) 2024-06-03T03:31:02.494 INFO:journalctl@ceph.osd.7.smithi151.stdout:Jun 03 03:31:02 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-7[55035]: 2024-06-03T03:31:02.168+0000 7f8e5df2b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:20.371699+0000 front 2024-06-03T03:30:20.371774+0000 (oldest deadline 2024-06-03T03:30:45.671759+0000) 2024-06-03T03:31:02.494 INFO:journalctl@ceph.osd.7.smithi151.stdout:Jun 03 03:31:02 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-7[55035]: 2024-06-03T03:31:02.168+0000 7f8e5df2b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.672527+0000 front 2024-06-03T03:30:25.672429+0000 (oldest deadline 2024-06-03T03:30:50.972365+0000) 2024-06-03T03:31:02.494 INFO:journalctl@ceph.osd.7.smithi151.stdout:Jun 03 03:31:02 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-7[55035]: 2024-06-03T03:31:02.168+0000 7f8e5df2b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:36.873180+0000 front 2024-06-03T03:30:36.873250+0000 (oldest deadline 2024-06-03T03:30:59.773270+0000) 2024-06-03T03:31:02.900 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:02 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:02.624+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:01.379911+0000 front 2024-06-03T03:30:01.380025+0000 (oldest deadline 2024-06-03T03:30:25.479720+0000) 2024-06-03T03:31:02.900 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:02 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:02.624+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:11.881579+0000 front 2024-06-03T03:30:11.881609+0000 (oldest deadline 2024-06-03T03:30:37.781638+0000) 2024-06-03T03:31:02.900 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:02 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:02.624+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:17.782146+0000 front 2024-06-03T03:30:17.782047+0000 (oldest deadline 2024-06-03T03:30:41.882207+0000) 2024-06-03T03:31:02.900 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:02 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:02.624+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:37.585191+0000 front 2024-06-03T03:30:37.585172+0000 (oldest deadline 2024-06-03T03:30:58.685181+0000) 2024-06-03T03:31:02.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:02 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:02.494+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:02.742111+0000 front 2024-06-03T03:30:02.742070+0000 (oldest deadline 2024-06-03T03:30:26.241792+0000) 2024-06-03T03:31:02.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:02 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:02.494+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.742816+0000 front 2024-06-03T03:30:09.742795+0000 (oldest deadline 2024-06-03T03:30:33.243036+0000) 2024-06-03T03:31:02.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:02 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:02.494+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:18.444645+0000 front 2024-06-03T03:30:18.444706+0000 (oldest deadline 2024-06-03T03:30:41.944716+0000) 2024-06-03T03:31:02.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:02 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:02.494+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.446009+0000 front 2024-06-03T03:30:25.445806+0000 (oldest deadline 2024-06-03T03:30:50.046043+0000) 2024-06-03T03:31:02.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:02 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:02.494+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:37.647415+0000 front 2024-06-03T03:30:37.647453+0000 (oldest deadline 2024-06-03T03:30:59.947552+0000) 2024-06-03T03:31:03.196 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:02 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:02.892+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.254189+0000 front 2024-06-03T03:30:09.253879+0000 (oldest deadline 2024-06-03T03:30:33.353839+0000) 2024-06-03T03:31:03.197 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:02 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:02.892+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:16.855414+0000 front 2024-06-03T03:30:16.855488+0000 (oldest deadline 2024-06-03T03:30:42.755658+0000) 2024-06-03T03:31:03.197 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:02 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:02.892+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.657633+0000 front 2024-06-03T03:30:25.658290+0000 (oldest deadline 2024-06-03T03:30:50.357531+0000) 2024-06-03T03:31:03.197 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:02 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:02.892+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:35.659635+0000 front 2024-06-03T03:30:35.660018+0000 (oldest deadline 2024-06-03T03:30:58.559861+0000) 2024-06-03T03:31:03.234 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:31:02 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:31:02.900+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:06.650666+0000 front 2024-06-03T03:30:06.650530+0000 (oldest deadline 2024-06-03T03:30:33.051199+0000) 2024-06-03T03:31:03.235 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:31:02 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:31:02.900+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:19.452818+0000 front 2024-06-03T03:30:19.452796+0000 (oldest deadline 2024-06-03T03:30:44.153028+0000) 2024-06-03T03:31:03.235 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:31:02 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:31:02.900+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:26.454415+0000 front 2024-06-03T03:30:26.454587+0000 (oldest deadline 2024-06-03T03:30:52.354304+0000) 2024-06-03T03:31:03.235 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:31:02 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:31:02.900+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:34.655361+0000 front 2024-06-03T03:30:34.655377+0000 (oldest deadline 2024-06-03T03:31:01.655600+0000) 2024-06-03T03:31:03.475 INFO:journalctl@ceph.osd.7.smithi151.stdout:Jun 03 03:31:03 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-7[55035]: 2024-06-03T03:31:03.200+0000 7f8e5df2b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:08.069472+0000 front 2024-06-03T03:30:08.069491+0000 (oldest deadline 2024-06-03T03:30:32.769733+0000) 2024-06-03T03:31:03.476 INFO:journalctl@ceph.osd.7.smithi151.stdout:Jun 03 03:31:03 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-7[55035]: 2024-06-03T03:31:03.200+0000 7f8e5df2b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:20.371699+0000 front 2024-06-03T03:30:20.371774+0000 (oldest deadline 2024-06-03T03:30:45.671759+0000) 2024-06-03T03:31:03.476 INFO:journalctl@ceph.osd.7.smithi151.stdout:Jun 03 03:31:03 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-7[55035]: 2024-06-03T03:31:03.200+0000 7f8e5df2b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.672527+0000 front 2024-06-03T03:30:25.672429+0000 (oldest deadline 2024-06-03T03:30:50.972365+0000) 2024-06-03T03:31:03.476 INFO:journalctl@ceph.osd.7.smithi151.stdout:Jun 03 03:31:03 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-7[55035]: 2024-06-03T03:31:03.200+0000 7f8e5df2b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:36.873180+0000 front 2024-06-03T03:30:36.873250+0000 (oldest deadline 2024-06-03T03:30:59.773270+0000) 2024-06-03T03:31:03.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:03 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:03.474+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:02.742111+0000 front 2024-06-03T03:30:02.742070+0000 (oldest deadline 2024-06-03T03:30:26.241792+0000) 2024-06-03T03:31:03.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:03 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:03.474+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.742816+0000 front 2024-06-03T03:30:09.742795+0000 (oldest deadline 2024-06-03T03:30:33.243036+0000) 2024-06-03T03:31:03.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:03 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:03.474+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:18.444645+0000 front 2024-06-03T03:30:18.444706+0000 (oldest deadline 2024-06-03T03:30:41.944716+0000) 2024-06-03T03:31:03.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:03 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:03.475+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.446009+0000 front 2024-06-03T03:30:25.445806+0000 (oldest deadline 2024-06-03T03:30:50.046043+0000) 2024-06-03T03:31:03.907 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:03 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:03.475+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:37.647415+0000 front 2024-06-03T03:30:37.647453+0000 (oldest deadline 2024-06-03T03:30:59.947552+0000) 2024-06-03T03:31:03.917 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:03 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:03.618+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:01.379911+0000 front 2024-06-03T03:30:01.380025+0000 (oldest deadline 2024-06-03T03:30:25.479720+0000) 2024-06-03T03:31:03.918 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:03 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:03.618+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:11.881579+0000 front 2024-06-03T03:30:11.881609+0000 (oldest deadline 2024-06-03T03:30:37.781638+0000) 2024-06-03T03:31:03.918 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:03 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:03.618+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:17.782146+0000 front 2024-06-03T03:30:17.782047+0000 (oldest deadline 2024-06-03T03:30:41.882207+0000) 2024-06-03T03:31:03.918 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:03 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:03.618+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:37.585191+0000 front 2024-06-03T03:30:37.585172+0000 (oldest deadline 2024-06-03T03:30:58.685181+0000) 2024-06-03T03:31:04.196 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:03 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:03.932+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.254189+0000 front 2024-06-03T03:30:09.253879+0000 (oldest deadline 2024-06-03T03:30:33.353839+0000) 2024-06-03T03:31:04.197 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:03 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:03.932+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:16.855414+0000 front 2024-06-03T03:30:16.855488+0000 (oldest deadline 2024-06-03T03:30:42.755658+0000) 2024-06-03T03:31:04.197 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:03 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:03.932+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.657633+0000 front 2024-06-03T03:30:25.658290+0000 (oldest deadline 2024-06-03T03:30:50.357531+0000) 2024-06-03T03:31:04.197 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:03 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:03.932+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:35.659635+0000 front 2024-06-03T03:30:35.660018+0000 (oldest deadline 2024-06-03T03:30:58.559861+0000) 2024-06-03T03:31:04.231 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:31:03 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:31:03.918+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:06.650666+0000 front 2024-06-03T03:30:06.650530+0000 (oldest deadline 2024-06-03T03:30:33.051199+0000) 2024-06-03T03:31:04.231 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:31:03 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:31:03.918+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:19.452818+0000 front 2024-06-03T03:30:19.452796+0000 (oldest deadline 2024-06-03T03:30:44.153028+0000) 2024-06-03T03:31:04.231 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:31:03 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:31:03.918+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:26.454415+0000 front 2024-06-03T03:30:26.454587+0000 (oldest deadline 2024-06-03T03:30:52.354304+0000) 2024-06-03T03:31:04.232 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:31:03 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:31:03.918+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:34.655361+0000 front 2024-06-03T03:30:34.655377+0000 (oldest deadline 2024-06-03T03:31:01.655600+0000) 2024-06-03T03:31:04.424 INFO:journalctl@ceph.osd.7.smithi151.stdout:Jun 03 03:31:04 smithi151 podman[71655]: 2024-06-03 03:31:04.162278637 +0000 UTC m=+5.646229381 container died 3872c2f9de02d4a76cb792a296eca02b3843fa92c6e84904a90bb5309862da54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-7, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, io.buildah.version=1.33.5) 2024-06-03T03:31:04.799 INFO:journalctl@ceph.osd.7.smithi151.stdout:Jun 03 03:31:04 smithi151 podman[71655]: 2024-06-03 03:31:04.515288602 +0000 UTC m=+5.999239346 container cleanup 3872c2f9de02d4a76cb792a296eca02b3843fa92c6e84904a90bb5309862da54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-7, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True) 2024-06-03T03:31:04.800 INFO:journalctl@ceph.osd.7.smithi151.stdout:Jun 03 03:31:04 smithi151 bash[71655]: ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-7 2024-06-03T03:31:04.800 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:04 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:04.470+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:02.742111+0000 front 2024-06-03T03:30:02.742070+0000 (oldest deadline 2024-06-03T03:30:26.241792+0000) 2024-06-03T03:31:04.800 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:04 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:04.470+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.742816+0000 front 2024-06-03T03:30:09.742795+0000 (oldest deadline 2024-06-03T03:30:33.243036+0000) 2024-06-03T03:31:04.800 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:04 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:04.470+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:18.444645+0000 front 2024-06-03T03:30:18.444706+0000 (oldest deadline 2024-06-03T03:30:41.944716+0000) 2024-06-03T03:31:04.800 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:04 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:04.470+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.446009+0000 front 2024-06-03T03:30:25.445806+0000 (oldest deadline 2024-06-03T03:30:50.046043+0000) 2024-06-03T03:31:04.800 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:04 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:04.470+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:37.647415+0000 front 2024-06-03T03:30:37.647453+0000 (oldest deadline 2024-06-03T03:30:59.947552+0000) 2024-06-03T03:31:04.893 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:04 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:04.641+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:01.379911+0000 front 2024-06-03T03:30:01.380025+0000 (oldest deadline 2024-06-03T03:30:25.479720+0000) 2024-06-03T03:31:04.893 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:04 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:04.641+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:11.881579+0000 front 2024-06-03T03:30:11.881609+0000 (oldest deadline 2024-06-03T03:30:37.781638+0000) 2024-06-03T03:31:04.893 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:04 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:04.641+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:17.782146+0000 front 2024-06-03T03:30:17.782047+0000 (oldest deadline 2024-06-03T03:30:41.882207+0000) 2024-06-03T03:31:04.893 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:04 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:04.641+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:37.585191+0000 front 2024-06-03T03:30:37.585172+0000 (oldest deadline 2024-06-03T03:30:58.685181+0000) 2024-06-03T03:31:05.132 INFO:journalctl@ceph.osd.7.smithi151.stdout:Jun 03 03:31:04 smithi151 podman[71672]: 2024-06-03 03:31:04.800080652 +0000 UTC m=+0.634770149 container remove 3872c2f9de02d4a76cb792a296eca02b3843fa92c6e84904a90bb5309862da54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-06-03T03:31:05.232 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:31:04 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:31:04.893+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:06.650666+0000 front 2024-06-03T03:30:06.650530+0000 (oldest deadline 2024-06-03T03:30:33.051199+0000) 2024-06-03T03:31:05.232 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:31:04 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:31:04.893+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:19.452818+0000 front 2024-06-03T03:30:19.452796+0000 (oldest deadline 2024-06-03T03:30:44.153028+0000) 2024-06-03T03:31:05.232 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:31:04 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:31:04.893+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:26.454415+0000 front 2024-06-03T03:30:26.454587+0000 (oldest deadline 2024-06-03T03:30:52.354304+0000) 2024-06-03T03:31:05.232 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:31:04 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:31:04.893+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:34.655361+0000 front 2024-06-03T03:30:34.655377+0000 (oldest deadline 2024-06-03T03:31:01.655600+0000) 2024-06-03T03:31:05.406 INFO:journalctl@ceph.osd.7.smithi151.stdout:Jun 03 03:31:05 smithi151 podman[71760]: 2024-06-03 03:31:05.035350714 +0000 UTC m=+0.017649104 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T03:31:05.407 INFO:journalctl@ceph.osd.7.smithi151.stdout:Jun 03 03:31:05 smithi151 podman[71760]: 2024-06-03 03:31:05.142852459 +0000 UTC m=+0.125150892 container create 85a483c92e5759009a5d6e2b58b226e8c4efed0caecf678d6163d7e67f78219d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-03T03:31:05.446 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:04 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:04.959+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.254189+0000 front 2024-06-03T03:30:09.253879+0000 (oldest deadline 2024-06-03T03:30:33.353839+0000) 2024-06-03T03:31:05.447 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:04 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:04.959+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:16.855414+0000 front 2024-06-03T03:30:16.855488+0000 (oldest deadline 2024-06-03T03:30:42.755658+0000) 2024-06-03T03:31:05.447 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:04 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:04.959+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.657633+0000 front 2024-06-03T03:30:25.658290+0000 (oldest deadline 2024-06-03T03:30:50.357531+0000) 2024-06-03T03:31:05.447 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:04 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:04.959+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:35.659635+0000 front 2024-06-03T03:30:35.660018+0000 (oldest deadline 2024-06-03T03:30:58.559861+0000) 2024-06-03T03:31:05.706 INFO:journalctl@ceph.osd.7.smithi151.stdout:Jun 03 03:31:05 smithi151 podman[71760]: 2024-06-03 03:31:05.413105144 +0000 UTC m=+0.395403526 container init 85a483c92e5759009a5d6e2b58b226e8c4efed0caecf678d6163d7e67f78219d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-03T03:31:05.706 INFO:journalctl@ceph.osd.7.smithi151.stdout:Jun 03 03:31:05 smithi151 podman[71760]: 2024-06-03 03:31:05.416625404 +0000 UTC m=+0.398923794 container start 85a483c92e5759009a5d6e2b58b226e8c4efed0caecf678d6163d7e67f78219d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=) 2024-06-03T03:31:05.707 INFO:journalctl@ceph.osd.7.smithi151.stdout:Jun 03 03:31:05 smithi151 podman[71760]: 2024-06-03 03:31:05.521316631 +0000 UTC m=+0.503615032 container attach 85a483c92e5759009a5d6e2b58b226e8c4efed0caecf678d6163d7e67f78219d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2) 2024-06-03T03:31:05.707 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:05 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:05.445+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:02.742111+0000 front 2024-06-03T03:30:02.742070+0000 (oldest deadline 2024-06-03T03:30:26.241792+0000) 2024-06-03T03:31:05.707 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:05 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:05.445+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.742816+0000 front 2024-06-03T03:30:09.742795+0000 (oldest deadline 2024-06-03T03:30:33.243036+0000) 2024-06-03T03:31:05.707 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:05 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:05.445+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:18.444645+0000 front 2024-06-03T03:30:18.444706+0000 (oldest deadline 2024-06-03T03:30:41.944716+0000) 2024-06-03T03:31:05.707 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:05 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:05.445+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.446009+0000 front 2024-06-03T03:30:25.445806+0000 (oldest deadline 2024-06-03T03:30:50.046043+0000) 2024-06-03T03:31:05.707 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:05 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:05.445+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:37.647415+0000 front 2024-06-03T03:30:37.647453+0000 (oldest deadline 2024-06-03T03:30:59.947552+0000) 2024-06-03T03:31:05.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:05 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:05.660+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:01.379911+0000 front 2024-06-03T03:30:01.380025+0000 (oldest deadline 2024-06-03T03:30:25.479720+0000) 2024-06-03T03:31:05.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:05 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:05.660+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:11.881579+0000 front 2024-06-03T03:30:11.881609+0000 (oldest deadline 2024-06-03T03:30:37.781638+0000) 2024-06-03T03:31:05.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:05 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:05.660+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:17.782146+0000 front 2024-06-03T03:30:17.782047+0000 (oldest deadline 2024-06-03T03:30:41.882207+0000) 2024-06-03T03:31:05.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:05 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:05.660+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:37.585191+0000 front 2024-06-03T03:30:37.585172+0000 (oldest deadline 2024-06-03T03:30:58.685181+0000) 2024-06-03T03:31:05.982 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:31:05 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:31:05.852+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:06.650666+0000 front 2024-06-03T03:30:06.650530+0000 (oldest deadline 2024-06-03T03:30:33.051199+0000) 2024-06-03T03:31:05.982 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:31:05 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:31:05.852+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:19.452818+0000 front 2024-06-03T03:30:19.452796+0000 (oldest deadline 2024-06-03T03:30:44.153028+0000) 2024-06-03T03:31:05.982 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:31:05 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:31:05.852+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:26.454415+0000 front 2024-06-03T03:30:26.454587+0000 (oldest deadline 2024-06-03T03:30:52.354304+0000) 2024-06-03T03:31:05.983 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:31:05 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:31:05.852+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:34.655361+0000 front 2024-06-03T03:30:34.655377+0000 (oldest deadline 2024-06-03T03:31:01.655600+0000) 2024-06-03T03:31:06.050 INFO:journalctl@ceph.osd.7.smithi151.stdout:Jun 03 03:31:05 smithi151 podman[71760]: 2024-06-03 03:31:05.706308849 +0000 UTC m=+0.688607243 container died 85a483c92e5759009a5d6e2b58b226e8c4efed0caecf678d6163d7e67f78219d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-7-deactivate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528) 2024-06-03T03:31:06.378 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@osd.7.service' 2024-06-03T03:31:06.418 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T03:31:06.418 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-06-03T03:31:06.418 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-06-03T03:31:06.418 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@osd.8 2024-06-03T03:31:06.446 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:05 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:05.983+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.254189+0000 front 2024-06-03T03:30:09.253879+0000 (oldest deadline 2024-06-03T03:30:33.353839+0000) 2024-06-03T03:31:06.446 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:05 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:05.983+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:16.855414+0000 front 2024-06-03T03:30:16.855488+0000 (oldest deadline 2024-06-03T03:30:42.755658+0000) 2024-06-03T03:31:06.447 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:05 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:05.983+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.657633+0000 front 2024-06-03T03:30:25.658290+0000 (oldest deadline 2024-06-03T03:30:50.357531+0000) 2024-06-03T03:31:06.447 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:05 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:05.983+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:35.659635+0000 front 2024-06-03T03:30:35.660018+0000 (oldest deadline 2024-06-03T03:30:58.559861+0000) 2024-06-03T03:31:06.632 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:06 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:06.434+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:02.742111+0000 front 2024-06-03T03:30:02.742070+0000 (oldest deadline 2024-06-03T03:30:26.241792+0000) 2024-06-03T03:31:06.633 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:06 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:06.434+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.742816+0000 front 2024-06-03T03:30:09.742795+0000 (oldest deadline 2024-06-03T03:30:33.243036+0000) 2024-06-03T03:31:06.633 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:06 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:06.434+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:18.444645+0000 front 2024-06-03T03:30:18.444706+0000 (oldest deadline 2024-06-03T03:30:41.944716+0000) 2024-06-03T03:31:06.633 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:06 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:06.434+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.446009+0000 front 2024-06-03T03:30:25.445806+0000 (oldest deadline 2024-06-03T03:30:50.046043+0000) 2024-06-03T03:31:06.633 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:06 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:06.434+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:37.647415+0000 front 2024-06-03T03:30:37.647453+0000 (oldest deadline 2024-06-03T03:30:59.947552+0000) 2024-06-03T03:31:06.732 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:31:06 smithi183 systemd[1]: Stopping Ceph osd.8 for a0e8473a-2156-11ef-bc9b-c7b262605968... 2024-06-03T03:31:06.732 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:06 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:06.666+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:01.379911+0000 front 2024-06-03T03:30:01.380025+0000 (oldest deadline 2024-06-03T03:30:25.479720+0000) 2024-06-03T03:31:06.732 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:06 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:06.666+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:11.881579+0000 front 2024-06-03T03:30:11.881609+0000 (oldest deadline 2024-06-03T03:30:37.781638+0000) 2024-06-03T03:31:06.732 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:06 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:06.666+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:17.782146+0000 front 2024-06-03T03:30:17.782047+0000 (oldest deadline 2024-06-03T03:30:41.882207+0000) 2024-06-03T03:31:06.732 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:06 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:06.666+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:37.585191+0000 front 2024-06-03T03:30:37.585172+0000 (oldest deadline 2024-06-03T03:30:58.685181+0000) 2024-06-03T03:31:07.232 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:31:06 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:31:06.735+0000 7f416967d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T03:31:07.232 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:31:06 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:31:06.736+0000 7f416967d700 -1 osd.8 103 *** Got signal Terminated *** 2024-06-03T03:31:07.232 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:31:06 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:31:06.736+0000 7f416967d700 -1 osd.8 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-03T03:31:07.232 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:31:06 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:31:06.837+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:06.650666+0000 front 2024-06-03T03:30:06.650530+0000 (oldest deadline 2024-06-03T03:30:33.051199+0000) 2024-06-03T03:31:07.232 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:31:06 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:31:06.837+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:19.452818+0000 front 2024-06-03T03:30:19.452796+0000 (oldest deadline 2024-06-03T03:30:44.153028+0000) 2024-06-03T03:31:07.232 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:31:06 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:31:06.837+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:26.454415+0000 front 2024-06-03T03:30:26.454587+0000 (oldest deadline 2024-06-03T03:30:52.354304+0000) 2024-06-03T03:31:07.232 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:31:06 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:31:06.837+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:34.655361+0000 front 2024-06-03T03:30:34.655377+0000 (oldest deadline 2024-06-03T03:31:01.655600+0000) 2024-06-03T03:31:07.446 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:06 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:06.949+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.254189+0000 front 2024-06-03T03:30:09.253879+0000 (oldest deadline 2024-06-03T03:30:33.353839+0000) 2024-06-03T03:31:07.447 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:06 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:06.949+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:16.855414+0000 front 2024-06-03T03:30:16.855488+0000 (oldest deadline 2024-06-03T03:30:42.755658+0000) 2024-06-03T03:31:07.447 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:06 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:06.949+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.657633+0000 front 2024-06-03T03:30:25.658290+0000 (oldest deadline 2024-06-03T03:30:50.357531+0000) 2024-06-03T03:31:07.447 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:06 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:06.949+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:35.659635+0000 front 2024-06-03T03:30:35.660018+0000 (oldest deadline 2024-06-03T03:30:58.559861+0000) 2024-06-03T03:31:07.656 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:07 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:07.386+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:02.742111+0000 front 2024-06-03T03:30:02.742070+0000 (oldest deadline 2024-06-03T03:30:26.241792+0000) 2024-06-03T03:31:07.656 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:07 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:07.386+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.742816+0000 front 2024-06-03T03:30:09.742795+0000 (oldest deadline 2024-06-03T03:30:33.243036+0000) 2024-06-03T03:31:07.656 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:07 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:07.386+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:18.444645+0000 front 2024-06-03T03:30:18.444706+0000 (oldest deadline 2024-06-03T03:30:41.944716+0000) 2024-06-03T03:31:07.656 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:07 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:07.386+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.446009+0000 front 2024-06-03T03:30:25.445806+0000 (oldest deadline 2024-06-03T03:30:50.046043+0000) 2024-06-03T03:31:07.656 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:07 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:07.386+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:37.647415+0000 front 2024-06-03T03:30:37.647453+0000 (oldest deadline 2024-06-03T03:30:59.947552+0000) 2024-06-03T03:31:07.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:07 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:07.674+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:01.379911+0000 front 2024-06-03T03:30:01.380025+0000 (oldest deadline 2024-06-03T03:30:25.479720+0000) 2024-06-03T03:31:07.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:07 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:07.674+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:11.881579+0000 front 2024-06-03T03:30:11.881609+0000 (oldest deadline 2024-06-03T03:30:37.781638+0000) 2024-06-03T03:31:07.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:07 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:07.674+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:17.782146+0000 front 2024-06-03T03:30:17.782047+0000 (oldest deadline 2024-06-03T03:30:41.882207+0000) 2024-06-03T03:31:07.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:07 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:07.674+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:37.585191+0000 front 2024-06-03T03:30:37.585172+0000 (oldest deadline 2024-06-03T03:30:58.685181+0000) 2024-06-03T03:31:07.982 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:31:07 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:31:07.837+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:06.650666+0000 front 2024-06-03T03:30:06.650530+0000 (oldest deadline 2024-06-03T03:30:33.051199+0000) 2024-06-03T03:31:07.982 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:31:07 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:31:07.837+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:19.452818+0000 front 2024-06-03T03:30:19.452796+0000 (oldest deadline 2024-06-03T03:30:44.153028+0000) 2024-06-03T03:31:07.982 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:31:07 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:31:07.837+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:26.454415+0000 front 2024-06-03T03:30:26.454587+0000 (oldest deadline 2024-06-03T03:30:52.354304+0000) 2024-06-03T03:31:07.983 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:31:07 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:31:07.837+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:34.655361+0000 front 2024-06-03T03:30:34.655377+0000 (oldest deadline 2024-06-03T03:31:01.655600+0000) 2024-06-03T03:31:08.446 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:07 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:07.993+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.254189+0000 front 2024-06-03T03:30:09.253879+0000 (oldest deadline 2024-06-03T03:30:33.353839+0000) 2024-06-03T03:31:08.447 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:07 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:07.993+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:16.855414+0000 front 2024-06-03T03:30:16.855488+0000 (oldest deadline 2024-06-03T03:30:42.755658+0000) 2024-06-03T03:31:08.447 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:07 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:07.993+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.657633+0000 front 2024-06-03T03:30:25.658290+0000 (oldest deadline 2024-06-03T03:30:50.357531+0000) 2024-06-03T03:31:08.447 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:07 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:07.993+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:35.659635+0000 front 2024-06-03T03:30:35.660018+0000 (oldest deadline 2024-06-03T03:30:58.559861+0000) 2024-06-03T03:31:08.656 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:08 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:08.374+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:02.742111+0000 front 2024-06-03T03:30:02.742070+0000 (oldest deadline 2024-06-03T03:30:26.241792+0000) 2024-06-03T03:31:08.656 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:08 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:08.374+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.742816+0000 front 2024-06-03T03:30:09.742795+0000 (oldest deadline 2024-06-03T03:30:33.243036+0000) 2024-06-03T03:31:08.656 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:08 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:08.374+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:18.444645+0000 front 2024-06-03T03:30:18.444706+0000 (oldest deadline 2024-06-03T03:30:41.944716+0000) 2024-06-03T03:31:08.656 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:08 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:08.374+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.446009+0000 front 2024-06-03T03:30:25.445806+0000 (oldest deadline 2024-06-03T03:30:50.046043+0000) 2024-06-03T03:31:08.657 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:08 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:08.374+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:37.647415+0000 front 2024-06-03T03:30:37.647453+0000 (oldest deadline 2024-06-03T03:30:59.947552+0000) 2024-06-03T03:31:08.657 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:08 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:08.374+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-06-03T03:30:45.749157+0000 front 2024-06-03T03:30:45.749116+0000 (oldest deadline 2024-06-03T03:31:08.049043+0000) 2024-06-03T03:31:08.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:08 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:08.651+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:01.379911+0000 front 2024-06-03T03:30:01.380025+0000 (oldest deadline 2024-06-03T03:30:25.479720+0000) 2024-06-03T03:31:08.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:08 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:08.651+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:11.881579+0000 front 2024-06-03T03:30:11.881609+0000 (oldest deadline 2024-06-03T03:30:37.781638+0000) 2024-06-03T03:31:08.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:08 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:08.651+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:17.782146+0000 front 2024-06-03T03:30:17.782047+0000 (oldest deadline 2024-06-03T03:30:41.882207+0000) 2024-06-03T03:31:08.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:08 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:08.651+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:37.585191+0000 front 2024-06-03T03:30:37.585172+0000 (oldest deadline 2024-06-03T03:30:58.685181+0000) 2024-06-03T03:31:08.982 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:31:08 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:31:08.840+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:06.650666+0000 front 2024-06-03T03:30:06.650530+0000 (oldest deadline 2024-06-03T03:30:33.051199+0000) 2024-06-03T03:31:08.982 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:31:08 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:31:08.840+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:19.452818+0000 front 2024-06-03T03:30:19.452796+0000 (oldest deadline 2024-06-03T03:30:44.153028+0000) 2024-06-03T03:31:08.982 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:31:08 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:31:08.840+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:26.454415+0000 front 2024-06-03T03:30:26.454587+0000 (oldest deadline 2024-06-03T03:30:52.354304+0000) 2024-06-03T03:31:08.982 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:31:08 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:31:08.840+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:34.655361+0000 front 2024-06-03T03:30:34.655377+0000 (oldest deadline 2024-06-03T03:31:01.655600+0000) 2024-06-03T03:31:08.982 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:31:08 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:31:08.840+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-06-03T03:30:41.655868+0000 front 2024-06-03T03:30:41.655911+0000 (oldest deadline 2024-06-03T03:31:08.656453+0000) 2024-06-03T03:31:09.446 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:08 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:08.956+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.254189+0000 front 2024-06-03T03:30:09.253879+0000 (oldest deadline 2024-06-03T03:30:33.353839+0000) 2024-06-03T03:31:09.447 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:08 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:08.956+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:16.855414+0000 front 2024-06-03T03:30:16.855488+0000 (oldest deadline 2024-06-03T03:30:42.755658+0000) 2024-06-03T03:31:09.447 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:08 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:08.956+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.657633+0000 front 2024-06-03T03:30:25.658290+0000 (oldest deadline 2024-06-03T03:30:50.357531+0000) 2024-06-03T03:31:09.447 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:08 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:08.956+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:35.659635+0000 front 2024-06-03T03:30:35.660018+0000 (oldest deadline 2024-06-03T03:30:58.559861+0000) 2024-06-03T03:31:09.656 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:09 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:09.397+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:02.742111+0000 front 2024-06-03T03:30:02.742070+0000 (oldest deadline 2024-06-03T03:30:26.241792+0000) 2024-06-03T03:31:09.656 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:09 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:09.397+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.742816+0000 front 2024-06-03T03:30:09.742795+0000 (oldest deadline 2024-06-03T03:30:33.243036+0000) 2024-06-03T03:31:09.657 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:09 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:09.397+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:18.444645+0000 front 2024-06-03T03:30:18.444706+0000 (oldest deadline 2024-06-03T03:30:41.944716+0000) 2024-06-03T03:31:09.657 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:09 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:09.397+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.446009+0000 front 2024-06-03T03:30:25.445806+0000 (oldest deadline 2024-06-03T03:30:50.046043+0000) 2024-06-03T03:31:09.657 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:09 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:09.397+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:37.647415+0000 front 2024-06-03T03:30:37.647453+0000 (oldest deadline 2024-06-03T03:30:59.947552+0000) 2024-06-03T03:31:09.657 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:09 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:09.397+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-06-03T03:30:45.749157+0000 front 2024-06-03T03:30:45.749116+0000 (oldest deadline 2024-06-03T03:31:08.049043+0000) 2024-06-03T03:31:09.983 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:09 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:09.618+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:01.379911+0000 front 2024-06-03T03:30:01.380025+0000 (oldest deadline 2024-06-03T03:30:25.479720+0000) 2024-06-03T03:31:09.983 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:09 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:09.618+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:11.881579+0000 front 2024-06-03T03:30:11.881609+0000 (oldest deadline 2024-06-03T03:30:37.781638+0000) 2024-06-03T03:31:09.983 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:09 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:09.618+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:17.782146+0000 front 2024-06-03T03:30:17.782047+0000 (oldest deadline 2024-06-03T03:30:41.882207+0000) 2024-06-03T03:31:09.983 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:09 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:09.618+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:37.585191+0000 front 2024-06-03T03:30:37.585172+0000 (oldest deadline 2024-06-03T03:30:58.685181+0000) 2024-06-03T03:31:09.983 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:09 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:09.618+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-06-03T03:30:43.886858+0000 front 2024-06-03T03:30:43.886895+0000 (oldest deadline 2024-06-03T03:31:09.186984+0000) 2024-06-03T03:31:09.983 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:31:09 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:31:09.847+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:06.650666+0000 front 2024-06-03T03:30:06.650530+0000 (oldest deadline 2024-06-03T03:30:33.051199+0000) 2024-06-03T03:31:09.983 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:31:09 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:31:09.847+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:19.452818+0000 front 2024-06-03T03:30:19.452796+0000 (oldest deadline 2024-06-03T03:30:44.153028+0000) 2024-06-03T03:31:09.984 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:31:09 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:31:09.847+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:26.454415+0000 front 2024-06-03T03:30:26.454587+0000 (oldest deadline 2024-06-03T03:30:52.354304+0000) 2024-06-03T03:31:09.984 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:31:09 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:31:09.847+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:34.655361+0000 front 2024-06-03T03:30:34.655377+0000 (oldest deadline 2024-06-03T03:31:01.655600+0000) 2024-06-03T03:31:09.984 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:31:09 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:31:09.847+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-06-03T03:30:41.655868+0000 front 2024-06-03T03:30:41.655911+0000 (oldest deadline 2024-06-03T03:31:08.656453+0000) 2024-06-03T03:31:10.197 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:09 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:09.924+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.254189+0000 front 2024-06-03T03:30:09.253879+0000 (oldest deadline 2024-06-03T03:30:33.353839+0000) 2024-06-03T03:31:10.197 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:09 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:09.924+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:16.855414+0000 front 2024-06-03T03:30:16.855488+0000 (oldest deadline 2024-06-03T03:30:42.755658+0000) 2024-06-03T03:31:10.197 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:09 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:09.924+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.657633+0000 front 2024-06-03T03:30:25.658290+0000 (oldest deadline 2024-06-03T03:30:50.357531+0000) 2024-06-03T03:31:10.197 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:09 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:09.924+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:35.659635+0000 front 2024-06-03T03:30:35.660018+0000 (oldest deadline 2024-06-03T03:30:58.559861+0000) 2024-06-03T03:31:10.197 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:09 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:09.924+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-06-03T03:30:44.864033+0000 front 2024-06-03T03:30:44.864222+0000 (oldest deadline 2024-06-03T03:31:09.564167+0000) 2024-06-03T03:31:10.656 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:10 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:10.353+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:02.742111+0000 front 2024-06-03T03:30:02.742070+0000 (oldest deadline 2024-06-03T03:30:26.241792+0000) 2024-06-03T03:31:10.656 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:10 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:10.353+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.742816+0000 front 2024-06-03T03:30:09.742795+0000 (oldest deadline 2024-06-03T03:30:33.243036+0000) 2024-06-03T03:31:10.657 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:10 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:10.353+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:18.444645+0000 front 2024-06-03T03:30:18.444706+0000 (oldest deadline 2024-06-03T03:30:41.944716+0000) 2024-06-03T03:31:10.657 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:10 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:10.353+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.446009+0000 front 2024-06-03T03:30:25.445806+0000 (oldest deadline 2024-06-03T03:30:50.046043+0000) 2024-06-03T03:31:10.657 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:10 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:10.353+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:37.647415+0000 front 2024-06-03T03:30:37.647453+0000 (oldest deadline 2024-06-03T03:30:59.947552+0000) 2024-06-03T03:31:10.657 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:10 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:10.353+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-06-03T03:30:45.749157+0000 front 2024-06-03T03:30:45.749116+0000 (oldest deadline 2024-06-03T03:31:08.049043+0000) 2024-06-03T03:31:10.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:10 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:10.651+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:01.379911+0000 front 2024-06-03T03:30:01.380025+0000 (oldest deadline 2024-06-03T03:30:25.479720+0000) 2024-06-03T03:31:10.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:10 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:10.651+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:11.881579+0000 front 2024-06-03T03:30:11.881609+0000 (oldest deadline 2024-06-03T03:30:37.781638+0000) 2024-06-03T03:31:10.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:10 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:10.651+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:17.782146+0000 front 2024-06-03T03:30:17.782047+0000 (oldest deadline 2024-06-03T03:30:41.882207+0000) 2024-06-03T03:31:10.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:10 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:10.651+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:37.585191+0000 front 2024-06-03T03:30:37.585172+0000 (oldest deadline 2024-06-03T03:30:58.685181+0000) 2024-06-03T03:31:10.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:10 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:10.651+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-06-03T03:30:43.886858+0000 front 2024-06-03T03:30:43.886895+0000 (oldest deadline 2024-06-03T03:31:09.186984+0000) 2024-06-03T03:31:10.983 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:31:10 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:31:10.891+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:06.650666+0000 front 2024-06-03T03:30:06.650530+0000 (oldest deadline 2024-06-03T03:30:33.051199+0000) 2024-06-03T03:31:10.983 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:31:10 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:31:10.891+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:19.452818+0000 front 2024-06-03T03:30:19.452796+0000 (oldest deadline 2024-06-03T03:30:44.153028+0000) 2024-06-03T03:31:10.983 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:31:10 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:31:10.891+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:26.454415+0000 front 2024-06-03T03:30:26.454587+0000 (oldest deadline 2024-06-03T03:30:52.354304+0000) 2024-06-03T03:31:10.983 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:31:10 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:31:10.891+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:34.655361+0000 front 2024-06-03T03:30:34.655377+0000 (oldest deadline 2024-06-03T03:31:01.655600+0000) 2024-06-03T03:31:10.983 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:31:10 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8[55915]: 2024-06-03T03:31:10.891+0000 7f4161c69700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-06-03T03:30:41.655868+0000 front 2024-06-03T03:30:41.655911+0000 (oldest deadline 2024-06-03T03:31:08.656453+0000) 2024-06-03T03:31:11.197 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:10 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:10.941+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.254189+0000 front 2024-06-03T03:30:09.253879+0000 (oldest deadline 2024-06-03T03:30:33.353839+0000) 2024-06-03T03:31:11.197 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:10 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:10.941+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:16.855414+0000 front 2024-06-03T03:30:16.855488+0000 (oldest deadline 2024-06-03T03:30:42.755658+0000) 2024-06-03T03:31:11.197 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:10 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:10.941+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.657633+0000 front 2024-06-03T03:30:25.658290+0000 (oldest deadline 2024-06-03T03:30:50.357531+0000) 2024-06-03T03:31:11.197 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:10 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:10.941+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:35.659635+0000 front 2024-06-03T03:30:35.660018+0000 (oldest deadline 2024-06-03T03:30:58.559861+0000) 2024-06-03T03:31:11.197 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:10 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:10.941+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-06-03T03:30:44.864033+0000 front 2024-06-03T03:30:44.864222+0000 (oldest deadline 2024-06-03T03:31:09.564167+0000) 2024-06-03T03:31:11.656 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:11 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:11.389+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:02.742111+0000 front 2024-06-03T03:30:02.742070+0000 (oldest deadline 2024-06-03T03:30:26.241792+0000) 2024-06-03T03:31:11.657 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:11 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:11.389+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.742816+0000 front 2024-06-03T03:30:09.742795+0000 (oldest deadline 2024-06-03T03:30:33.243036+0000) 2024-06-03T03:31:11.657 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:11 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:11.389+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:18.444645+0000 front 2024-06-03T03:30:18.444706+0000 (oldest deadline 2024-06-03T03:30:41.944716+0000) 2024-06-03T03:31:11.657 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:11 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:11.389+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.446009+0000 front 2024-06-03T03:30:25.445806+0000 (oldest deadline 2024-06-03T03:30:50.046043+0000) 2024-06-03T03:31:11.657 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:11 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:11.389+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:37.647415+0000 front 2024-06-03T03:30:37.647453+0000 (oldest deadline 2024-06-03T03:30:59.947552+0000) 2024-06-03T03:31:11.657 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:11 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:11.389+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-06-03T03:30:45.749157+0000 front 2024-06-03T03:30:45.749116+0000 (oldest deadline 2024-06-03T03:31:08.049043+0000) 2024-06-03T03:31:11.981 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:11 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:11.644+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:01.379911+0000 front 2024-06-03T03:30:01.380025+0000 (oldest deadline 2024-06-03T03:30:25.479720+0000) 2024-06-03T03:31:11.981 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:11 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:11.644+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:11.881579+0000 front 2024-06-03T03:30:11.881609+0000 (oldest deadline 2024-06-03T03:30:37.781638+0000) 2024-06-03T03:31:11.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:11 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:11.644+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:17.782146+0000 front 2024-06-03T03:30:17.782047+0000 (oldest deadline 2024-06-03T03:30:41.882207+0000) 2024-06-03T03:31:11.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:11 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:11.644+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:37.585191+0000 front 2024-06-03T03:30:37.585172+0000 (oldest deadline 2024-06-03T03:30:58.685181+0000) 2024-06-03T03:31:11.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:11 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:11.644+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-06-03T03:30:43.886858+0000 front 2024-06-03T03:30:43.886895+0000 (oldest deadline 2024-06-03T03:31:09.186984+0000) 2024-06-03T03:31:12.446 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:11 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:11.974+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.254189+0000 front 2024-06-03T03:30:09.253879+0000 (oldest deadline 2024-06-03T03:30:33.353839+0000) 2024-06-03T03:31:12.447 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:11 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:11.974+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:16.855414+0000 front 2024-06-03T03:30:16.855488+0000 (oldest deadline 2024-06-03T03:30:42.755658+0000) 2024-06-03T03:31:12.447 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:11 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:11.974+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.657633+0000 front 2024-06-03T03:30:25.658290+0000 (oldest deadline 2024-06-03T03:30:50.357531+0000) 2024-06-03T03:31:12.447 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:11 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:11.974+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:35.659635+0000 front 2024-06-03T03:30:35.660018+0000 (oldest deadline 2024-06-03T03:30:58.559861+0000) 2024-06-03T03:31:12.447 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:11 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:11.974+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-06-03T03:30:44.864033+0000 front 2024-06-03T03:30:44.864222+0000 (oldest deadline 2024-06-03T03:31:09.564167+0000) 2024-06-03T03:31:12.481 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:31:12 smithi183 podman[72267]: 2024-06-03 03:31:12.169934677 +0000 UTC m=+5.560562392 container died a3da1a0d4954eb3ebae67416c4c16da8a90a2f6ff89caa8a05621217c29ed9a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8, GIT_CLEAN=True, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-03T03:31:12.656 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:12 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:12.397+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:02.742111+0000 front 2024-06-03T03:30:02.742070+0000 (oldest deadline 2024-06-03T03:30:26.241792+0000) 2024-06-03T03:31:12.656 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:12 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:12.397+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.742816+0000 front 2024-06-03T03:30:09.742795+0000 (oldest deadline 2024-06-03T03:30:33.243036+0000) 2024-06-03T03:31:12.656 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:12 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:12.397+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:18.444645+0000 front 2024-06-03T03:30:18.444706+0000 (oldest deadline 2024-06-03T03:30:41.944716+0000) 2024-06-03T03:31:12.656 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:12 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:12.397+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.446009+0000 front 2024-06-03T03:30:25.445806+0000 (oldest deadline 2024-06-03T03:30:50.046043+0000) 2024-06-03T03:31:12.657 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:12 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:12.397+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:37.647415+0000 front 2024-06-03T03:30:37.647453+0000 (oldest deadline 2024-06-03T03:30:59.947552+0000) 2024-06-03T03:31:12.657 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:12 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:12.397+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-06-03T03:30:45.749157+0000 front 2024-06-03T03:30:45.749116+0000 (oldest deadline 2024-06-03T03:31:08.049043+0000) 2024-06-03T03:31:12.981 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:31:12 smithi183 podman[72267]: 2024-06-03 03:31:12.510655195 +0000 UTC m=+5.901282911 container cleanup a3da1a0d4954eb3ebae67416c4c16da8a90a2f6ff89caa8a05621217c29ed9a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-03T03:31:12.981 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:31:12 smithi183 bash[72267]: ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8 2024-06-03T03:31:12.982 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:31:12 smithi183 podman[72285]: 2024-06-03 03:31:12.695324475 +0000 UTC m=+0.518515772 container remove a3da1a0d4954eb3ebae67416c4c16da8a90a2f6ff89caa8a05621217c29ed9a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-03T03:31:12.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:12 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:12.617+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:01.379911+0000 front 2024-06-03T03:30:01.380025+0000 (oldest deadline 2024-06-03T03:30:25.479720+0000) 2024-06-03T03:31:12.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:12 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:12.617+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:11.881579+0000 front 2024-06-03T03:30:11.881609+0000 (oldest deadline 2024-06-03T03:30:37.781638+0000) 2024-06-03T03:31:12.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:12 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:12.617+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:17.782146+0000 front 2024-06-03T03:30:17.782047+0000 (oldest deadline 2024-06-03T03:30:41.882207+0000) 2024-06-03T03:31:12.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:12 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:12.617+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:37.585191+0000 front 2024-06-03T03:30:37.585172+0000 (oldest deadline 2024-06-03T03:30:58.685181+0000) 2024-06-03T03:31:12.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:12 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:12.617+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-06-03T03:30:43.886858+0000 front 2024-06-03T03:30:43.886895+0000 (oldest deadline 2024-06-03T03:31:09.186984+0000) 2024-06-03T03:31:13.431 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:31:13 smithi183 podman[72374]: 2024-06-03 03:31:13.112834026 +0000 UTC m=+0.105010658 container create 382b4581c0ae6a0d84d39bd06cb0395a94b0c0b11c8c99301418654ac3d00649 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8-deactivate, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True) 2024-06-03T03:31:13.432 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:31:13 smithi183 podman[72374]: 2024-06-03 03:31:13.024846674 +0000 UTC m=+0.017023312 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T03:31:13.432 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:31:13 smithi183 podman[72374]: 2024-06-03 03:31:13.349088955 +0000 UTC m=+0.341265595 container init 382b4581c0ae6a0d84d39bd06cb0395a94b0c0b11c8c99301418654ac3d00649 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8-deactivate, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-03T03:31:13.432 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:31:13 smithi183 podman[72374]: 2024-06-03 03:31:13.352369473 +0000 UTC m=+0.344546107 container start 382b4581c0ae6a0d84d39bd06cb0395a94b0c0b11c8c99301418654ac3d00649 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8-deactivate, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-06-03T03:31:13.432 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:31:13 smithi183 podman[72374]: 2024-06-03 03:31:13.431875923 +0000 UTC m=+0.424052569 container attach 382b4581c0ae6a0d84d39bd06cb0395a94b0c0b11c8c99301418654ac3d00649 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-06-03T03:31:13.446 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:13 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:13.023+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.254189+0000 front 2024-06-03T03:30:09.253879+0000 (oldest deadline 2024-06-03T03:30:33.353839+0000) 2024-06-03T03:31:13.447 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:13 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:13.023+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:16.855414+0000 front 2024-06-03T03:30:16.855488+0000 (oldest deadline 2024-06-03T03:30:42.755658+0000) 2024-06-03T03:31:13.447 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:13 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:13.023+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.657633+0000 front 2024-06-03T03:30:25.658290+0000 (oldest deadline 2024-06-03T03:30:50.357531+0000) 2024-06-03T03:31:13.447 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:13 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:13.023+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:35.659635+0000 front 2024-06-03T03:30:35.660018+0000 (oldest deadline 2024-06-03T03:30:58.559861+0000) 2024-06-03T03:31:13.447 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:13 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:13.023+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-06-03T03:30:44.864033+0000 front 2024-06-03T03:30:44.864222+0000 (oldest deadline 2024-06-03T03:31:09.564167+0000) 2024-06-03T03:31:13.732 INFO:journalctl@ceph.osd.8.smithi183.stdout:Jun 03 03:31:13 smithi183 podman[72374]: 2024-06-03 03:31:13.639128827 +0000 UTC m=+0.631305467 container died 382b4581c0ae6a0d84d39bd06cb0395a94b0c0b11c8c99301418654ac3d00649 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS) 2024-06-03T03:31:13.732 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:13 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:13.605+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:01.379911+0000 front 2024-06-03T03:30:01.380025+0000 (oldest deadline 2024-06-03T03:30:25.479720+0000) 2024-06-03T03:31:13.732 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:13 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:13.605+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:11.881579+0000 front 2024-06-03T03:30:11.881609+0000 (oldest deadline 2024-06-03T03:30:37.781638+0000) 2024-06-03T03:31:13.732 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:13 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:13.605+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:17.782146+0000 front 2024-06-03T03:30:17.782047+0000 (oldest deadline 2024-06-03T03:30:41.882207+0000) 2024-06-03T03:31:13.732 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:13 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:13.605+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:37.585191+0000 front 2024-06-03T03:30:37.585172+0000 (oldest deadline 2024-06-03T03:30:58.685181+0000) 2024-06-03T03:31:13.732 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:13 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:13.605+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-06-03T03:30:43.886858+0000 front 2024-06-03T03:30:43.886895+0000 (oldest deadline 2024-06-03T03:31:09.186984+0000) 2024-06-03T03:31:13.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:13 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:13.437+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:02.742111+0000 front 2024-06-03T03:30:02.742070+0000 (oldest deadline 2024-06-03T03:30:26.241792+0000) 2024-06-03T03:31:13.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:13 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:13.437+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.742816+0000 front 2024-06-03T03:30:09.742795+0000 (oldest deadline 2024-06-03T03:30:33.243036+0000) 2024-06-03T03:31:13.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:13 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:13.437+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:18.444645+0000 front 2024-06-03T03:30:18.444706+0000 (oldest deadline 2024-06-03T03:30:41.944716+0000) 2024-06-03T03:31:13.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:13 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:13.437+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.446009+0000 front 2024-06-03T03:30:25.445806+0000 (oldest deadline 2024-06-03T03:30:50.046043+0000) 2024-06-03T03:31:13.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:13 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:13.437+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:37.647415+0000 front 2024-06-03T03:30:37.647453+0000 (oldest deadline 2024-06-03T03:30:59.947552+0000) 2024-06-03T03:31:13.907 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:13 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:13.437+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-06-03T03:30:45.749157+0000 front 2024-06-03T03:30:45.749116+0000 (oldest deadline 2024-06-03T03:31:08.049043+0000) 2024-06-03T03:31:14.292 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@osd.8.service' 2024-06-03T03:31:14.330 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T03:31:14.330 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-06-03T03:31:14.331 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-06-03T03:31:14.331 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@osd.9 2024-06-03T03:31:14.406 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:14 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:14.039+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.254189+0000 front 2024-06-03T03:30:09.253879+0000 (oldest deadline 2024-06-03T03:30:33.353839+0000) 2024-06-03T03:31:14.406 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:14 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:14.039+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:16.855414+0000 front 2024-06-03T03:30:16.855488+0000 (oldest deadline 2024-06-03T03:30:42.755658+0000) 2024-06-03T03:31:14.406 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:14 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:14.039+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.657633+0000 front 2024-06-03T03:30:25.658290+0000 (oldest deadline 2024-06-03T03:30:50.357531+0000) 2024-06-03T03:31:14.406 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:14 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:14.039+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:35.659635+0000 front 2024-06-03T03:30:35.660018+0000 (oldest deadline 2024-06-03T03:30:58.559861+0000) 2024-06-03T03:31:14.406 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:14 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:14.039+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-06-03T03:30:44.864033+0000 front 2024-06-03T03:30:44.864222+0000 (oldest deadline 2024-06-03T03:31:09.564167+0000) 2024-06-03T03:31:14.696 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:14 smithi117 systemd[1]: Stopping Ceph osd.9 for a0e8473a-2156-11ef-bc9b-c7b262605968... 2024-06-03T03:31:14.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:14 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:14.455+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:02.742111+0000 front 2024-06-03T03:30:02.742070+0000 (oldest deadline 2024-06-03T03:30:26.241792+0000) 2024-06-03T03:31:14.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:14 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:14.455+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.742816+0000 front 2024-06-03T03:30:09.742795+0000 (oldest deadline 2024-06-03T03:30:33.243036+0000) 2024-06-03T03:31:14.907 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:14 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:14.455+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:18.444645+0000 front 2024-06-03T03:30:18.444706+0000 (oldest deadline 2024-06-03T03:30:41.944716+0000) 2024-06-03T03:31:14.907 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:14 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:14.455+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.446009+0000 front 2024-06-03T03:30:25.445806+0000 (oldest deadline 2024-06-03T03:30:50.046043+0000) 2024-06-03T03:31:14.907 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:14 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:14.455+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:37.647415+0000 front 2024-06-03T03:30:37.647453+0000 (oldest deadline 2024-06-03T03:30:59.947552+0000) 2024-06-03T03:31:14.907 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:14 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:14.455+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-06-03T03:30:45.749157+0000 front 2024-06-03T03:30:45.749116+0000 (oldest deadline 2024-06-03T03:31:08.049043+0000) 2024-06-03T03:31:14.981 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:14 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:14.624+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:01.379911+0000 front 2024-06-03T03:30:01.380025+0000 (oldest deadline 2024-06-03T03:30:25.479720+0000) 2024-06-03T03:31:14.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:14 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:14.624+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:11.881579+0000 front 2024-06-03T03:30:11.881609+0000 (oldest deadline 2024-06-03T03:30:37.781638+0000) 2024-06-03T03:31:14.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:14 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:14.624+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:17.782146+0000 front 2024-06-03T03:30:17.782047+0000 (oldest deadline 2024-06-03T03:30:41.882207+0000) 2024-06-03T03:31:14.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:14 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:14.624+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:37.585191+0000 front 2024-06-03T03:30:37.585172+0000 (oldest deadline 2024-06-03T03:30:58.685181+0000) 2024-06-03T03:31:14.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:14 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:14.624+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-06-03T03:30:43.886858+0000 front 2024-06-03T03:30:43.886895+0000 (oldest deadline 2024-06-03T03:31:09.186984+0000) 2024-06-03T03:31:15.197 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:14 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:14.897+0000 7f91aea00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T03:31:15.197 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:14 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:14.897+0000 7f91aea00700 -1 osd.9 103 *** Got signal Terminated *** 2024-06-03T03:31:15.198 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:14 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:14.897+0000 7f91aea00700 -1 osd.9 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-03T03:31:15.198 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:15 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:15.070+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.254189+0000 front 2024-06-03T03:30:09.253879+0000 (oldest deadline 2024-06-03T03:30:33.353839+0000) 2024-06-03T03:31:15.198 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:15 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:15.070+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:16.855414+0000 front 2024-06-03T03:30:16.855488+0000 (oldest deadline 2024-06-03T03:30:42.755658+0000) 2024-06-03T03:31:15.198 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:15 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:15.070+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.657633+0000 front 2024-06-03T03:30:25.658290+0000 (oldest deadline 2024-06-03T03:30:50.357531+0000) 2024-06-03T03:31:15.198 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:15 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:15.070+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:35.659635+0000 front 2024-06-03T03:30:35.660018+0000 (oldest deadline 2024-06-03T03:30:58.559861+0000) 2024-06-03T03:31:15.198 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:15 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:15.070+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-06-03T03:30:44.864033+0000 front 2024-06-03T03:30:44.864222+0000 (oldest deadline 2024-06-03T03:31:09.564167+0000) 2024-06-03T03:31:15.732 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:15 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:15.595+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:01.379911+0000 front 2024-06-03T03:30:01.380025+0000 (oldest deadline 2024-06-03T03:30:25.479720+0000) 2024-06-03T03:31:15.732 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:15 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:15.595+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:11.881579+0000 front 2024-06-03T03:30:11.881609+0000 (oldest deadline 2024-06-03T03:30:37.781638+0000) 2024-06-03T03:31:15.732 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:15 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:15.595+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:17.782146+0000 front 2024-06-03T03:30:17.782047+0000 (oldest deadline 2024-06-03T03:30:41.882207+0000) 2024-06-03T03:31:15.732 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:15 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:15.595+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:37.585191+0000 front 2024-06-03T03:30:37.585172+0000 (oldest deadline 2024-06-03T03:30:58.685181+0000) 2024-06-03T03:31:15.732 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:15 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:15.595+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-06-03T03:30:43.886858+0000 front 2024-06-03T03:30:43.886895+0000 (oldest deadline 2024-06-03T03:31:09.186984+0000) 2024-06-03T03:31:15.732 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:15 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:15.595+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6822 osd.6 since back 2024-06-03T03:30:53.288159+0000 front 2024-06-03T03:30:53.288255+0000 (oldest deadline 2024-06-03T03:31:15.587766+0000) 2024-06-03T03:31:15.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:15 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:15.454+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:02.742111+0000 front 2024-06-03T03:30:02.742070+0000 (oldest deadline 2024-06-03T03:30:26.241792+0000) 2024-06-03T03:31:15.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:15 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:15.454+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.742816+0000 front 2024-06-03T03:30:09.742795+0000 (oldest deadline 2024-06-03T03:30:33.243036+0000) 2024-06-03T03:31:15.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:15 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:15.454+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:18.444645+0000 front 2024-06-03T03:30:18.444706+0000 (oldest deadline 2024-06-03T03:30:41.944716+0000) 2024-06-03T03:31:15.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:15 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:15.454+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.446009+0000 front 2024-06-03T03:30:25.445806+0000 (oldest deadline 2024-06-03T03:30:50.046043+0000) 2024-06-03T03:31:15.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:15 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:15.454+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:37.647415+0000 front 2024-06-03T03:30:37.647453+0000 (oldest deadline 2024-06-03T03:30:59.947552+0000) 2024-06-03T03:31:15.907 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:15 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:15.454+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-06-03T03:30:45.749157+0000 front 2024-06-03T03:30:45.749116+0000 (oldest deadline 2024-06-03T03:31:08.049043+0000) 2024-06-03T03:31:16.446 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:16 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:16.050+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.254189+0000 front 2024-06-03T03:30:09.253879+0000 (oldest deadline 2024-06-03T03:30:33.353839+0000) 2024-06-03T03:31:16.447 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:16 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:16.050+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:16.855414+0000 front 2024-06-03T03:30:16.855488+0000 (oldest deadline 2024-06-03T03:30:42.755658+0000) 2024-06-03T03:31:16.447 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:16 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:16.050+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.657633+0000 front 2024-06-03T03:30:25.658290+0000 (oldest deadline 2024-06-03T03:30:50.357531+0000) 2024-06-03T03:31:16.447 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:16 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:16.050+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:35.659635+0000 front 2024-06-03T03:30:35.660018+0000 (oldest deadline 2024-06-03T03:30:58.559861+0000) 2024-06-03T03:31:16.447 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:16 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:16.050+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-06-03T03:30:44.864033+0000 front 2024-06-03T03:30:44.864222+0000 (oldest deadline 2024-06-03T03:31:09.564167+0000) 2024-06-03T03:31:16.447 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:16 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:16.050+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.117:6822 osd.6 since back 2024-06-03T03:30:52.465952+0000 front 2024-06-03T03:30:52.465985+0000 (oldest deadline 2024-06-03T03:31:15.965811+0000) 2024-06-03T03:31:16.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:16 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:16.442+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:02.742111+0000 front 2024-06-03T03:30:02.742070+0000 (oldest deadline 2024-06-03T03:30:26.241792+0000) 2024-06-03T03:31:16.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:16 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:16.442+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.742816+0000 front 2024-06-03T03:30:09.742795+0000 (oldest deadline 2024-06-03T03:30:33.243036+0000) 2024-06-03T03:31:16.907 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:16 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:16.442+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:18.444645+0000 front 2024-06-03T03:30:18.444706+0000 (oldest deadline 2024-06-03T03:30:41.944716+0000) 2024-06-03T03:31:16.907 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:16 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:16.442+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.446009+0000 front 2024-06-03T03:30:25.445806+0000 (oldest deadline 2024-06-03T03:30:50.046043+0000) 2024-06-03T03:31:16.907 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:16 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:16.442+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:37.647415+0000 front 2024-06-03T03:30:37.647453+0000 (oldest deadline 2024-06-03T03:30:59.947552+0000) 2024-06-03T03:31:16.907 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:16 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:16.442+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-06-03T03:30:45.749157+0000 front 2024-06-03T03:30:45.749116+0000 (oldest deadline 2024-06-03T03:31:08.049043+0000) 2024-06-03T03:31:16.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:16 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:16.644+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:01.379911+0000 front 2024-06-03T03:30:01.380025+0000 (oldest deadline 2024-06-03T03:30:25.479720+0000) 2024-06-03T03:31:16.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:16 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:16.644+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:11.881579+0000 front 2024-06-03T03:30:11.881609+0000 (oldest deadline 2024-06-03T03:30:37.781638+0000) 2024-06-03T03:31:16.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:16 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:16.644+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:17.782146+0000 front 2024-06-03T03:30:17.782047+0000 (oldest deadline 2024-06-03T03:30:41.882207+0000) 2024-06-03T03:31:16.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:16 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:16.644+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:37.585191+0000 front 2024-06-03T03:30:37.585172+0000 (oldest deadline 2024-06-03T03:30:58.685181+0000) 2024-06-03T03:31:16.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:16 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:16.644+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-06-03T03:30:43.886858+0000 front 2024-06-03T03:30:43.886895+0000 (oldest deadline 2024-06-03T03:31:09.186984+0000) 2024-06-03T03:31:16.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:16 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:16.644+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6822 osd.6 since back 2024-06-03T03:30:53.288159+0000 front 2024-06-03T03:30:53.288255+0000 (oldest deadline 2024-06-03T03:31:15.587766+0000) 2024-06-03T03:31:17.349 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:17 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:17.092+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.254189+0000 front 2024-06-03T03:30:09.253879+0000 (oldest deadline 2024-06-03T03:30:33.353839+0000) 2024-06-03T03:31:17.349 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:17 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:17.092+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:16.855414+0000 front 2024-06-03T03:30:16.855488+0000 (oldest deadline 2024-06-03T03:30:42.755658+0000) 2024-06-03T03:31:17.349 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:17 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:17.092+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.657633+0000 front 2024-06-03T03:30:25.658290+0000 (oldest deadline 2024-06-03T03:30:50.357531+0000) 2024-06-03T03:31:17.349 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:17 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:17.092+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:35.659635+0000 front 2024-06-03T03:30:35.660018+0000 (oldest deadline 2024-06-03T03:30:58.559861+0000) 2024-06-03T03:31:17.349 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:17 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:17.092+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-06-03T03:30:44.864033+0000 front 2024-06-03T03:30:44.864222+0000 (oldest deadline 2024-06-03T03:31:09.564167+0000) 2024-06-03T03:31:17.349 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:17 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:17.092+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.117:6822 osd.6 since back 2024-06-03T03:30:52.465952+0000 front 2024-06-03T03:30:52.465985+0000 (oldest deadline 2024-06-03T03:31:15.965811+0000) 2024-06-03T03:31:17.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:17 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:17.433+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:02.742111+0000 front 2024-06-03T03:30:02.742070+0000 (oldest deadline 2024-06-03T03:30:26.241792+0000) 2024-06-03T03:31:17.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:17 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:17.433+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.742816+0000 front 2024-06-03T03:30:09.742795+0000 (oldest deadline 2024-06-03T03:30:33.243036+0000) 2024-06-03T03:31:17.907 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:17 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:17.433+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:18.444645+0000 front 2024-06-03T03:30:18.444706+0000 (oldest deadline 2024-06-03T03:30:41.944716+0000) 2024-06-03T03:31:17.907 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:17 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:17.433+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.446009+0000 front 2024-06-03T03:30:25.445806+0000 (oldest deadline 2024-06-03T03:30:50.046043+0000) 2024-06-03T03:31:17.907 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:17 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:17.433+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:37.647415+0000 front 2024-06-03T03:30:37.647453+0000 (oldest deadline 2024-06-03T03:30:59.947552+0000) 2024-06-03T03:31:17.907 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:17 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:17.433+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-06-03T03:30:45.749157+0000 front 2024-06-03T03:30:45.749116+0000 (oldest deadline 2024-06-03T03:31:08.049043+0000) 2024-06-03T03:31:17.981 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:17 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:17.622+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:01.379911+0000 front 2024-06-03T03:30:01.380025+0000 (oldest deadline 2024-06-03T03:30:25.479720+0000) 2024-06-03T03:31:17.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:17 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:17.622+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:11.881579+0000 front 2024-06-03T03:30:11.881609+0000 (oldest deadline 2024-06-03T03:30:37.781638+0000) 2024-06-03T03:31:17.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:17 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:17.622+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:17.782146+0000 front 2024-06-03T03:30:17.782047+0000 (oldest deadline 2024-06-03T03:30:41.882207+0000) 2024-06-03T03:31:17.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:17 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:17.622+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:37.585191+0000 front 2024-06-03T03:30:37.585172+0000 (oldest deadline 2024-06-03T03:30:58.685181+0000) 2024-06-03T03:31:17.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:17 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:17.622+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-06-03T03:30:43.886858+0000 front 2024-06-03T03:30:43.886895+0000 (oldest deadline 2024-06-03T03:31:09.186984+0000) 2024-06-03T03:31:17.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:17 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:17.622+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6822 osd.6 since back 2024-06-03T03:30:53.288159+0000 front 2024-06-03T03:30:53.288255+0000 (oldest deadline 2024-06-03T03:31:15.587766+0000) 2024-06-03T03:31:18.447 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:18 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:18.101+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.254189+0000 front 2024-06-03T03:30:09.253879+0000 (oldest deadline 2024-06-03T03:30:33.353839+0000) 2024-06-03T03:31:18.447 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:18 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:18.101+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:16.855414+0000 front 2024-06-03T03:30:16.855488+0000 (oldest deadline 2024-06-03T03:30:42.755658+0000) 2024-06-03T03:31:18.447 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:18 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:18.101+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.657633+0000 front 2024-06-03T03:30:25.658290+0000 (oldest deadline 2024-06-03T03:30:50.357531+0000) 2024-06-03T03:31:18.447 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:18 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:18.101+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:35.659635+0000 front 2024-06-03T03:30:35.660018+0000 (oldest deadline 2024-06-03T03:30:58.559861+0000) 2024-06-03T03:31:18.447 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:18 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:18.101+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-06-03T03:30:44.864033+0000 front 2024-06-03T03:30:44.864222+0000 (oldest deadline 2024-06-03T03:31:09.564167+0000) 2024-06-03T03:31:18.448 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:18 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:18.101+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.117:6822 osd.6 since back 2024-06-03T03:30:52.465952+0000 front 2024-06-03T03:30:52.465985+0000 (oldest deadline 2024-06-03T03:31:15.965811+0000) 2024-06-03T03:31:18.712 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:18 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:18.453+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:02.742111+0000 front 2024-06-03T03:30:02.742070+0000 (oldest deadline 2024-06-03T03:30:26.241792+0000) 2024-06-03T03:31:18.712 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:18 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:18.453+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.742816+0000 front 2024-06-03T03:30:09.742795+0000 (oldest deadline 2024-06-03T03:30:33.243036+0000) 2024-06-03T03:31:18.712 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:18 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:18.453+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:18.444645+0000 front 2024-06-03T03:30:18.444706+0000 (oldest deadline 2024-06-03T03:30:41.944716+0000) 2024-06-03T03:31:18.712 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:18 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:18.453+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.446009+0000 front 2024-06-03T03:30:25.445806+0000 (oldest deadline 2024-06-03T03:30:50.046043+0000) 2024-06-03T03:31:18.712 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:18 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:18.453+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:37.647415+0000 front 2024-06-03T03:30:37.647453+0000 (oldest deadline 2024-06-03T03:30:59.947552+0000) 2024-06-03T03:31:18.712 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:18 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:18.453+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-06-03T03:30:45.749157+0000 front 2024-06-03T03:30:45.749116+0000 (oldest deadline 2024-06-03T03:31:08.049043+0000) 2024-06-03T03:31:18.981 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:18 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:18.612+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:01.379911+0000 front 2024-06-03T03:30:01.380025+0000 (oldest deadline 2024-06-03T03:30:25.479720+0000) 2024-06-03T03:31:18.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:18 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:18.612+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:11.881579+0000 front 2024-06-03T03:30:11.881609+0000 (oldest deadline 2024-06-03T03:30:37.781638+0000) 2024-06-03T03:31:18.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:18 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:18.612+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:17.782146+0000 front 2024-06-03T03:30:17.782047+0000 (oldest deadline 2024-06-03T03:30:41.882207+0000) 2024-06-03T03:31:18.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:18 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:18.612+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:37.585191+0000 front 2024-06-03T03:30:37.585172+0000 (oldest deadline 2024-06-03T03:30:58.685181+0000) 2024-06-03T03:31:18.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:18 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:18.612+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-06-03T03:30:43.886858+0000 front 2024-06-03T03:30:43.886895+0000 (oldest deadline 2024-06-03T03:31:09.186984+0000) 2024-06-03T03:31:18.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:18 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:18.612+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6822 osd.6 since back 2024-06-03T03:30:53.288159+0000 front 2024-06-03T03:30:53.288255+0000 (oldest deadline 2024-06-03T03:31:15.587766+0000) 2024-06-03T03:31:19.197 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:19 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:19.129+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.254189+0000 front 2024-06-03T03:30:09.253879+0000 (oldest deadline 2024-06-03T03:30:33.353839+0000) 2024-06-03T03:31:19.197 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:19 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:19.129+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:16.855414+0000 front 2024-06-03T03:30:16.855488+0000 (oldest deadline 2024-06-03T03:30:42.755658+0000) 2024-06-03T03:31:19.197 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:19 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:19.129+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.657633+0000 front 2024-06-03T03:30:25.658290+0000 (oldest deadline 2024-06-03T03:30:50.357531+0000) 2024-06-03T03:31:19.197 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:19 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:19.129+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:35.659635+0000 front 2024-06-03T03:30:35.660018+0000 (oldest deadline 2024-06-03T03:30:58.559861+0000) 2024-06-03T03:31:19.197 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:19 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:19.129+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-06-03T03:30:44.864033+0000 front 2024-06-03T03:30:44.864222+0000 (oldest deadline 2024-06-03T03:31:09.564167+0000) 2024-06-03T03:31:19.197 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:19 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9[64376]: 2024-06-03T03:31:19.129+0000 7f91a5200700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.117:6822 osd.6 since back 2024-06-03T03:30:52.465952+0000 front 2024-06-03T03:30:52.465985+0000 (oldest deadline 2024-06-03T03:31:15.965811+0000) 2024-06-03T03:31:19.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:19 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:19.455+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:02.742111+0000 front 2024-06-03T03:30:02.742070+0000 (oldest deadline 2024-06-03T03:30:26.241792+0000) 2024-06-03T03:31:19.907 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:19 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:19.455+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.742816+0000 front 2024-06-03T03:30:09.742795+0000 (oldest deadline 2024-06-03T03:30:33.243036+0000) 2024-06-03T03:31:19.907 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:19 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:19.455+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:18.444645+0000 front 2024-06-03T03:30:18.444706+0000 (oldest deadline 2024-06-03T03:30:41.944716+0000) 2024-06-03T03:31:19.907 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:19 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:19.455+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.446009+0000 front 2024-06-03T03:30:25.445806+0000 (oldest deadline 2024-06-03T03:30:50.046043+0000) 2024-06-03T03:31:19.907 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:19 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:19.455+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:37.647415+0000 front 2024-06-03T03:30:37.647453+0000 (oldest deadline 2024-06-03T03:30:59.947552+0000) 2024-06-03T03:31:19.907 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:19 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:19.455+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-06-03T03:30:45.749157+0000 front 2024-06-03T03:30:45.749116+0000 (oldest deadline 2024-06-03T03:31:08.049043+0000) 2024-06-03T03:31:19.907 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:19 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:19.455+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6822 osd.6 since back 2024-06-03T03:30:53.750889+0000 front 2024-06-03T03:30:53.751049+0000 (oldest deadline 2024-06-03T03:31:19.050596+0000) 2024-06-03T03:31:19.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:19 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:19.618+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:01.379911+0000 front 2024-06-03T03:30:01.380025+0000 (oldest deadline 2024-06-03T03:30:25.479720+0000) 2024-06-03T03:31:19.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:19 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:19.618+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:11.881579+0000 front 2024-06-03T03:30:11.881609+0000 (oldest deadline 2024-06-03T03:30:37.781638+0000) 2024-06-03T03:31:19.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:19 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:19.618+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:17.782146+0000 front 2024-06-03T03:30:17.782047+0000 (oldest deadline 2024-06-03T03:30:41.882207+0000) 2024-06-03T03:31:19.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:19 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:19.618+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:37.585191+0000 front 2024-06-03T03:30:37.585172+0000 (oldest deadline 2024-06-03T03:30:58.685181+0000) 2024-06-03T03:31:19.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:19 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:19.618+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-06-03T03:30:43.886858+0000 front 2024-06-03T03:30:43.886895+0000 (oldest deadline 2024-06-03T03:31:09.186984+0000) 2024-06-03T03:31:19.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:19 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:19.618+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6822 osd.6 since back 2024-06-03T03:30:53.288159+0000 front 2024-06-03T03:30:53.288255+0000 (oldest deadline 2024-06-03T03:31:15.587766+0000) 2024-06-03T03:31:20.609 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:20 smithi117 podman[98868]: 2024-06-03 03:31:20.286165964 +0000 UTC m=+5.522890654 container died ce935bdb182ee9902488f33eeab70b605c26d26d01c9a3efb42da45995dda305 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2) 2024-06-03T03:31:20.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:20 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:20.477+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:02.742111+0000 front 2024-06-03T03:30:02.742070+0000 (oldest deadline 2024-06-03T03:30:26.241792+0000) 2024-06-03T03:31:20.907 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:20 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:20.477+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.742816+0000 front 2024-06-03T03:30:09.742795+0000 (oldest deadline 2024-06-03T03:30:33.243036+0000) 2024-06-03T03:31:20.907 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:20 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:20.477+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:18.444645+0000 front 2024-06-03T03:30:18.444706+0000 (oldest deadline 2024-06-03T03:30:41.944716+0000) 2024-06-03T03:31:20.907 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:20 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:20.477+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.446009+0000 front 2024-06-03T03:30:25.445806+0000 (oldest deadline 2024-06-03T03:30:50.046043+0000) 2024-06-03T03:31:20.907 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:20 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:20.477+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:37.647415+0000 front 2024-06-03T03:30:37.647453+0000 (oldest deadline 2024-06-03T03:30:59.947552+0000) 2024-06-03T03:31:20.907 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:20 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:20.477+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-06-03T03:30:45.749157+0000 front 2024-06-03T03:30:45.749116+0000 (oldest deadline 2024-06-03T03:31:08.049043+0000) 2024-06-03T03:31:20.907 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:20 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:20.477+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6822 osd.6 since back 2024-06-03T03:30:53.750889+0000 front 2024-06-03T03:30:53.751049+0000 (oldest deadline 2024-06-03T03:31:19.050596+0000) 2024-06-03T03:31:20.946 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:20 smithi117 podman[98868]: 2024-06-03 03:31:20.807375131 +0000 UTC m=+6.044099850 container cleanup ce935bdb182ee9902488f33eeab70b605c26d26d01c9a3efb42da45995dda305 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-06-03T03:31:20.947 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:20 smithi117 bash[98868]: ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9 2024-06-03T03:31:20.981 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:20 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:20.640+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:01.379911+0000 front 2024-06-03T03:30:01.380025+0000 (oldest deadline 2024-06-03T03:30:25.479720+0000) 2024-06-03T03:31:20.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:20 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:20.640+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:11.881579+0000 front 2024-06-03T03:30:11.881609+0000 (oldest deadline 2024-06-03T03:30:37.781638+0000) 2024-06-03T03:31:20.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:20 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:20.640+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:17.782146+0000 front 2024-06-03T03:30:17.782047+0000 (oldest deadline 2024-06-03T03:30:41.882207+0000) 2024-06-03T03:31:20.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:20 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:20.640+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:37.585191+0000 front 2024-06-03T03:30:37.585172+0000 (oldest deadline 2024-06-03T03:30:58.685181+0000) 2024-06-03T03:31:20.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:20 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:20.640+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-06-03T03:30:43.886858+0000 front 2024-06-03T03:30:43.886895+0000 (oldest deadline 2024-06-03T03:31:09.186984+0000) 2024-06-03T03:31:20.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:20 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:20.640+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6822 osd.6 since back 2024-06-03T03:30:53.288159+0000 front 2024-06-03T03:30:53.288255+0000 (oldest deadline 2024-06-03T03:31:15.587766+0000) 2024-06-03T03:31:21.446 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:21 smithi117 podman[98897]: 2024-06-03 03:31:21.108732127 +0000 UTC m=+0.807786235 container remove ce935bdb182ee9902488f33eeab70b605c26d26d01c9a3efb42da45995dda305 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True) 2024-06-03T03:31:21.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:21 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:21.455+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:02.742111+0000 front 2024-06-03T03:30:02.742070+0000 (oldest deadline 2024-06-03T03:30:26.241792+0000) 2024-06-03T03:31:21.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:21 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:21.455+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.742816+0000 front 2024-06-03T03:30:09.742795+0000 (oldest deadline 2024-06-03T03:30:33.243036+0000) 2024-06-03T03:31:21.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:21 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:21.455+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:18.444645+0000 front 2024-06-03T03:30:18.444706+0000 (oldest deadline 2024-06-03T03:30:41.944716+0000) 2024-06-03T03:31:21.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:21 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:21.455+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.446009+0000 front 2024-06-03T03:30:25.445806+0000 (oldest deadline 2024-06-03T03:30:50.046043+0000) 2024-06-03T03:31:21.907 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:21 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:21.455+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:37.647415+0000 front 2024-06-03T03:30:37.647453+0000 (oldest deadline 2024-06-03T03:30:59.947552+0000) 2024-06-03T03:31:21.907 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:21 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:21.455+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-06-03T03:30:45.749157+0000 front 2024-06-03T03:30:45.749116+0000 (oldest deadline 2024-06-03T03:31:08.049043+0000) 2024-06-03T03:31:21.907 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:21 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:21.455+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6822 osd.6 since back 2024-06-03T03:30:53.750889+0000 front 2024-06-03T03:30:53.751049+0000 (oldest deadline 2024-06-03T03:31:19.050596+0000) 2024-06-03T03:31:21.981 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:21 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:21.615+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:01.379911+0000 front 2024-06-03T03:30:01.380025+0000 (oldest deadline 2024-06-03T03:30:25.479720+0000) 2024-06-03T03:31:21.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:21 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:21.615+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:11.881579+0000 front 2024-06-03T03:30:11.881609+0000 (oldest deadline 2024-06-03T03:30:37.781638+0000) 2024-06-03T03:31:21.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:21 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:21.615+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:17.782146+0000 front 2024-06-03T03:30:17.782047+0000 (oldest deadline 2024-06-03T03:30:41.882207+0000) 2024-06-03T03:31:21.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:21 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:21.615+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:37.585191+0000 front 2024-06-03T03:30:37.585172+0000 (oldest deadline 2024-06-03T03:30:58.685181+0000) 2024-06-03T03:31:21.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:21 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:21.615+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-06-03T03:30:43.886858+0000 front 2024-06-03T03:30:43.886895+0000 (oldest deadline 2024-06-03T03:31:09.186984+0000) 2024-06-03T03:31:21.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:21 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:21.615+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6822 osd.6 since back 2024-06-03T03:30:53.288159+0000 front 2024-06-03T03:30:53.288255+0000 (oldest deadline 2024-06-03T03:31:15.587766+0000) 2024-06-03T03:31:22.197 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:21 smithi117 podman[98984]: 2024-06-03 03:31:21.805611293 +0000 UTC m=+0.032208998 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T03:31:22.197 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:21 smithi117 podman[98984]: 2024-06-03 03:31:21.950989032 +0000 UTC m=+0.177586748 container create 612db306927b9f3bd6a78bb375f17fe3a9eb7720d681f20979a29ad0adac46d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, ceph=True, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=) 2024-06-03T03:31:22.696 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:22 smithi117 podman[98984]: 2024-06-03 03:31:22.263403604 +0000 UTC m=+0.490001366 container init 612db306927b9f3bd6a78bb375f17fe3a9eb7720d681f20979a29ad0adac46d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9-deactivate, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , ceph=True) 2024-06-03T03:31:22.696 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:22 smithi117 podman[98984]: 2024-06-03 03:31:22.274237755 +0000 UTC m=+0.500835504 container start 612db306927b9f3bd6a78bb375f17fe3a9eb7720d681f20979a29ad0adac46d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc) 2024-06-03T03:31:22.697 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:22 smithi117 podman[98984]: 2024-06-03 03:31:22.361684523 +0000 UTC m=+0.588282272 container attach 612db306927b9f3bd6a78bb375f17fe3a9eb7720d681f20979a29ad0adac46d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9-deactivate, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-06-03T03:31:22.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:22 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:22.491+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:02.742111+0000 front 2024-06-03T03:30:02.742070+0000 (oldest deadline 2024-06-03T03:30:26.241792+0000) 2024-06-03T03:31:22.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:22 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:22.491+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.742816+0000 front 2024-06-03T03:30:09.742795+0000 (oldest deadline 2024-06-03T03:30:33.243036+0000) 2024-06-03T03:31:22.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:22 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:22.491+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:18.444645+0000 front 2024-06-03T03:30:18.444706+0000 (oldest deadline 2024-06-03T03:30:41.944716+0000) 2024-06-03T03:31:22.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:22 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:22.491+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.446009+0000 front 2024-06-03T03:30:25.445806+0000 (oldest deadline 2024-06-03T03:30:50.046043+0000) 2024-06-03T03:31:22.907 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:22 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:22.491+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:37.647415+0000 front 2024-06-03T03:30:37.647453+0000 (oldest deadline 2024-06-03T03:30:59.947552+0000) 2024-06-03T03:31:22.907 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:22 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:22.491+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-06-03T03:30:45.749157+0000 front 2024-06-03T03:30:45.749116+0000 (oldest deadline 2024-06-03T03:31:08.049043+0000) 2024-06-03T03:31:22.907 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:22 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:22.491+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6822 osd.6 since back 2024-06-03T03:30:53.750889+0000 front 2024-06-03T03:30:53.751049+0000 (oldest deadline 2024-06-03T03:31:19.050596+0000) 2024-06-03T03:31:22.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:22 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:22.652+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:01.379911+0000 front 2024-06-03T03:30:01.380025+0000 (oldest deadline 2024-06-03T03:30:25.479720+0000) 2024-06-03T03:31:22.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:22 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:22.652+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:11.881579+0000 front 2024-06-03T03:30:11.881609+0000 (oldest deadline 2024-06-03T03:30:37.781638+0000) 2024-06-03T03:31:22.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:22 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:22.652+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:17.782146+0000 front 2024-06-03T03:30:17.782047+0000 (oldest deadline 2024-06-03T03:30:41.882207+0000) 2024-06-03T03:31:22.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:22 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:22.652+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:37.585191+0000 front 2024-06-03T03:30:37.585172+0000 (oldest deadline 2024-06-03T03:30:58.685181+0000) 2024-06-03T03:31:22.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:22 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:22.652+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-06-03T03:30:43.886858+0000 front 2024-06-03T03:30:43.886895+0000 (oldest deadline 2024-06-03T03:31:09.186984+0000) 2024-06-03T03:31:22.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:22 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:22.652+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6822 osd.6 since back 2024-06-03T03:30:53.288159+0000 front 2024-06-03T03:30:53.288255+0000 (oldest deadline 2024-06-03T03:31:15.587766+0000) 2024-06-03T03:31:23.141 INFO:journalctl@ceph.osd.9.smithi117.stdout:Jun 03 03:31:22 smithi117 podman[98984]: 2024-06-03 03:31:22.716270307 +0000 UTC m=+0.942868074 container died 612db306927b9f3bd6a78bb375f17fe3a9eb7720d681f20979a29ad0adac46d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-9-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-03T03:31:23.631 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@osd.9.service' 2024-06-03T03:31:23.743 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T03:31:23.744 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-06-03T03:31:23.744 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-06-03T03:31:23.744 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@osd.10 2024-06-03T03:31:23.772 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:23 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:23.499+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:02.742111+0000 front 2024-06-03T03:30:02.742070+0000 (oldest deadline 2024-06-03T03:30:26.241792+0000) 2024-06-03T03:31:23.772 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:23 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:23.499+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.742816+0000 front 2024-06-03T03:30:09.742795+0000 (oldest deadline 2024-06-03T03:30:33.243036+0000) 2024-06-03T03:31:23.772 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:23 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:23.499+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:18.444645+0000 front 2024-06-03T03:30:18.444706+0000 (oldest deadline 2024-06-03T03:30:41.944716+0000) 2024-06-03T03:31:23.772 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:23 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:23.499+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.446009+0000 front 2024-06-03T03:30:25.445806+0000 (oldest deadline 2024-06-03T03:30:50.046043+0000) 2024-06-03T03:31:23.772 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:23 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:23.499+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:37.647415+0000 front 2024-06-03T03:30:37.647453+0000 (oldest deadline 2024-06-03T03:30:59.947552+0000) 2024-06-03T03:31:23.773 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:23 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:23.499+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-06-03T03:30:45.749157+0000 front 2024-06-03T03:30:45.749116+0000 (oldest deadline 2024-06-03T03:31:08.049043+0000) 2024-06-03T03:31:23.773 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:23 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:23.499+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6822 osd.6 since back 2024-06-03T03:30:53.750889+0000 front 2024-06-03T03:30:53.751049+0000 (oldest deadline 2024-06-03T03:31:19.050596+0000) 2024-06-03T03:31:23.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:23 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:23.627+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:01.379911+0000 front 2024-06-03T03:30:01.380025+0000 (oldest deadline 2024-06-03T03:30:25.479720+0000) 2024-06-03T03:31:23.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:23 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:23.627+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:11.881579+0000 front 2024-06-03T03:30:11.881609+0000 (oldest deadline 2024-06-03T03:30:37.781638+0000) 2024-06-03T03:31:23.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:23 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:23.627+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:17.782146+0000 front 2024-06-03T03:30:17.782047+0000 (oldest deadline 2024-06-03T03:30:41.882207+0000) 2024-06-03T03:31:23.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:23 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:23.627+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:37.585191+0000 front 2024-06-03T03:30:37.585172+0000 (oldest deadline 2024-06-03T03:30:58.685181+0000) 2024-06-03T03:31:23.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:23 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:23.627+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-06-03T03:30:43.886858+0000 front 2024-06-03T03:30:43.886895+0000 (oldest deadline 2024-06-03T03:31:09.186984+0000) 2024-06-03T03:31:23.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:23 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:23.627+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6822 osd.6 since back 2024-06-03T03:30:53.288159+0000 front 2024-06-03T03:30:53.288255+0000 (oldest deadline 2024-06-03T03:31:15.587766+0000) 2024-06-03T03:31:24.063 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:23 smithi151 systemd[1]: Stopping Ceph osd.10 for a0e8473a-2156-11ef-bc9b-c7b262605968... 2024-06-03T03:31:24.063 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:24 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:24.062+0000 7fd515172700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T03:31:24.063 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:24 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:24.062+0000 7fd515172700 -1 osd.10 103 *** Got signal Terminated *** 2024-06-03T03:31:24.406 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:24 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:24.062+0000 7fd515172700 -1 osd.10 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-03T03:31:24.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:24 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:24.514+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:02.742111+0000 front 2024-06-03T03:30:02.742070+0000 (oldest deadline 2024-06-03T03:30:26.241792+0000) 2024-06-03T03:31:24.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:24 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:24.514+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.742816+0000 front 2024-06-03T03:30:09.742795+0000 (oldest deadline 2024-06-03T03:30:33.243036+0000) 2024-06-03T03:31:24.907 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:24 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:24.514+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:18.444645+0000 front 2024-06-03T03:30:18.444706+0000 (oldest deadline 2024-06-03T03:30:41.944716+0000) 2024-06-03T03:31:24.907 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:24 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:24.514+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.446009+0000 front 2024-06-03T03:30:25.445806+0000 (oldest deadline 2024-06-03T03:30:50.046043+0000) 2024-06-03T03:31:24.907 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:24 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:24.514+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:37.647415+0000 front 2024-06-03T03:30:37.647453+0000 (oldest deadline 2024-06-03T03:30:59.947552+0000) 2024-06-03T03:31:24.907 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:24 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:24.514+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-06-03T03:30:45.749157+0000 front 2024-06-03T03:30:45.749116+0000 (oldest deadline 2024-06-03T03:31:08.049043+0000) 2024-06-03T03:31:24.907 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:24 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:24.514+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6822 osd.6 since back 2024-06-03T03:30:53.750889+0000 front 2024-06-03T03:30:53.751049+0000 (oldest deadline 2024-06-03T03:31:19.050596+0000) 2024-06-03T03:31:24.981 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:24 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:24.615+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:01.379911+0000 front 2024-06-03T03:30:01.380025+0000 (oldest deadline 2024-06-03T03:30:25.479720+0000) 2024-06-03T03:31:24.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:24 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:24.615+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:11.881579+0000 front 2024-06-03T03:30:11.881609+0000 (oldest deadline 2024-06-03T03:30:37.781638+0000) 2024-06-03T03:31:24.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:24 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:24.615+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:17.782146+0000 front 2024-06-03T03:30:17.782047+0000 (oldest deadline 2024-06-03T03:30:41.882207+0000) 2024-06-03T03:31:24.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:24 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:24.615+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:37.585191+0000 front 2024-06-03T03:30:37.585172+0000 (oldest deadline 2024-06-03T03:30:58.685181+0000) 2024-06-03T03:31:24.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:24 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:24.615+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-06-03T03:30:43.886858+0000 front 2024-06-03T03:30:43.886895+0000 (oldest deadline 2024-06-03T03:31:09.186984+0000) 2024-06-03T03:31:24.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:24 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:24.615+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6822 osd.6 since back 2024-06-03T03:30:53.288159+0000 front 2024-06-03T03:30:53.288255+0000 (oldest deadline 2024-06-03T03:31:15.587766+0000) 2024-06-03T03:31:25.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:25 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:25.540+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:02.742111+0000 front 2024-06-03T03:30:02.742070+0000 (oldest deadline 2024-06-03T03:30:26.241792+0000) 2024-06-03T03:31:25.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:25 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:25.540+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.742816+0000 front 2024-06-03T03:30:09.742795+0000 (oldest deadline 2024-06-03T03:30:33.243036+0000) 2024-06-03T03:31:25.907 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:25 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:25.540+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:18.444645+0000 front 2024-06-03T03:30:18.444706+0000 (oldest deadline 2024-06-03T03:30:41.944716+0000) 2024-06-03T03:31:25.907 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:25 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:25.540+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.446009+0000 front 2024-06-03T03:30:25.445806+0000 (oldest deadline 2024-06-03T03:30:50.046043+0000) 2024-06-03T03:31:25.907 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:25 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:25.540+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:37.647415+0000 front 2024-06-03T03:30:37.647453+0000 (oldest deadline 2024-06-03T03:30:59.947552+0000) 2024-06-03T03:31:25.907 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:25 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:25.540+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-06-03T03:30:45.749157+0000 front 2024-06-03T03:30:45.749116+0000 (oldest deadline 2024-06-03T03:31:08.049043+0000) 2024-06-03T03:31:25.907 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:25 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:25.540+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6822 osd.6 since back 2024-06-03T03:30:53.750889+0000 front 2024-06-03T03:30:53.751049+0000 (oldest deadline 2024-06-03T03:31:19.050596+0000) 2024-06-03T03:31:25.981 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:25 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:25.638+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:01.379911+0000 front 2024-06-03T03:30:01.380025+0000 (oldest deadline 2024-06-03T03:30:25.479720+0000) 2024-06-03T03:31:25.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:25 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:25.638+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:11.881579+0000 front 2024-06-03T03:30:11.881609+0000 (oldest deadline 2024-06-03T03:30:37.781638+0000) 2024-06-03T03:31:25.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:25 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:25.638+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:17.782146+0000 front 2024-06-03T03:30:17.782047+0000 (oldest deadline 2024-06-03T03:30:41.882207+0000) 2024-06-03T03:31:25.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:25 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:25.638+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:37.585191+0000 front 2024-06-03T03:30:37.585172+0000 (oldest deadline 2024-06-03T03:30:58.685181+0000) 2024-06-03T03:31:25.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:25 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:25.638+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-06-03T03:30:43.886858+0000 front 2024-06-03T03:30:43.886895+0000 (oldest deadline 2024-06-03T03:31:09.186984+0000) 2024-06-03T03:31:25.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:25 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:25.638+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6822 osd.6 since back 2024-06-03T03:30:53.288159+0000 front 2024-06-03T03:30:53.288255+0000 (oldest deadline 2024-06-03T03:31:15.587766+0000) 2024-06-03T03:31:25.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:25 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:25.638+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6820 osd.7 since back 2024-06-03T03:31:03.089618+0000 front 2024-06-03T03:31:03.089721+0000 (oldest deadline 2024-06-03T03:31:24.789642+0000) 2024-06-03T03:31:26.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:26 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:26.494+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:02.742111+0000 front 2024-06-03T03:30:02.742070+0000 (oldest deadline 2024-06-03T03:30:26.241792+0000) 2024-06-03T03:31:26.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:26 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:26.494+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.742816+0000 front 2024-06-03T03:30:09.742795+0000 (oldest deadline 2024-06-03T03:30:33.243036+0000) 2024-06-03T03:31:26.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:26 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:26.494+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:18.444645+0000 front 2024-06-03T03:30:18.444706+0000 (oldest deadline 2024-06-03T03:30:41.944716+0000) 2024-06-03T03:31:26.907 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:26 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:26.494+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.446009+0000 front 2024-06-03T03:30:25.445806+0000 (oldest deadline 2024-06-03T03:30:50.046043+0000) 2024-06-03T03:31:26.907 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:26 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:26.494+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:37.647415+0000 front 2024-06-03T03:30:37.647453+0000 (oldest deadline 2024-06-03T03:30:59.947552+0000) 2024-06-03T03:31:26.907 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:26 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:26.494+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-06-03T03:30:45.749157+0000 front 2024-06-03T03:30:45.749116+0000 (oldest deadline 2024-06-03T03:31:08.049043+0000) 2024-06-03T03:31:26.907 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:26 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:26.494+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6822 osd.6 since back 2024-06-03T03:30:53.750889+0000 front 2024-06-03T03:30:53.751049+0000 (oldest deadline 2024-06-03T03:31:19.050596+0000) 2024-06-03T03:31:26.981 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:26 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:26.617+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:01.379911+0000 front 2024-06-03T03:30:01.380025+0000 (oldest deadline 2024-06-03T03:30:25.479720+0000) 2024-06-03T03:31:26.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:26 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:26.617+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:11.881579+0000 front 2024-06-03T03:30:11.881609+0000 (oldest deadline 2024-06-03T03:30:37.781638+0000) 2024-06-03T03:31:26.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:26 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:26.617+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:17.782146+0000 front 2024-06-03T03:30:17.782047+0000 (oldest deadline 2024-06-03T03:30:41.882207+0000) 2024-06-03T03:31:26.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:26 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:26.617+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:37.585191+0000 front 2024-06-03T03:30:37.585172+0000 (oldest deadline 2024-06-03T03:30:58.685181+0000) 2024-06-03T03:31:26.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:26 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:26.617+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-06-03T03:30:43.886858+0000 front 2024-06-03T03:30:43.886895+0000 (oldest deadline 2024-06-03T03:31:09.186984+0000) 2024-06-03T03:31:26.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:26 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:26.617+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6822 osd.6 since back 2024-06-03T03:30:53.288159+0000 front 2024-06-03T03:30:53.288255+0000 (oldest deadline 2024-06-03T03:31:15.587766+0000) 2024-06-03T03:31:26.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:26 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:26.617+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6820 osd.7 since back 2024-06-03T03:31:03.089618+0000 front 2024-06-03T03:31:03.089721+0000 (oldest deadline 2024-06-03T03:31:24.789642+0000) 2024-06-03T03:31:27.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:27 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:27.452+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:02.742111+0000 front 2024-06-03T03:30:02.742070+0000 (oldest deadline 2024-06-03T03:30:26.241792+0000) 2024-06-03T03:31:27.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:27 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:27.452+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.742816+0000 front 2024-06-03T03:30:09.742795+0000 (oldest deadline 2024-06-03T03:30:33.243036+0000) 2024-06-03T03:31:27.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:27 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:27.452+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:18.444645+0000 front 2024-06-03T03:30:18.444706+0000 (oldest deadline 2024-06-03T03:30:41.944716+0000) 2024-06-03T03:31:27.907 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:27 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:27.452+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.446009+0000 front 2024-06-03T03:30:25.445806+0000 (oldest deadline 2024-06-03T03:30:50.046043+0000) 2024-06-03T03:31:27.907 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:27 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:27.452+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:37.647415+0000 front 2024-06-03T03:30:37.647453+0000 (oldest deadline 2024-06-03T03:30:59.947552+0000) 2024-06-03T03:31:27.907 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:27 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:27.452+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-06-03T03:30:45.749157+0000 front 2024-06-03T03:30:45.749116+0000 (oldest deadline 2024-06-03T03:31:08.049043+0000) 2024-06-03T03:31:27.907 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:27 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:27.452+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6822 osd.6 since back 2024-06-03T03:30:53.750889+0000 front 2024-06-03T03:30:53.751049+0000 (oldest deadline 2024-06-03T03:31:19.050596+0000) 2024-06-03T03:31:27.981 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:27 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:27.649+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:01.379911+0000 front 2024-06-03T03:30:01.380025+0000 (oldest deadline 2024-06-03T03:30:25.479720+0000) 2024-06-03T03:31:27.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:27 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:27.649+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:11.881579+0000 front 2024-06-03T03:30:11.881609+0000 (oldest deadline 2024-06-03T03:30:37.781638+0000) 2024-06-03T03:31:27.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:27 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:27.649+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:17.782146+0000 front 2024-06-03T03:30:17.782047+0000 (oldest deadline 2024-06-03T03:30:41.882207+0000) 2024-06-03T03:31:27.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:27 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:27.649+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:37.585191+0000 front 2024-06-03T03:30:37.585172+0000 (oldest deadline 2024-06-03T03:30:58.685181+0000) 2024-06-03T03:31:27.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:27 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:27.649+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-06-03T03:30:43.886858+0000 front 2024-06-03T03:30:43.886895+0000 (oldest deadline 2024-06-03T03:31:09.186984+0000) 2024-06-03T03:31:27.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:27 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:27.649+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6822 osd.6 since back 2024-06-03T03:30:53.288159+0000 front 2024-06-03T03:30:53.288255+0000 (oldest deadline 2024-06-03T03:31:15.587766+0000) 2024-06-03T03:31:27.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:27 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:27.649+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6820 osd.7 since back 2024-06-03T03:31:03.089618+0000 front 2024-06-03T03:31:03.089721+0000 (oldest deadline 2024-06-03T03:31:24.789642+0000) 2024-06-03T03:31:28.906 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:28 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:28.444+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:02.742111+0000 front 2024-06-03T03:30:02.742070+0000 (oldest deadline 2024-06-03T03:30:26.241792+0000) 2024-06-03T03:31:28.907 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:28 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:28.444+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:09.742816+0000 front 2024-06-03T03:30:09.742795+0000 (oldest deadline 2024-06-03T03:30:33.243036+0000) 2024-06-03T03:31:28.907 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:28 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:28.444+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:18.444645+0000 front 2024-06-03T03:30:18.444706+0000 (oldest deadline 2024-06-03T03:30:41.944716+0000) 2024-06-03T03:31:28.907 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:28 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:28.444+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-06-03T03:30:25.446009+0000 front 2024-06-03T03:30:25.445806+0000 (oldest deadline 2024-06-03T03:30:50.046043+0000) 2024-06-03T03:31:28.907 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:28 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:28.444+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:37.647415+0000 front 2024-06-03T03:30:37.647453+0000 (oldest deadline 2024-06-03T03:30:59.947552+0000) 2024-06-03T03:31:28.907 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:28 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:28.444+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-06-03T03:30:45.749157+0000 front 2024-06-03T03:30:45.749116+0000 (oldest deadline 2024-06-03T03:31:08.049043+0000) 2024-06-03T03:31:28.907 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:28 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10[62045]: 2024-06-03T03:31:28.444+0000 7fd50d75e700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.117:6822 osd.6 since back 2024-06-03T03:30:53.750889+0000 front 2024-06-03T03:30:53.751049+0000 (oldest deadline 2024-06-03T03:31:19.050596+0000) 2024-06-03T03:31:28.981 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:28 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:28.687+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:01.379911+0000 front 2024-06-03T03:30:01.380025+0000 (oldest deadline 2024-06-03T03:30:25.479720+0000) 2024-06-03T03:31:28.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:28 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:28.687+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:11.881579+0000 front 2024-06-03T03:30:11.881609+0000 (oldest deadline 2024-06-03T03:30:37.781638+0000) 2024-06-03T03:31:28.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:28 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:28.687+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:17.782146+0000 front 2024-06-03T03:30:17.782047+0000 (oldest deadline 2024-06-03T03:30:41.882207+0000) 2024-06-03T03:31:28.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:28 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:28.687+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:37.585191+0000 front 2024-06-03T03:30:37.585172+0000 (oldest deadline 2024-06-03T03:30:58.685181+0000) 2024-06-03T03:31:28.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:28 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:28.687+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-06-03T03:30:43.886858+0000 front 2024-06-03T03:30:43.886895+0000 (oldest deadline 2024-06-03T03:31:09.186984+0000) 2024-06-03T03:31:28.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:28 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:28.687+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6822 osd.6 since back 2024-06-03T03:30:53.288159+0000 front 2024-06-03T03:30:53.288255+0000 (oldest deadline 2024-06-03T03:31:15.587766+0000) 2024-06-03T03:31:28.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:28 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:28.687+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6820 osd.7 since back 2024-06-03T03:31:03.089618+0000 front 2024-06-03T03:31:03.089721+0000 (oldest deadline 2024-06-03T03:31:24.789642+0000) 2024-06-03T03:31:29.755 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:29 smithi151 podman[71878]: 2024-06-03 03:31:29.489662136 +0000 UTC m=+5.552115832 container died a90a223158ba49c4b2bce76717bf1e257264c08ee5badc99ccd49d8d22ccbccf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-06-03T03:31:29.981 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:29 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:29.711+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:01.379911+0000 front 2024-06-03T03:30:01.380025+0000 (oldest deadline 2024-06-03T03:30:25.479720+0000) 2024-06-03T03:31:29.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:29 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:29.711+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:11.881579+0000 front 2024-06-03T03:30:11.881609+0000 (oldest deadline 2024-06-03T03:30:37.781638+0000) 2024-06-03T03:31:29.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:29 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:29.711+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:17.782146+0000 front 2024-06-03T03:30:17.782047+0000 (oldest deadline 2024-06-03T03:30:41.882207+0000) 2024-06-03T03:31:29.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:29 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:29.712+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:37.585191+0000 front 2024-06-03T03:30:37.585172+0000 (oldest deadline 2024-06-03T03:30:58.685181+0000) 2024-06-03T03:31:29.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:29 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:29.712+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-06-03T03:30:43.886858+0000 front 2024-06-03T03:30:43.886895+0000 (oldest deadline 2024-06-03T03:31:09.186984+0000) 2024-06-03T03:31:29.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:29 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:29.712+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6822 osd.6 since back 2024-06-03T03:30:53.288159+0000 front 2024-06-03T03:30:53.288255+0000 (oldest deadline 2024-06-03T03:31:15.587766+0000) 2024-06-03T03:31:29.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:29 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:29.712+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6820 osd.7 since back 2024-06-03T03:31:03.089618+0000 front 2024-06-03T03:31:03.089721+0000 (oldest deadline 2024-06-03T03:31:24.789642+0000) 2024-06-03T03:31:30.097 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:29 smithi151 podman[71878]: 2024-06-03 03:31:29.846061998 +0000 UTC m=+5.908515695 container cleanup a90a223158ba49c4b2bce76717bf1e257264c08ee5badc99ccd49d8d22ccbccf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-03T03:31:30.097 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:29 smithi151 bash[71878]: ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10 2024-06-03T03:31:30.406 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:30 smithi151 podman[71896]: 2024-06-03 03:31:30.097545107 +0000 UTC m=+0.608762127 container remove a90a223158ba49c4b2bce76717bf1e257264c08ee5badc99ccd49d8d22ccbccf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240528, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-06-03T03:31:30.684 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:30 smithi151 podman[71984]: 2024-06-03 03:31:30.324802293 +0000 UTC m=+0.017379818 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T03:31:30.684 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:30 smithi151 podman[71984]: 2024-06-03 03:31:30.431706796 +0000 UTC m=+0.124284302 container create f930cff41e3065e67e8b532002951e9c3404568e6e98c8706c36de097df4a5d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-03T03:31:30.684 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:30 smithi151 podman[71984]: 2024-06-03 03:31:30.684465924 +0000 UTC m=+0.377043441 container init f930cff41e3065e67e8b532002951e9c3404568e6e98c8706c36de097df4a5d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-03T03:31:30.980 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:30 smithi151 podman[71984]: 2024-06-03 03:31:30.687685318 +0000 UTC m=+0.380262835 container start f930cff41e3065e67e8b532002951e9c3404568e6e98c8706c36de097df4a5d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-3d89acc, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, io.buildah.version=1.33.5) 2024-06-03T03:31:30.980 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:30 smithi151 podman[71984]: 2024-06-03 03:31:30.767839929 +0000 UTC m=+0.460417439 container attach f930cff41e3065e67e8b532002951e9c3404568e6e98c8706c36de097df4a5d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-03T03:31:31.233 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:30 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:30.759+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:01.379911+0000 front 2024-06-03T03:30:01.380025+0000 (oldest deadline 2024-06-03T03:30:25.479720+0000) 2024-06-03T03:31:31.233 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:30 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:30.759+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:11.881579+0000 front 2024-06-03T03:30:11.881609+0000 (oldest deadline 2024-06-03T03:30:37.781638+0000) 2024-06-03T03:31:31.234 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:30 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:30.759+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:17.782146+0000 front 2024-06-03T03:30:17.782047+0000 (oldest deadline 2024-06-03T03:30:41.882207+0000) 2024-06-03T03:31:31.234 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:30 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:30.759+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:37.585191+0000 front 2024-06-03T03:30:37.585172+0000 (oldest deadline 2024-06-03T03:30:58.685181+0000) 2024-06-03T03:31:31.234 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:30 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:30.759+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-06-03T03:30:43.886858+0000 front 2024-06-03T03:30:43.886895+0000 (oldest deadline 2024-06-03T03:31:09.186984+0000) 2024-06-03T03:31:31.234 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:30 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:30.759+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6822 osd.6 since back 2024-06-03T03:30:53.288159+0000 front 2024-06-03T03:30:53.288255+0000 (oldest deadline 2024-06-03T03:31:15.587766+0000) 2024-06-03T03:31:31.234 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:30 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:30.759+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6820 osd.7 since back 2024-06-03T03:31:03.089618+0000 front 2024-06-03T03:31:03.089721+0000 (oldest deadline 2024-06-03T03:31:24.789642+0000) 2024-06-03T03:31:31.282 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:30 smithi151 podman[71984]: 2024-06-03 03:31:30.981074935 +0000 UTC m=+0.673652462 container died f930cff41e3065e67e8b532002951e9c3404568e6e98c8706c36de097df4a5d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240528) 2024-06-03T03:31:31.532 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@osd.10.service' 2024-06-03T03:31:31.596 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:31 smithi151 podman[72010]: 2024-06-03 03:31:31.507118128 +0000 UTC m=+0.520680627 container remove f930cff41e3065e67e8b532002951e9c3404568e6e98c8706c36de097df4a5d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-10-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-06-03T03:31:31.596 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:31 smithi151 systemd[1]: ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@osd.10.service: Deactivated successfully. 2024-06-03T03:31:31.597 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:31 smithi151 systemd[1]: Stopped Ceph osd.10 for a0e8473a-2156-11ef-bc9b-c7b262605968. 2024-06-03T03:31:31.597 INFO:journalctl@ceph.osd.10.smithi151.stdout:Jun 03 03:31:31 smithi151 systemd[1]: ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@osd.10.service: Consumed 9.971s CPU time. 2024-06-03T03:31:31.612 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T03:31:31.612 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-06-03T03:31:31.613 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-06-03T03:31:31.613 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@osd.11 2024-06-03T03:31:31.920 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:31 smithi183 systemd[1]: Stopping Ceph osd.11 for a0e8473a-2156-11ef-bc9b-c7b262605968... 2024-06-03T03:31:31.921 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:31 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:31.779+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:01.379911+0000 front 2024-06-03T03:30:01.380025+0000 (oldest deadline 2024-06-03T03:30:25.479720+0000) 2024-06-03T03:31:31.921 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:31 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:31.779+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:11.881579+0000 front 2024-06-03T03:30:11.881609+0000 (oldest deadline 2024-06-03T03:30:37.781638+0000) 2024-06-03T03:31:31.921 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:31 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:31.779+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:17.782146+0000 front 2024-06-03T03:30:17.782047+0000 (oldest deadline 2024-06-03T03:30:41.882207+0000) 2024-06-03T03:31:31.921 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:31 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:31.779+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:37.585191+0000 front 2024-06-03T03:30:37.585172+0000 (oldest deadline 2024-06-03T03:30:58.685181+0000) 2024-06-03T03:31:31.921 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:31 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:31.779+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-06-03T03:30:43.886858+0000 front 2024-06-03T03:30:43.886895+0000 (oldest deadline 2024-06-03T03:31:09.186984+0000) 2024-06-03T03:31:31.921 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:31 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:31.779+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6822 osd.6 since back 2024-06-03T03:30:53.288159+0000 front 2024-06-03T03:30:53.288255+0000 (oldest deadline 2024-06-03T03:31:15.587766+0000) 2024-06-03T03:31:31.921 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:31 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:31.779+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6820 osd.7 since back 2024-06-03T03:31:03.089618+0000 front 2024-06-03T03:31:03.089721+0000 (oldest deadline 2024-06-03T03:31:24.789642+0000) 2024-06-03T03:31:31.921 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:31 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:31.919+0000 7fd8bf28e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T03:31:31.921 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:31 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:31.919+0000 7fd8bf28e700 -1 osd.11 103 *** Got signal Terminated *** 2024-06-03T03:31:31.921 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:31 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:31.919+0000 7fd8bf28e700 -1 osd.11 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-03T03:31:33.232 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:32 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:32.732+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:01.379911+0000 front 2024-06-03T03:30:01.380025+0000 (oldest deadline 2024-06-03T03:30:25.479720+0000) 2024-06-03T03:31:33.232 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:32 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:32.732+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:11.881579+0000 front 2024-06-03T03:30:11.881609+0000 (oldest deadline 2024-06-03T03:30:37.781638+0000) 2024-06-03T03:31:33.232 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:32 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:32.732+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:17.782146+0000 front 2024-06-03T03:30:17.782047+0000 (oldest deadline 2024-06-03T03:30:41.882207+0000) 2024-06-03T03:31:33.232 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:32 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:32.732+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:37.585191+0000 front 2024-06-03T03:30:37.585172+0000 (oldest deadline 2024-06-03T03:30:58.685181+0000) 2024-06-03T03:31:33.232 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:32 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:32.732+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-06-03T03:30:43.886858+0000 front 2024-06-03T03:30:43.886895+0000 (oldest deadline 2024-06-03T03:31:09.186984+0000) 2024-06-03T03:31:33.232 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:32 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:32.732+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6822 osd.6 since back 2024-06-03T03:30:53.288159+0000 front 2024-06-03T03:30:53.288255+0000 (oldest deadline 2024-06-03T03:31:15.587766+0000) 2024-06-03T03:31:33.233 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:32 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:32.732+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6820 osd.7 since back 2024-06-03T03:31:03.089618+0000 front 2024-06-03T03:31:03.089721+0000 (oldest deadline 2024-06-03T03:31:24.789642+0000) 2024-06-03T03:31:33.233 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:32 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:32.732+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6820 osd.8 since back 2024-06-03T03:31:09.490679+0000 front 2024-06-03T03:31:09.490646+0000 (oldest deadline 2024-06-03T03:31:32.390687+0000) 2024-06-03T03:31:33.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:33 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:33.694+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:01.379911+0000 front 2024-06-03T03:30:01.380025+0000 (oldest deadline 2024-06-03T03:30:25.479720+0000) 2024-06-03T03:31:33.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:33 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:33.694+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:11.881579+0000 front 2024-06-03T03:30:11.881609+0000 (oldest deadline 2024-06-03T03:30:37.781638+0000) 2024-06-03T03:31:33.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:33 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:33.694+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:17.782146+0000 front 2024-06-03T03:30:17.782047+0000 (oldest deadline 2024-06-03T03:30:41.882207+0000) 2024-06-03T03:31:33.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:33 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:33.694+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:37.585191+0000 front 2024-06-03T03:30:37.585172+0000 (oldest deadline 2024-06-03T03:30:58.685181+0000) 2024-06-03T03:31:33.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:33 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:33.694+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-06-03T03:30:43.886858+0000 front 2024-06-03T03:30:43.886895+0000 (oldest deadline 2024-06-03T03:31:09.186984+0000) 2024-06-03T03:31:33.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:33 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:33.694+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6822 osd.6 since back 2024-06-03T03:30:53.288159+0000 front 2024-06-03T03:30:53.288255+0000 (oldest deadline 2024-06-03T03:31:15.587766+0000) 2024-06-03T03:31:33.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:33 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:33.694+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6820 osd.7 since back 2024-06-03T03:31:03.089618+0000 front 2024-06-03T03:31:03.089721+0000 (oldest deadline 2024-06-03T03:31:24.789642+0000) 2024-06-03T03:31:33.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:33 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:33.694+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6820 osd.8 since back 2024-06-03T03:31:09.490679+0000 front 2024-06-03T03:31:09.490646+0000 (oldest deadline 2024-06-03T03:31:32.390687+0000) 2024-06-03T03:31:34.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:34 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:34.692+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:01.379911+0000 front 2024-06-03T03:30:01.380025+0000 (oldest deadline 2024-06-03T03:30:25.479720+0000) 2024-06-03T03:31:34.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:34 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:34.692+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:11.881579+0000 front 2024-06-03T03:30:11.881609+0000 (oldest deadline 2024-06-03T03:30:37.781638+0000) 2024-06-03T03:31:34.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:34 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:34.692+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:17.782146+0000 front 2024-06-03T03:30:17.782047+0000 (oldest deadline 2024-06-03T03:30:41.882207+0000) 2024-06-03T03:31:34.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:34 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:34.692+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:37.585191+0000 front 2024-06-03T03:30:37.585172+0000 (oldest deadline 2024-06-03T03:30:58.685181+0000) 2024-06-03T03:31:34.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:34 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:34.692+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-06-03T03:30:43.886858+0000 front 2024-06-03T03:30:43.886895+0000 (oldest deadline 2024-06-03T03:31:09.186984+0000) 2024-06-03T03:31:34.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:34 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:34.692+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6822 osd.6 since back 2024-06-03T03:30:53.288159+0000 front 2024-06-03T03:30:53.288255+0000 (oldest deadline 2024-06-03T03:31:15.587766+0000) 2024-06-03T03:31:34.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:34 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:34.692+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6820 osd.7 since back 2024-06-03T03:31:03.089618+0000 front 2024-06-03T03:31:03.089721+0000 (oldest deadline 2024-06-03T03:31:24.789642+0000) 2024-06-03T03:31:34.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:34 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:34.692+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6820 osd.8 since back 2024-06-03T03:31:09.490679+0000 front 2024-06-03T03:31:09.490646+0000 (oldest deadline 2024-06-03T03:31:32.390687+0000) 2024-06-03T03:31:35.981 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:35 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:35.709+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:01.379911+0000 front 2024-06-03T03:30:01.380025+0000 (oldest deadline 2024-06-03T03:30:25.479720+0000) 2024-06-03T03:31:35.981 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:35 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:35.709+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:11.881579+0000 front 2024-06-03T03:30:11.881609+0000 (oldest deadline 2024-06-03T03:30:37.781638+0000) 2024-06-03T03:31:35.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:35 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:35.709+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:17.782146+0000 front 2024-06-03T03:30:17.782047+0000 (oldest deadline 2024-06-03T03:30:41.882207+0000) 2024-06-03T03:31:35.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:35 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:35.709+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:37.585191+0000 front 2024-06-03T03:30:37.585172+0000 (oldest deadline 2024-06-03T03:30:58.685181+0000) 2024-06-03T03:31:35.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:35 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:35.709+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-06-03T03:30:43.886858+0000 front 2024-06-03T03:30:43.886895+0000 (oldest deadline 2024-06-03T03:31:09.186984+0000) 2024-06-03T03:31:35.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:35 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:35.709+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6822 osd.6 since back 2024-06-03T03:30:53.288159+0000 front 2024-06-03T03:30:53.288255+0000 (oldest deadline 2024-06-03T03:31:15.587766+0000) 2024-06-03T03:31:35.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:35 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:35.709+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6820 osd.7 since back 2024-06-03T03:31:03.089618+0000 front 2024-06-03T03:31:03.089721+0000 (oldest deadline 2024-06-03T03:31:24.789642+0000) 2024-06-03T03:31:35.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:35 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:35.709+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6820 osd.8 since back 2024-06-03T03:31:09.490679+0000 front 2024-06-03T03:31:09.490646+0000 (oldest deadline 2024-06-03T03:31:32.390687+0000) 2024-06-03T03:31:36.983 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:36 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:36.688+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-06-03T03:30:01.379911+0000 front 2024-06-03T03:30:01.380025+0000 (oldest deadline 2024-06-03T03:30:25.479720+0000) 2024-06-03T03:31:36.983 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:36 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:36.688+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6804 osd.1 since back 2024-06-03T03:30:11.881579+0000 front 2024-06-03T03:30:11.881609+0000 (oldest deadline 2024-06-03T03:30:37.781638+0000) 2024-06-03T03:31:36.983 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:36 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:36.688+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-06-03T03:30:17.782146+0000 front 2024-06-03T03:30:17.782047+0000 (oldest deadline 2024-06-03T03:30:41.882207+0000) 2024-06-03T03:31:36.984 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:36 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:36.688+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6812 osd.4 since back 2024-06-03T03:30:37.585191+0000 front 2024-06-03T03:30:37.585172+0000 (oldest deadline 2024-06-03T03:30:58.685181+0000) 2024-06-03T03:31:36.984 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:36 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:36.688+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-06-03T03:30:43.886858+0000 front 2024-06-03T03:30:43.886895+0000 (oldest deadline 2024-06-03T03:31:09.186984+0000) 2024-06-03T03:31:36.984 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:36 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:36.688+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.117:6822 osd.6 since back 2024-06-03T03:30:53.288159+0000 front 2024-06-03T03:30:53.288255+0000 (oldest deadline 2024-06-03T03:31:15.587766+0000) 2024-06-03T03:31:36.984 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:36 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:36.688+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.151:6820 osd.7 since back 2024-06-03T03:31:03.089618+0000 front 2024-06-03T03:31:03.089721+0000 (oldest deadline 2024-06-03T03:31:24.789642+0000) 2024-06-03T03:31:36.984 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:36 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11[62631]: 2024-06-03T03:31:36.688+0000 7fd8b787a700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.183:6820 osd.8 since back 2024-06-03T03:31:09.490679+0000 front 2024-06-03T03:31:09.490646+0000 (oldest deadline 2024-06-03T03:31:32.390687+0000) 2024-06-03T03:31:37.663 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:37 smithi183 podman[72492]: 2024-06-03 03:31:37.404473311 +0000 UTC m=+5.592836365 container died e810848ad39abfa2938de1cfeabb66f2fdebd42a8779283ece6c129294ad14bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-03T03:31:37.981 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:37 smithi183 podman[72492]: 2024-06-03 03:31:37.753387875 +0000 UTC m=+5.941750928 container cleanup e810848ad39abfa2938de1cfeabb66f2fdebd42a8779283ece6c129294ad14bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-03T03:31:37.981 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:37 smithi183 bash[72492]: ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11 2024-06-03T03:31:38.374 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:38 smithi183 podman[72508]: 2024-06-03 03:31:38.029373783 +0000 UTC m=+0.624857177 container remove e810848ad39abfa2938de1cfeabb66f2fdebd42a8779283ece6c129294ad14bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=) 2024-06-03T03:31:38.682 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:38 smithi183 podman[72601]: 2024-06-03 03:31:38.27762903 +0000 UTC m=+0.017389489 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T03:31:38.682 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:38 smithi183 podman[72601]: 2024-06-03 03:31:38.380040792 +0000 UTC m=+0.119801278 container create dafbf733554fe7dbb53522b8b15cf7ffccf0a5fc3b50373942d1c91da72d7172 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc) 2024-06-03T03:31:38.682 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:38 smithi183 podman[72601]: 2024-06-03 03:31:38.591052787 +0000 UTC m=+0.330813230 container init dafbf733554fe7dbb53522b8b15cf7ffccf0a5fc3b50373942d1c91da72d7172 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=) 2024-06-03T03:31:38.682 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:38 smithi183 podman[72601]: 2024-06-03 03:31:38.594428111 +0000 UTC m=+0.334188551 container start dafbf733554fe7dbb53522b8b15cf7ffccf0a5fc3b50373942d1c91da72d7172 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-06-03T03:31:38.682 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:38 smithi183 podman[72601]: 2024-06-03 03:31:38.682224729 +0000 UTC m=+0.421985187 container attach dafbf733554fe7dbb53522b8b15cf7ffccf0a5fc3b50373942d1c91da72d7172 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-06-03T03:31:38.982 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:38 smithi183 podman[72601]: 2024-06-03 03:31:38.887775251 +0000 UTC m=+0.627535692 container died dafbf733554fe7dbb53522b8b15cf7ffccf0a5fc3b50373942d1c91da72d7172 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11-deactivate, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-06-03T03:31:39.433 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@osd.11.service' 2024-06-03T03:31:39.459 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:39 smithi183 podman[72628]: 2024-06-03 03:31:39.411358748 +0000 UTC m=+0.513566816 container remove dafbf733554fe7dbb53522b8b15cf7ffccf0a5fc3b50373942d1c91da72d7172 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-osd-11-deactivate, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-06-03T03:31:39.459 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:39 smithi183 systemd[1]: ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@osd.11.service: Deactivated successfully. 2024-06-03T03:31:39.459 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:39 smithi183 systemd[1]: Stopped Ceph osd.11 for a0e8473a-2156-11ef-bc9b-c7b262605968. 2024-06-03T03:31:39.459 INFO:journalctl@ceph.osd.11.smithi183.stdout:Jun 03 03:31:39 smithi183 systemd[1]: ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@osd.11.service: Consumed 8.221s CPU time. 2024-06-03T03:31:39.477 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T03:31:39.477 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-06-03T03:31:39.477 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-06-03T03:31:39.477 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@mds.a 2024-06-03T03:31:39.946 INFO:journalctl@ceph.mds.a.smithi117.stdout:Jun 03 03:31:39 smithi117 systemd[1]: Stopping Ceph mds.a for a0e8473a-2156-11ef-bc9b-c7b262605968... 2024-06-03T03:31:40.446 INFO:journalctl@ceph.mds.a.smithi117.stdout:Jun 03 03:31:40 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-a[71198]: 2024-06-03T03:31:40.039+0000 7f00b5200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T03:31:40.447 INFO:journalctl@ceph.mds.a.smithi117.stdout:Jun 03 03:31:40 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-a[71198]: 2024-06-03T03:31:40.039+0000 7f00b5200700 -1 mds.a *** got signal Terminated *** 2024-06-03T03:31:43.446 INFO:journalctl@ceph.mds.a.smithi117.stdout:Jun 03 03:31:43 smithi117 podman[99089]: 2024-06-03 03:31:43.153753155 +0000 UTC m=+3.253841898 container died 85a7fbdf4fdfcb6ce84706c3fa6c2af895304ca52171ea0574876d69742df3d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-a, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-06-03T03:31:43.756 INFO:journalctl@ceph.mds.a.smithi117.stdout:Jun 03 03:31:43 smithi117 podman[99089]: 2024-06-03 03:31:43.724938858 +0000 UTC m=+3.825027630 container cleanup 85a7fbdf4fdfcb6ce84706c3fa6c2af895304ca52171ea0574876d69742df3d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-a, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-06-03T03:31:43.756 INFO:journalctl@ceph.mds.a.smithi117.stdout:Jun 03 03:31:43 smithi117 bash[99089]: ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-a 2024-06-03T03:31:44.196 INFO:journalctl@ceph.mds.a.smithi117.stdout:Jun 03 03:31:43 smithi117 podman[99103]: 2024-06-03 03:31:43.95110629 +0000 UTC m=+0.778631399 container remove 85a7fbdf4fdfcb6ce84706c3fa6c2af895304ca52171ea0574876d69742df3d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-a, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-06-03T03:31:44.512 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@mds.a.service' 2024-06-03T03:31:44.624 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T03:31:44.625 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-06-03T03:31:44.625 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-06-03T03:31:44.625 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@mds.d 2024-06-03T03:31:45.196 INFO:journalctl@ceph.mds.d.smithi117.stdout:Jun 03 03:31:44 smithi117 systemd[1]: Stopping Ceph mds.d for a0e8473a-2156-11ef-bc9b-c7b262605968... 2024-06-03T03:31:45.696 INFO:journalctl@ceph.mds.d.smithi117.stdout:Jun 03 03:31:45 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-d[70018]: 2024-06-03T03:31:45.201+0000 7f74aaa00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T03:31:45.696 INFO:journalctl@ceph.mds.d.smithi117.stdout:Jun 03 03:31:45 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-d[70018]: 2024-06-03T03:31:45.201+0000 7f74aaa00700 -1 mds.d *** got signal Terminated *** 2024-06-03T03:31:49.485 INFO:journalctl@ceph.mds.d.smithi117.stdout:Jun 03 03:31:49 smithi117 podman[99212]: 2024-06-03 03:31:49.222479925 +0000 UTC m=+4.170399789 container died d731bf2a4312788c35dd6b92708cd0ead80445a1bd50e4baec045dc39902dccf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc) 2024-06-03T03:31:49.905 INFO:journalctl@ceph.mds.d.smithi117.stdout:Jun 03 03:31:49 smithi117 podman[99212]: 2024-06-03 03:31:49.678400344 +0000 UTC m=+4.626320208 container cleanup d731bf2a4312788c35dd6b92708cd0ead80445a1bd50e4baec045dc39902dccf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-d, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-03T03:31:49.906 INFO:journalctl@ceph.mds.d.smithi117.stdout:Jun 03 03:31:49 smithi117 bash[99212]: ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-d 2024-06-03T03:31:50.196 INFO:journalctl@ceph.mds.d.smithi117.stdout:Jun 03 03:31:49 smithi117 podman[99237]: 2024-06-03 03:31:49.904614342 +0000 UTC m=+0.664656096 container remove d731bf2a4312788c35dd6b92708cd0ead80445a1bd50e4baec045dc39902dccf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-d, GIT_CLEAN=True, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD) 2024-06-03T03:31:50.430 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@mds.d.service' 2024-06-03T03:31:50.539 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T03:31:50.539 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-06-03T03:31:50.540 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-06-03T03:31:50.540 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@mds.g 2024-06-03T03:31:50.946 INFO:journalctl@ceph.mds.g.smithi117.stdout:Jun 03 03:31:50 smithi117 systemd[1]: Stopping Ceph mds.g for a0e8473a-2156-11ef-bc9b-c7b262605968... 2024-06-03T03:31:51.446 INFO:journalctl@ceph.mds.g.smithi117.stdout:Jun 03 03:31:51 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-g[70802]: 2024-06-03T03:31:51.171+0000 7fd625800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T03:31:51.447 INFO:journalctl@ceph.mds.g.smithi117.stdout:Jun 03 03:31:51 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-g[70802]: 2024-06-03T03:31:51.171+0000 7fd625800700 -1 mds.g *** got signal Terminated *** 2024-06-03T03:31:53.822 INFO:journalctl@ceph.mds.g.smithi117.stdout:Jun 03 03:31:53 smithi117 podman[99344]: 2024-06-03 03:31:53.505713987 +0000 UTC m=+2.502979030 container died 7ea4d8f5f9746accef2fc49a6f4343d7231a9405df0e0f0077722066a5d8c1ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-g, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240528) 2024-06-03T03:31:54.401 INFO:journalctl@ceph.mds.g.smithi117.stdout:Jun 03 03:31:54 smithi117 podman[99344]: 2024-06-03 03:31:54.073184235 +0000 UTC m=+3.070449263 container cleanup 7ea4d8f5f9746accef2fc49a6f4343d7231a9405df0e0f0077722066a5d8c1ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-g, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-06-03T03:31:54.401 INFO:journalctl@ceph.mds.g.smithi117.stdout:Jun 03 03:31:54 smithi117 bash[99344]: ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-g 2024-06-03T03:31:54.696 INFO:journalctl@ceph.mds.g.smithi117.stdout:Jun 03 03:31:54 smithi117 podman[99357]: 2024-06-03 03:31:54.399968956 +0000 UTC m=+0.885857781 container remove 7ea4d8f5f9746accef2fc49a6f4343d7231a9405df0e0f0077722066a5d8c1ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-g, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528) 2024-06-03T03:31:55.037 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@mds.g.service' 2024-06-03T03:31:55.132 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T03:31:55.133 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-06-03T03:31:55.133 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-06-03T03:31:55.133 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@mds.j 2024-06-03T03:31:55.696 INFO:journalctl@ceph.mds.j.smithi117.stdout:Jun 03 03:31:55 smithi117 systemd[1]: Stopping Ceph mds.j for a0e8473a-2156-11ef-bc9b-c7b262605968... 2024-06-03T03:31:56.196 INFO:journalctl@ceph.mds.j.smithi117.stdout:Jun 03 03:31:55 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-j[69608]: 2024-06-03T03:31:55.701+0000 7fcf67800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T03:31:56.197 INFO:journalctl@ceph.mds.j.smithi117.stdout:Jun 03 03:31:55 smithi117 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-j[69608]: 2024-06-03T03:31:55.701+0000 7fcf67800700 -1 mds.j *** got signal Terminated *** 2024-06-03T03:31:59.447 INFO:journalctl@ceph.mds.j.smithi117.stdout:Jun 03 03:31:59 smithi117 podman[99466]: 2024-06-03 03:31:59.184599417 +0000 UTC m=+3.617379053 container stop acc516aa313c6e1f542a16c863765ce05cca030da9c0bc8585a8cf3d2dddb75f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-j, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-03T03:31:59.447 INFO:journalctl@ceph.mds.j.smithi117.stdout:Jun 03 03:31:59 smithi117 podman[99466]: 2024-06-03 03:31:59.215029953 +0000 UTC m=+3.647809585 container died acc516aa313c6e1f542a16c863765ce05cca030da9c0bc8585a8cf3d2dddb75f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-j, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-06-03T03:31:59.737 INFO:journalctl@ceph.mds.j.smithi117.stdout:Jun 03 03:31:59 smithi117 podman[99466]: 2024-06-03 03:31:59.726714716 +0000 UTC m=+4.159494349 container cleanup acc516aa313c6e1f542a16c863765ce05cca030da9c0bc8585a8cf3d2dddb75f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-j, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-06-03T03:31:59.738 INFO:journalctl@ceph.mds.j.smithi117.stdout:Jun 03 03:31:59 smithi117 bash[99466]: ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-j 2024-06-03T03:32:00.395 INFO:journalctl@ceph.mds.j.smithi117.stdout:Jun 03 03:31:59 smithi117 podman[99478]: 2024-06-03 03:31:59.994452694 +0000 UTC m=+0.787222884 container remove acc516aa313c6e1f542a16c863765ce05cca030da9c0bc8585a8cf3d2dddb75f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-j, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528) 2024-06-03T03:32:00.437 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@mds.j.service' 2024-06-03T03:32:00.541 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T03:32:00.542 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-06-03T03:32:00.542 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-06-03T03:32:00.542 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@mds.b 2024-06-03T03:32:00.849 INFO:journalctl@ceph.mds.b.smithi151.stdout:Jun 03 03:32:00 smithi151 systemd[1]: Stopping Ceph mds.b for a0e8473a-2156-11ef-bc9b-c7b262605968... 2024-06-03T03:32:01.156 INFO:journalctl@ceph.mds.b.smithi151.stdout:Jun 03 03:32:00 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-b[67242]: 2024-06-03T03:32:00.848+0000 7f87ad553700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T03:32:01.156 INFO:journalctl@ceph.mds.b.smithi151.stdout:Jun 03 03:32:00 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-b[67242]: 2024-06-03T03:32:00.848+0000 7f87ad553700 -1 mds.b *** got signal Terminated *** 2024-06-03T03:32:03.367 INFO:journalctl@ceph.mds.b.smithi151.stdout:Jun 03 03:32:03 smithi151 podman[72089]: 2024-06-03 03:32:03.111023224 +0000 UTC m=+2.372890538 container died d52b90d82890f0419d36ec5576119bdbc9cae6385ec09eb82bd963456ecf7b8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-b, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-3d89acc) 2024-06-03T03:32:03.656 INFO:journalctl@ceph.mds.b.smithi151.stdout:Jun 03 03:32:03 smithi151 podman[72089]: 2024-06-03 03:32:03.440984639 +0000 UTC m=+2.702851952 container cleanup d52b90d82890f0419d36ec5576119bdbc9cae6385ec09eb82bd963456ecf7b8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-b, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-06-03T03:32:03.656 INFO:journalctl@ceph.mds.b.smithi151.stdout:Jun 03 03:32:03 smithi151 bash[72089]: ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-b 2024-06-03T03:32:03.924 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@mds.b.service' 2024-06-03T03:32:03.960 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T03:32:03.960 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-06-03T03:32:03.960 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-06-03T03:32:03.961 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@mds.e 2024-06-03T03:32:04.406 INFO:journalctl@ceph.mds.e.smithi151.stdout:Jun 03 03:32:04 smithi151 systemd[1]: Stopping Ceph mds.e for a0e8473a-2156-11ef-bc9b-c7b262605968... 2024-06-03T03:32:04.406 INFO:journalctl@ceph.mds.e.smithi151.stdout:Jun 03 03:32:04 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-e[66639]: 2024-06-03T03:32:04.261+0000 7fade68f2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T03:32:04.406 INFO:journalctl@ceph.mds.e.smithi151.stdout:Jun 03 03:32:04 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-e[66639]: 2024-06-03T03:32:04.261+0000 7fade68f2700 -1 mds.e *** got signal Terminated *** 2024-06-03T03:32:07.396 INFO:journalctl@ceph.mds.e.smithi151.stdout:Jun 03 03:32:07 smithi151 podman[72222]: 2024-06-03 03:32:07.12975966 +0000 UTC m=+2.952794519 container died 427fca5011668a65822af965d667dc996432a1fc724a1baf615b3b0f59178bdf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-e, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, io.buildah.version=1.33.5) 2024-06-03T03:32:07.656 INFO:journalctl@ceph.mds.e.smithi151.stdout:Jun 03 03:32:07 smithi151 podman[72222]: 2024-06-03 03:32:07.495829048 +0000 UTC m=+3.318863899 container cleanup 427fca5011668a65822af965d667dc996432a1fc724a1baf615b3b0f59178bdf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-e, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5) 2024-06-03T03:32:07.656 INFO:journalctl@ceph.mds.e.smithi151.stdout:Jun 03 03:32:07 smithi151 bash[72222]: ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-e 2024-06-03T03:32:07.924 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@mds.e.service' 2024-06-03T03:32:07.956 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T03:32:07.956 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-06-03T03:32:07.957 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-06-03T03:32:07.957 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@mds.h 2024-06-03T03:32:08.406 INFO:journalctl@ceph.mds.h.smithi151.stdout:Jun 03 03:32:08 smithi151 systemd[1]: Stopping Ceph mds.h for a0e8473a-2156-11ef-bc9b-c7b262605968... 2024-06-03T03:32:08.406 INFO:journalctl@ceph.mds.h.smithi151.stdout:Jun 03 03:32:08 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-h[66942]: 2024-06-03T03:32:08.266+0000 7f5c7688a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T03:32:08.406 INFO:journalctl@ceph.mds.h.smithi151.stdout:Jun 03 03:32:08 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-h[66942]: 2024-06-03T03:32:08.266+0000 7f5c7688a700 -1 mds.h *** got signal Terminated *** 2024-06-03T03:32:11.384 INFO:journalctl@ceph.mds.h.smithi151.stdout:Jun 03 03:32:11 smithi151 podman[72344]: 2024-06-03 03:32:11.125949016 +0000 UTC m=+2.952186651 container died ec4dc6e6e0197519548bd55228fd58c49aa5baff6d11cfa517061b80d7229a26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-h, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, io.buildah.version=1.33.5) 2024-06-03T03:32:11.656 INFO:journalctl@ceph.mds.h.smithi151.stdout:Jun 03 03:32:11 smithi151 podman[72344]: 2024-06-03 03:32:11.476048217 +0000 UTC m=+3.302285865 container cleanup ec4dc6e6e0197519548bd55228fd58c49aa5baff6d11cfa517061b80d7229a26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True) 2024-06-03T03:32:11.656 INFO:journalctl@ceph.mds.h.smithi151.stdout:Jun 03 03:32:11 smithi151 bash[72344]: ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-h 2024-06-03T03:32:11.904 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@mds.h.service' 2024-06-03T03:32:11.936 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T03:32:11.936 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-06-03T03:32:11.937 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-06-03T03:32:11.937 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@mds.k 2024-06-03T03:32:12.156 INFO:journalctl@ceph.mds.k.smithi151.stdout:Jun 03 03:32:12 smithi151 systemd[1]: Stopping Ceph mds.k for a0e8473a-2156-11ef-bc9b-c7b262605968... 2024-06-03T03:32:12.656 INFO:journalctl@ceph.mds.k.smithi151.stdout:Jun 03 03:32:12 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-k[66328]: 2024-06-03T03:32:12.245+0000 7fb56a8ae700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T03:32:12.656 INFO:journalctl@ceph.mds.k.smithi151.stdout:Jun 03 03:32:12 smithi151 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-k[66328]: 2024-06-03T03:32:12.245+0000 7fb56a8ae700 -1 mds.k *** got signal Terminated *** 2024-06-03T03:32:15.380 INFO:journalctl@ceph.mds.k.smithi151.stdout:Jun 03 03:32:15 smithi151 podman[72464]: 2024-06-03 03:32:15.109060492 +0000 UTC m=+2.958485375 container died 8f37978860c8f3f84836688ea416e6287ef69b0dd92bcf246dab4998407f5195 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-k, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528) 2024-06-03T03:32:15.630 INFO:journalctl@ceph.mds.k.smithi151.stdout:Jun 03 03:32:15 smithi151 podman[72464]: 2024-06-03 03:32:15.463018963 +0000 UTC m=+3.312443838 container cleanup 8f37978860c8f3f84836688ea416e6287ef69b0dd92bcf246dab4998407f5195 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2) 2024-06-03T03:32:15.631 INFO:journalctl@ceph.mds.k.smithi151.stdout:Jun 03 03:32:15 smithi151 bash[72464]: ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-k 2024-06-03T03:32:15.836 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@mds.k.service' 2024-06-03T03:32:15.866 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T03:32:15.867 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-06-03T03:32:15.867 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-06-03T03:32:15.867 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@mds.c 2024-06-03T03:32:16.231 INFO:journalctl@ceph.mds.c.smithi183.stdout:Jun 03 03:32:15 smithi183 systemd[1]: Stopping Ceph mds.c for a0e8473a-2156-11ef-bc9b-c7b262605968... 2024-06-03T03:32:16.494 INFO:journalctl@ceph.mds.c.smithi183.stdout:Jun 03 03:32:16 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-c[67597]: 2024-06-03T03:32:16.242+0000 7f652cc72700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T03:32:16.494 INFO:journalctl@ceph.mds.c.smithi183.stdout:Jun 03 03:32:16 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-c[67597]: 2024-06-03T03:32:16.242+0000 7f652cc72700 -1 mds.c *** got signal Terminated *** 2024-06-03T03:32:19.482 INFO:journalctl@ceph.mds.c.smithi183.stdout:Jun 03 03:32:19 smithi183 podman[72709]: 2024-06-03 03:32:19.115569411 +0000 UTC m=+3.053620380 container died b98b31aef053566c44785052ac1906a03481c0e961c1abe3cc44d8df0ab9bad6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-c, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-06-03T03:32:19.852 INFO:journalctl@ceph.mds.c.smithi183.stdout:Jun 03 03:32:19 smithi183 podman[72709]: 2024-06-03 03:32:19.625364665 +0000 UTC m=+3.563415633 container cleanup b98b31aef053566c44785052ac1906a03481c0e961c1abe3cc44d8df0ab9bad6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-c, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5) 2024-06-03T03:32:19.852 INFO:journalctl@ceph.mds.c.smithi183.stdout:Jun 03 03:32:19 smithi183 bash[72709]: ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-c 2024-06-03T03:32:20.103 INFO:journalctl@ceph.mds.c.smithi183.stdout:Jun 03 03:32:19 smithi183 podman[72732]: 2024-06-03 03:32:19.852016839 +0000 UTC m=+0.734122711 container remove b98b31aef053566c44785052ac1906a03481c0e961c1abe3cc44d8df0ab9bad6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-c, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux ) 2024-06-03T03:32:20.107 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@mds.c.service' 2024-06-03T03:32:20.142 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T03:32:20.142 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-06-03T03:32:20.142 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-06-03T03:32:20.143 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@mds.f 2024-06-03T03:32:20.463 INFO:journalctl@ceph.mds.f.smithi183.stdout:Jun 03 03:32:20 smithi183 systemd[1]: Stopping Ceph mds.f for a0e8473a-2156-11ef-bc9b-c7b262605968... 2024-06-03T03:32:20.731 INFO:journalctl@ceph.mds.f.smithi183.stdout:Jun 03 03:32:20 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-f[67293]: 2024-06-03T03:32:20.463+0000 7f24f90ed700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T03:32:20.732 INFO:journalctl@ceph.mds.f.smithi183.stdout:Jun 03 03:32:20 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-f[67293]: 2024-06-03T03:32:20.463+0000 7f24f90ed700 -1 mds.f *** got signal Terminated *** 2024-06-03T03:32:23.372 INFO:journalctl@ceph.mds.f.smithi183.stdout:Jun 03 03:32:23 smithi183 podman[72843]: 2024-06-03 03:32:23.116436458 +0000 UTC m=+2.756248983 container died 9477330b41945ed7fd5ed49deb4a6f497382a7dcfcea2a04570a233589b6b131 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-f, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-06-03T03:32:23.730 INFO:journalctl@ceph.mds.f.smithi183.stdout:Jun 03 03:32:23 smithi183 podman[72843]: 2024-06-03 03:32:23.47123159 +0000 UTC m=+3.111044115 container cleanup 9477330b41945ed7fd5ed49deb4a6f497382a7dcfcea2a04570a233589b6b131 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-f, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2) 2024-06-03T03:32:23.730 INFO:journalctl@ceph.mds.f.smithi183.stdout:Jun 03 03:32:23 smithi183 bash[72843]: ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-f 2024-06-03T03:32:23.902 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@mds.f.service' 2024-06-03T03:32:23.934 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T03:32:23.934 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-06-03T03:32:23.935 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-06-03T03:32:23.935 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@mds.i 2024-06-03T03:32:24.481 INFO:journalctl@ceph.mds.i.smithi183.stdout:Jun 03 03:32:24 smithi183 systemd[1]: Stopping Ceph mds.i for a0e8473a-2156-11ef-bc9b-c7b262605968... 2024-06-03T03:32:24.482 INFO:journalctl@ceph.mds.i.smithi183.stdout:Jun 03 03:32:24 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-i[67909]: 2024-06-03T03:32:24.240+0000 7ffb4715c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T03:32:24.482 INFO:journalctl@ceph.mds.i.smithi183.stdout:Jun 03 03:32:24 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-i[67909]: 2024-06-03T03:32:24.240+0000 7ffb4715c700 -1 mds.i *** got signal Terminated *** 2024-06-03T03:32:27.231 INFO:journalctl@ceph.mds.i.smithi183.stdout:Jun 03 03:32:27 smithi183 podman[72962]: 2024-06-03 03:32:27.11663213 +0000 UTC m=+2.970494486 container died 71fa4d2bfe2ca211e5ad7fbe9d3510e2971251ce7223b690ad687f07daaf61ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-i, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-06-03T03:32:27.728 INFO:journalctl@ceph.mds.i.smithi183.stdout:Jun 03 03:32:27 smithi183 podman[72962]: 2024-06-03 03:32:27.486058004 +0000 UTC m=+3.339920361 container cleanup 71fa4d2bfe2ca211e5ad7fbe9d3510e2971251ce7223b690ad687f07daaf61ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528) 2024-06-03T03:32:27.728 INFO:journalctl@ceph.mds.i.smithi183.stdout:Jun 03 03:32:27 smithi183 bash[72962]: ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-i 2024-06-03T03:32:27.864 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@mds.i.service' 2024-06-03T03:32:27.896 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T03:32:27.896 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-06-03T03:32:27.896 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-06-03T03:32:27.897 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@mds.l 2024-06-03T03:32:27.981 INFO:journalctl@ceph.mds.l.smithi183.stdout:Jun 03 03:32:27 smithi183 systemd[1]: Stopping Ceph mds.l for a0e8473a-2156-11ef-bc9b-c7b262605968... 2024-06-03T03:32:28.481 INFO:journalctl@ceph.mds.l.smithi183.stdout:Jun 03 03:32:28 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-l[66988]: 2024-06-03T03:32:28.214+0000 7fe8ad33e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T03:32:28.481 INFO:journalctl@ceph.mds.l.smithi183.stdout:Jun 03 03:32:28 smithi183 ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-l[66988]: 2024-06-03T03:32:28.214+0000 7fe8ad33e700 -1 mds.l *** got signal Terminated *** 2024-06-03T03:32:31.382 INFO:journalctl@ceph.mds.l.smithi183.stdout:Jun 03 03:32:31 smithi183 podman[73080]: 2024-06-03 03:32:31.117283724 +0000 UTC m=+3.007320412 container died 148d60d50a9a8a210164585fdeab13b122a2c65809cb072385fe4c0e7b55dd0d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD) 2024-06-03T03:32:31.732 INFO:journalctl@ceph.mds.l.smithi183.stdout:Jun 03 03:32:31 smithi183 podman[73080]: 2024-06-03 03:32:31.464886624 +0000 UTC m=+3.354923354 container cleanup 148d60d50a9a8a210164585fdeab13b122a2c65809cb072385fe4c0e7b55dd0d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-l, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-06-03T03:32:31.732 INFO:journalctl@ceph.mds.l.smithi183.stdout:Jun 03 03:32:31 smithi183 bash[73080]: ceph-a0e8473a-2156-11ef-bc9b-c7b262605968-mds-l 2024-06-03T03:32:31.896 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a0e8473a-2156-11ef-bc9b-c7b262605968@mds.l.service' 2024-06-03T03:32:31.928 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T03:32:31.928 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-06-03T03:32:31.928 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 --force --keep-logs 2024-06-03T03:32:32.245 INFO:teuthology.orchestra.run.smithi117.stdout:Deleting cluster with fsid: a0e8473a-2156-11ef-bc9b-c7b262605968 2024-06-03T03:32:37.045 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 --force --keep-logs 2024-06-03T03:32:37.238 INFO:teuthology.orchestra.run.smithi151.stdout:Deleting cluster with fsid: a0e8473a-2156-11ef-bc9b-c7b262605968 2024-06-03T03:32:39.612 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 --force --keep-logs 2024-06-03T03:32:39.799 INFO:teuthology.orchestra.run.smithi183.stdout:Deleting cluster with fsid: a0e8473a-2156-11ef-bc9b-c7b262605968 2024-06-03T03:32:42.209 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-03T03:32:42.296 DEBUG:teuthology.orchestra.run.smithi151:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-03T03:32:42.327 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-03T03:32:42.353 INFO:tasks.cephadm:Archiving crash dumps... 2024-06-03T03:32:42.355 DEBUG:teuthology.misc:Transferring archived files from smithi117:/var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-06-02_21:24:02-fs-squid-distro-default-smithi/7738859/remote/smithi117/crash 2024-06-03T03:32:42.356 DEBUG:teuthology.orchestra.run.smithi117:> sudo tar c -f - -C /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/crash -- . 2024-06-03T03:32:42.466 INFO:teuthology.orchestra.run.smithi117.stderr:tar: /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/crash: Cannot open: No such file or directory 2024-06-03T03:32:42.466 INFO:teuthology.orchestra.run.smithi117.stderr:tar: Error is not recoverable: exiting now 2024-06-03T03:32:42.471 DEBUG:teuthology.misc:Transferring archived files from smithi151:/var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-06-02_21:24:02-fs-squid-distro-default-smithi/7738859/remote/smithi151/crash 2024-06-03T03:32:42.472 DEBUG:teuthology.orchestra.run.smithi151:> sudo tar c -f - -C /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/crash -- . 2024-06-03T03:32:42.514 INFO:teuthology.orchestra.run.smithi151.stderr:tar: /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/crash: Cannot open: No such file or directory 2024-06-03T03:32:42.515 INFO:teuthology.orchestra.run.smithi151.stderr:tar: Error is not recoverable: exiting now 2024-06-03T03:32:42.517 DEBUG:teuthology.misc:Transferring archived files from smithi183:/var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-06-02_21:24:02-fs-squid-distro-default-smithi/7738859/remote/smithi183/crash 2024-06-03T03:32:42.518 DEBUG:teuthology.orchestra.run.smithi183:> sudo tar c -f - -C /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/crash -- . 2024-06-03T03:32:42.552 INFO:teuthology.orchestra.run.smithi183.stderr:tar: /var/lib/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/crash: Cannot open: No such file or directory 2024-06-03T03:32:42.552 INFO:teuthology.orchestra.run.smithi183.stderr:tar: Error is not recoverable: exiting now 2024-06-03T03:32:42.554 INFO:tasks.cephadm:Checking cluster log for badness... 2024-06-03T03:32:42.554 DEBUG:teuthology.orchestra.run.smithi117:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-06-03T03:32:42.656 INFO:teuthology.orchestra.run.smithi117.stdout:2024-06-03T03:20:00.000145+0000 mon.a (mon.0) 1031 : cluster [WRN] Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled 2024-06-03T03:32:42.657 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-06-03T03:32:42.658 DEBUG:teuthology.orchestra.run.smithi117:> sudo egrep '\[SEC\]' /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-06-03T03:32:42.740 DEBUG:teuthology.orchestra.run.smithi117:> sudo egrep '\[ERR\]' /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-06-03T03:32:42.822 DEBUG:teuthology.orchestra.run.smithi117:> sudo egrep '\[WRN\]' /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-06-03T03:32:42.902 INFO:teuthology.orchestra.run.smithi117.stdout:2024-06-03T03:20:00.000145+0000 mon.a (mon.0) 1031 : cluster [WRN] Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled 2024-06-03T03:32:42.903 INFO:tasks.cephadm:Compressing logs... 2024-06-03T03:32:42.903 DEBUG:teuthology.orchestra.run.smithi117:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-06-03T03:32:42.907 DEBUG:teuthology.orchestra.run.smithi151:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-06-03T03:32:42.910 DEBUG:teuthology.orchestra.run.smithi183:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-06-03T03:32:42.932 INFO:teuthology.orchestra.run.smithi183.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph.log 2024-06-03T03:32:42.933 INFO:teuthology.orchestra.run.smithi183.stderr:: No such file or directory 2024-06-03T03:32:42.933 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-osd.2.log 2024-06-03T03:32:42.934 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-mds.f.log 2024-06-03T03:32:42.934 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph.audit.log 2024-06-03T03:32:42.934 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-osd.5.log 2024-06-03T03:32:42.934 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-mds.c.log 2024-06-03T03:32:42.936 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph.audit.log: /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-mon.c.log 2024-06-03T03:32:42.936 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-mgr.z.log 2024-06-03T03:32:42.936 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-mds.i.log 2024-06-03T03:32:42.936 INFO:teuthology.orchestra.run.smithi183.stderr: 87.1% -- replaced with /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph.log.gz 2024-06-03T03:32:42.937 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-mgr.z.log: 90.3% -- replaced with /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph.audit.log.gz 2024-06-03T03:32:42.938 INFO:teuthology.orchestra.run.smithi151.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph.log 2024-06-03T03:32:42.938 INFO:teuthology.orchestra.run.smithi151.stderr:: No such file or directory 2024-06-03T03:32:42.938 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph.audit.log 2024-06-03T03:32:42.938 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-osd.7.log 2024-06-03T03:32:42.938 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-mds.b.log 2024-06-03T03:32:42.942 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-osd.4.log 2024-06-03T03:32:42.942 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-osd.1.log 2024-06-03T03:32:42.942 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-mgr.y.log 2024-06-03T03:32:42.942 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-mds.h.log 2024-06-03T03:32:42.942 INFO:teuthology.orchestra.run.smithi151.stderr: 87.1%/var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-mgr.y.log: -- replaced with /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph.log.gz 2024-06-03T03:32:42.942 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-osd.10.log 2024-06-03T03:32:42.942 INFO:teuthology.orchestra.run.smithi151.stderr: 90.3% -- replaced with /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph.audit.log.gz 2024-06-03T03:32:42.942 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose/var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-mds.h.log: -- /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph.cephadm.log 2024-06-03T03:32:42.942 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-mds.e.log 2024-06-03T03:32:42.943 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph.cephadm.log: /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-osd.10.log: 82.4% -- replaced with /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph.cephadm.log.gz 2024-06-03T03:32:42.943 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-volume.log 2024-06-03T03:32:42.944 INFO:teuthology.orchestra.run.smithi183.stderr: 92.8% -- replaced with /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-mgr.z.log.gz 2024-06-03T03:32:42.944 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph.cephadm.log 2024-06-03T03:32:42.944 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-osd.11.log 2024-06-03T03:32:42.944 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-osd.8.log 2024-06-03T03:32:42.944 INFO:teuthology.orchestra.run.smithi183.stderr: 82.3% -- replaced with /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph.cephadm.log.gz 2024-06-03T03:32:42.945 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-mds.e.log: 92.6% -- replaced with /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-mgr.y.log.gz 2024-06-03T03:32:42.954 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-mon.b.log 2024-06-03T03:32:42.956 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-volume.log 2024-06-03T03:32:42.962 INFO:teuthology.orchestra.run.smithi117.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-06-03T03:32:42.963 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph.log 2024-06-03T03:32:42.964 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph.audit.log 2024-06-03T03:32:42.965 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-mds.g.log 2024-06-03T03:32:42.966 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-mds.k.log 2024-06-03T03:32:42.967 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-mds.j.log 2024-06-03T03:32:42.967 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-mds.g.log: 87.1% -- replaced with /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph.log.gz 2024-06-03T03:32:42.968 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-osd.3.log 2024-06-03T03:32:42.968 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-mon.b.log: 91.0% -- replaced with /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-volume.log.gz 2024-06-03T03:32:42.969 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-osd.0.log 2024-06-03T03:32:42.969 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-mds.l.log 2024-06-03T03:32:42.969 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-osd.3.log: 90.1% -- replaced with /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph.audit.log.gz 2024-06-03T03:32:42.970 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-mgr.x.log 2024-06-03T03:32:42.970 INFO:teuthology.orchestra.run.smithi117.stderr: 94.6% -- replaced with /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-mds.g.log.gz 2024-06-03T03:32:42.970 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-mon.a.log/var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-osd.0.log: 2024-06-03T03:32:42.971 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-osd.6.log 2024-06-03T03:32:42.973 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph.cephadm.log 2024-06-03T03:32:42.973 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-06-03T03:32:42.975 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-mds.a.log 2024-06-03T03:32:42.975 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-volume.log: 93.3% -- replaced with /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-mds.f.log.gz 2024-06-03T03:32:42.976 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph.cephadm.log: 82.7% -- replaced with /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph.cephadm.log.gz 2024-06-03T03:32:42.978 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-volume.log 2024-06-03T03:32:42.980 INFO:teuthology.orchestra.run.smithi183.stderr: 93.2% -- replaced with /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-mds.c.log.gz 2024-06-03T03:32:42.981 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-mds.k.log: 92.8% -- replaced with /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-mds.b.log.gz 2024-06-03T03:32:42.982 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-mds.d.log 2024-06-03T03:32:42.982 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-osd.9.log 2024-06-03T03:32:42.982 INFO:teuthology.orchestra.run.smithi183.stderr: 91.0% -- replaced with /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-volume.log.gz 2024-06-03T03:32:42.986 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-06-03T03:32:42.987 INFO:teuthology.orchestra.run.smithi151.stderr: 94.2% -- replaced with /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-mds.e.log.gz/var/log/ceph/cephadm.log: 2024-06-03T03:32:42.988 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-06-03T03:32:42.988 INFO:teuthology.orchestra.run.smithi151.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-06-03T03:32:42.989 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-06-03T03:32:42.989 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-mds.l.log: /var/log/ceph/cephadm.log: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-06-03T03:32:42.995 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/ceph-client.admin.log: 94.6% -- replaced with /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-mds.d.log.gz 2024-06-03T03:32:42.996 INFO:teuthology.orchestra.run.smithi117.stderr: 90.9% -- replaced with /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-volume.log.gz 2024-06-03T03:32:42.997 INFO:teuthology.orchestra.run.smithi117.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-06-03T03:32:43.012 INFO:teuthology.orchestra.run.smithi117.stderr: 93.9% -- replaced with /var/log/ceph/cephadm.log.gz 2024-06-03T03:32:43.016 INFO:teuthology.orchestra.run.smithi151.stderr: 93.6% -- replaced with /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-mds.h.log.gz 2024-06-03T03:32:43.024 INFO:teuthology.orchestra.run.smithi117.stderr: 94.3% -- replaced with /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-mds.j.log.gz 2024-06-03T03:32:43.025 INFO:teuthology.orchestra.run.smithi151.stderr: 94.3% -- replaced with /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-mds.k.log.gz 2024-06-03T03:32:43.025 INFO:teuthology.orchestra.run.smithi117.stderr: 93.4% -- replaced with /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-mds.a.log.gz 2024-06-03T03:32:43.049 INFO:teuthology.orchestra.run.smithi183.stderr: 94.3% -- replaced with /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-mds.l.log.gz 2024-06-03T03:32:43.180 INFO:teuthology.orchestra.run.smithi117.stderr: 90.1% -- replaced with /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-mgr.x.log.gz 2024-06-03T03:32:43.182 INFO:teuthology.orchestra.run.smithi183.stderr: 92.7% -- replaced with /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-mds.i.log.gz 2024-06-03T03:32:43.278 INFO:teuthology.orchestra.run.smithi151.stderr: 91.7% -- replaced with /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-mon.b.log.gz 2024-06-03T03:32:43.357 INFO:teuthology.orchestra.run.smithi183.stderr: 91.7% -- replaced with /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-mon.c.log.gz 2024-06-03T03:32:43.791 INFO:teuthology.orchestra.run.smithi151.stderr: 92.4% -- replaced with /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-osd.4.log.gz 2024-06-03T03:32:43.825 INFO:teuthology.orchestra.run.smithi117.stderr: 90.4% -- replaced with /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-mon.a.log.gz 2024-06-03T03:32:43.855 INFO:teuthology.orchestra.run.smithi117.stderr: 92.4% -- replaced with /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-osd.0.log.gz 2024-06-03T03:32:43.856 INFO:teuthology.orchestra.run.smithi151.stderr: 92.2% -- replaced with /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-osd.1.log.gz 2024-06-03T03:32:43.882 INFO:teuthology.orchestra.run.smithi183.stderr: 93.4% -- replaced with /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-osd.11.log.gz 2024-06-03T03:32:43.955 INFO:teuthology.orchestra.run.smithi117.stderr: 92.4% -- replaced with /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-osd.6.log.gz 2024-06-03T03:32:43.996 INFO:teuthology.orchestra.run.smithi117.stderr: 92.5% -- replaced with /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-osd.3.log.gz 2024-06-03T03:32:44.087 INFO:teuthology.orchestra.run.smithi183.stderr: 93.1% -- replaced with /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-osd.8.log.gz 2024-06-03T03:32:44.180 INFO:teuthology.orchestra.run.smithi151.stderr: 93.1% -- replaced with /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-osd.7.log.gz 2024-06-03T03:32:44.201 INFO:teuthology.orchestra.run.smithi151.stderr: 92.9% -- replaced with /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-osd.10.log.gz 2024-06-03T03:32:44.203 INFO:teuthology.orchestra.run.smithi151.stderr: 2024-06-03T03:32:44.203 INFO:teuthology.orchestra.run.smithi151.stderr:real 0m1.278s 2024-06-03T03:32:44.203 INFO:teuthology.orchestra.run.smithi151.stderr:user 0m4.656s 2024-06-03T03:32:44.203 INFO:teuthology.orchestra.run.smithi151.stderr:sys 0m0.187s 2024-06-03T03:32:44.234 INFO:teuthology.orchestra.run.smithi183.stderr: 92.9% -- replaced with /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-osd.2.log.gz 2024-06-03T03:32:44.277 INFO:teuthology.orchestra.run.smithi183.stderr: 93.0% -- replaced with /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-osd.5.log.gz 2024-06-03T03:32:44.279 INFO:teuthology.orchestra.run.smithi183.stderr: 2024-06-03T03:32:44.279 INFO:teuthology.orchestra.run.smithi183.stderr:real 0m1.356s 2024-06-03T03:32:44.279 INFO:teuthology.orchestra.run.smithi183.stderr:user 0m5.324s 2024-06-03T03:32:44.279 INFO:teuthology.orchestra.run.smithi183.stderr:sys 0m0.208s 2024-06-03T03:32:44.383 INFO:teuthology.orchestra.run.smithi117.stderr: 92.8% -- replaced with /var/log/ceph/a0e8473a-2156-11ef-bc9b-c7b262605968/ceph-osd.9.log.gz 2024-06-03T03:32:44.387 INFO:teuthology.orchestra.run.smithi117.stderr: 2024-06-03T03:32:44.387 INFO:teuthology.orchestra.run.smithi117.stderr:real 0m1.448s 2024-06-03T03:32:44.387 INFO:teuthology.orchestra.run.smithi117.stderr:user 0m5.074s 2024-06-03T03:32:44.387 INFO:teuthology.orchestra.run.smithi117.stderr:sys 0m0.473s 2024-06-03T03:32:44.388 INFO:tasks.cephadm:Archiving logs... 2024-06-03T03:32:44.388 DEBUG:teuthology.misc:Transferring archived files from smithi117:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-06-02_21:24:02-fs-squid-distro-default-smithi/7738859/remote/smithi117/log 2024-06-03T03:32:44.389 DEBUG:teuthology.orchestra.run.smithi117:> sudo tar c -f - -C /var/log/ceph -- . 2024-06-03T03:32:45.125 DEBUG:teuthology.misc:Transferring archived files from smithi151:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-06-02_21:24:02-fs-squid-distro-default-smithi/7738859/remote/smithi151/log 2024-06-03T03:32:45.126 DEBUG:teuthology.orchestra.run.smithi151:> sudo tar c -f - -C /var/log/ceph -- . 2024-06-03T03:32:45.820 DEBUG:teuthology.misc:Transferring archived files from smithi183:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-06-02_21:24:02-fs-squid-distro-default-smithi/7738859/remote/smithi183/log 2024-06-03T03:32:45.821 DEBUG:teuthology.orchestra.run.smithi183:> sudo tar c -f - -C /var/log/ceph -- . 2024-06-03T03:32:46.617 INFO:tasks.cephadm:Removing cluster... 2024-06-03T03:32:46.617 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 --force 2024-06-03T03:32:46.924 INFO:teuthology.orchestra.run.smithi117.stdout:Deleting cluster with fsid: a0e8473a-2156-11ef-bc9b-c7b262605968 2024-06-03T03:32:47.411 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 --force 2024-06-03T03:32:47.602 INFO:teuthology.orchestra.run.smithi151.stdout:Deleting cluster with fsid: a0e8473a-2156-11ef-bc9b-c7b262605968 2024-06-03T03:32:47.856 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a0e8473a-2156-11ef-bc9b-c7b262605968 --force 2024-06-03T03:32:48.036 INFO:teuthology.orchestra.run.smithi183.stdout:Deleting cluster with fsid: a0e8473a-2156-11ef-bc9b-c7b262605968 2024-06-03T03:32:48.292 INFO:tasks.cephadm:Removing cephadm ... 2024-06-03T03:32:48.292 DEBUG:teuthology.orchestra.run.smithi117:> rm -rf /home/ubuntu/cephtest/cephadm 2024-06-03T03:32:48.350 DEBUG:teuthology.orchestra.run.smithi151:> rm -rf /home/ubuntu/cephtest/cephadm 2024-06-03T03:32:48.365 DEBUG:teuthology.orchestra.run.smithi183:> rm -rf /home/ubuntu/cephtest/cephadm 2024-06-03T03:32:48.380 INFO:tasks.cephadm:Teardown complete 2024-06-03T03:32:48.381 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-06-03T03:32:48.392 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-06-03T03:32:48.392 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-06-03T03:32:48.400 DEBUG:teuthology.orchestra.run.smithi151:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-06-03T03:32:48.408 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-06-03T03:32:48.543 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-06-03T03:32:48.553 INFO:teuthology.task.clock:Checking final clock skew... 2024-06-03T03:32:48.554 DEBUG:teuthology.orchestra.run.smithi117:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-03T03:32:48.561 DEBUG:teuthology.orchestra.run.smithi151:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-03T03:32:48.563 DEBUG:teuthology.orchestra.run.smithi183:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-03T03:32:48.575 INFO:teuthology.orchestra.run.smithi151.stderr:bash: line 1: ntpq: command not found 2024-06-03T03:32:48.577 INFO:teuthology.orchestra.run.smithi183.stderr:bash: line 1: ntpq: command not found 2024-06-03T03:32:48.579 INFO:teuthology.orchestra.run.smithi151.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-03T03:32:48.579 INFO:teuthology.orchestra.run.smithi151.stdout:=============================================================================== 2024-06-03T03:32:48.579 INFO:teuthology.orchestra.run.smithi151.stdout:^* hv01.front.sepia.ceph.com 3 6 377 52 +925us[ +928us] +/- 45ms 2024-06-03T03:32:48.579 INFO:teuthology.orchestra.run.smithi151.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 55 -716us[ -713us] +/- 73ms 2024-06-03T03:32:48.579 INFO:teuthology.orchestra.run.smithi151.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 53 -662us[ -659us] +/- 63ms 2024-06-03T03:32:48.579 INFO:teuthology.orchestra.run.smithi151.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-06-03T03:32:48.581 INFO:teuthology.orchestra.run.smithi183.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-03T03:32:48.581 INFO:teuthology.orchestra.run.smithi183.stdout:=============================================================================== 2024-06-03T03:32:48.582 INFO:teuthology.orchestra.run.smithi183.stdout:^* hv01.front.sepia.ceph.com 3 6 377 51 +924us[ +925us] +/- 45ms 2024-06-03T03:32:48.582 INFO:teuthology.orchestra.run.smithi183.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 52 -715us[ -714us] +/- 73ms 2024-06-03T03:32:48.582 INFO:teuthology.orchestra.run.smithi183.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 54 -667us[ -666us] +/- 63ms 2024-06-03T03:32:48.582 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-06-03T03:32:48.607 INFO:teuthology.orchestra.run.smithi117.stderr:bash: line 1: ntpq: command not found 2024-06-03T03:32:48.617 INFO:teuthology.orchestra.run.smithi117.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-03T03:32:48.617 INFO:teuthology.orchestra.run.smithi117.stdout:=============================================================================== 2024-06-03T03:32:48.617 INFO:teuthology.orchestra.run.smithi117.stdout:^* hv01.front.sepia.ceph.com 3 7 377 52 +977us[ +933us] +/- 45ms 2024-06-03T03:32:48.617 INFO:teuthology.orchestra.run.smithi117.stdout:^+ hv02.front.sepia.ceph.com 3 7 377 52 -653us[ -653us] +/- 74ms 2024-06-03T03:32:48.617 INFO:teuthology.orchestra.run.smithi117.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 48 -659us[ -659us] +/- 63ms 2024-06-03T03:32:48.617 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-06-03T03:32:48.618 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-06-03T03:32:48.628 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-06-03T03:32:48.628 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-06-03T03:32:48.664 DEBUG:teuthology.orchestra.run.smithi117:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-06-03T03:32:48.873 DEBUG:teuthology.orchestra.run.smithi151:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-06-03T03:32:48.937 DEBUG:teuthology.orchestra.run.smithi183:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-06-03T03:32:49.002 DEBUG:teuthology.orchestra.run.smithi117:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-03T03:32:49.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-03T03:32:49.099 DEBUG:teuthology.orchestra.run.smithi151:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-03T03:32:49.125 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-03T03:32:49.125 DEBUG:teuthology.orchestra.run.smithi183:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-03T03:32:49.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-03T03:32:49.152 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-06-03T03:32:49.163 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-06-03T03:32:49.174 INFO:teuthology.task.internal:Duration was 1924.388384 seconds 2024-06-03T03:32:49.174 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-06-03T03:32:49.183 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-06-03T03:32:49.183 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-06-03T03:32:49.191 DEBUG:teuthology.orchestra.run.smithi151:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-06-03T03:32:49.193 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-06-03T03:32:49.228 INFO:teuthology.orchestra.run.smithi151.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-03T03:32:49.230 INFO:teuthology.orchestra.run.smithi183.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-03T03:32:49.316 INFO:teuthology.orchestra.run.smithi117.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-03T03:32:49.533 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-06-03T03:32:49.534 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi117.front.sepia.ceph.com 2024-06-03T03:32:49.534 DEBUG:teuthology.orchestra.run.smithi117:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-06-03T03:32:49.601 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi151.front.sepia.ceph.com 2024-06-03T03:32:49.602 DEBUG:teuthology.orchestra.run.smithi151:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-06-03T03:32:49.624 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi183.front.sepia.ceph.com 2024-06-03T03:32:49.625 DEBUG:teuthology.orchestra.run.smithi183:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-06-03T03:32:49.654 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-06-03T03:32:49.655 DEBUG:teuthology.orchestra.run.smithi117:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-06-03T03:32:49.662 DEBUG:teuthology.orchestra.run.smithi151:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-06-03T03:32:49.667 DEBUG:teuthology.orchestra.run.smithi183:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-06-03T03:32:49.739 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-06-03T03:32:49.739 DEBUG:teuthology.orchestra.run.smithi117:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-06-03T03:32:49.744 DEBUG:teuthology.orchestra.run.smithi151:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-06-03T03:32:49.746 DEBUG:teuthology.orchestra.run.smithi183:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-06-03T03:32:50.554 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-06-03T03:32:50.565 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-06-03T03:32:50.565 DEBUG:teuthology.orchestra.run.smithi117:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-06-03T03:32:50.644 DEBUG:teuthology.orchestra.run.smithi151:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-06-03T03:32:50.670 DEBUG:teuthology.orchestra.run.smithi183:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-06-03T03:32:50.697 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-06-03T03:32:50.709 DEBUG:teuthology.orchestra.run.smithi117:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-06-03T03:32:50.716 DEBUG:teuthology.orchestra.run.smithi151:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-06-03T03:32:50.718 DEBUG:teuthology.orchestra.run.smithi183:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-06-03T03:32:50.739 INFO:teuthology.orchestra.run.smithi151.stdout:kernel.core_pattern = core 2024-06-03T03:32:50.762 INFO:teuthology.orchestra.run.smithi183.stdout:kernel.core_pattern = core 2024-06-03T03:32:50.807 INFO:teuthology.orchestra.run.smithi117.stdout:kernel.core_pattern = core 2024-06-03T03:32:50.849 DEBUG:teuthology.orchestra.run.smithi117:> test -e /home/ubuntu/cephtest/archive/coredump 2024-06-03T03:32:50.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-03T03:32:50.895 DEBUG:teuthology.orchestra.run.smithi151:> test -e /home/ubuntu/cephtest/archive/coredump 2024-06-03T03:32:50.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-03T03:32:50.910 DEBUG:teuthology.orchestra.run.smithi183:> test -e /home/ubuntu/cephtest/archive/coredump 2024-06-03T03:32:50.925 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-03T03:32:50.926 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-06-03T03:32:50.941 INFO:teuthology.task.internal:Transferring archived files... 2024-06-03T03:32:50.942 DEBUG:teuthology.misc:Transferring archived files from smithi117:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-06-02_21:24:02-fs-squid-distro-default-smithi/7738859/remote/smithi117 2024-06-03T03:32:50.943 DEBUG:teuthology.orchestra.run.smithi117:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-06-03T03:32:51.053 DEBUG:teuthology.misc:Transferring archived files from smithi151:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-06-02_21:24:02-fs-squid-distro-default-smithi/7738859/remote/smithi151 2024-06-03T03:32:51.053 DEBUG:teuthology.orchestra.run.smithi151:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-06-03T03:32:51.090 DEBUG:teuthology.misc:Transferring archived files from smithi183:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-06-02_21:24:02-fs-squid-distro-default-smithi/7738859/remote/smithi183 2024-06-03T03:32:51.090 DEBUG:teuthology.orchestra.run.smithi183:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-06-03T03:32:51.126 INFO:teuthology.task.internal:Removing archive directory... 2024-06-03T03:32:51.126 DEBUG:teuthology.orchestra.run.smithi117:> rm -rf -- /home/ubuntu/cephtest/archive 2024-06-03T03:32:51.134 DEBUG:teuthology.orchestra.run.smithi151:> rm -rf -- /home/ubuntu/cephtest/archive 2024-06-03T03:32:51.136 DEBUG:teuthology.orchestra.run.smithi183:> rm -rf -- /home/ubuntu/cephtest/archive 2024-06-03T03:32:51.185 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-06-03T03:32:51.198 INFO:teuthology.task.internal:Not uploading archives. 2024-06-03T03:32:51.198 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-06-03T03:32:51.209 INFO:teuthology.task.internal:Tidying up after the test... 2024-06-03T03:32:51.209 DEBUG:teuthology.orchestra.run.smithi117:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-06-03T03:32:51.215 DEBUG:teuthology.orchestra.run.smithi151:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-06-03T03:32:51.217 DEBUG:teuthology.orchestra.run.smithi183:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-06-03T03:32:51.230 INFO:teuthology.orchestra.run.smithi151.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 3 03:32 /home/ubuntu/cephtest 2024-06-03T03:32:51.232 INFO:teuthology.orchestra.run.smithi183.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 3 03:32 /home/ubuntu/cephtest 2024-06-03T03:32:51.265 INFO:teuthology.orchestra.run.smithi117.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 3 03:32 /home/ubuntu/cephtest 2024-06-03T03:32:51.269 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-06-03T03:32:51.280 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-06-03T03:32:51.305 INFO:teuthology.run:Summary data: client.0-kernel-sha1: b9459a26b0888785d05c47deb35abc9d0354ab93 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/no 6-workunit/suites/fsync-tester}} duration: 1924.388384103775 failure_reason: '"2024-06-03T03:20:00.000145+0000 mon.a (mon.0) 1031 : cluster [WRN] Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled" in cluster log' flavor: default owner: scheduled_teuthology@teuthology success: false 2024-06-03T03:32:51.305 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-06-03T03:32:51.508 INFO:teuthology.run:FAIL